Waiting for health checks... running forever.

Describe the bug
Recently I have been unable to launch my supabase local project using the supabase start command.
The command remains stuck on Waiting for health check and never runs succesfully.

Studio is available normally.
Authentication or any service related to the database seem to work.

System information
Rerun the failing command with --create-ticket flag.

Ticket ID: d888349bfa5f409fb91e7b4eb4f71d98
Version of OS: macOS d888349bfa5f409fb91e7b4eb4f71d98
Version of CLI: 1.178.2
Version of Docker: Docker version 26.1.4, build 5650f9b102
Versions of services:

        SERVICE IMAGE      │      LOCAL       │ LINKED
  ─────────────────────────┼──────────────────┼─────────
    supabase/postgres      │ <IP_ADDRESS>        │ -
    supabase/gotrue        │ v2.151.0         │ -
    postgrest/postgrest    │ v12.0.1          │ -
    supabase/realtime      │ v2.28.32         │ -
    supabase/storage-api   │ v1.0.6           │ -
    supabase/edge-runtime  │ v1.54.6          │ -
    supabase/studio        │ 20240422-5cf8f30 │ -
    supabase/postgres-meta │ v0.80.0          │ -
    supabase/logflare      │ 1.4.0            │ -
    supabase/supavisor     │ 1.1.56           │ -
    darthsim/imgproxy      │ v3.8.0           │ -

✨  Done in 0.19s.

CONTAINER ID   IMAGE                                             COMMAND                  CREATED              STATUS                        PORTS                                                                  NAMES
f6e2d0a98988   public.ecr.aws/supabase/studio:20240422-5cf8f30   "docker-entrypoint.s…"   45 seconds ago       Up 43 seconds (healthy)       <IP_ADDRESS>:54323->3000/tcp, :::54323->3000/tcp                            supabase_studio_mentor
12431e0a3e21   public.ecr.aws/supabase/postgres-meta:v0.80.0     "docker-entrypoint.s…"   45 seconds ago       Up 44 seconds (healthy)       8080/tcp                                                               supabase_pg_meta_mentor
818fcf85ea2e   public.ecr.aws/supabase/edge-runtime:v1.54.6      "sh -c 'cat <<'EOF' …"   45 seconds ago       Up 44 seconds                 <IP_ADDRESS>:32768->8081/tcp, :::32768->8081/tcp                            supabase_edge_runtime_mentor
2d16a4fadf28   public.ecr.aws/supabase/imgproxy:v3.8.0           "imgproxy"               46 seconds ago       Up 44 seconds (healthy)       8080/tcp                                                               supabase_imgproxy_mentor
1fc177aa7072   public.ecr.aws/supabase/storage-api:v1.0.6        "docker-entrypoint.s…"   46 seconds ago       Up 45 seconds (healthy)       5000/tcp                                                               supabase_storage_mentor
48ddd978df6c   public.ecr.aws/supabase/postgrest:v12.0.1         "postgrest"              46 seconds ago       Up 45 seconds                 3000/tcp                                                               supabase_rest_mentor
9461c1d93559   public.ecr.aws/supabase/realtime:v2.28.32         "/usr/bin/tini -s -g…"   47 seconds ago       Up 46 seconds (healthy)       4000/tcp                                                               supabase_realtime_mentor
4df7ed5dc2a5   public.ecr.aws/supabase/inbucket:3.0.3            "/start-inbucket.sh …"   47 seconds ago       Up 46 seconds (healthy)       1100/tcp, 2500/tcp, <IP_ADDRESS>:54324->9000/tcp, :::54324->9000/tcp        supabase_inbucket_mentor
02a8792d2d43   public.ecr.aws/supabase/gotrue:v2.151.0           "auth"                   47 seconds ago       Up 46 seconds (healthy)       9999/tcp                                                               supabase_auth_mentor
c8e38227fed7   public.ecr.aws/supabase/kong:2.8.1                "sh -c 'cat <<'EOF' …"   47 seconds ago       Up 46 seconds (healthy)       8001/tcp, 8443-8444/tcp, <IP_ADDRESS>:54321->8000/tcp, :::54321->8000/tcp   supabase_kong_mentor
d7ab72e1bbfc   public.ecr.aws/supabase/postgres:<IP_ADDRESS>        "sh -c 'cat <<'EOF' …"   About a minute ago   Up About a minute (healthy)   <IP_ADDRESS>:54322->5432/tcp, :::54322->5432/tcp                            supabase_db_mentor

Here are the logs for postgrest:
27/Jun/2024:09:41:01 +0000: Starting PostgREST 12.0.1 (cd38da5)...
27/Jun/2024:09:41:01 +0000: Attempting to connect to the database...
27/Jun/2024:09:41:01 +0000: Connection successful
27/Jun/2024:09:41:01 +0000: Admin server listening on port 3001
27/Jun/2024:09:41:01 +0000: Listening on port 3000
27/Jun/2024:09:41:01 +0000: Listening for notifications on the pgrst channel
27/Jun/2024:09:41:01 +0000: Config reloaded
27/Jun/2024:09:41:01 +0000: Schema cache loaded

and edge-runtime:
Download https://deno.land/std/http/status.ts
Download<EMAIL_ADDRESS>Implicitly using latest version (0.224.0) for https://deno.land/std/http/status.ts
Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Serving functions on http://<IP_ADDRESS>:54321/functions/v1/<function-name>
Using supabase-edge-runtime-1.54.6 (compatible with Deno v1.43.0)

Could you kindly check if this is still reproducible in the beta release?
npx supabase@beta start

Same issue here, but the beta did not change it. Could it be an issue with the config.toml file?
config.toml.txt

Hmm I'm not sure what's exactly wrong as I can't seem to reproduce locally on macOS. But as a temporary workaround, you can pass in supabase start --ignore-health-check flag.
https://supabase.com/docs/reference/cli/introduction

Same issue here, but the beta did not change it. Could it be an issue with the config.toml file?
config.toml.txt
Got this when health check failed.
Serving functions on http://<IP_ADDRESS>:54321/functions/v1/<function-name>
Using supabase-edge-runtime-1.55.0 (compatible with Deno v1.43.0)
Stopping containers...
failed to execute http request: Head "http://<IP_ADDRESS>:54321/rest-admin/v1/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
failed to execute http request: Head "http://<IP_ADDRESS>:54321/functions/v1/_internal/health": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
Try rerunning the command with --debug to troubleshoot the error.

Got it working by recreating the project to default configuration, looks like there is a setting in newer versions blocking us, maybe you could post your project configuration to see ?

Hmm I'm not sure what's exactly wrong as I can't seem to reproduce locally on macOS. But as a temporary workaround, you can pass in supabase start --ignore-health-check flag.
https://supabase.com/docs/reference/cli/introduction

I'm having this same problem, and oddly even with the --ignore-health-check flag, the cli still hangs on Waiting for health checks...

Same issue here. I running on macOS 15. Could the issue be related to this macOS version somehow?

+1 on how you recreated the project to default configuration.

recreated the project to default configuration

You can try these commands to reinitialise the local project config. I'm not 100% sure if it solves the problem though.
supabase stop --no-backup
supabase init --force

Same issue here, but the beta did not change it. Could it be an issue with the config.toml file?
config.toml.txt
Got this when health check failed.
Serving functions on http://<IP_ADDRESS>:54321/functions/v1/<function-name>
Using supabase-edge-runtime-1.55.0 (compatible with Deno v1.43.0)
Stopping containers...
failed to execute http request: Head "http://<IP_ADDRESS>:54321/rest-admin/v1/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
failed to execute http request: Head "http://<IP_ADDRESS>:54321/functions/v1/_internal/health": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
Try rerunning the command with --debug to troubleshoot the error.

Got it working by recreating the project to default configuration, looks like there is a setting in newer versions blocking us, maybe you could post your project configuration to see ?

What do you mean by "recreating the project to default configuration"? Would you mind sharing how to do this? Thanks

+1 on how you recreated the project to default configuration.

Simply save your current configuration somewhere and reinitialize it

Yep! Just switching enabled to false worked for me. Thanks!

@sweatybridge closing the issue as there is a "fix" but feel free to re-open it if you need it.
Happy coding everyone!

Pruned containers: [3f649d1d9c453146d7db79a70b79caee2b73e435a29bdd7f83fb8e68e27b3899]
Pruned volumes: [supabase_db_platform supabase_config_platform]
Pruned network: [supabase_network_platform]
failed to connect to postgres: failed to connect to `host=<IP_ADDRESS> user=postgres database=postgres`: dial error (timeout: dial tcp <IP_ADDRESS>:54322: i/o timeout)


Create ROM

Check README.md for details.
Design File: 30 Points
Test Bench: 30 Points
Total Points: 60 points

can i get this asigned ?

can u assign me

@maverickayush7 please go ahead


upload_to_testflight broken unless run with verbose

New Issue Checklist

[x] Updated fastlane to the latest version
[x] I read the Contribution Guidelines
[x] I read docs.fastlane.tools
[x] I searched for existing GitHub issues

Issue Description
Was setting up CI (Bitrise) for a new project, using the same structure and configuration used for other projects, but my build uploads kept failing for some weird transporter reason. This only happens on CI, which makes it hard to reproduce for me (had to run it each time on CI, and SSH access / environment is bonkers and broken).
I just happened to stumble on the fact that adding --verbose magically fixed my build upload to TestFlight.
Downgrading seemed to solve it
Digging a bit further, I first thought I found a regression:

2.166.0 and older just works
2.167.0 is broken without --verbose
2.168.0 is broken without --verbose

But after temporarily downgrading to 2.166, recent builds seem to be failing with the same issue... So no regression, just something weird going on. Tried going back as far as 2.163, didn't help.
Adding --verbose to any of those versions instantly fixes it.
Complete output when running fastlane, including the stack trace and command used
I tried using fastlane ios development --capture_output but that didn't seem to do much on CI (nothing at all), so here's the CI log of the broken upload:

Partial output from CI of a broken upload

[02:03:54]: Successfully exported and compressed dSYM file
[02:03:54]: Successfully exported and signed the ipa file:
[02:03:54]: /Users/vagrant/git/fastlane/build/RemeCare Development.ipa
[02:03:54]: ----------------------------------
[02:03:54]: --- Step: upload_to_testflight ---
[02:03:54]: ----------------------------------
[02:03:54]: Ready to upload new build to TestFlight (App: )...
[02:03:54]: Fetching password for transporter from environment variable named `FASTLANE_APPLE_APPLICATION_SPECIFIC_PASSWORD`
[02:03:54]: Going to upload updated app to App Store Connect
[02:03:54]: This might take a few minutes. Please don't interrupt the script.
[02:04:11]: [Transporter Error Output]: ERROR ITMS-3000: "Line 3 column 51: value of attribute "apple_id" is invalid; must be an integer at XPath /package/software_assets"
[02:04:11]: [Transporter Error Output]: ERROR ITMS-3000: "Package "null" failed schema validation."
[02:04:11]: [Transporter Error Output]: The package name .itmsp contains an invalid character(s).  The valid characters are: A-Z, a-z, 0-9, dash, period, underscore, but the name cannot start with a dash, period, or underscore
[02:04:11]: Transporter transfer failed.
[02:04:11]:
[02:04:11]: ERROR ITMS-3000: "Line 3 column 51: value of attribute "apple_id" is invalid; must be an integer at XPath /package/software_assets"
ERROR ITMS-3000: "Package "null" failed schema validation."
The package name .itmsp contains an invalid character(s).  The valid characters are: A-Z, a-z, 0-9, dash, period, underscore, but the name cannot start with a dash, period, or underscore

Notice that the app "name" (or hash?) seems to be empty, and the itmsp file/package seems to have no name. Here's what I get for a successful upload with 2.166.0:

Partial output from CI of a successful upload

[18:30:55]: Successfully exported and compressed dSYM file
[18:30:55]: Successfully exported and signed the ipa file:
[18:30:55]: /Users/vagrant/git/fastlane/build/RemeCare Development.ipa
[18:30:55]: ----------------------------------
[18:30:55]: --- Step: upload_to_testflight ---
[18:30:55]: ----------------------------------
[18:30:55]: Ready to upload new build to TestFlight (App:<PHONE_NUMBER>)...
[18:30:56]: Fetching password for transporter from environment variable named `FASTLANE_APPLE_APPLICATION_SPECIFIC_PASSWORD`
[18:30:56]: Going to upload updated app to App Store Connect
[18:30:56]: This might take a few minutes. Please don't interrupt the script.
[18:32:12]: iTunes Transporter successfully finished its job
[18:32:12]: --------------------------------------------------------------------
[18:32:12]: Successfully uploaded package to App Store Connect. It might take a few minutes until it's visible online.
[18:32:12]: --------------------------------------------------------------------

Environment
🚫 fastlane environment 🚫
Stack

Key
Value

OS
10.15.7

Ruby
2.6.5

Bundler?
true

Git
git version 2.29.2

Installation Source
~/git/vendor/bundle/ruby/2.6.0/bin/fastlane

Host
Mac OS X 10.15.7 (19H2)

Ruby Lib Dir
~/.rbenv/versions/2.6.5/lib

OpenSSL Version
OpenSSL 1.1.1g  21 Apr 2020

Is contained
false

Is homebrew
false

Is installed via Fabric.app
false

Xcode Path
/Applications/Xcode.app/Contents/Developer/

Xcode Version
12.2

System Locale

Variable
Value

LANG
en_US.UTF-8
✅

LC_ALL
en_US.UTF-8
✅

LANGUAGE
en_US.UTF-8
✅

fastlane files:
`./fastlane/Fastfile`
fastlane_version '2.137.0'
default_platform :ios

actions_path('./../Pods/AppwiseCore/Fastlane Actions')

platform :ios do
  desc 'Submit a new Development Build to Apple TestFlight'
  lane :development do
    scheme = CredentialsManager::AppfileConfig.try_fetch_value(:scheme)
    archiveAndUpload(scheme: scheme)
  end

  desc 'Submit a new Lab Build to Apple TestFlight'
  lane :lab do
    scheme = CredentialsManager::AppfileConfig.try_fetch_value(:scheme)
    archiveAndUpload(scheme: scheme)
  end

  desc 'Submit a new Local Build to Apple TestFlight'
  lane :local do
    scheme = CredentialsManager::AppfileConfig.try_fetch_value(:scheme)
    archiveAndUpload(scheme: scheme)
  end

  desc 'Submit a new Production Build to Apple TestFlight'
  lane :production do
    scheme = CredentialsManager::AppfileConfig.try_fetch_value(:scheme)
    archiveAndUpload(scheme: scheme)
  end

  desc 'Submit a new Staging Build to Apple TestFlight'
  lane :staging do
    scheme = CredentialsManager::AppfileConfig.try_fetch_value(:scheme)
    archiveAndUpload(scheme: scheme)
  end

  desc 'Submit a new Test Build to Apple TestFlight'
  lane :test do
    scheme = CredentialsManager::AppfileConfig.try_fetch_value(:scheme)
    archiveAndUpload(scheme: scheme)
  end

  desc 'Deploy a new version to the App Store'
  lane :release do
    version_number = get_version_number
    sentry_project = CredentialsManager::AppfileConfig.try_fetch_value(:sentry_project)

    upload_to_app_store(
      submit_for_review: true,
      skip_binary_upload: true,
      force: true
    )
    sentry_finalize_release(
      version: "#{sentry_project}@#{version_number}"
    )
  end

  desc 'Download dSYMs from iTC and upload them to Sentry'
  lane :refresh_dsyms do
    identifiers = CredentialsManager::AppfileConfig.try_fetch_value(:identifiers)

    identifiers.each do |identifier|
      syncDsyms(identifier: identifier)
    end
  end

  desc "Imports, and then extracts localizations from the xcode project"
  lane :sync_translations do
    sync_localizations()
  end

  private_lane :archiveAndUpload do |options|
    scheme = options[:scheme]
    version_number = get_version_number
    sentry_project = CredentialsManager::AppfileConfig.try_fetch_value(:sentry_project)

    get_provisioning_profile(
      output_path: 'fastlane/build'
    ) unless ENV.key?('CI')
    build_ios_app(
      scheme: scheme,
      include_bitcode: true,
      output_directory: 'fastlane/build',
      output_name: "#{scheme}.ipa",
      silent: true
    )
    upload_to_testflight(
      skip_waiting_for_build_processing: true,
      itc_provider: '2PPBSYHSNC'
    )
    sentry_create_release(
      version: "#{sentry_project}@#{version_number}"
    )
    sentry_auto_set_commits(
      version: "#{sentry_project}@#{version_number}"
    )
  end

  private_lane :syncDsyms do |options|
    identifier = options[:identifier]

    download_dsyms(
      app_identifier: identifier,
      output_directory: 'fastlane/build'
    )
    sentry_upload_dsym
    clean_build_artifacts
    lane_context[SharedValues::DSYM_PATHS] = []
  end
end

`./fastlane/Appfile`
apple_id ENV['USER_APPLE_ID'] # Your Apple email address

team_name 'Remedus BVBA'
itc_team_name 'Remedus BVBA'
itc_provider '2PPBSYHSNC'

# Environment settings for Sentry
ENV['SENTRY_URL'] = 'https://sentry.appwi.se'
ENV['SENTRY_ORG_SLUG'] = 'appwise'
ENV['SENTRY_PROJECT_SLUG'] = 'remedus-ios'

# lane specific settings
for_platform :ios do
  app_identifier 'be.remedus.remecoachnew'
  setter(:scheme, 'RemeCare Production')
  setter(:sentry_project, ENV['SENTRY_PROJECT_SLUG'])

  setter(:identifiers, [
    'be.remedus.remecoachnew-Development',
    'be.remedus.remecoachnew-Local',
    'be.remedus.remecoachnew-Lab',
    'be.remedus.remecoachnew-Staging',
    'be.remedus.remecoachnew-Test',
    'be.remedus.remecoachnew'
  ])

  for_lane :development do
    app_identifier 'be.remedus.remecoachnew-Development'
    setter(:scheme, 'RemeCare Development')
  end

  for_lane :lab do
    app_identifier 'be.remedus.remecoachnew-Lab'
    setter(:scheme, 'RemeCare Lab')
  end

  for_lane :local do
    app_identifier 'be.remedus.remecoachnew-Local'
    setter(:scheme, 'RemeCare Local')
  end

  for_lane :staging do
    app_identifier 'be.remedus.remecoachnew-Staging'
    setter(:scheme, 'RemeCare Staging')
  end

  for_lane :test do
    app_identifier 'be.remedus.remecoachnew-Test'
    setter(:scheme, 'RemeCare Test')
  end
end

fastlane gems

Gem
Version
Update-Status

fastlane
2.167.0
🚫 Update available

Loaded fastlane plugins:

Plugin
Version
Update-Status

fastlane-plugin-sentry
1.7.0
✅ Up-To-Date

Loaded gems

Gem
Version

did_you_mean
1.3.0

bundler
2.1.4

rake
13.0.1

CFPropertyList
3.0.2

concurrent-ruby
1.1.7

i18n
1.8.5

minitest
5.14.2

thread_safe
0.3.6

tzinfo
1.2.8

activesupport
<IP_ADDRESS>

public_suffix
4.0.6

addressable
2.7.0

httpclient
2.8.3

json
2.3.1

algoliasearch
1.27.5

atomos
0.1.3

aws-eventstream
1.1.0

aws-partitions
1.399.0

aws-sigv4
1.2.2

jmespath
1.4.0

aws-sdk-core
3.109.3

aws-sdk-kms
1.39.0

aws-sdk-s3
1.85.0

babosa
1.0.4

claide
1.0.3

fuzzy_match
2.0.4

nap
1.1.0

netrc
0.11.0

ffi
1.13.1

ethon
0.12.0

typhoeus
1.4.0

cocoapods-core
1.10.0

cocoapods-deintegrate
1.0.4

cocoapods-downloader
1.4.0

cocoapods-plugins
1.0.0

cocoapods-search
1.0.0

cocoapods-trunk
1.5.0

cocoapods-try
1.2.0

colored2
3.1.2

escape
0.0.4

fourflusher
2.3.1

gh_inspector
1.1.3

molinillo
0.6.6

ruby-macho
1.4.0

nanaimo
0.3.0

xcodeproj
1.19.0

cocoapods
1.10.0

cocoapods-rome
1.0.1

colored
1.2

highline
1.7.10

commander-fastlane
4.4.6

declarative
0.0.20

declarative-option
0.1.0

digest-crc
0.6.1

unf_ext
<IP_ADDRESS>

unf
0.1.4

domain_name
0.5.20190701

dotenv
2.7.6

emoji_regex
3.2.1

excon
0.78.0

multipart-post
2.0.0

ruby2_keywords
0.0.2

faraday
1.1.0

http-cookie
1.0.3

faraday-cookie_jar
0.0.7

faraday_middleware
1.0.0

fastimage
2.2.0

jwt
2.2.2

memoist
0.16.2

multi_json
1.15.0

os
1.1.1

signet
0.14.0

googleauth
0.14.0

mini_mime
1.0.2

uber
0.1.0

representable
3.0.4

retriable
3.1.2

google-api-client
0.38.0

google-cloud-env
1.4.0

google-cloud-errors
1.0.1

google-cloud-core
1.5.0

google-cloud-storage
1.29.1

mini_magick
4.11.0

plist
3.5.0

rubyzip
2.3.0

security
0.1.3

naturally
2.2.0

simctl
1.6.8

slack-notifier
2.3.2

terminal-notifier
2.0.0

unicode-display_width
1.7.0

terminal-table
1.8.0

tty-screen
0.8.1

tty-cursor
0.7.1

tty-spinner
0.9.3

word_wrap
1.0.0

rouge
2.0.7

xcpretty
0.3.0

xcpretty-travis-formatter
1.0.0

fastlane-plugin-sentry
1.7.0

generated on: 2020-11-25

Could you please try to update fastlane and try it again?

Updated to 2.171.0 and I think it's solved. Ran a few different lanes on CI without verbose, and they seem to be working again 🎉

Updated to 2.171.0 and I think it's solved. Ran a few different lanes on CI without verbose, and they seem to be working again 🎉

Great news @djbe ! Thanks for checking and getting back to us. I'm closing this issue for now - if it becomes a problem again, let us know. 🤗

Great news @djbe ! Thanks for checking and getting back to us. I'm closing this issue for now - if it becomes a problem again, let us know. 🤗


support for 40G interface counters

I was not able to get the counters from et-XXX interfaces while using get_interfaces_counters(). After changing the yml file I can get the counters from et-XXX and emX interfaces

Coverage remained the same at 78.759% when pulling 4b7d391db4f6b00c53121c94d2e9f3899240b115 on ckishimo:develop into 29026c1a53423bcd2a17d14c90d7da6d7269c381 on napalm-automation:develop.


Vickel: Watch royal[\W_]+yachts[\W_]+miami(?!.com(?<=royalyachtsmiami.com))

Vickel requests the watch of the watch_keyword royal[\W_]*+yachts[\W_]*+miami(?!\.com(?<=royalyachtsmiami\.com)). See the MS search here and the Stack Exchange search in text, in URLs, and in code.

royal[\W_]*+yachts[\W_]*+miami(?!\.com(?<=royalyachtsmiami\.com)) has been seen in 0 true positives, 0 false positives, and 0 NAAs.

Approved by VLAZ in Charcoal HQ

Approved by VLAZ in Charcoal HQ


ExpressibleByIntegerLiteral erroneously overwrites (shadows?) existing initializer in type

Description
Adhering to ExpressibleByIntegerLiteral adds an initializer that is not declared in the protocol and overwrites or shadows an initializer that is actually declared in code.
This leads to erroneous behavior at the call site — in effect one initializer is called in code but a different one is actually executed. Code to reproduce the issue is attached below.
Steps to reproduce
Paste the following Swift code in an empty playground or project:
struct ABC: ExpressibleByIntegerLiteral {
    var value = 0

    init?(_ value: Int) {
        self.value = 1
    }

    init(integerLiteral value: IntegerLiteralType) {
        self.value = 2
    }
}

let a = ABC(1)
print(a.value)

Expected behavior
The expected behavior would be that ABC(1) invokes the initializer with unnamed parameter init?(_ value: Int). However, after running the code above you'd see that that line actually invokes init(integerLiteral value: IntegerLiteralType) without providing the named parameter and the printed value value is 2.
Screenshots
Here's a screenshot from my Xcode playground where I'm running the code:

Environment

Swift compiler version info: swiftlang-<IP_ADDRESS>.108 clang-15<IP_ADDRESS>
Xcode version info: Xcode 15.0 Build version 15A240d
Deployment target: macOS 13.6 (22G120)

So @airspeedswift has commented that this behavior is described in this proposal https://github.com/apple/swift-evolution/blob/main/proposals/0213-literal-init-via-coercion.md#source-compatibility and to summarize our conversation off GitHub — it could be useful to make the behavior easier to discover for users, either via adding a note to "ExpressibleBy*Literal" protocols documentation pages or a diagnostic.

I think this is a consequence of SE-0213: Literal initialization via coercion:

T(literal) should construct T using the appropriate literal protocol if possible.

If I remember correctly, the goal of this proposal was in fact to make sure that 1 as T and T(1) have the same behavior (if T conforms to the literal protocol).


⚠ Bofrost.fr has degraded performance

In 564e2cd, Bofrost.fr (https://www.bofrost.fr) experienced degraded performance:

HTTP code: 200
Response time: 1906 ms

Resolved: Bofrost.fr performance has improved in a3dc285 after 6 minutes.


go get failed

when I input go get go.pedge.io/lion , I got the following error
package go.pedge.io/lion: unrecognized import path "go.pedge.io/lion" (parse https://go.pedge.io/lion?go-get=1: no go-import meta tags ())

seems likes the host server is down

I'm sorry, I recently switched hosting providers and I didn't find out about this! Should be back up now, my apologies again.

@peter-edge the server down again

It's due to outages with https://sloppy.io, I'm doing all I can to ping them to get this back up but they have been down a lot recently https://status.sloppy.io and they recently sent an email saying they're suspending accounts at their will to make sure some customers stay up. I'll keep you updated.

Should be back up now.


build: upgradee angular 11

Updated Angular to version 11
close #121

@bbortt please review agagin

@bbortt please review. thanks

introduced in #126.


log: shouldn't log secret object

http://github.com/apache/apisix-ingress-controller/blob/3d0b9769894197ba3138cd3b96d775ba2163e0c8/pkg/ingress/secret.go#L376-L376

the link is invalid.

get

@PGDream do you want to pick up this task?

@PGDream do you want to pick up this task?

y

@PGDream Assigned, thanks

[WeOpen Star]I would like to help


Fixed YAML config to properly escape output requirements for interpolation

This PR fixes #1754 and #528

Disclaimer: This review was made by a crew of AI Agents.
Code Review Comment for PR #1762
Overview
This PR introduces significant improvements to file path handling by replacing the legacy os.path module with the more robust pathlib and enhances YAML configuration management with proper string interpolation techniques. These changes enhance code robustness, maintainability, and overall adherence to modern Python standards.
Positive Aspects

File Path Handling:

The migration to pathlib.Path is a welcome modernization, ensuring safer and more concise file path operations.
The use of expanduser() effectively addresses user directory paths, and resolve() ensures canonical paths are utilized.
Implementing directory creation with parents=True and exist_ok=True simplifies the process of making nested directories and reduces potential errors.

String Interpolation:

Introduced a clearer, more maintainable method for interpolating strings within YAML configurations, separating concerns effectively with the interpolate_only method.

Suggestions for Improvement

Error Handling in File Operations:

I recommend adding comprehensive error handling in the _save_file method. This could include checks for output_file validity and handling file-related exceptions elegantly, allowing the function to provide meaningful error messages. Example implementation:

def _save_file(self, result: Any) -> None:
    if self.output_file is None:
        raise ValueError("output_file is not set.")
    try:
        ...
    except (OSError, IOError) as e:
        raise RuntimeError(f"Failed to save output file: {e}")

Type Hinting:

Enhancing type hints throughout the code will improve readability and clarity. Adding type hints to the _save_file method will convey expected argument types clearly.

def _save_file(self, result: Union[Dict[Any, Any], str]) -> None:

Input Validation in String Interpolation:

Input validation and robust error handling for the interpolate_only method are critical to prevent runtime errors due to missing inputs. A validation check could be added early in the function:

if not input_string:
    return input_string

Constants for Repeated Strings:

Using constants (e.g., BRACE_OPEN, BRACE_CLOSE) instead of hardcoded strings for braces in the interpolate_only method will enhance maintainability and readability.

BRACE_OPEN = "{"
BRACE_CLOSE = "}"

General Recommendations

Unit Testing: It’s crucial to develop unit tests that cover all new functionalities, especially edge cases in path resolution and string interpolation. This ensures confidence in the reliability of added features.

Documentation: Enhance the docstrings:

_save_file should have comprehensive documentation demonstrating usage and explaining parameters.
Include examples in the interpolate_only method documentation to aid future developers in understanding its use.

Logging: Integrating logging statements will provide insights into execution flow and debugging support. For instance:
import logging

logger = logging.getLogger(__name__)

def _save_file(self, result: Any) -> None:
    logger.debug(f"Saving result to file: {self.output_file}")
    ...
    logger.info(f"Successfully saved result to {resolved_path}")

Conclusion
The changes presented in this PR exhibit a clear advancement in code quality and application functionality. With the recommended improvements aimed at error handling, type safety, thorough documentation, and testing, the robustness of the code will be significantly enhanced. These changes also align well with Python best practices. Overall, great job on these updates!


[Feature] Add News Letter Button in CSEdge Learn

Description
Use this pre-made code by LinkedIn
Add the button into the web at correct position and place that would fit for it.

Subscribe on LinkedIn

Screenshots
No response
Checklist

[X] I have checked the existing issues.
[X] I follow Contributing Guidelines of this project.
[X] I am a GSSoC'24 contributor.
[X] I want to work on this issue.

Code of Conduct

[X] I agree to follow this project's "Code of Conduct".

please assign me this issue

Can asssign to me I working on issue under Gssoc 24

please add label

please review my pr @CSEdgeOff

please review my pr @CSEdgeOff

https://github.com/CSEdgeOfficial/learn.csedge.courses/pull/414
kindy please review my pr Im made UI part you will merge the pr It useful to me work the issue @CSEdgeOff
Kindly please please check out it @CSEdgeOff


the program crashes when change package.use without root privileges

When I run command like "fluggie app-name +useflag" with attributes to change package.use/* files with regular user, which don't have admin privileges, then flaggie crashes.
With root privileges - it's ok.

I think, that file /usr/bin/flaggie must be placed at /usr/sbin directory.
I think, that flaggie must catch an errors when write files (not just write write, but try-write).

flaggie flaggie +test
At argv[2]='+test': test seems to be an incorrect flag for app-portage/flaggie
Traceback (most recent call last):
File "/usr/lib/python-exec/python3.5/flaggie", line 25, in
sys.exit(main(sys.argv))
File "/usr/lib64/python3.5/site-packages/flaggie/cli.py", line 208, in main
pfiles.write()
File "/usr/lib64/python3.5/site-packages/flaggie/packagefile.py", line 524, in write
f.write()
File "/usr/lib64/python3.5/site-packages/flaggie/packagefile.py", line 388, in write
f.write()
File "/usr/lib64/python3.5/site-packages/flaggie/packagefile.py", line 288, in write
dir=os.path.dirname(os.path.realpath(self.path)))
File "/usr/lib64/python3.5/tempfile.py", line 549, in NamedTemporaryFile
(fd, name) = _mkstemp_inner(dir, prefix, suffix, flags, output_type)
File "/usr/lib64/python3.5/tempfile.py", line 260, in _mkstemp_inner
fd = _os.open(file, flags, 0o600)
PermissionError: [Errno 13] Отказано в доступе: '/etc/portage/package.use/tmpxd74cptm'

sbin paths are ugly, and I'm not going to go into that mess. As for appropriate privileges, that's obvious. As for error messages, I don't really see much of a point in introducing pretty wrapper if the exception message clearly states what the problem is.


⚠ ClamAV has degraded performance

In d41778c, ClamAV (https://spamassassin.apache.org/) experienced degraded performance:

HTTP code: 200
Response time: 88 ms

Resolved: ClamAV performance has improved in 1260f0a.


Allow for a default optional resource class

We now have the choice of adding optional additional classes for resources which is wonderful. However, since our templates are format-centric, those optional classes tend to often be the same from resource to resource or there actually only may be one option that is always correct. Because of this, it would be useful to be able to offer a default "optional resource class", one that is already marked as in use, with, of course, the ability to uncheck or change.
in the template pcc:bf2:Monograph:Instance in dev, there are two choices of optional classes: http://id.loc.gov/ontologies/bibframe/Print & http://id.loc.gov/ontologies/bibframe/Manuscript
95% of the materials that a cataloger will catalog using this form will use the optional class http://id.loc.gov/ontologies/bibframe/Print. http://id.loc.gov/ontologies/bibframe/Manuscript will only be used in some very specific cases. So a default would be very useful for efficiency's sake
related issue #3421

This is completely implementable, but I would suggest it is too complex for the last week of the work cycle.


New user with many errors when running fMRI-prepped BOLD images through standard fc pipeline

Thank you for making this pipeline! We are excited to try it out in our lab. We have already run fMRIprep on multi-session resting state data, and we are attempting to run (or at least test) one of the RSFC pipelines. We tried following the instructions to the letter, but the pipeline is failing very early and often; there must be a problem with how we're specifying the location of our files or the design file(?) Per the instructions, we are trying to run a functional pipeline without the structural one first.
Cohort file
id0,id1,img
sub-abc123,ses-Baseline,sub-abc123/ses-Baseline/func/sub-abc123_ses-Baseline_task-rest_space-MNI152NLin2009cAsym_desc-preproc_bold.nii.gz
sub-abc123,ses-Session1,sub-abc123/ses-Session1/func/sub-abc123_ses-Session1_task-rest_space-MNI152NLin2009cAsym_desc-preproc_bold.nii.gz
sub-abc123,ses-Session4,sub-abc123/ses-Session4/func/sub-abc123_ses-Session4_task-rest_space-MNI152NLin2009cAsym_desc-preproc_bold.nii.gz
sub-abc123,ses-Session5,sub-abc123/ses-Session5/func/sub-abc123_ses-Session5_task-rest_space-MNI152NLin2009cAsym_desc-preproc_bold.nii.gz

Design File
To test things out, we're simply using an unaltered version of the following (from cloned Github repo):
fc-36p_despike.dsn

Error message
I'm not sure exactly what to post here--there are hundreds of errors. Here are a few snippets (clearly the pipeline isn't really running)
Constructing a pipeline based on user specifications
····································································
· [D][/design/design.dsn]
INFO: /root/matlab/startup.m does not exist ... creating
sed: cannot rename /design/sedHvoTT8: Device or resource busy
· [I][/scratch]
· [C][/cohort/cohort.csv]
Error in file(file, ifelse(append, "a", "w")) :
  cannot open the connection
Calls: write.table -> file
In addition: Warning message:
In file(file, ifelse(append, "a", "w")) :
  cannot open file '/cohort/cohort.csv': Read-only file system
Execution halted
· [O][/xcpOutput]
· [M][Executing serially]

Cohort file issue is the same as #268
Later, it's not finding the design file?
###################################################################
#  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  #
#                                                             	#
#  ⊗               	XCP ENGINE LOCALISER                	⊗  #
#                                                             	#
#  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  #
###################################################################
/xcpEngine/core/parseArgsLocaliser: line 113: /data/jux/BBL/projects/testACT/script/designfiles/fc-36p_despike.dsn: No such file or directory

Current processing step:
Localising subject metadata...
····································································
· [Parsing subject identifiers]
· [Creating output directory]
· [/xcpOutput/sub-abc123/ses-Baseline]
· [Initialising space mapping]
· [Initialising derivative maps]
· [Initialising atlas system]
· Ingress
· [Native space: sub-abc123_ses-Baseline_native]
· [Anatomical space: sub-abc123_ses-Baseline_anatomical]
fixed '' is not an image
· [Priming analytic audit]
· [Priming quality assessment]
····································································
Processing step complete:
Localising subject metadata...

Current processing step:
Localising image
····································································
· [Verifying that images are present]
· [Conforming and localising images]
····································································
Processing step complete:
Localising image

Current processing step:
Localising design
····································································
· [Writing localised design]
/xcpEngine/core/functions/printx: line 2: /data/jux/BBL/projects/testACT/script/designfiles/fc-36p_despike.dsn: No such file or directory
· [Evaluating subject variables]
cp: missing destination file operand after '/xcpOutput/sub-abc123/ses-Baseline/template.nii.gz'
Try 'cp --help' for more information.

later,
###################################################################
#  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  #
#                                                             	#
#  ⊗             	FMRI PREPROCESSING MODULE             	⊗  #
#                                                             	#
#  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  #
###################################################################

[I][/xcpOutput/sub-abc123/ses-Baseline/sub-abc123_ses-Baseline.nii.gz]
[O][/xcpOutput/sub-abc123/ses-Baseline/prestats]
/xcpEngine/core/functions/rerun: line 3: ((: == 1 : syntax error: operand expected (error token is "== 1 ")

Current processing step:
Ensure matching orientation
····································································
· Input: /scratch/sub-abc123_ses-Baseline-prestats-858165278~TEMP~.nii.gz
· Template:
· Output root:
· checking the orientation of img and template
NATIVE:LPI TEMPLATE:
· img and template orientation are not the same
· make it: LPI ->
/xcpEngine/core/functions/cleanup: line 3: ((: == 1 : syntax error: operand expected (error token is "== 1 ")

Module complete

later,
###################################################################
#  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  #
#                                                             	#
#  ·               	CONFOUND MODEL MODULE               	·  #
#                                                             	#
#  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  #
###################################################################

[I][/xcpOutput/sub-abc123/ses-Baseline/sub-abc123_ses-Baseline.nii.gz]
[O][/xcpOutput/sub-abc123/ses-Baseline/confound2]
/xcpEngine/core/functions/rerun: line 3: ((: == 1 : syntax error: operand expected (error token is "== 1 ")

Current processing step:
Generating confound matrix
····································································
· Generating temporal mask
Error in getopt_options(object, args) :
  Error in getopt(spec = spec, opt = args) : flag "i" requires an argument
Calls: parse_args -> parse_options -> getopt_options
Execution halted
Error in getopt_options(object, args) :
  Error in getopt(spec = spec, opt = args) : flag "i" requires an argument
Calls: parse_args -> parse_options -> getopt_options
Execution halted
Error in getopt_options(object, args) :
  Error in getopt(spec = spec, opt = args) : flag "i" requires an argument
Calls: parse_args -> parse_options -> getopt_options
Execution halted
· [Computing framewise displacement]
Error in file(file, "rt") : cannot open the connection
Calls: unname -> as.matrix -> read.table -> file
In addition: Warning message:
In file(file, "rt") :
  cannot open file '/xcpOutput/sub-abc123/ses-Baseline/confound2/mc/sub-abc123_ses-Baseline_realignment.1D': No such file or directory
Execution halted
· [Computing DVARS]
· [Selecting quality criteria]
· [Applying framewise threshold to time series]
Error in getopt_options(object, args) :
  Error in getopt(spec = spec, opt = args) : flag "m" requires an argument
Calls: parse_args -> parse_options -> getopt_options
Execution halted
· [Evaluating data quality]
/xcpEngine/core/functions/temporal_mask: line 197: /xcpOutput/sub-abc123/ses-Baseline/confound2/mc/sub-abc123_ses-Baseline_tmask.1D: No such file or directory
· relative maximum motion
Error in file(file, "rt") : cannot open the connection
Calls: read.table -> file
In addition: Warning message:
In file(file, "rt") :
  cannot open file '/xcpOutput/sub-abc123/ses-Baseline/confound2/mc/sub-abc123_ses-Baseline_relRMS.1D': No such file or directory
Execution halted
· relative mean motion
Error in file(file, "rt") : cannot open the connection
Calls: read.table -> file
In addition: Warning message:
In file(file, "rt") :
  cannot open file '/xcpOutput/sub-abc123/ses-Baseline/confound2/mc/sub-abc123_ses-Baseline_relRMS.1D': No such file or directory
Execution halted
The nuisance matrix is being selected base on design files
/xcpEngine/modules/confound2/confound2.mod: line 155: ((: == 1 : syntax error: operand expected (error token is "== 1 ")
/xcpEngine/modules/confound2/confound2.mod: line 172: ((: == 1 : syntax error: operand expected (error token is "== 1 ")
/xcpEngine/modules/confound2/confound2.mod: line 190: ((: == 1 : syntax error: operand expected (error token is "== 1 ")
/xcpEngine/modules/confound2/confound2.mod: line 215: ((: == 1 : syntax error: operand expected (error token is "== 1 ")
/xcpEngine/modules/confound2/confound2.mod: line 235: ((: == 1 : syntax error: operand expected (error token is "== 1 ")
/xcpEngine/modules/confound2/confound2.mod: line 271: ((: != 0 : syntax error: operand expected (error token is "!= 0 ")
/xcpEngine/modules/confound2/confound2.mod: line 300: ((: > 0 : syntax error: operand expected (error token is "> 0 ")
/xcpEngine/modules/confound2/confound2.mod: line 326: ((: > 1 : syntax error: operand expected (error token is "> 1 ")
/xcpEngine/modules/confound2/confound2.mod: line 346: ((: == 1 : syntax error: operand expected (error token is "== 1 ")
/xcpEngine/modules/confound2/confound2.mod: line 381: ((: == 1 : syntax error: operand expected (error token is "== 1 ")
/xcpEngine/modules/confound2/confound2.mod: line 403: ((: == 1 : syntax error: operand expected (error token is "== 1 ")
····································································

Runtime Information
xcpengine-docker \
  -c /data2/1305/ignore/xcp/1305_cohort.csv \
  -d /data2/Github/xcpEngine/designs/fc-36p_despike.dsn \
  -i /data2/1305/derivatives/xcp/tmp_abc \
  -o /data2/1305/derivatives/xcp \
  -r /data2/1305/derivatives/fmriprep

Additional context
Is it the case that it is not locating the image files correctly? Is it b/c the cohort file is read-only? Might I have to edit the design file? Apologies for being a bit lost with this.
Many thanks,
Brian

there is problem with acess to image in the  directory and all your command is good

Thanks for your answer. Do you think it is a permissions issue? I just tried running again and double-checked. Everything is open for read, write, and execute access. Or by access do you mean it's not finding the right images? All that's needed is the BOLD file, right?

I am experiencing virtually the same issue also running in Docker. I fixed the "read only" error for the cohort file by running Docker directly; however, this has not fixed the fact that the pipeline is not receiving my imaging files.
I tried running the cbf_all pipeline as well with similar results.

@bwinsto2 pls can you try singularity image or docker image pls. Sorry for late reply
@daphneneuro17 that should be a permission  issue at your end

@a3sha2 thanks for the reply, but I'm not sure what else to try. I downloaded the newest docker image and ran with the same command. I've tried with other datasets, but I'm receiving the same errors. I'm not sure how this could be a permissions issue because I've made sure the design file, cohort file, and all data files are open for read, write, and execute access for all users.
xcpengine-docker \   -c /data2/1305/ignore/xcp/1305_cohort.csv \   -d /data2/1305/ignore/xcp/fc-36p.dsn \   -i /data2/1305/derivatives/xcp/tmp_abc \   -o /data2/1305/derivatives/xcp \   -r /data2/1305/derivatives/fmriprep/
Do you have any ideas why the software is not finding my data files (I assume that is the problem)?
It sounds like @daphneneuro17 is experiencing the same problem.

It looks like this is the same (unresolved?) problem in #199 and #204
Here is my command for running it directly with docker, and the error I get:

Here is the analysis variables section of the design file:

I have tried keeping the design analysis variable in the design file as the default from the repository. I have also tried changing the variable to the place on my disk where I keep the file (referring to itself). I get the same error.
@a3sha2 or @mattcieslak any ideas? @a3sha2 you have noted previously that the issue is with "docker binding": is there anything more specific you could suggest to resolve this?

@bwinsto2 the problem is  directory of the design file

@bwinsto2  the main issue is design file, you need to give design file permission after you save it in another directory. For xcpengine to work, the design and cohort file must be read and write. try chmod +rwx  on both design and cohort file. the issues   #199 and #204 have resolved

It looks like you have the order of your arguments reversed for the -v flags. If your cohort file located on your machine at /data2/1305/ignore/xcp/1305_cohort.csv then your command should have -v /data2:/data
You can check by doing something like:
$ docker run --rm -it --entrypoint bash -v /data2:/data2 -v /tmp:/tmp
$ find /data2/1305

and this should show your files. Let us know if this works!

@mattcieslak thanks for looking into this! I suppose I misunderstood docker binding in this scenario. I did what you suggested, and the pipeline is running! Funny that adding a single character ("2") made all the difference.
Appreciate your help @a3sha2 and @mattcieslak

Suppose it's worth mentioning that my issue using the wrapper is perhaps unrelated(?).
I don't know if @daphneneuro17 has gotten it working by interacting with Docker directly, but it sounds like she had the same issue running w/ the wrapper.


Exporting model to ONNX

Hi, I wanted to export to ONNX from a trained model.
As mentionned here : you parse a tuple as input  #49
import os
import re
from typing import List, Optional, Tuple, Union

import click
import dnnlib
import numpy as np
import PIL.Image
import torch

import legacy

network_pkl = "my_model.pkl"

print('Loading networks from "%s"...' % network_pkl)
device = torch.device('cuda')
with dnnlib.util.open_url(network_pkl) as f:
    G = legacy.load_network_pkl(f)['G_ema'].to(device) # type: ignore

model = G

# Labels.
label = torch.zeros([1, G.c_dim], device=device)

z = torch.from_numpy(np.random.RandomState(0).randn(1, G.z_dim)).to(device).float()

# Export the model
torch.onnx.export(model,                            # model being run
                    (z, label),                          # model input (or a tuple for multiple inputs)
                    "export.onnx",                  # where to save the model (can be a file or file-like object)
                    export_params=True,             # store the trained parameter weights inside the model file
                    opset_version=9,                # the ONNX version to export the model to
                    do_constant_folding=True,       # whether to execute constant folding for optimization
                    input_names = ['latent', 'label'],       # the model's input names
                    output_names = ['image']        # the model's output names
                    )

It still gives error :
G:\Python\fastgan2\projected_gan\pg_modules\blocks.py:40: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert nc % 2 == 0, 'channels dont divide 2!'
G:\Python\fastgan2\projected_gan\pg_modules\blocks.py:41: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  nc = int(nc/2)
G:\Anaconda3\envs\fast_gan\lib\site-packages\torch\onnx\symbolic_helper.py:381: UserWarning: You are trying to export the model with onnx:Upsample for ONNX opset version 9. This operator might cause results to not match the expected results by PyTorch.
ONNX's Upsample/Resize operator did not match Pytorch's Interpolation until opset 11. Attributes to determine how to transform the input were added in onnx:Resize in opset 11 to support Pytorch's behavior (like coordinate_transformation_mode and nearest_mode).
We recommend using opset 11 and above for models using this operator.
  warnings.warn("You are trying to export the model with " + onnx_op + " for ONNX opset version "
G:\Anaconda3\envs\fast_gan\lib\site-packages\torch\onnx\symbolic_helper.py:258: UserWarning: ONNX export failed on adaptive_avg_pool2d because input size not accessible not supported
  warnings.warn("ONNX export failed on " + op + " because " + msg + " not supported")
Traceback (most recent call last):
  File "G:\Python\fastgan2\projected_gan\export.py", line 29, in <module>
    torch.onnx.export(model,                            # model being run
  File "G:\Anaconda3\envs\fast_gan\lib\site-packages\torch\onnx\__init__.py", line 316, in export
    return utils.export(model, args, f, export_params, verbose, training,
  File "G:\Anaconda3\envs\fast_gan\lib\site-packages\torch\onnx\utils.py", line 107, in export
    _export(model, args, f, export_params, verbose, training, input_names, output_names,
  File "G:\Anaconda3\envs\fast_gan\lib\site-packages\torch\onnx\utils.py", line 724, in _export
    _model_to_graph(model, args, verbose, input_names,
  File "G:\Anaconda3\envs\fast_gan\lib\site-packages\torch\onnx\utils.py", line 497, in _model_to_graph
    graph = _optimize_graph(graph, operator_export_type,
  File "G:\Anaconda3\envs\fast_gan\lib\site-packages\torch\onnx\utils.py", line 216, in _optimize_graph
    graph = torch._C._jit_pass_onnx(graph, operator_export_type)
  File "G:\Anaconda3\envs\fast_gan\lib\site-packages\torch\onnx\__init__.py", line 373, in _run_symbolic_function
    return utils._run_symbolic_function(*args, **kwargs)
  File "G:\Anaconda3\envs\fast_gan\lib\site-packages\torch\onnx\utils.py", line 1032, in _run_symbolic_function
    return symbolic_fn(g, *inputs, **attrs)
  File "G:\Anaconda3\envs\fast_gan\lib\site-packages\torch\onnx\symbolic_helper.py", line 172, in wrapper
    return fn(g, *args, **kwargs)
  File "G:\Anaconda3\envs\fast_gan\lib\site-packages\torch\onnx\symbolic_opset9.py", line 1346, in batch_norm
    weight, bias, running_mean, running_var = sym_help._batchnorm_helper(g, input, weight, bias, running_mean, running_var)
  File "G:\Anaconda3\envs\fast_gan\lib\site-packages\torch\onnx\symbolic_helper.py", line 726, in _batchnorm_helper
    raise RuntimeError("Unsupported: ONNX export of batch_norm for unknown "
RuntimeError: Unsupported: ONNX export of batch_norm for unknown channel size.

Thanks in advance

it seems that ONNX does not handle the abstractions in the code, and it will be a major change to support it. I think it should be possible to write a converter function for the BatchNorms, I am happy to accept PRs on this issue.


Vapor3 (Same as ErrorMiddleware)

Not sure why but valid requests aren't resolved correctly
Closes #2

Hi Tim, I was updating the tests when I came across ThrowingViewRenderer. The main problem is that ViewRenderer doesn't even exist anymore. The simplest solution would be to use the same flow as in Leaf's tests (and include Leaf as a test dependency)

Any status on this? 😄 🚀

I'll see if I can get some time on it!

Testing depends on https://github.com/vapor/vapor/issues/1637

@bre7 can you change the base branch to vapor3? Will make it easier for me to pull in and test

Done

Going to merge this so I can work on it locally on the flight!


Ldap bind credential related setting keys fixed.

Ldap bind credential related setting keys fixed.

Codecov Report

Merging #13 (6d64e19) into main (1798bd6) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main      #13   +/-   ##
=======================================
  Coverage   98.79%   98.79%
=======================================
  Files           8        8
  Lines         332      332
=======================================
  Hits          328      328
  Misses          4        4

Thanks for the correction!


Possible incorrect getDocumentHighlights behavior with global lexically scoped variables

There seem to be some related funny issues going on. When calling getDocumentHighlights on the position of the hello variable declaration in the following code snippet:
// Incorrect case:

class HelloWorld {
    constructor(public name: string) { }
    sayHello() {
        return `Hello, ${this.name}!`;
    }
};

let hello = new HelloWorld("TSLint");
hello.sayHello();

only the declaration is returned and not the usage on the next line. However, in the following code snippet, both uses are returned, correctly:
// Correct case:

class HelloWorld {
    constructor(public name: string) { }
    sayHello() {
        return `Hello, ${this.name}!`;
    }
};

if (true) {
    let hello = new HelloWorld("TSLint");
    hello.sayHello();
}

// Correct case (var vs. let):

class HelloWorld {
    constructor(public name: string) { }
    sayHello() {
        return `Hello, ${this.name}!`;
    }
};

var hello = new HelloWorld("TSLint");
hello.sayHello();

Also strangely, Sublime will correctly show both references in all three cases.
I've been trying to figure out the root cause but haven't been able to yet. It seems that the failure occurs specifically for lexically scoped variables declared at global scope, such as let declarations at global scope.
In case it helps, I did notice that in the getReferencesInNode function the array of possible positions contains exactly the two actual positions in both the incorrect and correct case. However, there is a difference when finding the relatedSymbol:
var referenceSymbol = typeChecker.getSymbolAtLocation(referenceLocation);
if (referenceSymbol) {
    var referenceSymbolDeclaration = referenceSymbol.valueDeclaration;
    var shorthandValueSymbol = typeChecker.getShorthandAssignmentValueSymbol(referenceSymbolDeclaration);
    var relatedSymbol = getRelatedSymbol(searchSymbols, referenceSymbol, referenceLocation);
    // NOTE (added for the Github issue):
    // relatedSymbol is undefined here in incorrect case, defined in the correct case
    if (relatedSymbol) {
        var referencedSymbol = getReferencedSymbol(relatedSymbol);
        referencedSymbol.references.push(getReferenceEntryFromNode(referenceLocation));
    }
    else if (!(referenceSymbol.flags & 67108864 /* Transient */) && searchSymbols.indexOf(shorthandValueSymbol) >= 0) {
        var referencedSymbol = getReferencedSymbol(shorthandValueSymbol);
        referencedSymbol.references.push(getReferenceEntryFromNode(referenceSymbolDeclaration.name));
    }
}

(Related to https://github.com/palantir/tslint/issues/570)

@DanielRosenwasser Figured out the problem I believe, In TSLint, we constructed a language service host object as such:
    export function createLanguageServiceHost(fileName: string, source: string) {
        const host: ts.LanguageServiceHost = {
            getCompilationSettings: () => Lint.createCompilerOptions(),
            getCurrentDirectory: () => "",
            getDefaultLibFileName: () => "lib.d.ts",
            getScriptFileNames: () => [fileName],
            getScriptSnapshot: () => ts.ScriptSnapshot.fromString(source),
            getScriptVersion: () => "1",
            log: (message) => { /* */ }
        };

        return host;
    }

The issue arises here:
getScriptSnapshot: () => ts.ScriptSnapshot.fromString(source),

When the language services seem requests a snapshot for the lib file, we were returning the source for the source file which is obviously non-sensical. Strangely enough, everything still worked fine in most cases, but not all cases (see above).
Do you have a suggestion on a good way to change this?

@mhegazy can check me on this, but consider just returning an empty string. That way there won't be any conflicting declarations. Let us know if that fixes the issue.

Thanks! This change you suggested seems to fix the issues we were having:
getScriptSnapshot: name => ts.ScriptSnapshot.fromString(name === fileName ? source : ""),

So this won't mess anything up? Seems a bit hacky and want to double-check on that to be safe

this looks good to me. an empty lib file should do the trick.

so is this issue resolved now?

Yes, thanks for the help. #3688 should be resolved as well, it was caused by the same issue I believe.


Output command task not outputting anything

Describe the bug
Have a pipeline setup to run terraform apply then run terraform output to get some parameters needed for a different stage in the pipeline. The terraform output gets nothing.
To Reproduce
Steps to reproduce the behavior:

Setup pipeline as (include yaml configuration or screenshots of classic ui editor)
Execute pipeline
See error

Expected behavior
I expect to get outputs from my terraform.
Screenshots

Pipeline Logs
/opt/hostedtoolcache/terraform/1.8.0/x64/terraform version
Terraform v1.8.0
on linux_amd64

provider registry.terraform.io/hashicorp/azurerm v3.97.1
Finishing: Terraform Output
/usr/bin/bash /home/vsts/work/_temp/cec1ea38-9319-4654-8190-ed530bcfdd74.sh
ftp_credentials is
test is

Agent Configuration

OS: Ubuntu 22.04.4
Hosted
Terraform version used: 1.8.0
AzureCLI version used: N/A

Mistake on my part. Ignore.


Set PAGEFLOW_EDITOR global in scrolled editor

Follow up for #1284. As long as we still load the paged js in the
editor, this global has to be set.
REDMINE-17269

Coverage decreased (-0.008%) to 99.017% when pulling 2c9b6de8bc42086484f2ea0242cbb94c75cf3ded on tf:scrolled-pageflow-editor-global into 0fa72a514f620f169737a4eb908630fd8d0064da on codevise:master.


Validation triggers

Validation, is not triggered when editing rules, which must be the case.

This might be related to JSON Forms (Notifications wise).

@edgarmueller I'm having some problems understanding how "allOf" and "additionalProperties" fit together. I get some unexpected validation errors when editing the rules if addtionalProperties is set to false. We can look at this together in the office.


Fixed compatiblity issues with python 3.9

According to #17, the code does not run with python 3.9. I updated tox to include 3.9 and also 3.12. I dropped 3.8 because it never worked and support for python will drop in October.
This PR contains code changes which makes it compatible with 3.9. Most of the changes fix the new typing syntax. In api.py the decorator for re-login has to be moved into a function instead a static method.

Summary by Sourcery
Fix compatibility issues with Python 3.9 by updating typing syntax and refactoring the re-login decorator. Update the build configuration to support Python 3.9 through 3.12 and remove older versions. Modify CI workflows to test against these versions and update documentation to reflect these changes.
Bug Fixes:

Fix compatibility issues with Python 3.9 by updating typing syntax and refactoring code.

Enhancements:

Refactor the re-login decorator in api.py to be a standalone function instead of a static method.

Build:

Update setup.cfg to include support for Python versions 3.9, 3.10, 3.11, and 3.12, and remove support for Python 3.7 and 3.8.

CI:

Modify the CI workflow to include testing for Python versions 3.9 and 3.12, in addition to 3.10 and 3.11.
Update the GitHub Actions upload-artifact step to use version 3.

Documentation:

Update developer documentation to include new tox environments for Python 3.9 and 3.12, and provide guidance on using pyenv with tox.

Tests:

Update test cases to use the Union type for typing annotations to ensure compatibility with Python 3.9.

This seems to work for me on a system that only has Python 3.9.


add persistent player data to local storage.

this will create and retrieve, for immediate use, the player name, number and number of attempts.

-- global var for swapping in/out the latest from local storage
-- add localStorage.set item method to click event for num
-- add if/else to sync localstore and global array

todo: make sure this is coming from arrays we are adding to, so we can show more than one game


Remove background color on figure element.

Removes video background here https://api.cannabis.ca.gov/?page_id=13062&preview=1&_ppp=be62ac1a46

Compare https://remove-video-background.pr.cannabis.ca.gov/test-for-seo-editor/
to https://cannabis.ca.gov/test-for-seo-editor/


bug: block hash out of range

Bug Report
Madara version:
349e7c1044eb9d715a6b06d259678ec9c4dd49fd
Current behavior:
Block hashes (or maybe other hashes too; haven't tested) reported are full 256-bit hashes.
Expected behavior:
All hashes should be bounded by the prime to be valid field elements.
Steps to reproduce:

Run Maara with --dev:cargo run -- --dev

Query the latest block hash:curl "http://localhost:9933/" -H "Content-Type: application/json" -d '{"jsonrpc":"2.0","id":1,"method":"starknet_blockHashAndNumber","params":[]}'

Check that the resulting block hash is 256-bit.

Related code:
N/A
Other information:
N/A

does this mean that hash return should be hash % P instead?

The issue is here if I'm right
        Ok(BlockHashAndNumber { block_hash: format!("{:#x}", block_hash), block_number })

The problem is not RPC related. Our whole codebase uses U256 and H256 instead of Felts. We did it initially because it was more handy but is back firing now.
In order to fix this:
In crates/primitives/starknet/src/block/header.rs, we must change this function.
    #[must_use]
    pub fn hash(&self) -> H256 {
        H256::from_slice(frame_support::Hashable::blake2_256(&self.block_number.encode()).as_slice())
    }

Its signature should return a Felt and its implem should respect the starknet specification defined here: https://docs.starknet.io/documentation/architecture_and_concepts/Blocks/header/#block_hash
Then we have to change everything that uses this type downstream: BlockHeader type definition, runtime storage, backend and rpc implem.
A problem we will have is that substrate expects some traits for the types it accepts in its storage. I think we will have to define our own SubstrateFelt(Felt) newtype.
I think such a type already exists with Felt252Wrapper(pub U256);  defined in crates/primitives/starknet/src/execution/program_wrapper.rs. I believe this is the type we should store in the runtime. But when using it in our program we should convert it to cairo_vm::felt::Felt252.

The core of our logic is to interact with Blockifier which in turn call cairo-vm. So we need to use cairo_vm::felt::Felt252 at some point, as it is the type expected by those libs. Therefore I think it is better to keep on with the Felt type defined by those libs, and propagate it along our codebase.
On the other hand, we are going to use starknet-rs types for the RPC layer, so we will probably need to convert it to your FieldElement at some point. But probably only for the RPC layer.
Wdyt?

see also https://github.com/keep-starknet-strange/madara/issues/128

@xJonathanLEI totally


Outer Skin Layer way messed up

First thought issue was 3D skin layers, but had no issues before. Disabled pack and went back to normal just without the animation (duh.) Assuming Pack isnt updated to 1.21.4 yet is the issue, or if that's just 1 issue of a few.

(same issue as #215)

I'm pretty sure this is a problem with EMF, they changed something in 1.21.2 and that broke skin layers
I'm pretty sure it's this change

As @SonaSudebta mentioned, I have a fork here with changes from @erinsc who made the fix, together with animation fixes from me https://github.com/ClaraTheRed/Fresh-Moves

How do i implement the fork?

You can download the resource pack here https://github.com/ClaraTheRed/Fresh-Moves/archive/refs/heads/main.zip
It includes both the version with the expressive eyes and the one without.


FlashDevelop versus VSCode compilation time difference

Hello Josh,
I noticed a big difference in terms of compilation time regarding FlashDevelop (~16s) versus VSCode (~34s). Here is the command ran from for each context:
FlashDevelop (14140594 bytes written in 16.085 seconds)
mxmlc-cli -load-config="C:\AIR\AIR_SDK_32_0/frameworks/air-config.xml" -load-config+=obj\Imago-DesktopConfig.xml -inline=true +configname=air -swf-version=43 -o obj\Imago-Desktop637081209041197666
Starting java as: java -Dsun.io.useCanonCaches=false -Xms32m -Xmx512m -Duser.language=en -Duser.region=US -Djava.util.Arrays.useLegacyMergeSort=true -Dapplication.home="C:\AIR\AIR_SDK_32_0" -jar "C:\AIR\AIR_SDK_32_0\lib\ascsh.jar"
INITIALIZING: ascsh v0.4 by Jeff Ward, simbulus.com
(fcsh)
Starting new compile.
Loading configuration: C:\AIR\AIR_SDK_32_0\frameworks\air-config.xml
Loading configuration: E:\Development\Air\LFI\obj\Imago-DesktopConfig.xml

14140594 bytes written to E:\Development\Air\LFI\obj\Imago-Desktop637081209041197666 in 16.085 seconds

VSCode (37171990 bytes written in 34.553 seconds)
Configuration file: e:\Development\Air\LFI\asconfig.desktop.json
Reading configuration file...
Validating configuration file...
Parsing configuration file...
SDK: c:\AIR\AIR_SDK_32_0
Compiling application...
C:\Program Files (x86)\Java\jre1.8.0_221\bin\java -Xms512M -Xmx1G -Dflexcompiler=c:\AIR\AIR_SDK_32_0 -Dflexlib=c:\AIR\AIR_SDK_32_0\frameworks -jar c:\AIR\AIR_SDK_32_0\lib\mxmlc-cli.jar +flexlib=c:\AIR\AIR_SDK_32_0\frameworks --debug=false +configname=air --output=bin/desktop/Imago-Desktop.swf --library-path+=lib --library-path+=lib/themes --source-path+=src --source-path+=../DragonBonesAS/DragonBones/src
--source-path+=../DragonBonesAS/Starling/Starling_2.x/src --source-path+=../Starling-Extension-Particle-System/src --default-size 800 480 --swf-version=43 --default-frame-rate=60 --define+=CONFIG::release,true --define+=CONFIG::debug,false --define+=CONFIG::isAir,true --define+=CONFIG::isAirMobile,false --define+=CONFIG::isAirDesktop,true --define+=CONFIG::embedSounds,true --define+=CONFIG::embedLevelChunks,true -- src/ch/adolio/Main.as
Loading configuration: c:\AIR\AIR_SDK_32_0\frameworks\air-config.xml

37171990 bytes written to E:\Development\Air\LFI\bin\desktop\Imago-Desktop.swf in 34.553 seconds

Apparently the number of bytes written are significantly different.

Do you have an idea from where could come this difference?
Is VSCode options better for a final release build?

Thanks in advance for your help.
Best,
Aurélien

It's difficult to say what is causing the size difference. It may be that more classes are being included for some reason. Perhaps some extra embedded assets.
You could try using a couple of different compiler options to see what differences there are:
The most useful would probably be size-report. The generated file will show how different parts of your app affect the final file size. Use the option in both VSCode and FlashDevelop and compare the two files.
You may also want to try using dump-config. Assuming that your compiler options are slightly different between FlashDevelop and VSCode, this might give you an idea of what you need to add or remove in VSCode's asconfig.json file.

I have this problem too with FlashBuilder, I tried many configurations, and the most close I got was with -load-externs=bin-debug/report.xml on modules asconfig, and -link-report=bin-debug/report.xml on the main app asconfig. But the true is that I'm not very confident, when I have some spare time I'll try the size-report too. Pls share results @Adolio

Thanks Josh for the investigation tracks! I didn't know about size-report & dump-config, very useful!
I finally found the time to restart digging into this... and I found my issue :) It was indeed coming from different compiler options linked to embedded assets. In the first case my sounds assets weren't embedded and in the second they were...
Sorry for the inconvenience & thanks again for the help!


[Feature request] give possibility to place  inside generated form

I find it strange there's no such request. Am I overlooked it?
Case is like this: I create form like that:
<Form .../>
<button type="submit"/>

Problem is, when focus is inside any input, and I click 'enter', button submit doesn't being clicked (and it shouldn't as it isn't inside .
I'd like to have possibility to have this button inside, not using custom templates / custom components.

The output of the Form component is a fieldset tag containing the fields. You can add a form tag wrapping the output:
<form action="..." method="..." or onClick={...}>
  <Form ... />
  <button type="submit"/>
</form>

Thanks!


POC/DRAFT: Add transforms

This is to showcase some of the changes I have in mind for #538 to do a major enhancement on the goss/gomega language. Check out the examples folder for an idea of where I'm going with this.
This is horrible code that needs to be cleaned up, just tried to get a quick braindump out there since the code explains it a lot better than my ticket.
For usage examples take a look at this:
https://github.com/aelsabbahy/goss/tree/add_transforms/examples

Hi there, I'm interested in having a fix for #483 and I would strongly prefer contributing to upstream istead of mainating my own fork. If I understand correctly, this MR is a potential fix: is there any way I could assist you in finishing it?

This PR is a potential fix in that it allows direct string matching of the command output. Rather than only limiting it to the current pattern matcher logic.
Here's an example with some work from this PR:
command:
  test_exec:
    exec: 'echo -e "This is an example output of a command\nGoss will print this in full when using the new string matcher\nas opposed to using the pattern matcher"'
    exit-status: 0
    stdout:
    - This is an example output of a command, broken
    - Goss will print this in full when using the new string matcher
    - as opposed to using the pattern matcher
    stderr: []
    timeout: 10000
  test_exec2:
    exec: 'echo -e "This is an example output of a command\nGoss will print this in full when using the new string matcher\nas opposed to using the pattern matcher"'
    exit-status: 0
    stdout: |
      This is an example output of a command, broken
      Goss will print this in full when using the new string matcher
      as opposed to using the pattern matcher
    stderr: []
    timeout: 10000

notice test_exec2 is testing stdout against a string rather than an array. The output of running this in the new version of goss will be:
.F..F.

Failures/Skipped:

Command: test_exec: stdout:
Expected
    {} # not implemented yet, will probably say "io.Reader{}", but it will not contain the command output
to contain patterns
    ["This is an example output of a command, broken","Goss will print this in full when using the new string matcher","as opposed to using the pattern matcher"]
the missing elements were
    ["This is an example output of a command, broken"]

Command: test_exec2: stdout:
Expected
    "This is an example output of a command\nGoss will print this in full when using the new string matcher\nas opposed to using the pattern matcher\n"
to equal
    "This is an example output of a command, broken\nGoss will print this in full when using the new string matcher\nas opposed to using the pattern matcher\n"

Total Duration: 0.002s
Count: 6, Failed: 2, Skipped: 0

This PR will be a pretty major refactor of how goss handles pattern matching and how results are output. It is a breaking change so it will be in goss v0.4.0.. and since v0.4.0 will be a breaking change, there's a few other breaking changes that I would like to have in there prior to release.
Here's my todo.md (and progress) for v0.4.0:
# TODO:

[X] Find way to get private unexported values from gomega matchers
[X] Implement gomega matcher for validate contains
[X] Remove old validatecontains code
[X] implement gjson transform
[X] Add equal matcher
[X] Use benumerically as defualt numbers
[X] Add toNumeric transformer
[-] create new testresult object
[x] Move all matcher data outside of matchers and into testresult object
[ ] Move all formatting related to testresult into outputs/
[-] Implement clean short failure messages for matchers
[ ] Check have-key message with matcher
[ ] Write docs
[ ] github.com/sanity-io/litter?
[ ] And matcher indent is funky/incorrect
[ ] get rid of have-key-with-value or dry out have-key-with-value?

# optional
[ ] catch .(string) type errors in gomega.go (if input is not controlled)
[ ] Implement have-patterns for array
[ ] all matcher potentially
[ ] investigate why string diff does not trigger for equal matcher
[ ] register matchers using init?

# Other v0.4.0 stories, since they potentially introduce breaking changes
[ ] psutil vs go-ps, possibly switch to the former
[ ] process, comm vs args. Can we match on either or with the psutil library?
[ ] rpm expand full version
[ ] rpm/deb version matchers? create generic generic version of semver.Range that can be used with a version interface. Create wrappers around 3rd party rpm/deb comparison libraries to honor version interface

Let me know if there's something in there that you're interested in helping out with. If I had to guess I'm about ~60% done with this PR. However, I haven't even started on any of the other v0.4.0 work.

I'll look into these early next week and will either do something or ask more questions then. Thank you for the detailed answer!

Awesome, please reach out to me before putting in work/time, I want to avoid a situation where you do work I already completed.
Also, is the string matcher change what you're looking for with regards to #483?
The reason it can't be done with patterns is because goss reads a file line by line. This is apparent if you run goss v on a GB text file or command output, the memory usage stays very low. Using string matcher would read the results into memory, but will have the benefit of displaying it.

I don't know yet how I would approach as the only thing I know for now is that delivering more information to the affected asset owner is a necessity. If I understand you correctly and it will work with text match but not with regexp, I am fine with it as it's infinitely better than the current situation which doesn't seem to have a workaround (at least I'm not aware of it).
I think I'll start with looking into this PR and figuring out if I can help with it, and then going trough the list and pinging you in PRs\issues.

My apologies for the delay, I don't know if I'll have time to look into this as I got a lot of other stuff on hands. @nloginov is interested in having this in place as well and will likely try to help you at some point in time.

@petemounce Tagging you on here since you are creating a bunch of PRs currently.
This is a major work stream that I'm working on for goss v4. I would love your feedback on it.
The manual.md changes and this file explain the differences:
examples/goss_awesome_gomega.yaml
I will most likely create a v4 branch, this + a few other PRs will be merged into it and hopefully I can get v4 out the door within a month. This is a fairly large re-write and may impact some of the PRs you have out there.

I like the input suggestion. The problem that I've been having is striking a balance between noisy default output and useful default outputs.
$ cat goss.yaml
matching:
  example:
    content: '{"foo": "bar", "moo" {"nested": "cow"}, "count": "15"}'
    matches:
      gjson:
        moo.nested: cow
        foo: {have-prefix: b}
        count: {le: 2}  # this will fail
        '@this': {have-key: "foo"}
        moo:
          and:
            - {have-key: "nested"}
            - {not: {have-key: "nested2"}}

And the different executions:
$ goss v
F

Failures/Skipped:

Matching: example: matches:
Expected
    15
to be <=
    2
the transform chain was
    [{"gjson":{"Path":"count"}},{"to-numeric":{}}]

Total Duration: 0.001s
Count: 1, Failed: 1, Skipped: 0
$ goss v -o include_raw
F

Failures/Skipped:

Matching: example: matches:
Expected
    15
to be <=
    2
the transform chain was
    [{"gjson":{"Path":"count"}},{"to-numeric":{}}]
the raw value was
    "{\"foo\": \"bar\", \"moo\" {\"nested\": \"cow\"}, \"count\": \"15\"}"

Total Duration: 0.001s
Count: 1, Failed: 1, Skipped: 0
$ goss v --format tap
1..1
not ok 1 - Matching: example: matches: Expected "15" to be <= 2 the transform chain was [{"gjson":{"Path":"count"}},{"to-numeric":{}}]
$ goss v --format tap -o include_raw
1..1
not ok 1 - Matching: example: matches: Expected "15" to be <= 2 the transform chain was [{"gjson":{"Path":"count"}},{"to-numeric":{}}] the raw value was "{\"foo\": \"bar\", \"moo\" {\"nested\": \"cow\"}, \"count\": \"15\"}"

-o include_raw vs -o exclude_raw is really more about what gets set as the default.
Errors are caught like other goss errors are. That said, I think the tansforms need more clearer error messages and to explain which transform encountered it, since this isn't very clear.
$ goss v
F

Failures/Skipped:

example: matches: Error: strconv.ParseFloat: parsing "15x": invalid syntax

Total Duration: 0.000s
Count: 1, Failed: 1, Skipped: 0

I like the input suggestion. The problem that I've been having is striking a balance between noisy default output and useful default outputs.

My preference is for an error to tell me all of the inputs and all of the outputs, so that I don't need to run the test again to find them.
I think that for machine-state assertions, that's more-than-normally valuable as a default due to the risk of intermittent failures. I think that because machine-state is not an isolated system; it's fraught with external influencers, dependencies, and those squishy meatbags changing things on what would otherwise be shining clean examples of computational perfection. :)

-o include_raw vs -o exclude_raw is really more about what gets set as the default. I initially had raw included by default, but got the opposite feedback. Need a good way to poll goss users on topics like this, any suggestions?

Maybe a gitter.im or community slack? I don't have any good suggestions, sorry.

@ripienaar Adding you here since you're pretty active with major goss changes as of late. This is probably the defining feature of v0.4.0, would love your input. The major change is the ability to do transforms and compare "15" <= 15, or use gjson to extract JSON values from a string (this allows for a much more powerful command/http tests).
At this point, I think it's just a discussion about UI and sane defaults, but feel free to review any part of this PR.
For example, using tap output and the example from the comment above https://github.com/aelsabbahy/goss/pull/576#issuecomment-689736096
currently the PR defaults to a "quiet" output
$ goss v --format tap
1..1
not ok 1 - Matching: example: matches: Expected {"count":"15","foo":"bar","moo":{"nested":"cow"}} to have key matching "foox" the transform chain was [{"gjson":{"Path":"@this"}}]

This can be changed by using -o include_raw to make it more verbose and include the "Raw" input:
$ goss v --format tap -o include_raw
1..1
not ok 1 - Matching: example: matches: Expected {"count":"15","foo":"bar","moo":{"nested":"cow"}} to have key matching "foox" the transform chain was [{"gjson":{"Path":"@this"}}] the raw value was "{\"foo\": \"bar\", \"moo\" {\"nested\": \"cow\"}, \"count\": \"15\"}"

@petemounce is proposing removing the include_raw and just having the transform chain include the "input" as part of it.
$ goss v --format tap
1..1
not ok 1 - Matching: example: matches: Expected {"count":"15","foo":"bar","moo":{"nested":"cow"}} to have key matching "foox" the transform chain was [{"gjson":{"Path":"@this","Input":"{\"foo\": \"bar\", \"moo\" {\"nested\": \"cow\"}"}}]

Close enough.. will tackle issues in the v4 branch.

@petemounce This is very close now, I think..
The failure is in the windows test:
...F.F..
Failures/Skipped:
Command: wrap a powershell with quotes - expect 0 because travis does not restrict anonymous logins: stdout:
Error
    HavePatterns matcher expects patterns to be a string. got:
        <int>: 0
Command: wrap a powershell - expect 0 because travis does not restrict anonymous logins: stdout:
Error
    HavePatterns matcher expects patterns to be a string. got:
        <int>: 0
Total Duration: 0.848s
Count: 8, Failed: 2, Skipped: 0

Any objection to this change/fix? https://github.com/aelsabbahy/goss/pull/576/commits/ac0e1ae65a419885ff0fd6aa967c6079ec310442


Folder/file sort grouping outside of sorting by name

What's the Problem?
Show folders first/last only when sorting by name. When sorting by Date/Size, it feels a bit funny to have folders not mix with files.
Solution/Idea
Add a new setting for this behaviour. (Like "List and sort directiories alongside files" but when not sorting by name)
Alternatives
Make this default behaviour. (either when "List and sort directiories alongside files" is on or off)
-or-
Be able to toggle folder grouping for each of the sorting setting, e.g. in the right click menu.
-or-
I noticed this actually when browsing downloads, so maybe just let the folder grouping loose when there.
Priorities

Capability
Priority

This proposal will allow end users to customize if folders are grouped when not sorting by name
Must

This proposal will allow end users to customize when folders are grouped depending on current sorting
Could

This proposal will allow end users to customize when folders are grouped when in download folder
Could

Files Version
<IP_ADDRESS>
Windows Version
Windows 11 21H2 22000.434
Comments
No response

I also have the same demand, hope can change "folders first/last" in any sort mode. THANKS.

This can be fixed by enabling "List and sort directory along side files" in the sorting menu


Add accessibility hint for voiceover support.

nice addition for the default hamburger button that many people may make use of.

Sorry for just getting to this. I'll review and test this week!
Thanks!

I'd actually like to see us consider localization here. I know this is used in projects with lots of languages, so adding in an English only accessibility label may be a regression for some people.


Suggested fix for issue #113

PR Checklist
Please check if your PR fulfills the following requirements:

[ ] Tests for the changes have been added (for bug fixes / features)

PR Type
What kind of change does this PR introduce?

[X] Bugfix
[ ] Feature
[ ] Code style update (formatting, local variables)
[ ] Refactoring (no functional changes, no api changes)
[ ] Build related changes
[ ] CI related changes
[ ] Other... Please describe:

What is the current behavior?
Suggested fix for infinite redirect loop as suggested in https://github.com/single-spa/single-spa-angular/issues/113
Issue Number: 113
What is the new behavior?
Not having an infinite redirect when links are clicked in quick succession. The issue can also be caused by having multiple angular applications mounted simultaneously if both applications have an angular router.
Does this PR introduce a breaking change?
[ ] Yes
[X] No

Other information

I am closing this PR since the implementation approach is flawed. If I'm missing something, let me know and we can reopen.


STM32G473 Crash while starting ADC reads via DMA

Describe the bug
Device crash while starting ADC reads via DMA.
It would seem that the DMA part of the ADC driver struct is never initialised, so dma_config calls an invalid pointer.
I also got a few warnings during compile, not sure if they are related.
-- west build: building application
[1/144] Generating include/generated/zephyr/version.h
-- Zephyr version: 3.7.99 (D:/cantest/zephyr), build: v3.7.0-3516-g57a8a7c3500a
[67/144] Building C object zephyr/drivers/adc/CMakeFiles/drivers__adc.dir/adc_stm32.c.obj
D:/cantest/zephyr/drivers/adc/adc_stm32.c:779:13: warning: 'dma_callback' defined but not used [-Wunused-function]
  779 | static void dma_callback(const struct device *dev, void *user_data,
      |             ^~~~~~~~~~~~
[86/144] Building C object zephyr/drivers/spi/CMakeFiles/drivers__spi.dir/spi_ll_stm32.c.obj
D:/cantest/zephyr/drivers/spi/spi_ll_stm32.c:126:13: warning: 'dma_callback' defined but not used [-Wunused-function]
  126 | static void dma_callback(const struct device *dma_dev, void *arg,
      |             ^~~~~~~~~~~~
[144/144] Linking CXX executable zephyr\zephyr.elf

To Reproduce
Run the ADC sequence sample app
Logs and console output
*** Booting Zephyr OS build v3.7.0-3516-g57a8a7c3500a ***
ADC sequence reading [0]:
[00:00:01.010,000] <err> os: ***** BUS FAULT *****
[00:00:01.015,000] <err> os:   Instruction bus error
[00:00:01.021,000] <err> os: r0/a1:  0x00000000  r1/a2:  0x00000000  r2/a3:  0x200002ec
[00:00:01.029,000] <err> os: r3/a4:  0x6ff7f6b5 r12/ip:  0x20000220 r14/lr:  0x08004b3b
[00:00:01.038,000] <err> os:  xpsr:  0x41000000
[00:00:01.043,000] <err> os: s[ 0]:  0x200035e4  s[ 1]:  0x00000000  s[ 2]:  0x0000000f  s[ 3]:  0x20003688
[00:00:01.054,000] <err> os: s[ 4]:  0x00000000  s[ 5]:  0xffffffff  s[ 6]:  0xffffffff  s[ 7]:  0x00000000
[00:00:01.064,000] <err> os: s[ 8]:  0x00000000  s[ 9]:  0xe000ed00  s[10]:  0x20000d68  s[11]:  0x00000000
[00:00:01.075,000] <err> os: s[12]:  0x0000000a  s[13]:  0x08009631  s[14]:  0x0800431c  s[15]:  0x61000000
[00:00:01.085,000] <err> os: fpscr:  0x00000000
[00:00:01.090,000] <err> os: Faulting instruction address (r15/pc): 0x70000000
[00:00:01.098,000] <err> os: >>> ZEPHYR FATAL ERROR 27: Unknown error on CPU 0
[00:00:01.106,000] <err> os: Current thread: 0x200007f0 (unknown)
[00:00:01.113,000] <err> os: Halting system

Environment (please complete the following information):

OS: Windows 10
Toolchain Zephyr SDK 0.16.5-1-win
v3.7.0-3516-g57a8a7c3500a
STM32G473

hello, @PeeJay could you please share your DTS. The stm32G4 serie has a DMAMUX in front of the DMA. So configuration is done for DMAMUX instance.

/dts-v1/;
#include <st/g4/stm32g473Xe.dtsi>
#include <st/g4/stm32g473v(b-c-e)tx-pinctrl.dtsi>
#include <zephyr/dt-bindings/input/input-event-codes.h>
#include <zephyr/dt-bindings/led/led.h>

/ {
// Analog IOs
&adc1 {
	#address-cells = <1>;
	#size-cells = <0>;

	pinctrl-0 = <&adc1_in15_pb0
				&adc1_in4_pa3
				&adc1_in3_pa2
				&adc1_in1_pa0
				&adc1_in9_pc3
				&adc1_in8_pc2
				&adc1_in7_pc1
				&adc1_in6_pc0
		    >;
	pinctrl-names = "default";
	st,adc-clock-source = <SYNC>;
	st,adc-prescaler = <4>;
	status = "okay";

	channel@f {
		reg = <15>;
		zephyr,gain = "ADC_GAIN_1";
		zephyr,reference = "ADC_REF_INTERNAL";
		zephyr,vref-mv = <2500>;
		zephyr,acquisition-time = <ADC_ACQ_TIME_DEFAULT>;
		zephyr,resolution = <12>;
		zephyr,oversampling = <32>;
	};

... etc ...

&dma1 {
	status = "okay";
};

&dmamux1 {
	status = "okay";
};

&clk_hsi {
	status = "okay";
};

&clk_hsi48 {
	status = "okay";
};

&pll {
	div-m = <4>;
	mul-n = <85>;
	div-p = <2>;
	div-q = <2>;
	div-r = <2>;
	clocks = <&clk_hsi>;
	status = "okay";
};

&rcc {
	clocks = <&pll>;
	clock-frequency = <DT_FREQ_M(170)>;
	ahb-prescaler = <1>;
	apb1-prescaler = <1>;
	apb2-prescaler = <1>;
};

Don't you have a phandle with DMAMUX channel declaration for the ADC , like tests/drivers/adc/adc_api/boards/nucleo_g474re.overlay:
	dmas = <&dmamux1 0 5 (STM32_DMA_PERIPH_RX | STM32_DMA_MEM_16BITS |
		STM32_DMA_PERIPH_16BITS)>;
	dma-names = "dmamux";

Ahh crud. Sorry! I looked everywhere to find documentation on how to setup the DMA correctly, but I never thought to look in there.
Just added it and now it's working. Thanks for the help!


release on pub.dev

I think most parts of the project are ready to use. @KNawm @hsluoyz please take a look and if it seems good enough, we can release it on pub.dev

@MrUnfunny we already have it: https://pub.dev/packages/casbin
Can we setup semantic-release to make release automatically?

Can we setup semantic-release to make release automatically?

I'm not sure about this. I'll look into it.

We can set up semantic-release for releasing on GitHub but right now there's no semantic-release plugin to publish on pub.dev. We can try to stitch flutter-pub-publisher with semantic-release to publish on both GitHub and pub.dev but we'll still need to manually change version number in pubspec.yaml and changelog.md. We can also try to create a plugin for semantic-release to auto-update pubspec and changelog files.

@MrUnfunny can this be used? https://github.com/themizzi/semantic-release-dart

@MrUnfunny can this be used? https://github.com/themizzi/semantic-release-dart

Looking at it's code, it seems to change pubspec only and not changelog. I'll try to tomowrrow.


[BUG] On iOS SKSurface.Snapshot returns null when called from SKGLView

Snapshot() method of SKSurface always returns null on iOS devices (both physical and virtual) when it's called inside SKGLView, but it seems to work when it is used from SKCanvasView. Actually it looks exactly like an old issue which was fixed in 1.6, but it works fine on Android devices this time. (https://github.com/mono/SkiaSharp/issues/436).
protected override void OnPaintSurface(SKPaintGLSurfaceEventArgs e)
{
   var snapshot = e.Surface.Snapshot();
   if(snapshot == null)
   {
   //code on iOS enters here
   }
}

Expected Behavior
Returns valid SKImage
Actual Behavior
Returns null
Basic Information

Version with issue:  2.80.2
Last known good version:  1.6
IDE:  Visual Studio for Mac / Visual Studio
Platform Target Frameworks:

iOS:  12.4

Target Devices:

iPad Air (4th gen)
iPad Pro (4th gen)
iPhone 12

I am facing similar problem with SKCanvasView on iOS, works fine on android.
SkiaSharp version : 2.80.2
iOS : 14.4

Same problem on iOS.


Drop dynamic wrapped client generation

/assign @dprotaso
We have lots of code generation to support using a dynamic client driving a typed kubernetes interface. This was introduced in https://github.com/knative/pkg/pull/2210 It's not clear if this is really being used anywhere.
What we do know is

it's not fully implemented (bunch of panics in the code)
it causes friction when updating k8s deps
it prevents downstream implementations from bumping their k8s version to a higher one
-- https://github.com/knative/pkg/issues/2725

I propose we drop this logic from knative/pkg
The only use of the dynamic injection  I found was https://github.com/tektoncd/triggers/pull/1207

cc @mattmoor in case you have more references to usages

We use this downstream to inject clients that exhibit different behavior.
cc @wlynch who I know spent a bunch of time looking at improving things a few weeks ago to streamline updates.

exhibit different behavior.

In what way?

We use this downstream...

I'm thinking of making the generation optional - so you can just generate these yourself downstream

@dprotaso I think for downstream codegen to work, the clients would need to be in their own go module, so that folks can go mod -replace with additional options.

I don't understand. I'm suggesting downstream add a generation script like so
https://github.com/knative/pkg/blob/db8a35330281c41c7e8e90df6059c23a36af0643/hack/update-codegen.sh#L46-L54
But add a flag --force-dynamic-clients or something to get the extra dynamic stuff

So looking at all the downstream packages that depend on injection.Dynamic I only see registration of dynamic clients and informers (due to codegen/cargo culting with no actual use).
The only downstream dependency that seems to setup the context with a dynamic client is tekton triggers
<EMAIL_ADDRESS>ctx = injection.Dynamic.SetupDynamic(ctx)

But then they setup the Default Informers and clients overriding everything. So injection.Dynamic doesn't really have any usage.
Also based on https://github.com/knative/pkg/pull/2210

One of the long-standing motivations for the injection-based controller architecture has been to enable us to synthesize different backing implementations, which can be made available to controller constructors by substituting what foo.Get(ctx) accesses.

I agree with this statement but I would envision the implementation to just use the same object graph as injection.Default vs. a separate one
ie. I would setup things this way
import "context"
import "k8s.io/client-go/dynamic"
import "knative.dev/pkg/client/injection/kube/client"
import "knative.dev/pkg/injection/clients/dynamicclient"

func main() {
  ctx := context.Background()
  dc := dynamic.NewForConfigOrDie(cfg)

 ctx = context.WithValue(ctx, dynamicclient.Key{}, dynamic.NewForConfigOrDie(cfg))
 ctx = context.WithValue(ctx, client.Key{}, myOwnKubernetesWrapper{dc})
}

type myOwnKubernetesWrapper struct {
   dc *dynamic.Client
}

var _ kubernetes.Interface = (*myOwnKubernetesWrapper)(nil)

Based on all this I'm inclined to delete the wrapped client, the informers, and injection.Dynamic - it should be fairly trivial to just remove registrations downstream (via codegen)

Thanks for the assessment @dprotaso !

Arf, this was used in tektoncd/pipeline 😅


Forum

This is  a bug | feature request.

Prerequisites

[ ] Are you running the latest version?
[ ] Are you reporting to the correct repository?
(enso is made of many specialized packages: https://github.com/familytree365)
[ ] Did you check the documentation?
[ ] Did you perform a cursory search?

Description

Steps to Reproduce

Expected behavior

Actual behavior

Categories creatable by admins or supervisor roles.
Topics created by anyone logged in.
Posts by anyone logged in.
Make use of Enso discussions / comments components.


Photo verification intro screen

Why?
Implement the introductory screen for the Photo Verification feature. This screen includes:

Instructions on taking photos (including recommended angles or views) with images as an example
A Terms of Service checkbox that unlocks the Start Photo Verification button
A Start Photo Verification button to initiate the photo capture process.
Use the same screen when coming from the Gallery screen (change the X button to back arrow and remove the agreement toggle button)

How?

Created PhotoVerificationIntro activity along with its layout and the respective fragment PhotoVerificationInstructionsFragment which will be reused in a next iteration to be opened in a BottomSheetDialog as well.
Navigate to the above screen on claiming results
Created a PhotoExampleAdapter which populates the respective RecyclerView with the PhotoExample UI Model which has an image and the texts below that image per the designs

Testing
Complete a mock claiming flow and at the last screen continue to photo verification to view the screen and ensure everything is OK.

Good catches @pantelisss , fixed!

Looks good!


fqdn/dnsproxy/proxy_test: increase timeout for DNS TCP client exchanges

Under heavy load, the round-trip-time (RTT) for DNS requests between a TCP client and a DNS proxy may exceed the 100ms timeout specified when creating the client in the dnsproxy tests.
This was observed on the test-PR #12298, with a RTT value going up to 296ms (under exceptional memory strain).
This might be the cause for the rare flakes reported in #12042. Let's increase this timeout. The timeout is only used a couple of times in the tests, so increasing it by a few hundred milliseconds would have no visible impact. And because we expect all requests from the TCP client to succeed on the L4 anyway (i.e. it should never time out in our tests), this should not prolong at all the execution of tests in the normal case.
Note that this timeout was at 1 second originally, and was reduced in commit 2b4badbf32b2 ("dnsproxy: Correct tests to account for matchPattern generation. Reduce timeout time").
Let's also retrieve and print the RTT value for that request in case of error, to get more info if this change were not enough to fix the flake.
Hopefully fixes: #12042

test-focus RuntimePrivilegedUnitTests

Coverage decreased (-0.009%) to 36.928% when pulling ef6f7e00e4e4bfff19d1745d4c07857e3746f707 on pr/qmonnet/dnsproxy_test_timeout into db88419c49ba38600946a8f50f769266dc978a84 on master.


feature: support IDOMElementDescriptors

Implement support for RFC #726

This PR does change existing error messages, as can be seen from the tests, which isn't an explicit goal of this change. Here's the explanation.
If an assertion couldn't find its element (whether because of a non-matching selector or because it was passed null), previously the error messaging would be inconsistent between its usage of <not found> vs. <unknown>. The code that produced those messages was refactored and is now driven by the dom element descriptor description functionality, so it would require extra code to restore the inconsistent behavior and keep the error message the same as it was before this PR.

Drafted https://github.com/mainmatter/qunit-dom/pull/2101 as per feedback, would this help dom-element-descriptors to create a "plugin" for qunit-dom?

@BobrImperator this is rebased and, pending your decision on the assertion messages change or any other feedback you might have, ready to merge!

https://github.com/mainmatter/qunit-dom/releases/tag/v3.2.0