DSD-435:  ProgressIndicator component

Fixes JIRA ticket DSD-435
This PR does the following:

Adds the ProgressIndicator component.
Note: there's a better way to do dark mode in Chakra but we don't have it configured. So while the darkMode prop works, we'll have to update it later to use the system "light" and "dark" values.
Note, there are some aria attribute issues that I noted in the spec doc which we can discuss together.

How has this been tested?
Storybook.
Checklist:

[ ] I have updated the Storybook documentation accordingly.
[ ] All new and existing tests passed.

Front End Review:

[ ] View the example in Storybook

Tugboat has finished building the preview for this pull request!
Link:

https://pr764-7a264s7c5xf6oqgdhpxhckspmzooqqsp.tugboat.qa

Dashboard:

https://dashboard.tugboat.qa/618ed4aae368652cd33c47ae

@bigfishdesign13 updated. Using Label does allow to remove some style rules.

Tugboat has finished building the preview for this pull request!
Link:

https://pr764-7a264s7c5xf6oqgdhpxhckspmzooqqsp.tugboat.qa

Dashboard:

https://dashboard.tugboat.qa/618ed4aae368652cd33c47ae


Minnesota - UP Migration - Validate Test Data

User Story:
As a developer, I want the Minnesota to validate test messages in staging, so that they can receive data in the format they expect.
Description/Use Case
[STLT] is already receiving data from the covid pipeline they will be onboarded to the UP where they will be receiving data from SimpleReport including Covid, Flu and RSV. We need to make sure the message we generate from the UP passes the STLTs validation. We can use SimpleReport to submit test messages to the STLT.
Risks/Impacts/Considerations
Notes doc and requirements
https://cdc.sharepoint.com/:w:/r/teams/ReportStream/Shared Documents/Engagement/Onboarding/Minnesota Onboarding Running Notes.docx?d=we9a6f9098aa245b8a6cac96c9df86fa5&csf=1&web=1&e=od5zlI
Dev Notes:

[x] Upload the newly created receiver settings to Staging
[x] Copy the receiver's secret key with the new receiver name in Azure's staging environment
[x] Verify we can connect to the newly created receiver
[x] Send a test message through Simple Report and make sure it gets routed to the STLT
[x] Send test messages according to the STLT needs

Acceptance Criteria

[ ] Created and sent data using Simple Report
[ ] Confirmed with STLT that data passes validation

PR for the transform is still in review and we don't have where to send the data to yet, so this ticket is not actually ready to be worked on.

Got email from MN.  I wonder,, should I response to them or SR should?

Will filter FLU out according to Glenn.

Done


update gpg keys

add @wilzbach
add nightly build bot
update public key signatures (expiration dates and ids)
sign updated keyring

Not sure why this is failing on the server, it's passing locally. Trying a restart.

Done @wilzbach
Not sure about GDC and LDC, let's see if there is interest and who would be the right person to add (@jpf91, @kinke), atm. none of those releases/tags are signed.

Done @wilzbach

Did you forget to push? I don't see @CyberShadow's key in the keyring (nor the updated fingerprint of mine).

Did you forget to push? I don't see @CyberShadow's key in the keyring (nor the updated fingerprint of mine).

Indeed @wilzbach, sorry.


version.sh: ignore feinsum, minikanren in conda env

feinsum has no pypi package. The latest tag of minikanren is not available on pypi, so this package also can't be installed via pip.
Both packages are in the generated requirements.txt, so this should not create issues.

This is ready for review. The remaining CI error is unrelated.


Specify unmatched_normal_id based on wildcard

For gambl we have three different genome builds, and each will need an unmatched normal specified. I've tried implementing this a couple of ways, most recently by including this in header.smk:
GENOME_DEFAULT_SWITCH = op.switch_on_wildcard("genome_build", {
    "grch37": "14-11247N",
    "hg38": "BLGSP-71-06-00286-99A-01D",
    "hs37d5": "SP116656"
})
shared_config["pairing_config"]["genome"]["unmatched_normal_id"] = GENOME_DEFAULT_SWITCH
genome_default = shared_config["pairing_config"]["genome"]["unmatched_normal_id"]
PAIRS = op.generate_pairs(SAMPLES, genome=("allow_unmatched", genome_default), mrna="no_normal")

I get the following error when I run the above with snakemake -np -s src/snakemake/header.smk:
AssertionError in line 71 of /projects/rmorin/projects/gambl-repos/gambl-lhilton/src/snakemake/header.smk:
There are 0 genome samples matching the normal ID <function switch_on_wildcard.<locals>._switch_on_wildcard at 0x7fb56528a440> (instead of just one).
  File "/projects/rmorin/projects/gambl-repos/gambl-lhilton/src/snakemake/header.smk", line 71, in <module>
  File "/home/lhilton/miniconda3/lib/python3.7/site-packages/oncopipe/__init__.py", line 1254, in generate_pairs
  File "/home/lhilton/miniconda3/lib/python3.7/site-packages/oncopipe/__init__.py", line 1072, in generate_runs

oncopipe version 1.0.2
snakemake-minimal version 5.14.0
What can I change to get this to work?

I updated oncopipe to 1.0.3 and the error persists.

Is it the same error if you change it to:
    "grch37": "14-11247N",
    "hg38": "BLGSP-71-06-00286-99A-01D",
    "hs37d5": "SP116656"
})

Your instinct with using a switch here was correct. For any directive loaded by snakemake, it will detect that the value is a function and will call the function, including if that function is a switch.
The issue here is that the unmatched_normal_id field is handled by oncopipe rather than snakemake, and I totally didn't think of the fact that unmatched_normal_id will need to be genome_build-specific. This requires an update to oncopipe, which I'm willing to implement tonight if this can wait until then.
I need to think of the simplest way of addressing this, but one idea is to change the pairing config keys from <seq_type> (e.g. genome) to <seq_type>--<genome_build> (e.g. genome--hg38), just like what's already in module file names. This way, a switch won't be necessary.
While I'm editing this part of oncopipe, I might simplify the pairing config in the project configuration from:
lcr-modules:
    _shared:
        [...]
        pairing_config:
            capture:
                unmatched_normal_id: "TCRBOA7-N-WEX"

To:
lcr-modules:
    _shared:
        [...]
        unmatched_normal_id:
            capture--grch37: "TCRBOA7-N-WEX"

Is this intuitive?

Yep, totally intuitive. Thanks!

To confirm (since I'm close to submitting a PR for gambl) the config would look like this, right:
lcr-modules:
  _shared:
    repository: "src/lcr-modules/"
    lcr-scripts: "src/lcr-scripts/"
    root_output_dir: "results/"
    scratch_directory: "null"
    unmatched_normal_id:
      genome--grch37: "14-11247N"
      genome--hg38: "BLGSP-71-06-00286-99A-01D"
      genome--hs37d5: "SP116656"

You will want to set scratch_directory to null (without quotes). Otherwise, the modules will write to a directory called null/. Other than that, it looks good! Actually, maybe change unmatched_normal_id to unmatched_normal_ids. That's the key name I'll use.

I already check for strings equal to "None" in the configuration to avoid this pitfall. I'll update the code to also warn the user for any string values of "null" as a safeguard.


neovim: withNodeJs regressed

Describe the bug
Before, the withNodeJs attribute cause node to be present on path for Neovim. As of a couple of days, this does not work anymore.
It may be caused by this commit: https://github.com/NixOS/nixpkgs/commit/12dafac23cc3d9998cc5a4ce3485bad1bbba3ae9
Especially this line which is removed:
https://github.com/NixOS/nixpkgs/commit/12dafac23cc3d9998cc5a4ce3485bad1bbba3ae9#diff-7bcde5ff83173c3032fdbc3addc7386ee1b86c5d6ce90bb33b0e54a853d6af70L90
(not sure though)
Notify maintainers
@manveru @rvolosatovs @teto
Metadata
 - system: `"x86_64-linux"`
 - host os: `Linux 6.10.14, NixOS, 24.11 (Vicuna), 24.11.20241014.a3c0b3b`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.24.9`
 - nixpkgs: `/nix/store/xnjw9gmfmpppdj6bxpw6cfkspc3h6xwl-source`

Add a :+1: reaction to issues you find important.

this was not intended but a plugin should use g:node_host_prog anyway so it should not have been breaking. What/how do you use withNodeJs for ?

coc.nvim is not a Node.js plugin itself, but depends on node binary present in PATH. I guess coc.nvim is popular enough that users notice the breakage and think withNodeJs regressed.
coc.nvim will use Node.js binary set in g:coc_node_path or COC_NODE_PATH env variable before searching in PATH (source). I worked around the issue by adding the variable to the wrapper:
[ "--set" "COC_NODE_PATH" (lib.getExe nodejs) ]

I relay on Node present in the path for few plugins.
I actually use neovim with home-manager and let lazy manage the plugins, I know it's not the nix-way but it used to work fine (most of the time :D) until the latest change.
I'll try the WA and hopefully it will be resolved later.

I need it for pyright-lsp

Diffing suggests that the problem is that the node directory is no longer in the path. The node executable itself is working:
--- /nix/store/8vwczaxz6zwrmz0h316p0g70bmzdvrpa-neovim-0.10.2/bin/nvim  1970-01-01 01:00:01
+++ /nix/store/x0zc9p2i8rw9md182dyna25ya6m6n73n-neovim-0.10.2/bin/nvim  1970-01-01 01:00:01
@@ -1,20 +1,5 @@
 #! /nix/store/0w0hhy7rmn5xisasdr4v4p958x114sj1-bash-5.2p32/bin/bash -e
-export NVIM_SYSTEM_RPLUGIN_MANIFEST='/nix/store/8vwczaxz6zwrmz0h316p0g70bmzdvrpa-neovim-0.10.2/rplugin.vim'
-export GEM_HOME='/nix/store/146nn1x9gnn134j6vf7d6fnjywy4w06r-neovim-ruby-env/lib/ruby/gems/3.3.0'
-PATH=${PATH:+':'$PATH':'}
-if [[ $PATH != *':''/nix/store/146nn1x9gnn134j6vf7d6fnjywy4w06r-neovim-ruby-env/bin'':'* ]]; then
-    PATH=$PATH'/nix/store/146nn1x9gnn134j6vf7d6fnjywy4w06r-neovim-ruby-env/bin'
-fi
-PATH=${PATH#':'}
-PATH=${PATH%':'}
-export PATH
-PATH=${PATH:+':'$PATH':'}
-if [[ $PATH != *':''/nix/store/fkapwdcjn9fm1n071jv7a75cw66x4wq7-nodejs-20.17.0/bin'':'* ]]; then
-    PATH=$PATH'/nix/store/fkapwdcjn9fm1n071jv7a75cw66x4wq7-nodejs-20.17.0/bin'
-fi
-PATH=${PATH#':'}
-PATH=${PATH%':'}
-export PATH
+export NVIM_SYSTEM_RPLUGIN_MANIFEST='/nix/store/x0zc9p2i8rw9md182dyna25ya6m6n73n-neovim-0.10.2/rplugin.vim'
 LUA_PATH=${LUA_PATH:+';'$LUA_PATH';'}
 LUA_PATH=${LUA_PATH/';''/nix/store/hqlfv4kzphmjq7ggis8pjjixl1dwpy9g-luajit-2.1.1713773202-env/share/lua/5.1/?/init.lua'';'/';'}
 LUA_PATH='/nix/store/hqlfv4kzphmjq7ggis8pjjixl1dwpy9g-luajit-2.1.1713773202-env/share/lua/5.1/?/init.lua'$LUA_PATH
@@ -33,4 +18,4 @@
 LUA_CPATH=${LUA_CPATH#';'}
 LUA_CPATH=${LUA_CPATH%';'}
 export LUA_CPATH
-exec -a "$0" "/nix/store/aqafkzaksbm4zk0343qi44xxlw6hvywn-neovim-unwrapped-0.10.2/bin/nvim"  --cmd "lua vim.g.node_host_prog='/nix/store/8vwczaxz6zwrmz0h316p0g70bmzdvrpa-neovim-0.10.2/bin/nvim-node';vim.g.loaded_perl_provider=0;vim.g.loaded_python_provider=0;vim.g.python3_
host_prog='/nix/store/8vwczaxz6zwrmz0h316p0g70bmzdvrpa-neovim-0.10.2/bin/nvim-python3';vim.g.ruby_host_prog='/nix/store/8vwczaxz6zwrmz0h316p0g70bmzdvrpa-neovim-0.10.2/bin/nvim-ruby'" "$@"
+exec "/nix/store/aqafkzaksbm4zk0343qi44xxlw6hvywn-neovim-unwrapped-0.10.2/bin/nvim"  --cmd "lua vim.g.node_host_prog='/nix/store/x0zc9p2i8rw9md182dyna25ya6m6n73n-neovim-0.10.2/bin/nvim-node';vim.g.loaded_perl_provider=0;vim.g.loaded_python_provider=0;vim.g.python3_host_pro
g='/nix/store/x0zc9p2i8rw9md182dyna25ya6m6n73n-neovim-0.10.2/bin/nvim-python3';vim.g.ruby_host_prog='/nix/store/x0zc9p2i8rw9md182dyna25ya6m6n73n-neovim-0.10.2/bin/nvim-ruby'" "$@"

I've submitted https://github.com/NixOS/nixpkgs/pull/350345 if anyone can confirm it fixes the issue

Yep, that fixes it.


skypatrol port 5021

Is there any way to change the skypatrol port 5021 udp to tcp if that's how you could do it?

You would have to modify server code.

I am trying to receive information from a skypatrol tt8750 the only way to integrate it was by protocol tcp 5008 enfora but are limited options in this protocol could change the device to binary for skypatrol protocol but I have problems programming it by udp

Thanks for replying, is there any way that uisted could enable me about project tracing how the code works I am beginner in java.
If so, how much money could this training cost me?

Send me an email to discuss it.


Update 'Mailing Address' based on Preferred Address Type

From idea in the Hub: https://powerofus.force.com/hub-ideas/1095
This is a lot more complex than phone an email though, due to the multi-address support.
WARNING: we should do https://github.com/SalesforceFoundation/HEDAP/issues/344 first.

**lurch: add

Tracking W-017028

@davidjray - WI has been closed (comments on on the WI). Please let me know if there is anything else we need to do with this Issue.
@wongmike


openapi: Migrate remaining properties from zulip-2.0.yaml to zulip.yaml.

zulip-2.0.yaml had only 4 endpoints. Compare them with their endpoints in zulip.yaml and change zulip.yaml accordingly.

@orientor thanks for investigating this!  A few requests:

Can you delete zulip-2.0.yaml and any references to it in this commit?
Can you comment on each of the changes with investigation into whether they are improvements or not?  I think in some cases the zulip-2.0.yaml code may be a bug.

@timabott changed and according to reviews and comments. Kindly check once again. Also is chat.zulip.org down? I am unable to access it.

@timabbott There was also security tag in zulip-2.0.yaml which describes authentication. This however is not present in zulip.yaml. I think adding this tag and a few others mentioned in the commit would make it more descriptive and accessible for both developers and users.

Great, thanks @orientor!  I did some edits to clean up the comments and am merging this as 6eb39c86682b9a5cdee2c816cf5f41fe4b2db898.

I think we should look at making more aggressive use of macros in the OpenAPI files.  E.g. we have a million copies of this:
        '200':
          description: Success.
          content:
            application/json:
              schema:
                allOf:
                - $ref: '#/components/schemas/JsonSuccess'
                - example:
                      {
                          "msg": "",
                          "result": "success"
                      }

If the endpoint doesn't return anything other than its success status, perhaps we should make that whole thing a reusable schema?
There's several other things we should be doing with this framework as well.


An issue with an importing .xsd.

Hello!
Thanks for the excellent work with this package!
I am working on making a pre-system for a new Swedish tax form implementation, in python. I needed a schema-validater and stumbled upon your work. The problem seems to arise from the fact that there is an instance .xsd-file and a component .xsd-file. The instance file imports the component file.
The problem I am running into is an element is fully qualified in the instance file as agd:avsandare, but is referenced as avsandare in the component file. The definition of the prefix "agd" is given in the instance file so I am not sure where the problem lies.
I am attaching the files I have used to cause the situation. They are all public files so there is no sensitive information being sent.
Thanks again for everything!
Brian Kuehn
Handen, Sweden
Files for Swedish Tax Form.zip

Hello again!
I've re-done the tests and now they are working. It was late last night. I'll have to attest to the fact that I was doing something wrong.
Sorry for the bother...
Brian

Don't worry (I saw, a wrong filename in a namespace import ...) this is another public schema and instance that I keep for my local collection of additional test cases.
Bye
Davide

Good that this will be of some help then!!
Brian
Skickat från min iPhone

28 maj 2019 kl. 10:28 skrev Davide Brunato<EMAIL_ADDRESS>Don't worry (I saw, a wrong filename in a namespace import ...) this is another public schema and instance that I keep for my local collection of additional test cases.
Bye
Davide
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub, or mute the thread.


Docker script returned exit code 1 after multiple tests run and Allure Reporter is Closed

Dear All
I am anticipating an answer to say "this is not a DevExpress problem"
but I would like to use Allure Reporting in our Jenkins builds and we have issues with random crashing.
I'm afraid I cannot reproduce this for you easily.
And I cannot find any existing or past Allure issues about this.
My issue is that docker exits ungracefully, when I use the allure reporter in docker containers on Jenkins.
This happens most of the time as soon as the Allure report is closed.
I tried various memory allocations and concurrent vs sequential runs but it doesnt work.
I have tried to reproduce this locally on my Mac but my docker run commands always pass successfully with the allure reporter.
And sometimes they pass on Jenkins slave too.
Please note the testcafe-reporter-allure plug in has not been maintained for a year.
And I have no way of contacting the author of the plugin.
In the meantime we have stopped using Allure.
I could not see any existing issues for this problem, is there anyone that can help me to shed some light?
Do you know how to contact the testcafe-reporter-allure contributor called azohra-core?
What is your Test Scenario?
Running TestCafe multiple tests in a docker container with Allure Reporter - in Jenkins Pipeline
What is the Current behavior?
Most of the time the jenkins step will fail because of an error code in Docker container
"script returned exit code 1" immediately after Allure Reporter Closed
What is the Expected behavior?
To successfully run my tests in a docker container using --reporter allure without crashing
Steps to Reproduce:
**1/  Docker file contains this sort of thing **
RUN npm install -g testcafe
RUN npm install
COPY . /test-suite
ENTRYPOINT ["testcafe"]
CMD ["--version"]
2/ Jenkins Pipeline
Build Image with this command
sh 'docker build -t test_cafe .'
Example of a Run Docker with this command
sh 'docker run --name="accounts" -t --memory=4g test_cafe "chromium:headless --no-sandbox" tests/Accounts/*.js -c 2 -u --skip-js-errors --env=test --locale=AU --reporter allure'
3/ Allure Plugin / Pipeline Stage -  allure will generate Reports
4/ Report is produced and attached to the build
Your Environment details:

testcafe version:                  "testcafe": "^1.5.0"
node.js version:                   v 12.3.1
allure version :  "testcafe-reporter-allure": "^1.0.5"
browser name and version: chromium headless
platform and version:          Linux Slave
other:          screenshot of the offending crash

Hello @mishmaccas ,
Please provide us with docker build logs. They might help us understand what happened during test execution.

Sorry for delayed response.
I wanted to send the logs but I cannot reproduce this anymore.
I changed my docker repository to alpine:latest ...
Scripts are now executing nicely using allure reporting in docker containers.
Thank you for your responses. I am closing this issue.


Have you tried?

wo̊̊̊̊̊̊̊̊̊̊̊̊̊̊̊̊̊̊̊̊rked for me҉
but first needed to set:

and:

no you need to set   to

@Purpzie, that's only for arm64.  It's definitely   and   for x86_64, as mentioned here https://nocode.readthedocs.io/

Wow, that makes sense now.
This workaround fixed it for me.

Thanks guys, I'm closing this issue so I can reopen it soon.


Allow array join to passthrough strings

I propose to modify the Sightly spec so that the array join works on simple strings by passing them through: ${'test' @ join=', '} will output test
Right now we must wrap the string into an ugly single-element array: ${['test'] @ join=', '}

Fixed in f2afcf2.


Error in migration engine. Reason: entered unreachable code

Command: prisma db push
Version: 4.6.1
Binary Version: 694eea289a8462c80264df36757e4fdc129b1b32
Report: https://prisma-errors.netlify.app/report/14435
OS: x64 win32 10.0.22000
JS Stacktrace:
Error: Error in migration engine.
Reason: [migration-engine\connectors\sql-migration-connector\src\sql_renderer\mysql_renderer.rs:521:97] internal error: entered unreachable code

Rust Stacktrace:
Starting migration engine RPC server
The migration was not applied because it triggered warnings and the force flag was not passed.
[migration-engine\connectors\sql-migration-connector\src\sql_renderer\mysql_renderer.rs:521:97] internal error: entered unreachable code

relationMode: "prisma"
Latest main
Push the data model in the error two times
unreachable code

Simplified data model that makes this bug to happen:
model Class {
  id    Int      @id
  when  DateTime @default(dbgenerated()) @db.Time
}

Push this two times, the second one panics.


Simplify with use of fragments

The whole code can be widely simplified by use of Ecto fragments. No need for custom SQL or the Repo.execute_or_load function.
Fragments allow us to insert custom SQL parts into otherwise regular Ecto queries. We can use the levenshtein function using:
    from q in query,
      where: fragment("levenshtein(?, ?)", q.name, ^query_string)) < 5,
      order_by: fragment("levenshtein(?, ?)", q.name, ^query_string),
      limit: 10

But we can do even better! Since query syntax is based on macros, we can extend it with macros! Let's define a levenshtein function for the query syntax:
  defmacrop levenshtein(lhs, rhs) do
    quote do
      fragment("levenshtein(?, ?)", unquote(lhs), unquote(rhs))
    end
  end

We can then use it in our query:
  def fuzzy_name_search(query \\ Todo, query_string) do
    from q in query,
      where: levenshtein(q.name, ^query_string) < 5,
      order_by: levenshtein(q.name, ^query_string)
  end

We can then use it like this:
Todo.fuzzy_name_search("custom") |> Repo.all

# We also made our query composable
import Ecto.Query
Todo.fuzzy_name_search("custom") |> limit(10) |> Repo.all

def long_names(query \\ Todo) do
  from q in query, where: fragment("len(?)", t.name) > 5
end

Todo.long_names |> Todo.fuzzy_name_search("custom") |> limit(10) |> Repo.all

@michalmuskala Thanks a ton!
I'm gonna fire this up quickly and then update the README in a separate commit to explain the subtleties of this then merge everything up!
I wish I would have read far enough in properly in Ecto to get this up and running. Its awesome to see a more elegant solution. When originally implementing this @ work we actually had a bit of discussion about the approach but didn't dedicate significant time to it since it was working and not creating any bottlenecks or performance worries.

Yeah, README needs some updates.
The performance of both solutions should be very similar, generated SQL is nearly exactly the same as the one you were using before.

Can this be merged? This makes the example much better, more standard and educational.
Also the "create extension" part can be a migration execute, can't it?

@princemaple I'm going to update it to be using Ecto 2.x, this is still in the 1.x API, so I dont want to merge it but I'm going to base my 2.0 rewrite off of it.

Cool!
On Sun, 4 Sep 2016, 03:52 yburyug<EMAIL_ADDRESS>wrote:

@princemaple https://github.com/princemaple I'm going to update it to
be using Ecto 2.x, this is still in the 1.x API, so I dont want to merge it
but I'm going to base my 2.0 rewrite off of it.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/cometaworks/fast_fuzzy_search_in_phoenix_and_ecto/pull/4#issuecomment-244560468,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABRKNHGWMzYhreFCGUnjIHUT2dgdSF3Kks5qmbPegaJpZM4HS4mV
.


Godot 4 Beta 15 error - Invalid type in function 'open_resource'

Describe the bug
The new version of Godot, Version 4.0 beta 15, causes a bug to appear in plugin.gd
res://addons/dialogue_manager/plugin.gd:78 - Invalid type in function 'open_resource' in base 'Control ()'. The Object-derived class of argument 1 (null instance) is not a subclass of the expected argument class.

I can't seem to reproduce that error. When does it show up for you?

When I attempted to enable the v2.8.5 build in beta 15 I got the following

Opening plugin.gd results in the following

I can't seem to reproduce that error. When does it show up for you?

I cant reproduce it too. But it happens, when i installed the plugin for the first time and created first dialog file.

I just made a fresh project, installed the addon, and created/run dialogue without seeing the error.
I'll close this until it can be reproduced.

I just managed to replicate it.


net/http: support "gzip" as a Transfer Encoding

What version of Go are you using (go version)?
go version go1.10.3 windows/amd64
Does this issue reproduce with the latest release?
Yes.
What operating system and processor architecture are you using (go env)?
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\xxxx\AppData\Local\go-build
set GOEXE=.exe
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOOS=windows
set GOPATH=C:\Users\xxxx\go
set GORACE=
set GOROOT=C:\Go
set GOTMPDIR=
set GOTOOLDIR=C:\Go\pkg\tool\windows_amd64
set GCCGO=gccgo
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=C:\Users\xxxx\AppData\Local\Temp\go-build557436246=/tmp/go-build -gno-record-gcc-switches
What did you do?
I am requesting a URL with code below:
package main

import (
	"fmt"
	"net/http"
	"time"
)

func main() {

	urlReq2 := "https://www.azulseguros.com.br/azul-cms/wp-content/themes/azul2016/integracao-azul-leve.php?timestamp=NGJYHX7MXNS"
	req, err := http.NewRequest("GET", urlReq2, nil)
	netClient := &http.Client{
		Timeout: time.Second * 10,
	}
	_, err = netClient.Do(req)
	if err != nil {
		fmt.Println(err)
	}

}

I got this error:
net/http: HTTP/1.x transport connection broken: unsupported transfer encoding "gzip"
What did you expect to see?
I expect to be foward to other page and get HTML.
In other languages like nodejs and c#, the same request is OK.
Running cURL also bring me the result.
What did you see instead?
Error on netClient.Do():
net/http: HTTP/1.x transport connection broken: unsupported transfer encoding "gzip"

Thank you for filing this issue @rv-dtomaz and welcome to the Go project!
So I believe we only support "identity" and "chunked" transfer encodings. However, I've retitled this
as a feature request and unfortunately we are in the Go1.12 so this will have to wait until Go1.13.
@bradfitz if I may indulge you, perhaps we should document that we only support "identity" and "chunked" transfer encodings, for this cycle? In the next cycle we can do a net/http hackathon where we examine all supported transfer encodings and perhaps add them.

Hi @odeke-em ,thanks a lot for your quick suport.
I suspected that this was the problem.
Thanks a lot.
If I can help in solution, I am here.
best

@odeke-em Is there any workaround about this ? Something that we could do temporary.
I would really apreciate.
best

@odeke-em Is there any workaround about this ? Something that we could do temporary.
I would really apreciate.

Great question @rv-dtomaz! My apologies for the late reply, I just saw this on waking up early.
So currently the work around that I can think of is a bit of a hackasauraus but it'll produce the identical response to the last response from curl -i -L $URL
It basically involves trying the request, if it spits out the error you've encountered, then:
a) Make a connection the server
b) Speak HTTP/1.1 to it
c) If it returns "Transfer-Encoding":"gzip", turn that into "Content-Encoding":"gzip"
d) Rewire the entire request's body and invoke http.ReadResponse
and here it is https://gist.github.com/odeke-em/39c8ecb7522edf80f1d91df71e415340 or inlined below
package main

import (
	"bufio"
	"bytes"
	"crypto/tls"
	"fmt"
	"io"
	"log"
	"net"
	"net/http"
	"net/http/httputil"
	"net/textproto"
	"strings"
	"time"
)

func main() {
	uri := "https://www.azulseguros.com.br/azul-cms/wp-content/themes/azul2016/integracao-azul-leve.php?timestamp=NGJYHX7MXNS"
	req, err := http.NewRequest("GET", uri, nil)
	netClient := &http.Client{
		Timeout:   time.Second * 10,
		Transport: &transferEncodingRedactor{base: http.DefaultTransport.(*http.Transport)},
	}
	res, err := netClient.Do(req)
	if err != nil {
		log.Fatalf("Failed to make request: %v", err)
	}
	blob, _ := httputil.DumpResponse(res, true)
	fmt.Printf("%s\n", blob)
	_ = res.Body.Close()
}

type transferEncodingRedactor struct {
	base *http.Transport
}

func (rt *transferEncodingRedactor) RoundTrip(req *http.Request) (*http.Response, error) {
	// Optimistic route i.e common case.
	res, err := rt.base.RoundTrip(req)
	if err == nil || res != nil {
		return res, err
	}

	// An error occured here but now time ot examine the contents
	if !strings.Contains(err.Error(), "unsupported transfer encoding") {
		return res, err
	}

	u := req.URL
	var conn net.Conn
	switch req.URL.Scheme {
	case "https":
		tr := rt.base
		if tr == nil {
			tr = http.DefaultTransport.(*http.Transport)
		}
		conn, err = tls.Dial("tcp", "www.azulseguros.com.br:443", tr.TLSClientConfig)
	default:
		conn, err = net.Dial("tcp", "www.azulseguros.com.br:80")
	}

	if err != nil {
		return nil, fmt.Errorf("Dial error: %v", err)
	}

	// Ensure we close the connection after.
	// Perhaps you might want to reuse it per host?
	defer conn.Close()
	br := bufio.NewReader(conn)
	bw := bufio.NewWriter(conn)

	s := u.Path
	if q := u.RawQuery; len(q) > 0 {
		s += "?" + q
	}

	intro := fmt.Sprintf("%s %s HTTP/1.1\r\nHost: %s\r\n\r\n", req.Method, s, u.Host)
	fmt.Fprint(bw, intro)
	bw.Flush()

	// And now we've got the case of an unsupported transfer
	// encoding time for a raw fetch and header rewrite.
	tp := textproto.NewReader(br)

	protoLine, err := tp.ReadLine()
	if err != nil {
		return nil, fmt.Errorf("ReadLine error: %v", err)
	}
	// Time to parse the headers
	rhdr, err := tp.ReadMIMEHeader()
	if err != nil {
		return nil, fmt.Errorf("ReadMIMEHeader: %v", err)
	}
	hdr := http.Header(rhdr)
	// Now rewrite the header
	if te := hdr.Get("Transfer-Encoding"); te == "gzip" {
		hdr.Set("Content-Encoding", "gzip")
		delete(hdr, "Transfer-Encoding")
	}

	// After this rewrite, reassemble the already read parts back
	// and then get them to ReadResponse
	hdrBuf := new(bytes.Buffer)
	if err := hdr.Write(hdrBuf); err != nil {
		return nil, fmt.Errorf("Rewriting header to wire: %v", err)
	}

	reconstructed := io.MultiReader(strings.NewReader(protoLine+"\r\n"), hdrBuf, strings.NewReader("\r\n"), br)
	return http.ReadResponse(bufio.NewReader(reconstructed), req)
}

Response
HTTP/1.1 200 OK
Content-Type: text/html;charset=UTF-8
Date: Mon, 10 Dec 2018 16:52:57 GMT
Server: WildFly/10
Set-Cookie: JSESSIONID=4pq22naRmHI-EuX1FrYq5PUml_z1WOD4LPsdpp3K.sentraprdlb6:site-emissao-inst2; path=/AzulLeve
Set-Cookie: ASAzul-Zafira_SSL=FDABABAK; Expires=Mon, 10-Dec-2018 18:52:57 GMT; Path=/
Vary: Accept-Encoding
X-Powered-By: Undertow/1

<!DOCTYPE html>
<link type="text/css" rel="stylesheet" href="/AzulLeve/javax.faces.resource/theme.css.jsf?ln=primefaces-redmond" /><link type="text/css" rel="stylesheet" href="/AzulLeve/javax.faces.resource/primefaces.css.jsf;jsessionid=4pq22naRmHI-EuX1FrYq5PUml_z1WOD4LPsdpp3K.sentraprdlb6:site-emissao-inst2?ln=primefaces&v=5.3&v=5.3&v=5.3" />

	<title>
		AZUL AUTO LEVE / POPULAR
	</title>
	<meta http-equiv="expires" content="0" />
	<meta http-equiv="pragma" content="no-cache" />
	<meta http-equiv="cache-control" content="no-cache" />
	<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
	<meta http-equiv="X-UA-Compatible" content="IE=9" />
	<meta http-equiv="X-UA-Compatible" content="IE=8" />
	<meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1" />
	<link rel="stylesheet" href="/AzulLeve/resources/css/application.css" type="text/css" />

AZUL AUTO LEVE

Sessão expirada.

@bradfitz might cringe at me :)

Hello,  @odeke-em
will this be supported soon? I am encountering the error

In the next cycle we can do a net/http hackathon where we examine all supported transfer encodings and perhaps add them.

Thanks!

@monis0395 thanks for the ping! Did you see my suggested workaround in https://github.com/golang/go/issues/29162#issuecomment-445888885?
For sure, I'll prioritize this and roll out a CL to fix it in the next 4 days.

Thank for the quick reply, sorry could respond earlier
yes I tried but it didn't work, just the error message got shorter 😛
from net/http: HTTP/1.x transport connection broken: unsupported transfer encoding "gzip"
to unsupported transfer encoding "gzip"
it's great to hear that it would be fix so soon 🙌🙌🙌

Hi @odeke-em
I tried debugging it more and found that  at line 86 the values are
protoLine: HTTP/1.1 500 Internal Server Error
err: nil

The endpoint which I am hitting is a http2 and content-type is text/event-stream is it because of that?
I dont if this helps but when I use fasthttp for that endpoint it works but I wanna avoid using fasthttp
Thanks

Hi @odeke-em
I tried your code from https://golang.org/cl/166517, I copied tranfer.go to my local and checked my endpoint
Now I am getting 200 OK status 👍 but the contents are empty when I read them from response
any clue?
Thanks

Hi @odeke-em
Yesterday I spent more time on this issue. I found the issue why the work around wasn't working from the suggested in #29162 (comment).
Instead of this
	intro := fmt.Sprintf("%s %s HTTP/1.1\r\nHost: %s\r\n\r\n", req.Method, s, u.Host)
	fmt.Fprint(bw, intro)
	bw.Flush()

	// And now we've got the case of an unsupported transfer
	// encoding time for a raw fetch and header rewrite.
        tp := textproto.NewReader(br)

       protoLine, err := tp.ReadLine()

I changed it to this
        intro := fmt.Sprintf("%s /%s HTTP/1.1\r\nHost:%s\r\n\r\n", req.Method, s, u.Host)
	fmt.Fprint(conn, intro)

	br := bufio.NewReader(conn)
	// And now we've got the case of an unsupported transfer
	// encoding time for a raw fetch and header rewrite.
        tp := textproto.NewReader(br)

       protoLine, err := tp.ReadLine()

Here is the full gist https://gist.github.com/monis0395/0202b2129e943989e83a828553077cde
But it still did not work when I am hitting to my server.  I get protoLine = HTTP/1.1 500. I think some problem with the way I am connect to the server. I am completely new with textproto, can you please suggest where can i read more about it?
Also I tried the change u made in transfer.go it is working now. This time I added each of your change manually, last time I had download the file transfer.go. I don't know why it didn't work last time 😕
Good news is that It works now 😁
I want to use your change in my production code, so is it possible to use it without changing in the go/src/net/http/tranfer.go? Any bright ideas?  Because I want to avoid changing the golang source code.
Thanks!

Sorry for the late reply @monis0395, I was quite swamped in the past quarters but I've made some small updates on CL https://go-review.googlesource.com/c/go/+/166517 and I shall kindly ping @bradfitz to take a look so that perhaps we can include it for Go1.14.

I want to use your change in my production code, so is it possible to use it without changing in the go/src/net/http/tranfer.go? Any bright ideas? Because I want to avoid changing the golang source code.

Unfortunately you'd need to deploy a custom fork otherwise we'd have to write a custom server which gets tricky ;)

@monis0395 if you get gotip or what will become Go1.14 (whose release candidates will be made soon) you'll have this in production. Cheers and thank you for the feature request!

Transfer-Encoding is theoretically interesting in the context of uploads, i. e. to have a smaller representation on the wire, but can be used to transmit zip bombs to exploit unbounded buffers.

The Go 1.23 release notes contain:

On-the-fly compression should use the Transfer-Encoding header instead of Content-Encoding.

But right now the Go stdlib doesn't support Transfer-Encoding. What's the status on this feature request? Is there any chance this gets picked up in the near future? If not, then I'd suggest not recommending in the release notes to use it.

CC @neild


Newer firefox version needs geckodriver

See: https://github.com/SeleniumHQ/selenium/issues/2559
https://github.com/mozilla/geckodriver
It's basically the same thing as chromedriver.

Fixed in 005c3d2d3900b3748824f79a784540998fcf411a


HTML-Export: Support direct links to pages (#page-1)

Is your feature request related to a problem? Please describe.
When I have exported my prototype as a HTML and host it for my colleagues to see, I want to send them direct links to individual pages and not only to the main-page.
Describe the solution you'd like
I would like to have fragment identifies (e.g. #page-1) at the end of the URL, so I can directly link to pages.

Thank you for your request. We will add it to our backlog!


Docs: ForwardRef convention

Details
This PR implements changes to forwardRef documentation, introduced in the proposal linked below.
What's missing:
We are still in the middle of discussion on how to support TS generics with combination of forwarded refs. Two main possibilties are discussed in the proposal. A brief summary:

Assert generic types at the component declaration level

import React from 'react';
import DraggableFlatList from 'react-native-draggable-flatlist';
import {FlatList} from 'react-native-gesture-handler';
import type {DraggableListProps, DraggableListType} from './types';

function DraggableList<T>({renderClone, shouldUsePortal, ...viewProps}: DraggableListProps<T>, ref: React.ForwardedRef<FlatList<T>>) {
    return (
        <DraggableFlatList
            ref={ref}
            // eslint-disable-next-line react/jsx-props-no-spreading
            {...viewProps}
        />
    );
}

DraggableList.displayName = 'DraggableList';

// We have to assert the type here because we use generic forwrded ref
// https://fettblog.eu/typescript-react-generic-forward-refs/#option-1%3A-type-assertion
export default React.forwardRef(DraggableList) as DraggableListType;

Patch React's internal forwardRef typings

declare module "react" {
  function forwardRef<T, P = {}>(
    render: (props: P, ref: React.Ref<T>) => React.ReactElement | null
  ): (props: P & React.RefAttributes<T>) => React.ReactElement | null;
}

Feel free to join the discussion in the slack channel or here in comments below :)
Fixed Issues

$ : N/A
PROPOSAL: https://expensify.slack.com/archives/C01GTK53T8Q/p1695110739750909
Tests

N/A
Offline tests

N/A
QA Steps

N/A
PR Author Checklist

[x] I linked the correct issue in the ### Fixed Issues section above
[x] I wrote clear testing steps that cover the changes made in this PR

[x] I added steps for local testing in the Tests section
[x] I added steps for the expected offline behavior in the Offline steps section
[x] I added steps for Staging and/or Production testing in the QA steps section
[x] I added steps to cover failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[x] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)
[x] I tested this PR with a High Traffic account against the staging or production API to ensure there are no regressions (e.g. long loading states that impact usability).

[x] I included screenshots or videos for tests on all platforms
[x] I ran the tests on all platforms & verified they passed on:

[x] Android / native
[x] Android / Chrome
[x] iOS / native
[x] iOS / Safari
[x] MacOS / Chrome / Safari
[x] MacOS / Desktop

[x] I verified there are no console errors (if there's a console error not related to the PR, report it or open an issue for it to be fixed)
[x] I followed proper code patterns (see Reviewing the code)

[x] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick)
[x] I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
[x] I verified that comments were added to code that is not self explanatory
[x] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[x] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method

[x] If any non-english text was added/modified, I verified the translation was requested/reviewed in #expensify-open-source and it was approved by an internal Expensify engineer. Link to Slack message:

[x] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[x] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is approved by marketing by adding the Waiting for Copy label for a copy review on the original GH to get the correct copy.
[x] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[x] I verified the JSDocs style guidelines (in STYLE.md) were followed

[x] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[x] I followed the guidelines as stated in the Review Guidelines
[x] I tested other components that can be impacted by my changes (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar are working as expected)
[x] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[x] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[x] I verified that if a function's arguments changed that all usages have also been updated correctly
[x] If a new component is created I verified that:

[x] A similar component doesn't exist in the codebase
[x] All props are defined accurately and each prop has a /** comment above it */
[x] The file is named correctly
[x] The component has a clear name that is non-ambiguous and the purpose of the component can be inferred from the name alone
[x] The only data being stored in the state is data necessary for rendering and nothing else
[x] If we are not using the full Onyx data that we loaded, I've added the proper selector in order to ensure the component only re-renders when the data it is using changes
[x] For Class Components, any internal methods passed to components event handlers are bound to this properly so there are no scoping issues (i.e. for onClick={this.submit} the method this.submit should be bound to this in the constructor)
[x] Any internal methods bound to this are necessary to be bound (i.e. avoid this.submit = this.submit.bind(this); if this.submit is never passed to a component event handler like onClick)
[x] All JSX used for rendering exists in the render method
[x] The component has the minimum amount of code necessary for its purpose, and it is broken down into smaller components in order to separate concerns and functions

[x] If any new file was added I verified that:

[x] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[x] If a new CSS style is added I verified that:

[x] A similar style doesn't already exist
[x] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(themeColors.componentBG))

[x] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[x] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[x] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[x] If the PR modifies a component or page that can be accessed by a direct deeplink, I verified that the code functions as expected when the deeplink is used - from a logged in and logged out account.
[x] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[x] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.
[x] I have checked off every checkbox in the PR author checklist, including those that don't apply to this PR.

Screenshots/Videos

Web

Mobile Web - Chrome

Mobile Web - Safari

Desktop

iOS

Android

@blazejkustra as we spoke about it, please let me know if you have any remarks and further improvements when it comes to TS approach. Thanks! :)

Reviewer Checklist

[ ] I have verified the author checklist is complete (all boxes are checked off).
[ ] I verified the correct issue is linked in the ### Fixed Issues section above
[ ] I verified testing steps are clear and they cover the changes made in this PR

[ ] I verified the steps for local testing are in the Tests section
[ ] I verified the steps for Staging and/or Production testing are in the QA steps section
[ ] I verified the steps cover any possible failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[ ] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)

[ ] I checked that screenshots or videos are included for tests on all platforms
[ ] I included screenshots or videos for tests on all platforms
[ ] I verified tests pass on all platforms & I tested again on:

[ ] Android / native
[ ] Android / Chrome
[ ] iOS / native
[ ] iOS / Safari
[ ] MacOS / Chrome / Safari
[ ] MacOS / Desktop

[ ] If there are any errors in the console that are unrelated to this PR, I either fixed them (preferred) or linked to where I reported them in Slack
[ ] I verified proper code patterns were followed (see Reviewing the code)

[ ] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick).
[ ] I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
[ ] I verified that comments were added to code that is not self explanatory
[ ] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[ ] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method
[ ] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[ ] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is approved by marketing by adding the Waiting for Copy label for a copy review on the original GH to get the correct copy.
[ ] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[ ] I verified the JSDocs style guidelines (in STYLE.md) were followed

[ ] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[ ] I verified that this PR follows the guidelines as stated in the Review Guidelines
[ ] I verified other components that can be impacted by these changes have been tested, and I retested again (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar have been tested & I retested again)
[ ] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[ ] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[ ] If a new component is created I verified that:

[ ] A similar component doesn't exist in the codebase
[ ] All props are defined accurately and each prop has a /** comment above it */
[ ] The file is named correctly
[ ] The component has a clear name that is non-ambiguous and the purpose of the component can be inferred from the name alone
[ ] The only data being stored in the state is data necessary for rendering and nothing else
[ ] For Class Components, any internal methods passed to components event handlers are bound to this properly so there are no scoping issues (i.e. for onClick={this.submit} the method this.submit should be bound to this in the constructor)
[ ] Any internal methods bound to this are necessary to be bound (i.e. avoid this.submit = this.submit.bind(this); if this.submit is never passed to a component event handler like onClick)
[ ] All JSX used for rendering exists in the render method
[ ] The component has the minimum amount of code necessary for its purpose, and it is broken down into smaller components in order to separate concerns and functions

[ ] If any new file was added I verified that:

[ ] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[ ] If a new CSS style is added I verified that:

[ ] A similar style doesn't already exist
[ ] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(themeColors.componentBG)

[ ] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[ ] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[ ] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[ ] If the PR modifies a component or page that can be accessed by a direct deeplink, I verified that the code functions as expected when the deeplink is used - from a logged in and logged out account.
[ ] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[ ] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.
[ ] I have checked off every checkbox in the PR reviewer checklist, including those that don't apply to this PR.

Screenshots/Videos

Web

Mobile Web - Chrome

Mobile Web - Safari

Desktop

iOS

Android

:hand: This PR was not deployed to staging yet because QA is ongoing. It will be automatically deployed to staging after the next production release.

@kacper-mikolajczak Looks good to me! Thanks for this improvement!

🚀 Deployed to staging by https://github.com/luacmartins in version: 1.3.73-0 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

🚀 Deployed to production by https://github.com/mountiny in version: 1.3.73-1 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
failure ❌

🕸 web 🕸
success ✅

🚀 Deployed to staging by https://github.com/luacmartins in version: 1.3.74-0 🚀

platform
result

🤖 android 🤖
failure ❌

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

🚀 Deployed to production by https://github.com/chiragsalian in version: 1.3.74-3 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
failure ❌

🕸 web 🕸
success ✅


Sanitize error details

cc @javierarce

:+1: :boom:

Frontend tests were OK :+1: (details)

thx! :shipit:

Frontend tests were OK :+1: (details)


Subreddit Streams

Is there functionality in this project for subreddit streams in a similar fashion to what PRAW can achieve?
http://praw.readthedocs.io/en/latest/code_overview/models/subreddit.html
for comment in reddit.subreddit('iama').stream.comments():
print(comment)

Not exactly. We can get close to this behavior using Paginators.
For v0.9.0:
Paginator<Comment> stream = new CommentStream(redditClient, "<subreddit>");
stream.setSorting(Sorting.NEW);
stream.setLimit(Paginator.RECOMMENDED_MAX_LIMIT); // 100 items per page

for (Listing<Comment> page : stream) {
    // <do something with page>
    // Reset the Paginator to fetch the first page
    stream.reset();
}

For v1.0.0 (currently unreleased)
BarebonesPaginator<Comment> stream = redditClient.latestComments("<subreddit>")
    .limit(Paginator.RECOMMENDED_MAX_LIMIT)
    .build();

for (Listing<Comment> page : stream) {
    // <do something with page>
    // Reset the Paginator to fetch the first page
    stream.restart();
}

This isn't perfect -- if more comments are created faster than you can request them you're going to miss out on some data. If the opposite is true, you'll get some duplicate comments.

No worries, thanks!


State Resolution: Reloaded

Documentation: https://github.com/matrix-org/matrix-doc/pull/1441
PR: #1610
Discussion should happen on the PR, not here - this issue exists just for tracking the MSC metadata.

Done :tada:


Patch Custom Logo Badges

Add "Custom Logo" Section:

add how to create badges with custom logo using Base64 images and Shields.io
add examples of badges with custom logo
update README.md
update README.rst

Fix:

fixed broken link line in README.rst

Hi @GabriOliv,
It looks good.

[ ] Could you check on your branch to see that the <details>...</details> markup with Markdown inside works fine on the GitHub (automatic) preview for the new README.md file please? I have tried it elsewhere and was sometimes disappointed to see that inside it only pure HTML was allowed. Please include a screenshot if you can, here by replying with a comment.

[ ] Also maybe there are too many examples and it will take too much vertical space (horizontal space is fine). Can you check that the markdown code is well displayed in one line, and not many lines?

Thanks

Task 1:

Task 2:

Great! Thanks!


libhdhomerun: add comment about udp firewall rule suggestion

Motivation for this change
libhdhomerun requires UDP port 65001 to be open in order to detect and communicate with tuners. I've therefore included a comment about a firewall suggestion.
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[X] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[X] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[X] Fits CONTRIBUTING.md.

@rycee
I've only included a comment. Will you rebase this?

Looks good, rebased to master in e768fce754dab3e95b47cdcd27ecabd2e2d126a0.


Graph view not working for transient resources

When selecting the graph view for  transient (non-saved) resource - it shows up empty. As soon as I save the resource it shows up.

the reason for this seems to be that the ResourceGraphTab only passes local/cluster resources to the GraphView component, without "mixing in" the corresponding transient resources..


[SPARK-36606][DOCS][TESTS] Enhance the docs and tests of try_add/try_divide

What changes were proposed in this pull request?

The try_add function allows the following inputs:

number, number
date, number
date, interval
timestamp, interval
interval, interval

And, the try_divide function allows the following inputs:

number, number
interval, number

However, in the current code, there are only examples and tests about the (number, number) inputs. We should enhance the docs to let users know that the functions can be used for datetime and interval operations too.
Why are the changes needed?

Improve documentation and tests.
Does this PR introduce any user-facing change?

No
How was this patch tested?

New UT
Also build docs for preview:

Test build #142842 has started for PR 33861 at commit 2ede9c8.

Kubernetes integration test unable to build dist.
exiting with code: 1
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47344/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47344/

Test build #142842 has finished for PR 33861 at commit 2ede9c8.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142842/

Merged to master and branch-3.2.


Use provider.runtime to guess esbuild target

A nice to have feature request.
Docs say

The default JavaScript syntax target is set to ES2017

Plugin could use provider.runtime to change the target accordingly
In my case I ended up with
provider:
  runtime: nodejs14.x

custom:
  esbuild:
    target: es2020

And thank you for the super fast compiler!

Agreed this is a nice to have feature
One FYI, the target can actually be the Node version. Examples:
provider:
  runtime: nodejs14.x

custom:
  esbuild:
    target: node14

provider:
  runtime: nodejs12.x

custom:
  esbuild:
    target: node12

Source: https://esbuild.github.io/api/#target
That way you don't need to worry about which es version is compatible with which node version, esbuild already has that specific information to make sure the right features are used.


404 Error when querying object immediately

SDK Component

Using the operate API Client with the following code :
const instance = await zbc.createProcessInstance({
    bpmnProcessId: `c8-sdk-demo`,
    variables: {
        humanTaskStatus: 'Needs doing',
    },
})

console.log(`[Zeebe] Finished Process Instance ${instance.processInstanceKey}`)

const flow = await operate.getProcessInstanceSequenceFlows(res.processInstanceKey);
// 404 Error

const instance = await operate.getProcessInstance(res.processInstanceKey);
// 404 Error

Expected Behavior
Access to data related to the process while creating these objects on the fly returns 404 as it takes time to make these objects available which is confusing and results in an unpredictable application behavior as well as making it hard to unit test apps built with the SDK.
Current Behavior
Error for the Sequence Flow

Result of opening the same link in the browser immediately  after the error:

Error for the Process Instance

Result of opening the same link in the browser immediately  after the error:

Possible Solution
I think the main issue is related to how elastic search works and that it needs time to index the data unless its manually refreshed.
Option 1:
I suggest we introduce a method that allows the developer to refresh the index effected when needed (as optional considering the performance impact on ELK )or await for data to come back knowing that the create function returned 200 OK
Option 2:
another option is adding an option to get data as a streams and check the result in an interval and return when ready then kill the interval or something like function generators like Observables to subscribe to changes.
Option 3 (after browser support is resolved - ISSUE #79):
to elevate the experience of using these apis while building front-ends is using EventSource to get changes when its available
Steps to Reproduce

create instance
query instance
query sequence flow.

Context (Environment)
I am building a wizard application on my self-hosted environment that has each step represented as a userTask to be able to evaluate data and decide on the next step using DMNs and proper logic which requires real-time access to data and process variables to drive the UI.

Yes, this is a "known characteristic" of Camunda 8's distributed, eventually-consistent architecture.
Some great thoughts and suggested approaches.
I like Option 2 for the SDK.
Option 1 involves changes to the core system, which we can't do here.
Option 3 is from a hypothetical future.
Option 2 looks good - it's a realistic ergonomic to address this scenario. In my integration tests, I use awaited timeouts to introduce delays - and I have to tune them sometimes depending on load and location. Having a subscribeable callback would be more ergonomic.
Cleaning up resources and dealing with timeouts are two aspects that I can see.
What do you think about an API like this:
const instance = await zbc.createProcessInstance({
    bpmnProcessId: `c8-sdk-demo`,
    variables: {
        humanTaskStatus: 'Needs doing',
    },
})

console.log(`[Zeebe] Finished Process Instance ${instance.processInstanceKey}`)

const flow = await camunda8.getProcessInstanceSequenceFlows({
  processInstanceKey: res.processInstanceKey,
  timeout: 10000,
  pollInterval: 500
}).catch(() => console.log(`Didn't get it within ten seconds`)

pollInterval is optional, with some sensible default.
Under the hood, the call is executed every pollInterval until we get back a result, or we throw the 404 if we hit the timeout.
Users can "subscribe" to the Promise outcome with a callback function in then, or await it for synchronous looking code.
What  are your thoughts on that?

I like that I think it will enhance the experience a lot  , so basically something like this :
import  { HTTPError } from 'got';

..... class code ....

public async getProcessInstance(
	processInstanceKey: number | string
): Promise < ProcessInstance > {
	const headers = await this.getHeaders();
	const rest = await this.rest;
	// can also be used later for a benchmarking utility
	const requestStartTime = Date.now();

	while(true) {
		try {
			const response = rest(`process-instances/${processInstanceKey}`, {
				headers,
				parseJson: (text) => losslessParse(text, ProcessInstance),
			});

			return response.json();
		} catch (error) {
			if (error instanceof HTTPError && error.response.statusCode === 404) {
				// If we receive a 404, it means the process instance key is not found or not available yet
				// We'll retry until we reach the timeout
				if (Date.now() - requestStartTime >= timeout) {
					throw new Error('Timeout reached while waiting for process instance sequence flows');
				}
			} else {
				// Handle other types of errors
				console.error('Error fetching process instance sequence flows:', error);
				throw error;
			}
		}

		// Wait for the specified poll interval before retrying
		await new Promise(resolve => setTimeout(resolve, pollInterval));
	}
};

the other concept I had will change the way the sdk is used so maybe its not a good idea for now
// Define a function generator that fetches data at regular intervals
async function* dataGenerator(interval: number): AsyncGenerator<any, void, unknown> {
    while (true) {
        try {
            const data = await fetchData();
            if (data) { // or 404
                // If data is available, yield it
                yield data;
                // Break the loop if data is available
                break;
            }
        } catch (error) {
            console.error('Error fetching data:', error);
        }
        // Wait for the specified interval before trying again
        await new Promise(resolve => setTimeout(resolve, interval));
    }
}

// Usage example
(async () => {
    const generator = dataGenerator(5000); // Fetch data every 5 seconds
    for await (const data of generator) {
        console.log('Data:', data);
        // Use the fetched data here
    }
})();

A related issue are the search API calls - on Operate and Tasklist you can search for processes.
const processes = await operate
  .searchProcessInstances({
    filter: {
      processDefinitionKey,
      state: 'ACTIVE',
    },
})

That is not going to return a 404 while waiting for eventual consistency - it is going to return a 200 with zero results.
At some point it will return a 200 with one or more results.
So these APIs could benefit from a generator API.
What do you think about a search subscription that is an event emitter that wraps the dataGenerator and emits data event?
That might be a more familiar API for most developers. So something like:
const processSubscription = operate
  .subscribeProcessInstances({
    filter: {
      processDefinitionKey,
      state: 'ACTIVE',
    },
})

processSubscription.on('data', process => console.log(process))

@jwulf if we have fix multiple issues at once I think it would be great , the only thing I see here is that what if the search is for something that dose not exist , the subscription needs to close after an interval , so maybe we should merge both solution in one and add it to all our apis.
const processSubscription = operate
  .subscribeProcessInstances({
    timeout: 10000, // Optiona With default value of 5000
    pollInterval: 500 // Optional with default value of 300
    filter: {
      processDefinitionKey,
      state: 'ACTIVE',
    },
  })

processSubscription.on('data', process => {
  console.log(process)
  processSubscription.close()
  // effectively processSubscription.once() with cleanup
})

Should the timeout default to something or should it default to infinite?
Think about this use case: refreshing tasks from tasklist as they appear, adding them to a tasklist in a frontend. That is a subscription that you would want to be persistent.
When an timeout for the subscription is specified, it should emit an event when it times out - and then maybe only if no data was received.

The timeout needs to have default to value just to avoid any memory leaks on the server as this has a filter "its looking for something" the default can be something a little high like 50000 MS and the interval will keep refreshing when new data received.
in the use case mentioned (No process or task based filters ) I would design it around ServerSentEvents or WebSockets , something like fetch-event-source as it will let the browser handle the logic of having an interval on the front-end and on the backend its exactly like emit event.
operate
  .subscribeProcessInstances({
    filter: {
      processDefinitionKey,
      state: 'ACTIVE',
    },
    async onopen(response) {

    },
    onmessage(msg) {

    },
    onclose() {

    },
    onerror(err) {

    }
});

we can add events to manage the full life cycle of the subscription like the methods above , in the case of any ID based filter the timeout will trigger the on error callback any state or assignee like filters will keep going.


Default NODE_ENV to test

Followup to https://github.com/vitest-dev/vscode/pull/309
When running the extension on a fresh install I could see the NODE_ENV by default is coming through as true. If we want to match the behavior of startVitest in  https://github.com/vitest-dev/vitest/blob/5c7e9ca05491aeda225ce4616f06eefcd068c0b4/packages/vitest/src/node/cli/cli-api.ts#L40 we should default to test.

Thanks 👍🏻

Omg, bad typo... Thanks for the fix!


ember data emitting warning because of 'extractMeta' function

WARNING: You've defined 'extractMeta' in (unknown mixin) which is not used for serializers extending JSONAPISerializer. Read more at http://emberjs.com/api/data/classes/DS.JSONAPISerializer.html#toc_customizing-meta on how to customize meta when using JSON API.
Defined in ember data code here and was added in 2.11.0-beta.1
Perhaps we can rename this function to avoid the warning?

I dont know if its because this Warning but I also cant get any model property on Route. Example:
let languages = this.store.findAll('language'); languages.get('language');
Ps: I have language field inside _embedded
Thanks,

Modified the code and it seems to work fine. Extract Meta is not used by JSONAPIAdapter. It seems to just be called that in the mixin as a naming convention

This seems to have come back again with a newer version of ember-data. Now seeing:
WARNING: You've defined 'extractMeta' in (unknown) which is not used for serializers extending JSONAPISerializer. Read more at https://emberjs.com/api/data/classes/DS.JSONAPISerializer on how to customize meta when using JSON API.
ember-data version: 3.9.0
emver-data-hal-9000 version: 0.3.3


MINOR: Fix ReadOnlySessionStore java docs

Fixes some typos in javadocs of readOnly SessionStore, and minor typos in WindowStore / KeyValueStore.

@mimaison Please check this when you have some time

@AyoubOm thanks for your contribution!


Create an Action (webhook) in Hasura to check whether the reward has been claimed

... and add a boolean isClaimed (or whatever) field to the reward response

Issues with creating action in Hasura needs investigating
type UpdatedReward {
  is_claimed : String
}

type achievements2 extends achievement {
  reward_status : UpdatedReward
}

this is done


Created directory recursively before creating write stream

In case you do not want to store binaries in your repository (.zip files) and do not have a zips folder, the script fails with a file not found error, because the parent directory doesn't exist

Thank you for the contribution. The request has been merged.


Add a help command for the ztunnel cli

This should probably be replaced by something like clap but even this is a small quality of life improvement. Feel free to close if you're not interested.

Hi @film42. Thanks for your PR.
I'm waiting for a istio member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.
Once the patch is verified, the new status will be reflected by the ok-to-test label.
I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.


Wrong cmd slot of xread in cluster

Issue tracker is used for reporting bugs and discussing new features. Please use
stackoverflow for supporting issues.

Expected Behavior

Xread should not occur MOVED error in cluster.
Current Behavior

Xread always occur MOVED error in cluster.
Possible Solution

Steps to Reproduce

github.com/go-redis/redis/v7 v7.4.0
In cluster mode
XRead(&redis.XReadArgs{Streams: []string{"test_stream_pub_sub", "$"}, Count:   100, Block:   0})

Wrong cmd args position used to slot, should be "test_stream_pub_sub" instead of "count".
Incidentally please check other cmd about stream.

Context (Environment)

Detailed Description

Possible Implementation

go-redis uses command information from Redis
command info xread
1) 1) "xread"
   2) (integer) -4
   3) 1) readonly
      2) movablekeys
   4) (integer) 1
   5) (integer) 1
   6) (integer) 1
   7) 1) @read
      2) @stream
      3) @slow
      4) @blocking

Here 4) (integer) 1 is the position (1) of the key argument. So it uses args[1] which is count. So everything works as expected (but incorrectly).

go-redis uses command information from Redis
command info xread
1) 1) "xread"
   2) (integer) -4
   3) 1) readonly
      2) movablekeys
   4) (integer) 1
   5) (integer) 1
   6) (integer) 1
   7) 1) @read
      2) @stream
      3) @slow
      4) @blocking

Here 4) (integer) 1 is the position (1) of the key argument. So it uses args[1] which is count. So everything works as expected (but incorrectly).

go-redis uses command information from Redis
command info xread
1) 1) "xread"
   2) (integer) -4
   3) 1) readonly
      2) movablekeys
   4) (integer) 1
   5) (integer) 1
   6) (integer) 1
   7) 1) @read
      2) @stream
      3) @slow
      4) @blocking

Here 4) (integer) 1 is the position (1) of the key argument. So it uses args[1] which is count. So everything works as expected (but incorrectly).

go-redis uses command information from Redis
command info xread
1) 1) "xread"
   2) (integer) -4
   3) 1) readonly
      2) movablekeys
   4) (integer) 1
   5) (integer) 1
   6) (integer) 1
   7) 1) @read
      2) @stream
      3) @slow
      4) @blocking

Here 4) (integer) 1 is the position (1) of the key argument. So it uses args[1] which is count. So everything works as expected (but incorrectly).

There is movablekeys in command flags.It is the note from https://redis.io/commands/command.
"movablekeys - keys have no pre-determined position. You must discover keys yourself."

So the client should do more to discover keys, right?

Should be fixed by https://github.com/go-redis/redis/pull/1502


Create CODE_OF_CONDUCT.md

Hi, I just added a code of conduct from covenant template.

Merged as admin, I can't approve my PR


Enable NvEnc AV1 encoding on Linux

This looks way too simple to work, but then again, VA-API was basically the same, except it also expected an encoder profile and some other bits and bops for 10bit support, but the Linux NvEnc pipeline never sets a profile for HEVC, either - so I guess it just defaults to the MAIN profile. Somebody should probably test this before it gets merged.

Hello,
I've tested this on my Arch EndeavourOS, Kernel 6.7.5, Nvidia 4080, driver version 550, Quest 3 - and it's not working.
I get:
"Dropping video packet. Reason: waiting for IDR frame"
error, when setting it to "AV1 AMD Only".

The pr completely forgets to enable the flag in xtask to build with av1 support idk how I missed that


Error while loading module using Packer

Found this startup loading error while loading treesitter-context module:
packer.nvim: Error running config for nvim-treesitter-context: [string "..."]:0: loop or previous error loading module 'treesitter-context'

I'm using this following configuration on Packer
   use {
      "romgrk/nvim-treesitter-context",
       config = function()
           local ctx = require "treesitter-context"

           ctx.setup({
             patterns = { -- Match patterns for TS nodes. These get wrapped to match at word boundaries.
                 -- For all filetypes
                 -- Note that setting an entry here replaces all other patterns for this entry.
                 -- By setting the 'default' entry below, you can control which nodes you want to
                 -- appear in the context window.
                 default = {
                     'class',
                     'function',
                     'method',
                     'for', -- These won't appear in the context
                     'while',
                     'if',
                     'switch',
                     'case',
                 },
                 -- Example for a specific filetype.
                 -- If a pattern is missing, *open a PR* so everyone can benefit.
                 --   rust = {
                 --       'impl_item',
                 --   },
             },
           })

      end,
      event = "BufWinEnter",
   }

Run PackerInstall then PackerCompile after opening nvim, I found that error on startup.
It seems like module treesitter-context is not loaded properly. Am I missed something?

Unsure what's going on. Did you resolve it? Does it work with another package manager?

Sorry late reply. Yes, I've already fixed the issue. I use NvChad, and the problem occurs because nvim-treesitter/nvim-treesitter will only be loaded when event BufRead occurs, can be seen here. So nvim-treesitter-context won't loaded properly with above configuration. Here's the config I use right now, and it's works!
   use {
      "romgrk/nvim-treesitter-context",
      event = "BufRead", -- make sure it's match with `nvim-treesitter/nvim-treesitter` plugin, don't mess it up
      requires = { "nvim-treesitter/nvim-treesitter" }, -- nvim-treesitter-context depends on nvim-treesitter
      config = function()
         local ctx = require "treesitter-context"
         ctx.setup {}
      end,
   }

I will marked it as done, and close the issue 👍


[Fix] Strict concurrency warnings

Description
This PR fixes some concurrency warnings.
To test, enable strict concurrency = complete from build settings.

Thanks for the thoughts.


Add latest Opera Desktop Versions

Change/Update request
Browser: Opera Desktop
Actual Situation: Latest Opera Desktop Version is 86
Target Situation: Include Opera Desktop Versions 87-89 (89 has been released on Jul 7: https://blogs.opera.com/desktop/2022/07/opera-89-stable/)
Thanks!

Done!


code for training

Hi, would you please release the code for training the StereoDNN?

I think the paper is bad


cData not prepopulating in DEV and Test Environments

Background:
Since OCI migration TESt and DEV have not had data pulling into form

Steps to Reproduce:
log in to environment with IDIR
Actual Behavior:
Expected Behaviour:
Environment:
Acceptance Criteria:
Test:

[ ] Component functionally test
[ ] Integration
[ ] End to end UX
[ ] End to end functionality
[ ] if data impacting confirm w/BI/DIO team

Tested different forms in TEST & DEV!


ATTESTATION_SUBNET_COUNT should be dynamic

ATTESTATION_SUBNET_COUNT is currently a spec constant that defines the valid range of subnets persistent committees should randomly subscribe to (this value is currently set to 64).  When the network has a smaller number of validators, this results in nodes joining subnets with committee counts of 0.  Spreading persistent committee members across unused subnets could significantly inhibit our ability to find subnets in a timely fashion.
The following table compares active validators to resulting committee counts:

ACTIVE VALIDATORS
COMMITTEE COUNT

16384
4

32768
8

...
...

262144
64

In other words,  we don't need 64 attestation subnets until we reach 262,144 active validators.
To fix this, I suggest that we replace  ATTESTATION_SUBNET_COUNT with a method that calculates it based on the committee count.  Consider something like this to start:
def get_attestation_subnet_count_at_slot(state: BeaconState, slot: Slot) -> uint64:
    return min(get_committee_count_at_slot(state,slot), 64)

that seems fine if we assume a static validator set, but we really should consider how many validators could exit or activate within a PERSISTENT_COMMITTEE_PERIOD to ensure we create enough attestation subnets.  To do that, we should take a look at worst case validator churn limits.  From the spec, get_validator_churn_limit() is defined as follows:
def get_validator_churn_limit(state: BeaconState) -> uint64:
    """
    Return the validator churn limit for the current epoch.
    """
    active_validator_indices = get_active_validator_indices(state, get_current_epoch(state))
    return max(MIN_PER_EPOCH_CHURN_LIMIT, len(active_validator_indices) // CHURN_LIMIT_QUOTIENT)

In order to have a churn limit > 4 , this would have to be true:
len(active_validator_indices)  > 4 * CHURN_LIMIT_QUOTIENT
In other words, we would need 262,144 active validators.  From the table above, we know this is the exact amount needed to reach a committee count of 64.  This means that we can calculate the max increase/decrease in committee counts within a PERSISTEN_COMMITTEE_PERIOD as follows:
Change in num active validators = 4*PERSISTEN_COMMITTEE_PERIOD = +/- 8,192 validators
Change in committee count = 8,192/SLOTS_PER_EPOCH/TARGET_COMMITTEE_SIZE = +/- 2

It's probably safe to only consider the max increase in committee count (worst, worst case we have 2 additional attestation subnets).  As a result, we can modify our new method get_attestation_subnet_count_at_slot() from above to account for the worst case scenario where 2 additional committees could be added within a PERSISTENT_COMMITTEE_PERIOD:
def get_attestation_subnet_count_at_slot(state: BeaconState, slot: Slot) -> uint64:
    max_additional_committees = MIN_PER_EPOCH_CHURN_LIMIT*PERSISTEN_COMMITTEE_PERIOD//SLOTS_PER_EPOCH//TARGET_COMMITTEE_SIZE
    return min(get_committee_count_at_slot(state,slot) + max_additional_committees, 64)

If we want to retain the ability to control the committee to attestation subnets ratio, we could add a configurable spec constant called COMMITTEES_PER_SUBNET that can be set to 1 and get_attestation_subnet_count_at_slot could be modified as follows:
def get_attestation_subnet_count_at_slot(state: BeaconState, slot: Slot) -> uint64:
    max_additional_committees = MIN_PER_EPOCH_CHURN_LIMIT*PERSISTEN_COMMITTEE_PERIOD//SLOTS_PER_EPOCH//TARGET_COMMITTEE_SIZE
    return min(get_committee_count_at_slot(state,slot) + max_additional_committees, 64) // COMMITTEES_PER_SUBNET

Credit/blame to @mkalinin for talking this out with me and @djrtwo for pointing out that low validator counts could result in persistent committee members joining unused subnets.

Also we need EPOCHS_PER_RANDOM_SUBNET_SUBSCRIPTION * 2 (54 hours) warm-up period when validators are required to join randomly to new subnet, but it's not involved in any attestation/block duties. Otherwise we have 0 nodes skeleton when new subnet is added. It couldn't be discovered.

Thanks @jrhea! I had similar thoughts as we were discussing the attnets debugging work you've been doing.
I think that predicting the validator set size in some future epoch and accounting for how many committees will be is not worth the added complexity here (especially re: @zilm13's comment about the warm-up).
Another couple of notes:

Phase 1 won't have a notion of unused subnets due to the actual assignment of shards to each so replicating full subnets (and the size of persistent vals on the used subnets) to some extent seems reasonable here
Having more validators on persistent subnets has diminishing returns in terms of increased bandwidth to each node.

The biggest arguments might lie in being able to find peers more easily in the dht but this might not be a realistic approximation of what comes in phase 1.

especially re: @zilm13's comment about the warm-up

I don't see his comment as really being an issue.  The warmup is accounted for in the PERSISTENT_COMMITTEE_PERIOD of 9 days.  If that constant is going away, we simple use EPOCHS_PER_RANDOM_SUBNET_SUBSCRIPTION * 2 instead.  We are just looking for a worst case here to apply as a lead.  With a 9 day lead...we are looking at an additional 2 committees.

Phase 1 won't have a notion of unused subnets due to the actual assignment of shards to each so replicating full subnets (and the size of persistent vals on the used subnets) to some extent seems reasonable here

My understanding was that we would be using topic discovery in phase 1.  If so, then i don't much value in this experiment.

Having more validators on persistent subnets has diminishing returns in terms of increased bandwidth to each node.

I believe this can be addressed, but it is part of a solution to another concern i have.

The biggest arguments might lie in being able to find peers more easily in the dht but this might not be a realistic approximation of what comes in phase 1.

Again, i thought we plan on implementing topic discovery for phase 1. if not, i see your point.  If we are, then i would consider this argument less compelling.

I think that predicting the validator set size in some future epoch and accounting for how many committees will be is not worth the added complexity here

I forgot to comment on this.  Ya, I figured you might say that and you would be a better judge of the ramifications of this.  Your call boss

This would make sense to use the first option
def get_attestation_subnet_count_at_slot(state: BeaconState, slot: Slot) -> uint64:
    return min(get_committee_count_at_slot(state,slot), 64)

Using the current active validator set seems like a good gauge as any for determining the amount of subnets.
Although there might be a possibility of newly added validators biasing the subnets through this period EPOCHS_PER_RANDOM_SUBNET_SUBSCRIPTION * 2. It isn't something we should worry about.
The biggest arguments might lie in being able to find peers more easily in the dht but this might not be a realistic approximation of what comes in phase 1.

While that maybe true, the initial genesis validator set would only have 4 committees. Having validators subscribe to the other 60 empty subnets seems a bit pointless. You would have 94% of your validators not participating in a functioning subnet.

You would have 94% of your validators not participating in a functioning subnet.

Which is totally fine. If we move to phase 1 with similar number of validators (this is feasible though unlikely), all persistent subnets would then be used, but they would also be the same size as those today. The size of the persistent subnet approximates the phase 1 behaviour more than all of the validators being assigned to subnets that are actually used.

Using the current active validator set seems like a good gauge as any for determining the amount of subnets.

But this does not adequately capture the moment in which the subnet count would jump up and as @zilm13 noted, the subnet would be under-provisioned in number of validators.

But this does not adequately capture the moment in which the subnet count would jump up and as @zilm13 noted, the subnet would be under-provisioned in number of validators.

That's why I suggested adding a calculation that gives the worst case possible number of new committees.
def get_attestation_subnet_count_at_slot(state: BeaconState, slot: Slot) -> uint64:
    max_additional_committees = MIN_PER_EPOCH_CHURN_LIMIT*PERSISTENT_COMMITTEE_PERIOD//SLOTS_PER_EPOCH//TARGET_COMMITTEE_SIZE
    return min(get_committee_count_at_slot(state,slot) + max_additional_committees, 64) // COMMITTEES_PER_SUBNET

Granted that PERSISTENT_COMMITTEE_PERIOD  should be changed to EPOCHS_PER_RANDOM_SUBNET_SUBSCRIPTION * 2, but considering that PERSISTENT_COMMITTEE_PERIOD = 2048 epochs and only results in an additional 2 committees....i think switching to EPOCHS_PER_RANDOM_SUBNET_SUBSCRIPTION * 2 = 512 epochs is even easier.
What am i missing?

@djrtwo if I remember correctly, one of the possibilities we discussed was reverting back to using all 64 committees so that our network is more similar to how it will be in phase 1.  Is that still on the table?

working on that now
On the table, but not certain worth the complexity

fix merged! thanks @jrhea !


Use a Python entry point for 'display_urdf'

On many platforms, the unversioned python executable is not Python 2. A better solution here would be to change this package to use an entry point, as shebang python scripts don't work correctly on Windows.
This same change applies to Noetic, but there isn't a Noetic-specific branch to make the change against.

remove the shebang, make the script not executable, and add a console_scripts entrypoint in the setup.py

Done. I split it into three commits. The first resolves a ROS 2 bug, the second does the move, and the third fixes up the script as an entry point. We should rebase-merge this to preserve the change history of display_urdf.

I don't think any changes are needed in ROS 1

You're right, I forgot about that. Good call - no changes necessary for ROS 1.

@cottsay Friendly ping here; I think this is close to getting done, with just a couple more nits to fix from @sloretz .

Okay, I dropped the argv stuff. Again, please rebase-merge this PR when you're happy with it - I updated the commits to include a reference to this PR so they look good when rebase-merged.

I'm going to go ahead and merge this, thanks for the patch @cottsay