GET BUG in 4.4.0

Hi,
When I perform a GET on this URL : "https://hopteam.hop.fr" using 4.3.1 version, it works fine (response is 200)
If I do the same GET with 4.4.0, responses is 404
Here is my code
           request = new Request.Builder()                 .url(url_hopteam)                 .build();             response = client.newCall(request).execute();
The client is
       client = new OkHttpClient.Builder()             .connectTimeout(28, TimeUnit.SECONDS)             .writeTimeout(20, TimeUnit.SECONDS)             .readTimeout(28, TimeUnit.SECONDS)             .addNetworkInterceptor(new LoggingInterceptor())             .cookieJar(cookieJar)             .connectionSpecs(Arrays.asList(ConnectionSpec.MODERN_TLS, ConnectionSpec.COMPATIBLE_TLS))             .build();
In my app build.gradle, I have
    implementation 'com.squareup.okhttp3:okhttp:4.4.0'
In the project build.gradle, I have
       classpath 'com.android.tools.build:gradle:3.5.2'

I suspect this is a server misconfiguration, potentially triggered by a subtle change in OkHttp behaviour with redirects, connection coalescing and DNS ordering.
$ okurl --debug https://hopteam.hop.fr 2>&1 | grep Location
11:40:19.114	Location: https://office.hop.fr/opensso/saml2/jsp/spSSOInit.jsp?metaAlias=/MSAD/sp&idpEntityID=http://madfs.hop.fr/adfs/services/trust&NameIDFormat=urn:oasis:names:tc:SAML:2.0:nameid-format:transient&binding=HTTP-POST&goto=https://hopteam.hop.fr/home
11:40:19.514	Location: https://madfs.hop.fr/adfs/ls/?SAMLRequest=...

I'll check the certificates to confirm this.

Did we inadvertently started coalescing http/1 connections? Other scarier regression would be coalescing redirect calls without a 2nd DNS check

I suspect the latter.

$ nslookup hopteam.hop.fr | grep Address
Address: <IP_ADDRESS>
$ nslookup madfs.hop.fr | grep Address
Address: <IP_ADDRESS>
$ nslookup office.hop.fr | grep Address
Address: <IP_ADDRESS>

Cert indicates this is fine

It does look like it's affecting HTTP/1.1 - but not strongly confirmed.
n.b. For HTTP/2 - A reminder of the difference in Firefox and Chrome connection coalescing
https://daniel.haxx.se/blog/2016/08/18/http2-connection-coalescing/

The Firefox way
Host A has two addresses, host B has two addresses. The lists of addresses are not the same, but there is an overlap – both lists contain <IP_ADDRESS>. And the host A has already stated that it is authoritative for B as well. In this situation, Firefox will not make a second connect to host B. It will reuse the connection to host A and ask for host B’s content over that single shared connection. This is the most aggressive coalescing method in use.
one connection
The Chrome way
Chrome features a slightly less aggressive coalescing. In the example above, when the browser has connected to <IP_ADDRESS> for the first host name, Chrome will require that the IPs for host B contains that specific IP for it to reuse that connection.  If the returned IPs for host B really are <IP_ADDRESS> and <IP_ADDRESS>, it clearly doesn’t contain <IP_ADDRESS> and so Chrome will create a new connection to host B.
Chrome will reuse the connection to host A if resolving host B returns a list that contains the specific IP of the connection host A is already using.

@MS893 Thanks for the detailed bug report!

I just published 4.4.1 to Maven Central. It should be available in under an hour.

You are the best team ! Thank you, I'm happy if I was useful.


Error on the path using Zeit V2

Environment details

OS: on zeit deployment
Node.js version: latest
npm version: latest
@google-cloud/vision version: 1.0.0

Steps to reproduce
You can view source online here : https://zeit.co/deployments/testvision-80c8uqjob.now.sh/source
The bug exists only on the deployment with zeit on their new version (2).
Just trying to require the library generates the error.
You can see the logs here:  https://zeit.co/gmulti/testvision/80c8uqjob/logs
And, online URL : https://testvision-80c8uqjob.now.sh
I also proposed a post on spectrum to tell them about it:
https://spectrum.chat/zeit/serve/google-vision-1-0-0-not-work~966bc190-10ca-43a5-82c9-4d31f3c739e0

This has just been reported here : https://github.com/zeit/ncc/issues/398

Same issue with @google-cloud/datastore

@Gmulti @ecjs my guess is that this bug will crop up on all of our libraries which depend on grpc-js; ncc is not an environment we test in unfortunately.
We'll make an effort to dig into this as soon as possible, and if anything is figured out on the ncc side of things, we'll happily try to land a patch.

Having same issue

looping in @alexander-fenster, who's team works more closely with grpc-js.

As of today, most @google-cloud/* libraries use @grpc/grpc-js, protobufjs, and google-auth-library, which are the three major dependencies. @Gmulti If you could try compiling the similar single-line require file using these two libraries it might help to pinpoint the error to one of these dependencies.
The linked issue also says that googleapis also has an issue, which is weird, since googleapis is a very straightforward HTTP/1 client library, very different from @google-cloud onces (but again, it uses the auth library). So trying to pinpoint which dependency breaks everything will help solving the problem for you all.
Thanks!


Implement Node Metadata Handling

Summary
In order to better handle requests from Enso Studio, the Enso language server needs to have the ability to reason about node metadata in the program source. Having this ability will mean that messages across the wire can be far more lightweight. Furthermore, it will ensure that the metadata is updated correctly on changes sent between the client and the server.
Value
The client and the server will be operating on the same set of node metadata.
Specification

When recieving a synchronisation or diff message from the client, or sending one to the client, the server must ensure that its internal representation of the node metadata is kept up to date.
This metadata includes node positioning and other relevant elements.
It may be possible to implement this using a library shared between client and server, but this is yet to be determined.

Acceptance Criteria & Test Cases

The language server handles node metadata correctly.

Superseded by a new plan.


Fixed Change input selection icon #127

Fixed Change input selection icon #127 , and removed the 'x' icon from checked box

duplicate of #128

@Dhruvnotfound implemented this, but I will add another issue that you could work on


Scheduling applications to a k8s namespace based on CF space

As far as I can tell eirini only supports specifying a single namespace for running user applications: https://github.com/cloudfoundry-incubator/eirini#configuring-opi-with-cloud-foundry-and-kubernetes
Is it possible to configure eirini to schedule applications to many namespaces? I'd like to be able to deploy applications bound to a CF space to a dedicated k8s namespace.

Hi @zankich - as of today we assume just one namespace for all CF apps, but only because we haven't heard strong use cases for doing better yet and one namespace was therefore The Simplest Thing That Could Work.
We'd be very receptive to ideas about better ways to do this, but we'd need to think through what the best boundary is: for example should a CF space == namespace or should it be CF org == namespace or should it be CF isolation segment == namespace?
Can you outline a bit more about your use case for wanting a specific space, and do you have strong opinions about whether it should be at the org, space or isolation segment level?

To support tenancy constructs between space developers, I'm pretty confident the mapping should be space == namespace. If we map things at a higher level, there will be no way we can ever hope to offer kubectl access to developers as they'll be able to access their space-neighbor's resources. Namespaces are the most granular tenancy construct available in k8s, and spaces are the most granular available in CF. It's easier to construct less-granular tenancy groupings than it is to divide what is already the most granular thing available.
They both have the word "space" in there, too! Very convenient.
In terms of use cases, there are lots once we assume a world where k8s is the only backend we care about. Space quota enforcement could be pushed down to namespaces. Data services could be provisioned and shared in new ways without exposing them to entire orgs or foundations.
There's even some potential that we could externalize CF's Authorization model into k8s, effectively a sort of perm_ng using k8s Users, Roles, and Rolebindings.
That said, I'm not entirely sure that OPI needs to concern itself with the CF<->k8s mapping for spaces. Each request could take an additional, optional namespace parameter and error if that namespace isn't already available.

I think that makes a lot of sense @cwlbraa, I've created https://www.pivotaltracker.com/story/show/172350759 for the Eirini side.
Very cool that this way the problem of figuring out the right mapping of CAPI concepts to namespaces can live in CAPI, which seems obviously right (though I guess this means we've going to leak the concept of "namespace" out of OPI, I think that's fine tho -- we really don't need to pretend at this point that we can productively avoid leaking kubernetes concepts around).

@julz It is probably hijacking, but as isolation segments were mentioned above I thought I'd add my 2 cents anyway.
I would like to see Eirini deploying CF apps not only into separate namespaces, but also into separate k8s clusters. I suppose the mapping would be isolation segment == cluster. Would it make sense, to introduce such a capability early on together with the introduction of a namespace parameter?

@zankich Is there already an issue or tracker item for the CAPI side of this? I would comment that we might want to go for cf-<org>-<space> as a naming pattern to prevent clashes with non-cf namespaces and between different cf orgs.
But just mentioned here it will go lost, as Eirini should stick to simply using whatever namespace was provided.

@loewenstein We're also interested in multicluster scheduling, but getting working CF apps in multiple clusters would require a bunch of common system components deployed and configured in those clusters -- Would scheduling just the LRPs across clusters have some benefit that I'm missing?

A working CF app would need an envoy as Istio ingress gateway and another as sidecar. In addition, logs and metrics would have to be scraped. What other system components am I missing?
We would gain an isolation level between apps similar to that of cf4bosh with isolation segments or at least that is what I suppose. Currently, with cf-for-k8s we do not even get this isolation between apps and system components.

for example should a CF space == namespace

If possible, I will be very interested to have this level of relation between a space and namespace as by example RBAC or resources to be assigned could be defined at this level

CF org

We could use the org name/level to tag the applications, using the k8s label, part of that org as org: my CF org in order to let a user to query easily all the applications, k8s resources belonging to that org.
Of course, this proposition could also be extended to the space level to also create a k8s label for the space - space: my space

perm_ng using k8s Users, Roles, and Rolebindings.

If you can manage that correctly, that would be fantastic as currently there is no easy way to manage RBAC, assign it to a user's profile (developer, ...) under kubernetes


hotfix flaw comments

#545 Added the field order to the FlawComments model but the field was excluded from the comment creation endpoint, so an error was thrown when trying to create a comment.

Fixes OSIDB-3547

This does not seem to me as a step to the right direction. User should not care for the any order here. (S)he is always creating the very last comment and the backend should make sure that the order will be applied in this sense. Or do I miss something?
If it is mean as some temporary hotfix which is going to be removed later, it is not a good idea. API removals are breaking changes and require major release.

After further discussion and testing with @dmonzonis we think that removing the field is a better solution, internally OSIDB creates the field sequentially so the user does not need to do it

@jsvob The field was added as required on this pull request https://github.com/RedHatProductSecurity/osidb/pull/752
seems like the behavior of django changed after the update because I was unable to reproduce it locally until I rebuilt the containers with the new version

I think that all commits should be merged into one as they relate to the same thing.


create docker image

The stackdriver_exporter image is not listed on https://hub.docker.com/u/prometheuscommunity
Using the docker run line in the README does not work, access denied

I'm in the process of cutting a new release that will provide the latest tag.

I've released 0.7.0. This should now be fixed.

Fantastic, thanks for the quick response


[NFC] Update SwiftPM-Package.xcscheme

Since this file hasn't been added to .gitignore, we have to update it after the list of targets is updated.

@swift-ci test windows


Tooltip of elided long layer names doesn't give full name when coming from a plugin

🐛 Bug Report
When having layers with long names, the names are elided, which is good...
Normally when mousing over them, the full name is provided in the tool tip.
This works for example for a sample image -- e.g. astronaut renamed something really long.
But, when using a plugin to open, after mousing over them, the full layer name is not provided, instead info about the filename and plugin is provided -- useful, but not exactly the most helpful when layers have long names that start the same. Especially since this is also provided when mousing the canvas. E.g.:

The only way to see the full name is to expand the layer list, which is annoying.
💡 Steps to Reproduce

open an image file with a plugin -- can be any file, e.g. a jpg or png using builtins
make the name of the layer really long, enough to elide, e.g. add qwertyuiopasdfghjklzxcvbnmqwertyuiop
mouse over the layer name

💡 Expected Behavior
Disambiguating long-named layers is really important. Knowing the file name of origin and plugin is nice.
Seeing the full text of an elided label should be preserved when a plugin is used.
Ideally, when using a plugin the tooltip would gain the file name and plugin name as a 2nd line, with the full layer name being still provided.
🌎 Environment
napari: 0.4.18
Platform: macOS-13.6.2-arm64-arm-64bit
System: MacOS 13.6.2
Python: 3.10.12 | packaged by conda-forge | (main, Jun 23 2023, 22:41:52) [Clang 15.0.7 ]
Qt: 5.15.8
PyQt5: 5.15.9
NumPy: 1.25.2
SciPy: 1.11.1
Dask: 2023.8.0
VisPy: 0.12.2
magicgui: 0.7.2
superqt: unknown
in-n-out: 0.1.8
app-model: 0.2.0
npe2: 0.7.2
OpenGL:

GL version:  2.1 Metal - 83.1
MAX_TEXTURE_SIZE: 16384

Screens:

screen 1: resolution 2056x1329, scale 2.0
screen 2: resolution 1920x1200, scale 1.0

Settings path:

/Users/sobolp/Library/Application Support/napari/napari-418_d279d6cf5d4193876ed97858e757fe322037331b/settings.yaml
Plugins:
napari: 0.4.18 (77 contributions)
napari-aicsimageio: 0.7.2 (2 contributions)
napari-animation: 0.0.7 (2 contributions)
napari-console: 0.0.8 (0 contributions)
napari-ome-zarr: 0.5.2 (2 contributions)
napari-svg: 0.1.10 (2 contributions)
napari-workshop-browser: 0.0.3 (2 contributions)
ome-types: 0.4.1 (2 contributions)

💡 Additional Context
No response

@psobolewskiPhD the tooltip is generated here:
https://github.com/napari/napari/blob/9d0c449553eaf1acc3be1bf9bc0c8b3eec05afc6/napari/_qt/containers/qt_layer_model.py#L37-L41
If you have a better idea of how it should look, then it may be simple PR, that could be cherry-picked to 0.4.18.
I do not have the code to determine if label is elited, but we could always add this information.


Use [*]Button.styleFrom in theme

Reduces the verbosity of theme.dart by using static Flutter methods to generate each ButtonStyle instead of repeating WidgetState over and over.

Apparently the indentation was also fixed... ignore whitespace changes in the diff for convenience


CI: Only push to report repo and create badge for upstream

Avoid failures like https://github.com/hugovk/coveragepy/actions/runs/7854238034/job/21434885977 when forks update from upstream/master.

Thanks!


Issue with api link

Since I have a Website with TechnicSolder in it, the issue is when I go to this link http://windmillgaming.tk/TechnicSolder/Solder/public/api/, it redirects me to my website because of the Trailing Slash, without the Trailing Slash after api it takes me to the correct api link for Solder, is there any way I can fix that? here is my .htaccess file for my Website: http://pastebin.com/rMwhHnD8

oh and im using the LAMP stack on Ubuntu 14.04, I tried installing the LEMP Stack, but Nginx is very confusing when it comes to Rewrites

You should probably look into setting up virtual hosts on your Apache server.

how does one do that?

ok so i got the Virtual Host Working at solder.windmillgaming.tk, now im having an issue where this shows Error in exception handler.

ok nvm, forgot to set read/write perms

ok so it still tries to redirect me to http://solder.windmillgaming.tk/api

ok it works with http://solder.windmillgaming.tk/TechnicSolder/public/index.php/api/ sadly I cant use .htaccess but its fine

ok good its working now :D


ISSUE-773: Ignore null values for a field in avro record to fix runti…

…me exceptions
Change-Id: Ifb6cf88a2e0e6e777d6eb46226a212f13d2c6f5c

@priyank5485 @satishd should we make this as configurable option and default to true.
ignore.invalid.messages on all the sources side and also extend this to unparsable event.
I think for the most users they might want to ignore incase of mal-formed or null events but for some it might important to throw an error an stop processing.  In case of mal-formed event we should also log that.

@harshach I guess you intended to put earlier comment for https://github.com/hortonworks/registry/pull/168

@harshach Agree with you on having a setting like that. I remember discussing this with @satishd His suggestion was to add a checked exception to avro deserializers in SR. That way SAM source using SR deserializer can have a catch block for the checked exception and based on the config you suggested we can ignore the message totally and log it. However if ignore.invalid.messages was set to false, not sure how to handle that. Need to think more about that.

@harshach @priyank5485 It may be better to extend this to earlier concept of invalid/unparsed messages stream at source level in which users can get complete control of handling invalid messages.
@priyank5485 My suggestion was to catch SerDesException thrown by deserializer in spouts and take respective action. Deserializer can not catch and log as it has to return some value back to the caller of deserialize method.

@satishd Having invalid/unparsed stream sounds like a possible approach. If we decide to do that, it will need more changes and probably some UI changes as well. We can cover it in another issue if both of you are okay. Also, now i remember your suggestion. Just curious cant we have a catch all block in SR deserializer and throw it as a checked exception that is a part of deserialize method? Why does deserialize have to return something?

Just curious cant we have a catch all block in SR deserializer and throw it as a checked exception that is a part of deserialize method? Why does deserialize have to return something?

@priyank5485 What I meant is deserializer can not just ignore invalid messages by catching exceptions and log them without throwing exception or returning payloads back to consumer of deserializer. Consumers of deserializer need to handle the logic of handling errors and take respective action.
I am fine with addressing invalid messages at source with a new issue.

@satishd Raised this https://github.com/hortonworks/streamline/issues/808 for invalid or messages that dont adhere to schema

discussed with @priyank5485 . Since the current change is already running in a customer env and proven to be working. Adding Optional.empty() could provide additional issues like here https://github.com/hortonworks/streamline/blob/master/streams/functions/src/main/java/com/hortonworks/streamline/streams/udaf/NumberSum.java#L41 . We need to check across the codebase to make sure we ignore Optional.empty().
Lets go with deleting the key for this release. Lets open a Issue to make sure we make changes at the core and thought through different scenarios where this approach might provide issues.
I am +1 on merging this for the current release.

Merged this in to HDF-3.0-maint. Not merging in to master. Will raise a new issue to track avro null fields in StreamlineEvent

Raised this https://github.com/hortonworks/streamline/issues/821


How can I have the same access in development mode and production mode?

Describe the feature
I want to do something after the service starts, such as executing a scheduled task, starting another service, or something else.
While developing the pattern, I can use the hooks: listen implementation in nuxt.config.ts, but it won't work after npm run build.
I looked at the nitro document, i can use presets to define entry (https://nitro.unjs.io/deploy/custom-presets), but it only work in production mode. This is annoying, I can do this easily in nuxt2, is there a solution to implement it in nuxt3?
Additional information

[ ] Would you be willing to help implement this feature?
[ ] Could this feature be implemented as a module?

Final checks

[X] Read the contribution guide.
[X] Check existing discussions and issues.

You can use a nitro plugin to do this.


Implement journald input plugin

Implement SDA journald TCP receiver in LA as a plugin.
Document helper service running basically “journalctl | netcat” as we did in CoreOS.
c.f. #42

With this https://github.com/sematext/logagent-js/commit/dd22fdd0862f3c5e210d39518d76b3efc987b42b we can use systemd.journald-upload.service to forward logs to logagent.

Released. Documentation https://sematext.com/docs/logagent/input-plugin-journald-upload/


Add a small documentation for ApprovalTests

Following #9607 : a documentation should be generated with a few examples, as done for Property-based Testing:

simple example
example with a parametrized junit test
example of use of combinations

@murdos I want to work on this issue. How can I proceed further?

@Harsh4902: you should read the https://github.com/approvals/ApprovalTests.java documentation in order to get familiar with it, and then add a markdown documentation to the approval-tests module, as done in the jwik module.

@murdos can you please review this PR?


Set default value of crs_validate_utf8_encoding TX variable

I'm working on some new features for util/crs-rules-check/rules-check.py script. Before I push them, we need to fix some errors.
This fix sets up the missing default value of a TX variable. There is no other case like this, all other TX variable has default value.

I'm not sure why the tests are failing…
Should we add a back-reference to the associated rule in crs-setup.conf? Like:
# Default check for UTF8 encoding validation (rule 900950 in crs-setup.conf)
SecRule &TX:crs_validate_utf8_encoding "@eq 0" \
    ⋮

Also, I noticed that the other back-references are inconsistent, some are missing, and at least one is incorrect. But that can be for another PR :smile:

I'm not sure why the tests are failing…

maybe because it's the new Docker images?
The CI system says:
5:18PM FTL ftw/run: can't connect to destination &{DestAddr:<IP_ADDRESS> Port:80 Protocol:http} - unexpected error found. Is your waf running?
	running 911100-1:
Error: Process completed with exit code 1.

and
collecting ... collected 3168 items

tests/regression/CRS_Tests.py::test_crs[953120.yaml -- 953120-0] ERROR
tests/regression/CRS_Tests.py::test_crs[953120.yaml -- 953120-1] ERROR
tests/regression/CRS_Tests.py::test_crs[953120.yaml -- 953120-2] ERROR
tests/regression/CRS_Tests.py::test_crs[953120.yaml -- 953120-3] ERROR
tests/regression/CRS_Tests.py::test_crs[953120.yaml -- 953120-4] ERROR
...

Should we add a back-reference to the associated rule in crs-setup.conf? Like:
# Default check for UTF8 encoding validation (rule 900950 in crs-setup.conf)
SecRule &TX:crs_validate_utf8_encoding "@eq 0" \
    ⋮

oh, that's a good idea!

Also, I noticed that the other back-references in REQUEST-901-INITIALIZATION.conf are inconsistent, some are missing, and at least one is incorrect. But that can be for another PR :smile:

:smile:
Sure. First I'd like to fix the logical errors like this, then we can fix the others.

Great work 🥳


fix(callout): updated directive prop def

An update to Angular 1.4.9 introduced a regression in ngOfficeUiFabric as explained in #139. This fixes the issue for the uif-callout directive.

Manually merged to branch dev


python3Packages.qiling: 1.4.1 -> 1.4.2

Description of changes
https://github.com/qilingframework/qiling/releases/tag/1.4.2
Fixes #164892

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
22.05 Release Notes (or backporting 21.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 164946 run on x86_64-linux 1

2 packages built:

python310Packages.qiling
python39Packages.qiling


chore: split Analysis.Convex.Cone.Basic

Splits Mathlib.Analysis.Convex.Cone.Basic, to move Riesz extension and Hahn-Banach out of the basic file about definitions.

Thanks! Definitely a worthwhile split.
bors merge


[BUG] Engine crash when loading assets

Describe the bug
The engine crashes while loading when assets has been loaded once, and then the folder Library is deleted.
To Reproduce
Steps to reproduce the behavior:

Delete all .meta files and the folder Library.
Start the engine and import all the Assets folder.
Close the engine.
Delete the folder Library.
Start the engine again.

Expected behavior
Not crashing.

After talking with @yalania , we concluded that this is a expected behaviour. But, this behaviour can be improved.


Streamline GetBlockSubsidy / fix coin cap comment

GetBlockSubsidy() can be streamlined a bit. Also fixing a comment about projected coin cap.

This is going to break after #951, do not merge this yet, I'll fix it after #951 (which is more important)

Like expected, needs a rebase

rebased, pls review :)

To me properly reviewing this change means making sure that these code changes never affect the actual value returned by the function (I assume no actual change to the computation is intended).  Unfortunately I don't have time for that right now since I have new sentinel functionality to implement.
I wonder if it's a good idea to be making a lot of refactoring changes now.  It seems to me there are still quite a few substantive questions about 12.1 that we need to figure out, maybe our time would be better spent on those questions at the moment.
I think some projects try to reserve time periods for refactoring/improving code vs. making core changes.  Maybe we could consider putting some of these changes off to say early in the 12.2 cycle ?

I would agree but I have most of refactoring postponed for like several month now waiting for the governance PRs and would like to clean that up, not redoing this again in 12.2. If you have no time to review each and every PR - that's fine, if no one have time - that's fine too, github doesn't charge fees for having PRs open :D

utACK 2339409

Oops, somehow missed the &&  :+1:
But I at least looked :-)


--no-header should stop terraform-docs from reading the main.tf file

Prerequisites
Put an x into the box(es) that apply:

[x] This pull request fixes a bug.
[ ] This pull request adds a feature.
[ ] This pull request enhances existing functionality.
[ ] This pull request introduces breaking change.

For more information, see the Contributing Guide.
Description
Our projects do not have that file. Even with --no-header, terraform-docs crashes. This PR stops the tool from reading a file when --no-header is set.
I'm not sure if the test is in the correct location. Let me know if that works for you
Checklist
Put an x into all boxes that apply:

[x] I have read the Contributing Guidelines.

Tests

[x] I have added tests to cover my changes.
[x] All tests pass when I run make test.

Documentation

[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.

Code Style

[x] My code follows the code style of this project.

Codecov Report

Merging #224 into master will increase coverage by 0.04%.
The diff coverage is 75.00%.

@@            Coverage Diff             @@
##           master     #224      +/-   ##
==========================================
+ Coverage   92.45%   92.49%   +0.04%
==========================================
  Files          20       20
  Lines         928      933       +5
==========================================
+ Hits          858      863       +5
  Misses         48       48
  Partials       22       22

Impacted Files
Coverage Δ

internal/module/module.go
91.66% <75.00%> (+0.20%)
:arrow_up:

Thanks for reviewing. I agree with your comments. Done!


In-place editing deletion with a form

I'm trying to remove an item from a table, but I need a form to allow the user to submit a reason for removing the item. Is there an easy way to combine the UX of rendering a form like the "edit" examples and then removing the row when they submit that form like the "delete" example?

I think quickest way to do this would be editing confirmation with some content_tags in https://github.com/Sology/smart_listing/blob/master/app/views/smart_listing/_action_delete.html.erb
After that when item is being removed form the list You can setup an event on "smart_listing:remove" and saving in Your db or wherever You like this reason for deleting an item. Does that sound ok?
BTW I'm so sorry for this delayed answer.

Thanks! That looks like a good solution.


fix(es6-lib): es6 library

add missing library in es6 to translate in es5
Proposed Changes

Potential Breaking Changes

Acceptance Criteria

[ ] The proposed changes are covered by unit tests
[ ] The potential breaking changes are clearly identified
[ ] README.md is adjusted to reflect the proposed changes (if relevant)

Codecov Report

Merging #1498 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #1498   +/-   ##
=======================================
  Coverage   98.82%   98.82%
=======================================
  Files         499      499
  Lines       10323    10323
  Branches     2385     2385
=======================================
  Hits        10202    10202
  Misses        121      121

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 861a70a...0a2c2fe. Read the comment docs.


Consider installing docfx from chocolatey

The pinned version of docfx (2.43.4) https://github.com/Azure/azure-sdk-for-net/blob/master/eng/pipelines/docindex.yml#L26
Is available on Chocolatey. Should we be installing from there? https://chocolatey.org/packages/docfx/2.43.4

Benefits of this approach would include:

Fewer steps to install (run choco install instead of the script)
Keeps docfx out of the working directory
Chocolatey is already installed on Windows agents (https://github.com/actions/virtual-environments/blob/master/images/win/Windows2019-Readme.md#chocolatey)

In the past, chocolatey hasn't been used in our scenarios because we were concerned about how there is no package verification for packages it downloads and installs.  We should investigate whether or not the packages are now signed and verified which come from chocolatey and then make a call on this and other packages we could pick up from there.
How are we getting docfx today?

From the chocalatey website:

We don't require cryptographically signing packages yet, that is a future enhancement

Ironically, looking at the docfx download on their website, none of the binaries in the zip file are signed, so maybe there's nothing more chocolatey could do here anyway.  We also checked the NUPKG for docfx and while the package is signed, the files it contains are not.
So our concerns about chocolatey are irrelevant when it comes to docfx because they don't sign anyway.

Issue opened on DocFX: https://github.com/dotnet/docfx/issues/5610
Bottom line, it won't make any difference for installing DocFX with chocolatey because DocFX isn't signed anyway.
Regardless, the concern is still there that chocolatey makes no commitment to verify signatures of the things it installs.  So we're concerned about opening up the precident of using chocolatey without any commitment from the tool on these security baselines.

@vhvb1989 -- We are avoiding chocolatey because it does not verify package signatures. I've opened an issue to move off of that for doxygen.


I have recently contributed by opening issues addressing a layer reuse issue related to go build targets and another related to the buildpack respecting GOFLAGS. I have also submitted a PR simplifying go build flags.

This has been open far too long. @ForestEckhardt can we get a vote from you?

Congrats @fg-j. I will add you to the contribution group in short order!


Some Missing ADFs for examples and example variants

Hi @alpine9000
first I want to thank you for this collection of amiga examples. It does cover a lot of hardware programming aspects. Yours example collection is therefore currently used as test programs for a new amiga emulator called vAmiga. That is an Amiga-Emulator made specific for macOS. We want to get it pass every of yours example. Fortunately most of yours examples are already compiled/assemled and packed into an ADF. So we can visually compare with SAE. But for certain tests there is no ADF at all or there are variants and only one variant is in the ADF.
Here is an overview about what of your examples is already mastered by vAmiga
dirkwhoffmann/vAmiga#35
In order to help dirkwhoffmann I tried to get some of the missing ADFs build. But I run into difficulties. Especially the imagecon program  gave me hard times, lots of makefile errors.
Maybe you are still around and maybe vAmiga does made you curious and also interests you and you maybe also don't mind to post the imagecon tool as a binary for macOS under yours github release section ? Or you could just drag it into this issue ?
Especially we do like to get your example#10 the blitspeed example running. With it we can tune the blitter to get the exact speeds.
Here is the issue to get it run: dirkwhoffmann/vAmiga#38
Do you maybe still have the ADFs with the Variants 1 - 5 Bitplanes somewhere on your hardrive?
Again thank you for this great repository ! Amiga rulez. 😜

I checked out a fresh version of the repo and got it building. (note: the tools build fine using clang on OSX, so no need to install gcc).
I also included most of the missing ADF files as well as the ADFs for each of the 2,4,8,16 and 32 color modes in the blit speed example.
I wouldnt bother with the 028.bs example as that was just the beginnings of the Blocky Skies game.

Hi alpine,
again thank you very much that you help us out 😃.
all the best
❤

Many thanks from my side, too!!!


run changie workflow only if go files updated. doc,test only updates …

Issues
Have changie workflow check for changie logs only when actual *.go files are updated. Github workflow, docs, and testdata/test only updates do not need changie logs. Dependabot can't merge Dependabot PRs with changie workflows failing.

Changelog

[X] List your changes here
[ ] Make a changie entry, N/A github workflow update

Tophatting

nvm :)


Fix long filenames

Improved fix for #8 (file name too long).  This PR moves the trim code to _get_non_existant_name so the trimming happens during the increment loop.
So, if a file name is too long in the form: REALLY_LONG_NAME.ext
and is too long by 4 characters, it'll be trimmed to: REALLY_LONG_.ext (dropping the 4 characters in NAME).
If REALL_LONG_.ext exists, it'll be incremented to: REALLY_L (1).ext (dropping characters so that the entire name fits within the max file name length.
I think this is the best that can be done to preserve the ability to generate safe names but also trim names that are too long.  An alternate approach would be to catch the OSError and warn the user the note name is too long and cannot be exported then continue on with other notes.  I think most users would prefer the note to be exported with a modified name though and this is likely an edge case affecting very few notes.  (In my library, I had 2 notes out of 6,422 that had names which were too long.)
All tests pass and flake8 is clean for the changes.

Thank you VERY much for your help! I really appreciate your having taken time to fix this issue and contribute to my project.
I will refactor it a bit before release if you don't mind :)

Glad I could help and THANK YOU for this very useful tool!


Allow non-unique mobile phones.

This issue is similar to #645.
In our existing solution the customers have several accounts with same certificate and with the same phone number. In order to migrate our solution of identity model to MBR, I need the configuration support for this.
I would create MobilePhoneIsUnique configuration property for this.

Sorry, I don't have the bandwidth to try to make this change.

Given that I don't foresee making this change, I'll close this issue. Thanks.


[English]: Golden Signals

Details (optional)
Optional details

Can I work on it ?

Here you go, @PadminiRai. Please make sure to read the style guide before getting started and let us know if you have any questions.

@remote007 how's it going? Is there anything we can do to help this move forward?

Hi @JasonMorgan , I edited the content it's in the draft format rn. I fell ill and am taking some rest. Will create asap. Thanks for asking.

Apologies for the delay, but this term no longer meets the requirements to be included in the cloud native glossary. I will go ahead and close this issue.


fix: Disables smartfin.org link

Recommend disabling this link until we can confirm smartfin.org is stable and under control.


Signature Verification Failed

Error Code: Generic_Verify_Fail
You might be under attack, there might ba a network problem, or you be missing a recently added Tor Browser verification key.
Click start to refresh the keyring and try again. if the message persists report the above code here:
https://github.com/micahflee/torbrowser-launcher/issues

This happens to me on Ubuntu 18.10 with the Tor Browser Launcher via apt.

See https://github.com/micahflee/torbrowser-launcher/issues/382#issuecomment-451603188


Unfortunately, an unexpected error occurred, and Bundler cannot continue.

I ran the command /usr/local/bin/bundle install --deployment --without development test postgres aws.
I expected Bundler to install success.
Instead, an error happened.The error message is:
Unfortunately, an unexpected error occurred, and Bundler cannot continue.
Error details
Errno::ENOENT: No such file or directory - git
  /usr/local/lib/ruby/gems/2.0.0/gems/bundler-1.12.5/lib/bundler/source/git/git_proxy.rb:149:in ``'
  /usr/local/lib/ruby/gems/2.0.0/gems/bundler-1.12.5/lib/bundler/source/git/git_proxy.rb:149:in `block in git'
  /usr/local/lib/ruby/gems/2.0.0/gems/bundler-1.12.5/lib/bundler/shared_helpers.rb:72:in `call'
  /usr/local/lib/ruby/gems/2.0.0/gems/bundler-1.12.5/lib/bundler/shared_helpers.rb:72:in `with_clean_git_env'
  /usr/local/lib/ruby/gems/2.0.0/gems/bundler-1.12.5/lib/bundler/source/git/git_proxy.rb:149:in `git'
  /usr/local/lib/ruby/gems/2.0.0/gems/bundler-1.12.5/lib/bundler/source/git/git_proxy.rb:156:in `block in has_revision_cached?'
  /usr/local/lib/ruby/gems/2.0.0/gems/bundler-1.12.5/lib/bundler/shared_helpers.rb:54:in `chdir'
  /usr/local/lib/ruby/gems/2.0.0/gems/bundler-1.12.5/lib/bundler/shared_helpers.rb:54:in `block in chdir'
  /usr/local/lib/ruby/2.0.0/monitor.rb:211:in `mon_synchronize'
  /usr/local/lib/ruby/gems/2.0.0/gems/bundler-1.12.5/lib/bundler/shared_helpers.rb:53:in `chdir'
  /usr/local/lib/ruby/gems/2.0.0/gems/bundler-1.12.5/lib/bundler/source/git/git_proxy.rb:204:in `in_path'
  /usr/local/lib/ruby/gems/2.0.0/gems/bundler-1.12.5/lib/bundler/source/git/git_proxy.rb:156:in `has_revision_cached?'
  /usr/local/lib/ruby/gems/2.0.0/gems/bundler-1.12.5/lib/bundler/source/git/git_proxy.rb:88:in `checkout'
  /usr/local/lib/ruby/gems/2.0.0/gems/bundler-1.12.5/lib/bundler/source/git.rb:154:in `specs'
  /usr/local/lib/ruby/gems/2.0.0/gems/bundler-1.12.5/lib/bundler/lazy_specification.rb:56:in `__materialize__'
  /usr/local/lib/ruby/gems/2.0.0/gems/bundler-1.12.5/lib/bundler/spec_set.rb:91:in `block in materialize'
  /usr/local/lib/ruby/gems/2.0.0/gems/bundler-1.12.5/lib/bundler/spec_set.rb:88:in `map!'
  /usr/local/lib/ruby/gems/2.0.0/gems/bundler-1.12.5/lib/bundler/spec_set.rb:88:in `materialize'
  /usr/local/lib/ruby/gems/2.0.0/gems/bundler-1.12.5/lib/bundler/definition.rb:140:in `specs'
  /usr/local/lib/ruby/gems/2.0.0/gems/bundler-1.12.5/lib/bundler/definition.rb:129:in `resolve_remotely!'
  /usr/local/lib/ruby/gems/2.0.0/gems/bundler-1.12.5/lib/bundler/installer.rb:195:in `resolve_if_need'
  /usr/local/lib/ruby/gems/2.0.0/gems/bundler-1.12.5/lib/bundler/installer.rb:70:in `run'
  /usr/local/lib/ruby/gems/2.0.0/gems/bundler-1.12.5/lib/bundler/installer.rb:22:in `install'
  /usr/local/lib/ruby/gems/2.0.0/gems/bundler-1.12.5/lib/bundler/cli/install.rb:102:in `run'
  /usr/local/lib/ruby/gems/2.0.0/gems/bundler-1.12.5/lib/bundler/cli.rb:175:in `install'
  /usr/local/lib/ruby/gems/2.0.0/gems/bundler-1.12.5/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
  /usr/local/lib/ruby/gems/2.0.0/gems/bundler-1.12.5/lib/bundler/vendor/thor/lib/thor/invocation.rb:126:in `invoke_command'
  /usr/local/lib/ruby/gems/2.0.0/gems/bundler-1.12.5/lib/bundler/vendor/thor/lib/thor.rb:359:in `dispatch'
  /usr/local/lib/ruby/gems/2.0.0/gems/bundler-1.12.5/lib/bundler/vendor/thor/lib/thor/base.rb:440:in `start'
  /usr/local/lib/ruby/gems/2.0.0/gems/bundler-1.12.5/lib/bundler/cli.rb:11:in `start'
  /usr/local/lib/ruby/gems/2.0.0/gems/bundler-1.12.5/exe/bundle:27:in `block in <top (required)>'
  /usr/local/lib/ruby/gems/2.0.0/gems/bundler-1.12.5/lib/bundler/friendly_errors.rb:98:in `with_friendly_errors'
  /usr/local/lib/ruby/gems/2.0.0/gems/bundler-1.12.5/exe/bundle:19:in `<top (required)>'
  /usr/local/bin/bundle:23:in `load'
  /usr/local/bin/bundle:23:in `<main>'

Environment
Bundler   1.12.5
Rubygems  2.0.14
Ruby      2.0.0p353 (2013-11-22 revision 43784) [i686-linux]
GEM_HOME  /home/git/gitlab/vendor/bundle/ruby/2.0.0
GEM_PATH
Git       not installed
  Bundler settings

frozen
Set for your local app (/home/git/gitlab/.bundle/config): "1"
path
Set for your local app (/home/git/gitlab/.bundle/config): "vendor/bundle"
without
Set for your local app (/home/git/gitlab/.bundle/config): "development:test:postgres:aws:puma"
disable_shared_gems
Set for your local app (/home/git/gitlab/.bundle/config): true
orig_path
Set via BUNDLE_ORIG_PATH: "/usr/bin:/bin"

Can you please share the output of bundle env and is git installed + available to the user?

@chuganghong any update on this?

Closing due to lack of a response.


feat: log preservation in LFMerge or app container

From https://trello.com/c/zLfOjZid/76-log-preservation-for-lfmerge-need-to-discuss-how-to-do-it-how-much-to-keep
We need a way to preserve log files from LFMerge across kubernetes container restarts, since these logs can help to diagnose user problems that have happened in the past.

What is the "docker way" of keeping around container logs?  I see at least two ways of retaining logs:

use some acceptable log retention facility in k8s / docker for the container log
ensure the lfmerge logs are written to the filesystem and have log retention/rotation policy defined and then create a volume for that log directory

@longrunningprocess what do you think we should do for this?

What is the "docker way" of keeping around container logs? I see at least two ways of retaining logs:

use some acceptable log retention facility in k8s / docker for the container log
ensure the lfmerge logs are written to the filesystem and have log retention/rotation policy defined and then create a volume for that log directory

@longrunningprocess what do you think we should do for this?

Those a both good options.  I'm in conversation with LTOps about the best way forward since we're not the only ones who would need this... we need to find a general solution and best practice for all teams coming onto the k8s platform.

OK thank you for your investigation and for reporting back here when you know more.  I am also adding the research label.

walked through a bunch of the Grafana features today in staging and the log viewer is really nice.  LTOps wants to give it a little more time to make some decisions about retention constraints, e.g., how much space and/or duration.

LTOps is still waiting to hear from affected teams on whether they like the Grafana/Loki solution.  They also need to decide retention periods which they will do at one of their next meetings.

LTOps is still testing some different configurations and it not ready to put it into prod just yet.

update from LTOps:

I hope to install in prod this week, although we didn't discuss retention this week. (always too many new requests).  I have the retention in Dallas set to 1 week, after confirming at 2 days.  So that appears to be working, but I'll know in a day or so.  So I should be able to start in AWS with a week or so, until we discuss again.  It is adjustable.

node upgrades occurring right now, will test things out once LTOps is finished.


Android: Updated toolchain file and dependencies

This PR supercedes the branch feature/android_toolchain.
Replaced the toolchain file with a new version based on zuhowei's fork, which supports the latest NDK versions as well as 64 bit targets.
Removed the STL dependency from sfml-activity rather than relying on some implementation implicitly linked by default.
Deleted project.properties, which wasn't supposed to be part of the repository code.
Updated CMake files to properly support Nvidia's Nsight Tegra Visual Studio Edition as an alternative to using the toolchain file (you might still have to update the STL implementation as that's not yet set through CMake).

Some Considerations:

getLibraryName() is used only once. As such we could skip the whole string handling and merge this function into loadLibrary().

Updated the branch. It's now possible to build SFML without utilizing the toolchain file if you're using Visual Studio and Tegra Nsight VS Edition:
cmake -G "Visual Studio 12 2013" -DCMAKE_SYSTEM_NAME=Android -DANDROID_NATIVE_API_LEVEL=9 -DANDROID_ABI=armeabi path/to/source

getLibraryName() is used only once. As such we could skip the whole string handling and merge this function into loadLibrary().

What are the chances that it will be used multiple times at a later point?
Is the "string handling" an issue or can it be an issue?
Is it a "public" function?
Let me know if you intend to keep working on this PR or rather when it's ready to be merged.

What are the chances that it will be used multiple times at a later point?

It's private (i.e. even static) and very unlikely to be reused. I can hardly think of any reason to determine the current library name. This is a bit like determining/using argv[0] in a desktop program, but even less useful since you can't use it for recursive calls. Only case I could imagine using this would be to more or less DRM a shared library so it can only be used by a specific other shared library, but at the same time the common user will never even see these names.

I'm going to squash and merge this today unless I hear anything which would prevent this. I'll leave getLibraryName() the way it is right now.

You can squash it, I'll then merge it. :wink:

I wouldn't add the examples/android/.gitignore file. It doesn't really serve any purpose.

Well, it keeps git's status clean when building in-tree (which you typically do), but then again it's not necessary... hm...

It's never really recommended to build in tree and if you want the status to be clean we'd have to add a ton of additional git ignores, which is just ugly and doesn't really help.

when building in-tree (which you typically do)

Nooooooooooooooooo. Nobody does that. Not in a source tree that you're constantly working in, at least. Please make you a favor, and create your build directory out of the source tree :stuck_out_tongue:

I obviously do. :D But okay, point taken, removing it.

There you go, squashed, rebased and ready. :)

This PR has been added to my merge list, meaning it will be merged soon, unless someone raises any concerns.

There are tabs all over the place in CMake files, uppercase functions ("SET" instead of "set"), and what's the thing with Nsight?

Gna, there shouldn't be tabs~ Nsight is a plugin for Visual Studio providing NDK integration (compiling/linking, debugging, etc.).

Okay, updated the branch - I didn't touch the new toolchain file though, since I'd like to keep it 1:1 to the original (linked above).

If it's an "external dependency" you should exlude it from the .gitattributes (and .editorconfig).

Added. EditorConfig doesn't support exclusions as far as I know, but should be fine (since the shouldn't be edited anyway; similar to stb stuff).

Nsight is a plugin for Visual Studio providing NDK integration

Ok but my question was more: why does it need special stuff (find_host_package) in addition to the regular toolchain?

The latest change on master has created a conflict with this branch. Make sure to rebase and resolve.

Ok but my question was more: why does it need special stuff (find_host_package) in addition to the regular toolchain?
Simply because it doesn't use the toolchain at all. It's a "special" target built into CMake, which is triggered by setting CMAKE_SYSTEM_NAME to Android while creating a MSVC project.

Going to add a short note together with the rebase/merge.

Done.

This PR has been added to my merge list, meaning it will be merged soon, unless someone raises any concerns.

Building this for testing I noticed that you added (at least) three options, but didn't mark them with the SFML_ prefix.
Also is it possible to only "display"/set them when one is actually trying to build for Android, otherwise it can be a bit irritating.

They shouldn't appear when you're not building for Android. Do they still appear for you? If so, that's a bug. Unfortunately they have to be set before the toolchain is invoked (using CMake's project() command), so there's no way in using your usual CMake variables for branching.
We could wrap those with a SFML_ prefix, but behind the scenes we'd still have to set the non-prefixed variables. Those are used by the toolchain file and CMake.

I've updated the branch. Try again, they should be gone now. This still leaves the name thing though.

Ah I guess the naming is okay if it's used by the toolchain. I was mostly confused to see the options on my standard MinGW build.
Seems to work fine now. :+1:


Calling a Repository method without checking permissions

By default calling the repository methods require the permission of logged in user. Which is fine while calling the repository from endpoints. But withing server-side code I want to ignore the permission checks.
new MyRepository().Retrieve(connection, request);
Is there any way to ignore the permission check for every repository call withing my server-side code?
I know connection.ById(id) does not require permission, but it does not load master-detail fields.
Something like the following will be great.
new MyRepository(ignorePermissionCheck: false).Retrieve(connection, request);

You may use TransientGrantingPermissionService, or Impersonation system

Thank you.
I have found detail about TransientGrantingPermissionService in this issue https://github.com/volkanceylan/Serenity/issues/1383


improve clarity through updated docstrings and comment

This should clear up any confusion about what's going on in this method.
Fixes #360

I'm going to add one more thing regarding the log variational distribution.

Codecov Report

Merging #373 into master will increase coverage by 0.06%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master     #373      +/-   ##
==========================================
+ Coverage   88.77%   88.84%   +0.06%
==========================================
  Files          32       32
  Lines        3048     3048
==========================================
+ Hits         2706     2708       +2
+ Misses        342      340       -2

Impacted Files
Coverage Δ

scvi/inference/posterior.py
92.55% <ø> (ø)
:arrow_up:

scvi/inference/autotune.py
69.67% <0%> (+0.5%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9c4fde4...26e22eb. Read the comment docs.

@gabmis I reverted the Black formatting. Good point.


User creation architecture

The core goal of the membership epic is for users to be able to provision themselves in Sierra. We have a few options for how this is to be achieved technically:

Naïve solution: add a user creation endpoint to the identity API (TypeScript, this repo in /packages/apps/identity-api) which calls the Sierra API to create a user. This is appealingly simple, but it has a number of disadvantages:

It circumvents Auth0 security mechanisms and conveniences.
How do we log the user in post signup?
By bypassing Auth0, we couple ourselves (more) to Sierra, making it harder to consider alternative identity sources.
We have to maintain a signup experience entirely separately to the login experience, which is hosted within Auth0 (universal login).

New universal login + database script: this is the minimal change within our current architecture; we would use the registration form as provided by Auth0 and we would implement the Create user custom database script to provision users in Sierra.
There is one key disadvantage here: the registration form only supports username, email and password fields. In order to populate other mandatory fields (namely, first and last names), we would have to take a progressive profiling approach whereby a custom action would direct the user to input further fields immediately after signup.
It's worth noting that we will likely have to take a progressive profiling approach in later phases of the membership project (for collecting user addresses etc).

Classic universal login + database script: as above, but reverting to the classic universal login experience, which is much more customisable than the new universal login despite some disadvantages.
This would require migrating our existing signup and password reset pages to classic universal login, too, but on the other hand it would enable us to customise those too.

Bits and pieces:

Additional signup fields in Classic universal login with Lock: https://auth0.com/docs/libraries/lock/lock-configuration#additionalsignupfields-array-
The Next.js Auth0 SDK does not support non-universal login: https://github.com/auth0/nextjs-auth0/issues/141

Slack conversations I've found (interesting context):

https://wellcome.slack.com/archives/C0198JT4GTG/p1619170523187400
https://wellcome.slack.com/archives/C0198JT4GTG/p1615474324049800?thread_ts=1615474105.049300&cid=C0198JT4GTG

I've done a quick spike on option 2, where we continue to work with universal login.
It looks like we can redirect to another form at the point of signing up with universal login. I’ve tried this is a sandbox developer account and used an online sample of a consent form (in lieu of a full sign up page with consent form). It looks like, despite universal login meaning we need to get a user’s email and password first, we can still redirect to a sign up form (as per the original user flows) as part of the membership process. Note: the verify your email address email is sent as soon as you fill out email and password and hit submit).
Mel, i’m tired, explain that again:
We can continue to use universal login for membership work by just adding one step to the start of the user flow here, email and password first.
(To see video of the sign up process from sandbox account click here)

I have tried to update the user flow to allow for this extra step, you can check the miro board here
I will now try to go through tickets we have to add these considerations where relevant. We will probably need a few more tickets relating to this work too.

We decided to keep this ticket 'in progress' so that it is a reminder of our 3 possible approaches, even though we've gone with option 2. It'll also remind me to update Carla when she's back.


chore(main): release 3.1.2

:warning: IMPORTANT: Release will be prefixed by 'v' (as in v1.2.3). Therefore it is crucial that the pre-commit hook is tweaked in this PR to point to the correct release tag, as this does not work today. Also, the __about__.py file must be manually bumped.
3.1.2 (2024-11-16)
Bug Fixes

parser: catch prefix "project." (not "project") (#233) (b15d8df)

Documentation

format README (#232) (58bc805)

This PR was generated with Release Please. See documentation.

:robot: Created releases:

v3.1.2
:sunflower:


Define images with URL instead of data URIs

Hi, I'm new to Quill and ReactQuill, but integration works fine up to now. We have started to create our own handlers, especially to define images with URL instead of data URIs. We already found a solution how to do it in vanilla Quill, but got stuck when transferring the solution to ReactQuill.
The solution in vanilla Quill looks like the following:
var toolbarOptions = [
            ['bold', 'italic', 'underline', 'strike'],        // toggled buttons
            ['blockquote', 'code-block'],

            [{ 'header': 1 }, { 'header': 2 }],               // custom button values
            [{ 'list': 'ordered' }, { 'list': 'bullet' }],
            [{ 'script': 'sub' }, { 'script': 'super' }],      // superscript/subscript
            [{ 'indent': '-1' }, { 'indent': '+1' }],          // outdent/indent
            [{ 'direction': 'rtl' }],                         // text direction

            [{ 'size': ['small', false, 'large', 'huge'] }],  // custom dropdown
            [{ 'header': [1, 2, 3, 4, 5, 6, false] }],

            [{ 'color': [] }, { 'background': [] }],          // dropdown with defaults from theme
            [{ 'font': [] }],
            [{ 'align': [] }],

            ['clean'],                                         // remove formatting button
            ['image']
        ];

        var quill = new Quill('#editor', {
            theme: 'snow',
            modules: {
                toolbar: {
                    container: toolbarOptions,
                    handlers: {
                        image: imageHandler
                    }
                }
            },
        });

        function imageHandler() {
            var range = this.quill.getSelection();
            var value = prompt('What is the image URL');
            if(value){
                this.quill.insertEmbed(range.index, 'image', value, Quill.sources.USER);
            }
        }

Transfer to ReactQuill is no problem for instantiation of the editor and configuration of the modules. But we cannot find any description of how to migrate the handler function. Maybe you can give us some hint, how the imageHandler function should look with ReactQuill or a link to a proposed solution.
Andreas

Same here!

Instead of using this.quill, you can use a ref to access the Quill instance created by ReactQuill

Thanks for your prompt reply. I've already checked this option in the Readme, but got a little bit confused by that. There is a warning "please avoid from imperatively manipulating the instance, to avoid getting ReactQuill and Quill out-of-sync", so I wasn't sure about the correct and save solution.
We are using Hooks and therefor ref.current to get the reference. Is this save to use?
And by the way, where can I reference to the Quill.sources.USER definition? When I import the Quill namespace from react-quill, there is no sources definition.
That's why I was asking for an example, just to be on the save side. Many thanks for your support.

I think it should be safe to reference the current value of the ref as long as you don't create closures over a stale reference.

And by the way, where can I reference to the Quill.sources.USER definition? When I import the Quill namespace from react-quill, there is no sources definition.

I'm not sure why the export is not typed, are you sure that it exists on the current Quill 1.3 and is not part of the beta 2.0 branch? In any case it's actually just a string: "user"
https://github.com/DefinitelyTyped/DefinitelyTyped/blob/master/types/quill/index.d.ts#L23

Thank you.

This should work with React v.17, Next.js vs.12 and React Quill v.2:
import React, { useMemo, useRef } from "react"
import dynamic from "next/dynamic"
import "react-quill/dist/quill.snow.css"

// Import ReactQuill only on the client side.
const ReactQuill = dynamic(
  async () => {
    const ReactQuill = (await import("react-quill")).default

    return ({
      forwardedRef,
      ...rest
    }) => <ReactQuill ref={forwardedRef} {...rest} />
  },
  {
    ssr: false,
  },
)

export const Editor = ({ value, onValueChange }) => {
  const quillRef = useRef(null)

  // @see https://stackoverflow.com/a/64146927/2270233
  const modules = useMemo(
    () => ({
      toolbar: {
        container: [
          [{ header: [1, 2, 3, false] }],
          ["image"],
        ],
        handlers: {
          image: imageHandler,
        },
      },
    }),
    [],
  )

  function imageHandler() {
    if (!quillRef.current) return

    const editor = quillRef.current.getEditor()
    const range = editor.getSelection()
    const value = prompt("Please enter the image URL")

    if (value && range) {
      editor.insertEmbed(range.index, "image", value, "user")
    }
  }

  return (
      <ReactQuill
        forwardedRef={quillRef}
        theme="snow"
        defaultValue={value}
        onChange={onValueChange}
        modules={modules}
      />
  )
}


Make cursor a pointer for down_panel tab links.

Add cursor: pointer style to tablist style settings in down_panel/style.js.

Looks good to me, Thanks!


Allow BcolzMinuteBarWriter to append to most recent day

Minutely data can now be appended to bcolz files even when
minutes in the same day have already been written. For example,
previously attempting to write data for the minute 2016-05-11 16:30
would raise an exception if any OHLCV data for 2016-05-11 had been
written to the same file.
Trying to overwrite existing minutes still raises a
BcolzMinuteOverlappingData exception.
Note that previously all sids' bcolz files ended at the same time.
This is no longer necessarily the case. The last record in each
sid's bcolz file now corresponds to the latest minute for which
OHLCV data is provided to the writer.

Coverage decreased (-0.01%) to 81.461% when pulling 9eb8f4430ee880dc300c2914b8cae02a4e65aeb9 on modify-minutes-writer into 47da15592c5f5755bc603904da79bc3dadab3a99 on master.

We should add some tests in test_minute_bar_writer exercising the new behavior, e.g. a test that writes up to some value before the market close, and then successfully writes another value in that same day.

Coverage decreased (-0.008%) to 81.463% when pulling b1c96532f4eadab5233888e7b9c2eb68279bd494 on modify-minutes-writer into 47da15592c5f5755bc603904da79bc3dadab3a99 on master.

Coverage decreased (-0.04%) to 81.429% when pulling 1829981a79396b275c01225a39386807f9ddeae0 on modify-minutes-writer into 47da15592c5f5755bc603904da79bc3dadab3a99 on master.

Coverage decreased (-0.01%) to 81.483% when pulling 8217cdb1bd6c6555d28f29c124110220c9787ba3 on modify-minutes-writer into 784d5f4a16e4c0dadbab5af4c4ea0769de37fc6a on master.

Coverage increased (+0.009%) to 81.506% when pulling 8217cdb1bd6c6555d28f29c124110220c9787ba3 on modify-minutes-writer into 784d5f4a16e4c0dadbab5af4c4ea0769de37fc6a on master.


BUG 831833

What does the bug make reference to?
https://github.com/Microsoft/TypeScript/blob/9a89147587c06ba51181ff2ee5ade69a98b171ea/tests/cases/conformance/types/typeRelationships/subtypesAndSuperTypes/enumIsNotASubtypeOfAnythingButNumber.ts#L123

Old internal bug report. Contents reproduced:

enum E { A }

interface I19 {
    [x: string]: Object;
    foo: E; // error, failed in <IP_ADDRESS>
}

interface I20 {
    [x: string]: {};
    foo: E; // error, worked in <IP_ADDRESS>
}

var x: Object;
x = E.A; // error, worked in <IP_ADDRESS>
var y: {};
y = E.A;

function f16<T extends E>(x: T) {
   var r13 = true ? E.A : x; // error
   var r13 = true ? x : E.A; // error
}

Expected: no errors
Actual: enum isn't a subtype or assignable to Object or {}

I'm confused what should be the behavior?

    interface I19 {
        [x: string]: Object;
        foo: E; // BUG 831833
        ~~~~~~~
!!! error TS2411: Property 'foo' of type 'E' is not assignable to string index type 'Object'.
    }

    interface I20 {
        [x: string]: {};
        foo: E; // BUG 831833
        ~~~~~~~
!!! error TS2411: Property 'foo' of type 'E' is not assignable to string index type '{}'.
    }

Is that correct?

I believe that bug (and thus that test) refer to out-of-date behavior. All types are assignable to {} and Object, so those shouldn't be an error.


Guaranteed Scheduling For Critical Add-On Pods task does not link to details of built in PriorityClasses

Folks - on this page, regarding this language: "To mark a Pod as critical, set priorityClassName for that Pod to system-cluster-critical or system-node-critical. system-node-critical is the highest available priority, even higher than system-cluster-critical." This immediately brings to mind the question: "what do you mean by even higher, i.e. what is the effect?" A little searching yields this documentation which answers the question nicely. Consider adding some clarification on the effect of node vs cluster priority in your docs.

/retitle Guaranteed Scheduling For Critical Add-On Pods task does not link to details of built in PriorityClasses
We could and should update https://kubernetes.io/docs/concepts/scheduling-eviction/pod-priority-preemption/#priorityclass to incorporate details about the built-in priority classes.
It is OK to copy text verbatim from https://kubernetes.io/blog/2023/01/12/protect-mission-critical-pods-priorityclass/ and you can also use your own wording.
/language en
/help
/triage accepted
/priority backlog

We should also update https://kubernetes.io/docs/tasks/administer-cluster/guaranteed-scheduling-critical-addon-pods/ to link to https://kubernetes.io/docs/concepts/scheduling-eviction/pod-priority-preemption/#priorityclass once we update it.

I am working on this issue.
/assign

@rajeshdeshpande02 Are you still working on this issue

Unassigning @rajeshdeshpande02 as didn't receive any updates. Please assign if you get back to this or anyone can also assign to this issue
/unassign @rajeshdeshpande02


feat(config): add diff option

Description
Closes #2401

Please don't delete this checklist! Before submitting the PR, please make sure you do the following:

[x] It's really useful if your PR references an issue where it is discussed ahead of time. If the feature is substantial or introduces breaking changes without a discussion, PR might be closed.
[x] Ideally, include a test that fails without this PR but passes with it.
[x] Please, don't make changes to pnpm-lock.yaml unless you introduce a new test example.

Tests

[x] Run the tests with pnpm test:ci.

Documentation

[x] If you introduce new functionality, document it. You can run documentation with pnpm run docs command.

Changesets

[x] Changes in changelog are generated from PR name. Please, make sure that it explains your changes in an understandable manner. Please, prefix changeset messages with feat:, fix:, perf:, docs:, or chore:.

I don't see this option working in the browser?

I wasn't aware of that, just added browser mode support.


Component Library storybook examples should not use
 tags to generate margins

Problem
The Card storybook example relies on using the
 tag to create margins between the text blocks and buttons.
Although this goes against accessibility guidelines, this is seen as the recommended way to use the component, as it is what is documented.
Solution
Ensure that the
 tag is only used for line breaks within blocks of text, and use either the

 component, and/or CSS margin otherwise.

That's correct, thanks for bringing this to our attention!


Support multiple input indices for document level monitors

Signed-off-by: Angie Zhang<EMAIL_ADDRESS>Issue #, if available: #474
Description of changes: Support multiple indices as input for Document level monitors
CheckList:
[ ] Commits are signed per the DCO using --signoff
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Codecov Report

Merging #481 (77a7f80) into main (6d3fb50) will decrease coverage by 0.01%.
The diff coverage is 80.00%.

@@             Coverage Diff              @@
##               main     #481      +/-   ##
============================================
- Coverage     76.89%   76.87%   -0.02%
  Complexity      176      176
============================================
  Files           166      166
  Lines          8370     8372       +2
  Branches       1232     1232
============================================
  Hits           6436     6436
- Misses         1329     1332       +3
+ Partials        605      604       -1

Impacted Files
Coverage Δ

.../opensearch/alerting/DocumentLevelMonitorRunner.kt
82.72% <80.00%> (+0.86%)
:arrow_up:

...ing/model/destination/DestinationContextFactory.kt
75.00% <0.00%> (-3.58%)
:arrow_down:

...destinationmigration/DestinationConversionUtils.kt
68.88% <0.00%> (-2.23%)
:arrow_down:

...rg/opensearch/alerting/BucketLevelMonitorRunner.kt
75.38% <0.00%> (-1.54%)
:arrow_down:

...rch/alerting/opensearchapi/OpenSearchExtensions.kt
54.05% <0.00%> (-1.36%)
:arrow_down:

...nationmigration/DestinationMigrationCoordinator.kt
68.08% <0.00%> (+4.25%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6d3fb50...77a7f80. Read the comment docs.

Closing this PR for now as there has been a lot of changes since and will have to redo some of the work and get it done in conjunction with alerting dashboards change to support this.


fix(test): test

:tada: This PR is included in version 1.0.4-rc.1 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Webpack 5: deterministic output, better build performance and caching

🚀 Feature
We should upgrade to Webpack 5 for various reasons
(just creating the issue to reference it in blog post).
Related to this issue with undeterministic static output: https://github.com/facebook/docusaurus/issues/3383

I'd be happy to take this.

I'd be happy to take this.

It's something I wanted to work on just after the i18n release, so tell me on discord if you give it a try :)
I think we can split the work in multiple parts, like the minimum upgrade, and another PR for perf/caching improvements

It's something I wanted to work on just after the i18n release, so tell me on discord if you give it a try :)
I think we can split the work in multiple parts, like the minimum upgrade, and another PR for perf/caching improvements

If It's useful for anyone, I reviewed the dependencies that are related to webpack:

library
actual version
compatible?

"@endiliey/static-site-generator-webpack-plugin
^4.0.0",
Unknown, probably works

"@svgr/webpack
^5.5.0",
Unknown, probably works

"copy-webpack-plugin
^6.3.0",
^7.0.0 ok

"html-webpack-plugin
^4.5.0",
^5@beta ok

"optimize-css-assets-webpack-plugin
^5.0.4",
For webpack v5 or above please use [css-minimizer-webpack-plugin](https://github.com/webpack-contrib/css-minimizer-webpack-plugin) instead.

"pnp-webpack-plugin
^1.6.4",
Unknown, probably works

"terser-webpack-plugin
^4.1.0",
^5.1.1 ok

"webpack
^4.44.1",
^5.x.x

"webpack-bundle-analyzer
^3.6.1",
^4.3.0  ok

"webpack-dev-server
^3.11.0",
not required since v5

"webpack-merge
^4.2.2",
Unknown, probably works

webpackbar
^5.0.0-3
has changes in one commit for fixing some bugs with webpack 5, probably works with actual version

If It's useful for anyone, I reviewed the dependencies that are related to webpack:

library
actual version
compatible?

"@endiliey/static-site-generator-webpack-plugin
^4.0.0",
Unknown, probably works

"@svgr/webpack
^5.5.0",
Unknown, probably works

"copy-webpack-plugin
^6.3.0",
^7.0.0 ok

"html-webpack-plugin
^4.5.0",
^5@beta ok

"optimize-css-assets-webpack-plugin
^5.0.4",
For webpack v5 or above please use [css-minimizer-webpack-plugin](https://github.com/webpack-contrib/css-minimizer-webpack-plugin) instead.

"pnp-webpack-plugin
^1.6.4",
Unknown, probably works

"terser-webpack-plugin
^4.1.0",
^5.1.1 ok

"webpack
^4.44.1",
^5.x.x

"webpack-bundle-analyzer
^3.6.1",
^4.3.0  ok

"webpack-dev-server
^3.11.0",
not required since v5

"webpack-merge
^4.2.2",
Unknown, probably works

webpackbar
^5.0.0-3
has changes in one commit for fixing some bugs with webpack 5, probably works with actual version

webpack-merge latest changes are very breaking and I'm pretty sure it will work on WP5.  It should be upgraded sometime in the future but this will be very breaking for plugins, so we should leave it for now.
I'll start bumping these other dependencies other then webpack for the direct WP5 support.

webpack-merge latest changes are very breaking and I'm pretty sure it will work on WP5.  It should be upgraded sometime in the future but this will be very breaking for plugins, so we should leave it for now.
I'll start bumping these other dependencies other then webpack for the direct WP5 support.

Progress report
In my PR, I have successfully repaired development and production builds.
Dependencies
This change requires updating a ton of dependencies. It has also added, removed and changed a few, such as:
- pnp-webpack-plugin
* @endiliey/responsive-loader -> responsive-loader-modern (my fork, stops webpack from throwing due to a webpack 2 backwards compatibility check that breaks things)
* optimize-css-assets-webpack-plugin -> css-minimizer-webpack-plugin (see note below!)
* react-loadable-ssr-addon -> react-loadable-ssr-addon-webpack-5 (also my fork, stops it from trying to use chunk data that is no longer exposed)

Things that do work

Development server is perfectly fine
Production build works (throws a type error on netlify right now, about to fix after clicking the comment button on this)

Things that don't work

CSS in the production build is horribly broken in a number of ways.  I have traced the issue back to mini-css-extract-plugin.
Speaking of CSS, the new CSS compressing plugin just throws an error whenever I tried to get it working.  Will post more details soon.

Speed
I have not measured speed impact/changes yet.

We are anxiously awaiting this as we have recently migrated our monorepo to Webpack 5, and Docusaurus is the one project that's keeping us from making it a complete migration. Thanks for all the work that's being done on this!

You don't need pnp-webpack-plugin plugin, because webpack has built-in PnP support

You don't need pnp-webpack-plugin plugin, because webpack has built-in PnP support

Yeah we removed that

@anshulrgoyal by the way, its 0.5 seconds faster after the upgrade.


Trezors: Update to work with firmware 2.3 and 1.9

Trezor firmware versions 2.3 and 1.9 changed a couple of things that were causing HWI to not work with them. This PR fixes those things.

Changed promptpin to use GetPublicKey message. The Ping message used previously no longer has a pin prompt option.
Updated messages and passphrase callback to match new passphrase workflow
Check for a session being initialized by doing a GetPublicKey and seeing if a FirmwareError is returned. If so, send an Initialize to start a session.

Due to the passphrase and session changes to the Trezor T, users using firmware 2.3 will be required to enter the passphrase on the host in the same way that is done with the Trezor 1

Entering passphrase on the device is definitively possible via python-trezor 0.12, see https://github.com/spesmilo/electrum/pull/6064

i don't understand why the previous code disallowed entering passphrase on host, and the new code enforces passphrase on host. If entering on device ever worked, there is no reason it should not work now. If entering on host works for T1, there is no reason it should not work for TT.

I think I understand why you disabled passphrase-on-host for TT. Is it because there is no place to prompt to enter on host / on device?
If that's the case, I'd suggest a different behavior: if passphrase is not specified on the command line, ask for passphrase on device. Otherwise use the supplied passphrase.
(Alternately, you could reuse promptpin to invoke the passphrase-on-device dialog on TT.)
I still don't understand why entering on host was briefly disallowed.

Entering passphrase on the device is definitively possible via python-trezor 0.12, see spesmilo/electrum#6064

The session changes mean that a user must enter their passphrase for every single command.

If that's the case, I'd suggest a different behavior: if passphrase is not specified on the command line, ask for passphrase on device. Otherwise use the supplied passphrase.

That's not a bad idea.

I still don't understand why entering on host was briefly disallowed.

I prefer that as much sensitive user info be entered on the device itself. So the preference is to have PINs and passphrases entered on the device. Previously the T would let you do that and it would cache the passphrase so users only had to do it once. So entering on the host was disallowed in order to require users enter the passphrase on the device. But with the session changes users would have to enter the passphrase with every single command they do. I found this experience to be cumbersome and annoying so I reversed the requirement: users must now enter on the host (this reversal is partially because we have no way to prompt users for which they want to do).

The session changes mean that a user must enter their passphrase for every single command.

Not if you implement sessions like we suggested earlier.

Dne pá 17. 4. 2020 18:31 uživatel Andrew Chow<EMAIL_ADDRESS>napsal:

The session changes mean that a user must enter their passphrase for every
single command.  I
thought you resolved that by never ever calling Initialize.
the only difference is that now you can't call GetPublicNode on a freshly
plugged-in device, because there is no default session.
once you or something else calls Initialize for the first time (so a
session is created), that session remains open for as long as you don't
call Initialize again.

Not if you implement sessions like we suggested earlier.

Perhaps I misunderstand how sessions work. Or perhaps you are misunderstanding how HWI works. I'll try to explain why I don't want to do that.
As I understand it, sessions require the host software to store state. The device returns a 32 byte session id in a Features message and this session id must be sent in an Initialize message at the beginning of each connection in order to use the passphrase cached for that session. If it is not sent, the device begins a new session which means that the passphrase must be entered again (or a different passphrase entered).
Now the way that HWI works is that it connects to a device, does whatever is needed to complete the action specified by the user, and then disconnects. It must disconnect because it is a command line tool like trezorctl. For example. if the user does getxpub m/0h/0, then it connects to the device, sends Initialize, handles the response, sends a GetPublicKey message, handles the passphrase callback, etc. At the end, it gets the xpub, closes the device connection, returns the xpub, and exits. In this scenario, the user had to enter his passphrase. If we forced passphrase on device, he would have entered it on the device. Now suppose that user does getxpub m/0h/1. We do the same thing as above and the user has to enter his passphrase again. This is not a good experience.
So to have the good experience of the passphrase being cached on the first getxpub, we need to send that session id given in the Features message the first time he did getxpub. But how do we get that session id from the first invocation of getxpub to the second invocation? We can't just "store it in a variable" as you have suggested previously; the program exits entirely so anything that was in memory is lost. We'd have to make HWI store some state on disk. This means that we need to now add an entire framework for storing data on disk. We need to know where the data is going to be, allow the user to change the location, handle file I/O errors, etc. It's a whole lot more work just to store a single 32 byte string.
The crux of the problem is that I (and others) think that HWI shouldn't store anything on disk. It shouldn't need to store any state. No other device requires this. The problem it is trying to solve does not need to store state except for this one specific thing that you have recently added to your firmware. Frankly, I'm tired of having to constantly add Trezor specific things just to keep supporting your devices. This isn't a Trezor utility, it's a general purpose hardware wallet tool and it really should not have device specific commands and options (we already have 3 Trezor specific commands and I'd really like to get rid of them).
So if storing the session id in memory is not possible, and storing it on disk is unwanted, what other options are there? The only thing I can think of is that we make the user remember the session id and enter it on the command line every time. So now the user has to remember a 64 character hex string and remember to pass it in with a Trezor specific command line argument. But this workflow is ostensibly the same thing as making the user give the passphrase on the command line every time. Sure it is not the passphrase itself, but currently the session id is effectively just a proxy for the passphrase. However user passphrases are easier for the user to remember and does not change over time. The user experience between passing in a session id and passing in their passphrase is the same - on device they don't have to enter their passphrase for every command. It just has to be part of the command invocation which is easy to copy and paste.
This is why I've chosen to require the user enter the passphrase on the host. If I have misunderstood how sessions work or if you have any other ideas for how we could actually make use of sessions, please let me know.

The session changes mean that a user must enter their passphrase for every
single command.  I
thought you resolved that by never ever calling Initialize.
the only difference is that now you can't call GetPublicNode on a freshly
plugged-in device, because there is no default session.
once you or something else calls Initialize for the first time (so a
session is created), that session remains open for as long as you don't
call Initialize again.

This was not true in my testing. I guess that's a bug.
In my experimentation, every time I did something without sending Initialize after opening the connection, I always got a non-specific FirmwareError. When I looked into it, this error was because there was no session for that connection.

Your utility is being used essentially as a library for projects like Wasabi, BTCPay and maybe others. If it's not acceptable to pass session_id as a parameter to your utility, then it is not meant to be used as a library and you should communicate this clearly in the readme. I will then point these projects to read that particular line in the readme and recommend them using something else.

So now the user has to remember a 64 character hex string and remember to pass it in with a Trezor specific command line argument.

I am not sure what's the problem here. If I understand it correctly, a user needs to provide  -d <path> on everycall. This is not different to providing -d <path> -s <session>. You can even combine these two into one string. I am pretty users copy-paste the path from the enumerate call anyway as they can change on every device connection. From that regard, it is just making the "path" identifier longer.

@molnard @lontivero This conversation does not look good. Could you handle the situation?

@prusnak

If I understand it correctly, a user needs to provide -d  on every call now.

No.  You can get away with only specifying the master xpub fingerprint, which is unchanging as long as the seed stays the same.  E.g.:
$ ./hwi.py enumerate
[{"type": "trezor", "path": "webusb:002:2", "model": "trezor_t", "needs_pin_sent": false, "needs_passphrase_sent": false, "fingerprint": "01234567"}]

$ ./hwi.py -f 01234567 getdescriptors
{"receive": ["pkh([01234567/44h/0h/0h]xpub[...]

# <physically disconnect device, reconnect, enter pin>

$ ./hwi.py -f 01234567 getdescriptors
{"receive": ["pkh([01234567/44h/0h/0h]xpub[...]

Having to run ./hwi.py enumerate each time I connect the device and then modify any commands I want to reuse from my bash history with a new sessionid sounds annoying; it'd be nice if that could be avoided.

@prusnak Would it be possible to implement backwards compatibility with a new firmware?
It would be really uncomfortable for both of us getting numerous support requests regarding it :/

If a change results in user programs breaking, it's a bug in the kernel. - Linus

@harding I typically do the poor-man's version as well: -t <device type> which of course only works if there is one of each type plugged in

which is unchanging as long as the seed stays the same

You ignore the passphrase completely in the scenario you are describing. The whole discussion here is because of the passphrase.

Would it be possible to implement backwards compatibility with a new firmware?

We are backward compatible. If your client doesn't understand the new API it will fall back to entering the passphrase on the host. What we are discussing here is to bring a new feature: a user can enter the passphrase on the device and they do not have to enter it again for future invocations of HWI if they provide the correct session_id.

@prusnak

You ignore the passphrase completely in the scenario you are describing. The whole discussion here is because of the passphrase.

That's fair, I've never used a passphrase with a Trezor.  However, it seems likely to me given @instagibbs's comment that -d <device_id> doesn't need to be specified for every command even if a passphrase is in use.  (I don't have a spare Trezor to use to test that assertion, though.)

Following up on what @matejcik said, I have been able to restore the previous behavior of having the passphrase entered on device once. After some further testing, the non-specific FirmwareError was only occurring if an Initialize had never been sent. The most recent changes are to do a GetPublicKey to see if it fails, and if it does, then an Initialize will be sent. So we are able to keep the current session open. I no longer was able to reproduce that non-specific FirmwareError on new connections to the device, so I will just call that either a failure in testing methodology, or a bug in a previous firmware revision.
What users experience with these changes should be identical to what they experienced previously.

You ignore the passphrase completely in the scenario you are describing.

I don't think he does. The scenario he describes with using the fingerprint is a common use case. The fingerprint is dependent on the passphrase as it is derived from the key at m/0h. With sessions, the user will always have to provide the session id in order to get the same fingerprint.

We are backward compatible.

While the bulk of this discussion has been about the passphrases, there are other issues this PR fixes which are due to the API not being backwards compatible. Namely the changes to Ping are not backwards compatible as we were expecting the pin_protection field to still be in Ping and have some effect. But that is no longer the case.

@prusnak as you mentioned above, HWI is being used by a couple of projects like Wasabi, BTCPayServer and maybe others too. I know that many of our users love to be able to use their Trezor wallets with Wasabi and the same has to be the case with other projects too.
Two different Wasabi users reported yesterday that they cannot use Trezor with Wasabi after upgrading the firmware and I don't know how many other users could have been affected. Anyway, this is one of those situations where all our projects lose so, could it be possible for Trezor to release a new firmware that makes the interface behaves as before?

Two different Wasabi users reported yesterday that they cannot use Trezor with Wasabi

Unusable Trezor with Wasabi will be fixed when this PR is merged.

could it be possible for Trezor to release a new firmware that makes the interface behaves as before?

No, sorry. Take your time to understand the discussed issue and you'll see there is no regression. And with the change we introduced, the Wasabi UX will be much better for the user. (As they won't have to enter the passphrase again even when switching between different wallets).

Thank you for the cooperation here @prusnak @achow101 @matejcik and @harding !
I agree that we should focus on long-term goals. With all respect, HWI and projects built on the top should be taken into account when checking the compatibility of the upcoming firmware. This issue struct out of the blue and surprised us, more importantly, the users. That is the worst thing can happen on any term.
It seems we can fix this current issue. Also, it attracts the attention that these kinds of changes cannot be done one-sidedly. Preventing cooperation is required.
@achow101 elegant solution! Let me know if I can do some testings on it. Wasabi will publish a new hotfix release with the fix, hopefully soonish.

Here are builds of this PR for testing
hwi-mac.tar.gz
hwi-linux.tar.gz
hwi-win.tar.gz

This version works on old firmware of Trezor T, this is good enough ACK for me.
Testing the new version of firmware now

Updated my Trezor, and it still work. Thank @achow101 !

Tested with numerous HW wallets and firmwares. Works well!

Tested with the binary on macOS but got Could not open client or get fingerprint information: FirmwareError: Firmware error", "code": -13.
However, when I tested (on macOS) from source it worked well for me, both models One and T, both older and the new firmware and both with and without passphrases.
@molnard just to be sure, was the passphrase enabled when you plugged in the device?
For me it does trigger the passphrase input.

I tested with the binaries only (updated according to the edited comment https://github.com/bitcoin-core/HWI/pull/319#issuecomment-615884187)
@ben-kaufman yes the same device is working on windows, there I can enter my passphrase during the enumerate phase.
I have another Trezor_T without the passphrase. It works fine on both OS.

@molnard Yes, I get the same error with the binary, but not when using from the source.

Here are builds of this PR for testing
hwi-mac.tar.gz
hwi-linux.tar.gz
hwi-win.tar.gz
edit: wrong mac build.

@achow101 are you sure the new macOS build linked is correct?

@achow101 are you sure the new macOS build linked is correct?

Nope. I don't have a mac so I can't test it out myself. The macOS builds come from travis, so I probably grabbed the wrong one. But now I can't find the build from this PR...
If the source works, then it should be fine. It looks like there are also some other issues with macOS builds currently.

No problem, it is working - good job, thx!

On Linux I am getting the following error after running our unit tests on HWI:
Error Message:
   WalletWasabi.Hwi.Exceptions.HwiException : 'hwi --testnet enumerate' exited with incorrect exit code: 255.

Zero is the desired. Did you change the exit code?

Installed an Ubuntu 18.04 and dig deeper. With the new release it is not working.
[1810] Error loading Python lib '/tmp/_MEInzgskp/libpython3.6m.so.1.0': dlopen: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.28' not found (required by /tmp/_MEInzgskp/libpython3.6m.so.1.0)
2020-04-21 15:52:43 ERROR	ConnectHardwareWalletViewModel (140)	WalletWasabi.Hwi.Exceptions.HwiException: 'hwi enumerate' exited with incorrect exit code: 255.
   at WalletWasabi.Hwi.HwiClient.SendCommandAsync(IEnumerable`1 options, Nullable`1 command, String commandArguments, Boolean openConsole, CancellationToken cancel, Boolean isRecursion) in /home/user/WalletWasabi/WalletWasabi/Hwi/HwiClient.cs:line 57
   at WalletWasabi.Hwi.HwiClient.EnumerateAsync(CancellationToken cancel, Boolean isRecursion) in /home/user/WalletWasabi/WalletWasabi/Hwi/HwiClient.cs:line 249
   at WalletWasabi.Gui.Tabs.WalletManager.HardwareWallets.ConnectHardwareWalletViewModel.EnumerateIfHardwareWalletsAsync() in /home/user/WalletWasabi/WalletWasabi.Gui/Tabs/WalletManager/HardwareWallets/ConnectHardwareWalletViewModel.cs:line 414
   at WalletWasabi.Gui.Tabs.WalletManager.HardwareWallets.ConnectHardwareWalletViewModel.<>c__DisplayClass6_0.<<-ctor>b__13>d.MoveNext() in /home/user/WalletWasabi/WalletWasabi.Gui/Tabs/WalletManager/HardwareWallets/ConnectHardwareWalletViewModel.cs:line 129

There weren't any changes to exist codes. I'm not seeing anything unusual with it on Arch.
If you use HWI by itself, do you get the same issue? Is there any error output besides the exit code?

There weren't any changes to exit codes. I'm not seeing anything unusual with it on Arch.
If you use HWI by itself, do you get the same issue? Is there any error output besides the exit code?

According to @lontivero:

Yes, Ubuntu 18.04 comes with libc6 (2.27) and this new hwi requires (2.28).

This upgrade is necessary? It will break HWI for Ubuntu 18.04 users.

According to @lontivero:

Yes, Ubuntu 18.04 comes with libc6 (2.27) and this new hwi requires (2.28).

This upgrade is necessary? It will break HWI for Ubuntu 18.04 users.

Hmm. That's odd. The docker container that does the build should be using debian stretch which has an older libc6. I'll investigate that.

@molnard @lontivero I've redone a build from scratch (nuked the docker containers and rebuilt them) and uploaded the results. These match the travis results too (the previous ones didn't) so the containers were probably dirty from build system changes I was experimenting with. Can you try the new binaries? They should be linked to an older libc.

Where are the new binaries?

On the releases page. I overwrote the original release.

It works in the Ubuntu 18.04 (no libc6 error). Thank you!

Tested, works well! Thx!


RemoteEvent/RemoteFunctions

I'm not sure if this is out of scope for lemur, but I want to run extensive unit testing on my RemoteEvents to make sure they're bulletproof.
Currently, neither of these objects are supported in lemur. For this to be added, I'm sure that fake players would also need to be implemented. Roblox itself runs tests like this (as shown by TestService.NumberOfPlayers).

I think this is within scope!
We'll need to start tracking scripts vs local scripts, and probably need a proper event loop first in order to resolve them, though!

I feel that's extremely helpful, but not exactly necessary for a lot of the testing I need to do, unless I'm not thinking big enough.


Reactive auth user

So I've tried to follow the docs and 2.0 PR #216
Got everything working pretty well. I can login and send messages back and forth.
(Followed this guide for the flow https://docs.feathersjs.com/guides/basics/setup.html)
I made a profile page to update the logged in users information.
The flow i used for that is
const { User } = this.$FeathersVuex.api;
let user = User.getFromStore(authUser._id);
if (!user) {
  user = await User.get(authUser._id);
}
this.user = user.clone();

I then edit the copy and call this.user.save() once I'm done to commit the changes and update the server.
This does not appear to update authenticated user in the store state.auth.user.
To fix this i added the following to my makeAuthPlugin which takes the user information returned from the server upon login and creates a User model using that information and now the auth.user automatically updates since i now clone the model from auth and save that instead.
actions: {
  responseHandler({ commit, state }, response) {
    const apiService = "api";
    if (state.userService) {
      let user = response[state.responseEntityField];
      const Model = Object.keys(models[apiService])
        .map(modelName => models[apiService][modelName])
        .find(model => model.servicePath === state.userService);
      if (Model) {
        user = new Model({ ...response.user });
        commit("setUser", user);
      }
    }
  }
}

Am i doing something wrong, since i would expect the auth.user to be reactive by default, or was that changed in the 2.0 pre-release? And if that is the case, is the above approach the correct one to solve the issue?
I'm using feathers: 4.3.7 on the server and client and feathers-vuex: 2.0.0-pre78

@Hiws could you try using<EMAIL_ADDRESS>(non-pre).  I released it yesterday.

The code you have posted is almost exactly identical to what's in the final released version.

@marshallswain
It works perfectly after upgrading!
Thanks you for all the great work, thoroughly impressed with what you've made.