Alert for StopInfo survey

Hi all,
We're trying to deploy a survey that understands why people contribute information to StopInfo (or why they might want to) for an upcoming research conference. We also want to be able to improve the feature in any way we can in order to increase and maintain contributions. We were thinking it would be really useful to get participants by having an alert pop up to alert OBA users of the survey when they tap the info button to get into StopInfo.
I wanted to see if this was feasible, as we have a relatively short time-frame to work with. Ideally we'd have to launch a new version sometime next week. @aaronbrethorst did you have a specific timeframe in mind for the next release? If it won't work out, that's okay -- just thought I'd check to see if it coincides with a launch.
As to functionality, the alert would contain a link to the survey to open in Safari (we've played around with just having the link in StopInfo itself within the web view -- but it displays poorly in app, there's bad routing, and unfortunately there is no way to break out of the web view to open Safari (it doesn't have a URL scheme like other apps). Once they tap on the info button next to the stop name, or "About this Stop" under the arrival info, a pop-up would appear that asks them if they're interested in taking the survey... and they can tap "Yes," "No," or "Don't show this again." I'm open to other suggestions. We would take it out with the next update.
I'm generally wary of having pop-ups in apps asking to rate the app or visit other sites -- they're pretty annoying -- but if we could get even 20-30 more people because of this, it'd be worth it to us.
I'll also be posting a more in-depth list of StopInfo improvements or fixes within the next couple of days. Analytics show it's accessed on average 1000 or so times a day (on weekdays), and it's been reported as being really useful by our blind and low vision participants, so it's a feature we want to stick around.

Seems like it would be easier to do this in the web UI using something like Bootstrap modals.

@bbodenmiller I've tried a similar approach -- problem is that it still gets stuck in the web view controller because it thinks it's the native browser. I need the link to open in mobile Safari.

@caitbonnar I don't have a good suggestion for you, offhand, unfortunately. Loading this in-app would definitely be the easiest way to go, and it's a bummer that won't work.
I wasn't planning on releasing anything in the next week, but maybe you can just make that change off master, and resubmit to Apple as 2.3.2?

@caitbonnar any particular reason why it needs to be in native safari? I believe you can load iframes in the bootstrap modal.

@bbodenmiller Basically just the usability is really bad. If they tap the app's 'Back' button at all (which is more salient than the survey's back button), the results get messed up. We don't really want to run that risk. If 'back' is pressed in Safari, it for some reason behaves better (I've been testing out a lot of different ways). The iframe may help with routing but I think is subject to the same problem.
I'll talk to my team today and see what they say. It may be more trouble than it's worth, and there are still issues with it displaying correctly in the mobile browser (since we have to use certain survey forms for IRB, and they are not the best on mobile).
Thanks to both of you for the input!

Have you thought about asking users to enter their email to be emailed the survey?

@bbodenmiller We have a list of e-mails from people who have previously signed in to contribute, and are contacting them with the survey that way. However we'd like to reach a broader base, and especially allow people to be anonymous if they so choose (in accordance with our IRB), so we'd like the alert in addition.
I talked with my team and they are pushing for us to include the alert, even if it requires an update with only that in place. @aaronbrethorst If this is still okay, I will work on getting that submitted today as 2.3.2. If you'd like me to include any other minor updates, let me know!

@caitbonnar that sounds fine with me. I'd rather not include anything else, as it increases risk of destabilization. Thanks, though :)

Sounds good, will let you know when this is submitted. Thanks!

@aaronbrethorst @bbodenmiller @barbeau For future ref, the alert has so far worked really well for us -- it's been out for less than a day in the App Store and we've already gained over 100 new responses and some pretty valuable feedback. :-)

:+1: :+1: :+1:

Very nice!!  BTW - we were looking at the Google Analytics data today in context of testing OBA Android analytics, and we stumbled on some iOS StopInfo events being reported for regions outside of Puget Sound.  I believe @caitbonnar that you're using this analytics data for a paper, so just a heads up that you should filter by region before using the StopInfo numbers to make sure they reflect users actually accessing the StopInfo pages for Puget Sound.  I'll open an issue shortly for what we believe is a bug allowing StopInfo events to be reported in GA from outside of Puget Sound.

@barbeau Yeah, I noticed that too -- will definitely make sure to do that. :-) The reason people can get to StopInfo outside of Puget Sound is because we always display the "About this Stop" button below the arrival times. A quick fix would be to take that out completely for regions outside of Puget Sound.
However, that was done intentionally at first, to see if other regions were interested. We have an info page that it brings them to if they're outside of Puget Sound. (At least we used to - I'll have to check with Meg that it's still in there.) It gives details about the project and mentions it is currently only in Puget Sound. Maybe it would be easy to have a quick button or something that allows someone to request StopInfo for their region.

@caitbonnar Ok, thanks, good to know.  Yes, the info page for outside Puget Sound does still work.
We were confused because the info button is only displayed next to the stop name if there is a regional StopInfo server (i.e., you're in Puget Sound), but the "About this Stop" button is always shown, even if you're outside Puget Sound.  So, both buttons have the same resulting action, but they are displayed under different conditions.  I would suggest normalizing it so both buttons are shown/hidden together.  I do agree that it would be good to capture interest in the feature from regions that don't use it, but in the app analytics I'd like to be able to distinguish between interest in the feature, and actual use of the feature.  Right now from the app analytics it appears as if Tampa users are using StopInfo, even though it doesn't exist yet in Tampa.  I suppose you could cross-reference with StopInfo analytics logs, but that seems to be a complicated solution.
Does that make sense?  If so I can open an issue for these items so we can discuss the exact solution further.

@barbeau Yeah, I think that was the product of trying to hurriedly code it in before our study... I agree that it should definitely be normalized. After this study finishes, I plan on opening an issue about better StopInfo integration too.
That makes sense about the analytics. We could also fire the analytics only if the region is currently Puget Sound. However because StopInfo is a web view, its analytics would go through the web site, which would be separate from the mobile analytics. We could send a parameter that would identify whether StopInfo was loaded from iOS, Android, or the Web, though. But you're right, it is more complicated.
Feel free to open the issue now if you'd like -- I can tack on more StopInfo integration changes, or just reference the issue later from a separate issue. Whatever would be best for keeping things organized.

@caitbonnar Ok, thanks!  I'll go ahead an open an issue just so this doesn't get lost.  Feel free to tack on or open a new one if the other things aren't exactly related.


Create sst_GO

!

What is this?

We are transition this repo to https://github.com/sst/v2 and are closing this issue for now. If it is still relevant please reopen it there - sorry for the trouble


⚠ Bofrost.lu has degraded performance

In 0c301d3, Bofrost.lu (https://www.bofrost.lu) experienced degraded performance:

HTTP code: 200
Response time: 1490 ms

Resolved: Bofrost.lu performance has improved in 9e20c71 after 1 hour, 18 minutes.


gcc11: Update back to 11.3 on Darwin

Description of changes
Update back to GCC 11.3 for Darwin, with the patchset used by Homebrew. Removed libgcc dependency from graphite2 as it does not appear to be needed (tested on aarch64-darwin and x86_64-darwin).

Description of removed patch
https://github.com/NixOS/nixpkgs/pull/171792 downgraded gcc11 to 11.2 because it broke the standalone libgcc:
make: *** No rule to make target 'emutls_s.o', needed by 'libemutls_w.a'.  Stop.

Because we build the standalone libgcc with --disable-shared, libemutls_s.o is natually not built. I wrote a small patch to fix it.

Ref: #137877

Things done

Built on platform(s)

[x] x86_64-linux (on master, for the libgcc change)
[ ] aarch64-linux
[ ] x86_64-darwin
[x] aarch64-darwin (on master)

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[x] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
22.11 Release Notes (or backporting 22.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

No objection from me, but I won't do a proper review or testing of this.

Even though this is probably too late for 22.11, is it possible to get a jobset so we can test in the meantime?

Yes, but I believe it's inconvenient for the current commits.  They're in a place where even stdenv binaries are missing, and I'm not speaking of possible regressions that haven't been discovered there yet.  Better do that with this pair of commits based on recent master or at least staging-next branch (though there's still a large rustc regression on staging-next).

And I'm personally convinced that this PR would fall under that freeze.  The gcc condition probably doesn't really apply on Darwin, and minor update shouldn't be a "breaking change".

Builds in progress at https://hydra.nixos.org/jobset/nixpkgs/pr-196565-darwin-gcc11.3

Though I'm surprised that the testing branch is based on a commit from August (0f5230c0ddf).

Though I'm surprised that the testing branch is based on a commit from August (0f5230c).

Oops, the local master ref in my nixpkgs checkout on my Mac was severely outdated. Rebased.

So, diffs in failures can be observed (after rebuilds progress) like

x86_64: https://hydra.nixos.org/eval/1783747?compare=1783745
aarch64: https://hydra.nixos.org/eval/1783747?compare=1783740

Let's put this on hold while we wait for clarifications from @Ericson2314 regarding the libgcc package (https://github.com/iains/gcc-darwin-arm64/pull/99).
graphite2, the only package buildable on Darwin requiring this, doesn't seem to require it at all now (cc @sternenseemann, author of #123420).

In the interest of pushing this forward, let's remove the libgcc patch for now. I removed libgcc dependency from graphite2 as it does not appear to be needed (tested on aarch64-darwin and x86_64-darwin).

I changed the patch URL as suggested and tested on darwin-gcc-11-3-master. Now we need someone to trigger the eval on Hydra again.

It's been a week, and here are the results of the new eval:

x86_64: https://hydra.nixos.org/eval/1784232?compare=1783745
aarch64: https://hydra.nixos.org/eval/1784232?compare=1783745

There are a lot of timeouts with empty logs which I'm not sure how to fix.

Those were issues in the builder machines; I can restart the builds.

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/prs-already-reviewed/2617/685


issue with iterating over a dict

If query_params is a dict, the the following code fails to iterate.
https://github.com/kubernetes-client/python-base/blob/11da619c9ef7b2e26fe162e5feeb72d5bedcc5c9/stream/ws_client.py#L238
For Python 2.x:
for key, value in d.iteritems():
For Python 3.x:
for key, value in d.items():

Hi @sujeeth961991,
Thanks for reporting. Could you please create a pull request to fix the issue?
I think we should use .items() to iterate over the items, as this function will be compatible with both python versions (iteritems is removed from 3.x as the default behaviour is to return an iterator).

/assign @iamneha

cc @roycaihw @brendandburns

Interesting. I tried to log kwargs in websocket_call and ran e2e tests. It showed that kwargs was
{'body': None, 'query_params': [('command', ['/bin/sh', '-c', 'for i in $(seq 1 3); do date; done']), ('stderr', False), ('stdin', False), ('stdout', True), ('tty', False)], 'post_params': [], '_request_timeout': None, '_preload_content': True, 'headers': {'Content-Type': 'application/json', 'Accept': '*/*', 'User-Agent': 'Swagger-Codegen/8.0.0-snapshot/python'}}

where kwargs.get("query_params", {}) was
[('command', ['/bin/sh', '-c', 'for i in $(seq 1 3); do date; done']), ('stderr', False), ('stdin', False), ('stdout', True), ('tty', False)]

It seems that query_params is not a dict but a list of tuples?

It is a list of tuples of a (key, value) pair:
https://github.com/kubernetes-client/python-base/blob/8d7b6f7dc3158ff5c45fd828e965606d1c0ecfeb/stream/ws_client.py#L237

@iamneha See code here: https://github.com/kubernetes-client/python-base/blob/8d7b6f7dc3158ff5c45fd828e965606d1c0ecfeb/stream/ws_client.py#L238-242
The (key, value) pair are appended to the query_param list - and therefore it is always a list of tuples.

@micw523 I think you are confusing query_params with kwargs.get("query_params").
You can see test_pod_apis
Here you will see we stream the API and send command that need to be executed. kwargs is coming from this e2e tests which is in form of tuple.
And you are right, the query_params you are taking about is tuple. We are passing this tuple to the url which execute inidividual command params during API call

@iamneha You’re right. I was not talking about the correct query_params here. However, The e2e test called connect_get_namespaced_pod_exec, which also packed the parameters into a list of tuples. It’s in the core_v1_api.py and it’s too large for github to show so no link here :)

@micw523 yes you are right nothing needs to be changed here :)
cc @roycaihw

@iamneha @micw523 Thanks for looking into this issue :)
/close


AdaptiveByteBufAllocator: Make pooling of AdaptiveByteBuf magazine local

Motivation:
To reduce the usage of FastThreadLocal and make the AdaptiveByteBufAllocator also play nice with virtual threads we should just pool the AdapativeByteBuf instances in the Magazines itself.
Modifications:
Remove usage of Recycler for pooling AdaptiveByteBuf Use a bounded MPMC queue that is stored in the Magazine itself for pooling these instances Result:
Fixes #14279

@normanmaurer I was investigating why there were test failures on https://github.com/netty/netty/pull/14332 and tried to recreate the PR from first principles. I think it might actually be a bug the MPSC queue. Regardless, I ended up with a very similar but slightly different implementation.

I'm all in to have this - it trades the virtue of simplicity with some cost in the optimized common path i.e.

if a thread is an event loop one it has to perform a CAS using the mpsc q to release the buffer wrapper

Another option could be to have a Recycler impl which has the same tunable we're used to AND using a pluggable concept of ownership. I'm preparing a small PoC to show what I mean

@normanmaurer Take a look. The allocate method does the fallback allocation now, instead of communicating potential allocation failure.

@franz1981 any other comments or can we merge this one ?

Yep this morning I can share my proposal, yesterday I have been sidetracked the whole day : 🙏

@franz1981 ok just so I am sure I understood correctly... we should hold back until you shared your proposal, correct ?

@normanmaurer @chrisvest it -> https://github.com/franz1981/netty/tree/4.1-mag-local-pool
I'm not fixing the comment at https://github.com/netty/netty/pull/14414#discussion_r1816359191 or the ones from Norman, but proposing a different fast(er)-path for event loop confined magazines (as usual :P )
I would have preferred to avoid another fasthreadlocal::get TBH, given that we already do it for the event loop-local magazine, but in this way we can reuse the Recycler without modifying it.
I believe there is some bench which correctly use fast thread local threads on JMH which can be used to compare against the 2.
The branch I've sent got a sole concern, perf wise:

I made the q-based handle (shared magazines) to be an EnhancedHandle child (which impl the recycle methods)
The Recycler already have 2 EnhancedHandle i.e NOOP_HANDLE and the default one
it means that AdpativeByteBuf::deallocate can be megamorphic - I'm sending an additional commit in the branch to improve it

@franz1981 The two approaches sound equivalent in terms of code style, so we could just let the benchmarks decide. So how about we look at that in a follow-up? I know it can take lot of time to tweak benchmarks back and forth.

Right now I suggest already to bring the fast thread local solution in too - which uses array deq for the fastest path - which is very difficult to beat, unless there is something very wrong in.
If you don't feel confident for this I can send a separate PR with the follow up changes

@chrisvest I've sent by accident my changes into your branch - now I've forcibly pushed your original commits removing mine :"(
I'll send a separate draft PR on top of your changes here to help @normanmaurer as well to see what changes I'm proposing


sink(cdc): close MemoryQuota to stop SinkManager correctly

What problem does this PR solve?
Issue Number: close #9073
Issue Number: close #9067
What is changed and how it works?
In SinkManager.Run or SinkManager.Close, we wait a WaitGroup, which is associated to sink workers and redo workers.
But sink workers and redo workers can be blocked on MemQuota.BlockAcquire, so the wait can't be finished forever.
Check List
Tests

Unit test

Questions
Will it cause performance regression or break compatibility?
Do you need to update user documentation, design documentation or monitoring documentation?
Release note
None

/merge

/retest


Is it possible to send queries to multiple databases behind the scenes?

The docs mention the possibilities of an A/B test. Is it possible to configure this to in any way send the same query to multiple databases simultaneously? I realize that the caller wouldn't realize that this was happening, but this would be an ideal way to test a new DB configuration, by putting another copy of the database behind pgbouncer-rr, and then running all queries through the primary and the test DB simultaneously.

Hi @Taytay - as you note, the calling client could only support one target database, however, you could potentially capture the query statement in your routing or rewriting python function, and pass it to another process that would act as a new client to replace the query to additional databases. I haven't done this, but I think it should be doable. Good luck.

Thanks very much! I'll close this.


Set help link to our Pages site

Running the help command currently opens a subwindow containing a link to the AB3 user guide, with a copy URL button. Pressing the help button in the tab bar or pressing the F1 keyboard shortcut opens the AB3 user guide in the browser directly. These links should be updated to our currently hosted version of our user guide.
As a side note, we could also edit the UG to mention these methods of navigation.

https://ay2324s1-cs2103-w14-3.github.io/tp/UserGuide.html


DPL-871 -  As a developer I would like the ability to view old runs in Traction

User story
As a developer I would like to add a fall back option for SMRTLink Versions and System Names so that old PacBio runs in Traction can still be viewed. Currently if you go to Traction production and try view an old run (TRACTION-RUN1), it errors when you try and view the run.
Who are the primary contacts for this story
Ben T
Who is the nominated tester for UAT
Ben T
Acceptance criteria
To be considered successful the solution must allow:

[ ] When I go to a run with no SMRTLink Version or System Name I can still see the run plate and wells

@BenTopping
So do we want to populate these fields for old runs with the correct information?
Or do we want to leave them blank, and make the code handle it?
Thanks

@BenTopping So do we want to populate these fields for old runs with the correct information?
Or do we want to leave them blank, and make the code handle it?
Thanks

@KatyTaylor
Depends how tricky it is to populate them with the correct information. I would say that populating them is the preferred way because it provides more useful information about a run, but at the same time if some of the data doesn't exist in traction-service (like system name or smrt link version) Traction will reject any updates to the run anyway.
So I guess my answer is populate them if its fairly easy to do, but if it requires a large amount of work then just leave them blank and make the code handle it and it is not that much more valuable to populate them.

The 4 system names referenced in pacbio > run.rb (traction service) are Sequel II, Sequel I, Sequel IIe and Revio.
There aren't any Sequel I runs in the database.
SELECT system_name, IF(system_name = 0, "Sequel II", IF(system_name = 1, "Sequel I", IF(system_name = 2, "Sequel IIe", "Revio"))) AS system_name, COUNT(*) AS num_runs
FROM pacbio_runs
GROUP BY system_name;

Before and after, having changed run 3 in the test data to have System Name "Sequel II", and SMRT Link Version "v10".
The reason no fields display for the Well Defaults section is that there are no fields configured for SMRT Link v10 in config > PacbioRunWellComponents.json - assume this is intended, or at least out of scope.
Before:

After:

PRs:

https://github.com/sanger/traction-service/pull/1123
https://github.com/sanger/traction-ui/pull/1355/files


No support for collections or tables

Please add some collections/models or tables

I will add it, although it should be pretty easy to just use the key value store instead as you will have better control and it will be easier


7th, 8th 2 days target tasks

Priority: High

[ ] Obtain a JWT from the existing smart contract for identity verification.

Time Allocation: 1 hour
TODO: Investigate and implement the functionality to retrieve a JWT from the smart contract. Test the retrieval process and ensure its compatibility with the identity verification flow.

Priority: Medium

[ ] Run the authentication flow of other projects locally to deepen the understanding.

Time Allocation: 1 hour
TODO: Set up the local environment for the authentication flow of other projects. Follow the documentation or guides provided by those projects to execute the authentication flow and gain a better understanding of the process.

Priority: Medium

[ ] Explore the data gathered in OpSci to understand the available datasets.

Time Allocation: 1 hour
TODO: Access OpSci's data repository and browse through the available datasets. Analyze the data to gain insights into the types, formats, and potential use cases of the gathered data.

Priority: Low

[ ] Contact the administrators of OpSci.

Time Allocation: 30 minutes
TODO: Reach out to the administrators of OpSci via email or other communication channels. Inquire about any specific questions, seek guidance, or request additional information related to the project and its objectives.

Priority: Low

[ ] Create a flowchart (using Mermaid) of the authentication process for your own project.

Time Allocation: 1 hour
TODO: Map out the authentication flow of your project using Mermaid or any other diagramming tool. Visualize the steps, user interactions, and data flows involved in the authentication process.

Priority: Low

[ ] Implement the retrieval of files based on CID locally.

Time Allocation: 1 hour
TODO: Develop a local setup to retrieve files based on their Content Identifier (CID). Use IPFS or related libraries/tools to fetch files using the CID and validate the retrieval process.

Priority: Low

[ ] Test the retrieval of files from GitHub based on the stored CID.

Time Allocation: 1 hour
TODO: Experiment with fetching files from GitHub using the stored CID. Utilize the GitHub API or suitable libraries/tools to retrieve files based on their CID and verify the successful retrieval.

Priority: Low

[ ] Determine whether to focus on IPFS files or Filecoin files.

Time Allocation: 30 minutes
TODO: Make a decision regarding whether to target IPFS files or Filecoin files based on the requirements and objectives of your project. Consider the advantages and constraints of each option before finalizing your choice.

1:  Obtain a JWT from the existing smart contract for identity verification
opsci verse : local run (https://github.com/opscientia/verse)

Run the authentication flow of other projects locally to deepen the understanding.
opsci verse : local run (https://github.com/opscientia/verse)

Trying data fetch with local lotus command.
posted following question to filecoin support channel
===
I'm trying to use  local PC lotus node.
I tested  DL data which CID is on following site.
https://datasets.filecoin.io/google-open-images?page=16
I installed lotus with brew to my macbook.
original data download instructions of above site says  lotus client helper retrieve XXX but it cause following error

No help topic for 'helper'

if  I cut  helper option,  it cause another error.
lotus client  retrieve --miner f096540 bafykbzacedojx7clth6jn6bvx6i2cpfnhvioonnodtukjolv3y6paplokynxc @/data/maris/google_full/open-images-dataset-train3/farm5.staticflickr.com/9.zip
ERROR: could not get API info for FullNode: repo directory does not exist. Make sure your configuration is correct

Shoud I use another way to install lotus?


Add configuration mechanism

Fixes #176
You can now either use a config file or environment variables to manipulate the way that the ALM core starts.
We no longer have any but these two command line switches for the core process:

-config: Takes a filepath to the config file to read
-printConfig: Prints the config (including merged environment variables) and exits

If no configuration file is specified when the core is started, these are the defaults:
#------------------------
# Postgres configuration
#------------------------

postgres.host: localhost
postgres.password: mysecretpassword
postgres.port: 5432
postgres.sslmode: disable
# The number of times alm server will attempt to open a connection to the database before it gives up
postgres.connection.maxretries: 50
# Number of seconds to wait before trying to connect again
postgres.connection.retrysleep: 1s

#------------------------
# HTTP configuration
#------------------------

http.address: <IP_ADDRESS>:8080

#------------------------
# Misc.
#------------------------

# Enable development related features, e.g. token generation endpoint
developer.mode.enabled: false

Although this is a YAML file, we highly suggest to stick to this rather lenghty notation instead of nesting structs.
To override configuration values using environment variables, use the prefix
ALMIGHTY_ and replace the dots in the variables names with underscores.
For example to override postgres.password, set the environment variable ALMIGHTY_POSTGRES_PASSWORD to the value of you liking.
NOTE: Environment variables override the default values and the ones you've set in your config file.

Current coverage is 47.24% (diff: 0.00%)

Merging #274 into master will decrease coverage by 0.42%

@@             master       #274   diff @@
==========================================
  Files            33         33
  Lines          1204       1217    +13
  Methods           0          0
  Messages          0          0
  Branches          0          0
==========================================
+ Hits            574        575     +1
- Misses          589        603    +14
+ Partials         41         39     -2

Powered by Codecov. Last update 1efe0b9...cbffb74


Fix embed button

Needed to remove a stray 'global' to get the modal for adding an embed url to display

Thanks! I'll get this merged in soon and include in the 0.1.0 release


BUG: local variable 'trans_constr' referenced before assignment

Originally posted to community forum
https://support.gurobi.com/hc/en-us/community/posts/15116202641425
The error is coming from these lines:
https://github.com/Gurobi/gurobi-machinelearning/blob/88f953c7b17e4d3c9168dc10b0ad1a270f4112f2/src/gurobi_ml/sklearn/column_transformer.py#LL76C1-L80C1
Not sure if it's a case of missing line or wrong indentation or something else.

Thanks for opening the issue.
It was missing properly handling the error.


add instructions for autoload for sbcl

Ah sorry. Didn't realize you was using the same branch as #1.
Can you perhaps make a new PR? Thank you

Or maybe you can give #4 a try. 😊


Switch first and second result in quickOpen if first result is the current editor

Fixes #159299
This code intercepts quick-open pick lists and if the first pick is the same file as the current editor, it switches the first two results.
It's my first attempt to contribute to VSCode therefore it's more of an RFC, I guess. 8-)
To test this, go to quick open (ctrl-p or cmd-p), type something to find a file, open that file, then go to quick open, type the same something as before, and the same file shouldn't be the first pick this time.

Another benefit of this reordering is that a quick cmd-p enter sequence takes you to the previous editor.

Another benefit of this reordering is that a quick cmd-p enter sequence takes you to the previous editor.

The disadvantage of course is that cmd-p cmd-p does not take you to the previous editor anymore which is the current behaviour.

Quite right @nikita-r; I didn't know you could navigate in that list with cmd-p. As I don't use it, I need to learn more about that pattern of use: do you use the second cmd-p also after typing a query, or only when the first cmd-p already shows what you want?

I don't really know how I use it TBH :) the secondary purpose of cmd-p is somewhat unexpected probably, but some people may be used to it by now...
I have read the issue though and I have a comment on it: consider Angular projects where there are three "paired" files: ProductList.component.html, ProductList.component.ts, ProductList.component.scss.  Now, following the issue logic, you'd want the current file to be the third instead of the second.  Hard to come up with a definitive logic here consistent with the Principle of Least Surprise.

Again good point @nikita-r !
I think if you have multiple files, you would subconsciously expect to have to look in the list (not expect there to be one obviously right answer), or you'd add a distinguisher in the query. I could imagine if I worked with angular I'd learn to just type ht/ts/cs to the end of the query (e.g. plistht) to get where I want. But it feels entirely unnecessary when it's simply the other matching file.
I consider it very unlikely that the user would want to go to the file they're already in. I feel that when typing that quick open query, I'm specifying the file I want among the other files. Maybe the switching behaviour could be triggered by a non-empty query, and even controlled by settings.
In Atom, when I needed this years ago, I ended up making an extension that tweaked the default behavior of the editor. I think VSCode extensions are more constrained (not that that's necessarily a bad thing) so I don't think the same approach is available here.

I have rebased the branch. The rationale stands: when using the "open file" picker, when I have typed something to search, I'm unlikely to want to go to the active file, therefore it shouldn't be the first result.
This PR switches the first and second results if the first result is the active file.


Fabo/2217 limited to max precision

Closes #2217
Description:

Thank you! 🚀

For contributor:

[ ] Added entries in PENDING.md with issue # and GitHub username
[ ] Reviewed Files changed in the github PR explorer
[ ] Attach screenshots of the UI components on the PR description (if applicable)
[ ] Scope of work approved for big PRs

For reviewer:

[ ] Manually tested the changes on the UI

fixed, but still shows scientific notation on transactions:

Codecov Report

Merging #2258 into develop will increase coverage by 0.01%.
The diff coverage is 100%.

@@             Coverage Diff             @@
##           develop    #2258      +/-   ##
===========================================
+ Coverage    96.67%   96.68%   +0.01%
===========================================
  Files          108      108
  Lines         2313     2321       +8
  Branches       115      115
===========================================
+ Hits          2236     2244       +8
  Misses          65       65
  Partials        12       12

Impacted Files
Coverage Δ

app/src/renderer/scripts/num.js
100% <100%> (ø)
:arrow_up:

...rc/renderer/components/governance/ModalPropose.vue
100% <100%> (ø)
:arrow_up:

app/src/renderer/components/wallet/SendModal.vue
85.29% <100%> (ø)
:arrow_up:

.../renderer/components/staking/UndelegationModal.vue
100% <100%> (ø)
:arrow_up:

app/src/renderer/components/common/TmFormMsg.vue
97.77% <100%> (+0.05%)
:arrow_up:

...rer/components/transactions/LiStakeTransaction.vue
94.11% <100%> (ø)
:arrow_up:

...rc/renderer/components/staking/DelegationModal.vue
92.59% <100%> (ø)
:arrow_up:

...rc/renderer/components/governance/ModalDeposit.vue
100% <100%> (ø)
:arrow_up:


Appropriate getting started tutorial and defaults

          I was just trying out a lower variance neg bin sampling... I think deciding the appropriate defaults is an issue.

Originally posted by @SamuelBrand1 in https://github.com/CDCgov/Rt-without-renewal/pull/173#discussion_r1540834169
I think we need to reconsider the getting started example. The default model to demonstrate should be a renewal model, in my opinion.
Also, the direct infections model probably shouldn't have negative binomial sampling? It rather obviously difficult to identify compared to latent process variation in a way that (say) a renewal model wouldn't find problematic.

Agree

#169 Introduced quite a few changes in how we handle model objects so we now need to also reflect these a bit better in a the getting started tutorial.


Doc request: Intero doesn't document keybindings in C-h m

Behavior: open mode help with C-h m.
What happens: docs are shown, and haskell-mode documents its keybindings as a good citizen.
What should IMHO happen: intero should also document its keybindings, so I could check how to jump back (in the context of #81).
If I need to motivate that intero should follow Emacs conventions, read on.
Caveat: I'm not an Emacs expert (I grew up with Vim), so I might be misunderstanding Emacs conventions.
Context
intero does seem impressive for a young program, and I'll keep using it. But FYI, the user experience doesn't "just work" for me, so the claims on the homepage feel a bit exaggerate to me.
I'm aware of https://github.com/commercialhaskell/intero#default-key-bindings, but Emacs has its rules and they're embedded into muscle memory.
I found this issue, #81 and #82 within few minutes simply by installing intero, loading a file, jumping to a definition and trying to go back by muscle memory.
 Sorry.

But FYI, the user experience doesn't "just work" for me, so the claims on the homepage feel a bit exaggerate to me.

I just confirmed this was announced a few hours ago and found
https://www.reddit.com/r/haskell/comments/4mrotr/intero_for_emacs/d3xu7zf
So I would still revise the homepage a bit, but huge kudos nonetheless!

No problem, adding them is a one-liner. Emacs auto-generates these. Thanks for reporting!


Make it possible use alternate link patterns

On the forums, Magnus asked for links of the form §201906202320. The pattern PAT_LINK = re.compile(r'\§(\d+)')  could work.
Accepting wiki links would also be nice.

Would capturing '@(\d+)' be a way of visualizing relationships to tags?
An experimental generic regex argument would be best (use at your own risk, I suppose)


docs(zh-cn): Reviewed 62_what-is-the-rouge-metric.srt

docs(zh-cn): Reviewed 62_what-is-the-rouge-metric.srt #390
Issues
Fixed several content error in original english subtitle
FIX 1
timestamp: 00:00:18,180 --> 00:00:21,180
modification:
tech --> text
FIX 2
timestamp: 00:02:34,290 --> 00:02:36,870
modification:
abstracter --> abstractive
FIX 3
timestamp: 00:03:21,660 --> 00:03:23,910
modification:
load metric --> load_metric

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.


typo in lib/misc.zsh

https://github.com/robbyrussell/oh-my-zsh/blob/master/lib/misc.zsh#L3
should be if [[ -e "$d/url-quote-magic" ]]; then

allready reported


"Rast" is listed in the drop-down menu

Description:
When user writes ”Rast" the location appears in the drop-down menu next to the comune and the county which is a part according to the law.
Precondition:
The website is up an running.

Step 1
Write in the search bar "Rast"

Expected results
The location appears in the drop-down menu as "Sat Rast, DOLJ (Rast)".


Porting ethereum-types to zappy

This package: https://www.npmjs.com/package/ethereum-types is required for the arbiter (depends on @zapjs/types)
This package from the official ethereum repository seems like it will do the trick.
Should be added to requirements.txt/setup.py.

can't find documentation for ethereum-types, but seems like we can just use the https://github.com/zapproject/zappy/tree/main/packages/artifacts jsons


Follow Disqus' instructions to add comments

Easy Peasy :smile:

Sorry it's a mistake, I wanted to send a PR to my own master branch :stuck_out_tongue:


Scroll Bars

This tool is fantastic thanks.
Is there any way it can automatically show scroll bars like Power BI does with a wide column chart for example? It allows scrolling whilst preserving the y axis labels.
I've seen the Vega workarounds using a parameter slider or hconcat but I would love it to be consistent with the other visuals in Power BI.
Thanks.

Hi there, and thanks very much for your feedback :)
What you're after is not a Vega feature, and will not be possible to do from within Deneb - the workarounds you have already listed are the currently established ways of being able to manage it.
If this is something that you'd like in Vega or Vega-Lite, then I would suggest making a feature request in the appropriate repository for the team to have a look at. It certainly sounds sensible, but I know from experience in developing a dedicated custom visual that it might be a fair amount of effort to get it to work in a more flexible scenario like this, and it's unlikely that the team would be able to extend this to supporting the canvas renderer if they could achieve it.


WIP: Add OWNERS to Upgrading Kubeflow deployment

Looking into the feature requests, having an upgradeability path seems to be referred often. Wondering who should own these docs.
@jlewi do you have someone in mind?
cc @8bitmp3

I'm not a domain expert but happy to be just a reviewer @RFMVasconcelos Thank you!

I'm not a domain expert but happy to be just a reviewer @RFMVasconcelos Thank you!

+1 I can help here


Fixed a bug in the dataBody when messages value are null.

Had a problem with the view when trying to open a topic with some null values.
Here is a fix for that bug.

GG


Remove ToMisoString constraint on text

Will get around having to pack everything.

Re-exported Data.Monoid from Miso.String
Removed silly ToMisoString Value instance.
Added ms function as a concise toMisoString

cc @cocreature, can I get a review here.

Should be fixed now.


HSEARCH-1837 Allow Integrators to customize default directory providers ...

...for shortcuts
https://hibernate.atlassian.net/browse/HSEARCH-1837

Hi @gustavonalle , I've merged it, thanks! Looks like you know exactly how we structure things.
I've made one single change: please see my second commit, just moved one class to a different package.

BTW, didn't want to delay the fix for a minor question, but still have one doubt: why didn't you actually implement all of the default methods in the base class?

why didn't you actually implement all of the default methods in the base class

Mainly to make extensions easier, like in infinispan I will be doing this:
https://github.com/gustavonalle/infinispan/blob/ISPN-5311/query/src/main/java/org/infinispan/query/backend/DirectoryProviderService.java

I guessed you were going to extend the base class, but you could still override any method even if it was implemented there right?
Do you mean you prefer them to be abstract? Ok, I was just curious.


Issues with scrolling and large tables (>500 rows)

This issue was discovered when I tried testing DC using a table containing 1000 rows to see how it would handle. There are a two issues here;

The scrollbar behaviour is not as expected, especially the horizontal scrollbar
Scrolling of the table vs scrolling of the application and the visibility of the menus et. al.

Desired Behaviour
If the number of columns in the table exceed the number of columns that can be displayed on the screen the horizontal scrollbar should display at the bottom of the screen, irrespective of the number of rows loaded.
When user clicks on the horizontal scrollbar at the bottom of the screen it should allow them to move it  sideways to scroll through the columns offscreen.
When user clicks on the vertical scrollbar at the right of the screen it should allow them to move it up and down to scroll through rows offscreen.
Additional rows should load as the user scrolls down until the last row at the bottom of the table is reached.
The top menu (Validate | Column | Table etc.), tabs, column reference (A,B,C,D etc.) and column headings should all remain visible as the user scrolls down.
To save data and load time, DC should only display a set number of rows by default (which it appears to do). However, there needs to be an option that allows the user to increase the maximum number of rows to load, including loading all rows.
There also needs to be the ability to pin and unpin the top menu and tabs to the top of the screen so that the user has the ability to control whether they remain visible or be hidden as they scroll.
Current Behaviour
When a large table is loaded that contains more than the maximum number of rows visible on the screen DC only loads the rows that can be displayed. This is fine, but when the user scrolls down and clicks on the horizontal scrollbar to scroll sideways, DC loads additional rows at which point the horizontal scrollbar (which is supposed to allow the display of additional columns, not rows) disappears.
This means the user cannot use the horizontal scrollbar to navigate sideways until all rows in the table are loaded. If there are >100 rows in the table, this will become a problem.
The top menu (Validate | Column | Table etc.), tabs, column reference (A,B,C,D etc.) and column headings  also disappear from view if the scrollbars are used.
There is no easy way to navigate to the last row at the very bottom of the table.
Steps to Reproduce

Open Data Curator using the icon
Open a new file (any type) with >100 rows
Start scrolling

Your Environment
Data Curator version 0.2.3
Windows 7
Images

related to #42

Working well on macOS with Data Curator v0.3.1
Will test on windows when 0.3.1 binary is released

Tested works well on Windows 7.


Minor doc fixes

Hi,
I just found & fixed some minor doc issues.

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: sbueringer
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approver: sdake
If they are not already assigned, you can assign the PR to them by writing /assign @sdake in a comment when ready.
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

content/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Hi @sbueringer. Thanks for your PR.
I'm waiting for a istio member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.
I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

/assign @sdake


NOISSUE - Check For Subject During Policy Addition

Signed-off-by: rodneyosodo<EMAIL_ADDRESS>What does this do?
On things services check if the subject is a thing or a user.

If the subject is a thing, check the following:

The user adding the policy is the owner of the thing
The user adding the policy has c_share action on the thing

If the subject is a user, check the following:

The user adding the policy is the owner of the user
The user adding the policy has c_share action on the user

Which issue(s) does this PR fix/relate to?
No issue
List any changes that modify/break current functionality

Ensures that the object column in the policies table references the id column of the groups table.
Check for subject in addition to checking for object

Have you included tests for your changes?
Yes
Did you document any new/modified functionality?
No
Notes
To be merged after https://github.com/mainflux/mainflux/pull/1825

Please resolve conflicts and remove duplicate APIs (ShareClient = AddPolicy). Don't forget to update API docs accordingly.

That is done at https://github.com/mainflux/mainflux/pull/1825

LGTM

@dborovcanin please review this one


Flutter WEB support?

Any plans to bring flutter WEB support?

Many browsers support pdf viewing , but some mobile web browsers not!
Also, browsers are not as flexible as flutter for customization.
So it seems that web support is needed.
Thanks.


Azure: Deploy cloud-config into guest clusters to fix e2e

The conformance tests expect this to exist for Azure and will fail to
start if it doesn't:
https://github.com/openshift/origin/blob/625733dd1ce7ebf40c3dd0abd693f7bb54f2d580/test/extended/util/cluster/cluster.go#L186
The config does not contain any secret information, so this should be
fine.

What this PR does / why we need it:
Which issue(s) this PR fixes (optional, use fixes #<issue_number>(, fixes #<issue_number>, ...) format, where issue_number might be a GitHub issue, or a Jira story:
Fixes #
Checklist

[ ] Subject and description added to both, commit and PR.
[ ] Relevant issues have been referenced.
[ ] This change includes docs.
[ ] This change includes unit tests.

@csrwng mind having a look? This blocks azure conformance tests

/lgtm


feat: 明细表支持同名列渲染 closes #2502 #2510

👀 PR includes

🐛 Bugfix

[x] Solve the issue and close #0

📝 Description

明细表使用指标名作为 id, 不支持同名的场景, 修改为使用 field 作为唯一标识, 已解决同名场景下的交互 (刷选/复制) 异常.
🖼 Screenshot

Before
After

🔗 Related issue link

closes #2502 #2510
🔍 Self-Check before the merge

[ ] Add or update relevant docs.
[ ] Add or update relevant demos.
[ ] Add or update test case.
[ ] Add or update relevant TypeScript definitions.

Codecov Report
Attention: Patch coverage is 88.88889% with 1 lines in your changes are missing coverage. Please review.

Project coverage is 78.71%. Comparing base (ecdc9c8) to head (7cd3193).
Report is 259 commits behind head on next.

Files
Patch %
Lines

.../s2-core/src/utils/layout/generate-header-nodes.ts
50.00%
0 Missing and 1 partial :warning:

Additional details and impacted files
@@            Coverage Diff             @@
##             next    #2568      +/-   ##
==========================================
+ Coverage   75.77%   78.71%   +2.94%
==========================================
  Files         257      274      +17
  Lines       11994    12365     +371
  Branches     2464     2522      +58
==========================================
+ Hits         9088     9733     +645
+ Misses       1398     1099     -299
- Partials     1508     1533      +25

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

:tada: This PR is included in version @antv/s2-v2.0.0-next.13 :tada:
The release is available on:

npm package (@next dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


CosyVoice2 训练报错 ImportError: There is no such entity as torchaudio.functional.compute_kaldi_pitch

windows11 使用项目Dockerfile 创建docker镜像运行训练脚本 examples\libritts\cosyvoice\run.sh报错。模型是CosyVoice2-0.5B

[2024-12-20 01:25:35,226] [INFO] [real_accelerator.py:203:get_accelerator] Setting ds_accelerator to cuda (auto detect)
 [WARNING]  async_io requires the dev libaio .so object and headers but these were not found.
 [WARNING]  async_io: please install the libaio-dev package with apt
 [WARNING]  If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found.
 [WARNING]  Please specify the CUTLASS repo directory as environment variable $CUTLASS_PATH
 [WARNING]  sparse_attn requires a torch version >= 1.5 and < 2.0 but detected 2.3
 [WARNING]  using untested triton version (2.3.1), only 1.0.0 is known to be compatible
/opt/conda/envs/cosyvoice/lib/python3.8/site-packages/torch/nn/utils/weight_norm.py:28: UserWarning: torch.nn.utils.weight_norm is deprecated in favor of torch.nn.utils.parametrizations.weight_norm.
  warnings.warn("torch.nn.utils.weight_norm is deprecated in favor of torch.nn.utils.parametrizations.weight_norm.")
/opt/conda/envs/cosyvoice/lib/python3.8/site-packages/torch/nn/utils/weight_norm.py:28: UserWarning: torch.nn.utils.weight_norm is deprecated in favor of torch.nn.utils.parametrizations.weight_norm.
  warnings.warn("torch.nn.utils.weight_norm is deprecated in favor of torch.nn.utils.parametrizations.weight_norm.")
/workspace/CosyVoice/train_data/CosyVoice/cosyvoice/dataset/processor.py:24: UserWarning: torchaudio._backend.set_audio_backend has been deprecated. With dispatcher enabled, this function is no-op. You can remove the function call.
  torchaudio.set_audio_backend('soundfile')
Traceback (most recent call last):
  File "cosyvoice/bin/train.py", line 170, in <module>
    main()
  File "/opt/conda/envs/cosyvoice/lib/python3.8/site-packages/torch/distributed/elastic/multiprocessing/errors/__init__.py", line 347, in wrapper
    return f(*args, **kwargs)
  File "cosyvoice/bin/train.py", line 101, in main
    configs = load_hyperpyyaml(f, overrides=override_dict)
  File "/opt/conda/envs/cosyvoice/lib/python3.8/site-packages/hyperpyyaml/core.py", line 188, in load_hyperpyyaml
    hparams = yaml.load(yaml_stream, Loader=loader)
  File "/opt/conda/envs/cosyvoice/lib/python3.8/site-packages/yaml/__init__.py", line 81, in load
    return loader.get_single_data()
  File "/opt/conda/envs/cosyvoice/lib/python3.8/site-packages/ruamel/yaml/constructor.py", line 116, in get_single_data
    return self.construct_document(node)
  File "/opt/conda/envs/cosyvoice/lib/python3.8/site-packages/ruamel/yaml/constructor.py", line 120, in construct_document
    data = self.construct_object(node)
  File "/opt/conda/envs/cosyvoice/lib/python3.8/site-packages/ruamel/yaml/constructor.py", line 147, in construct_object
    data = self.construct_non_recursive_object(node)
  File "/opt/conda/envs/cosyvoice/lib/python3.8/site-packages/ruamel/yaml/constructor.py", line 188, in construct_non_recursive_object
    for _dummy in generator:
  File "/opt/conda/envs/cosyvoice/lib/python3.8/site-packages/ruamel/yaml/constructor.py", line 633, in construct_yaml_map
    value = self.construct_mapping(node)
  File "/opt/conda/envs/cosyvoice/lib/python3.8/site-packages/ruamel/yaml/constructor.py", line 429, in construct_mapping
    return BaseConstructor.construct_mapping(self, node, deep=deep)
  File "/opt/conda/envs/cosyvoice/lib/python3.8/site-packages/ruamel/yaml/constructor.py", line 244, in construct_mapping
    value = self.construct_object(value_node, deep=deep)
  File "/opt/conda/envs/cosyvoice/lib/python3.8/site-packages/ruamel/yaml/constructor.py", line 147, in construct_object
    data = self.construct_non_recursive_object(node)
  File "/opt/conda/envs/cosyvoice/lib/python3.8/site-packages/ruamel/yaml/constructor.py", line 183, in construct_non_recursive_object
    data = constructor(self, tag_suffix, node)
  File "/opt/conda/envs/cosyvoice/lib/python3.8/site-packages/hyperpyyaml/core.py", line 491, in _construct_name
    raise ImportError("There is no such entity as %s" % callable_string)
ImportError: There is no such entity as torchaudio.functional.compute_kaldi_pitch
2024-12-20 01:25:38,002 DEBUG Attempting to acquire lock<PHONE_NUMBER>41872 on /root/.triton/autotune/Fp16Matmul_2d_kernel.pickle.lock
2024-12-20 01:25:38,002 DEBUG Lock<PHONE_NUMBER>41872 acquired on /root/.triton/autotune/Fp16Matmul_2d_kernel.pickle.lock
2024-12-20 01:25:38,002 DEBUG Attempting to release lock<PHONE_NUMBER>41872 on /root/.triton/autotune/Fp16Matmul_2d_kernel.pickle.lock
2024-12-20 01:25:38,002 DEBUG Lock<PHONE_NUMBER>41872 released on /root/.triton/autotune/Fp16Matmul_2d_kernel.pickle.lock
2024-12-20 01:25:38,017 DEBUG Attempting to acquire lock<PHONE_NUMBER>42448 on /root/.triton/autotune/Fp16Matmul_4d_kernel.pickle.lock
2024-12-20 01:25:38,018 DEBUG Lock<PHONE_NUMBER>42448 acquired on /root/.triton/autotune/Fp16Matmul_4d_kernel.pickle.lock
2024-12-20 01:25:38,018 DEBUG Attempting to release lock<PHONE_NUMBER>42448 on /root/.triton/autotune/Fp16Matmul_4d_kernel.pickle.lock
2024-12-20 01:25:38,018 DEBUG Lock<PHONE_NUMBER>42448 released on /root/.triton/autotune/Fp16Matmul_4d_kernel.pickle.lock
E1220 01:25:38.572536<PHONE_NUMBER>79296 torch/distributed/elastic/multiprocessing/api.py:826] failed (exitcode: 1) local_rank: 0 (pid: 1248) of binary: /opt/conda/envs/cosyvoice/bin/python3.8
E1220 01:25:38.576262<PHONE_NUMBER>79296 torch/distributed/elastic/multiprocessing/errors/error_handler.py:136] no error file defined for parent, to copy child error file (/tmp/torchelastic_h0m8d_fd/1986_62c__ac6/attempt_0/0/error.json)
Traceback (most recent call last):
  File "/opt/conda/envs/cosyvoice/bin/torchrun", line 8, in <module>
    sys.exit(main())
  File "/opt/conda/envs/cosyvoice/lib/python3.8/site-packages/torch/distributed/elastic/multiprocessing/errors/__init__.py", line 347, in wrapper
    return f(*args, **kwargs)
  File "/opt/conda/envs/cosyvoice/lib/python3.8/site-packages/torch/distributed/run.py", line 879, in main
    run(args)
  File "/opt/conda/envs/cosyvoice/lib/python3.8/site-packages/torch/distributed/run.py", line 870, in run
    elastic_launch(
  File "/opt/conda/envs/cosyvoice/lib/python3.8/site-packages/torch/distributed/launcher/api.py", line 132, in __call__
    return launch_agent(self._config, self._entrypoint, list(args))
  File "/opt/conda/envs/cosyvoice/lib/python3.8/site-packages/torch/distributed/launcher/api.py", line 263, in launch_agent
    raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError:
============================================================
cosyvoice/bin/train.py FAILED
------------------------------------------------------------
Failures:
  <NO_OTHER_FAILURES>
------------------------------------------------------------
Root Cause (first observed failure):
[0]:
  time      : 2024-12-20_01:25:37
  host      : 78209458e1b2
  rank      : 0 (local_rank: 0)
  exitcode  : 1 (pid: 1248)
  error_file: /tmp/torchelastic_h0m8d_fd/1986_62c__ac6/attempt_0/0/error.json
  traceback : Traceback (most recent call last):
    File "/opt/conda/envs/cosyvoice/lib/python3.8/site-packages/torch/distributed/elastic/multiprocessing/errors/__init__.py", line 347, in wrapper
      return f(*args, **kwargs)
    File "cosyvoice/bin/train.py", line 101, in main
      configs = load_hyperpyyaml(f, overrides=override_dict)
    File "/opt/conda/envs/cosyvoice/lib/python3.8/site-packages/hyperpyyaml/core.py", line 188, in load_hyperpyyaml
      hparams = yaml.load(yaml_stream, Loader=loader)
    File "/opt/conda/envs/cosyvoice/lib/python3.8/site-packages/yaml/__init__.py", line 81, in load
      return loader.get_single_data()
    File "/opt/conda/envs/cosyvoice/lib/python3.8/site-packages/ruamel/yaml/constructor.py", line 116, in get_single_data
      return self.construct_document(node)
    File "/opt/conda/envs/cosyvoice/lib/python3.8/site-packages/ruamel/yaml/constructor.py", line 120, in construct_document
      data = self.construct_object(node)
    File "/opt/conda/envs/cosyvoice/lib/python3.8/site-packages/ruamel/yaml/constructor.py", line 147, in construct_object
      data = self.construct_non_recursive_object(node)
    File "/opt/conda/envs/cosyvoice/lib/python3.8/site-packages/ruamel/yaml/constructor.py", line 188, in construct_non_recursive_object
      for _dummy in generator:
    File "/opt/conda/envs/cosyvoice/lib/python3.8/site-packages/ruamel/yaml/constructor.py", line 633, in construct_yaml_map
      value = self.construct_mapping(node)
    File "/opt/conda/envs/cosyvoice/lib/python3.8/site-packages/ruamel/yaml/constructor.py", line 429, in construct_mapping
      return BaseConstructor.construct_mapping(self, node, deep=deep)
    File "/opt/conda/envs/cosyvoice/lib/python3.8/site-packages/ruamel/yaml/constructor.py", line 244, in construct_mapping
      value = self.construct_object(value_node, deep=deep)
    File "/opt/conda/envs/cosyvoice/lib/python3.8/site-packages/ruamel/yaml/constructor.py", line 147, in construct_object
      data = self.construct_non_recursive_object(node)
    File "/opt/conda/envs/cosyvoice/lib/python3.8/site-packages/ruamel/yaml/constructor.py", line 183, in construct_non_recursive_object
      data = constructor(self, tag_suffix, node)
    File "/opt/conda/envs/cosyvoice/lib/python3.8/site-packages/hyperpyyaml/core.py", line 491, in _construct_name
      raise ImportError("There is no such entity as %s" % callable_string)
  ImportError: There is no such entity as torchaudio.functional.compute_kaldi_pitch

将torchaudio的版本降到2.0.1就可以训练了。

#758


update and clean up go mod dependencies

@tariq1890: The following test failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

gencheck_client-go
16321be5909888265416c60e6f17650613d8bed5
link
/test gencheck_client-go

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

Thanks!
Can you run ‘make gen’ to get rid of the gen errors?
Also curious, does the patch version of the k8s client fix things that are required?

@Jason-ZW ALready done :)

Also curious, does the patch version of the k8s client fix things that are required?

Not that I know of. Just ensuring good dependency hygiene


100% CPU Lockup with subkey expiry on 5.3.3

Describe the bug
Migrated from redis to keydb for the subkey expiry feature. On most hosts this worked fine, but on one it got into a high CPU lockup state. Every host has different data, and I am unsure what is different about this host, but this was unexpected as redis has been used fine for ages.
Log Files
CPU was on 100%, running commands like info took up to 2 minutes, and continued to slow down.
Output of info:
keydbinfo.log
Running a bgsave here, the hang seemed to be before accepting the connection:
$$  date; time redis-cli -p 6389 bgsave; date
Wed 27 May 10:17:49 UTC 2020
(error) ERR Background save already in progress

real    1m59.435s
user    0m0.001s
sys     0m0.000s
Wed 27 May 10:19:49 UTC 2020

Log file for that time:
17160:17362:M 27 May 2020 10:17:23.961 - Client closed connection
17160:17362:M 27 May 2020 10:17:23.961 - Client closed connection
17160:17362:M 27 May 2020 10:17:23.962 - Client closed connection
17160:17362:M 27 May 2020 10:19:48.990 - Accepted <IP_ADDRESS>:38274
17160:17362:M 27 May 2020 10:19:48.991 - Accepted <IP_ADDRESS>:35410
17160:17362:M 27 May 2020 10:19:48.991 - Accepted <IP_ADDRESS>:48794
17160:17362:M 27 May 2020 10:19:48.991 - Accepted <IP_ADDRESS>:33910
17160:17362:M 27 May 2020 10:19:48.991 - Accepted <IP_ADDRESS>:39824
17160:17362:M 27 May 2020 10:19:48.991 - Accepted <IP_ADDRESS>:47330
17160:17362:M 27 May 2020 10:19:48.991 - Accepted <IP_ADDRESS>:49230
17160:17362:M 27 May 2020 10:19:49.055 * Background saving started by pid 32518
17160:17362:M 27 May 2020 10:19:49.056 - DB 6: 60007 keys (60003 volatile) in 65536 slots HT.
17160:17362:M 27 May 2020 10:19:49.056 - 21 clients connected (0 replicas), 328125192 bytes in use
17160:17362:M 27 May 2020 10:19:49.109 - Client closed connection
17160:17362:M 27 May 2020 10:19:49.109 - Client closed connection
17160:17362:M 27 May 2020 10:19:49.109 - Client closed connection
17160:17362:M 27 May 2020 10:19:49.109 - Client closed connection
17160:17362:M 27 May 2020 10:19:49.109 - Client closed connection
17160:17362:M 27 May 2020 10:19:49.109 - Client closed connection
17160:17362:M 27 May 2020 10:19:49.109 - Client closed connection
32518:17362:C 27 May 2020 10:19:51.363 * DB saved on disk
32518:17362:C 27 May 2020 10:19:51.377 * RDB: 46 MB of memory used by copy-on-write
17160:17362:M 27 May 2020 10:19:51.473 * Background saving terminated with success
17160:17362:M 27 May 2020 10:19:55.395 - DB 6: 60160 keys (60157 volatile) in 65536 slots HT.
17160:17362:M 27 May 2020 10:19:55.395 - 14 clients connected (0 replicas), 368008936 bytes in use
17160:17362:M 27 May 2020 10:19:59.079 - Accepted <IP_ADDRESS>:41888
17160:17362:M 27 May 2020 10:19:59.657 - Client closed connection
17160:17362:M 27 May 2020 10:20:02.029 - DB 6: 68047 keys (68044 volatile) in 131072 slots HT.
17160:17362:M 27 May 2020 10:20:02.029 - 14 clients connected (0 replicas), 375634136 bytes in use

Logs on start up have lots of this for many of the various subkeys we use, all key:null:
17309:17309:M 27 May 2020 10:40:39.349 # Corrupt subexpire entry in RDB skipping. key: (null) subkey: pra2
17309:17309:M 27 May 2020 10:40:39.349 # Corrupt subexpire entry in RDB skipping. key: (null) subkey: rrc2
17309:17309:M 27 May 2020 10:40:39.369 # Corrupt subexpire entry in RDB skipping. key: (null) subkey: j5
17309:17309:M 27 May 2020 10:40:39.369 # Corrupt subexpire entry in RDB skipping. key: (null) subkey: sn
17309:17309:M 27 May 2020 10:40:39.369 # Corrupt subexpire entry in RDB skipping. key: (null) subkey: gh
17309:17309:M 27 May 2020 10:40:39.369 # Corrupt subexpire entry in RDB skipping. key: (null) subkey: sv
17309:17309:M 27 May 2020 10:40:39.369 # Corrupt subexpire entry in RDB skipping. key: (null) subkey: sc
17309:17309:M 27 May 2020 10:40:39.369 # Corrupt subexpire entry in RDB skipping. key: (null) subkey: cn
17309:17309:M 27 May 2020 10:40:39.369 # Corrupt subexpire entry in RDB skipping. key: (null) subkey: ca

I think we use the same expiry time (14400) for subkeys as well as their parent keys. The parent keys will get refreshed occasionally, and the subkeys won't.
Flame graphs:
Database load took a very long time (multiple minutes, for 78k keys - 400mb), flamegraph in the zip below. Note exe is called keydb-serverOLD as we moved it out of the way to put redis back on, but it is 5.3.3
Then when running, here is a 1 minute flame graph (gdb confirmed the same functions):
flame-graph.zip
And screenshot:

To Reproduce
I tried to reproduce this by copying the database out and running it standalone on another port (had to do it before it expired). When data was being loaded from the RDB it was super slow.
It was also spamming out this in the logs while doing the ping benchmark below
Corrupt subexpire entry in RDB skipping. key: (null) subkey: CX
Here is some benchmark data (it got slower as it ran) before I ctrl+c'd it:
====== PING_INLINE ======
  100000 requests completed in 327.07 seconds
  50 parallel clients
  3 bytes payload
  keep alive: 1
  host configuration "save": 3600 100000
  host configuration "appendonly": no
  multi-thread: no

0.00% <= 0.1 milliseconds
0.54% <= 0.2 milliseconds
19.94% <= 0.3 milliseconds
59.78% <= 0.4 milliseconds
78.01% <= 0.5 milliseconds
85.70% <= 0.6 milliseconds
93.66% <= 0.7 milliseconds
95.92% <= 0.8 milliseconds
96.87% <= 0.9 milliseconds
97.18% <= 1.0 milliseconds
...
99.90% <= 27059 milliseconds
99.90% <= 36692 milliseconds
99.91% <= 36693 milliseconds
99.95% <= 48527 milliseconds
99.99% <= 48528 milliseconds
100.00% <= 48528 milliseconds
305.74 requests per second

Once it finished loading the DB:
11284:11284:M 27 May 2020 11:27:00.246 * DB loaded from disk: 340.658 seconds
11284:11284:M 27 May 2020 11:27:00.246 * Ready to accept connections
11284:18340:M 27 May 2020 11:27:00.246   Thread 0 alive.

Then it got fast again. This speedup didn't happen when the data was actively being written to when used live, it stayed slow and on 100% CPU.
$$  keydb-benchmark -p 6390
====== PING_INLINE ======
  100000 requests completed in 1.02 seconds
  50 parallel clients
  3 bytes payload
  keep alive: 1
  host configuration "save": 3600 100000
  host configuration "appendonly": no
  multi-thread: no

0.00% <= 0.1 milliseconds
3.82% <= 0.2 milliseconds
51.79% <= 0.3 milliseconds
80.65% <= 0.4 milliseconds
92.42% <= 0.5 milliseconds
97.84% <= 0.6 milliseconds
99.16% <= 0.7 milliseconds
99.50% <= 0.8 milliseconds
99.63% <= 0.9 milliseconds
99.75% <= 1.0 milliseconds
99.84% <= 1.1 milliseconds
99.85% <= 1.3 milliseconds
99.86% <= 1.4 milliseconds
99.88% <= 1.5 milliseconds
99.89% <= 1.6 milliseconds
99.92% <= 1.7 milliseconds
99.92% <= 1.8 milliseconds
99.93% <= 1.9 milliseconds
99.93% <= 2 milliseconds
99.95% <= 8 milliseconds
99.96% <= 9 milliseconds
100.00% <= 9 milliseconds
98039.22 requests per second

Changing back to redis, and wiping the rdb dump resolved the issue. When trying keydb again, the error hangs continued.

Awesome! That flame graph helps tremendously.

It seems the "corrupt subkey expire entry in RDB" can be caused by the set being expired while the database is offline/loading and a subkey expire is still set. This can be reproduced with:
keydb>  multi
keydb>  sadd testset val1
keydb>  expiremember testset val1 300
keydb>  pexpire testset 1
keydb>  debug reload
keydb>  exec

This reproduces on a single server without special config.

Thanks Ben!
So the issue here is:

Load an expired key
Notice the key is expired during load, and delete it (never gets added to the database)
Hit the subkey expire field for the key we just deleted and get very confused because there is no matching primary key for it.

The performance aspect seems to be primarily from the error handling being slow but I will do a bit more investigation on load perf in the happy path.

Fix e6a9a6b42 added to unstable + test.

This is in 6.0.8-RC


Filter out non-mutating open events

I had to do this in my setup to avoid infinite loop of reloading.

Thanks for your contribution! This will be released in v1.6.4 soon.


Add admin.apps.requests.cancel API

This pull request adds admin.apps.requests.cancel API method to the Web API client. The API is available only for Enterprise Grid orgs/workspaces.
Category (place an x in each of the [ ])

[ ] bolt (Bolt for Java)
[ ] bolt-{sub modules} (Bolt for Java - optional modules)
[x] slack-api-client (Slack API Clients)
[ ] slack-api-model (Slack API Data Models)
[ ] slack-api-*-kotlin-extension (Kotlin Extensions for Slack API Clients)
[ ] slack-app-backend (The primitive layer of Bolt for Java)

Requirements
Please read the Contributing guidelines and Code of Conduct before creating this issue or pull request. By submitting, you are agreeing to the those rules.

Thanks @mwbrooks for checking this!


Integrate oauth support if required/useful

We'll have to see how much this will work with desktop devices since bunq still doesn't have a desktop interface from which you could confirm an oauth request

On hold until this is looked into by bunq:
https://together.bunq.com/d/3436-oauth-returning-private-information-when-creating-a-session


[gherkin-lint] WIP - fleshing out a walking skeleton

Please send PRs against the gherkin-lint branch :-)


add support for all s3 canned-acls

Added
aws-exec-read
bucket-owner-read
bucket-owner-full-control
log-delivery-write s3
canned-acls from supported list here https://docs.aws.amazon.com/AmazonS3/latest/userguide/acl-overview.html#canned-acl

Description of your changes

Fixes #
I have:

[ ] Read and followed Crossplane's contribution process.
[ ] Run make reviewable test to ensure this PR is ready for review.

How has this code been tested

@rpoluri do these two work? It seems like AWS ignores them?
bucket-owner-read | Object | Object owner gets FULL_CONTROL. Bucket owner gets READ access. If you specify this canned ACL when creating a bucket, Amazon S3 ignores it.
bucket-owner-full-control | Object | Both the object owner and the bucket owner get FULL_CONTROL over the object. If you specify this canned ACL when creating a bucket, Amazon S3 ignores it.

if ACL grants contains bucket-owner-full-control  only - it is default private acl

@haarchri so we should merge as-is? I've tested it OK, except I didn't see any effect of the two mentioned.

Yes

okay it is safe
@rpoluri  thanks for contribution


TypeError: expected str, bytes or os.PathLike object, not NoneType

File "C:\Users\Fadak\Desktop\voice\Retrieval-based-Voice-Conversion-WebUI-main\infer-web.py", line 138, in
for root, dirs, files in os.walk(index_root, topdown=False):
File "C:\Program Files\Python39\lib\os.py", line 342, in walk
return _walk(fspath(top), topdown, onerror, followlinks)
TypeError: expected str, bytes or os.PathLike object, not NoneType
app could not recognized my gpu so as I saw in issues, i reinstalled pytorch and now it can recognize my gpu, but the app cant run succesfuly.

It seems that the program cannot find the index_root directory, which is assigned by index_root = os.getenv("index_root"), and the index_root in the .env file refer to the ./logs directory. Please check your ./logs directory exists under the project root directory.

Thank you for your help. I reinstalled the project and it works well


Custom IDP with Amplify and Auth at Edge

Hello there
after searching for hours the different issues (closed & open) on both Amplify project and here i couldn't find a similar solution.
Question
Did someone actually manage to make auth-at-Edge stack work with amplify integration and an custom IDP through Cognito ?
Quick problem overview
Amplify modify the state during a custom IDP signIn hence blocking the parse-auth process.
Context
We are using this stack for our applications with the default Cognito UI and it work great (both Cognito user-pool & Azure IDP connections).
We  would want to use a custom Login page with amplify to replace the hosted UI (same as shared here issue49)
We created a spa with amplify & the auth module accessible on /login and modified the check-auth to redirect on our custom login page. Setup the stack for cookie compatibility amplify.
The solution works with amplify when login through Cognito user-pool. Even though the parse-auth function is completely skip during the authentification process from amplify (Auth.signIn) (maybe i'm already doing something wrong here).
But when using the Auth.federatedSignIn({ customProvider: 'my-providerName-setup-inCognito'}); the state generated from the check-auth is not send to oauth2/authorize request. Amplify generate it's own state.
It does have the option to pass a customState attribute to the federatedSignIn methods. But even with that, it still modify it.
So when the authentification flow from IDP finish and that we end up on the parse-auth, the lambda refuse the connexion because the state cannot be parse as intended here
parsedState = JSON.parse(     Buffer.from(common.urlSafe.parse(state), "base64").toString()   );
Did someone managed a similar solution ?
Possible solutions
1 - Not using amplify and do all the IDP flow manually in the custom login page (still weird for a compliant amplify stack)
2 - Override amplify class to get and set the state correctly -> cumbersome & not easy to maintain later on
3 - Pass the state in a header during check-auth et set it again in parse-auth ->   Isn't that posing a security risk ? (like losing the whole interest of the state)
It's a solution i take by heart and would really wanna be able to achieve !
Thank you for the reading ❤

Well mate...
I was so focus to make it work along this stack i did not though of that.
So in that case both Cognito login & IDP login skip the parse auth lambda (and won't use the Auth-At-edge state generated by the check-auth).
Thanks for your quick reply. It worked (for the few test i just did).
Well i still need to do lot more test to see how it behave with the rest of the stack (refresh token and all). On top of that i would want to reduce the number of cookie amplify is returning, will try to modify a bit the behavior on that sens.
And need to try if i can redirect directly to my home instead of my login to avoid useless load + redirect after auth.
Will get back to you on Monday with my differents findings.
Thanks again ! you already made my weekend ❤

nb : I was already checking how to inject the auth-at-edge state in the amplify call. And then in the Edge get it back and transform it back in the correct format -_- So i'm avoiding quite a lot with this

Okay let's see! Your welcome and I hope this is a good path forward for you 👍🏻👍🏻

Hi @AzaxSyndrom

By default amplify generate cookie with a dot before the domain ".mycookiedomain" Reading about it, it's suppose to be a standard.

Actually it is better to not use the leading dot, because that makes cookies readable by subdomains too.
To get Amplify to work with that and prevent the double cookies that you see, use a single space " " as the Amplify cookieStorage domain setting:
Amplify.configure({
  Auth: {
    ...,
    cookieStorage: {
      domain: " ", // Use a single space " " for host-only cookies
      ...
    },
  },
});

Read the comment here for more details: https://github.com/aws-samples/cloudfront-authorization-at-edge/blob/996a07f6bee12b4b62503ac763d5465ee0a06a1c/src/cfn-custom-resources/react-app/index.ts#L64C23-L64C23

About the signOut only clearing ID token, it should clear all cookies actually so we need to dig in.
The reason this probably happens is because you cannot delete cookies, only overwrite them with an empty value and expiry set (then browser understands it must discard it). However, overwrite like that only works if you use the exact same cookie settings as you did when you initially set the cookies. So e.g. exact same path, domain, etc. Have a look if that is the case for you? Note that this PR did fix an issue like this: https://github.com/aws-samples/cloudfront-authorization-at-edge/pull/207 So make sure you are on a version of Auth@Edge that includes that fix (v2.1.2 onwards)

Hi
getting back to you about those signOut cookies and the cookieDomain. It all works
With the latest version of amplify the clearing cookie problem is gone.
We have a working solution.
Thanks again !


66: Integrate DurationParameter as a Python timedelta.

Looks good, but also got the feeling that it is quite some effort to make Python types to be Protobuf compatible after looking #65 #68, but it is how it is.

@cwang39403  Yeah we have gathered quite a bit of code but it also does quite a bit for us. While it may look mostly as if it is only to convert from/to protobuf, it actually does a bit more:

Pack/unpack Python objects into protobuf Struct (which doesn't support int, timedelta or datetime)
Describe how a parameter should be named and what it does in a workflow definition.
How to build up this parameter description based on a json file or a protobuf message describing a workflow definition.
How to dump this parameter to a protobuf message describing a workflow definition.

All in all, it is tightly coupling the workflow definition stuff which is shared from the orchestrator to the SDK, it also helps with validating the params supplied to a job submission and how to parse the config at the worker. It looks deceivingly bloated, but it also appears to fulfill quite a few functionalities. It still feels weird to me as well, but I do not have a better idea currently :P


torch undefined symbol: iJIT_NotifyEvent error

Describe the bug
Received the following error when executing esm-fold using the image
# on the host machine
> % docker run --rm $USER/esmfold:test -h
Traceback (most recent call last):
  File "/home/vscode/.conda/envs/py39-esmfold/bin/esm-fold", line 5, in <module>
    from esm.scripts.fold import main
  File "/home/vscode/.conda/envs/py39-esmfold/lib/python3.9/site-packages/esm/__init__.py", line 8, in <module>
    from .data import Alphabet, BatchConverter, FastaBatchedDataset  # noqa
  File "/home/vscode/.conda/envs/py39-esmfold/lib/python3.9/site-packages/esm/data.py", line 12, in <module>
    import torch
  File "/home/vscode/.conda/envs/py39-esmfold/lib/python3.9/site-packages/torch/__init__.py", line 202, in <module>
    from torch._C import *  # noqa: F403
ImportError: /home/vscode/.conda/envs/py39-esmfold/lib/python3.9/site-packages/torch/lib/libtorch_cpu.so: undefined symbol: iJIT_NotifyEvent

which looks like torch installation problem
To Reproduce
Start the container
docker run --rm -it --entrypoint zsh $USER/esmfold:base

Then inside the container
> conda activate py39-esmfold
(py39-esmfold) > python -c 'import torch'

Expected behavior
Expected to receive the same error:
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/vscode/.conda/envs/openfold-venv/lib/python3.9/site-packages/torch/__init__.py", line 202, in <module>
    from torch._C import *  # noqa: F403
ImportError: /home/vscode/.conda/envs/openfold-venv/lib/python3.9/site-packages/torch/lib/libtorch_cpu.so: undefined symbol: iJIT_NotifyEvent

Screenshots
If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):

OS: Ubuntu 22.04

Additional context
Add any other context about the problem here.

fixed in v0.0.2


WIP: Auto-generate embeddings for documents and queries

This PR adds support for generating embeddings for documents and queries
How to use

As a quirk of the current implementation, have the field "_vectors": null in at least one of the documents of your dataset.
Run the indexing like normal
To perform vector search requests with auto-generation of the embeddings for the query, use the vector field of the request, but pass it the string value of your query.

Example: { "vector": "batman" }
TODO

[x] Decide on model provisioning strategy
[x] Allow configuring model
[ ] In search response, reinstate similarity computation and display vector that was computed
[x] Decide sane defaults
[x] Conditionally enable autoembedding depending on settings
[x] no longer require `"_vectors": null"

Hi @dureuill!
The auto-generated embedding looks really promising!
I was wondering about your planning on how the model will be attached to an index? Will all indexes using the same model or an index may have a different model than the other?
Assuming we have two indexes so we can perform image similarity search and text search :

Index for storing movies poster/image embedding using openai/clip-vit-base-patch32
Index for storing movies metadata embedding in multilingual using sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2

Will that be possible? Thank you!
Hats off to your work!

hi @hdytsgt :wave:
Thanks for the encouraging message ❤

I was wondering about your planning on how the model will be attached to an index? Will all indexes using the same model or an index may have a different model than the other?

A single index may use multiple models. Different indexes may use different models.

Assuming we have two indexes so we can perform image similarity search and text search :

Note that for the upcoming v1.6, we're only supporting the auto embedding of text via BERT models, no image similarity. You will be able to do image similarity by providing the embeddings yourself, like previously.
It would look like this:

Enable the experimental vector store feature:

 curl \
  -X PATCH 'http://localhost:7700/experimental-features/' \
  -H 'Content-Type: application/json' -H 'Authorization: Bearer foo' \
--data-binary '{ "vectorStore": true
  }'

Setup the embedders for your index of interest by sending the settings for your index:

{
  "embedders": {
    "default": {
      "source": { "huggingFace": {}},
      "documentTemplate": {
        "template": "A movie titled '{{doc.title}}' whose description starts with {{doc.overview|truncatewords: 20}}"
      }
    },
    "image": {
      "source": { "userProvided": { "dimensions": 512 } }
    },
    "translation": {
      "source": { "huggingFace": {
        "model": "sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2"
      }},
      "documentTemplate": {
        "template": "A movie titled '{{doc.title}}' whose description starts with {{doc.overview|truncatewords: 20}}"
      }
    }
  }
}

You then add some documents:

[{"id":9607,"title":"Super Mario Bros.","overview":"Mario and Luigi, plumbers from Brooklyn, find themselves in an alternate universe where evolved dinosaurs live in hi-tech squalor. They're the only hope to save our universe from invasion by the dino dictator, Koopa.","genres":["Science Fiction","Adventure","Comedy","Family"],"poster":"https://image.tmdb.org/t/p/w500/lE8KFoPErukKBRTy9De25ClpxH9.jpg","release_date":738547200,
"_vectors": {
  "image":  // need to specify the embedding of correct dimension for the `image` category
[0.0,
 0.0,
 ...,
 0.0,
 0.0]
}
// no need to specify for the `default` and `translation` embedders, the embedding will be computed automatically
},
{"id":9323,"title":"Ghost in the Shell","overview":"In the year 2029, the barriers of our world have been broken down by the net and by cybernetics, but this brings new vulnerability to humans in the form of brain-hacking. When a highly-wanted hacker known as 'The Puppetmaster' begins involving them in politics, Section 9, a group of cybernetically enhanced cops, are called in to investigate and stop the Puppetmaster.","genres":["Action","Animation","Science Fiction"],"poster":"https://image.tmdb.org/t/p/w500/9gC88zYUBARRSThcG93MvW14sqx.jpg","release_date":816652800,
"_vectors": { "image":
[0.0,
 0.0,
 ...,
 0.0,
 0.0]
}}]

Lastly, you can do a search:

{
  "q": "Plumbers and dinosaurs",  // will be embedded by the default embedder
  "limit": 20,
  "hybrid": {
    "semanticRatio": 1.0 // do a full vector search
  }
}

to get a translation and benefit from the multi lingual capabilities of the paraphrase-multilingual-MiniLM-L12-v2 model, you would specify the embedder:
{
  "q": "Le film avec les plombiers et les dinosaures",  // will be embedded by the "translation" embedder
  "limit": 20,
  "hybrid": {
    "semanticRatio": 1.0,  // do a full vector search
    "embedder": "translation" // provide embedder name here
  }
}

lastly, to do an image similarity search, you will need to pass the embedding yourself:
{
  "vector": [0.0, ..., 0.0],  // provided by you
  "limit": 20,
  "hybrid": {
    "semanticRatio": 1.0,  // do a full vector search
    "embedder": "image", // compare with the "image" embeddings from the index
  }
}

We are interested in adding image embedding eventually, but this poses some design questions (we are handling text so far in Meilisearch) and won't make the cut for v1.6

Hats off to your work!

Thank you 😊  I'm racing to finish the work for v1.6-RC.0 that's planned next week!

Oh wow! Searching while defining which model to use is actually pretty cool and handy! 🎉
Hopefully that doesn't affect the performance too, considering loading the uncached model is relatively slow.
Thank you for the detailed explanation, can't wait to try v1.6.
Awesome work @dureuill!

Superseded by #4226


Uncaught ReferenceError: vendor is not defined

Uncaught ReferenceError: vendor is not defined
    at Object.0 (external "vendor":1)
    at __webpack_require__ (bootstrap cdec395…:659)
    at fn (bootstrap cdec395…:83)
    at Object.41 (index.js from dll-reference vendor:1)
    at __webpack_require__ (bootstrap cdec395…:659)
    at fn (bootstrap cdec395…:83)
    at Object.<anonymous> (client:3)
    at Object../node_modules/webpack-dev-server/client/index.js?http:/localhost:1212 (client?fd75:181)
    at __webpack_require__ (bootstrap cdec395…:659)
    at fn (bootstrap cdec395…:83)
0	@external "vendor":1
__webpack_require__	@bootstrap e86c1e1…:659
fn	@bootstrap e86c1e1…:83
41	@index.js from dll-reference vendor:1
__webpack_require__	@bootstrap e86c1e1…:659
fn	@bootstrap e86c1e1…:83
(anonymous)	@client:3
./node_modules/webpack-dev-server/client/index.js?http:/localhost:1212	@client?fd75:181
__webpack_require__	@bootstrap e86c1e1…:659
fn	@bootstrap e86c1e1…:83
93	@bundle.js:25006
__webpack_require__	@bootstrap e86c1e1…:659
module.exports../app/actions/notes_actions.js.Object.defineProperty.value	@bootstrap e86c1e1…:708
(anonymous)	@bootstrap e86c1e1…:708

Sorry for the terrible formatting.
This error appears every half dozen or so hot-reloads. It resolves on reloading again. Not a big deal, but I don't see it reported yet as best I can tell.

This has to do with the DLL config. Try running npm run build-dll again. Did you change of the DLL configs?

Have not been able to reproduce, tried running npm run build-dll but same issue. Perhaps it's an issue with a specific package I'm using? Not sure. I wonder if anyone else will have this issue or if it's just me.

To temporarily circumvent this issue, make vendor and empty array

Thanks very much for the tip. I'll try it and report back.

Unfortunately this leads to the following:
Webpack has been initialised using a configuration object that does not match the API schema.
I wouldn't worry about this possible bug. If other people have it  we should find out in the next few weeks.

Interesting. Try commenting this line and see if it works

Unfortunately commenting out that line leads to the original error nonstop.
I have noticed one way this error occurs is on virtually any manual reload of the page. It then requires a full restart of the boilerplate, which makes this issue substantially more difficult to work through than I originally thought.

Seems to always been accompanied by the following error (but I cannot confirm yet that this is always the case):
[14096:0412/135953.728:ERROR:CONSOLE(6639)] "Uncaught (in promise) TypeError: Cannot read property 'scriptId' of null", source: chrome-devtools://devtools/bundled/inspector.js (6639)
I am seeing this error on almost every refresh manual or automatic now requiring a full reload of the boilerplate.

It seems like what's happening is that the hot-reload process is somehow setting read-only in the Windows file properties for some files in the node_modules subdirectories (top level and app folder level)
When this happens, I have to close the boilerplate, clear the read-only flag from all the files in the directories, and then run npm run dev again.
Without that I get stuck with the error indefinitely.

I'm wondering if it's related to these:
https://github.com/gaearon/react-hot-loader/issues/526
https://github.com/webpack/webpack/issues/4581

I've migrated other projects to the dll version of this boilerplate and I can confirm that react-hot-loader still works. Not sure why its broken for you

This may be due to native node modules, for example electron-spellchecker
I haven't had enough time to confirm this, but in the short time I had after removing electron-spellchecker from my application, I have not seen this error.

I have not risked adding back electron-spellchecker yet, but since removal I have only had the error on manual refreshes, and it is consistently resolved by a second refresh.

I ran into the same issue today. Very confusing. It's inconsistent but started flaring up badly enough that I found a bandaid:
<script>
  function insertBundle () {
    // Dynamically insert the renderer process
    const script = document.createElement('script')
    const port = process.env.PORT || 1212
    script.src = (process.env.HOT)
      ? 'http://localhost:' + port + '/dist/bundle.js'
      : './dist/bundle.js'
    // HACK: Writing the script path should be done with HtmlWebpackPlugin
    document.body.appendChild(script)
  }

  // Dynamically include DLL if in development
  if (process.env.NODE_ENV === 'development') {
    const dllScript = document.createElement('script')
    dllScript.src = '../dll/vendor.dll.js'
    document.body.appendChild(dllScript)
    setTimeout(insertBundle, 100)
  } else {
    insertBundle()
  }
</script>

My guess, which is a total guess, is that there is some kind of weird race condition between which of the 2 dynamically inserted scripts get executed first.
I started by just hard coding
<script src='../dll/vendor.dll.js'></script>

and commenting out the dynamic insertion of the DLL file. That worked 100% of the time. So I started playing with the setTimeout approach. I played with a few different wait times, and found that, at least on my machine, as I descended below 100ms it felt like bundle.js was increasingly winning the race.
Odd, but it works for now. Hope this is helpful @Slapbox. Gonna get back to it!

Thanks very much @afrojas I'll definitely give it a try and report back!

@Slapbox did you resolve this? One way to debug this is to view the actually dll file. I would recommend using a very fast text editor like vscode or sublime. It will crash atom for sure

It's not fully resolved on my end. I haven't had time to try suggested resolutions yet. I have resolved it enough that I can work around it until I can sync with the master repo again. I find doubletapping F5 reduces the concurrences of the error on manual refreshes.

Fixed in 0.11.0

Oh my god a versioned release!? I just didn't even think we were doing versions anymore.

I didnt know i had permission on github releases to push releases even tho I'm not the owner. Will start pushing releases more frequently

Will start pushing releases more frequently

@amilajack Please do. Keeping up to date with boilerplate is certainly a pain.

I'm also going to work on a tool that can automate parts of the migration process. Also going to work on a UPGRADE.md guide

The current version of app.html, namely creating an array of script srcs and writing them to the DOM with the defer attribute, fixed the behavior I, and I believe @Slapbox, were seeing.
Thanks @amilajack!

Only Rebuild webpack vendor.js and Clear your history Browse


FID on CelebA 64x64

Would you like to share how to generate .npz file for evaluating performance on CelebA 64x64? When evaluating CelebA 64x64, you claimed in paper that you used full training set but you said you randomly picked up 50k images from training set on github readme, which is confusing. Can you show more details?
Here is how I did.
First of all, I generate .npz file as  reference:
import numpy as np
from PIL import Image
import random
import math
import blobfile as bf
from tqdm import tqdm

def _list_image_files_recursively(data_dir):
    results = []
    for entry in sorted(bf.listdir(data_dir)):
        full_path = bf.join(data_dir, entry)
        ext = entry.split(".")[-1]
        if "." in entry and ext.lower() in ["jpg", "jpeg", "png", "gif"]:
            results.append(full_path)
        elif bf.isdir(full_path):
            results.extend(_list_image_files_recursively(full_path))
    return results

def center_crop_arr(pil_image, image_size):
    # We are not on a new enough PIL to support the `reducing_gap`
    # argument, which uses BOX downsampling at powers of two first.
    # Thus, we do it by hand to improve downsample quality.
    while min(*pil_image.size) >= 2 * image_size:
        pil_image = pil_image.resize(
            tuple(x // 2 for x in pil_image.size), resample=Image.BOX
        )

    scale = image_size / min(*pil_image.size)
    pil_image = pil_image.resize(
        tuple(round(x * scale) for x in pil_image.size), resample=Image.BICUBIC
    )

    arr = np.array(pil_image)
    crop_y = (arr.shape[0] - image_size) // 2
    crop_x = (arr.shape[1] - image_size) // 2
    return arr[crop_y : crop_y + image_size, crop_x : crop_x + image_size]

local_images = _list_image_files_recursively("./img_align_celeba")
len(local_images)

num_samples = 50_000
random_indices = random.sample(range(len(local_images)), num_samples)
assert len(set(random_indices)) == num_samples
resolution = 64
arrs = []
for idx in tqdm(random_indices):
    path = local_images[idx]
    with bf.BlobFile(path, "rb") as f:
        pil_image = Image.open(f)
        pil_image.load()
    pil_image = pil_image.convert("RGB")

    arr = center_crop_arr(pil_image, resolution)

    arrs.append(arr)

arrs = np.stack(arrs)
np.savez("celeba64_50k.npz", arrs)

Then I run the evaluation script:
python evaluations/evaluator.py celeba64_50k.npz 1000t_50000x64x64x3.npz

Here 1000t_50000x64x64x3.npz stores samples generated by your CelebA 64x64 ADM-IP.pt using 1000 steps. The result is:
Inception Score: 3.248073101043701
FID: 9.09117351570518
sFID: 37.492244654769365
Precision: 0.45012
Recall: 0.61274

Can you help me figure out where I did wrong? I will be grateful about that.

My bad. After checking your scripts and readme in ./datasets, I got the correct FID:
Inception Score: 3.2480759620666504
FID: 1.5056962961214708
sFID: 3.43160755706333
Precision: 0.67378
Recall: 0.6176486557189325

However, this raise another issue. As you can see, I got 9.09 for FID before when I use PIL.Image.resize, this is what DiffusionVAE did. It seems that resize functions of cv2 and PIL yield different results.


Update OSX pools in release/3.1 and release/5.0 branches

OSX 10.14 pool is being retired, we should update (probably to latest: https://devblogs.microsoft.com/devops/hosted-pipelines-image-deprecation/).

Addressed with https://github.com/dotnet/source-build/pull/2701 and https://github.com/dotnet/source-build/pull/2702.


Return taxonomy

As @ElDeveloper pointed our in #1 it would be very handy to obtain taxonomic lineage information for every inserted fragment. This is now realized by a third output.
We obtain the lineages by simply traversing fragment node to root node in the insertion tree and collect all taxonomic labels.
Note: resulting lineages are only as good as the provided reference tree!

Ready for a review @wasade @mortonjt


weaveworks/weave-kube:2.0.5 image missing in DockerHub.

What you expected to happen?
docker pull weaveworks/weave-kube:2.0.5 completes successfully.
What happened?
$ docker pull weaveworks/weave-kube:2.0.5
Error response from daemon: manifest for weaveworks/weave-kube:2.0.5 not found

How to reproduce it?
docker pull weaveworks/weave-kube:2.0.5

Version 2.0.5 in the process of being released.
Would be interested to know what led to you attempting to pull the image.

Got notified of a new tag via https://about.sibbell.com .

It's there now, closing,

I don't know anything about sibbell, but we create a tag first, then build all the software and test it, then make the release visible on GitHub.  So it would be better if they reacted to the latter event.

Thanks for the info. Understood - wont bother you again :)
Sibell is just a s small web service which scans starred repo for new tags/releases and sends emails when a new one is detected. It detected the tag (and eventually the release).


feat: Use unified view

Issue #, if available: n/a
Description of changes:

We should use same view when no nodes are found/ready. This PR adds helpStyle bar at the bootom matching when listing nodes.
Also local build updated ATTRIBUTION versions which are updated with this PR as well.
Bump the deps

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Cheers for mergeing @tzneal 👍. Would you mind cutting fresh release?

Done, waiting on the homebrew PR to merge still though.


I receive "Availability Status: Closed" alerts, how to filter them?

I receive "Availability Status: Closed" notifications for the following cmd:
camply  campsites --rec-area 2931 --start-date 2023-06-09 --end-date 2023-06-13 --notifications pushbullet --search-forever --equipment RV 0 --campground 232461
I can see the code checks the status against CAMPSITE_UNAVAILABLE_STRINGS constant list, "Closed" is in this list, but I still get the alerts where Status is "Closed", how to filter them further to make sure I receive alerts only for "Available" sites?

Hey @vallkor , The "Closed" logic was just added yesterday on 0.16.2, can you confirm you're running with the latest version of camply (0.18.0)?
camply campsites --rec-area 2931 --start-date 2023-06-09 --end-date 2023-06-13 --continuous --equipment RV 0 --campground 232461 --debug

Thank you, @juftin , issue has been fixed with 0.18.0 version. I had an older package installed using Pip and didn't know my python is still using it, I updated packages and it is now fixed. Thanks for a great tool!