Track source location in change detection

Objective

Make it possible to know what changed your component or resource.
Common need when debugging, when you want to know the last code location that mutated a value in the ECS.
This feature would be very useful for the editor alongside system stepping.

Solution

Adds the caller location to column data.

Testing

The component_change_detection example now shows where the component was mutated:

2024-06-26T06:56:42.980958Z  INFO component_change_detection: Entity { index: 1, generation: 1 } changed: Ref(MyComponent(2.0)) from: "examples/ecs/component_change_detection.rs:34:23"

Changelog

Added source locations to ECS change detection.

Migration Guide

Added caller field to many internal ECS functions used with change detection. Use Location::caller() to provide the source of the function call.

Mut is one of the hottest paths in Bevy. I'm concerned about the performance implications of introducing anything (even the cheapest ops) to it. I'd like to either see compelling evidence that this won't meaningfully affect performance, especially given that this is a "debugging tool". I'd feel much more comfortable if this was behind a feature flag.

I really like this feature and think it'll be very useful, but I'll second the request for putting it behind an off-by-default feature flag.

Yup, I assumed as much. I was hoping to get a read on whether this is even an acceptable approach to take for the feature.

Is there a reason you're de referencing the static reference you get from Location::caller? Couldn't you just pass the reference around?

Location is two u32s and one &str, making it 16 bytes on 64-bit platforms. Storing a reference instead would bring it down to 8 bytes, which might have some mild performance gains because it would fit in a register.
Furthermore, why do you use core::panic::Location as compared with the std variant?

static reference: I started with this, but was having trouble with the deref making it hard to update the field, so I switched to an owned value to get it working. I agree it should be fixed if this is going to be merged.
using core: because it can't hurt? I wasn't sure if bevy_ecs was every intended to be used in a no_std context, and figured it would be safer to use core for a type so ubiquitous.

2. Changes from commands completely break this approach because commands are deferred, and the call stack can't be traced to the calling location by the time the command is actually executed.

Is it possible to find the Location in Commands::insert, then pass that forward to the actual command?

using core: because it can't hurt? I wasn't sure if bevy_ecs was ever intended to be used in a no_std context, and figured it would be safer to use core for a type so ubiquitous.

I would like this, one day :) Probably core + alloc, but still! Not a priority, but might as well do it when there's no cost to it.

Good call! That looks pretty straightforward.

Spent far to long trying to figure out why a camera transform from glb was getting reset, was correct on spawn, next frame its Identity, this found the issue in seconds.  Was calling remove_parent_in_place before TransformPropagate had ran.  Huge help.

This is close to finished, it just needs a system to track the caller from Commands. That's past my capabilities, but hopefully someone else is up for the task!

I've updated this to now work with commands, as well as a pass on naming, and updated the change detection example to more effectively teach these concepts.

I like this and I want it. Should Commands::add and Command::push be track_caller so that custom user commands can provide call tracing? Or would that be something user should do on the methods of their extension trait.

If we do want to support custom commands like this, it would need more documentation. I feel like this is probably follow-up PR territory.

follow-up PR territory.

Fair enough, agreed.

Is there a reason you're de referencing the static reference you get from Location::caller? Couldn't you just pass the reference around?

Location is two u32s and one &str, making it 16 bytes on 64-bit platforms. Storing a reference instead would bring it down to 8 bytes, which might have some mild performance gains because it would fit in a register.

at the risk of sounding incredibly pedantic: 24 bytes. &str is the pointer and the length, so it is already 16 bytes.
remember that this is a pretty magical struct that gets willed into existence by the compiler during codegen.

insert_with_caller & friends aren't pub. So the actual question is: Do we want to make them pub?

I think so. This will be needed for custom impls, as you said. We can document the context around this on the with_caller methods. I wish there was a way to make this pattern nicer, but alas.

at the risk of sounding incredibly pedantic: 24 bytes. &str is the pointer and the length, so it is already 16 bytes.

Hah, I forgot about wide pointers. Thanks!

Thank you to everyone involved with the authoring or reviewing of this PR! This work is relatively important and needs release notes! Head over to https://github.com/bevyengine/bevy-website/issues/1666 if you'd like to help out.


Generate social media gifs

Options for this issue:

Keep it simple and generate gifs as we always have, or
Try to automatically generate custom gifs for each WSC, e.g., for use with their twitter accounts. This may involve a shell/python script to use headless browser to render a d3 in canvas (turn into raster) then capture output as png/gif.

Notes on option 2: last summer's interns took this approach: Node script (written in javascript). uses a node package to create a png from an svg. Mary says, "There is a package that can generate an gif from an svg but I don't know how similar it is to the other one."

I think the relevant work is here: https://github.com/USGS-VIZLAB/active-flood-viz/blob/master/floodviz/thumbnail/thumbnail.js

would Option 3: generate gifs w/ R be different than option 1? (is option 1 a screen capture?)

Seems we can use gifsicle to have different delays per frame, which is nice because it lets us have a longer running gif w/o increasing the file size (e.g., frames during transitions are short, frames looking at a category are longer)
gifsicle -b a.gif -d50 "#0" "#1" -d100 "#2" "#3"

which sets the delays of frames 0 and 1 to 50, and frames 2 and 3 to 100

I am working on coding this up for each state+PR.

using a state specific UTM zone to get each state to look good. I am using the median UTM number from this spreadsheet
I think the bounding boxes of states will let us get away with ~3 different kinds of gif layouts: 1) tall (California), 2) flat (North Carolina), or squarish (Colorado). We should be able to do some aspect ratio calc with the bounding box of the sp object.
I am adding a USGS watermark and a url to the main viz.
Here is a starting point for North Carolina w/o pies or legend etc:

Not sure if there is a single tool way to do this (this example is imagemagick and gifsicle):
use R to generate ordered pngs, then imagemagick to turn them into a gif:
convert -loop 0 -delay 10 01_CA_water_use_pie0.png 01_CA_water_use_pie1.png 01_CA_water_use_pie2.png 01_CA_water_use_pie3.png 01_CA_water_use_pie4.png 01_CA_water_use_pie5.png 02_CA_water_use_irrigation0.png 01_CA_water_use_pie5.png 01_CA_water_use_pie4.png 01_CA_water_use_pie3.png 01_CA_water_use_pie2.png 01_CA_water_use_pie1.png out.gif

then gifsicle to change the frame delays for individual frames:
gifsicle -b out.gif -d150 "#0" -d10 "#2" "#3" "#4" "#5" "#6" "#7" "#8" "#9" "#10" "#11"

delay of 150 for "pause" frames and 10 for "transition" frames

I have the three different types implemented on https://github.com/jread-usgs/viz-scratch/tree/wu_gif_transitions, which are tall (e.g., CA, IN), wide (e.g., NC, TN), squarish (e.g., CO, NE). Things seem to be in good shape for social media release. A few changes I would like to make before bringing the code in: 1) re-order the categories to go from largest to smallest for each state, 2) transition between the pie and three largest categories, and 3) mess around w/ pause durations and frame numbers to get the right feel.

Any reason this is over in scratch and not in this repo? Like how we ported the code for the national map over here?

no, no reason. I can move it over as part of progressing through the other GIF tasks.

This code is now in this repo


Fix allocated node counter and open/close node linked lists

Hi, I'm Sergio Sota, from Spain. This is my very first pull request. Sorry for my terrible English :-P I have used your astar algorithm implementation in a bot for an AI contest (check here https://github.com/marcan/lighthouses_aicontest) that will take place this july on Bilbao (Euskal Encounter 26)
The thing is... sometimes somewhere some bot hung (I run many instances of it in parallel with different parameters in order to train) So I begun the bug hunt... it took me almost 2 weeks but finally I think I have found the problem.

When there is no more memory (I used the default 1000 node size) the program tries to free all nodes: the open list, the closed list, the goal node... but there is another node that doesn't get freed: the "n" node (the one pulled out from the open list at the beginning and used here and there) I used EnsureMemoryFree method assertion just after that and the program stops (it's not big deal since the astar object gets destroyed once you finish your search)

If you select not to use FSA memory allocator... when a node gets freed there is a "m_AllocateNodeCount--" but when a node gets allocated there is no "m_AllocateNodeCount++"

This is, I believe, the most important issue: all nodes, once allocated in memory, have references to other nodes... so we can't realloc any of them or the solution chain could be broken. e.g. (A) -> (B) -> (C) if we realloc B to other place it will still point to C, but now A is pointing to the position B occupied. If FSA doesn't use that block then we are lucky, but if it is somehow refilled with other block D... then we have (A) -> (D) -> (?)
In other words... when we have a successor that will replace other old open/close node (because this successor has a better path) we must change the old/close node in place (because other node could be pointing to it) Right now the successor is pulled in as a new node and the old open/closed node is erased... and that could broke the chain.

Well, I have tested the modified version and it is working ok so far! Thanks for you time and for your great project :-)

Hi Sergio. Many thanks for using my code and for spending all that time diagnosing and fixing this issue! I will spend some time reviewing the code this weekend.

I have checked the changes and didn't see any problems.
Thanks again, Sergio

Great! Thank you! :-D


Added pop up messages for installs

Requested in issue #2116

How does this behave for users who want to automatically apply an exported winutil config? Will this block the execution of the config import and therefore require user input before any further tweaks get applied?
If I recall correctly when importing a config, the applications are installed first.
If so, this would break the "zero touch" workflow for importing config files

How does this behave for users who want to automatically apply an exported winutil config? Will this block the execution of the config import and therefore require user input before any further tweaks get applied? If I recall correctly when importing a config, the applications are installed first. If so, this would break the "zero touch" workflow for importing config files

It shouldn't affect it as it was only added to say that installs are done at the end.

All right, I've taken another look at the code, and I have two improvements.

Move the line $sync.ProcessRunning = $False in front of the Popup.
-- The installs are started in a runspace so this would block execution until the button is pressed. (This doesn't become immediately clear, because the program installed are started last when importing a config)
Move the Console Output in front of the Popup as well. (Also relevant for #2134)
-- With the console output after the Popup, the terminal message will only be printed once the user clicks on the popup, so your order of Log-Messages in the terminal can get messed up if you wait.

@ChrisTitusTech I do not know how to re commit after changes to this PR.

You can move it to draft form then bring it back.
As far as the PR, I don't think a popup is a great solution and I'm struggling to find what would be. I think the best course from here is something that changes the GUI application directly that the user is staring at.


Add output file to export-schema

Feature Request Type

[ ] Core functionality
[ ] Alteration (enhancement/optimization) of existing feature(s)
[ X ] New behavior

Description
Currently, schema export only supports printing to stdout. I would like to save to a file, and while it's possible to use redirection to do so, I think saving schema to a file is a common enough use case that it should have native support. I've made a diff here: https://github.com/strawberry-graphql/strawberry/compare/main...stillmatic:strawberry:main and am happy to submit that as a PR.

I think that's fair 😊
happy to review the PR, I'd already suggest to use pathlib (and write a test) 😊


Let instant execution capture Task.timeout, Task.enabled and Task.onlyIf

@bot-gradle test this

OK, I've already triggered ReadyForMerge build for you.

@bot-gradle test this

OK, I've already triggered ReadyForMerge build for you.


Example with no-collation is misleading and incorrect in Fabric DW

Source: https://github.com/MicrosoftDocs/sql-docs/blob/live/docs/t-sql/statements/collation-precedence-transact-sql.md
There is one misleading issue in Collation Precedence - SQL Server | Microsoft Learn doc.
In the examples, the following code should not return error if GreekCol and Latingol have different collations:
SELECT (CASE WHEN id > 10 THEN GreekCol ELSE LatinCol END) COLLATE Latin1_General_CI_AS
FROM TestTab;

However, this is true only if both columns are NVARCHAR. If we replace the NVARCHAR column type with VARCHAR and even if we put UTF8 collation on VARCHAR, the expression will fail.
This is a big problem for FabricDw where nvarchar is not supported and all columns are varchar. A reader might try to do this in fabric DW and since they cannot create nvarchar columns they would believe that FabricDw doesn't work.
We need to clarify that this is applicable only to NVARCHAR, and the example is not applicable to Fabric DW.
The full repro is below:
DROP TABLE IF EXISTS TestTab
GO
CREATE TABLE TestTab (
   id int,
   GreekCol varchar(10) collate greek_ci_as,
   LatinCol varchar(10) collate latin1_general_cs_as,
   NGreekCol nvarchar(10) collate greek_ci_as,
   NLatinCol nvarchar(10) collate latin1_general_cs_as,
   U8GreekCol varchar(10) collate Greek_100_CI_AS_SC_UTF8,
   U8LatinCol varchar(10) collate Latin1_General_100_CI_AS_SC_UTF8
   )
INSERT TestTab VALUES (1, N'A', N'a', N'A', N'a', N'A', N'a');
GO

--NVARCHAR
SELECT (CASE WHEN id > 10 THEN NGreekCol ELSE NLatinCol END) COLLATE Latin1_General_CI_AS
FROM TestTab;
-- Works fine like in documentation.

--VARCHAR non-UTF8
SELECT (CASE WHEN id > 10 THEN GreekCol ELSE LatinCol END) COLLATE Latin1_General_CI_AS
FROM TestTab;
--Msg 457, Level 16, State 1, Line 39
--Implicit conversion of varchar value to varchar cannot be performed because the collation of the value is unresolved
--due to a collation conflict between "Latin1_General_CS_AS" and "Greek_CI_AS" in CASE operator.

--VARCHAR UTF8
SELECT (CASE WHEN id > 10 THEN U8GreekCol ELSE U8LatinCol END) COLLATE Latin1_General_CI_AS
FROM TestTab;
--Msg 457, Level 16, State 1, Line 48
--Implicit conversion of varchar value to varchar cannot be performed because the collation of the value is unresolved
--due to a collation conflict between "Latin1_General_100_CI_AS_SC_UTF8" and "Greek_100_CI_AS_SC_UTF8" in CASE operator.

#reassign:WilliamDAssafMSFT I'll work on this, thanks for the attention here @jovanpop-msft

@jovanpop-msft  I believe all of the examples assume nvarchar columns in GreekCol and LatinCol because of the test table example under ### Examples of Collation Rules. Seems logical then that we should just remove the applicability of this article to Synapse and Fabric, yes?

Closing, taking this internally.


Add Date to downloaded Measurement Metadata

What are you missing?
Add the upload date to the downloaded metadata.
Why do you need a solution?
Sometimes users want to know which measurements were sequenced together (= uploaded together). This information is already depicted on the browser, but is not visible if you download the measurement metadata.
Ideas
Ideally the column would show only the date and exclude the time (ex. show 2024-07-24, instead of 2024-07-24 09:56). This makes filtering in excel possible.
Code of Conduct

[X] I agree to follow this project's Code of Conduct

Hi,
The information is used to download only samples from a specific sequencing batch, which often correspond to the upload date from the lab. It is also important to know which samples are in which batch to look for sequencing batch effect during the analysis.
Best, Morgana

Hi @oquendoM ,
since we have the batch information, would it not make sense to have that information annotated upon measurement metadata export? We already provide the sample id and sample name, would just be another property.
What do you think?
Best, Sven

Unfortunately sequencing batch and delivered batch (this one normally matches the metadata submission by the user) does not always match. The upload date would still be more informative in my opinion. But the complain of the impossibility to easily match sample metadata to the "raw data" metadata is a complain I heard already from the qPortal. This is normally relevant for big projects (>100 samples).

hm, but then I doubt the use of the batch property at all. We might need to differentiate between submission batch and measurement batch.
This can e.g. be realised, by providing an additional property for measurements. Like Measurement Run or sth alike?

Yes something like a run ID. Would this need to be added to the measurement metadata manually?

Would this need to be added to the measurement metadata manually?

yes, we would introduce a property for measurements :)


Support HAL

Also upgrade to
Spring 4.1.6.RELEASE
Hateoas 0.16.0.RELEASE

Discover and list commands can now follow _links objects in addition to links arrays in JSON response.


feat(material-icon): allow for custom tags

I would like to propose adding a tag property on material-icon's. This would allow more extensible and semantic markup. A use case for this would be for actionItem's within top-app-bar. At present the examples in mdc-web [0] display material-icons as buttons. Buttons are tabbable without having to supply a tabindex.

@mgr34 you mean something like this:
<MaterialIcon tag='button' ... />

Yes this is a pattern we have, and it would make sense that it also continues in this component. I'm open to this idea.


Capture files not saved

$ kubectl capture app-586868cdc-8dkgm -ns mynamespace -M 30 --snaplen 256
Sysdig is starting to capture system calls:

Node: gke-cluster-default-pool-93caf4f1-6b2j
Pod: app-586868cdc-8dkgm
Duration: 30 seconds
Parameters for Sysdig: -S -M 30 -pk -z -w /capture-app-586868cdc-8dkgm-1592332339.scap.gz  --snaplen 256

The capture has been downloaded to your hard disk at:
/Users/john/capture-app-586868cdc-8dkgm-1592332339.scap.gz

$ ls /Users/john/capture-app-586868cdc-8dkgm-1592332339.scap.gz
gls: cannot access '/Users/john/capture-app-586868cdc-8dkgm-1592332339.scap.gz': No such file or directory

$ kubectl version
Client Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.0", GitCommit:"70132b0f130acc0bed193d9ba59dd186f0e634cf", GitTreeState:"clean", BuildDate:"2019-12-13T11:51:44Z", GoVersion:"go1.13.4", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"15+", GitVersion:"v1.15.9-gke.24", GitCommit:"39e41a8d6b7221b901a95d3af358dea6994b4a40", GitTreeState:"clean", BuildDate:"2020-02-29T01:24:35Z", GoVersion:"go1.12.12b4", Compiler:"gc", Platform:"linux/amd64"}

Not 100% sure why it isn't working, but the message that it wrote out the file makes it seem like kubectl-capture isn't validating the results of the sysdig capture.

Oh.. this is just a script lol.
build_capture_pod nor build_capture_pod_for_ebpf nor kubectl take into account the desired namespace

$ kubectl logs capture-app-586868cdc-8dkgm-1592340541
* Setting up /usr/src links from host
* Unloading sysdig-probe, if present
* Running dkms install for sysdig
Error! echo
Your kernel headers for kernel 4.19.102+ cannot be found at
/lib/modules/4.19.102+/build or /lib/modules/4.19.102+/source.
* Running dkms build failed, couldn't find /var/lib/dkms/sysdig/0.26.7/build/make.log
* Trying to load a system sysdig-probe, if present
* Trying to find precompiled sysdig-probe for 4.19.102+
Found kernel config at /proc/config.gz
* Trying to download precompiled module from https://s3.amazonaws.com/download.draios.com/stable/sysdig-probe-binaries/sysdig-probe-0.26.7-x86_64-4.19.102%2B-e0d915406c87215c571075f54ded2193.ko
Download of sysdig-probe for version 0.26.7 failed. This is because the probe for this particular version does not exist in the repo.
Consider compiling your own sysdig-probe and loading it or getting in touch with the sysdig community
* Capturing system calls
Unable to load the driver
error opening device /host/dev/sysdig0. Make sure you have root credentials and that the sysdig-probe module is loaded.
----------------------
Event           #Calls
----------------------

Can corroborate that this is an issue for me using:
$ kubectl version
Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.5", GitCommit:"e6503f8d8f769ace2f338794c914a96fc335df0f", GitTreeState:"clean", BuildDate:"2020-06-27T00:36:48Z", GoVersion:"go1.14.4", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"14+", GitVersion:"v1.14.10-gke.45", GitCommit:"42bef28c2031a74fc68840fce56834ff7ea08518", GitTreeState:"clean", BuildDate:"2020-06-17T19:54:40Z", GoVersion:"go1.12.12b4", Compiler:"gc", Platform:"linux/amd64"}

any news?

Doesn't seems to be due to version mismatch.
Client Version: version.Info{Major:"1", Minor:"21", GitVersion:"v1.21.4", GitCommit:"3cce4a82b44f032d0cd1a1790e6d2f5a55d20aae", GitTreeState:"clean", BuildDate:"2021-08-11T18:16:05Z", GoVersion:"go1.16.7", Compiler:"gc", Platform:"darwin/amd64"} Server Version: version.Info{Major:"1", Minor:"21", GitVersion:"v1.21.2", GitCommit:"092fbfbf53427de67cac1e9fa54aaa09a28371d7", GitTreeState:"clean", BuildDate:"2021-06-16T12:53:14Z", GoVersion:"go1.16.5", Compiler:"gc", Platform:"linux/amd64"}
What I found in logs is this for minikube
`Your kernel headers for kernel 5.10.47-linuxkit cannot be found at
/lib/modules/5.10.47-linuxkit/build or /lib/modules/5.10.47-linuxkit/source.

Running dkms build failed, couldn't find /var/lib/dkms/sysdig/0.27.1/build/make.log
Trying to load a system sysdig-probe, if present
Trying to find precompiled sysdig-probe for 5.10.47-linuxkit
Found kernel config at /proc/config.gz
Trying to download precompiled module from https://download.sysdig.com/stable/sysdig-probe-binaries/sysdig-probe-0.27.1-x86_64-5.10.47-linuxkit-4d8a36681e85a6d2064c34eff88cb0c2.ko
curl: (22) The requested URL returned error: 404
Capturing system calls
Unable to load the driver
error opening device /host/dev/sysdig0. Make sure you have root credentials and that the sysdig-probe module is loaded.

Event           #Calls
----------------------`


Change min-height of modal

Hi, i am try to change min-height after modal rendered; I already tried the function height: 200, and through custom class using customClass: 'reciept-print-modal', without success...
Is there any way to change the height of swal({ });?

Are you asking about setting height or min-height?

@limonte about min-height ckeck my problem

I'm trying to change by css but without success
.sweet-alert {     height: 491px;     margin-bottom: 5%;     padding: 20px 20px 0px 20px; }

.sweet-alert

why don't you inspect the modal for the correct class-name? :) It's .swal2-modal, here's the example of setting min-height: https://jsfiddle.net/ad3quksn/130/


getLocalRef fails when using fully qualified refs

This is a follow on issue to #90 (I wasn't able to reopen my issue closed by someone else because I'm not a collaborator, I think).
Behavior
Fix #91 worked well for me when using a short name, but it does break when I use a fully qualified ref. I think that the filter logic here breaks down when shortName is fully qualified, e.g., if shortName == 'refs/remotes/origin/{branch_name}, then match[1] == '{branch_name}' when match !== null.
Maybe a fix?
I think that this can be fixed by modifying getShortName to also strip 'refs/remotes/origin' from the ref.

Thanks for reporting the issue. I will look into it again as soon as I will find some time. Hopefully this time I will get the solution right :)

Awesome. Thanks, @dorny.


DocsArchive-<IP_ADDRESS>Auto archive content

Please don't merge this PR until target archive repo PR https://github.com/OPS-E2E-PPE/docs-archive-test-target_17486e4e-f9cb-4e19-9080-491670e42a0f/pull/1 is merged into live branch.
Auto archive content to https://github.com/OPS-E2E-PPE/docs-archive-test-target_17486e4e-f9cb-4e19-9080-491670e42a0f.git

Docs Build status updates of commit 79ce44b:
:x: Validation status: errors
Please follow instructions here which may help to resolve issue.

File
Status
Preview URL
Details

:x:Error

Details

[Error]

Cannot sync git repo to specified commit because branch Archive_master_2019-10-22-03-01-17 has been deleted or has been force pushed: fatal: Couldn't find remote ref Archive_master_2019-10-22-03-01-17

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Discrepancies between search & autocomplete results

Hello,
I am encountering a problem on your demo page. When I input "14 rue Fernand Rey", no results are found. But when I press "Enter",  the right address is shown in the results (14 rue Fernand Rey, Lyon, France).
Why is it displayed with the "search" endpoint, and not with the "autocomplete" endpoint?
Thanks!

Hi @dway8, thanks for reaching out.
Autocomplete and search endpoints behave differently at this time because only search supports address Interpolation. Interpolation is the process of estimating house numbers where an exact match is not found in the data. We are working on adding this functionality to autocomplete as well but it will still be another months or so before that makes it to production.


Bundler auto_install prints to STDOUT and breaks connection

Description
Reproduction steps:

Use the rbenv version manager
Open a Ruby file in VSCode with the Ruby LSP extension installed
See the below error:

Here's the output from the VSCode extension:
2024-06-04 08:01:10.831 [info] (kickstarter) Ruby LSP> Running bundle install for the custom bundle. This may take a while...
Ruby LSP> Command: (bundle check || bundle install) 1>&2

2024-06-04 08:01:11.342 [info] (kickstarter) Resolving dependencies...

2024-06-04 08:01:12.020 [info] (kickstarter) The Gemfile's dependencies are satisfied

2024-06-04 08:01:14.725 [info] (kickstarter) Initializing Ruby LSP v0.17.1...

2024-06-04 08:01:14.752 [info] (kickstarter) [Error - 8:01:14 AM] Client Ruby LSP: connection to server is erroring.
Header must provide a Content-Length property.
{"resolving dependencies...\ncontent-length":"1599"}
Shutting down server.
2024-06-04 08:01:14.753 [info] (kickstarter) [Error - 8:01:14 AM] Stopping server failed
2024-06-04 08:01:14.753 [info] (kickstarter) Error: Client is not running and can't be stopped. It's current state is: starting
	at ws.shutdown (/Users/aidanlavis/.vscode/extensions/shopify.ruby-lsp-0.7.1/out/extension.js:41:8468)
	at ws.stop (/Users/aidanlavis/.vscode/extensions/shopify.ruby-lsp-0.7.1/out/extension.js:41:8047)
	at ws.stop (/Users/aidanlavis/.vscode/extensions/shopify.ruby-lsp-0.7.1/out/extension.js:44:38355)
	at ws.handleConnectionError (/Users/aidanlavis/.vscode/extensions/shopify.ruby-lsp-0.7.1/out/extension.js:43:63)
	at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

I originally posted this thread in slack, then I posted this comment in a different issue, but @vinistock recommended I start a new issue.

I just tried:
rails new foo
cd foo
code .

and the Ruby LSP works.  So it must be specific to the repo I'm working in - going to investigate to see if I can figure out why,

Not sure what changed, but I'm getting a slightly different error now:
2024-06-04 20:46:35.050 [info] (kickstarter) Ruby LSP> Running bundle install for the custom bundle. This may take a while...
Ruby LSP> Command: (bundle check || bundle install) 1>&2

2024-06-04 20:46:35.566 [info] (kickstarter) Resolving dependencies...

2024-06-04 20:46:36.242 [info] (kickstarter) The Gemfile's dependencies are satisfied

2024-06-04 20:46:38.802 [info] (kickstarter) Initializing Ruby LSP v0.17.1...

2024-06-04 20:46:38.828 [info] (kickstarter) [Error - 8:46:38 PM] Client Ruby LSP: connection to server is erroring.
Header must provide a Content-Length property.
{"resolving dependencies...\ncontent-length":"1599"}
Shutting down server.
2024-06-04 20:46:38.828 [info] (kickstarter) [Error - 8:46:38 PM] Stopping server failed
2024-06-04 20:46:38.828 [info] (kickstarter) Error: Client is not running and can't be stopped. It's current state is: starting
	at ws.shutdown (/Users/aidanlavis/.vscode/extensions/shopify.ruby-lsp-0.7.1/out/extension.js:41:8468)
	at ws.stop (/Users/aidanlavis/.vscode/extensions/shopify.ruby-lsp-0.7.1/out/extension.js:41:8047)
	at ws.stop (/Users/aidanlavis/.vscode/extensions/shopify.ruby-lsp-0.7.1/out/extension.js:44:38355)
	at ws.handleConnectionError (/Users/aidanlavis/.vscode/extensions/shopify.ruby-lsp-0.7.1/out/extension.js:43:63)
	at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
2024-06-04 20:46:38.829 [info] (kickstarter) [Error - 8:46:38 PM] Client Ruby LSP: connection to server is erroring.
Header must provide a Content-Length property.
{"{\"id\"":"0,\"result\":{\"capabilities\":{\"positionEncoding\":\"utf-16\",\"textDocumentSync\":{\"openClose\":true,\"change\":2},\"completionProvider\":{\"triggerCharacters\":[\"/\",\"\\\"\",\"'\",\":\",\"@\"],\"resolveProvider\":true,\"completionItem\":{\"labelDetailsSupport\":true}},\"hoverProvider\":{},\"signatureHelpProvider\":{\"triggerCharacters\":[\"(\",\" \",\",\"]},\"definitionProvider\":true,\"documentHighlightProvider\":true,\"documentSymbolProvider\":{\"symbolKind\":{\"value_set\":[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26]},\"hierarchicalDocumentSymbolSupport\":true},\"codeActionProvider\":{\"resolveProvider\":true},\"codeLensProvider\":{},\"documentLinkProvider\":{},\"documentFormattingProvider\":true,\"documentOnTypeFormattingProvider\":{\"firstTriggerCharacter\":\"{\",\"moreTriggerCharacter\":[\"\\n\",\"|\",\"d\"]},\"foldingRangeProvider\":{\"lineFoldingOnly\":true},\"selectionRangeProvider\":true,\"semanticTokensProvider\":{\"documentSelector\":[{\"language\":\"ruby\"}],\"legend\":{\"tokenTypes\":[\"namespace\",\"type\",\"class\",\"enum\",\"interface\",\"struct\",\"typeParameter\",\"parameter\",\"variable\",\"property\",\"enumMember\",\"event\",\"function\",\"method\",\"macro\",\"keyword\",\"modifier\",\"comment\",\"string\",\"number\",\"regexp\",\"operator\",\"decorator\"],\"tokenModifiers\":[\"declaration\",\"definition\",\"readonly\",\"static\",\"deprecated\",\"abstract\",\"async\",\"modification\",\"documentation\",\"default_library\"]},\"range\":true,\"full\":{\"delta\":false}},\"inlayHintProvider\":{},\"diagnosticProvider\":{\"interFileDependencies\":false,\"workspaceDiagnostics\":false},\"workspaceSymbolProvider\":true},\"serverInfo\":{\"name\":\"Ruby LSP\",\"version\":\"0.17.1\"},\"formatter\":\"rubocop\"},\"jsonrpc\":\"2.0\"}Content-Length: 242"}
Shutting down server.
2024-06-04 20:46:38.829 [info] (kickstarter) [Error - 8:46:38 PM] Stopping server failed
2024-06-04 20:46:38.829 [info] (kickstarter) Error: Client is not running and can't be stopped. It's current state is: starting
	at ws.shutdown (/Users/aidanlavis/.vscode/extensions/shopify.ruby-lsp-0.7.1/out/extension.js:41:8468)
	at ws.stop (/Users/aidanlavis/.vscode/extensions/shopify.ruby-lsp-0.7.1/out/extension.js:41:8047)
	at ws.stop (/Users/aidanlavis/.vscode/extensions/shopify.ruby-lsp-0.7.1/out/extension.js:44:38355)
	at ws.handleConnectionError (/Users/aidanlavis/.vscode/extensions/shopify.ruby-lsp-0.7.1/out/extension.js:43:63)
	at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

aha I figured it out!
I ran this yesterday:
bundle config auto_install true

this created a .bundle/config file.
If I delete the .bundle folder, then Ruby LSP works perfectly!  If I run bundle config auto_install true again it breaks again.

auto_install resets bundler state here: https://github.com/rubygems/rubygems/blob/a15e2cb599e1415d0e9577eb7ca2d0df985a4c38/bundler/lib/bundler.rb#L183-L190
That includes the ui level set previously. Probably a bundler bug? That seems unexpected

Good find. That certainly explains why it's not respecting the UI level.
I'm not sure if it's intentional or not. If they want to keep the code that way, we will need to turn off auto install before launching the server.

Let's see https://github.com/rubygems/rubygems/issues/7769

@Earlopain
Some additional context from the slack thread that didn't make it into this GitHub issue (sorry!) is that it only seems to fail when running on my repo - a fresh rails repo works fine, regardless of whether auto_install turned on or not.
For my repo:
> bundler -v
Bundler version 2.5.3
> gem -v
3.5.11

For a fresh rails app:
> bundler -v
Bundler version 2.5.11
> gem -v
3.5.11

I tried updating my repo to use 2.5.11:
> bundler -v
Bundler version 2.5.11
> gem -v
3.5.11

And then I also tried using the latest version of bundler:
> bundler -v
Bundler version 2.5.13
> gem -v
3.5.11

and I'm still seeing the error.

Given that it works fine for a fresh rails repo and it only fails on my (closed source) repo, I think this is going to be pretty hard to reproduce.
Feel free to close this issue if you all want to - I've been perfectly happy just not using auto_install while I'm working!  That said if you all want to continue looking into the issue I'm happy to answer any questions you might have about the repo.

It would be very helpful if you could try to make a minimal reproduction for this. I haven't had a chance to look into it myself again but last time I couldn't verify my assumption about an issue with bundler itself.


QUESTION: need help setting up pulse counter

Dear,
First of all: great work on the Homie framework!
I'm in trouble with the convention and how to do this in de code. I find the temperature example somehow confusing.
Everything is working great, using the git-version. But I need some help with the setup/convention. I've got a interrupt-based pulse counter. Idea is to make a node WaterMeterNode("WaterMeter", "Counter").
The node has 2 properties: counter and totalcounter. The latter should be settable. Both have a number of litres water as "content". So the unit would be "litres"?
Could anyone help me out to get my head straight?
By the way: I use a debouncer in the interrupt handler since I wasn't fully sure how the Bounce library works. Publishing is done by a ticker, so also interrupt-based. So I make a "snapshot" of the counters when it's time to publish. That was I won't be missing any pulses during publishing.

to see what I've made from it:
https://github.com/bertmelis/homie-utility/blob/master/homie-utility.ino

Guessing you figured this out?  Just for reference wanted to post this here, haven't tried it yet, but here's one I found for a pulse meter:
https://github.com/yves-ledermann/homie-esp8266-supplements/blob/master/src/Nodes/PulseCounter.cpp


mysql backup restore table issue

i use to backup new tables. just in case a table runs out space and i need to make an another one.  field names and their types are copied correctly, but 'allow nulls' is set to yes, and the 'default value' is set to null, even though the backuped table has the 'allow nulls' set to false and the 'default value' is set to empty string.

Hi,
Webmin uses mysqldump command to create dumps. If your original table explicitly defined columns to be NOT NULL and/or have a DEFAULT value other than NULL, these would be reflected in the mysqldump output.
For a column that is supposed to be NOT NULL with an empty string as the default, the CREATE TABLE part should look something like this:
`column_name` VARCHAR(255) NOT NULL DEFAULT ''

If you have more questions, please consider posting them in our forum, as this is primarily a bug tracker!


[region-isolation] Excise 'transfer' in favor of 'sending'

Transfer is the old spelling for sending. I left it in to ease cherry-picking... but I think we have reached a point where that is less of a concern.

@swift-ci smoke test

I did this by hand using clangd's rename symbol and by doing the comments myself in combination with query-replace and git grep.

Forgot to update the unittest = /.

@swift-ci smoke test

Failed in something unrelated in Linux.

@swift-ci smoke test linux platform


Error when attempting to alias a custom domain on pro plan

Not blocking for me (I just downgraded to the Premium plan), but the pro plan does not support custom domain aliasing (d'oh!).
Error! Custom domains are only enabled for premium accounts. Please upgrade by running 'now upgrade'.

This should have been fixed yesterday, if it still keeps on happening please reopen the issue


new tool TensileLibLogicToYaml that converts Liblogic file to tensile yaml

changes for TensileLibLogicToYaml that converts Liblogic file to tensile yaml is done , reviewed, validated with test cases

Please fix the static analysis fail

I'm merging the PR on behalf of Feroz.


Opentsdb emitter extension

A well test metric emitter which emits druid metrics to OpenTSDB. Last pr(#5303) was closed by accident and can not be reopened.

#5303

The travis-ci build failed, but it's not due to my commits. @jon-wei Can you help restart the failed job.


⚠ Punto de Fuga has degraded performance

In 2208977, Punto de Fuga (http://puntodefugapodcast.com) experienced degraded performance:

HTTP code: 200
Response time: 7063 ms

Resolved: Punto de Fuga performance has improved in bb51027.


RigidBody center of gravity issue

Godot version: 3.1.1

OS/device including version: Windows 10 x64

Issue description:

In the following scene the collision shape of the corner desk was auto-generated via Mesh->Create Trimesh Collision sibling.

As the RigidBody origin is located outside on the side of the Collision shape Im' expecting the desk to behave like a pendelum. It doesn't. It behaves as if its center of gravity was located somwhere inside the mesh:

But... After I add a simple CollisionShape, and disble it (!) I'm getting the proper result:

The result:

[EDIT] Other things that work:

duplicate the collision shape instead of adding a new one.
use convex sibling instead of trimesh

Steps to reproduce:
See above
Minimal reproduction project:

Phys.zip

Convex shape decomposition got some updates recently, can you try on a master build to see if still affects it? (I do not know if there are new options for it too)

A Concave Collider is treated as a box actually and center of gravity will be recalculated, not the origin position.

Still valid in 3.2.3 rc3

The rigid body can only have convex collisions. A trimesh (concave) shape, which is made of triangles, is too complex to be simulated.
So the physics engine can't simulate properly the dynamics of the RigidBody that has a trimesh shape.

Happens only with ConcavePolygonShape, and there's a warning in the editor:
ConcavePolygonShape doesn't support RigidBody in another mode than static.
This is a documented limitation, closing.

Happens only with ConcavePolygonShape, and there's a warning in the editor:
ConcavePolygonShape doesn't support RigidBody in another mode than static.
This is a documented limitation, closing.


duplicate events and minor pathing

So I have a i9 with 8 cores, and using all 8 cores to run the game (cause i mean, i made the error of going to Intel, and missed out on a 16 core amd). Also other then harmony, base rimworld, and rimthreaded, I also was running royalty dlc too (cause I am totally rich and need to flaunt my wealth around (Im a broke college kid, send help))
But some events seem to improperly reserve I believe, because say when I got raided, it triggered 2 raids, or say guests triggered twice, though mad animal event and wander joining did not seem to trigger it. (screen shot of events

along with event log

)
Also minor pathing bug (photo of what happened), so i was building in mountain area, and I tell them to build a wall, with a corner of wood, which was blocked off from the inside, well no issue, I do recall they would still be able to build it (and if not, they could come from the outside, where it is wide open). Regardless, my pawns would go back and fourth inside, with wood, about to build it, but stopping when they got close, and then turning around to grab wood to go build it on loop, until I just had them build a corner inside, and then they went around to build it externally.

Minor stuff, sure, but hay, everything helps (plus the ideal of a late game duplicate raid scares me o.o)

So I triggered a raid in dev mode, and it did not seem to trigger it, neither in 1x or 3x speed. I do not fully understand how this happened....

Thanks for the bug report. If there is a way to reproduce this, I will definitely look into it. I suspect it is caused by some sort of race condition. If that is the case, it should be pretty easy to fix, once I locate the part of the code that spawns these kinds of events.

Thanks for the bug report. If there is a way to reproduce this, I will definitely look into it. I suspect it is caused by some sort of race condition. If that is the case, it should be pretty easy to fix, once I locate the part of the code that spawns these kinds of events.

I started a new game with royalty and the bug was there from the start. All games that i start has the same issue.

I fixed the pathing bug and the double events might have something to do with colony item wealth which was fixed as well. Please reopen if either are still an issue, especially if there is a way to produce double events consistently.


I can't get request payload when used $auth.signup(user)

Hi!,
I have a problem with satellizer api signup, when call the method $auth.signup(user) in my backend made with CodeIgniter I can't retrieve the values sent in user.
I tried everything
The methos of my library REST for options call
$this->options('name'); // this is null, link library
https://github.com/chriskacerguis/codeigniter-restserver
I tried get payload manually with this
$request_body = file_get_contents('php://input');
$data = json_decode($request_body); //this is null too
I don't know that can be please, help me!
Thank you all!!
Best regards.

Fix with this #430
Summarizing, the browser send two request one OPTIONS for check if the server accept custom headers and after request POST with data.
Thanks for your time!


Emulator Refactor

This is an open issue moving on from #105 .
I have working TensorFlow models for both the naive GP optimization as well as the Habib log-likelihood function. Here is, in general, what the code looks like-
Naive
X = tf.Variable(pca.gparams)
Y = tf.Variable(pca.w)
conf_priors = np.array(Starfish.config.PCA["priors"]).T

# Create the trainable variables in our model. Constrain to be strictly
# positive by wrapping in softplus and adding a little nudge away from zero.
amplitude = (np.finfo(np.float64).tiny +
           tf.nn.softplus(
               tf.Variable(10. * tf.ones(pca.m, dtype=tf.float64)),
               name='amplitude'))
length_scale = (np.finfo(np.float64).tiny +
               tf.nn.softplus(
                  tf.Variable(np.tile([300., 30., 30.], (pca.m, 1)), dtype=np.float64),
                  name='length_scale'))

# Put Gamma prior on the length scales
rv_scale = tfd.Gamma(
  concentration=conf_priors[0],
  rate=conf_priors[1],
  name='length_scale_prior')

# Define ARD kernel and GP likelihood over observations
kernel = InputScaledKernel(tfk.ExponentiatedQuadratic(amplitude),
                         length_scale,
                         name='ARD_kernel')
gp = tfd.GaussianProcess(kernel=kernel, index_points=X)

# Joint log prob of length_scale params and data
log_likelihood = (tf.reduce_sum(gp.log_prob(Y)) +
                tf.reduce_sum(rv_scale.log_prob(length_scale)))

# Optimization target (neg log likelihood) and optimizer. Use a fairly
# generous learning rate (adaptive optimizer will adapt :))
loss = -log_likelihood
opt = tf.train.AdamOptimizer(.1).minimize(loss)

This optimizes pretty well, and takes ~50s for 1000 iterations. Here is the log-like by iteration plot which shows that we can get away with optimizing for probably 200 iterations

After training, we can save the hyperparameters and use them for recreating the GP and sampling from it, very similarly to how the current code works. I wish the weights were easily to visualize with plots, so I could show the process. Something I could definitely work on.
Habib
This is a little more complex, but works about the same
def habib_joint_log_prob(lam_xi, a, len_scales):
    """
    habib lnlike
    """
    gparams = pca.gparams
    PhiPhi = pca.PhiPhi.astype('f8')
    w_hat = pca.w_hat
    M = pca.M
    m = pca.m

    conf_priors = np.array(Starfish.config.PCA['priors']).T

    # Put Gamma prior on the length scales
    rv_scale = tfd.Gamma(
          concentration=conf_priors[0],
          rate=conf_priors[1],
          name='length_scale_prior')

    # Define ARD kernel and GP likelihood over observations
    kernel = InputScaledKernel(tfk.ExponentiatedQuadratic(a),
                             len_scales,
                             name='ARD_kernel')
    sigs = kernel.matrix(gparams, gparams)

    # This is garbage that I have to do to create a block diagonal
    paddings = tf.constant([[0, (m-1)*len(gparams)],[0, 0]])
    Sig_w = tf.pad(sigs[0], paddings)
    for i in range(1, m):
        paddings = tf.constant([[i*len(gparams), (m-i-1)*len(gparams)],[0, 0]])
        zeros = tf.pad(sigs[i], paddings)
        Sig_w = tf.concat([Sig_w, zeros], 1)

    C = (1. / lam_xi) * PhiPhi + Sig_w

    sign, pref = tf.linalg.slogdet(C)
    rhs = tf.linalg.solve(C, np.expand_dims(w_hat, -1))
    central = tf.matmul(tf.expand_dims(w_hat, 0), rhs)
    lnp = -0.5 * (pref + central + M * m * np.log(2. * np.pi))

    return tf.dtypes.cast(tf.squeeze(lnp) + tf.reduce_sum(rv_scale.log_prob(len_scales)), tf.float64)

lam_xi = (np.finfo(np.float64).tiny +
             tf.nn.softplus(tf.Variable(1., dtype=tf.float64), name='lambda_xi'))
amplitude = (np.finfo(np.float64).tiny +
           tf.nn.softplus(
               tf.Variable(10. * tf.ones(pca.m, dtype=tf.float64)),
               name='amplitude'))
length_scale = (np.finfo(np.float64).tiny +
               tf.nn.softplus(
                  tf.Variable(np.tile([300., 30., 30.], (pca.m, 1)), dtype=np.float64),
                  name='length_scale'))

loss = -habib_joint_log_prob(lam_xi, amplitude, length_scale)
opt = tf.train.AdamOptimizer(.1).minimize(loss)

So, the main divergences from the current habib implementation is that I've opted to use an ARD Kernel from TensorFlow to get the covariance instead of Starfish.covariance.k. I then have some garbage to create the block diagonals (TF doesn't have a tf.block_diag, but I've submitted a feature request), and finally, I simply swap out the numpy functions to the tensorflow functions for the linear algebra.
Here is a plot of the loss function per iteration as before-

This method would require sampling weights the "hard" way by calculating the many matrices and sampling. Or perhaps we can use the optimized parameters and use the available GP code (like above) for sampling, since it offers a GP.sample() method.
Notes
Using Tensorflow as a backend means that none of the above code actually evaluates any code. Instead, it creates a graph of every operation and optimally distrubes them among the CPU and GPU. To actually evaluate the code, a simple wrapper has to be used like
num_iters = 1000
loss_ = np.empty(num_iters, dtype=np.float64)
with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    for i in tqdm.trange(num_iters):
        lx_, amp_, len_scales_, loss_[i], _ = sess.run([lam_xi, amplitude, length_scale, loss, opt])

In addition, there is a lot more concern for datatypes, as you can see in the above code, as well as special concerns for constraining the trainable values. Something that is super cool, though, is that these graphs that tensorflow makes can actually be visualized and interpreted (a la TensorBoard) and have built in methods for saving models and exploring them.
I do have an MCMC implementation using tfp.mcmc to explore the posterior of the hyperparameters. Unfortunately, it runs somewhat slowly and I would guess it could take an hour to get actual convergence. This could just be my poor understanding of MCMC samplers and options, but at first glance I'm not impressed by the performance. HOWEVER, as a proof of concept this shows that we can completely drop the emcee dependence from the emulator code.
Does this look good enough to bring into the emulator code and add TensorFlow as a dependence?

To extend this, I want to create a few plots showing samples from the GPs as well as some sort of timing evaluation, especially against the number of components of the PCA.

This looks really exciting. I have a few questions, since I'm just getting acquainted with Tensorflow myself.

The naive optimization (I think what we were calling emulator v1) looks pretty good to me. If the results you plotted for the weights are typical, then I'd say this is a good candidate for merging and providing the v1 option to users. Nice work!

I haven't yet had success locating what the functional form of the TF ARD kernel is. Do you happen to have a link?

For the Habib implementation, I'm not sure I understand what you mean by sampling the weights the hard way. Does this mean that training the emulator will be more difficult? Or does it mean we can't marginalize the likelihood over the emulated spectrum when we go and run a fit to a spectrum?

Regardless, I'm super excited about this proof of concept. It seems like it could be a really powerful improvement for the code!

@iancze Thanks :) . I'm actually going to try and see what PyTorch implementation would look like. The TF code is a lot of boilerplate and PyTorch might end up with significantly cleaner code.
As for the the TF ARD Kernel you can't find it because I had to custom make it with the help of a TFP engineer. See here for discussion and implementation. PyTorch's Pyro (the equivalent relationship as Tensorflow and Tensorflow Probability, it seems) has a native ARD Kernel for their GP module.
I think I am mistaken in what I mean by the "hard way". As I look at it again, the likelihood is just used for training, and then we can use the trained hyperparameters in the standard GP to sample our weights. Correct me if I'm wrong on that.

I've gotten up and running with PyTorch/Pyro and so far I much much much prefer it to TensorFlow. Here are some examples of doing a Naive fit-
import torch
import pyro
import pyro.contrib.gp as gp
import pyro.distributions as dist

grid = HDF5Interface()
pca = PCAGrid.create(grid)

X = torch.from_numpy(pca.gparams)
y = torch.from_numpy(pca.w)

amplitude = torch.tensor(20.)
lengthscale = torch.tensor([300., 30., 30.])

kernel = gp.kernels.RBF(input_dim=3, variance=amplitude, lengthscale=lengthscale)
gpr = gp.models.GPRegression(X, y, kernel, noise=torch.tensor(1.))
optimizer = torch.optim.Adam(gpr.parameters(), lr=0.01)
loss_fn = pyro.infer.Trace_ELBO().differentiable_loss
losses = []
num_steps = int(1e4)
for i in tqdm.trange(num_steps):
    optimizer.zero_grad()
    loss = loss_fn(gpr.model, gpr.guide)
    loss.backward()
    optimizer.step()
    losses.append(loss.item())

Here are the fits:

The biggest problem I have right now is that I don't currently have an easy way of batching this. In other words, Its fitting one GP for all weights instead of (in this example) 4 independent GPs. I have opened a discussion with Pyro to discuss this. As far as speed, on my CPU (8 core) it performs about 50 optimization steps per second. For the 10,000 steps above it takes me ~3 minutes. Which is phenomenal

Update- I have the GP working with priors properly but not batching correctly (so it still fits 1 GP broadcasted to m weight functions). The best fit parameters seem a lot more appropriate than what I was seeing beforehand. Also this is a very pretty plot and I wanted to share it-

So, to clarify, what I'm currently doing is training ONE Gaussian process to fit all of the eigenspectra weights. This means there is one amplitude and one length-3 lengthscale parameter fit for all the data points. This is what is produced above, and IMO doesn't seem like a bad fit. I could fit m separate GPs but I was hoping for a less tedious way of achieving the same effect within Pyro.

Ah I see. I think this could work, but will probably end up sacrificing a bit of accuracy in the interpolated spectra, since the amplitudes will need to be different for each weight. For example, in your plot above, the GP looks pretty good for w0, but will be conveying too much uncertainty in all the other eigenspectra weights. If you wanted to reduce dimensionality it might work if you used similar length scale parameters for all weights but used a different amplitude for each one.

Okay, so I've trained (in my case) 6 GPs completely separately, using a for-loop.
amplitudes = 20 * torch.ones(pca.m)
lengthscales = torch.tensor([300., 30., 30.]).expand(pca.m, 3)

kernels = [gp.kernels.RBF(input_dim=3, variance=amplitudes[i], lengthscale=lengthscales[i]) for i in range(pca.m)]
priors = torch.tensor(
    [[2, 2, 2],
    [0.0075, 0.075, 0.075]]
)
[kernel.set_prior("lengthscale", dist.Gamma(priors[0], priors[1]).to_event()) for kernel in kernels]
[kernel.set_prior("variance", dist.Uniform(10, 150)) for kernel in kernels]

gprs = [gp.models.GPRegression(X, yi, kernel, noise=torch.tensor(1.)) for yi, kernel in zip(y, kernels)]

optimizers = [torch.optim.Adam(gpr.parameters(), lr=0.01) for gpr in gprs]
loss_fn = pyro.infer.Trace_ELBO().differentiable_loss
num_steps = int(1e4)
losses = np.zeros((pca.m, num_steps))
for i, (gpr, opt) in enumerate(zip(gprs, optimizers)):
    for j in tqdm.trange(num_steps):
        opt.zero_grad()
        loss = loss_fn(gpr.model, gpr.guide)
        loss.backward()
        opt.step()
        losses[i][j] = loss.item()

Losses

Fits

The parameters for each kernel are independent. I really like how these plots look I'm not sure why. I ought to be able to find a way to vectorize these cheaply using the torch.multiprocessing to process each kernel independently but still parallelized.

Yes, this looks much better, and I think completely reproduces the v1 emulator. About how long did the training take?

each 1e4 fit took ~ 2:45 on my machine so about 15 minutes total for 6 eigenspectra. As you can see from the losses this can probably reduced to 5e3 steps each cutting the time in  half.

That's great! If you're happy with the PyTorch version, then I'd say this is a great candidate for merging into the codebase as sort of the go-to emulator version. Folks who need the complexity of the Habib emulator can spend more time training it, but for most inference problems I think this faster version will suffice.

This is great to hear, and thanks for all of the work. Happy Holidays to you (and all our github watchers :) ) as well!
v1 emulator
You showed a nice implementation of this a few posts back. Given the install difficulties with PyTorch that you mentioned, I was wondering how hard this might be to implement this with a simple GP without a PyTorch dependency? Perhaps via something that's a bit easier to install, whether it be celerite, george, scikit-learn, or even the custom Starfish implementation formerly in covariance.py? This stripped down emulator might be a nice thing to include in a v0.3.0 release that most users could use without needing to worry about installing PyTorch if they might not need it.
PyTorch
That said, I think moving to support PyTorch would be a worthwhile research goal, especially given the speedups you have shown with the Habib et al. likelihood function. I think that this currently makes sense as an optional addition to the v1 emulator, and if it's so fast, it makes sense as the primary implementation of the Habib et al. likelihood function.
Regarding the tensor notation, if you could write your implementation up in a tex document somewhere this might make it easier to figure out how things fit together and we could iterate on an implementation. I'm a bit rusty with tensor math but maybe this well help us converge on the best way to calculate the Habib likelihood with something like Tensorflow or PyTorch.
Regarding V12, let me refresh my memory with what I wrote 3+ years ago...
"The Mm × m matrix V12 (and its transpose V21) describe the relations between the set of
parameters in the grid and the newly chosen parameters to interpolate at theta_star. The structure of V12 is set by evaluating [the emulator GP kernel] k_i (Equation (29, ApJ version)) across a series of rows of {theta_star}^grid like in Sigma^grid, for i = 1, 2, ..., m, and across m columns of theta_star."
Ok, this is a lot of notation jargon just to explain what's going on. Essentially, we're just doing standard GP posterior prediction of the eigenspectra weights onto a single point, theta_star, conditioned on all of the existing eigenspectra weights measured at the grid points. But, because the emulator output is multidimensional (there are m eigenspectra weights) that's why this formulation has some extra rows/columns compared to the standard GP prediction operation to a single point (see Rasmussen and Williams, 2005, eqns 2.21 - 2.24).
The lam_xi term is called a "nugget term" and is basically the white-noise scatter of the emulator, in flux space, like you mentioned. I'm not sure I fully understand your comment about the conditional and the standard kernel. The math leading up to Eqn 42, ApJ is introducing the GP prior to the grid of eigenspectra weights, and then marginalizing over all possible GP weights (i.e., getting the GP marginal likelihood). The thing that we actually need to calculate is Eqn 42 (and the priors, Eqn 43, 44, and 46). The whole posterior is put together in Eqn 45. Is there a fast way to implement a standard GP hyperparameter optimization in PyTorch and this is why you are asking about a standard kernel?

For the last point-
The "standard kernel" is the same as A.10 (arxiv) used to create block matrix Sig_w (eqn. A.13). Our hyperparameters train the amplitude and lengthscale of this standard kernel (across all batches and feature dimensions) in addition to the lam_xi. My initial understanding was that we could train these kernel parameters using the Habib likelihood and then just throw them into a standard GP (eqn. A13) for sampling the posterior function (i.e. for getting the weights once trained). My current understanding is that the Habib implementation samples the posterior function using a conditional likelihood that includes lam_xi and PhiPhi (arxiv eqn. A.30) which means we can't just naively throw our trained hyperparameters into a normal GP. Unless we can, which is my confusion.

[this is in response to your new but not newest comment]

I've also been thinking about the overall organization of the emulator code. As of right now, I think we should not put as much emphasis on the PCAGrid class. I think the two points where users should work is either a GridInterface + Interpolator or an Emulator. The intermediate PCAGrid class is merely a wrapper for the linear algebra that facilitates the PCA and ought to only be used in the Emulator.

This is great that you are thinking about reorganizing the emulator / interpolator into a more parallel type of functionality. In general, I think your concepts are great and JSON saving is always a good idea. (Though, I always wonder if there are better scientific formats for this kind of stuff). I have a few comments about envisioned use cases and how a user might go about training things.
PCA decomposition
It's true that this actual process can be pretty quick, and in theory the eigenspectra don't need to be stored on disk. However, for analysis purposes, we definitely should have a use pattern where actual eigenspectra can be stored to disk for plotting purposes and to analyze what spectra features they might actually tracking. People might want to experiment with how they are tuning the emulator by varying the number of eigenspectra weights, and having access to them in a persistent manner (with the settings used to generate the eigenspectra) is really important here. Also, if PyTorch goes and changes its PCA algorithm from one release to the next (or maybe even just default parameters), we won't have a way to verify reproducibility of eigenspectra.
Second, the pattern in which I used Starfish was the following. Store ~5 Gb library of synthetic spectra on my home disk somewhere. Use PCA to decompose these into eigenspectra, and train the emulator. Then, transfer the eigenspectra and hyperparameters to a cluster where I fit an actual spectrum. In that case, a trained emulator (or even just the PCA spectra themselves) are very easy to transfer from a staging computer to HPC without needing the full synthetic grid. There may be better use patterns but at least I found this convenient.
Some other patterns to consider
Say a user trains an emulator based on some subset of their synthetic grid. Then, they do a fit of a spectrum and find that their posterior butts up against the edge of their subset, and they need to retrain a new emulator. Is there anything they can easily use from the previous training? Maybe the answer's no, but it feels like there would be some hints for GP hyperparameter settings.
Or, say a user trains an emulator using m eigenspectra weights. Then, they decide that they need to be more (or less) accurate and want an emulator with m + 1 or m - 1 eigenspectra. Is there anything they can reuse from the first training?
[response to newest comment]

For the last point-
The "standard kernel" is the same as A.10 (arxiv) used to create block matrix Sig_w (eqn. A.13).

The GP hyperparameters determined from training using A.23 (and kernel equation A.10) are the same ones used to do emulation via eqn A.30. I think everything is consistent here among the training and utilization steps. Hopefully that makes sense but please keep asking questions if I'm struggling to make clear what's going on.

It would be just as easy to ship an emulator as HDF5 with the eigenspectra built in, too. I wanted to try and see what the bare minimum model-persistence would require.
For the other patterns, I (naively) believe we should be able to add points onto a Gaussian Process without immediate detriment if the appended grid points are close to the original grid points. Since our training of the Kernel parameters ought to have optimized to a local optimum, we should have a somewhat well-defined function-space to sample from using the GP. Since we assume our weights are a smooth function, this means we should be able to just use these sampled functions on any set of grid points. This means the user could just sample outside the borders but at the cost of increased covariance.  What adding extra grid points into the training data would do, however, is further constrain our function-space and therefore lower the covariance of the function sampling (I think).

+1 for the emulator.hdf5 idea. I think that makes a lot of sense.
I think you're probably right on the hyperparameter settings, but, the reason I was thinking there might be a substantial change is because even if you keep m total eigenspectra, if you expand your subgrid slightly, then the eigenspectra themselves will change. Thus, the GP hyperparameters might change as well even if you've only added a few spectra on the edge of the grid. If these spectra corresponded to a temperature range where some new important spectral line or feature was being introduced, this could substantially change the eigenspectra from the PCA. I guess we won't know for sure without some detailed tests, but back when I was constructing the emulator this seemed to have a big effect.

Long time since I've visited this issue, but refactor/core-numpy includes a fully (as far as I can tell through inspection and testing) functional rewrite of the emulator. Documentation will be included to showcase the new operation modes, alongside a jupyter notebook example that provides some more insight into the background of the emulator.


MixtureModelPipeline that lets you specify a cluster to be tagged…

… as outliers
Also has python scripts that automate diagnostic process.

Looks fine to me, save for the coverage...

Added comments and improved coverage


Application's CSS file is properly commented.

Instructor has advised that changes to the CSS are not required. Based on this closing this


No X11 Display Variable was set

I have a freshly built version of Ghidra on my Ubuntu 22.04 server that is not wanting to launch a window. When I ran ghidraDebug I saw the error messages about: "No x11 display variable was set, or no headful library support was found, but this program performed an operation which requires it".
I am opening it through a local Gnome desktop, and have the appropriate version Java installed and in use, and my env variables are correctly exported. Is there something to do with compatibility between Ghidra and Ubuntu together that I am missing? Any help, greatly appreciated.

Did you install Java through apt? If so, what package did you install? You have to be careful not to install the headless version of the JDK that they like to offer by default.

Hey @ryanmkurtz thank you for the help. That was indeed the issue. Something so simple- awesome note to self.
Good to step away and come back with fresh eyes, can't believe I missed it! Thank you again.


Show descriptive error message in expression editor

Describe the Bug with repro steps
Create a simple flow.
When an item is created -> Compose. in compose, add an expression concat("1", "2") and try saving. It says expression is invalid. Can we add ' Expression is invalid. Please use single quote' when we know double quotation is used to represent a string ?
This is a basic fundamental scenario for users where they are getting really confused. In expression, we accept single quotation, in filter query, we accept double quotation, we need to make it super clear in UX.
What type of Logic App Is this happening in?
Consumption (Portal)
Are you using new designer or old designer
New Designer
Did you refer to the TSG before filing this issue? https://aka.ms/lauxtsg
Yes
Workflow JSON
No response
Screenshots or Videos
No response
Browser
Chrome
Additional context
No response

@valentina-vallalta can you take this one? ERic should be able to help you out. This one is medium priority though so focus on higher priority bugs you may have first.


Add sprint and slide logic

Animations are added in #21 but we need to map sprinting to a key, increase Stroheim's speed, and make it so he can slide

#34 addresses this issue


Using Umask on the Windows platform will generate compilation errors.

On the Windows platform, there is no concept of Umask, so the submitJob function of cp_service.go will generate compilation errors on the Windows platform. We can avoid this problem by using the equivalent of setting file permissions.

thanks for your contribution, Once tested, it will be merged

thanks for your contribution, Once tested, it will be merged
@Normalnoise
Why is the submitJob function modified in the master branch, but my PR is not merged?


hover touch effect not working in Xamarin ios sample

Hi Hover example works on uwp but not ios

Hi @trossellcullenm
Thank you for the report.
Does iOS support Hover?

Hi Andre,
I was using your toucheffect sample and I have been trying code using this nugget and also  the xamarin.community toolkit.effects sample with the hover function which is not working. Is there no way of making this work? -- it  does work perfectly on uwp.

Hi Andre,
I was using your toucheffect sample and I have been trying code using this nugget and also  the xamarin.community toolkit.effects sample with the hover function which is not working. Is there no way of making this work? -- it  does work perfectly on uwp.

@trossellcullenm how do you expect the hover to work? Are you using iOS with a pointer device like a mouse?
I don't think hover is supported with touch input

@trossellcullenm how do you expect the hover to work? Are you using iOS with a pointer device like a mouse?
I don't think hover is supported with touch input

Hi Gerald,
On UWP hover works with a mouse or track pad and also with a touch screen holding a finger over an image triggers an event. On the sample app on the hover event the label should rotate. I use the hover event to provide information about the image which is underneath. Hover does not do anything on xamarin.ios.

Hi Gerald,
On UWP hover works with a mouse or track pad and also with a touch screen holding a finger over an image triggers an event. On the sample app on the hover event the label should rotate. I use the hover event to provide information about the image which is underneath. Hover does not do anything on xamarin.ios.

I'm suspecting iOS does not expose any states like hover, but not sure. We might need to investigate on that.

I'm suspecting iOS does not expose any states like hover, but not sure. We might need to investigate on that.


577 test all parameter types of the dense matrix cpu rosenbrock on the analytical policy tests

Closes #577

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 93.27%.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #593   +/-   ##
=======================================
  Coverage   93.27%   93.27%
=======================================
  Files          49       49
  Lines        3466     3466
=======================================
  Hits         3233     3233
  Misses        233      233

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


QU: How do I insert text between the brackets of a function?

Once I've navigated to the function like the example below, I'd like to fill it with a string of code.
render() {
  // insert here
}

Is this possible?

@chrisgervang, I was going to implement this after removing nodes, but since you asked for it, it's now implemented in 0.55.0. Let me know if you run into any issues.
Use the setBodyText method:
methodDeclaration.setBodyText("// insert here");

Or:
sourceFile.addFunction({
    name: "myFunction",
    bodyText: "// insert here"
});

Thank you so much! Great job with this library. It's amazing work!

Thanks for the compliment. I'm glad it's useful! :)


Rewrite proxy to use token source

Rewrite proxy to use Token Source

@ahmetb oops i didn't mean to create pr to upstream. closing.


Using delta counters instead of cumulative counters for span RED metrics

Resolved via https://github.com/wavefrontHQ/wavefront-opentracing-sdk-csharp/commit/01767855a4148bd6e91b672aee5fa8df0f6090ed


Add chat flow for Document Server

back end for file manager.
this should not be valid once jimber is informed that this method should not be needed. @ken-de-moor

worked on branch development_vdc_jimber_chatflows

worked on branch development_vdc_jimber_chatflows

Chatflow is not added to dashboard

Chatflow is not added to dashboard

I shouldn't hit empty page, should be implicit inside the filebrowser chatflow

I shouldn't hit empty page, should be implicit inside the filebrowser chatflow

blocked on: https://github.com/threefoldtech/vdc-solutions-charts/issues/56

blocked on: https://github.com/threefoldtech/vdc-solutions-charts/issues/56

there should be no chatflow for document server.


Rollback gci and golangci-lint versions

At some point we diverged in the version of gci from the version golangci-lint is using. That led to issues with gci not reporting any issues and golangci-lint reporting bogus issues. Oddly enough I could only manifest this issues locally, the GitHub Checks workflow never failed. Given that we're not using any of the newer features of either gci or golangci-lint, I opted to rollback their versions  instead.
gci from 0.8.2 to 0.8.0 and golangci-lint from 1.50.0 to 1.50.1.

Turns out it was a Golang version issue. The problem would manifest with 1.18 but not with 1.18.7 which is used on GitHub Workflows.


Extensible key bindings

It would be nice to let devs extend the keybindings in slidem-deck, maybe by adding a slot to the keybindings divs

Excellent suggestion, I will include it in the next release

I added a slot where you can add keybindings for navigation. Use the forward and backward slots for appropriate navigation direction.
Example:
<slidem-deck loading>
  <gluon-keybinding slot="forward" key=" "></gluon-keybinding>
  <gluon-keybinding slot="backward" key="Delete"></gluon-keybinding>
  <gluon-keybinding slot="backward" key="Backspace"></gluon-keybinding>

Note: gluon-keybinding uses Key Values and not Key Codes, so the correct key for Space is " ".
Note2: IE11 is not spec compliant and uses "Spacebar", so you have to add both to support older browsers.


(BadRequest) The VM size of AgentPoolProfile:nodepool1 is not allowed in your subscription in location 'westus'.  For more details, please visit https://aka.ms/aks/quotas-skus-regions

Problem encountered on https://dotnet.microsoft.com/en-us/learn/aspnet/deploy-microservice-tutorial/azure-resources
Operating System: windows
Provide details about the problem you're experiencing. Include your operating system version, exact error message, code sample, and anything else that is relevant.
on command: az aks create --resource-group IoutilityApp --name IoutilityCluster --node-count 1 --enable-addons http_application_routing --generate-ssh-keys
response : Resource provider 'Microsoft.ContainerService' used by this operation is not registered. We are registering for you.
Registration succeeded.
(BadRequest) The VM size of AgentPoolProfile:nodepool1 is not allowed in your subscription in location 'westus'.  For more details, please visit https://aka.ms/aks/quotas-skus-regions
Code: BadRequest

This appears to be an issue with your Azure subscription. Would recommend contacting Azure support for this issue.


[HOLD] [$250] Migrate BaseValidateCodeForm.js to function component

Class Component Migration
Filenames

[ ] src/pages/signin/ValidateCodeForm/BaseValidateCodeForm.js

type: React.Component
has state values: true
has refs: true
has context: false
lifecycle methods: componentDidMount,componentDidUpdate

Task

We currently have some class components in our codebase that we would like to refactor to a function component.
Here's a link with some general advice on how to refactor a class component to a function component: https://react.dev/reference/react/Component#alternatives
If you need additional guidance, please ask in #expensify-open-source
Test for any regressions and verify that there are no breaking changes

⚠ Could not update price automatically because there is no linked Upwork Job ID. The BZ team member will need to update the price manually in Upwork.

Heads up! The pricing for this issue has been adjusted based on the scope of the work and the fact that no complex bug investigations or proposal are required.

I'd like to work on this issue when it goes to external

I'd love to work on this.


Support vendor pseudo elements

Fix for: #3
This adds a VENDOR_PSEUDO_ELEMENTS regex that is appended to HAS_PSEUDO_CLASSES_ELEMENTS

Solid work – thanks!


Fixes 376: add description for status DHCP leases

This PR enhances the Status->DHCP IPv4 Leases and Status->DHCP IPv6 Leases pages by adding the description of the static lease (if available).

Done. Please merge if acceptable :)

Thanks, Frank!


Frontier - Add Gravity Bridge DAI, ETH, USDC, USDT

This commit adds metadata for major Ethereum tokens brought to Osmosis
over Gravity Bridge. This will allow users of these pools to see the
token name and decimals.

closed in favor of #144


Data permission

Thank you for your awesome work!
However, when I tried to download the dataset( https://osf.io/983qz/), I got a permission error as follows:

Please help me.
Best wishes.

Hi, I get the very same issue when trying to request access to the dataset on OSF:

Is there any alternative way to get the training datasets?
Thank you a lot in advance!

I contacted<EMAIL_ADDRESS>as suggested in the error message, and I received this answer:

It appears that project was caught in our spam filter. Thanks for alerting us, the project is public again. Don't hesitate to reach out in the future with additional questions or feedback.

I can see the dataset now!

Thank you for catching and resolving this issue, closing it since there isn't anything else for me to do.


Innovation search should allow filter by tags

This issue was moved to FirstToDisclose/firsttodisclose.net#114


PWN-8654 - NewTokenAdapterDiffCallback.kt fix - make areItemsTheSame same logic as areContentsTheSame

Jira Ticket
https://p2pvalidator.atlassian.net/browse/PWN-8654
Description of Work
PWN-8654 - NewTokenAdapterDiffCallback.kt fix - make areItemsTheSame same logic as areContentsTheSame

to release?


Tag v1.0.0

Let's tag a v1.0.0 before the end of September. I don't see the API changing much till then, just waiting for a bit more testing in production. Not blocked on #1.

Finally tagged. Lots of great improvements since the initial plan.


Failed: memory not established

In the documents tab I upload a PDF file and shows in the list.
But on the Chat tab, there is a message from the bot with the file name
To Reproduce
Steps to reproduce the behavior:

Go to 'Documents'
Click on 'Upload'
Upload PDF file
Go to 'Chat'
Ask "write a summary of the file"
Bot answers: I apologize, but I don't have access to the content of the file you mentioned. Is there anything else I can help you with?

Expected behavior
A clear and concise description of what you expected to happen.
The file should be in the list and the chat should suggest what to do with it.
I should be able to ask about the file. No?
Screenshots
If applicable, add screenshots to help explain your problem.

Platform

OS: Windows
IDE: N/A

Help fixing it please
Thank you

Thank you, I will explore these steps and inverstigate the behavior you've identified.

Two things going on here:

Copilot does not have a semantic understanding of the imported files as physical artifacts. (The files listed on the Documents are retreived from application storage. not semantic memory/vector db.)  In short, refering to "the file" is ambiguous for the model...but the memory was established.  As a test, please ask a question that pertains to some specific document content.

Examining the chat history in the prompt reveals one possible factor (empty chat entry):

We are currently in the processing of shifting our memory architecture and improving functionality in this area might be best addressed within that that scope (as opposed to modifying the current code).  Stay posted!

Requesting the summary of such a large document (stored as many chunks in the vectordb) means that no single prompt will be able to accesss the all the chunks for summarization.  A planner may be able to navigate this objective across multiple steps...unclear.  But if the document itself contains summary sections, then copilot can draw on those summaries.  In short, your satisfaction with the response to summarize may be dependent on the level of detail or completeness expected.

Please let me know if you have any issues or follow-up questions.  In the meantime, I'll raise this so the experience can be improved as part of the new memory architecture.

I notice that .TXT files are processed fine. only .PDF are an issue.
When will office files be readable for summarization and such?

I'm unable to differentiate behavior for .txt and .pdf
We are currently working on a new memory archicture that does support word documents and is extensbile to many document types.


Creating a GLFW_MAXIMIZED and GLFW_DECORATED window on Windows causes window to overlap taskbar

As requested:
From #899, which was fixed to make glfwMaximizeWindow() maximize to workarea rather than to full screen size.

This is still an issue for windows created with glfwWindowHint(GLFW_MAXIMIZED, GLFW_TRUE) and glfwWindowHint(GLFW_DECORATED, GLFW_FALSE).
A possible workaround is to call glfwMaximizeWindow() after window creation.

A similar fix as with https://github.com/glfw/glfw/commit/80d181f12dbb3b19fa649638252fa90eacf3d6a2 is required for the initial creation of the window here.
Best,
Jonathan

This should finally be fixed now with a730acf8e50e7efa70e79e42c511a97f27c97e4d. Thank you for your patience!


probability

Hi! Is there any way I can see the probability? Something like:
model = Subseq(1)
model.probability()

Hey @localcreator
Sorry for the late response, I don't get notifications and I don't know why...
What would be the probability here? The % of each "word" in the "corpus"? Or maybe you meant the probability of a prediction given a sequence?

Hi! Simple probability in %, just as it is implemented in sklearn. This fits better: "Or maybe you meant the probability of a prediction given a sequence?"

I am not familiar with probabilities in sklearn.
Are they relative probabilities with the different outcomes?
For instance, if the algorithm sees the two outcome A and B. And A is 3 times more likely, does it return A: 75%, B: 25% or more something like A: 6%, B: 2%, Others: 92%?
Currently, none of them are actually implemented. But I guess we could easily do something about the relative probabilities as an indicator and not an absolute truth

This fits better: "For instance, if the algorithm sees the two outcomes A and B. And A is 3 times more likely, does it return A: 75%, B: 25% or more something like A: 6%, B: 2%, Others: 92%?"
model.probability() simply prints these probabilities for each prediction. The result for the next forecast A, B, C, D may look like [0.7, 0.1, 0.05, 0.15]. And here it will be clear that the most likely forecast will be A[0.7]

It would make sense to have them to see how more likely a prediction is compared to another.
I think we could compute this from the calculated weights from FrequencyArray. And this is what get_k_best_letter does to get the predictions.
If you would like to give it a shot, I would be happy to help.
Otherwise, I will try to schedule this on my side, but this won't be done this week unfortunately

Unfortunately, I have little experience editing someone else's code. So far, the only hope is that someday you will have time to do this

Alright :+1:
Thanks for opening the issue, that's a good idea.
I will try to get time free soon :)

Hi! Can I find out if there is any progress in solving this issue?

Hello,
I was going to give my pet projects a bit of love next week.
I was also going to ask you if you still needs it. Apparently yes, I will try to release next version by next week :)

Yes, of course, I have been waiting with great desire and still continue to wait for this update for more than a year! Somewhere I have already heard about one week, it seems it was last year)))

Hello,
please kindly note that this is an unsponsored open-source project. I am working on this in my free time and take no financial benefits from it.
To be honest, there is no technical interest on my side to make this improvement, it was only to help you. Now, it's not even sure I will prioritize this.

I understand, sorry for the unfortunate joke, sir

In any case, regardless of your desire to continue working on this project, I am sincerely grateful for your work done!

Hello, I am having a look at this.
Would the weights be enough for your use case? Something like {'A': 0.04, 'B': 0.01} would be enough?
And so on client side you could compute {'A': 80%, 'B': 20%}.
I could also display the total_weights, but I am afraid the total_weights would confuse more than it would help in this kind of case.
What do you think?
I try to not take any mathematic liberties on the research paper I don't own. And I feel like showing percentages such as {'A': 0.75} would be a bit misleading, people would think subseq is sure at 75% that A is the prediction, which is wrong.

I don't know, it's worth a try. It is not yet clear why C is present in 2 sentences, but the weight is 8, and not, for example, 2 or 4. Here it is at your discretion, as you consider more correct.

Because internal weights is a bit more subtile than just counting the number of sentence in which you can find the letter.
Full code for reference is there. But maybe the research paper is clearer.
If that's okay for you, I will release this version tomorrow with a version for Python 3.11

Thanks! Yes, of course, that would be great!

That's published under version 1.0.4 !
I also released for python 3.11, you shouldn't have issues anymore.
Can you test and tell me if this is fine for your use case?
 I think this issue can be closed. Thank you for your help, sir!

Glad to hear that solves your issues :tada:
Sorry for the delay, and sorry for my bad reaction. I misunderstood your joke, I guess the written communication does not help.
Happy to help again in the future, don't hesitate to ping me on issues, I don't always receive an email for that.
Thanks for the star :)


feat: add fill_hl config option

Make the colors of the filler space configurable.
From :h syntax.txt, I think we should use TabLineFill instead of TabLine. This might break people's tabline color, so I make the highlight group a config option and naively pass it to the render() function.
							*hl-TabLine*
TabLine		Tab pages line, not active tab page label.
							*hl-TabLineFill*
TabLineFill	Tab pages line, where there are no labels.

This makes sense to me, I don't think the config variable makes sense to pass to render() though. It's already in the global config table, we can just use the global from within render() instead. I'll look over this in more detail tonight though.


Split Bill -Split Bill flow is blocked after the final button "Split $XX" is clicked

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Action Performed:

Open staging.new.expensify.com on Safari
Click FAB and select "Split Bill" or go to any group, click "+" and select "Split Bill"
Follow the flow to the last screen and click the final button "Split $XX"

Expected Result:
"Spli Bill" flow finished successfully and IOU preview appeared in the group
Actual Result:
"Split Bill" flow is blocked after the final button "Split $XX" is clicked. The cursor has a "Blocked" icon
Workaround:
Unknown
Platforms:

Which of our officially supported platforms is this issue occurring on?

[ ] Android / native
[ ] Android / Chrome
[ ] iOS / native
[ ] iOS / Safari
[x] MacOS / Chrome / Safari
[ ] MacOS / Desktop

Version Number: <IP_ADDRESS>
Reproducible in staging?: Yes
Reproducible in production?: Yes
If this was caught during regression testing, add the test name, ID and link from TestRail:
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Notes/Photos/Videos: Any additional supporting documentation
https://github.com/Expensify/App/assets/78819774/da8b4939-183d-449a-85ba-fbbd97054ee6
https://github.com/Expensify/App/assets/78819774/ff8814d4-f044-42a2-af71-134d9f4d817b
Expensify/Expensify Issue URL:
Issue reported by: Applause - Internal Team
Slack conversation:
View all open jobs on GitHub

I can't reproduce on the latest staging

I'm also unable to reproduce this one on staging. Closing.


Support show stopper validator

The flow will support a show stopper validation error.
The next validator will no be applied when some validator returns the show stopper error.

retest this please

retest this please


Python 3.7 Alpine (3.7.3) unittest.mock.MagicMock, pytest-cov and isinstance failure

The tag python:3.7.3-alpine created on March 28, 2019 08:47 AM has an issue/weird interaction with unittest.mock.MagicMock, pytest/pytest-cov and isinstance. The isinstance(...) check fails on MagicMock objects... even when setting the spec kwarg.
Here is an example:

Test folder structure:

- test -
        |_ requirements.txt
        |_ test_isinstance.py
        |_ Dockerfile

File: requirements.txt:

pytest==4.1.1
pytest-cov==2.6.1

File: test_isinstance.py:

from unittest.mock import MagicMock

class ClassA:
    pass

class TestIsInstance:

    def test_isinstance(self):
        class_a_mock = MagicMock(spec=ClassA)
        assert isinstance(class_a_mock, ClassA)

File: Dockerfile:

FROM python:3.7.3-alpine
ENV PYTHONPATH=/home/python/
ADD requirements.txt /home/python/requirements.txt
RUN pip install --no-cache-dir -r /home/python/requirements.txt
WORKDIR /home/python
ADD test_isinstance.py /home/python/test_isinstance.py

Build the image:docker build -t isinstance .
Run the tests:

With Coverage (fails):

➜ docker run --rm isinstance python -m pytest --cov=test_isinstance test_isinstance.py
============================= test session starts ==============================
platform linux -- Python 3.7.3, pytest-4.1.1, py-1.8.0, pluggy-0.9.0
rootdir: /home/python, inifile:
plugins: cov-2.6.1
collected 1 item

test_isinstance.py F                                                     [100%]

=================================== FAILURES ===================================
________________________ TestIsInstance.test_isinstance ________________________

self = <test_isinstance.TestIsInstance object at 0x7f37d947b908>

    def test_isinstance(self):
        class_a_mock = MagicMock(spec=ClassA)
>       assert isinstance(class_a_mock, ClassA)
E       AssertionError: assert False
E        +  where False = isinstance(<MagicMock spec='ClassA' id='139877845284904'>, ClassA)

test_isinstance.py:12: AssertionError

----------- coverage: platform linux, python 3.7.3-final-0 -----------
Name                 Stmts   Miss  Cover
----------------------------------------
test_isinstance.py       7      0   100%

=========================== 1 failed in 0.32 seconds ===========================

Without Coverage (passes):

➜ docker run --rm isinstance python -m pytest test_isinstance.py
============================= test session starts ==============================
platform linux -- Python 3.7.3, pytest-4.1.1, py-1.8.0, pluggy-0.9.0
rootdir: /home/python, inifile:
plugins: cov-2.6.1
collected 1 item

test_isinstance.py .                                                     [100%]

=========================== 1 passed in 0.04 seconds ===========================

The above does not happen on earlier versions of the image, e.g. python:3.7.2-alpine.

I wanted to add that I also have python 3 installed locally and the tests work fine:
➜ python --version
Python 3.7.3

➜ python -m pytest --cov=test_isinstance test_isinstance.py
=================================================================== test session starts ===================================================================
platform darwin -- Python 3.7.3, pytest-4.1.1, py-1.7.0, pluggy-0.8.1
rootdir: /path/to/test, inifile:
plugins: requests-mock-1.5.2, cov-2.6.1
collected 1 item

test_isinstance.py .                                                                                [100%]

---------- coverage: platform darwin, python 3.7.3-final-0 -----------
Name                 Stmts   Miss  Cover
----------------------------------------
test_isinstance.py       7      0   100%

================================================================ 1 passed in 0.06 seconds =================================================================

Alpine uses  musl libc instead of glibc and friends, so certain software might run into issues depending on the depth of their libc requirements.
https://github.com/docker-library/docs/tree/master/python#pythonversion-alpine
python:3.7.3-slim is probably what you want
$ docker image ls | grep '3.7.3'
python                                     3.7.3-alpine        a93594ce93e7        4 days ago          90.9MB
python                                     3.7.3-slim          6ba8638f69d7        4 days ago          143MB
python                                     3.7.3               59a8c21b72d4        4 days ago          929MB

@wglambert I think it had to do with it. After switching to python:3.7.3-slim it seemed to work properly. Thanks.
I do have to add, it is weird that this does work for python:3.7.2-alpine so something must have changed between 3.7.2-alpine and 3.7.3-alpine.

Nothing changed in the way we package or build Python in the image between those two versions, so this was either a change in Alpine or a change in Python (with the more likely candidate being the latter).
The best I can suggest is digging through https://github.com/python/cpython/compare/v3.7.2...v3.7.3 to see if anything seems relevant to have broken musl support in this language API, unfortunately.
I think it's worth pointing out that Python upstream really doesn't support musl very well officially, so I think any fix is going to be best-effort (and will have a much higher chance if there's a patch and it's well-tested and obviously doesn't break glibc).

Fair enough, thank you for the feedback @tianon

:+1:
I'm going to close this for now since there doesn't appear to be much (if anything) we can do about this here in the image and that fixes will likely have to happen upstream.


Rebrand

Changes references of PolyMC to Prism Launcher including variables.

[ ] Ensure docker paths work
[ ] Ensure directories work
[ ] Rename meta-polymc repo to (or create) meta-prismlauncher

We could also just make this generic and change everything to launcher

I can do that instead if you want. I don't have a preference.

Done


Add necessary accessibility properties to content navigation component

Make sure that content navigation component is using a navigation landmark (enclosed into nav tag) and has ARIA label (has aria-labelledby attribute).
Make sure that content menu’s visual heading (which is usually “On this page”) has id attribute that is referenced in nav’s aria-labelledby attribute.

@zakiya since you are currently working on it maybe you can incorporate this a11y fixes?

This is done and merged.