Layer combinations causing some scene objects to not render

I have a program that works with validation layers off, but if I turn on certain validation layers, certain graphics pipelines will fail to render their objects, and yet no error (or warning) is reported from the validation layers via the debug report.
The debug callback does print output when I have written bad code, and also whenever I turn on the information or debug bits, so I'm quite sure output would be printed if it was returned by the layer.
The clear color always renders, it is just the scene objects drawn within a bound pipeline that don't render (cmd_draw_indexed after binding descriptor sets, vertex buffers, and index buffers).
The pattern of which layers affect the rendering is oddly complex.  Of the 6 standard validation layers, there are 2^6 combinations of enabling of them.  I've tried most of them, and when I try them all I will post my results here, but it is not clearly just one layer, nor is any layer clearly exempt.  I'm doing this testing on <IP_ADDRESS>, but there were similar problems with <IP_ADDRESS> and I haven't gone back to see if the behavior is identical.
The most interesting combination so far is with "parameter validation" combined with "swapchain" OR "unique objects"... those combinations render objects in one graphics pipeline, but not the other.
I am using linux with nvidia drivers, and I've reproduced this same behavior on multiple computers now.  This behavior started after nvidia driver 375, but persists in 378.  It did not occur prior to 375 but I can no longer verify this (ubuntu wont allow me to revert the driver).  The behavior also occurs on Arch Linux under nvidia driver 378.  It happens with both my nvidia cards (GTSX 1050 Ti and GTX 650).  So it is quite reproducable.
I happen to be programming in rust and I am using the rust ash library, but I've scoured its code and I don't believe this to be relevant.

Combinations that cause both pipelines of objects to not render:
(threading & object tracker & core validation)
or (threading & parameter validation)
or (parameter validation & core validation)
or (parameter validation & object tracker)
Combinations that break one of the two pipelines:
(parameter validation & swapchain -- without unique objects)
or (parameter validation & unique objects -- without swapchain)
oddly, parameter validation & swapchain & unique objects combined renders fine.

@mikedilger, this is just creepy.  Is your program using any extensions at all?  The codegen for some of these is tricky, especially related to the unique_objects layer.
And param_val + uniqueobjects fails, but then adding inthe swapchain layer works?  Yikes.
Is it possible that you could share something we could debug here?

I agree it's creepy.  It feels like memory corruption.
I have created a private repo and added you as a collaborator.  The README should explain how to get into the situation where you can see/test the behavior

I'm using Surface, XlibSurface and DebugReport extensions

Hey @mikedilger, tried to build this morning but no joy.  There seem to be a lot of moving parts here and to simplify thing maybe we could try one of these options:  1) get us a 'simple' straight Vulkan app that exhibits the issue, i.e., without the intermediate libraries; 2) Add a case to vk_layer_validation_tests that shows the problem; or 3) send us a trace and/or api_dump of  working and failing cases.
Also, the use case we should entirely focus on here is running with all the layers or using VK_LAYER_LUNARG_standard_validation instead.

I've just stripped out the networking stuff and the unused 'sarek' and 'siege-gui' crates, in order to get closer to "A simple straight Vulkan app".
I fixed the compilation issue (ring crate was yanked due to a security issue).
I think it should work again.
How do I get an api dump?  I remember doing that once, but google is being especially unhelpful right now.

@mikedilger , thanks for the update -- we have been able to reproduce the issue here.  Trying to rule out driver issues first.  So far, it seems we only get visible output with no layers, or with just parameter_validation.

@mikedilger, we duplicated the behavior on an Intel system, so it's less likely to be a driver issue. Our next two cool ideas were that it might be a thread synchronization problem -- can you combine your window and graphics threads to see if that makes a difference?  And we are unable to tell if XInitThreads was called before XOpenDisplay, which could be very important.  Let us know and we'll go from there...

I've combined the graphics and window threads.  That did not make a difference.
I've updated to the newest winit crate as it had some x11 related bugs, but it made no difference.
As for XInitThreads and XOpenDisplay, the code in winit looks like this:
impl XConnection {
    pub fn new(error_handler: XErrorHandler) -> Result<XConnection, XNotSupported> {
        // opening the libraries
        let xlib = try!(ffi::Xlib::open());
        let xcursor = try!(ffi::Xcursor::open());
        let xf86vmode = try!(ffi::Xf86vmode::open());
        let xinput2 = try!(ffi::XInput2::open());
        let xlib_xcb = try!(ffi::Xlib_xcb::open());

        unsafe { (xlib.XInitThreads)() };
        unsafe { (xlib.XSetErrorHandler)(error_handler) };

        // calling XOpenDisplay
        let display = unsafe {
            let display = (xlib.XOpenDisplay)(ptr::null());
            if display.is_null() {
                return Err(XNotSupported::XOpenDisplayFailed);
            }
            display
        };

        Ok(XConnection {
            xlib: xlib,
            xf86vmode: xf86vmode,
            xcursor: xcursor,
            xinput2: xinput2,
            xlib_xcb: xlib_xcb,
            display: display,
            latest_error: Mutex::new(None),
        })
    }
...
}

@mikedilger, bummer.  To speed things up and get some smarter folks involved, can you please add @jeremy-lunarg and @Tony-LunarG to your private repo?

done

Interesting. If I set breakpoints on both occurrences of get_physical_device_properties, I can make known bad configurations behave correctly. And, coincidentally, I noticed in api-dumps of known good configurations that there are three calls to get_physical_device_properties (as opposed to two). Getting closer...

I shouldn't have been calling it twice, and I just pushed a commit to call it only once.  I can't test it because I'm on FreeBSD at the moment.

Any progress?
I think get_physical_device_properties() would have been called on each physical device in a loop, and then prior to my last commit, one more time in the renderer for setting up the memory allocator.  With 2 devices, you would expect 3 calls from only 2 code points.
If breakpoints themselves can cause rendering to come back, ... maybe it is a memory alignment issue?
What I notice is that the things that make it render or fail to render are subtle tiny changes having nothing to do with anything seemingly remotely related.  It's like gremlins.
One possibility is that I am corrupting some memory somewhere, which has difficult-to-trace follow-on effects.  That's what is seems like to me, and even though rust is a memory-safe language, there are tons of 'unsafe' regions meaning I easily could have messed up memory and the rust compiler would not have protected me.  This would usually give a SEGV, but not always.
More likely, I passed a bad pointer or bad parameter (e.g. wrong size) and libvulkan using my bad pointer overwrote some internal structures causing the rendering to fail.  I will keep looking.

When it doesn't render, I noticed in the api_dump the results from getPhysicalDeviceMemoryProperties() are quite different - instead of mostly being HOST_COHERENT and HOST_CACHED bits, its mostly DEVICE_LOCAL and LAZILY_ALLOCATED.  I thnk there is a bug in there somewhere.

I have spent a decent amount stumped on this one. :weary:
I did several diffs using api_dump and ltrace. Also did some heap checking with Valgrind. Everything "appears" reasonable. Then, I tried the layers with the demos that come with vulkano. They worked as expected. My gut feeling is there's some memory stomping going on your application. But, of course, now that I say that we'll probably find a bug in the layers. :smirk:

If breakpoints themselves can cause rendering to come back, ... maybe it is a memory alignment issue?

This ended up being my inexperience with Rust debugging. Not a real observation. Sorry about that.

Thanks Jeremy and Mark for looking into this. I've been rebuilding my engine, partly because of this bug, partly to switch to a different system level library (dropping ash, choosing vks... still not using vulkano as it's still too immature) and also I know vulkan better now.
So maybe this just remains a mystery for now (until I get curious again).

I'm still curious. :wink:

Closing for now.  @mikedilger, if you come across a solid repro case just reopen this puppy and we'll get right on it.  Thanks!


Community Feedback Request - Release a 2.0 if React removes the patents clause?

One of the big reasons that Phosphor includes its own virtual DOM implementation, is because of React's patents clause. In the wake of the Apache foundation forbidding the "react+patents" license in dependencies, Facebook relicensed RocksDB as dual Apache/GPL. It's reasonable to assume that React will follow suit:
https://github.com/facebook/react/issues/10191
Should that happen, what is the communities feedback on releasing a Phosphor 2.0 which does not include @phosphor/virtualdom and replaces all internal uses of that package with React?
If we do this, it will be the first external runtime dependency for Phosphor.
cc @ellisonbg @jasongrout @birdsarah @bollwyvl @jdetle @blink1073 @afshin @ian-r-rose @gnestor

I think this would be a really great thing! It would save @sccolbert a ton of work and open up the project to a lot more contributors from the React community. It would also allow bundled extensions to make use of open source React components. It could also make it more straight-forward to provide and consume JupyterLab UI components (e.g. dropdown, right-click menu, toolbar, etc.).

I agree completely with grant. Should it happen it makes total sense to
switch over.
On Mon, Jul 17, 2017 at 12:23 AM Grant Nestor<EMAIL_ADDRESS>wrote:

I think this would be a really great thing! It would save @sccolbert
https://github.com/sccolbert a ton of work and open up the project to a
lot more contributors from the React community. It would also allow bundled
extensions to make use of open source React components. It could also make
it more straight-forward to provide and consume JupyterLab UI components
(e.g. dropdown, right-click menu, toolbar, etc.).
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/phosphorjs/phosphor/issues/302#issuecomment-315670341,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AKcARroqpR6j3IiqEucyvvWfz8icWay7ks5sOu_PgaJpZM4OZb_a
.

--
Have a wonderful day,
John J. Detlefs

I also concur with Grant.

I agree with @gnestor as well
Even without the benefit broader community uptake or support, these gains are themselves nontrivial:

100% inter-compatibility guarantee when functional components are exported
Free high-end implementation of virtual DOM whose ongoing development comes from upstream and is reliable

What components use a VDOM implementation right now? The menu bar, the tab bar, and the command palette?
One thing not addressed in the above notes: of all the VDOM implementations out there, why React specifically? For one, it clearly has the largest community adoption, and presumably the greatest momentum. It does seem like we can't make the vdom implementation swappable because of the hard dependency.
Thinking of embedding one of these phosphor components in another page: Are there any problems with having React as a dependency that interfere with other uses of React on a page? Does React require being a singleton, so there would need to be some sort of deduping and possible version conflicts if React is used elsewhere on the page?

What components use a VDOM implementation right now? The menu bar, the tab bar, and the command palette?

CommandPalette, Menu, MenuBar, and TabBar.

why React specifically

For all the reasons you mention.

Are there any problems with having React as a dependency that interfere with other uses of React on a page?

React would just need to be importable through whatever bundler you choose. We wouldn't be shipping our own copy.

React would just need to be importable through whatever bundler you choose. We wouldn't be shipping our own copy.

Oh right, I forgot you aren't distributing a webpack bundle.

Are there any problems with having React as a dependency that interfere with other uses of React on a page? Does React require being a singleton, so there would need to be some sort of deduping and possible version conflicts if React is used elsewhere on the page?

Yes, having more than 1 version of React loaded on a page can definitely cause issues. However, seeing as though we are using webpack to bundle everything together (including 3rd party extensions.), we should be able to dedupe and control which version of React is provided.

I would be in favor of using React if the license change follows.
On Mon, Jul 17, 2017 at 7:30 AM, Grant Nestor<EMAIL_ADDRESS>wrote:

Are there any problems with having React as a dependency that interfere
with other uses of React on a page? Does React require being a singleton,
so there would need to be some sort of deduping and possible version
conflicts if React is used elsewhere on the page?
Yes, having more than 1 version of React loaded on a page can definitely
cause issues. However, seeing as though we are using webpack to bundle
everything together (including 3rd party extensions.), we should be able to
dedupe and control which version of React is provided.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/phosphorjs/phosphor/issues/302#issuecomment-315772646,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AABr0E00_m9jFnNyCSRUwolnL6LvIh09ks5sO2_7gaJpZM4OZb_a
.

--
Brian E. Granger
Associate Professor of Physics and Data Science
Cal Poly State University, San Luis Obispo
@ellisonbg on Twitter and GitHub
<EMAIL_ADDRESS>and<EMAIL_ADDRESS>

Anything that deduplicates effort seems like a good thing.
We have been building up a collection of ui elements that we call ui-kit that all rely on VDom. We have the intention to make it public, just haven't yet. This has been a decent amount of work and if there's a chance this means that we could move to something from a larger community effort that would be great.

It appears that Facebook has reached a no-op decision:
https://code.facebook.com/posts/112130496157735/explaining-react-s-license
I am very disheartened by this. The most painful part is that the post appears to clarify Facebook's intent to use the patent clause as a weapon in their overall patent strategy. I would summarize their position in this way, "if you are feeling uncomfortable with the react.js patent retaliation clause, then good, cause that is why it is there."
I will be working with NumFOCUS folks this week to figure out a path forward.

Yep. Saw that last night :(
It seems the wider community is also realizing they weren't paying close enough attention to the clause. https://news.ycombinator.com/item?id=15050841

https://medium.com/@raulk/if-youre-a-startup-you-should-not-use-react-reflecting-on-the-bsd-patents-license-b049d4a67dd2

In light of this, I think the safest way forward right now is to not adopt React in Phosphor.

https://code.facebook.com/posts/300798627056246

Reopening in light of the license change.

@phosphor/virtualdom is currently about 940 lines of code, of which maybe a half is element, attribute and property names. Doesn't sound so nice to depend on React just for that :/
For me the best things about PhosphorJS are TypeScript and that it doesn't bring in junk like lodash, jQuery, React and dozens of totally useless and incompetent left-pad style packages. How much extra work would it be to design so that React is the "official" vdom implementation, but 3rd parties could replace it?
Dojo 2 will use maquette which is also all TypeScript and with a codebase about the size of the current @phosphor/virtualdom. Not saying it should be chosen instead, but if there's a suitable abstraction in place in PhosphorJS, for example I could maintain a phosphor-maquette package to replace React in my own projects.

What would it take to have phosphor widgets that are compatible with react, while still maintaining widgets that are compatible with the phosphor vdom?

@jdetle not worth the effort.

@jjrv, an API compatible virtual DOM implementation (like preact-compat or a shim on top of something like maquette) could be swapped out at bundle time using something like require.alias in WebPack.

@blink1073 in that case it would be nice if a subset of React API not much larger than the current vdom API was used. Any necessary shim could then be kept small. I'm sure PhosphorJS would also work great in some Angular projects and it would be silly for them to also include React.

To be clear, we're only talking about @phosphor/widgets here. The rest of the Phosphor packages don't depend on vdom.

@jjrv It's also equally silly for me to maintain a separate vdom library at this point, given that React is more complete, has a large community, several useful third part components, and now an MIT license.

Why not use Inferno instead? Its it is smaller and much faster than React, and in particular its virtual DOM diff implementation is much faster than React's.
https://github.com/infernojs/inferno

Hi, How can we make a react-phosphor / react-jupyterlab type thing so we can embed JupyterLab in our React app and share state? Typescript noob but studying it now

@saulshanabrook's response on Gitter:

One short answer: You can't. Not only is JupyterLab not built with React it doesn't have global state storage.

Longer answer: You could wrap JupyterLab in a React component, but you would have to manually link the state to any changes in JupyterLab's application. We have been transitioning some of JupyterLab to use React internally and if you are interested in solving this problem long term, we would love to have your help with that. If that is something you are interested in, I am happy to help get you started there.

Which parts of JupyterLab are you looking to control externally?

@gnestor

We need to be able to control the location / dimensions of the lab, so it can fit inside a desired position on the app (Potentially options.hostId passed to jupyterlab can control this)

We need to pass maps of nodes / links / selections to jupyterlab kernel, so user can prototype graph algorithms quickly. When the kernel updates the graph, it could dispatch CRUD operations to the parent react component, and when the user updates graph data in the app, it could pass this data as a prop to the jupyterlab

like
then inside jupyterlab component we could just render the lab one time, and watch for changes to the graph data, and dispatch updates to react state

This all sounds possible but I can't say for sure without seeing some example code.

We need to be able to control the location / dimensions of the lab, so it can fit inside a desired position on the app (Potentially options.hostId passed to jupyterlab can control this)

I'm not familiar with options.hostId but you can certainly put JupyterLab in an iframe and control the dimensions of that.

We need to pass maps of nodes / links / selections to jupyterlab kernel, so user can prototype graph algorithms quickly. When the kernel updates the graph, it could dispatch CRUD operations to the parent react component, and when the user updates graph data in the app, it could pass this data as a prop to the jupyterlab

You can create a labextension that hooks into kernels and respond to kernel events, messages, etc. and communicate with the parent React app from there.

here's a repo where i've attempted to create a react app for jupyterlab
https://github.com/bionicles/lab-component
I added index.js of examples/app to a "useEffect" hook inside a boilerplate create-react-app
"yarn start" and it tries to work but cannot find a kernel server and has some bugs:
the jupyterlab stuff automatically fills the whole screen when we want it to just fill the parent div
Unexpected token < in JSON at position 0
TypeError: Cannot read property '0' of undefined
next I tried adding main.py also from examples/app -- but python main.py fails as there is no template.html

For reference, @bionicles created https://github.com/jupyterlab/jupyterlab/issues/6380 to continue the conversation about embedding in React.

@jasongrout recently brought up the point that by depending on React in @phosphor/widgets, we would have to bump the major version of our library when React has a major version bump.  The TypeScript definitions for React have also been historically brittle.  In JupyterLab, we have been forced to declare the types as a pinned dependency.
I think instead we should make it easier to use React with Phosphor.

Add the RenderWidget
Utilize the fact that jsx can now be declared per module in TypeScript
Add a pass through for React renderers to the Phosphor vdom
Add examples of using React in a Phosphor widget and vice versa.
Investigate allowing React components to be used as the renderers for Phosphor widgets.


getcompletion() can get cmdline-completion result

I have extended getcompletion().
getcompletion() can get cmdline-completion result.
I think it is useful to implement Vim completion.

Note:  Current problem {pat} is partial matched when {type} is "cmdline".
It is not big problem.  But I don't know why.

Instead of duplicating the code, you can use the following:
 ExpandInit(&xpc);

if (STRCMP(get_tv_string(&argvars[1]), "cmdline") == 0)

{

  set_one_cmd_context(&xpc, get_tv_string(&argvars[0]));

  xpc.xp_pattern_len = (int)STRLEN(xpc.xp_pattern);

}

else

{
xpc.xp_pattern = get_tv_string(&argvars[0]);
xpc.xp_pattern_len = (int)STRLEN(xpc.xp_pattern);
xpc.xp_context = cmdcomplete_str_to_type(get_tv_string(&argvars[1]));
@@ -4459,6 +4466,7 @@ f_getcompletion(typval_T *argvars, typval_T *rettv)
xpc.xp_pattern_len = (int)STRLEN(xpc.xp_pattern);
}
#endif

}
pat = addstar(xpc.xp_pattern, xpc.xp_pattern_len, xpc.xp_context);
if ((rettv_list_alloc(rettv) != FAIL) && (pat != NULL))

Fixed the problems.

Note: Current problem {pat} is partial matched when {type} is "cmdline".
It is not big problem. But I don't know why.

The problem is fixed.

I have added {type} type check.


Gh 885 reduce times byte array is copied

Can't reopen the old pull request, but will review comments.

Codecov Report

Merging #1020 into develop will increase coverage by 1.66%.
The diff coverage is 82.55%.

@@              Coverage Diff              @@
##             develop    #1020      +/-   ##
=============================================
+ Coverage      61.96%   63.62%   +1.66%
+ Complexity      1077      965     -112
=============================================
  Files            510      484      -26
  Lines          15308    13976    -1332
  Branches        1744     1589     -155
=============================================
- Hits            9485     8892     -593
+ Misses          5081     4440     -641
+ Partials         742      644      -98

Impacted Files
Coverage Δ
Complexity Δ

...liser/implementation/StringToStringSerialiser.java
60% <ø> (ø)
3 <0> (?)

...isation/ToStringSerialiser/ToStringSerialiser.java
50% <ø> (ø)
1 <0> (?)

...ulostore/key/core/impl/CoreKeyGroupByCombiner.java
64.74% <ø> (+2.52%)
0 <0> (ø)
:arrow_down:

...v/gchq/gaffer/serialisation/ToBytesSerialiser.java
25% <0%> (-75%)
2 <0> (ø)

...erialisation/implementation/BooleanSerialiser.java
25% <0%> (+25%)
2 <0> (+2)
:arrow_up:

...v/gchq/gaffer/commonutil/ByteArrayEscapeUtils.java
0% <0%> (ø)
0 <0> (ø)
:arrow_down:

...n/implementation/raw/CompactRawLongSerialiser.java
71.42% <100%> (-11.91%)
5 <1> (ø)

...y/core/impl/byteEntity/ByteEntityRangeFactory.java
94.7% <100%> (+0.03%)
0 <0> (ø)
:arrow_down:

...r/serialisation/implementation/JavaSerialiser.java
70.37% <100%> (ø)
8 <2> (+1)
:arrow_up:

.../impl/classic/ClassicAccumuloElementConverter.java
80.3% <100%> (-1.13%)
0 <0> (ø)

... and 149 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ce05edb...06f9f08. Read the comment docs.

@gaffer01 This is latest branch I'd like you to take a look at.


CORE-6406 bash script to allow install of cli and plugins from s3 bucket

This installer will allow install of the cli and plugins via curl  | bash <-v for checksum verification>

Looks like there is a genuine build failure with this PR atm:

[2022-09-07T16:24:09.042Z] Caused by: org.gradle.api.publish.maven.InvalidMavenPublicationException: Invalid publication 'maven': artifact file does not exist: '/var/jenkins/workspace/Corda5_corda-runtime-os_PR-1990/tools/plugins/build/generatedScripts/corda-cli-installer-script.sh'

[2022-09-07T16:24:09.042Z] 	at org.gradle.api.publish.maven.internal.publisher.ValidatingMavenPublisher.checkCanPublish(ValidatingMavenPublisher.java:129)

[2022-09-07T16:24:09.042Z] 	at org.gradle.api.publish.maven.internal.publisher.ValidatingMavenPublisher.validateArtifacts(ValidatingMavenPublisher.java:103)

[2022-09-07T16:24:09.042Z] 	at org.gradle.api.publish.maven.internal.publisher.ValidatingMavenPublisher.publish(ValidatingMavenPublisher.java:47)

[2022-09-07T16:24:09.042Z] 	at org.gradle.api.publish.maven.tasks.PublishToMavenLocal$1.publish(PublishToMavenLocal.java:60)

[2022-09-07T16:24:09.042Z] 	at org.gradle.api.publish.internal.PublishOperation.run(PublishOperation.java:43)

[2022-09-07T16:24:09.042Z] 	... 125 more

Looks like there is a genuine build failure with this PR atm:

[2022-09-07T16:24:09.042Z] Caused by: org.gradle.api.publish.maven.InvalidMavenPublicationException: Invalid publication 'maven': artifact file does not exist: '/var/jenkins/workspace/Corda5_corda-runtime-os_PR-1990/tools/plugins/build/generatedScripts/corda-cli-installer-script.sh'

[2022-09-07T16:24:09.042Z] 	at org.gradle.api.publish.maven.internal.publisher.ValidatingMavenPublisher.checkCanPublish(ValidatingMavenPublisher.java:129)

[2022-09-07T16:24:09.042Z] 	at org.gradle.api.publish.maven.internal.publisher.ValidatingMavenPublisher.validateArtifacts(ValidatingMavenPublisher.java:103)

[2022-09-07T16:24:09.042Z] 	at org.gradle.api.publish.maven.internal.publisher.ValidatingMavenPublisher.publish(ValidatingMavenPublisher.java:47)

[2022-09-07T16:24:09.042Z] 	at org.gradle.api.publish.maven.tasks.PublishToMavenLocal$1.publish(PublishToMavenLocal.java:60)

[2022-09-07T16:24:09.042Z] 	at org.gradle.api.publish.internal.PublishOperation.run(PublishOperation.java:43)

[2022-09-07T16:24:09.042Z] 	... 125 more

Think this is down to the question you asked me @ronanbrowne I suspect the task that needs to be ran to actually generate that script isn't being invoked in the pipeline


NPM package updates; Storybook, downshift, focus-trap-react, immer, redux, uuid

This PR updates a set of core admin UI related packages, many of which have not been updated for a few years and are no longer maintained in their current version.
Details

Core updates to several admin user facing libraries, many of these packages are years out of date and needed some attention.
Longer term, I think we could simplify our package landscape (especially the immer/Redux tools), for now though this PR does the minimum possible changes for us to adopt the latest versions of these packages.
Small fix to Eslint for Storybook checks, found while fixing the Storybook changes.

Updates applied

Package
Old Version
New Version

Major

◯ @storybook/addon-controls
^7.6.14
^8.4.2

◯ @storybook/addon-docs
^7.6.14
^8.4.2

◯ @storybook/react
^7.6.14
^8.4.2

◯ @storybook/react-webpack5
^7.6.14
^8.4.2

◯ downshift
^7.6.2
^9.0.8

◯ focus-trap-react
^8.11.3
^10.3.0

◯ immer
^9.0.21
^10.1.1

◯ react-redux (v9 blocked)
^7.2.2
^8.1.3

◯ redux
^4.2.1
^5.0.1

◯ redux-thunk
^2.4.2
^3.1.0

◯ reselect
^4.1.8
^5.1.1

◯ storybook
^7.6.14
^8.4.2

◯ storybook-django
^0.5.1
^1.0.0

◯ uuid
^9.0.1
^11.0.2

Minor

◯ eslint (major blocked, next minor only)
^8.56.0
^8.57.1

◯ eslint-plugin-storybook
^0.6.15
^0.11.0

Other blocked updates

We are starting to be critically blocked by our Draftail dependencies relying on specific React versions, beyond that, I have been able to update a lot of libraries.
I did not adjust the draft-js package as it was causing an error when trying to update, due to  our draftail 2.0.0-rc usage, this will require more time and input from @thibaudcolas probably. See also https://github.com/springload/draftail/issues/213#issuecomment-826158209
Stylelint - waiting on PR https://github.com/wagtail/stylelint-config-wagtail/pull/45
Eslint - waiting on issue to be worked on https://github.com/wagtail/eslint-config-wagtail/issues/22

Testing

Comments will be impacted, smoke testing required for a set of comments behaviours, uses Redux packages, Immer and Focus trap react. Note that the bug https://github.com/wagtail/wagtail/issues/11021 is not related to this package update and is an existing bug.
Storybook (running locally), there are console warnings shown but these are not new and not fundamental issues.
Page explorer - It uses Redux and the focus trap component.
StreamField / Rich text - Combo box (block chooser) - This uses the Downshift library.

Relevant upgrade release notes

Storybook & related packages

https://storybook.js.org/docs/migration-guide

focus-trap-react - v8 to v10

https://github.com/focus-trap/focus-trap-react/releases/tag/v9.0.0
https://github.com/focus-trap/focus-trap-react/releases/tag/v10.0.0

downshift - v7 to v9

https://github.com/downshift-js/downshift/releases/tag/v9.0.0
https://github.com/downshift-js/downshift/releases/tag/v8.0.0
https://github.com/downshift-js/downshift/blob/master/src/hooks/MIGRATION_V8.md

uuid - v9 to v11

https://github.com/uuidjs/uuid/blob/main/CHANGELOG.md#1100-2024-10-26
https://github.com/uuidjs/uuid/blob/main/CHANGELOG.md#1000-2024-06-07

immer v9 to v10

https://github.com/immerjs/immer/releases/tag/v10.0.0

Update Redux ecosystem

https://github.com/reduxjs/redux-thunk/releases/tag/v3.1.0
https://github.com/reduxjs/redux/releases/tag/v5.0.0
https://github.com/reduxjs/reselect/releases/tag/v5.0.1
https://github.com/reduxjs/react-redux/releases/tag/v8.0.0 (note: react-redux cannot be updated further due to React dependency, which is blocked by Draftail)

Outdated package reference
Note: These are not the updates I have done (see above for that), instead this is a reference of what updates still need to be done before & after this PR.
Before

31 outdated packages

Package
Old Version
New Version

Major

◯ @storybook/addon-controls
^7.6.14
^8.4.2

◯ @storybook/addon-docs
^7.6.14
^8.4.2

◯ @storybook/react
^7.6.14
^8.4.2

◯ @storybook/react-webpack5
^7.6.14
^8.4.2

◯ @types/react
^16.14.21
^18.3.12

◯ @types/react-dom
^16.0
^18.3

◯ @typescript-eslint/eslint-plugin
^6.21.0
^8.13.0

◯ @typescript-eslint/parser
^6.21.0
^8.13.0

◯ css-loader
^6.10.0
^7.1.2

◯ cssnano
^5.1.15
^7.0.6

◯ downshift
^7.6.2
^9.0.8

◯ eslint
^8.56.0
^9.14.0

◯ focus-trap-react
^8.11.3
^10.3.0

◯ immer
^9.0.21
^10.1.1

◯ react
^16.14.0
^18.3.1

◯ react-dom
^16.14.0
^18.3.1

◯ react-redux
^7.2.2
^9.1.2

◯ react-test-renderer
^16.14.0
^18.3.1

◯ react-transition-group
^1.1.3
^4.4.5

◯ redux
^4.2.1
^5.0.1

◯ redux-thunk
^2.4.2
^3.1.0

◯ reselect
^4.1.8
^5.1.1

◯ sass-loader
^14.1.0
^16.0.3

◯ storybook
^7.6.14
^8.4.2

◯ storybook-django
^0.5.1
^1.0.0

◯ stylelint
^15.11.0
^16.10.0

◯ uuid
^9.0.1
^11.0.2

Major version zero

◯ @types/draft-js
^0.10.45
^0.11.18

◯ draft-js
^0.10.5
^0.11.7

◯ eslint-plugin-storybook
^0.6.15
^0.11.0

◯ telepath-unpack
^0.0.3
^0.0.4

After

17 outdated packages

Package
Old Version
New Version

Major

◯ @types/react
^16.14.21
^18.3.12

◯ @types/react-dom
^16.0
^18.3

◯ @typescript-eslint/eslint-plugin
^6.21.0
^8.13.0

◯ @typescript-eslint/parser
^6.21.0
^8.13.0

◯ css-loader
^6.10.0
^7.1.2

◯ cssnano
^5.1.15
^7.0.6

◯ eslint
^8.57.1
^9.14.0

◯ react
^16.14.0
^18.3.1

◯ react-dom
^16.14.0
^18.3.1

◯ react-redux
^8.1.3
^9.1.2

◯ react-test-renderer
^16.14.0
^18.3.1

◯ react-transition-group
^1.1.3
^4.4.5

◯ sass-loader
^14.1.0
^16.0.3

◯ stylelint
^15.11.0
^16.10.0

Major version zero

◯ @types/draft-js
^0.10.45
^0.11.18

◯ draft-js
^0.10.5
^0.11.7

◯ telepath-unpack
^0.0.3
^0.0.4

@lb- I am not sure upgrading draft.js to 0.11 is a good idea given https://github.com/springload/draftail/issues/213#issuecomment-826158209

Good call @zerolab  - I will update my description. I did try to update but got a bunch of errors but I had not seen that issue.

Description updated to be a bit clearer as to what's changed and what is just a reference of 'not yet updated'.

OK, looks like we will need this PR merged to update Storybook https://github.com/torchbox/storybook-django/pull/15
If there's no traction on that soon (ping @thibaudcolas ), I will redo this PR without the Storybook update.

OK, rebased and removed Storybook updates. Instead I have raised https://github.com/wagtail/wagtail/issues/12553 for that to come later when the downline dependency is updated.


How to enable GPU building with gcc?

https://github.com/NVIDIA/stdexec/blob/3af6f4f2e46a6387890a6e7bf9ef03c278bdcaf6/.github/workflows/ci.gpu.yml#L22-L23
The ci gate of GPU use mpic++ or clang++ to compile with GPU support, and both CMAKE_CXX_COMPILER and CMAKE_CUDA_COMPILER are set to the same compiler.
How can I compile with GPU support with g++ and nvc++?

How to properly set CMAKE_CXX_COMPILER and CMAKE_CUDA_COMPILER?
I want to build the master source instead of stdexec bundled with hpc sdk

Maybe it's better to refine the document on how to enable GPU building.

But in CI test, both clang++ and mpic++ can support GPU?
My question is how to properly set CMAKE variables (set CMAKE_CXX_COMPILER or CMAKE_CUDA_COMPILER to nvc++?) so that I can build master with GPU support.

Thanks. It works for me now.


Gamepad API events

Gamepad API
Introducing gamepad events, I'll let pictures spoke for them self:

I've tested it on Firefox, Chrome and Edge with success using an Xbox One Controller.
Key events are ignored while at least one gamepad is connected.
This is still a draft as I await for your feedback and suggestions.
Could someone with controllers of other vendors try this ?
Roadmap

[ ] Table should show gamepad buttons / axes when a gamepad is connected.
[ ] Verify responsiveness.
[ ] ...

Tests

[x] Xbox One controller
[ ] Xbox Series X controller
[ ] Dual Shock controller
[ ] Joy-con controller
[ ] ...

License
SVG comes from Gamepad W3C Editor's draft and is linked along with W3C license. Thanks @sgraham for the original asset.
@palashmon Eventually, is it ok to include my name with this contribution ? (see footer)

thats cool

man this is really cool! I'm sorry I didn't see this earlier! Will look a bit more into it

Thanks, I figure I may have refactored too much code...
I also noticed that you started a with-nextjs branch with React. If you feel this content is ok but need it on with-nextjs, I'd be more than happy to code it for React.


Powershell colorscheme doesn't get updated by colortool.

It looks like one can run two different kind of powershell, one with the normal way of launching it from the start, or from the win+x menu, and one with the right click open powershell here method. The normal ps colorscheme doesn't get update by the colortool while the cmd console, and the wsl console do get the new colorscheme. Here is a screenshot of this happening:

That's because these are different mechanisms by which you can launch powershell. One involves the Start Menu's Powershell shortcut, and the other uses the registry's values for powershell.exe. Colortool will only be able to set the values for the version that was launched, the other will remain unmodified.
See this blog post  for details.

I see, thank you for the link and the information about settings mechanism. I would never have guessed that this was the reason for the behaviour, because I did not modify the setting for that shortcut, I didn't think it would retain those settings. This was really helpfull.


Tabbing appears to not follow visual order

Description
The tab order appears to now be following visual order when switching using the pane:show-next-item or pane:show-previous-item commands (regardless of the shortcut used to initiate this command)
Steps to Reproduce

Open a number of tabs
Close a tab (not the last one)
Open a new tab

Expected behavior: The tabs should switch as order of display.
Actual behavior: The tabs switching to the new tab before the tab which was the last in the list before the new tab was opened.
Versions
Atom: 1.7.3
APM: 1.9.2
Running Mac OS X 10.11.4

In Atom 1.7, ctrl-tab will switch between the most recently used (MRU) tabs in an Atom window instead of switching to the tab to the right of the current tab.
http://blog.atom.io/2016/04/12/atom-1-7-and-1-8-beta.html
Generally speaking, you may read a recent post about managing issue.

There was a bug with tab index being incorrect if you used the Add new tabs at end configuration option.
This was fixed in 1.7.4, can you still reproduce this in 1.7.4?

This actually appears to be working again in 1.8.0 and I can no longer reproduce the above.

Then it is likely that this was fixed by https://github.com/atom/tabs/pull/302.

Thanks @neilwhitespace for following up. Let us know if you can reproduce this again in the future!


Can't compile Raffle.sol because of value type incompatibility

Compiler error, using soldiity version 0.8.7
TypeError: Invalid type for argument in function call. Invalid implicit conversion from literal_string "" to bytes calldata requested.
  --> contracts/Lottery.sol:85:45:
   |
85 |         (bool upKeepNeeded, ) = checkUpkeep("");
   |                                             ^^

Error HH600: Compilation failed

Here are the two functions:
function checkUpkeep(
        bytes calldata /* checkData */
    )
        public
        view
        override
        returns (
            bool upkeepNeeded,
            bytes memory /* performData */
        )
    {
        bool isOpen = s_lotteryState == LotteryState.OPEN;
        bool timePassed = (block.timestamp - s_lastTimestamp) > s_interval;
        bool hasPlayers = s_players.length > 0;
        upkeepNeeded = (isOpen && timePassed && hasPlayers);
    }

    function performUpkeep(
        bytes calldata /* performData */
    ) external override {
        (bool upKeepNeeded, ) = checkUpkeep("");
        if (!upKeepNeeded) {
            revert Lottery__UpKeppNotNeeded(
                address(this).balance,
                s_players.length,
                uint256(s_lotteryState)
            );
        }
        s_lotteryState = LotteryState.CALCULATING_WINNER;
        uint256 requestId = i_vrfCoordinator.requestRandomWords(
            i_gasLane,
            i_subscriptionId,
            REQUEST_CONFIRMATIONS,
            i_callbackGasLimit,
            NUM_WORDS
        );
        emit RequestedRaffleWinner(requestId);
    }

Answer latter in the video at 14:52:55


test: create autotest for Accounts tc1854

What ❔

Added autotest for Accounts tc1854 [Account API] /api?module=account&action=txlistinternal&txhash= response returns elements

Why ❔

Checklist

[ ] PR title corresponds to the body of PR (we generate changelog entries from PRs).
[ ] Tests for the changes have been added / updated.
[ ] Documentation comments have been added / updated.

@abilevych requested changes added - could you please review again?
Thanks in advance

@abilevych PR is fixed and ready for review

@abilevych conflict resolved - ready for review


[frontend][dex] The clientSecret to connect from kuberpult to dex is exposed in the logs

The secret and id to connect from Kuberpult to DEX are passed to the frontend service as environment variables and there also to the config.
The whole config is logged inside frontend-service/pkg/cmd/server.go.
Kuberpult version - since dex was added, but also on latest (2.27.0).
How to reconstruct it:

Clone the repo
Edit docker-compose-earthly.yml
Add dex environments to the frontend-services env variables:

      - KUBERPULT_DEX_CLIENT_ID=1234
      - KUBERPULT_DEX_CLIENT_SECRET=reallysecretdexconnectionsecret

Run make kuberpult-earthly
Check logs:

kuberpult-frontend-service  | {"level":"warn","ts":1714054532.8153377,"caller":"cmd/server.go:144","msg":"config: \n{cd-service:8443 false  http://kuberpult-cd-service:80    false https://cd.dev.freiheit.systems/ tools  false https://login.microsoftonline.com/    false 1234 reallysecretdexconnectionsecret    https://github.com/freiheit-com/kuberpult/commit/{commit} https://github.com/freiheit-com/fdc-standard-setup-dev-env-manifest/tree/{branch}/{dir} main localhost:* user-local-dev-docker-compose<EMAIL_ADDRESS>2m0s 10m0s false false}"}

In #1550 I removed the logging of the secret.
Revolution ID: DSN-R3VBYG

Closed by #1550, #1572


unable to build

I got an error in the final step of building yosys-slang.
My system:  Ubuntu 22.04.4 LTS
Yosys version: Yosys 0.44+9 (git sha1 4b9f45273, g++ 11.4.0-1ubuntu1~22.04 -fPIC -fPIC -O3)
Error while executing yosys-config:
yosys-config --build build/slang.so slang_frontend.cc initial_eval.cc builder.cc -Ibuild/slang_install/include -std=c++20 -DSLANG_BOOST_SINGLE_HEADER -Lbuild/slang_install/lib -lsvlang -lfmt
/usr/bin/ld: build/slang_install/lib/libfmt.a(format.cc.o): warning: relocation against _ZTIN3fmt3v1012format_facetISt6localeEE' in read-only section .text'
/usr/bin/ld: build/slang_install/lib/libfmt.a(format.cc.o): relocation R_X86_64_PC32 against symbol `stderr@@GLIBC_2.2.5' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: bad value
collect2: error: ld returned 1 exit status

Thanks, let me take a look. Is this with yosys from Ubuntu’s distribution?

No, I have built yosys from source (g++ 11.4.0)

So to reproduce I will install as much as I need from packages, then compile Yosys in default configuration, install it, then attempt to compile yosys-slang, is that right?

Yes, I did it this way

Solved. The README reports the following switches for cmake, that result in the ld error:
cmake -S third_party/slang -B build/slang -DCMAKE_INSTALL_PREFIX=build/slang_install -DSLANG_USE_MIMALLOC=OFF
Using instead all the swicthes found in the script build.sh, yosys-slang is correctly build:
cmake -S third_party/slang -B build/slang -DCMAKE_INSTALL_PREFIX=build/slang_install -DSLANG_USE_MIMALLOC=OFF \
        -DSLANG_INCLUDE_TESTS=OFF -DCMAKE_BUILD_TYPE=Release -DSLANG_INCLUDE_TOOLS=OFF \
        -DCMAKE_CXX_FLAGS="-fPIC" -DBoost_NO_BOOST_CMAKE=ON

PLease, update the README

Thanks! I worked on reproducing this but got stuck by slang throwing a compiler error when built in a Ubuntu container.

Please, update the README

I am in the process of moving to a Makefile which will invoke the cmake step for you so those README instructions will be retired soon.


Update airframe-log to 23.10.0

About this PR
📦 Updates org.wvlet.airframe:airframe-log from 23.3.4 to 23.10.0
📜 GitHub Release Notes - Version Diff
Usage
✅ Please merge!
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

⚙ Adjust future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "org.wvlet.airframe", artifactId = "airframe-log" } ]

Or, add this to slow down future updates of this dependency:
dependencyOverrides = [{
  pullRequests = { frequency = "30 days" },
  dependency = { groupId = "org.wvlet.airframe", artifactId = "airframe-log" }
}]

labels: library-update, early-semver-minor, semver-spec-minor, commit-count:1

Superseded by #760.


[BUG] Stucked at staring up the management container. Infinite Loop

Description
Hello! Thanks for a great product!
I am trying to start self-hosted server. Did everything in Quick and Advanced Manual from the official site
The docker-compose of management container after starting goes to infinite reboot
Steps To Reproduce
Installing on ubuntu 2204 docker.io and docker-compose from apt
Walkthrough the manual
Try to start -> Fail with management container
Compose Version
docker-compose version 1.29.2, build unknown
docker-py version: 5.0.3
CPython version: 3.10.12
OpenSSL version: OpenSSL 3.0.2 15 Mar 2022

Docker Environment
Client:
 Version:    24.0.5
 Context:    default
 Debug Mode: false

Server:
 Containers: 4
  Running: 3
  Paused: 0
  Stopped: 1
 Images: 5
 Server Version: 24.0.5
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: runc io.containerd.runc.v2
 Default Runtime: runc
 Init Binary: docker-init
 containerd version:
 runc version:
 init version:
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 5.15.0-101-generic
 Operating System: Ubuntu 22.04.4 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 1
 Total Memory: 1.912GiB
 Name: PQVPN-NetBird
 ID: d0590919-37e4-4af2-a215-06424bf2afa2
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  <IP_ADDRESS>/8
 Live Restore Enabled: false

Logs for management container

management_1  | Error: failed to initialize database: illegal base64 data at input byte 0

management_1  | 2024-03-27T16:00:57Z INFO management/cmd/management.go:449: loading OIDC configuration from the provided IDP configuration endpoint https://keycloak/.well-known/openid-configuration

management_1  | 2024-03-27T16:00:57Z INFO management/cmd/management.go:454: loaded OIDC configuration from the provided IDP configuration endpoint: https://keycloak/.well-known/openid-configuration

management_1  | 2024-03-27T16:00:57Z INFO management/cmd/management.go:456: overriding HttpConfig.AuthIssuer with a new value https://keycloak, previously configured value: https://keycloak

management_1  | 2024-03-27T16:00:57Z INFO management/cmd/management.go:460: overriding HttpConfig.AuthKeysLocation (JWT certs) with a new value https://keycloak/protocol/openid-connect/certs, previously configured value: https://keycloak/protocol/openid-connect/certs

management_1  | 2024-03-27T16:00:57Z INFO management/cmd/management.go:465: overriding DeviceAuthorizationFlow.TokenEndpoint with a new value: https://keycloak/protocol/openid-connect/token, previously configured value: https://keycloak/protocol/openid-connect/token

management_1  | 2024-03-27T16:00:57Z INFO management/cmd/management.go:468: overriding DeviceAuthorizationFlow.DeviceAuthEndpoint with a new value: https://keycloak/protocol/openid-connect/auth/device, previously configured value: https://keycloak/protocol/openid-connect/auth/device

management_1  | 2024-03-27T16:00:57Z INFO management/cmd/management.go:476: overriding DeviceAuthorizationFlow.ProviderConfig.Domain with a new value: auth.site.no, previously configured value:

management_1  | 2024-03-27T16:00:57Z INFO management/cmd/management.go:486: overriding PKCEAuthorizationFlow.TokenEndpoint with a new value: keycloak/protocol/openid-connect/token, previously configured value: keycloak/protocol/openid-connect/token

management_1  | 2024-03-27T16:00:57Z INFO management/cmd/management.go:489: overriding PKCEAuthorizationFlow.AuthorizationEndpoint with a new value: keycloak/protocol/openid-connect/auth, previously configured value: keycloak/protocol/openid-connect/auth

management_1  | 2024-03-27T16:00:57Z INFO management/server/telemetry/app_metrics.go:177: enabled application metrics and exposing on http://<IP_ADDRESS>:8081

management_1  | 2024-03-27T16:00:57Z INFO management/server/store.go:92: using SQLite store engine

management_1  | Error: failed to initialize database: illegal base64 data at input byte 0

management_1  | 2024-03-27T16:01:04Z INFO management/cmd/management.go:449: loading OIDC configuration from the provided IDP configuration endpoint keycloak/.well-known/openid-configuration

management_1  | 2024-03-27T16:01:04Z INFO management/cmd/management.go:454: loaded OIDC configuration from the provided IDP configuration endpoint: keycloak/.well-known/openid-configuration

management_1  | 2024-03-27T16:01:04Z INFO management/cmd/management.go:456: overriding HttpConfig.AuthIssuer with a new value keycloak, previously configured value: keycloak

management_1  | 2024-03-27T16:01:04Z INFO management/cmd/management.go:460: overriding HttpConfig.AuthKeysLocation (JWT certs) with a new value keycloak/protocol/openid-connect/certs, previously configured value: keycloak/protocol/openid-connect/certs

management_1  | 2024-03-27T16:01:04Z INFO management/cmd/management.go:465: overriding DeviceAuthorizationFlow.TokenEndpoint with a new value: keycloak/protocol/openid-connect/token, previously configured value: keycloak/protocol/openid-connect/token

management_1  | 2024-03-27T16:01:04Z INFO management/cmd/management.go:468: overriding DeviceAuthorizationFlow.DeviceAuthEndpoint with a new value: keycloak/protocol/openid-connect/auth/device, previously configured value: keycloak/protocol/openid-connect/auth/device

management_1  | 2024-03-27T16:01:04Z INFO management/cmd/management.go:476: overriding DeviceAuthorizationFlow.ProviderConfig.Domain with a new value: auth.site.no, previously configured value:

management_1  | 2024-03-27T16:01:04Z INFO management/cmd/management.go:486: overriding PKCEAuthorizationFlow.TokenEndpoint with a new value: keycloak/protocol/openid-connect/token, previously configured value: keycloak/protocol/openid-connect/token

management_1  | 2024-03-27T16:01:04Z INFO management/cmd/management.go:489: overriding PKCEAuthorizationFlow.AuthorizationEndpoint with a new value: keycloak/protocol/openid-connect/auth, previously configured value: keycloak/protocol/openid-connect/auth

management_1  | 2024-03-27T16:01:04Z INFO management/server/telemetry/app_metrics.go:177: enabled application metrics and exposing on http://<IP_ADDRESS>:8081

management_1  | 2024-03-27T16:01:04Z INFO management/server/store.go:92: using SQLite store engine

management_1  | Error: failed to initialize database: illegal base64 data at input byte 0

Hi @Luccifer, the script will generate a set of config files. Can you share the content of your management.json? It looks like the DataStoreEncryptionKey is missing or wrongly created. Did you have any errors when running the script?

Hi @Luccifer, the script will generate a set of config files. Can you share the content of your management.json? It looks like the DataStoreEncryptionKey is missing or wrongly created. Did you have any errors when running the script?

Ofc!
Here is it:
{
    "Stuns": [
        {
            "Proto": "udp",
            "URI": "stun:$TURN_DOMAIN:3478",
            "Username": "",
            "Password": null
        }
    ],
    "TURNConfig": {
        "Turns": [
            {
                "Proto": "udp",
                "URI": "turn:$TURN_DOMAIN:3478",
                "Username": "$TURN_USER",
                "Password": "$TURN_PASSWORD"
            }
        ],
        "CredentialsTTL": "12h",
        "Secret": "secret",
        "TimeBasedCredentials": false
    },
    "Signal": {
        "Proto": "$NETBIRD_SIGNAL_PROTOCOL",
        "URI": "$NETBIRD_DOMAIN:$NETBIRD_SIGNAL_PORT",
        "Username": "",
        "Password": null
    },
    "ReverseProxy": {
        "TrustedHTTPProxies": [],
        "TrustedHTTPProxiesCount": 0,
        "TrustedPeers": [
             "<IP_ADDRESS>/0"
        ]
    },
    "Datadir": "",
    "DataStoreEncryptionKey": "$NETBIRD_DATASTORE_ENC_KEY",
    "StoreConfig": {
        "Engine": "$NETBIRD_STORE_CONFIG_ENGINE"
    },
    "HttpConfig": {
        "Address": "<IP_ADDRESS>:$NETBIRD_MGMT_API_PORT",
        "AuthIssuer": "$NETBIRD_AUTH_AUTHORITY",
        "AuthAudience": "$NETBIRD_AUTH_AUDIENCE",
        "AuthKeysLocation": "$NETBIRD_AUTH_JWT_CERTS",
        "AuthUserIDClaim": "$NETBIRD_AUTH_USER_ID_CLAIM",
        "CertFile":"$NETBIRD_MGMT_API_CERT_FILE",
        "CertKey":"$NETBIRD_MGMT_API_CERT_KEY_FILE",
        "IdpSignKeyRefreshEnabled": "$NETBIRD_MGMT_IDP_SIGNKEY_REFRESH",
        "OIDCConfigEndpoint":"$NETBIRD_AUTH_OIDC_CONFIGURATION_ENDPOINT"
    },
    "IdpManagerConfig": {
        "ManagerType": "$NETBIRD_MGMT_IDP",
        "ClientConfig": {
            "Issuer": "$NETBIRD_AUTH_AUTHORITY",
            "TokenEndpoint": "$NETBIRD_AUTH_TOKEN_ENDPOINT",
            "ClientID": "$NETBIRD_IDP_MGMT_CLIENT_ID",
            "ClientSecret": "$NETBIRD_IDP_MGMT_CLIENT_SECRET",
            "GrantType": "client_credentials"
        },
        "ExtraConfig": "$NETBIRD_IDP_MGMT_EXTRA_CONFIG",
        "Auth0ClientCredentials": null,
        "AzureClientCredentials": null,
        "KeycloakClientCredentials": null,
        "ZitadelClientCredentials": null
     },
    "DeviceAuthorizationFlow": {
        "Provider": "$NETBIRD_AUTH_DEVICE_AUTH_PROVIDER",
        "ProviderConfig": {
          "Audience": "$NETBIRD_AUTH_DEVICE_AUTH_AUDIENCE",
          "AuthorizationEndpoint": "",
          "Domain": "$NETBIRD_AUTH0_DOMAIN",
          "ClientID": "$NETBIRD_AUTH_DEVICE_AUTH_CLIENT_ID",
          "ClientSecret": "",
          "TokenEndpoint": "$NETBIRD_AUTH_TOKEN_ENDPOINT",
          "DeviceAuthEndpoint": "$NETBIRD_AUTH_DEVICE_AUTH_ENDPOINT",
          "Scope": "$NETBIRD_AUTH_DEVICE_AUTH_SCOPE",
          "UseIDToken": "$NETBIRD_AUTH_DEVICE_AUTH_USE_ID_TOKEN",
          "RedirectURLs": null
         }
    },
    "PKCEAuthorizationFlow": {
        "ProviderConfig": {
            "Audience": "$NETBIRD_AUTH_PKCE_AUDIENCE",
            "ClientID": "$NETBIRD_AUTH_CLIENT_ID",
            "ClientSecret": "$NETBIRD_AUTH_CLIENT_SECRET",
            "Domain": "",
            "AuthorizationEndpoint": "$NETBIRD_AUTH_PKCE_AUTHORIZATION_ENDPOINT",
            "TokenEndpoint": "$NETBIRD_AUTH_TOKEN_ENDPOINT",
            "Scope": "$NETBIRD_AUTH_SUPPORTED_SCOPES"
        }
    }
}

I also added quotes in this json at 61, 78 and 91-92 LOC

Did enyone managed to make it start from scratch with advanced tutorial, assuming management.json has issues in json-code + some mappings are abscent

Well after 5th clean installation VM Ubuntu 2204 proxmox I made it working with standalone docker compose.
But now I can't access the dashboard it always gives 502, meanvile I am using reverse-proxy with default ports like 80,443,10000,33073, etc... Ports in firewall are forwarding correctly, cause other services are fine with the familiar settings.
At the attempt of connecting by httP - key cloak gives invalid redirect uri, but if I use SSL https:// 502 error appearing...

I switched to PostgreSQL and the error went away.


Fix memory leak (adds close hook to RewriterContainer)

This PR fixes the possible memory leak described in #177 by properly closing the RewriterContainer.

Fixes #177


"as" in WOQL should take an optional third argument specifying casting type.

This would be a huge improvement of having to manually typcast all over the place in csv files.

This is now the case.


Discord -> Matrix: Gif feature should result in image, not link

When using the "Gif" feature in Discord:

it results in some image:

but on the Matrix side it's shared as a link:

I personally prefer the status quo. I feel like converting links to images should be left up to url previews (and I personally like to disable url previews).
Images are also more costly to send than mere links.

IMO the bridge's job should be to translate messages with a look and behavior as close to the original as possible, to avoid creating disconnects between the two groups of users. Transforming them into something different that's more convenient for the user is the job of the client.

Transforming them into something different that's more convenient for the user is the job of the client.

By that logic, shouldn't it be the client's job to enable url previews by default or not? How can the client transform it to something more convenient? Is the client expected to reverse image search every image it receives and ask the users if they would want to convert it to a matching url?
Your proposal does indeed make the "look" from Discord -> Matrix close to the original as possible, but as far as "behavior" goes, it would be quite different. It also causes inconsistencies.
Discord -> Matrix would create a seemingly unnecessary image for Matrix, which fails to replicate the behavior.
Sending the url from Matrix -> Discord would cause the same problem you outlined but in reverse (link shown on Matrix's side, image on Discord), and there would be no sane way to fix that without relying on url previews anyways.
The only workaround I can imagine to that is to expect Matrix users to never send links and send images instead, but that would also end up sending an image to Discord when a link would have sufficed for them. That would just introduce another disparity in behavior.
I don't think your proposal makes sense in terms of consistent behavior between both sides of the bridge.

I personally disagree with this change as well.
Discord also does have an option to disable URL previews. Which would make this even more inconsistent.
With URL preview:

Without:

So by forcing to always send Gifs as images this would just cause even more inconsistencies and force Matrix users to see an image, where Discord users don't necessarily see one.

just have it be an option? in the settings?

nobody sees the gifs on the matrix end

That really depends on the client you're using.
They're fine with Element Desktop (I can get the full playing GIF when i click on the preview image)

and look good on Element mobile and even animate there

I am on element mobile, latest version too. my friends don't see gif previews either and they're all on different homeservers, so I don't know how we all aren't seeing gif previews lol


Issue 180: Prefetch read

Change log description
Add prefetch for async segment reader.
Purpose of the change
Fix #180
What the code does
Add a AsyncSegmentReaderWrapper that maintains an internal buffer and a read offset. Every time read request comes in it will fetches data from the buffer and issues a background read request to server if needed  to fill the buffer. Basically this wrapper will enable caller to read from memory instead of waiting for server to response.

Codecov Report

Merging #189 (6fda5cf) into master (05ce59a) will increase coverage by 4.88%.
The diff coverage is 94.44%.

@@            Coverage Diff             @@
##           master     #189      +/-   ##
==========================================
+ Coverage   74.39%   79.27%   +4.88%
==========================================
  Files          51       51
  Lines        9416     9906     +490
==========================================
+ Hits         7005     7853     +848
+ Misses       2411     2053     -358

Impacted Files
Coverage Δ

src/error.rs
8.51% <0.00%> (+4.06%)
:arrow_up:

src/segment_reader.rs
67.37% <93.85%> (+11.61%)
:arrow_up:

config/src/lib.rs
100.00% <100.00%> (ø)

src/byte_stream.rs
91.12% <100.00%> (+1.76%)
:arrow_up:

src/client_factory.rs
98.75% <100.00%> (ø)

src/raw_client.rs
94.87% <100.00%> (+0.84%)
:arrow_up:

auth/src/lib.rs
86.66% <0.00%> (-1.51%)
:arrow_down:

src/reader_group/reader_group_state.rs
88.59% <0.00%> (+1.30%)
:arrow_up:

... and 5 more


Thoughts on JSON serialisation and pydantic?

Looking at the example on how to create a StructureData instance:
from aiida_atomistic.data.structure import StructureData

properties_dict = {
    "cell":{"value":[[3.5, 0.0, 0.0], [0.0, 3.5, 0.0], [0.0, 0.0, 3.5]]},
    "pbc":{"value":[True,True,True]},
    "positions":{"value":[[0.0, 0.0, 0.0],[1.5, 1.5, 1.5]]},
    "symbols":{"value":["Li","Li"]},
    }

structure = StructureData(properties = properties_dict)

And the fact the entire structure data is now captured by the properties, after https://github.com/aiidateam/aiida-atomistic/issues/10 the constructor would effectively only have the properties input. This immediately made me think of just having a JSON serializing method (to_json, from_json, or similar). But then I thought about @sphuber's AEP (https://github.com/aiidateam/AEP/pull/40) - which I still have to read 🙈 - and if we shouldn't be thinking about how to integrate the new StructureData with the concepts there.
I'll have to read up some more before having a solid opinion on what to do, but I already wanted to raise the issue.

I think what you are proposing would be automatically supported with https://github.com/aiidateam/aiida-core/pull/6255
It would allow defining a schema for the StructureData class through a pydantic model:
class StructureData

    class Model(Data.Model):
        pbc: list[bool] = MetadataField(description='Periodicity along each cell axis')
        cell: list[list[float]] = MetadataField(description='The cell parameters')
        positions: list[list[float | int]] = MetadataField(description='The atomic positions')
        symbols: list[str]= MetadataField(description='The atomic symbol labels')

You would then be able to create an instance from JSON serialized data as follows:
data = {
  'cell': [[3.5, 0.0, 0.0], [0.0, 3.5, 0.0], [0.0, 0.0, 3.5]],
  'pbc': [True, True, True],
  'positions': [[0.0, 0.0, 0.0], [1.5, 1.5, 1.5]],
  'symbols': ['Li', 'Li'],
}
structure = StructureData.from_serialized(data)
serialized = structure.serialize()

Note that your class wouldn't have to implement from_serialized and serialize as these are implemented on the aiida.orm.entity:Entity base class.
I haven't tried this, but may be worth doing, also as a way of testing the pydantic PR in aiida-core.


InvalidStateError when Attempting to Update Message Read Status in Dexie 3.2.4

Issue Description
I'm encountering an InvalidStateError with Dexie version 3.2.4 when trying to update the read status of a message in my application's database. The error message suggests an attempt to execute an operation on an objectStore of an IDBTransaction that has already finished. This occurs in a seemingly straightforward operation where I first check if a message exists and then attempt to update its readed status.
Steps to Reproduce
Initialize the database with tables for messages, contacts, and entities. Ensure messages have a unique index on msgId.
Attempt to update a message's readed status by first retrieving the message by its ID and then calling the update method on the tgMessages table.
The error occurs upon executing the update operation, despite the message existing in the database.
Expected Behavior
The message's readed status should be updated without any errors, and the operation should complete successfully within the context of an active transaction.
Actual Behavior
The operation fails with the following error:
InvalidStateError: Failed to execute 'objectStore' on 'IDBTransaction': The transaction has finished.
export class ChatMessageDatabase extends Dexie {
    tgMessages: Dexie.Table<TgMessage, number> = null as any;
    constructor(userId: number) {
        super(`ChatMessageDatabase_userId_${userId}`);

        this.version(1).stores({
            tgMessages: '++id,&msgId,contactId,senderId,receiverId',
        });
        this.tgMessages = this.table('tgMessages');

async updateMessageReaded(message: TgMessage) {
    try {
        await this.transaction('rw', this.tgMessages, async () => {
            const msg = await this.tgMessages.get(message.id);
            if (msg) {
                await this.tgMessages.update(message.id, { readed: true });
                console.log('Successfully updated message read status', message);
            } else {
                console.log('Message not found');
            }
        });
    } catch (err) {
        console.error('Failed to update message read status', err);
    }
}

Additional Context
Dexie version: 3.2.4
Browser: [Include browser name and version]
The issue occurs consistently when attempting to update any message's readed status.
The database initialization and update operation are part of an application managing chat messages.
I have tried several approaches to ensure the operation executes within an active transaction context, including wrapping the operation in a this.transaction call, but the error persists.
I use the following code which works fine:
async updateMessageReadedDirectly(message: TgMessage) {
        return new Promise((resolve, reject) => {
            const request = indexedDB.open(this.name);
            request.onerror = function (event) {
                reject('Database open error');
            };
            request.onsuccess = function (event) {
                //@ts-ignore
                const db = event.target.result;
                const transaction = db.transaction(['tgMessages'], 'readwrite');
                const store = transaction.objectStore('tgMessages');
                const getRequest = store.get(message.id);

                getRequest.onsuccess = function () {
                    const data = getRequest.result;
                    data.readed = message.readed;
                    const updateRequest = store.put(data);
                    updateRequest.onsuccess = function () {
                        resolve('Update success');
                    };
                    updateRequest.onerror = function (event: any) {
                        reject('Update failed');
                    };
                };

                getRequest.onerror = function (event: any) {
                    reject('Get failed');
                };
            };
        });
    }

Any insights or suggestions on how to resolve this error would be greatly appreciated.

Is the error happening on a real browser or in a unit test? What browser engine and version are you using? Reason I ask is because unit tests usually use fakeIndexedDB or indexedDBShim which cannot survive transactions between awaits, just like browsers had problem with for a while ago.

If the case is an old browser, fakeIndexedDB or indexedDBShim, one solution is to transpile the code to ES2016 (before await was introduced).


create install.sh for Xubuntu 20.04

Confirmed working with Enhanced session
Don't forget to enable the hv socket
 set-VM -VMName "yourvnnamehere" -EnhancedSessionTransportType HvSocket

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: sub6as sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.

Just to add this script was inspired by pull #95


Issue with running example 1

I install seisflows with this.
Installing the devel branch gives errors, the master branch is fine. So the devel branch may have some bugs
I met this priblem below:
(seisflows) [zhangchang@gpu01 example1]$ seisflows examples run 1
                                @@@@@@@@@@
                           .@@@@.    .%&<EMAIL_ADDRESS>                        @@@@   @@@@   &@@@@@@ ,%@
                     @@@@   @@@,  /@@              @
                    @@@   @@@@   @@@              @
                  @@@@   @@@@   @@@                @  @
                  @@@   @@@@   ,@@@                @ @
                 @@@@   @@@@    @@@@              @@ @ @
                 @@@@   @@@@@    @@@@@          @@@ @@ @
                 @@@@    @@@@@     @@@@@@@@@@@@@@  @@  @
                  @@@@    @@@@@@        @@@&     @@@  @
                  @@@@@     @@@@@@@@         %@@@@#  @@
                    @@@@#      @@@@@@@@@@@@@@@@@   @@
                     &@@@@@          @@@@(       @@&
                        @@@@@@@             /@@@@
                            @@@@@@@@@@@@@@@@@
                                @@@@@@@@@@

================================================================================
SEISFLOWS EXAMPLE 1
///////////////////
This is a [SPECFEM2D] [WORKSTATION] example, which will run an inversion to
assess misfit between two homogeneous halfspace models with slightly different
velocities. [1 events, 1 station, 1 iterations]. The tasks involved include:

(optional) Download, configure, compile SPECFEM2D
[Setup] a SPECFEM2D working directory
[Setup] starting model from 'Tape2007' example
[Setup] target model w/ perturbed starting model
[Setup] a SeisFlows working directory
[Run] the inversion workflow
================================================================================

================================================================================
EXAMPLE SETUP
Configuring SPECFEM2D with command: ./configure
configure: WARNING: required lex library not found; giving up on flex
fatal: No names found, cannot describe anything.
Making SPECFEM2D binaries with command: make all
================================================================================
GENERATING INITIAL MODEL

Setting the SPECFEM2D Par_file for SeisFlows compatiblility
NPROC: 1 -> 1
setup_with_binary_database: 0 -> 1
SAVE_MODEL: default -> binary
save_ASCII_kernels: .true. -> .false.
Running SPECFEM2D with command: ./bin/xmeshfem2D > OUTPUT_FILES/mesher.log.txt
Running SPECFEM2D with command: ./bin/xspecfem2D > OUTPUT_FILES/solver.log.txt
Cleaning up after xspecfem2d, setting up for new run

================================================================================
GENERATING TRUE/TARGET MODEL

Updating initial homogeneous velocity model values
VELOCITY_MODEL:

1 1 2600.d0 5800.d0 3500.0d0 0 0 10.d0 10.d0 0 0 0 0 0 0
->
1 1 2600.d0 5900.d0 3550.0d0 0 0 10.d0 10.d0 0 0 0 0 0 0
Running SPECFEM2D with command: ./bin/xmeshfem2D > OUTPUT_FILES/mesher.log.txt
Running SPECFEM2D with command: ./bin/xspecfem2D > OUTPUT_FILES/solver.log.txt

Cleaning up after xspecfem2d, setting up for new run
created parameter file: parameters.yaml
workflow: forward -> inversion
configuring SeisFlows parameter file
ntask: 1 -> 1
materials: acoustic -> elastic
update_density: False -> False
generate_data: False -> True
syn_data_format: ascii -> ascii
obs_data_format: ASCII -> ascii
unit_output: VEL -> disp
nproc: 1 -> 1
start: 1 -> 1
end: 1 -> 1
step_count_max: 10 -> 5
components: null -> Y
attenuation: False -> False
plot_waveforms: False -> True
misfit: waveform -> traveltime
adjoint: waveform -> traveltime
path_specfem_bin: null -> /home/zhangchang/example1/specfem2d_workdir/bin
path_specfem_data: null -> /home/zhangchang/example1/specfem2d_workdir/DATA
path_model_init: null ->
/home/zhangchang/example1/specfem2d_workdir/OUTPUT_FILES_INIT
path_model_true: null ->
/home/zhangchang/example1/specfem2d_workdir/OUTPUT_FILES_TRUE
Finalizing SPECFEM2D Par_file for SeisFlows example
MODEL: default -> gll
use_existing_STATIONS: .false. -> .true.
NPROC: 1 -> 1
Using 1 stations in this SeisFlows example

================================================================================
COMPLETE EXAMPLE SETUP
================================================================================
RUNNING SEISFLOWS EXAMPLE WORKFLOW
2024-04-29 07:01:56 [INFO] | running setup for all modules
2024-04-29 07:01:56 [DEBU] | workflow.Inversion
2024-04-29 07:02:07 [DEBU] | system.Workstation
2024-04-29 07:02:09 [DEBU] | solver.Specfem2D
2024-04-29 07:02:11 [INFO] | initializing 1 solver directories
2024-04-29 07:02:12 [DEBU] | source 0: 001
2024-04-29 07:02:17 [DEBU] | linking source '001' as 'mainsolver'
2024-04-29 07:02:18 [DEBU] | preprocess.Default
2024-04-29 07:02:19 [DEBU] | optimize.Gradient
2024-04-29 07:02:21 [INFO] | no optimization checkpoint file, assume 1st iteration
2024-04-29 07:02:25 [INFO] |
                          RUNNING ITERATION 01

================================================================================
2024-04-29 07:02:25 [INFO] |
////////////////////////////////////////////////////////////////////////////////
GENERATE SYNTHETIC DATA
////////////////////////////////////////////////////////////////////////////////
2024-04-29 07:02:25 [INFO] | checking true/target model parameters:
2024-04-29 07:02:25 [INFO] | vp: min=5900.000; mean=5900.000; max=5900.000
2024-04-29 07:02:25 [INFO] | vs: min=3550.000; mean=3550.000; max=3550.000
2024-04-29 07:02:25 [INFO] | evaluating objective function for source 001
2024-04-29 07:02:25 [DEBU] | running forward simulation with 'Specfem2D'
2024-04-29 07:02:25 [DEBU] | running executable with cmd: 'bin/xmeshfem2D'
2024-04-29 07:02:25 [DEBU] | running executable with cmd: 'bin/xspecfem2D'
2024-04-29 07:02:34 [DEBU] | checkpointing workflow to seisflows state file
Traceback (most recent call last):
File "/home/zhangchang/miniconda3/envs/seisflows/bin/seisflows", line 8, in
sys.exit(main())
^^^^^^
File "/home/zhangchang/seisflows/seisflows/seisflows.py", line 1431, in main
sf()
File "/home/zhangchang/seisflows/seisflows/seisflows.py", line 451, in call
getattr(self, self._args.command)(**vars(self._args))
File "/home/zhangchang/seisflows/seisflows/seisflows.py", line 804, in submit
system.submit(workdir=self._args.workdir,
File "/home/zhangchang/seisflows/seisflows/system/workstation.py", line 211, in submit
workflow.run()
File "/home/zhangchang/seisflows/seisflows/workflow/inversion.py", line 197, in run
super().run()  # Runs task list
^^^^^^^^^^^^^
File "/home/zhangchang/seisflows/seisflows/workflow/forward.py", line 283, in run
func()
File "/home/zhangchang/seisflows/seisflows/workflow/inversion.py", line 239, in generate_synthetic_data
super().generate_synthetic_data(**kwargs)
File "/home/zhangchang/seisflows/seisflows/workflow/forward.py", line 315, in generate_synthetic_data
self.system.run([self._generate_synthetic_data_single], **kwargs)
File "/home/zhangchang/seisflows/seisflows/system/workstation.py", line 250, in run
func(**kwargs)
File "/home/zhangchang/seisflows/seisflows/workflow/forward.py", line 346, in _generate_synthetic_data_single
self.run_forward_simulations(
File "/home/zhangchang/seisflows/seisflows/workflow/inversion.py", line 419, in run_forward_simulations
super().run_forward_simulations(path_model, save_traces=save_traces,
File "/home/zhangchang/seisflows/seisflows/workflow/forward.py", line 582, in run_forward_simulations
self.solver.forward_simulation(save_traces=save_traces,
File "/home/zhangchang/seisflows/seisflows/solver/specfem.py", line 634, in forward_simulation
if self.prune_scratch:
^^^^^^^^^^^^^^^^^^
AttributeError: 'Specfem2D' object has no attribute 'prune_scratch'
================================================================================
EXAMPLE FAILED
Command 'seisflows submit' returned non-zero exit status 1.

Hi @trinitite271, sorry there was a small bug in the devel branch that I just fixed in #212, can you try again and see if that works?


Custom single-particle dispersion for HubbardMom1D

The single particle dispersion relation used in HubbardMom1D and BoseHubbardMom1D2C can be changed with the keyword argument dispersion. See the docstring of HubbardMom1D.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

7 of 7 (100.0%)  changed or added relevant lines in 2 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.007%) to 86.63%

Totals

Change from base Build<PHONE_NUMBER>:
0.007%

Covered Lines:
3110

Relevant Lines:
3590

💛  - Coveralls


Feature/allow unconfigured run

This allows the Coopr kerberos-client service to be run without any configuration to simply install the kerberos client libraries.  Previously, it would attempt to kadmin and fail.

[x] splits the keytab generation logic into its own recipe and conditionally includes it only if any keytabs are defined
[x] rubocop and testing updates

Finished in 6.09 seconds (files took 3.16 seconds to load)
16 examples, 0 failures

One comment, otherwise :+1: LGTM


feat(@vtmn/react): update icon variants

Pull Request checklist
🚨 Please review the guidelines for contributing to this repository.

[x] Make sure you are requesting to pull a topic/feature/bugfix branch. Don't request your main!
[x] Check the commit's or even all commits' message styles matches our requested structure.
[x] Check your code additions will fail neither code linting checks.
[x] Check PR name, it must follow the https://www.conventionalcommits.org pattern
[x] If it's a new component in CSS, ask for design review

Description
This is an update to react icon component variants

It allows icons to have an inherited color.
Why : This makes sense as icons doesn't have color restrictions, and when associated with text, an icon should be able to hold the same color.
How : This PR propose a neutral variant keyword, when stated outputted inline color value is inherit

It changes the variant keyword danger as negative
and output color value as var(--vtmn-semantic-color_content-negative),
rather than var(--vtmn-semantic-color_content-danger) (that doesn't exist - stating a danger variant right now output a black color)

Does this introduce a breaking change?

Yes

danger wording variant, if ever used...
<VtmnIcon variant="danger" value="home-fill" />

...should be update to negative
<VtmnIcon variant="negative" value="home-fill" />

❤ Thank you!

Here is a new proposition without "neutral" variant.
icon color inherit from its parent whenever no variant is stated

For me it's ok, thanks @crotru :tada:


World metadatas no uploaded after fail

Describe the bug
World metadatas are only uploaded once, at world creation.
If an error occurs on first deployement before uploading metadatas, no metadatas will be uploaded for dojo::world::world
Moreover there is no easy way to update world metadatas
To Reproduce
with dojo starter, start a migration and cancel after base & world contract has been deployed
relaunch migration
Additional context
0.7.2

related: #2079
https://github.com/dojoengine/dojo/issues/2079#issuecomment-2185712129

Closed in #2203.

Can i try this ?

I am applying to this issue via OnlyDust platform.
My background and how it can be leveraged
i want to start me open source journey and i thinks it'd be a great place for me.... can i please get this issue?

I am applying to this issue via OnlyDust platform.
My background and how it can be leveraged
I'm an IT student persuing final year

I am applying to this issue via OnlyDust platform.
My background and how it can be leveraged
i'm a blockchain developer from 2022 and here is my github profile - https://github.com/jaikumar9
How I plan on tackling this issue
Allow metadata updates after deployment with an admin function.

I am applying to this issue via OnlyDust platform.
My background and how it can be leveraged
Experience programming in the languages used in the project, so I can complete the issue
How I plan on tackling this issue
Having experience in programming, I can easily detect errors in the code and program the issue in compliance with programming standards.

can i try this issue?

I am applying to this issue via OnlyDust platform.
My background and how it can be leveraged
love to work on this

I am applying to this issue via OnlyDust platform.
My background and how it can be leveraged
I am a Python programmer and also a Cairo programmer also I am pleased to say my skills can be of great value in this project thank you
How I plan on tackling this issue
I will approach this project by using the knowledge I have on Cairo and as a lover of games I will be of great assistance

I am applying to this issue via OnlyDust platform.
My background and how it can be leveraged
In the system, world metadata is only uploaded once during the world creation process. If an error occurs during the first deployment—specifically before the metadata is successfully uploaded—then the metadata will not be uploaded for dojo::world::world. Moreover, there is currently no straightforward method to update the world metadata after the fact.
How I plan on tackling this issue
Steps to Reproduce
Use the Dojo starter kit to initiate a migration.
Begin the deployment of the base and world contracts.
Cancel the migration before it is fully complete (i.e., after the base and world contracts have been deployed, but before the metadata has been uploaded).
Attempt to relaunch the migration.
Expected Behavior
Metadata should either be uploaded successfully on relaunch or a mechanism should exist to update or retry metadata uploads.
Observed Behavior
No world metadata is uploaded if the deployment is canceled before the metadata step.
There is no clear way to update or re-upload metadata once the error has occurred.
Suggested Fix
Implement a way to retry metadata uploads after a failed deployment or introduce a manual method to update world metadata after deployment.

I am applying to this issue via OnlyDust platform.
My background and how it can be leveraged
I come from a programming and web development background having worked on various client-based projects and freelanced on People Per Hour and contributed to open-source initiatives. I have also worked with responsive and stateful websites and applications which included careful migration and deployment processes. My work in mortgages has reinforced my problem-solving skills which are aimed towards reliability and long term solutions of the system. This is the way why I think I can use both of these relevant aspects for analyzing the bug in Dojo’s world metadata handling and making sure that such things don’t happen again.
How I plan on tackling this issue
First I would investigate why metadata is only uploaded once and not updated on subsequent deployments if errors occur. To do this:
Reproduce the Bug: I would follow the steps (initiating and canceling the migration after base and world contracts have been deployed) and look at the logs and contract states to see where metadata handling fails.
Fix Metadata Upload Logic: I would refactor the metadata upload process so it’s not tied to the world creation phase. Instead I would make sure metadata can be uploaded or updated at any point post creation. This could mean:
Implementing a retry mechanism for metadata upload failures during deployment.
Adding the ability to update world metadata after initial deployment, perhaps by introducing a separate function or migration step for metadata updates.
Test Thoroughly: After the fix I would write comprehensive tests to simulate various deployment failures (e.g. server crashes, network issues) to make sure the metadata upload mechanism is robust.
Optimize and Document: Finally I would document this new process clearly and make sure any future contributors or users of the platform can update metadata without having to redeploy the entire world contract.

I am applying to this issue via OnlyDust platform.
My background and how it can be leveraged
I have 1 year experience in Cairo language, Rust, Dojo Engine. Additionally, I have problem solving skills, self-learning skills, and some soft skills.

I am applying to this issue via OnlyDust platform.
My background and how it can be leveraged
Given your scenario and the nature of the bug, my background in Python, data handling, and logical problem-solving could be helpful in diagnosing and resolving this issue. Specifically:
Programming Experience: I could assist in crafting solutions for ensuring metadata is always uploaded, either by retrying failed uploads or allowing manual updates.
Debugging & Error Handling: Experience with debugging and error handling could be used to identify and resolve the error during deployment, ensuring that metadata is not skipped.
System Design: Understanding of system architectures can help propose ways to allow post-deployment metadata updates or a retry mechanism in case of failure.
How I plan on tackling this issue
Identify Where the Issue Occurs:
The first thing to do is pinpoint exactly where in the deployment process the failure occurs. This requires reviewing:
The logic handling the uploading of metadata.
The point in the process where errors or cancellations can lead to incomplete deployments.
Questions to consider:
When exactly is the metadata uploaded in the flow?
Is there any checkpoint or transaction rollback when deployment fails?
What error message is thrown when deployment is canceled?
2. Implement Transactional Metadata Upload:
A robust deployment process should ensure metadata is uploaded only when the world creation is successful. To achieve this, I would:
Ensure the metadata upload is transactional. This means if the deployment fails or is canceled, any action related to metadata upload would be rolled back. This could be done by:
Moving metadata upload to a later stage of deployment (after the world contract is confirmed as deployed).
Guaranteeing consistency: Use an event-based approach or a flag in the contract to signal if the metadata has been successfully uploaded.
3. Error Handling and Retry Mechanism:
Add error handling and a retry mechanism for metadata uploads. If the initial deployment is interrupted:
Catch deployment errors: When the deployment process is canceled or fails, catch these errors and:
Log the failure state (including whether the metadata was uploaded or not).
Prompt the user to retry the metadata upload upon re-running the migration.
Automatic Retry: Implement an automatic retry process that, upon detecting incomplete metadata, triggers a new attempt to upload it without restarting the entire world creation process.
Example:
try:
deploy_world()
upload_metadata()
except DeploymentError:
log("Deployment failed. Retrying metadata upload.")
retry_metadata_upload()
4. Create a Metadata Update Mechanism:
Since there is no easy way to update world metadata, a feature should be introduced that allows this after the world has been created.
Manual Update API/Functionality: Implement a manual method (either through CLI, smart contract functions, or a dedicated update tool) to allow users to upload or update metadata at any point after the world is deployed.
Versioning: Consider adding versioning or timestamps to metadata updates so that any updated information is traceable.
Security: Ensure that only authorized parties (e.g., world owners) can make these updates to avoid malicious changes.
Example:
def update_metadata(world_id, new_metadata):
if authorized_user:
world_metadata[world_id] = new_metadata
log("Metadata updated successfully.")
else:
log("Unauthorized access.")
5. Post-Deployment Checks:
Implement a post-deployment validation step that verifies if the metadata has been uploaded correctly. If metadata is missing or incomplete, prompt the user with options to fix the issue before concluding the process.
Verification: After the contracts are deployed, check if metadata exists for the world.
Prompt for Re-upload: If not, ask the user if they want to retry the metadata upload.
6. Unit Tests and Migrations Validation:
Once the fix is in place, write unit tests and conduct end-to-end testing of the entire migration process to ensure:
The metadata is uploaded properly in all cases (success, failure, or cancellation).
Retrying works as expected when errors occur.
Example Solution Flow:
Deploy base and world contracts.
Upload metadata (only if deployment is successful).
If unsuccessful, allow retry without redeploying the entire contract.
In case of failure: Catch the failure, log it, and prompt for manual or automatic retry.
Introduce metadata update functionality to handle cases where metadata changes are needed after world creation.

I am applying to this issue via OnlyDust platform.
My background and how it can be leveraged
I am a back-end developer with strong experience in systems architecture, databases, and cloud infrastructure. I specialize in frameworks like NestJS and database management systems such as PostgreSQL, which enable me to design reliable and scalable backend services. Additionally, I am proficient in TypeScript, which strengthens my ability to handle complex codebases with a clear focus on typing and structure.
My background enables me to understand issues related to deployment processes and system upgrades. Given my experience, I can identify bottlenecks in software deployments, data synchronization problems, and architectural pitfalls. This skill set would allow me to analyze and resolve bugs like the metadata upload issue in your system.
How I plan on tackling this issue
Analyze and Debug the Existing Code: My first step would be to examine the existing deployment logic related to the dojo::world::world metadatas. I would review how and when the metadatas are triggered for upload and investigate if they are tied to specific conditions during world creation.
Implement a Retry Mechanism: Since the issue occurs when the initial deployment fails, I would introduce a retry mechanism that allows metadata uploads to re-attempt in case of errors. This could be done by adding error-handling mechanisms that ensure metadata upload is retried after failure rather than abandoned.
Add Metadata Update Functionality: I would introduce a separate function or API endpoint allowing updates to metadatas after initial world creation. This would allow administrators or developers to manually update world metadata whenever required, resolving the problem of immutability.
Testing and Validation: I would thoroughly test this fix by simulating various scenarios such as deployment failures, re-attempted migrations, and metadata updates. This would help ensure that the solution is robust across different environments.
Analyze and Debug the Existing Code: My first step would be to examine the existing deployment logic related to the dojo::world::world metadatas. I would review how and when the metadatas are triggered for upload and investigate if they are tied to specific conditions during world creation.
Implement a Retry Mechanism: Since the issue occurs when the initial deployment fails, I would introduce a retry mechanism that allows metadata uploads to re-attempt in case of errors. This could be done by adding error-handling mechanisms that ensure metadata upload is retried after failure rather than abandoned.
Add Metadata Update Functionality: I would introduce a separate function or API endpoint allowing updates to metadatas after initial world creation. This would allow administrators or developers to manually update world metadata whenever required, resolving the problem of immutability.
Testing and Validation: I would thoroughly test this fix by simulating various scenarios such as deployment failures, re-attempted migrations, and metadata updates. This would help ensure that the solution is robust across different environments.

I am applying to this issue via OnlyDust platform.
My background and how it can be leveraged
I am Full-Stack Web Developer with over 4+ years of experience
How I plan on tackling this issue
To address the issue of world metadata only being uploaded once during world creation in the Dojo framework, and the complications that arise if an error occurs during the initial deployment, follow these steps:
Understanding the Problem
Single Upload Limitation: The metadata for dojo::world::world is only uploaded at the time of world creation. If the deployment fails before this upload, you will not have a way to upload the metadata later.
Difficulty in Updating Metadata: Once the metadata is set, there is no straightforward method to update it, which can lead to issues if changes are needed post-deployment.
Steps to Reproduce the Issue
To reproduce this issue using the Dojo starter project:
Start a migration process.
Cancel it after the base and world contracts have been deployed.
Relaunch the migration to observe that the metadata does not upload again.
Proposed Solutions

Error Handling During Deployment
Implement robust error handling in your migration scripts to ensure that any errors encountered during deployment are logged and addressed before proceeding. This includes:
Validation Checks: Before initiating a deployment, validate all necessary parameters and configurations.
Rollback Mechanism: If an error occurs, ensure that you can roll back to a previous stable state without losing any data.
Manual Metadata Upload
If you encounter this issue:
Create a separate script or function that allows for manual uploading of metadata after initial deployment.
This could involve directly interacting with the smart contract or using an API endpoint designed for this purpose.
Update Mechanism for Metadata
Develop a mechanism to allow updates to world metadata:
Consider implementing a function within your smart contract that allows for metadata updates under specific conditions (e.g., only by an admin).
This function should include checks to ensure that updates are valid and do not compromise the integrity of the world state.
Utilize Middleware Features
With the introduction of before and after middleware in Dojo 5, leverage these features to manage processes around world creation and metadata uploads:
Use before middleware to run validation and setup tasks before deploying worlds.
Use after middleware to handle any cleanup or notifications after deployment.

May I try my hand at this?

pls, can you pls assign this to me

May I handle this issue?

Sorry for the delay on this one guys.
The metadata needs to be a bit reworked before being assigned.
Currently they are not uploaded at all, but the new migration flow should be easier to ensure they are uploaded only if they change.
Will ping here once ready to be taken.

Let me handle this issue!

Can I work on this, please?


Feature/seab 2443/prevent curators from changing user permissions

Description
Changes the setUserPrivileges endpoint so that curators cannot change any user's permissions, for example, to make a user an admin or a curator.
The if at line 1043 in the original code prevented an update to the User object if the requested permissions matched the user permissions.  I thought it was worth deleting it for the sake of simplicity, even though doing so might trigger an extra db access every now and then.
Issue
https://ucsc-cgl.atlassian.net/browse/SEAB-2443
Please make sure that you've checked the following before submitting your pull request. Thanks!

[x] Check that you pass the basic style checks and unit tests by running mvn clean install
[x] Follow the existing JPA patterns for queries, using named parameters, to avoid SQL injection
[x] Check the Snyk dashboard to ensure you are not introducing new high/critical vulnerabilities
[x] Assume that inputs to the API can be malicious, and sanitize and/or check for Denial of Service type values, e.g., massive sizes
[x] Do not serve user-uploaded binary images through the Dockstore API
[x] Ensure that endpoints that only allow privileged access enforce that with the @RolesAllowed annotation
[x] Do not create cookies, although this may change in the future

Thanks, this is a welcome change to make least privileges more clear.


Allow to select and copy text of legend items.

What problem does this feature solve?
Currently it's impossible to select text of legend items and copy it into buffer (and then paste somewhere).
What does the proposed API look like?
Legend component could be extended with extra property, like 'selectable' with values 'true', 'false'. Such property switches ability of the legend items to be selectable.

Unless we provide an HTML renderer for legend component,  we can only use SVG renderer and set the SVG container's style by CSS user-select: auto to enable text selection. In canvas, it may be quite complicated and not very practical.
Here is a demo for SVG renderer https://codepen.io/plainheart/pen/abZRgRM

Is it possible to achieve this without SVG but using events (like 'legendselected') ?

Maybe you can use a third-party library like clipboardjs, in the callback of legendselected event, set the text into system clipboard manually by its API.

Thanks for help, will try to make something )))


Allow velocity context values to be passed into final group notification generator

When creating grouped notifications, it is helpful if some values can be stored and passed into the final VelocityContext being used for that generator.  This allows a common notification need - outputting the file name on which a manual action notification is targeted without having to have it repeated over and over in the notification.

Completed - leaving open for any feedback by a project that is testing this to ensure it meets their needs.

CI build passed.


Allow drop into item with no child

When an item has no child then we cannot drop an item into it. This issue is for example visible on
http://angular-ui-tree.github.io/website/tree.html : try to drag node1.1.1 into node1.2
Now the item can be dropped and a class angular-ui-tree-appending-child is added onto hovered element.

I've cloned your repo but this doesn't seem to work. @nterray can you provide a working JSFiddle?

+1


Colors from Tailwind

Describe the problem
Difficult for getting beautiful colors without using Tailwind colors
Describe the proposed solution
<div class="color:blue-500 background:red-500">Hello</di>

Alternatives considered

Importance
would make my life easier

Hi, I have transfered this issue to the web.
Do you have for example any of your favorite Palette you would like to use?
You can import CSS theme from other framework/library like  https://open-props.style/ or some generated online and in Stylify config add:
const config = {
  compiler: {
    // This will enable css variables
    rewriteVariablesByCssVariables: true,
    externalVariables: [
        // Name of used variables like color:$blue1
        'variaable1', 'variable2', 'variable3', blue1
    ]
  }
}


[REVIEW] GitHub Profile @snitin315

GitHub Profile Link:

https://github.com/snitin315

Liked your profile!

Liked your profile!

Thanks!

Great profile! I loved it. 😄

Great profile! I loved it. 😄

Thank you.

suggest aligning and writing the about me section in text not as image.

Thank you for sharing, we will review on a live stream soon 👍

Reviewed on live stream https://youtu.be/i2t50B5lmIU
Looks good, some thoughts...

love the shoutout to past sponsors
try to have the contribution graph a bit more balanced towards issues and PRs (partly due to new year of stats)