Can we coordinate efforts to support Graal Native Image?

Issue details
Graal Native Image allows building an entire game, including what would normally be a bundled JRE, into one executable. This also offers some resistance to decompilation, since assets are stored in the executable rather than in the flimsy protection of a JAR. Various people have worked with Native Image before, and I'm not sure if anyone has a 100% optimal way to use it yet. In the PR https://github.com/libgdx/libgdx/pull/7112 , @Berstanio and I discussed some of this, and I feel like an issue for this partly-missing support is a more appropriate venue to invite further discussion.
Some existing work has been done by anyicomplex (who doesn't seem active on GitHub anymore) at https://github.com/anyicomplex?tab=repositories , Berstanio at https://github.com/Berstanio/gdx-graalhelper , and at least a few other people I remember hearing about. I've gotten native image working with the headless backend (see https://github.com/tommyettinger/noisegen/blob/main/graalvm-env/build_native.bat , which is nasty and uses machine-specific paths ), but not LWJGL3, which is a goal for me. The shared libraries are a problem for LWJGL3 and native images, and getting them to load correctly from inside a platform jar is tricky. If anyone has examples that work, particularly with the current libGDX stable release (1.11.0), that would be greatly appreciated.
Apparently, native images can be built for iOS, but I know nothing about iOS, so I'd find any explanation of how it works there useful.
Reproduction steps/code
Building a JAR into a native image is done after the JAR is built, but currently some Graal dependencies seem to be required in the native binary. I would absolutely love to be wrong about this, and I really hope there's a way to keep the large Graal dependency out of an .exe or the like. The configuration to get native images built is pretty arcane, and it would be helpful to have some good documentation here. At least when using anyicomplex' version of gdx-svmhelper, figuring out what versions of Graal and libGDX were compatible with that library was very challenging.
Version of libGDX and/or relevant dependencies
1.11.0 and onward.
Stacktrace
Not relevant.
Please select the affected platforms

[ ] Android
[ ? ] iOS
[ ] HTML/GWT
[ x ] Windows
[ x ] Linux
[ x ] macOS

Apparently, native images can be built for iOS, but I know nothing about iOS, so I'd find any explanation of how it works there useful.

On iOS it does a normal native-image execution, with the main difference that it exits after a relocatable object file has been created. This object file than gets linked together with the main app. Besides that, it is just native-image.
On iOS we also don't need to worry about libraries, because they are currently statically linked and even if not, they would be extracted to frameworks in any case.

but currently some Graal dependencies seem to be required in the native binary. I would absolutely love to be wrong about this, and I really hope there's a way to keep the large Graal dependency out of an .exe or the like.

They don't end up in the final executable. The dependencies are needed to be able to produce code, that will run at image build time, for custom configuration (e.g. here: https://github.com/Berstanio/gdx-graalhelper/blob/06d0a663981cb5eb72bf61617697dbcbcea9e3f0/src/main/java/com/anyicomplex/gdx/svm/ReflectionRegistrationFeature.java#L162-L166)
The classes won't end up in the final image. No jar would also contain the graal dependencies, since they are provided by native-image, so they are only compileOnly dependencies.

At least when using anyicomplex' version of gdx-svmhelper, figuring out what versions of Graal and libGDX were compatible with that library was very challenging.

That is true, this is why I would be in favor to integrate it into libGDX, so that the native-image configuration gets distributed with the jars and get picked up automatically.

The last part sounds really good, I just have no idea what might be needed. It looks like gdx-graalhelper would need to be updated to the most recent libGDX version or incorporated in some way into libGDX itself. There's also the critical need for LWJGL3 to work with Graal, since most (actually all) of the games I know of that want to resist decompilation are on desktop and have no options other than getting Graal native image to work, or using an old version of Excelsior JET (if they can somehow get one).

I picked up some work on native-imaga again, so here are a few notes:

The shared libraries are a problem for LWJGL3 and native images, and getting them to load correctly from inside a platform jar is tricky

I don't know why I thought that, we can just extract the shared librarys from the native-image as we do with the jars.
I got a native image build so far, loading librarys seems to work. Problematic is atm https://github.com/LWJGL/lwjgl3/blob/4f9767cd5eb87c0e682ec5107b3601bc409d93a9/modules/lwjgl/core/src/main/java/org/lwjgl/system/ThreadLocalUtil.java#L157 , cause it leads to a segfault.

Nice work! Does this need a library dependency, or does it just need Gradle config, or is it as simple as running the Graal tools with the right arguments?

At the moment it needs a modified lwjgl openGL native to run. https://github.com/LWJGL/lwjgl3/issues/875
The rest is only configuration which can be applied in different ways, either with a dependency or manually by the user.

With just a few lines of code, I was able to get rid of the need of a lwjgl3 fork: https://github.com/Berstanio/gdx-graalhelper/commit/4867e9e45eeb62103642246ce4e6532e5d287e43
Now it's only a dependency/configuration thing.

Awesome! I'll try to take a look soon. It might be good to take a quick break from the TextraTypist library debugging I've been doing, and see what I can get running on Graal!

Okay, I haven't documented anything yet, so just the rough overview:

You need lwjgl3-3.3.2
You need to build the gdx-graalhelper branch from source, the "dev" branch
You should use graalvm 22.3.1 Java 11
As extra native-image parameters you need to pass "--no-fallback --report-unsupported-elements-at-runtime"
Depending on your platform you also need to add "-H:IncludeResources=".*dylib.*" or "-H:IncludeResources=".*so.*"
However I would recommend for testing purpose to just pass -H:IncludeResources=".*.* to just include everything
If something doesn't work because of reflection, you can add the "@CollectForReflection" annotation to the class and everything inside will be registered for reflection (java 11 is required)
If something libGDX internal fails for reflection, than it should probably be added to the gdx-graalhelper lib

If anything else, feel free to hit me up on discord.

Okay, a few updates:
I completly revamped the whole reflection registration.  Now it should be way more presice thus reducing resulting imaga size. It should be also way better at automatically registering reflection needs for Pooling and JSON serialization.
bullets JNI registration is still kinda garbage, but SWIG is also a complicated code generator. We will probably need some kind of script to effectivly dump out all needed JNI access. However, that is low prio.
The limitation for Java 11 was lifted, this was mainly done with the fact in mind, that next graal release won't ship java 11 anymore. However, now we rely on some internal API, which isn't great.
Related graal issue: https://github.com/oracle/graal/issues/4925
All native-image parameters should be now passed automatically. Only the "-H:IncludeResources=" should be left needed.
At some point I would like to have at least the native inclusion automatically, but we are blocked by https://github.com/oracle/graal/issues/4767
We are also waiting for https://github.com/oracle/graal/issues/6391 to be fixed. It is also under investigation, whether the struct is Thread local, the same way it is in the jvms. If it is not it wouldn't be a big deal atm., therefor this is rather low prio.
@CollectForReflection was renamed to @CollectForGDXJsonSerialization
At this point, I would prepare the library to be pr'ed for integration into the main libGDX repo. This is useful, since it enables user to build native-images easily without extra dependencys. This is also side effect free, since the configuration will only be read by graal.
Thats all from me for the moment I think!

Just following up on this; I've been using Berstanio's Graal helper for a while now in one repo, and even though there are a few small issues I encountered, it's mostly been solid. The file sizes of zipped .exe files are so much smaller than the bundled JREs with games produced by jpackage... Using Graal 17 produces zipped .exe files under 20MB in size, as long as the assets aren't huge.
I think we can definitely start pushing for wider adoption soon. Maybe that should start with a wiki article?

@tommyettinger I tried your sample on my linux machine. I created a Jar file via the "jar" task and then I was able to create a native binary. However when I try to run that binary I get the following error
[LWJGL] Failed to load a library. Possible solutions:
	a) Add the directory that contains the shared library to -Djava.library.path or -Dorg.lwjgl.librarypath.
	b) Add the JAR that contains the shared library to the classpath.
[LWJGL] Enable debug mode with -Dorg.lwjgl.util.Debug=true for better diagnostics.
[LWJGL] Enable the SharedLibraryLoader debug mode with -Dorg.lwjgl.util.DebugLoader=true for better diagnostics.
Exception in thread "main" java.lang.UnsatisfiedLinkError: Failed to locate library: liblwjgl.so
	at org.lwjgl.system.Library.loadSystem(Library.java:174)
	at org.lwjgl.system.Library.loadSystem(Library.java:64)
	at org.lwjgl.system.Library.<clinit>(Library.java:52)
	at com.github.yellowstonegames.lwjgl3.Lwjgl3Launcher.main(Lwjgl3Launcher.java:20)

Any idea what I'm missing? Is there maybe more to do in order to compile for linux?

@tommyettinger I tried your sample on my linux machine. I created a Jar file via the "jar" task and then I was able to create a native binary. However when I try to run that binary I get the following error
[LWJGL] Failed to load a library. Possible solutions:
	a) Add the directory that contains the shared library to -Djava.library.path or -Dorg.lwjgl.librarypath.
	b) Add the JAR that contains the shared library to the classpath.
[LWJGL] Enable debug mode with -Dorg.lwjgl.util.Debug=true for better diagnostics.
[LWJGL] Enable the SharedLibraryLoader debug mode with -Dorg.lwjgl.util.DebugLoader=true for better diagnostics.
Exception in thread "main" java.lang.UnsatisfiedLinkError: Failed to locate library: liblwjgl.so
	at org.lwjgl.system.Library.loadSystem(Library.java:174)
	at org.lwjgl.system.Library.loadSystem(Library.java:64)
	at org.lwjgl.system.Library.<clinit>(Library.java:52)
	at com.github.yellowstonegames.lwjgl3.Lwjgl3Launcher.main(Lwjgl3Launcher.java:20)

Any idea what I'm missing? Is there maybe more to do in order to compile for linux?

The repo doesn't include linux build steps, as it seems.
https://github.com/yellowstonegames/DetritusScion/blob/main/graal-native/build_native.bat
You need to adjust this for linux accordingly, in case you haven't done that.

You need to adjust this for linux accordingly, in case you haven't done that.

Thanks for pointing me to the right direction. I created a bash script and compiled the executable using the flags as in the .bat script. It runs fine now although I noticed different behaviors across different GraalVM versions but they all do compile and generate a perfectly runnable executable for linux :)
Another observation is that the executable file size is 50MB which is quite big as opposed to the fat jar which is around ~18MB

Great to hear that it works!

Another observation is that the executable file size is 50MB which is quite big as opposed to the fat jar which is around ~18MB

That is very much expected. For a better comparison you would need to compare JRE + fat jar vs executable, since your executable contains also runtime classes.
Another example is, the size of your executable will vary very much based on the amount of reflection you use, since it needs to generate more code to support this. A jar is totally unaffected by that.

The .bat files in DetritusScion are, as you found, just commands that run directly in cmd.exe or a shell. They don't use any Batch-specific features other than maybe using a backslash to continue a line. I often just copy the line it runs into my cmd.exe session, rather than editing the .bat when I want to make a small change. They would probably function as a Linux executable with a shebang, but I don't know if that would let them still work as a Batch file.
As for the size of the executable: try compressing it, I have gotten excellent results! JAR files are already compressed using DEFLATE (the same algorithm as a .zip), but Graal Native Image executables are not by default compressed. They also tend to have a lot of repetitive or duplicate data inside them, it seems, because zipping a Graal Native Image .exe tends to shrink it to about 1/3 the original size, while it will barely make any change to the size of a .zip or .jar file.

As for the size of the executable: try compressing it, I have gotten excellent results!

I don't suppose you mean that I simply wrap the binary into a .zip file?
Otherwise I'm not sure how to do that. Does GraalVM support compression? I'll check it out tho thanks.
I've been trying to streamline this process with other 2D sample games as well but the steps are not always the same. Plus I had a hard time making it work when there's already an Android module in the way. I'll do some more digging and maybe come up with a tutorial at some point.

I don't suppose you mean that I simply wrap the binary into a .zip file?

Actually, that's exactly what I mean! The size of (the .exe) matters less than the size of (everything needed to run the game), so when measuring the bandwidth used by a server to distribute your game, you're probably measuring the size of zipped files (or some other form of compression) rather than uncompressed ones.
When you compare the sizes of a) a JAR without a JVM, b) that JAR without a JVM, zipped, c) that JAR packed into a native binary and zipped, Windows x64, and d) that JAR packed with jpackageimage and zipped:

The Graal Native Image zip is less than half the size of the jpackageimage zip, and less than 1MB larger than the JAR zipped (and the JAR wouldn't be runnable to any users without a JVM, plus no Java 8+ JVMs I know of fit in under 1MB).
The jpackageimage zip includes everything in lwjgl3/build/distributions, while the Graal Native zip includes the .exe and 5 small .dll files (I don't know if they're always required). The zipped JAR is just a .jar file inside a zip archive, which for most users isn't enough to run the game. I haven't compared with Packr yet.


Fragmenting sent handshake messages

Description
Previously handshake messages being sent out would not be fragmented. Most handshake messages are pretty small but a couple of exceptions, such as Certificate, mean the packet size can be greater than the MTU, which then causes some packet loss and the handshake would never be able to complete. This changes the internalSend function to fragment handshakes to the set MTU.
We set MTU as a config option, which defaults to 1200
Reference issue

This is ready for review now. A couple of notes:
I don't love the Marshal() interface, but it didn't really make sense to fit it anywhere else. I can't fragment before packet.Marshal(), because the handshake is just an in-memory struct still; we have no idea the full size of the handshake message itself. And I can't do it after because we need to manipulate the record and handshake headers to support fragmentation properly to spec. It might take a larger refactor to have a cleaner interface. Open to thoughts here.
I also realizing I needed to tackle the
// sequenceNumber and messageSequence line up, may need to be re-evaluated
tech debt because the possibility of fragmentation means these will not align anymore, so there is slightly more code tweaked to fix this case.
One last change was a small improvement to the way the "handshake completed signaling" works so that we don't end up in a situation where we could cause a panic calling an already-closed channel. I can split this out if desired / I may move this change to https://github.com/pion/dtls/pull/113

I am going to spend some time with this, but I am not a big fan of how the MTU is split out all over the code base.
I need to think over it, but it would be really nice if we didn't have to change Marshal() but just had a single function that took care of fragmenting for us.
I don't think there is much danger to double parse/allocate since this isn't a hot path. I do think this could start to introduce substantial code complexity. Maybe this is the only way we can do it, but I will think it over and come back!

@RyanGordon re handshake complete stuff, merge away! Thank you so much for putting it in a smaller PR, it makes it so much easier to just worry about a single thing.

What do you think of keeping something in memory that maps original sequence number -> fragmented
Then only when it comes time to encrypt and send we can say We are sending N, but N-1  was actually these three so we need to account for that.
This adds more state/complexity to that function. But I just think in the long run it will pay off to have something that is very contained/testable.

I agree it would be nicer if we could keep Marshal() the same. I am going to think more on this but the basic complexity is the way the inheritance works with marshaling:
recordHeader.Marshal -> handshakeHeader.Marshal -> handshake.Marshal
If we reversed it, then it becomes much easier to keep the fragmenting and other stateful logic out of Marshal
i.e. very rough-cut:
internalSend(message) {
  var packets [][]byte
  if message.type == handshake {
    handshakes := fragment_handshake(message.handshake)
    for handshake in handshakes {
       record = ...
       packets = append(packets, record.Marshal())
   }
  } else {
   packets = [message.Marshal()]
  }

  // send packets
}

fragment_handshake(handshake) {
  content := handshake.content.Marshal()
  origHeader := handshake.header
  fragments = split(content, MTU_SIZE)

  handshakes = []
  for fragment in fragments {
     fragmentHeader = copy(origHeader)
     fragmentHeader.offset = ...
     fragmentHeader.len = ...
     fragmentHeader.content = fragment
     handshakes = append(handshakes, fragmentHeader)
  }

  return handshakes
}

This is fantastic @RyanGordon!
I am really impressed how fast you rewrote/fixed up this PR. Reading through the code I like it a lot more, it is just nice that it is only in one place.
I think that is the hardest part about OpenSSL. A lot of the TLS code duplicates logic/ a lot of things are interwoven so it is hard to follow.
This is a merge from me!

So checking out the PR I think that if we fire messages for a flight multiple times this will cause issues.
It looks like we increment the sequenceNumber every time we send a fragment. I think this a good design decision.
However if we re-run a flight multiple times the SequenceNumber will not account for that. I think the only change we need to make is to record sequenceNumber at start of flight and then we should be good! Then every time we re-run that flight we need to reset back.
Happy to take care of it also!

Another thing I realized, we currently might send different packets with the same sequenceNumber.
If this flight handler has run multiple times https://github.com/pion/dtls/blob/master/server_handlers.go#L481 we don't increment the global sequenceNumber counter until we have moved onto the next flight.
If we hit an error during handshaking https://github.com/pion/dtls/blob/master/conn.go#L391 we will reuse a sequence number that has already been used. The global sequenceNumber will just be lower than what has actually gone out.
We need to handle sequenceNumbers and replaying flights better in general. The code is confusing error prone. Maybe we should 'replay flights' by pulling them from the local handshake cache? Then we can increment the global sequenceNumber cache w/e we want.

So checking out the PR I think that if we fire messages for a flight multiple times this will cause issues.
It looks like we increment the sequenceNumber every time we send a fragment. That works great for the happy case.
However if we re-run a flight multiple times the SequenceNumber will not account for that. I think the only change we need to make is to record sequenceNumber at start of flight and then we should be good. Then every time we re-run that flight we need to reset back.
Happy to take care of it also!

Hmm, I think this is actually how it works because c.handshakeMessageSequence is used for the handshake header messageSequence field and c.state.localSequenceNumber is only used for the record header sequenceNumber field
c.handshakeMessageSequence is assigned based on the old logic that would correctly set the handshake header messageSequence to the right value for the state of the sequence, including replays.
And now c.state.localSequenceNumber is only used for the record and is incremented every time a record is sent so it is always unique for each sent record (even if the record contains retransmission content). At least according to RFC6347 <IP_ADDRESS> seems to say that the record layer sequence number should always be unique or duplicate packets could be dropped by the receiver depending on the library implementation.

@RyanGordon
(These are just assumptions, please correct me where I am wrong!)

https://github.com/pion/dtls/blob/bc94af06cd09f3ec7fb246e5e0a264382da4e19e/conn.go#L379 always increments the SequenceNumber. We will never have a message with the same sequence number twice.

This means that each call to internal send will use new sequenceNumbers https://github.com/pion/dtls/blob/bc94af06cd09f3ec7fb246e5e0a264382da4e19e/client_handlers.go#L292

pion/dtls itself is unable to handle gaps in sequence numbers. It is very explicit about the SequenceNumber that is used for certain messages https://github.com/pion/dtls/blob/bc94af06cd09f3ec7fb246e5e0a264382da4e19e/server_handlers.go#L188

Hmm okay, that is interesting. I need to do more research into this to make sure I am understanding the spec correctly and that this doesn't introduce breaking behavior. (Currently) My understanding is that the record "sequence number" (I believe) should always be incrementing even on re-transmit. The handshake "message sequence" is the one that should have the specific ordering and be re-used depending on the state of the handshake and used to know where in the handshake the sender is.
I will be double checking what the spec behavior should be, and what impact these changes  would have and what would need to be changed to avoid any impact

Fantastic, sorry for this review being a doozy! I really appreciate how much you are doing to land this.
I would see what OpenSSL does on retransmissions (if it increments the SequenceNumber), and honestly if E2E Lossy passes I think we are in great shape.
I really need to make a nice way for us to do E2E Lossy against OpenSSL.

I really need to make a nice way for us to do E2E Lossy against OpenSSL.
I'm working on that. Once I've recovered from a 9hr timezone shift I'll submit

I really need to make a nice way for us to do E2E Lossy against OpenSSL.
I'm working on that. Once I've recovered from a 9hr timezone shift I'll submit

Well that is just a fantastic surprise. Thanks for grabbing that!

Ok, this is ready for a final review @Sean-Der, et al. Tests should pass but golangci-lint seems to be doing something weird (related to Go 1.13 release?)

Nice @RyanGordon !
Mind rebasing this off of you 1.12 pin I will make sure I review this tonight! If it passes everything that is for sure going to be a merge from me.
really nice work


[app] Add tracing support via Jaeger or Zipkin

kobs now supports tracing via the OpenTelemetry Jaeger or Zipkin
exporter.
At this point not all functions are instrumented, but most of them are
creating spans now. We also propergate the trace information on all
outgoing http requests and try to get them from incoming requests, so
that kobs can be seamlessly integregated with other applications which
are support this.

[x] I added a CHANGELOG entry for this change.
[ ] I adjusted the corresponding documentation for this change.
[ ] I adjusted the values.yaml file and the documentation for all values.

Codecov Report

Merging #361 (c1f2c8f) into main (7883065) will increase coverage by 0.62%.
The diff coverage is 56.07%.

@@            Coverage Diff             @@
##             main     #361      +/-   ##
==========================================
+ Coverage   46.46%   47.09%   +0.62%
==========================================
  Files         128      130       +2
  Lines        9190     9938     +748
==========================================
+ Hits         4270     4680     +410
- Misses       4834     5160     +326
- Partials       86       98      +12

Impacted Files
Coverage Δ

cmd/kobs/hub/hub.go
19.39% <0.00%> (-1.12%)
:arrow_down:

cmd/kobs/satellite/satellite.go
13.09% <0.00%> (-1.58%)
:arrow_down:

pkg/hub/api/plugins/plugins.go
0.00% <ø> (ø)

pkg/hub/hub.go
0.00% <0.00%> (ø)

pkg/kube/clusters/cluster/terminal/terminal.go
0.00% <0.00%> (ø)

pkg/middleware/httptracer/httptracer.go
0.00% <0.00%> (ø)

pkg/satellite/api/resources/resources.go
3.30% <0.00%> (ø)

pkg/satellite/plugins/plugins.go
0.00% <0.00%> (ø)

pkg/satellite/satellite.go
0.00% <0.00%> (ø)

pkg/tracer/tracer.go
0.00% <0.00%> (ø)

... and 28 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7883065...c1f2c8f. Read the comment docs.


Container singleton doesn't respect arguments

Registering a singleton with arguments results in having the same singleton for different arguments
App::bind('logger', function ($container, $type = 'default') { return new Logger($type); });
App::make('logger', 'default'); // creates a new instance
App::make('logger', 'default'); // returns the same singleton - OK
App::make('logger', 'another'); // returns the same singleton - confusing?

Obviously if I call a function with a different argument, I expect a different result.
I this case I would expect a different singleton for each set of arguments.

App::bind() should not create singletons at all!?

Apologies, I meant singleton(). Code above edited.
Thanks, @franzliedke

I think by definition, singletons shouldn't even receive constructor arguments...
Do you really need a shared object?

You can't bind classes with arguments to the closure like that. That makes no sense?

I can.
It does make sense for me.
This functionality probably is there for a reason?
On 04.05.2015. 17:44, Graham Campbell wrote:

You can't bind classes with arguments to the closure like that. That
makes no sense?
—
Reply to this email directly or view it on GitHub
https://github.com/laravel/framework/issues/8593#issuecomment-98736349.


Add handlers for  and

Added handlers for <h2> (H2Elem) and <h5> (H5Elem). They are just copy/pastes from H1Elem with the names changed.

Thanks very much for the PR @grkuntzmd - someone also drew my attention to https://www.reddit.com/r/golang/comments/8b7jh8/custom_element_in_myitcvioreact/ which I'm assuming is a related question?
<main> is indeed a core HTML 5 element and so belongs in myitcv.io/react, alongside H1 etc.
Auto-generating elements from the HTML spec is on the rather long TODO list, hence there are a number of elements missing right now.
Hence you could include <main> in this PR.
Apologies - I'm also missing contribution guidelines for this repo (also on the TODO list!) and to be honest I think I'm missing a couple of scripts to ease the process.
So it might be quickest for me to add the missing elements for you if you like?

@grkuntzmd if you take a look at #118 I've add a helper script _scripts/generate_react.sh which in effect shows how to re-run go generate on the myitcv.io/react package when you add a new element (code generation also runs in the core). Hence you'll see in the https://github.com/myitcv/react/pull/118/files a number of additional files as part of my commit; notably the gen_*.go files.
Thinking behind pushing up #118 was to effectively act as a contribution guideline of sorts.
So if you'd like to adapt this PR I'll happily merge it.
Otherwise if you'd prefer to abandon this, let me know and I'll merge #118.
Thanks again

@grkuntzmd

For the tabindex attribute, since even a zero is significant, I made it a *int. If it is non-nil that attributes is added to the HTML. I have made the required modifications to reactGen.

I agree this is a sensible decision for now.
Just so we're singing from the same hymn sheet, I've been referring to https://developer.mozilla.org/en-US/docs/Web/HTML/Global_attributes/tabindex.

I've made the changes that you requested. I cleaned up the element generator.

I am going to withdraw my pull request. I have decided not to use GopherJS for the project that I am doing and so will not have a chance to use the changes that I am proposing. I believe that a developer should use his own tools, if for no other reason than to make sure they actually work.
Thank you for the time you have spent reviewing the code. Perhaps in the future, if I decide to use GopherJS, I will contact you again.

I am going to withdraw my pull request. I have decided not to use GopherJS for the project that I am doing and so will not have a chance to use the changes that I am proposing. I believe that a developer should use his own tools, if for no other reason than to make sure they actually work.

@grkuntzmd I quite understand; and agree, that's a good approach to take.
Thanks for showing such a keen interest in the first place; I enjoyed our exchange!
All the best with your project.


[fireTV, AndroidTV] Template starter focus issues on Home screen

Describe the bug
Template starter has focus issues on Home screen
To Reproduce

[ ] 1. fireTV, Atv

Home screen is open AND focus is on "Try me!" button
Click up
Focus is lost -> Focus should be on "Try me!" button

[ ] 2. fireTV, Atv

No focus initially after app is opened

Desktop (please complete the following information):

OS: macOS Monterey 12.6
Node Version: 16.14.2
RNV Version: 0.37.0-rc.9
@rnv/template-starter: 0.37.0-rc.9

@rnv/template-starter: 0.38.0-canary.1

[ ] tvOS - button is not focused at all
[ ] androidTV - No focus initially after app is opened, but when button is focused color doesn't change

@rnv/template-starter: 1.0.0-rc.18

webos, tizen -> button is not focused at all

webos, tizen issue is tracked on https://github.com/flexn-io/renative/issues/1579


Disable zooming in/out 'on Scroll' but keep it 'on Click', when using ZoomableGroup?

Hello,
Have you been able figure this out? I am working on something similar and to figure this out would be great!

Hi @ayaseef,
Does this go in the right direction? You can do some quite granular filtering via the filterZoomEvent prop.
https://codesandbox.io/s/react-simple-maps-zoom-event-filtering-szbfy?file=/src/MapChart.js

If anyone has an issue with typescript with this, just add the following to your .d.ts file
(the refernce should be at the top of the ts file)
/// <reference types="react-simple-maps/index.d.ts"/>

declare module 'react-simple-maps' {
  interface ZoomableGroupProps {
    filterZoomEvent: (event: { type: 'wheel' }) => boolean;
  }
}


Fix downloading dataset with gdown in deeplabv3_plus.py

What's the error you're seeing? This code used to work recently.

The error is in
!gdown https://drive.google.com/uc?id=1B9A9UCJYMwTL4oBEo4RZfbMZMaZhKJaz

It throw the message:
Access denied with the following error:
Cannot retrieve the public link of the file. You may need to change
the permission to 'Anyone with the link', or have had many accesses.

You may still be able to access the file from the browser:
 https://drive.google.com/uc?id=1B9A9UCJYMwTL4oBEo4RZfbMZMaZhKJaz

unzip:  cannot find or open instance-level-human-parsing.zip, instance-level-human-parsing.zip.zip or instance-level-human-parsing.zip.ZIP.
So I proposed to change that cell to
``gdown "1B9A9UCJYMwTL4oBEo4RZfbMZMaZhKJaz&confirm=t"

Ok, sounds good! Please replicate the fix in the md and ipynb files.

Ok, sounds good! Please replicate the fix in the md and ipynb files.

Ok, I did what you suggested.. (I modified the files : deeplabv3_plus.ipynb, deeplabv3_plus.py and deeplabv3_plus.md)
The thing is that I tested the command just in the colab file that you provided online (https://colab.research.google.com/github/keras-team/keras-io/blob/master/examples/vision/ipynb/deeplabv3_plus.ipynb), and there it worked well


extend graph navigation - allow for graph to be maintained during node expansion/navigation

allow for graph to be maintained while user requested additions enter the dataset

modifying GraphContainer such that it uses the redux store rather than component local state
lots of tweaks to graph component and related utils due to change in data flow
updating async spinner to use data from central store, as well

video example: https://www.useloom.com/share/bb0a7f160db24987b9ad7f2ecbc35ff9

@davemasselink could you rebase the PR from the latest master?

@fzxu, yes... I'll re-base to make sure it'll merge cleanly with master
and make a couple tweaks based on your comments and another issue I discovered

LGTM. feel free to merge it yourself if you are not making additional changes to this PR.

@fzxu, thx... yeah, I'll merge once I've pushed my final changes and all checks are still passing


chore(upgrade): Upgrade Core Version

[x] Code is up-to-date with the main branch.
[x] You've successfully built and run the tests locally.
[ ] There are new or updated unit tests validating the change.

:pencil: Description

Upgrade Core version to 0.7.1

:link: Related Issues
N/A

Thank you for pushing this, but we did not (and we won't) publish the 0.7.1 core to the maven central. We upgraded the core to this version just to distinguish it from the published 0.7.0 release, as they have a few differences.


database record limits for acceptable lookup performance

I see that all the database implementations are using the iterator from the for finding and querying records. E.g. the document based implementation for get is using the iterator until it finds a matching key. I assume this linear search does not scale for several 10000 or million records. What are the limits here to have an acceptable access performance? Are there any memory recommendations and maximum number of records?

I have tested this with 10000 simple documents. It took 1 min and 43 seconds to insert the data which is pretty slow. Syncing the data to a different node took 57 seconds. Searching the document 5000 in the middle took 26 seconds. Any ideas how to improve this?
Since the search is linear I would assume then search in 100 000 documents would need 260 seconds.

What database types are you testing against? If you are using KeyValue what happens if you use KeyValueIndexed?
Also, can you describe your benchmarking environment in more detail? For example, is this on Node.js or browser?
Any addtional information you can provide about your benchmarking will help us set up something similar in our own benchmarks.

This is a test project: https://github.com/koh-osug/orbitdb-benchmark
I'm using node.js, version v20.11.0. I'm using the document store.

https://github.com/koh-osug/orbitdb-benchmark/issues/1#issuecomment-2040465544
I was looking into some performance issues as well
see also: https://github.com/ipfs/kubo/issues/10383

Regarding write speed:
We have noticed an improved write speed when changing entry, index and heads storage to MemoryStorage (from LevelStorage). Could be related to how LevelStorage stores to disk. This will require further investigation.
Regarding query speed:
The legacy "useRefs" has been removed. This should result in a quicker lookup because it isn't needing to follow various ref paths.
We've also noticed increasing LRU cache size from 1000 to 10000 improves subsequent queries.
Could you please install the next version of OrbitDB which removes useRefs and re-run your benchmarks to see if you are also seeing improved query times.

Since the search is linear I would assume then search in 100 000 documents would need 260 seconds.

Yes you are correct. LevelDB (which is used for the index) does what it is intended to do; get an item quickly. However, it isn't so great at searching text. For this, something else would need to be used, something that is designed for text querying. Whether this is something that is integrated into OrbitDB or whether an external indexing solution is used is up for discussion.

Regarding write speed:
We have noticed an improved write speed when changing entry, index and heads storage to MemoryStorage (from LevelStorage). Could be related to how LevelStorage stores to disk. This will require further investigation.
Regarding query speed:
The legacy "useRefs" has been removed. This should result in a quicker lookup because it isn't needing to follow various ref paths.
We've also noticed increasing LRU cache size from 1000 to 10000 improves subsequent queries.
Could you please install the next version of OrbitDB which removes useRefs and re-run your benchmarks to see if you are also seeing improved query times.

Since the search is linear I would assume then search in 100 000 documents would need 260 seconds.

Yes you are correct. LevelDB (which is used for the index) does what it is intended to do; get an item quickly. However, it isn't so great at searching text. For this, something else would need to be used, something that is designed for text querying. Whether this is something that is integrated into OrbitDB or whether an external indexing solution is used is up for discussion.

I was intending to figure out K Nearest Neighbors search personally, I will look into the changes you've referenced here.

I was working on sharding orbitdb, and it took 8 shards 100 seconds to do 32k inserts on a Xeon E5 V4 2690, i tried to open the slave on my laptop on the same LAN using only mdns, but the process failed with a "want for [random string] arborted", i will try tomorrow with a direct dial, and will try to record how long it takes the 32k records to be sent to the slave database, which is the source for which the KNN search will be done after the 32k records are ingested.


[SMALLFIX] Allow setOwn/setMode on Alluxio but not propagating to object

store UFS.

@calvinjia

LGTM. Thanks for the fix

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/11315/
Test PASSed.

LGTM


Storm 1311 Migration of UI from Clojure to Jersey/Grizzly I

This PR contains the basic frameworks and the GET requests. PUT's and storm.py changes to follow

Can I suggest perhaps a different path forward?  instead of creating the jersey/jackson server part first, instead start migrating the ui implementation to java first while writing some unit tests for them, and then put in the final part that replaces the routing.
i.e.
replace cluster-summary with UIHelpers.getClusterInfo, write unit tests for UIHelpers.getClusterInfo and then verify that the UI works as expected.
Then try replacing json-resonse everywhere in core.clj with something new in UIHelpers...

PR is missing a lot of changes - will put up again in some time.


Server window shows a player online after disconnecting

Version <IP_ADDRESS>
OS: Windows 10

Start Server
Note the 0 Players Online on Windows Tab
Open Client and log in as any character
Note the 1 Players Online on Server Tab
Log out
1 Players Online is still visible on Server Tab

Expecting to see 0 Players Online

I figured out what is causing this, it should be fixed when I am able to push my changes to the main repo.


Update jsoniter-scala-core, ... to 2.1.2

Updates

com.github.plokhotnyuk.jsoniter-scala:jsoniter-scala-core
com.github.plokhotnyuk.jsoniter-scala:jsoniter-scala-macros

from 2.0.4 to 2.1.2.
Release Notes/Changelog
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "com.github.plokhotnyuk.jsoniter-scala" } ]

labels: library-update, semver-minor

Codecov Report

Merging #42 into master will not change coverage.
The diff coverage is n/a.

@@         Coverage Diff          @@
##           master   #42   +/-   ##
====================================
  Coverage       0%    0%
====================================
  Files          24    24
  Lines         229   229
  Branches        8     8
====================================
  Misses        229   229

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 72206b4...a2c45f1. Read the comment docs.


fix(plugin-less): remove duplicate calls

Summary
remove duplicate calls
Related Links

Checklist

[ ] Tests updated.
[ ] Documentation updated.

👍🏻


TPH: Validate shared columns and constraints

It is possible to share columns, foreign keys, and indexes across a hierarchy. For example:
class Animal {
    public int Id { get; set; }
}

class Cat : Animal {
    public string Breed { get; set; }
}

class Dog: Animal {
    public string Breed { get; set; }
}

We should validate (or combine) the configuration on each of the properties. Because conflicts:
modelBuilder.Entity<Cat>().Property(c => c.Breed).HasMaxLength(30);
modelBuilder.Entity<Dog>().Property(d => d.Breed).HasMaxLength(15);

It the current behavior last one wins?

The only place it currently matters is Migrations. I just did Distinct((x, y) => x.Name == y.Name), so one (undefined) of them will win.

:trollface: It's almost like we should maintain a separate model for SSpace with some sort of mapping between the two...

:see_no_evil:

Triage: For the moment we will throw if the two columns are not configured in the same way. We can relax this in the future.


Added support for $elementMatch

Fixes issue #76.
Permits creation of more complicated queries like:
[
    "documents": [
        "$elemMatch": [
            "value": "1234"
        ]
    ]
]

I think the enum case .element should be renamed to something more descriptive. What it does is that it checks for an array at the given key to contain the given element(s)/criteria.
Given that second part (the criteria) there should also be an option to give AQT criteria as the value of this enum case.
What are your thoughts?

Do you have any recommendation of a better name for .element? Perhaps .arrayElement?
The enum case I proposed does support adding AQT Criteria. For example:
[ "documents": [ "$elemMatch": [ "value": "1234" ] ] ]

Would be represented by:
AQT.element(key: "documents", match: .valEquals(key: "value", val: "1234"))

The proposed renamed alternative could be:
AQT.arrayElement(key: "documents", match: .valEquals(key: "value", val: "1234"))

This match parameter receives a AQT value, which could even be an .and for example. Here is an actual example of a complex query which I am currently using in my server (this is the most extreme case I have):
AQT.and([
    .greaterThanOrEqual(key: "modifiedDate", val: Date()),
    .or([
        .contains(key: "name", val: "\\b\(filterString)", options: .caseInsensitive),
        .element(key: "documents", match: .contains(key: "value", val: "^\(filterString)", options: [])),
        .element(key: "associations", match: .contains(key: "houseNumber", val: "^\(filterString)", options: .caseInsensitive)),
        .contains(key: "clubName", val: "\\b\(filterString)", options: .caseInsensitive)
    ]),
    .element(key: "associations", match: .and([
        .valEquals(key: "permission", val: "admin"),
        .valEquals(key: "clubId", val: "SUDN"),
        .greaterThanOrEqual(key: "allowedEndDate", val: Date()),
        .valNotEquals(key: "disabled", val: true)
     ]))
])
```

Sorry for the inactivity. My laptop's USB-c ports were damaged so I'm "digitally handicapped" now.
I think containsElement would be the best fit.

was the rename ok?

Whoops! Sorry! I did not come back to you last week 😬
Looks fine! I'll merge it right away


Prevent creation of tags that are too similar to already existing tags.

To prevent clutter and pollution of the tagspace it could be of interest to use the postgres similarity extensions which uses the Trigram similarity measure to determine how similar entities are.
This is quite easy to implement just a matter of enabling the pg_trgm extension by means of a migration and adding a before_save hook on the tag model.
We can experiment with the similarity threshold but my suggestion would be to use a threshold of 0.3
I've implemented similar functionality quite recently in another rails app so if this sounds interesting let me know and I'll create a PR whenever I have some spare time.

Hi Alexander,
This definitely sounds interesting, but it would be even nicer if we could fix this by suggesting better tags in the tag autocomplete functionality, maybe using this same postgres extension.
It is possible that people would need to create tags that are very similar but have a distinct different meaning for users, blocking those could be annoying.

We could do both ;) but yeah better suggestions is also a possible use-case for similarity measures. We could basically have a class method that would return all tags that meet a certain similarity threshold and sort them by similarity.
I understand your concerns regarding blocking similar tags as a workaround we could allow teams to specify the threshold themselves (per tenant) then if a team wants to use semantic versioning tags (which will alway be very similar) then they can choose a very loose threshold.
The downside of not limiting duplicate tags is the risk of a polluted tagspace which in my opinion far outweighs the possible frustration of a few users as it limits the value of tags and the level of insight they provide.

The ability to merge tags could also help here


Sprint12 Staging

Coverage decreased (-1.07%) to 26.462% when pulling ae3fc7438e9d552fb9024b565fc4126c60e7b9ef on sprint12-staging into 07d784063050b242281dfce393229c5a11bd29fe on master.


[TES-3] Content intro

The Description of a searched result page content including movie cover image,name, genre

done


Corrected example 1

https://github.com/MicrosoftDocs/office-docs-powershell/issues/3546

Docs Build status updates of commit 4e7672e:
:clock10: Full build: the average full build time is 13 min(s) 49 sec(s), this is based on the last 462 build(s) for this repository.

Docs Build status updates of commit 4e7672e:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

exchange/exchange-ps/exchange/devices/Set-DeviceConditionalAccessRule.md
:white_check_mark:Succeeded
View (exchange-ps)

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

Docs Build status updates of commit 9524f1e:
:clock10: Full build: the average full build time is 13 min(s) 38 sec(s), this is based on the last 493 build(s) for this repository.

Docs Build status updates of commit 9524f1e:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

exchange/exchange-ps/exchange/devices/Set-DeviceConditionalAccessRule.md
:white_check_mark:Succeeded
View (exchange-ps)

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

Docs Build status updates of commit f7dbd58:
:clock10: Full build: the average full build time is 13 min(s) 37 sec(s), this is based on the last 496 build(s) for this repository.

Docs Build status updates of commit f7dbd58:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

exchange/exchange-ps/exchange/devices/Set-DeviceConditionalAccessRule.md
:white_check_mark:Succeeded
View (exchange-ps)

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


async support

It doesn't look like you can use an async http client to send requests to the Neo4j backend.
It should be pretty trivial to fix:
in remote/RemoteDBAccess.java
// this
response = iBuilder.post(Entity.entity(json, MediaType.APPLICATION_JSON_TYPE));
// becomes
response = iBuilder.async().post(..., callback)

and we would need to have some .executeAsync() method added.
If need be, i'd be happy to contribute with a PR.. thoughts?

Hi,
I am today working with a customer, I will have look at the issue and contact you again tomorrow (Aug. 27).
Best regards,
Wolfgang
-----Original Message-----
From: ヨーント<EMAIL_ADDRESS>To: Wolfgang-Schuetzelhofer/jcypher<EMAIL_ADDRESS>Sent: Tue, Aug 25, 2015 4:48 pm
Subject: [jcypher] async support? (#5)
It doesn't look like you can use an async http client to send requests to the Neo4j backend.
It should be pretty trivial to fix:
in remote/RemoteDBAccess.java
// this
response = iBuilder.post(Entity.entity(json, MediaType.APPLICATION_JSON_TYPE));
// becomes
response = iBuilder.async().post(..., callback)
and we would need to have some .executeAsync() method added.
If need be, i'd be happy to contribute with a PR.. thoughts?
—
Reply to this email directly or view it on GitHub.

Hi,
before an async execution can work in a multithreaded environment, I have to do some work on threadsafety as JCypher maintains quite a lot of state information about a domain in memory.
That work on threadsafety in order to allow multithreaded concurrent access has to be done anyway and is planned for the next release.
Thereafter I will provide an executeAsync() just as you proposed.
Contributions and ideas are always welcome.
Best regards,
Wolfgang
P.S.: I am out of office the next week


Deprecated iOS values not shown as Deprecated in java

i am used to just looking at the API from AndroidStudio and just looking at the java one, and when it all looks good e.g. UIStatusBarStyle.BlackOpaque i feel ive found the right thing and i move on, then it can take me a long time to realize that i have actually used the wrong thing, and https://developer.apple.com/documentation/uikit/uistatusbarstyle/uistatusbarstyleblackopaque?language=objc is actually deprecated and replaced by a new value, and all this time i am scratching my head trying to work out why the app keeps doing unexpected things

This should be solved now.


Unresolved reference: containerSize in LocalWindowInfo.current.containerSize

Describe the bug
If you use  LocalWindowInfo.current.containerSize in common Main the error Message "Unresolved reference: containerSize" apppears.
For Desktop it works as expected.
Affected platforms
Select one of the platforms below:

Android

Versions

Kotlin version*: 1.9.20
Compose Multiplatform version*: 1.5.10

To Reproduce
Steps and/or the code snippet to reproduce the behavior:

Create a multiplatform Project.
Add the lines
 var x by remember { mutableStateOf(0) }
x = LocalWindowInfo.current.containerSize.width
Start on Android

Expected behavior
There schouldn't be an error.

This API is not available in common/Android - it's declared in non-common actual implementation.
It's probably about an IDE bug that shows you API that is not available in common (desktop's Java implementation doesn't fit Android). We're tracking it here - #3503. Closing as duplicate.
PS JetBrains doesn't publish Android artifacts, so if you are facing with an Android only problem, it's better to use Google's issue tracker as mentioned in the template. But it's not this case.

Hey
I don’t know why IDE suggests containerSize hint in commonMain, but containerSize is not marked as actual value under the hood
actual interface WindowInfo {
    actual val isWindowFocused: Boolean

    actual val keyboardModifiers: PointerKeyboardModifiers
        get() = WindowInfoImpl.GlobalKeyboardModifiers.value

    val containerSize: IntSize get() = IntSize.Zero
}

so you can’t use it in commonMain
You need to define your own expect function like this:
@Composable
expect fun getScreenWidth(): Dp

Android implementation:
@Composable
actual fun getScreenWidth() = LocalConfiguration.current
    .screenWidthDp
    .dp

Others:
@OptIn(ExperimentalComposeUiApi::class)
@Composable
actual fun getScreenWidth() = LocalWindowInfo.current
    .containerSize
    .width
    .dp

@nowiwr01w WindowInfo.containerSize is in pixels, not dp. The correct way to get dp is
with(density) { containerSize.width.toDp() }

Please check the following ticket on YouTrack for follow-ups to this issue. GitHub issues will be closed in the coming weeks.


camel-quarkus-main - Uses reflection for setting its name

Notice this in the http-log example
2020-09-09 13:52:20,310 INFO  [org.apa.cam.imp.eng.DefaultBeanIntrospection] (Quarkus Main Thread) Invoked: 1 times (overall) [Method: setProperty, Target: org.apache.camel.quarkus.core.FastCamelContext@5a2266eb, Arguments: [name, quarkus-camel-example-http-log]]
This is with master branch (eg camel 3.5 as of today)

So FastCamelContext needs to either have a custom generated configurer or we need to make it an instanceof ExtendedCamelContext which would have configurer out of the box.

@lburgazzoli there is some fast factory finder resolver that causes it to not find ExtendedCamelContext.class - maybe its due its using FQN name now?

it is likely that, will have a look

Okay the example can run native compiled (I guess its because the FastCamelContext is open for reflection)

(main) Stopping BeanIntrospection which was invoked: 1 times

The issue is caused by the fact that the configurer for the ExtendedCamelContext has a wrong factory file which is org.apache.camel.impl.ExtendedCamelContext instead of org.apache.camel.ExtendedCamelContext
Opened an issue upstream https://issues.apache.org/jira/browse/CAMEL-15518


Update button appears but throws an error when a user tries to update

Describe the bug
The bug consists in the update button appearing on top-left corner, but when i click on it and on relaunch, an error is thrown with a long stacktrace and a message 'Uncaught Exception: Error: Tried to install update but no update was discovered.'
To Reproduce
Steps to reproduce the behavior:
The update button must be available

Click on 'update'
Click on 'Relaunch to update Lens'
An error will be trown with the following stacktrace

Uncaught Exception:
Error: Tried to install update but no update was discovered.
at /Applications/Lens.app/Contents/Resources/app.asar/node_modules/@lensapp/core-main/dist/index.js:7848:15
at IpcMainImpl.nativeOnCallback (/Applications/Lens.app/Contents/Resources/app.asar/node_modules/@lensapp/messaging-for-main/dist/index.js:60:9)
at IpcMainImpl.emit (node:events:514:28)
at IpcMainImpl.emit (node:domain:489:12)
at WebContents.<anonymous> (node:electron/js2c/browser_init:2:79287)
at WebContents.emit (node:events:514:28)
at WebContents.emit (node:domain:489:12)

Expected behavior
I should update lens.
Screenshots

Environment (please complete the following information):

Lens Version:

Lens: 2024.8.291605-latest
Extension API: 6.10.39
Electron: 27.3.11
Chrome: 118.0.5993.159
Node: 18.17.1
© 2023 Mirantis, Inc.

OS: MacOs Sequoia M3 15.0.1 (24A348)

Hello iFabrycio,
Thank you for reaching out to Lens support!
Thank you for reporting a bug.
We are working on your issue. Stand by for further updates.
Best Regards,
Tikhon Kudinov
Lens Support Engineer
Mirantis, Inc

What update channel is selected for you in the App Settings?


Reproduce Logistic Classifier results

Is it possible to provide the code or architecture for the logistic classifier you used for downstream evaluation of this model embeddings?

The logistic classifier is implemented as the default sklearn classifier, following the example on this page: https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html

Im having trouble replicating the results in supplementary figure 2b. Would you mind sharing the logistic classifier + umap code you used to generate that figure? Were the embeddings generated with the 33 layer or 4 layer model?

For the paper all results were with the 33 layer model.
Unfortunately we cannot share the data for tabula sapiens v2 just yet.
To generate silhouette scores:
from sklearn.metrics import silhouette_samples
def run_benchmark(ad):
    X = ad.obsm["X_embed"]
    labels=ad.obs["cell_ontology_class"]
    scores = (silhouette_samples(X, labels)+ 1) / 2
    return {labels[i]:scores[i] for i in range(len(scores))}

(the +1 and /2 are to keep in line with SCIB).
To generate the UMAPS:
fig, (ax1, ax2, ax3) = plt.subplots(1, 3, sharex=False, sharey=False, figsize=(12, 3), frameon=False, dpi=600)
sc.pl.umap(geneformer_ad_new, color="cell_ontology_class", groups=["b cell"], ax=ax1, show=False)
sc.pl.umap(scgpt_ad_new, color="cell_ontology_class", groups=["b cell"], ax=ax2, show=False)
sc.pl.umap(uce_ad_new, color="cell_ontology_class", groups=["b cell"], ax=ax3, show=False)
plt.savefig("figures/sfig_bcells_comp_umap.svg")

To transfer labels:
from sklearn.linear_model import LogisticRegression
X, y = ica_uce, ica_cts
clf = LogisticRegression(random_state=0).fit(X, y)

ica_uce is the UCE embeddings for the immune cell atlas, ica_cts is the cell types.
tabula_pred = clf.predict(uce_b_cells.obsm["X_uce"])
uce_b_cells.obs["ica_pred_ct"] = tabula_pred

uce_b_cells is the subset of tabula sapiens v2 that was originally classified as b cells.
fig, ax = plt.subplots(1, 1, sharex=False, sharey=False, figsize=(4, 3), frameon=False, dpi=600)

sc.pl.umap(uce_b_cells, color="ica_pred_ct", legend_loc="on data", groups=["naive B cell", "memory B cell"], show=False, ax=ax);

plt.savefig("figures/sfig_bcell_preds.svg")

Thank you! Which observation key did you use for training the logistic regression model on tissue immune atlas (i.e. "Manually_curated_celltype" or "Predicted_labels_Celltypist" etc.) ?

The cell_type column from the dataset on cellxgene, which has 35 clusters. https://cellxgene.cziscience.com/e/1b9d8702-5af8-4142-85ed-020eb06ec4f6.cxg/
I'm not sure which of those two columns that translates to. There might have been some mapping of the original author annotations to the cellXgene cell type names.

Is the data set on that link TabulaSapiensv1? or v2?

Supplementary figure 2b is on tabula sapiens v2 which is not public.
That link is to the cross-tissue immune atlas which is public: https://www.science.org/doi/10.1126/science.abl5197 and was used to transfer the more fine grained B cell annotations.
You can find the UCE embeddings for the immune atlas dataset using the CellXGene API: https://cellxgene.cziscience.com/census-models
I can also upload the h5ad with embeddings, which will be very slightly different than what is on the cellXgene website.

oh yes right my apologies for the confusion. Does the h5ad you have include the cellXGene 'cell_type' labels? The original dataset from source only includes the labels listed under 'Author Categories' so im just looking to find out how to download those 'cell_type' labels

If you download the dataset from https://cellxgene.cziscience.com/e/1b9d8702-5af8-4142-85ed-020eb06ec4f6.cxg/ it will have a cell_type column in the .obs!


scroll_to is now its own request

It looks like this:
scroll_to [line, col]
I'm not sure the best way to handle this, since we lose the height and first_line information that we get with other update requests. This may mean that we need to maintain a local copy of the current "lines" buffer to handle the scrolling, but I'm not sure at this point.

Closed via https://github.com/little-dude/xi-tui/commit/4afc8db5a836aeda3b8131444d27c75c00a92506


Queries with matching files break insertion into database

Steps to replicate:
initial download query
Second download query with matching files
Result:
IntegrityError: (sqlite3.IntegrityError) UNIQUE constraint failed:
file.file_id
[SQL: INSERT INTO file (file_id, dataset_id, master_id, url, version,
filename, local_path, data_node, checksum, checksum_type, size, status, sha)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)]

(Background on this error at: https://sqlalche.me/e/20/gkpj)
See
/gpfscmip/gpfsdata/esgf/esgpull2/log/esgpull-update-2023-04-21_09-58-03.log
for error log.

Further details:
initial selection :
    distrib:       True
    experiment_id: piControl
    frequency:     mon
    mip_era:       CMIP6
    source_id:     CESM2, CMCC-ESM2, CanESM5-CanOE, GISS-E2-1-G, UKESM1-0-LL
    variable_id:   dfe, mlotstmax, no3, phydiat, psl, si, tos, vo
    variant_label: r1i1p1f1

Second selection :
    distrib:       True
    experiment_id: piControl
    frequency:     mon
    mip_era:       CMIP6
    source_id:     CESM2, CMCC-ESM2, CanESM5-CanOE, GISS-E2-1-G, UKESM1-0-LL
    variable_id:   dfe, mlotstmax, no3, phydiat, psl, si, tos, vo

When running
esgpull update <tag of the second selection>
User is prompted with a download confirmation message, which subsequently fails:
846 new files (1.4 TiB).

Aborted!

For the record, this issue has been identified as caused by non-matching checksum fields on 2 replicas of the same file, and has nothing to do with multiple queries pointing to the same files.
This case of miscomputed checksums is currently unhandled and gives the error provided here.
One solution might be to ignore all replicas of that file in this case, since downloading either will probably raise on when file's checksum is computed.

@AtefBN This is fixed with the latest version 0.6.0, live on the conda IPSL channel


[0.9.x] [KOGITO-1750] [cherry-pick] Set simple mgmt console test as smoke test

https://issues.redhat.com/browse/KOGITO-1750
cherry-pick: https://github.com/kiegroup/kogito-cloud-operator/pull/280
Many thanks for submiting your Pull Request :heart:!
Please make sure that your PR meets the following requirements:

[x] You have read the contributors guide
[x] Pull Request title is properly formatted: [KOGITO-XYZ] Subject
[x] Pull Request contains link to the JIRA issue
[x] Pull Request contains description of the issue
[x] Pull Request does not include fixes for issues other than the main ticket
[ ] Your feature/bug fix has a unit test that verifies it
[ ] You've tested the new feature/bug fix in an actual OpenShift cluster

@radtriste saw that you just add a new commit, is this PR ready?
Just merged the one against master.

I just rebased


Clear cache on non-GET form submit

Closes #193
When handling a form submission, turbo clears cache but then re-creates a cache of the current page before navigating to the new page.
  async formSubmissionSucceededWithResponse(formSubmission: FormSubmission, fetchResponse: FetchResponse) {
    if (formSubmission == this.formSubmission) {
      ///...
      if (responseHTML) {
        if (formSubmission.method != FetchMethod.get) {
          this.view.clearSnapshotCache() //<= clear cache
        }

        ///...
        this.proposeVisit(fetchResponse.location, visitOptions) //<= recreates cache
      }
    }
  }

This initial commit is to add a failing test.  I do not have a solution for solving the issue.

@seanpdoyle - one question with this MR, it only covers visits (not turbo-frame or turbo-stream).  In other words:

Visit Non-Get Form Submission clears cache
Turbo-frame Non-Get Form Submission DOES NOT clear cache (should it clear?)
Turbo-stream Non-Get Form Submission DOES NOT clear cache (should it clear?)

FYI: I currently have a situation using turbo-stream where I expected it to clear the cache, but it did not.

I cloned this locally, rolled back the implementation changes, and wasn't able to get the new test to fail.

Odd.  It failed for me locally and in the Github CI pipeline before adding the implementation.

Can we rebase and try again?

@dhh - I can rebase, but there's still an outstanding conversation with @seanpdoyle that needs a decisions.
see: https://github.com/hotwired/turbo/pull/495/files#r765369966

I think we're good to not trigger before:cache in this case, when no caching is happening. We have other lifecycle hooks to use for general transitions. I think we're good to rebase this.


[core] Methods to return PersistentAppStorage, Temp, Desktop system folders

We have methods like GetApplicationDirectory() and GetWorkingDirectory(), but it would be really useful to have a few more system folders that'd be useful for games and apps in a cross-platform manner. I'd like to suggest at least these folders "PersistentAppStorage", "Temp", "Desktop".
PersistentAppStorage
A folder like C://Users/<user>/AppData/LocalLow/ or C://Users/<user>/AppData/Roaming/ on Windows, ~/Library/Application Support/ on Mac, $XDG_CONFIG_HOME/ or $HOME/.config/ on Linux, /storage/emulated/<userid>/Android/data/<packagename>/files on Android, /var/mobile/Containers/Data/Application/<guid>/Documents on iOS.
Why? A lot of programs and games use those folders to save persistent files such as savefiles or app configurations in their own subfolder.
Temp
A folder like C://Users/<user>/AppData/Local/Temp/ on Windows.
Why? We can use the temporary directory to store and load temporary files needed by our application. They will be cleared by the OS later on.
Desktop
A folder like C://Users/<user>/Desktop/ on Windows.
Why? We can use this folder for storing immediately visible data to the user. Also could be used as a default/initial saving location.
These would be the proposed functions:
const char *GetPersistentAppStorageDirectory(void);
const char *GetTempDirectory(void);
const char *GetDesktopDirectory(void);

or as a single method with a flag
const char *GetSystemDirectory(int flag); // SYSTEM_DIRECTORY_TEMP, SYSTEM_DIRECTORY_APPSTORAGE, SYSTEM_DIRECTORY_DESKTOP

This way could be more condensed and future proof if Raylib decides to include more system folders (e.g. HOME, DOCUMENTS, MUSIC, VIDEOS, FONTS).
What should happen if the target platform doesn't have e.g. a Desktop interface/folder?*
In that case, the methods could fall back and try to return the value from GetApplicationDirectory().

IMHO, there are so many possible env vars that their use is very situational. If you know what you want to use, probably could just do something like this instead:
#include "raylib.h"
#include <stdlib.h>

int main(void) {
    InitWindow(800, 450, "getenv test");
    SetTargetFPS(60);

    #if defined (PLATFORM_DESKTOP)
        #if defined(_WIN32)
            const char *str = getenv("APPDATA");
        #elif defined(__linux__)
            const char *str = getenv("XDG_CONFIG_HOME");
        #elif defined(__APPLE__)
            const char *str = getenv("SOMETHING");
        #endif
    #elif defined(PLATFOM_ANDROID)
            const char *str = getenv("SOMETHING");
    #endif

    TraceLog(LOG_INFO, "str: %s", str);

    while (!WindowShouldClose()) {
        BeginDrawing();
        ClearBackground(RAYWHITE);
        EndDrawing();
    }
    CloseWindow();
    return 0;
}

I think this is well outside raylib's scope, and those directories and paths aren't even analogous/existing on every platform
It's situational and something a user should manage for himself

I think it should work well in raylib-extras, as more specific helper functions
See https://github.com/raylib-extras/extras-c

I think querying those directories is a bit out-of-scope for raylib and it implies a good amount of platform-specific code (I did some of those queries on one of my tools using Win32 API, it was a pain). So, not plans to implement it for the moment.


tt_lib.tensor.reduce operation failed with low PCC [Grayskull]

Describe the bug
tt_lib.tensor.reduce operation, with math_op='max' and dim='H', breaks with low PCC value error in some test cases.
tt_lib.tensor.reduce operation, with math_op='sum' and dim='H', breaks with low PCC value error in some test cases.
To Reproduce
Steps to reproduce the behavior:

Checkout main branch
Run unit tests test_reduce_max_h.py and test_reduce_sum_h.py using these commands:
pytest tests/tt_eager/python_api_testing/non_working_unit_tests/grayskull/test_reduce_max_h.py
pytest tests/tt_eager/python_api_testing/non_working_unit_tests/grayskull/test_reduce_sum_h.py

Expected behavior
There is a test case presented in the unit test tests/tt_eager/python_api_testing/non_working_unit_tests/grayskull/test_reduce_max_h.py and it is are expected to fail with low PCC value.
Getting Additional info for the operation under test and its behavior
To get additional information and results for different combinations of input shapes, types, layouts and memory configs for which this operation was tested you can also run locally sweeps for tt_lib.tensor.reduce and check the results. To do this you should:

Follow the Getting Started page to setup the repo, environment variables and python-env
Activate source build/python_env/bin/activate
Run sweeps by using python tests/tt_eager/python_api_testing/sweep_tests/run_pytorch_test.py -i tests/tt_eager/python_api_testing/sweep_tests/test_configs/ci_sweep_tests_broken/grayskull/ttlib_reduce_max_h_test.yaml -o ./result-sweeps
After the run is completed all test sweeps results should be available inside specified output directory (in this case ./result-sweeps). There you will find reduce_max_h_sweep.csv which holds all executed sweeps, among which you can also find the ones that failed and were recreated by the unit test, which you can get by searching unique data_seed field.

@banekg Are you still facing this issue? When I test this test today it is passed. Please find the above image
For sweep test I didn't find such yaml file

@jliangTT and @banekg  This issue is not occurring in WHB0. Even after repetitive runs, the test is passing consistently with no mismatch. Since it is happening only in GS, I suspect the problem is not with the logic involved in the computation.
@jliangTT Let me know how to proceed further.

hey this is good progress. I am going to mark this as blocked. But as this is p2, let's steer our effort to other high priority issue.

Hi @umadevimcw, I'm still getting failed unit tests, for both operations.

@jliangTT @banekg Keeping the input memconfig as SYSTEM_MEMORY is creating an issue. Updated the config and created the PR. With these changes tests are passing

@banekg Please find the PR #7929

@tt-aho Please share your inputs on this. When the input memconfig is system memory we are facing mismatch issue as shown in the image here
https://github.com/tenstorrent/tt-metal/issues/7007#issuecomment-2058772702

I will need to take a look. To confirm the details, this only happens on GS?
And input memconfig is system memory means we are passing TT tensors on host to the op?


Add workaround for flash data cache corruption on read-while-write

@lucasdemarchi

@zehortigoza
Do you investigate the cache reset (FLASH_ACR_DCRST) and determine it was not needed?

@davids5 It is reseting,  I did not try without reset but I can give I try if you think that it don't need...
  /* reset data cache */
  value |= FLASH_ACR_DCRST;
  putreg32(value, STM32_FLASH_A

@zehortigoza - sorry I missed it.  Thank you!


Appstudio update test-component-pac-fyqp

Pipelines as Code configuration proposal

Pipelines as Code CI/test-component-pac-fyqp-on-pull-request has failed.

StatusDurationName

✅ Succeeded
9 seconds
appstudio-init

✅ Succeeded
14 seconds
clone-repository

✅ Succeeded
16 seconds
sast-snyk-check

✅ Succeeded
14 seconds
appstudio-configure-build

✅ Succeeded
2 minutes
build-container

🏃 Running
---
get-clair-results

🏃 Running
---
sanity-inspect-image

✅ Succeeded
13 seconds
sast-go

✅ Succeeded
23 seconds
sast-java-sec-check

🏃 Running
---
clamav-scan

Pipelines as Code CI/test-component-pac-fyqp-on-pull-request has failed.

StatusDurationName

✅ Succeeded
9 seconds
appstudio-init

✅ Succeeded
14 seconds
clone-repository

✅ Succeeded
16 seconds
sast-snyk-check

✅ Succeeded
14 seconds
appstudio-configure-build

✅ Succeeded
2 minutes
build-container

✅ Succeeded
30 seconds
sanity-inspect-image

🏃 Running
---
get-clair-results

✅ Succeeded
23 seconds
sast-java-sec-check

🏃 Running
---
clamav-scan

✅ Succeeded
13 seconds
sast-go

Pipelines as Code CI/test-component-pac-fyqp-on-pull-request has failed.

StatusDurationName

✅ Succeeded
9 seconds
appstudio-init

✅ Succeeded
14 seconds
clone-repository

✅ Succeeded
16 seconds
sast-snyk-check

✅ Succeeded
14 seconds
appstudio-configure-build

✅ Succeeded
2 minutes
build-container

✅ Succeeded
30 seconds
sanity-inspect-image

🏃 Running
---
get-clair-results

✅ Succeeded
23 seconds
sast-java-sec-check

✅ Succeeded
13 seconds
sast-go

🏃 Running
---
clamav-scan


ASP.NET Core hosted Blazor app fails to run in Docker

Repro steps:

Create a new Blazor (ASP.NET Core hosted) app with the Preview 4 bits
Build and run - it works
Right click, Add Docker support (Linux)
Build and run on Docker Linux

Actual result:
Unhandled Exception: System.ArgumentException: The directory name  is invalid.
Parameter name: path
   at System.IO.FileSystemWatcher.CheckPathValidity(String path)
   at System.IO.FileSystemWatcher..ctor(String path)
   at Microsoft.AspNetCore.Builder.AutoRebuildExtensions.WatchFileSystem(BlazorConfig config, Action onWrite)
   at Microsoft.AspNetCore.Builder.BlazorApplicationBuilderExtensions.UseBlazor(IApplicationBuilder app, BlazorOptions options)
   at BlazorDocker.Server.Startup.Configure(IApplicationBuilder app, IWebHostEnvironment env) in C:\Users\daroth\Source\Repos\BlazorDocker\BlazorDocker.Server\Startup.cs:line 43
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.AspNetCore.Hosting.Internal.ConventionBasedStartup.Configure(IApplicationBuilder app)
   at Microsoft.AspNetCore.Mvc.Filters.MiddlewareFilterBuilderStartupFilter.<>c__DisplayClass0_0.<Configure>g__MiddlewareFilterBuilder|0(IApplicationBuilder builder)
   at Microsoft.AspNetCore.HostFilteringStartupFilter.<>c__DisplayClass0_0.<Configure>b__0(IApplicationBuilder app)
   at Microsoft.AspNetCore.Hosting.Internal.WebHost.BuildApplication()
   at Microsoft.AspNetCore.Hosting.Internal.WebHost.StartAsync(CancellationToken cancellationToken)
   at Microsoft.AspNetCore.Hosting.WebHostExtensions.RunAsync(IWebHost host, CancellationToken token, String startupMessage)
   at Microsoft.AspNetCore.Hosting.WebHostExtensions.RunAsync(IWebHost host, CancellationToken token, String startupMessage)
   at Microsoft.AspNetCore.Hosting.WebHostExtensions.RunAsync(IWebHost host, CancellationToken token)
   at Microsoft.AspNetCore.Hosting.WebHostExtensions.Run(IWebHost host)
   at BlazorDocker.Server.Program.Main(String[] args) in C:\Users\daroth\Source\Repos\BlazorDocker\BlazorDocker.Server\Program.cs:line 11

More repro steps:

Disable Blazor auto-build by adding <BlazorRebuildOnFileChange>false</BlazorRebuildOnFileChange> to the client project file.
Build and run the app

Actual result:
Unhandled Exception: System.ArgumentException: The path must be absolute.
   at Microsoft.Extensions.FileProviders.PhysicalFileProvider..ctor(String root, ExclusionFilters filters)
Parameter name: root
   at Microsoft.AspNetCore.Builder.BlazorApplicationBuilderExtensions.UseBlazor(IApplicationBuilder app, BlazorOptions options)
   at Microsoft.Extensions.FileProviders.PhysicalFileProvider..ctor(String root, ExclusionFilters filters)
   at BlazorDocker.Server.Startup.Configure(IApplicationBuilder app, IWebHostEnvironment env) in C:\Users\daroth\Source\Repos\BlazorDocker\BlazorDocker.Server\Startup.cs:line 43
   at Microsoft.AspNetCore.Builder.BlazorApplicationBuilderExtensions.UseBlazor(IApplicationBuilder app, BlazorOptions options)
--- End of stack trace from previous location where exception was thrown ---
   at BlazorDocker.Server.Startup.Configure(IApplicationBuilder app, IWebHostEnvironment env) in C:\Users\daroth\Source\Repos\BlazorDocker\BlazorDocker.Server\Startup.cs:line 43
   at Microsoft.AspNetCore.Hosting.Internal.ConventionBasedStartup.Configure(IApplicationBuilder app)
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.AspNetCore.Mvc.Filters.MiddlewareFilterBuilderStartupFilter.<>c__DisplayClass0_0.<Configure>g__MiddlewareFilterBuilder|0(IApplicationBuilder builder)
   at Microsoft.AspNetCore.Hosting.Internal.ConventionBasedStartup.Configure(IApplicationBuilder app)
   at Microsoft.AspNetCore.HostFilteringStartupFilter.<>c__DisplayClass0_0.<Configure>b__0(IApplicationBuilder app)
   at Microsoft.AspNetCore.Mvc.Filters.MiddlewareFilterBuilderStartupFilter.<>c__DisplayClass0_0.<Configure>g__MiddlewareFilterBuilder|0(IApplicationBuilder builder)
   at Microsoft.AspNetCore.HostFilteringStartupFilter.<>c__DisplayClass0_0.<Configure>b__0(IApplicationBuilder app)
   at Microsoft.AspNetCore.Hosting.Internal.WebHost.BuildApplication()
   at Microsoft.AspNetCore.Hosting.Internal.WebHost.BuildApplication()
crit: Microsoft.AspNetCore.Hosting.Internal.WebHost[6]
   at Microsoft.AspNetCore.Hosting.Internal.WebHost.StartAsync(CancellationToken cancellationToken)
      Application startup exception
   at Microsoft.AspNetCore.Hosting.WebHostExtensions.RunAsync(IWebHost host, CancellationToken token, String startupMessage)
System.ArgumentException: The path must be absolute.
Parameter name: root
   at Microsoft.AspNetCore.Hosting.WebHostExtensions.RunAsync(IWebHost host, CancellationToken token, String startupMessage)
   at Microsoft.Extensions.FileProviders.PhysicalFileProvider..ctor(String root, ExclusionFilters filters)
   at Microsoft.AspNetCore.Hosting.WebHostExtensions.RunAsync(IWebHost host, CancellationToken token)
   at Microsoft.AspNetCore.Builder.BlazorApplicationBuilderExtensions.UseBlazor(IApplicationBuilder app, BlazorOptions options)
   at Microsoft.AspNetCore.Hosting.WebHostExtensions.Run(IWebHost host)
   at BlazorDocker.Server.Program.Main(String[] args) in C:\Users\daroth\Source\Repos\BlazorDocker\BlazorDocker.Server\Program.cs:line 11
   at BlazorDocker.Server.Startup.Configure(IApplicationBuilder app, IWebHostEnvironment env) in C:\Users\daroth\Source\Repos\BlazorDocker\BlazorDocker.Server\Startup.cs:line 43
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.AspNetCore.Hosting.Internal.ConventionBasedStartup.Configure(IApplicationBuilder app)
   at Microsoft.AspNetCore.Mvc.Filters.MiddlewareFilterBuilderStartupFilter.<>c__DisplayClass0_0.<Configure>g__MiddlewareFilterBuilder|0(IApplicationBuilder builder)
   at Microsoft.AspNetCore.HostFilteringStartupFilter.<>c__DisplayClass0_0.<Configure>b__0(IApplicationBuilder app)
   at Microsoft.AspNetCore.Hosting.Internal.WebHost.BuildApplication()

Still fails because the generated file BlazorASPNETCoreHosted.Client.blazor.config is generating a filepath with Windows style. Since this is Docker on Linux, it fails:
BlazorASPNETCoreHosted.Client.blazor.config:

D:\DevSpikes\BlazorASPNETCoreHosted\BlazorASPNETCoreHosted.Client\BlazorASPNETCoreHosted.Client.csproj
bin\Debug\netstandard2.0\BlazorASPNETCoreHosted.Client.dll
autorebuild:true
debug:true

I tried to remove the Windows style and have it as a relative path, and I was able to run it on Docker okay:
BlazorASPNETCoreHosted.Client.blazor.config:

BlazorASPNETCoreHosted.Client.csproj
bin\Debug\netstandard2.0\BlazorASPNETCoreHosted.Client.dll
autorebuild:true
debug:true

However, whenever I compile this project from Visual Studio, this path is re-generated, so it is a bug that needs to be fixed.

@CESARDELATORRE I agree this is a bug we should fix. However the fix won't be out before Build, because it's too late.
In the meantime, you might be able to use a hacky workaround like the following. To have your custom .blazor.config regenerated on each build, add something like the following to your client app's .csproj:
  <Target Name="FixBlazorConfigForDocker" AfterTargets="GenerateBlazorMetadataFile">
    <WriteLinesToFile File="$(BlazorMetadataFilePath)" Lines="BlazorASPNETCoreHosted.Client.csproj" Overwrite="true" Encoding="Unicode"/>
    <WriteLinesToFile File="$(BlazorMetadataFilePath)" Lines="$(OutDir)$(AssemblyName).dll" Overwrite="false" Encoding="Unicode"/>
    <WriteLinesToFile File="$(BlazorMetadataFilePath)" Condition="'$(BlazorRebuildOnFileChange)'=='true'" Lines="autorebuild:true" Overwrite="false" Encoding="Unicode"/>
    <WriteLinesToFile File="$(BlazorMetadataFilePath)" Condition="'$(BlazorEnableDebugging)'=='true'" Lines="debug:true" Overwrite="false" Encoding="Unicode"/>
  </Target>

This will produce the same content that you showed above, i.e.:
BlazorASPNETCoreHosted.Client.csproj
bin\Debug\netstandard2.0\BlazorASPNETCoreHosted.Client.dll
autorebuild:true
debug:true

@SteveSandersonMS - Thanks Steve. That hack in the client .csproj works perfectly when running the app on RELEASE mode from Visual Studio even re-generating the build from scratch (I tested it deleting /bin /obj folders), which is good so I don't need to manually fix the .blazor.config file. :)
However, if you run the project in DEBUG mode (F5) from Visual Studio, it still impacts braking the app and you still get the following exception:
System.ArgumentException
HResult=0x80070057
Message=The directory name  is invalid.
Source=System.IO.FileSystem.Watcher
StackTrace:
at System.IO.FileSystemWatcher.CheckPathValidity(String path)
at System.IO.FileSystemWatcher..ctor(String path)
at Microsoft.AspNetCore.Builder.AutoRebuildExtensions.WatchFileSystem(BlazorConfig config, Action onWrite)
at Microsoft.AspNetCore.Builder.BlazorApplicationBuilderExtensions.UseBlazor(IApplicationBuilder app, BlazorOptions options)
at BlazorSentimentAnalysis.Server.Startup.Configure(IApplicationBuilder app, IWebHostEnvironment env) in D:\DevSpikes\ScalableSentimentAnalysisBlazorWebApp-Docker\BlazorSentimentAnalysis.Server\Startup.cs:line 60
So the fix on the generation code for the bug must be impacting additional DEBUG features that don't understand now the relative path.
In any case, thanks a lot for the workarounds. I can manage with these workarounds until the bug is fixed, eventually. Thanks! 👍

The fix above worked for me on Preview5 , I am not sure if this is related to the issue, but I also get a System.FileNotFoundException /src/BlazorASPNETCoreHosted.Client/wwwroot when trying this with preview6 and adjusting the docer images to use preview6 as well. With or without the target, the issue remains the same. Let me know if I should open a new seperate issue. Thanks.

There is an issue with Preview6 now:
Unhandled Exception: System.ArgumentException: The path must be absolute.
Parameter name: root
at Microsoft.Extensions.FileProviders.PhysicalFileProvider..ctor(String root, ExclusionFilters filters)
at Microsoft.Extensions.FileProviders.PhysicalFileProvider..ctor(String root)
at Microsoft.AspNetCore.Builder.BlazorHostingEndpointRouteBuilderExtensions.MapFallbackToClientSideBlazor(IEndpointRouteBuilder endpoints, String clientAssemblyFilePath, String pattern, String filePath)
at Microsoft.AspNetCore.Builder.BlazorHostingEndpointRouteBuilderExtensions.MapFallbackToClientSideBlazor[TClientApp](IEndpointRouteBuilder endpoints, String pattern, String filePath)
at Favorites.Server.Startup.<>c.b__4_0(IEndpointRouteBuilder endpoints) in C:\Users\Herman\Source\Workspaces\Default\Favorites\Favorites.Server\Startup.cs:line 72
at Microsoft.AspNetCore.Builder.EndpointRoutingApplicationBuilderExtensions.UseEndpoints(IApplicationBuilder builder, Action`1 configure)
at Favorites.Server.Startup.Configure(IApplicationBuilder app, IWebHostEnvironment env) in C:\Users\Herman\Source\Workspaces\Default\Favorites\Favorites.Server\Startup.cs:line 73
at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor, Boolean wrapExceptions)
at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
at Microsoft.AspNetCore.Hosting.Internal.ConfigureBuilder.Invoke(Object instance, IApplicationBuilder builder)
at Microsoft.AspNetCore.Hosting.Internal.ConfigureBuilder.<>c__DisplayClass4_0.b__0(IApplicationBuilder builder)
at Microsoft.AspNetCore.Hosting.Internal.ConventionBasedStartup.Configure(IApplicationBuilder app)
at Microsoft.AspNetCore.Mvc.Filters.MiddlewareFilterBuilderStartupFilter.<>c__DisplayClass0_0.g__MiddlewareFilterBuilder|0(IApplicationBuilder builder)
at Microsoft.AspNetCore.HostFilteringStartupFilter.<>c__DisplayClass0_0.b__0(IApplicationBuilder app)
at Microsoft.AspNetCore.Hosting.Internal.WebHost.BuildApplication()
at Microsoft.AspNetCore.Hosting.Internal.WebHost.StartAsync(CancellationToken cancellationToken)
at Microsoft.AspNetCore.Hosting.WebHostExtensions.RunAsync(IWebHost host, CancellationToken token, String startupMessage)
at Microsoft.AspNetCore.Hosting.WebHostExtensions.RunAsync(IWebHost host, CancellationToken token, String startupMessage)
at Microsoft.AspNetCore.Hosting.WebHostExtensions.RunAsync(IWebHost host, CancellationToken token)
at Microsoft.AspNetCore.Hosting.WebHostExtensions.Run(IWebHost host)

We're also hitting this in dotnet/try:
 System.ArgumentException: The path must be absolute.
Parameter name: root
   at Microsoft.Extensions.FileProviders.PhysicalFileProvider..ctor(String root, ExclusionFilters filters)
   at Microsoft.Extensions.FileProviders.PhysicalFileProvider..ctor(String root)
   at Microsoft.AspNetCore.Builder.BlazorHostingEndpointRouteBuilderExtensions.MapFallbackToClientSideBlazor(IEndpointRouteBuilder endpoints, String clientAssemblyFilePath, String pattern, String filePath)
   at Microsoft.AspNetCore.Builder.BlazorHostingEndpointRouteBuilderExtensions.MapFallbackToClientSideBlazor[TClientApp](IEndpointRouteBuilder endpoints, String filePath)
   at MLS.Agent.Startup.<>c.<Configure>b__12_3(IEndpointRouteBuilder endpoints) in F:\workspace.13\_work\1\s\MLS.Agent\Startup.cs:line 178
   at Microsoft.AspNetCore.Builder.EndpointRoutingApplicationBuilderExtensions.UseEndpoints(IApplicationBuilder builder, Action`1 configure)
   at MLS.Agent.Startup.<>c.<Configure>b__12_1(IApplicationBuilder blazor) in F:\workspace.13\_work\1\s\MLS.Agent\Startup.cs:line 179
   at Microsoft.AspNetCore.Builder.MapExtensions.Map(IApplicationBuilder app, PathString pathMatch, Action`1 configuration)
   at MLS.Agent.Startup.Configure(IApplicationBuilder app, IHostApplicationLifetime lifetime, IBrowserLauncher browserLauncher, IDirectoryAccessor directoryAccessor, PackageRegistry packageRegistry) in F:\workspace.13\_work\1\s\MLS.Agent\Startup.cs:line 182
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor, Boolean wrapExceptions)
   at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at Microsoft.AspNetCore.Hosting.Internal.ConfigureBuilder.Invoke(Object instance, IApplicationBuilder builder)
   at Microsoft.AspNetCore.Hosting.Internal.ConfigureBuilder.<>c__DisplayClass4_0.<Build>b__0(IApplicationBuilder builder)
   at Microsoft.AspNetCore.Hosting.Internal.ConventionBasedStartup.Configure(IApplicationBuilder app)
   at Microsoft.AspNetCore.Mvc.Filters.MiddlewareFilterBuilderStartupFilter.<>c__DisplayClass0_0.<Configure>g__MiddlewareFilterBuilder|0(IApplicationBuilder builder)
   at Microsoft.AspNetCore.Hosting.Internal.WebHost.BuildApplication()
   at Microsoft.AspNetCore.Hosting.Internal.WebHost.StartAsync(CancellationToken cancellationToken)
   at Microsoft.AspNetCore.Hosting.WebHostExtensions.RunAsync(IWebHost host, CancellationToken token, String startupMessage)
   at Microsoft.AspNetCore.Hosting.WebHostExtensions.RunAsync(IWebHost host, CancellationToken token, String startupMessage)
   at Microsoft.AspNetCore.Hosting.WebHostExtensions.RunAsync(IWebHost host, CancellationToken token)
   at Microsoft.AspNetCore.Hosting.WebHostExtensions.Run(IWebHost host)

@RealTadango In my case, this was happening because we were calling "UseContentRoot" in a way that could cause it to be something other than the location of our assembly. Changing it to:
.UseContentRoot(Path.GetDirectoryName(typeof(Program).Assembly.Location))

seems to help.

@rchande I don't use UseContentRoot

Any workaround for this? UseContentRoot didn't work for me

Still a problem with preview7

@javiercn Possibly related to #11609?

It is, but blazor has its own mechanism to expose the files to the application host.

This will be fixed soon.

With 3.0.100-preview9-014004 still the same.
`
System.ArgumentException
HResult=0x80070057
Message=The path must be absolute.
Source=Microsoft.Extensions.FileProviders.Physical
StackTrace:
at Microsoft.Extensions.FileProviders.PhysicalFileProvider..ctor(String root, ExclusionFilters filters)
at Microsoft.Extensions.FileProviders.PhysicalFileProvider..ctor(String root)
at Microsoft.AspNetCore.Builder.BlazorHostingApplicationBuilderExtensions.UseClientSideBlazorFiles(IApplicationBuilder app, String clientAssemblyFilePath)
at Microsoft.AspNetCore.Builder.BlazorHostingApplicationBuilderExtensions.UseClientSideBlazorFiles[TClientApp](IApplicationBuilder app)
at BlazorDocker.Server.Startup.Configure(IApplicationBuilder app, IWebHostEnvironment env) in C:\Users\jbogad\source\repos\BlazorDocker\Server\Startup.cs:line 37
at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor, Boolean wrapExceptions)
at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
at Microsoft.AspNetCore.Hosting.ConfigureBuilder.Invoke(Object instance, IApplicationBuilder builder)
at Microsoft.AspNetCore.Hosting.ConfigureBuilder.<>c__DisplayClass4_0.b__0(IApplicationBuilder builder)
at Microsoft.AspNetCore.Hosting.ConventionBasedStartup.Configure(IApplicationBuilder app)
`

I have the same issue with .net core 3 preview 9.
The path is clientAssemblyFilePath == "/app/bin/Debug/netcoreapp3.0/MessagingManagement.UI.dll"
Any ETA when it will be fixed?
System.ArgumentException
HResult=0x80070057
Message=The path must be absolute.
Source=Microsoft.Extensions.FileProviders.Physical
StackTrace:
at Microsoft.Extensions.FileProviders.PhysicalFileProvider..ctor(String root, ExclusionFilters filters)
at Microsoft.Extensions.FileProviders.PhysicalFileProvider..ctor(String root)
at Microsoft.AspNetCore.Builder.BlazorHostingApplicationBuilderExtensions.UseClientSideBlazorFiles(IApplicationBuilder app, String clientAssemblyFilePath)
at Microsoft.AspNetCore.Builder.BlazorHostingApplicationBuilderExtensions.UseClientSideBlazorFiles[TClientApp](IApplicationBuilder app)
at MessagingManagement.Server.Startup.Configure(IApplicationBuilder app, IWebHostEnvironment env) in C:\Projects\PoC\MessagingManagementUI\src\MessagingManagement.Server\Startup.cs:line 57

The fix is available in the public preview version of visual studio and will be generally available in the next update.
Closing this as there's no need for further action.

@javiercn @mkArtakMSFT This is marked as closed with 3.1.0-preview 1. However, this error still occurs with an ASP.NET Core hosted WebAssemlby project.
.NET Core Version: 3.1.0-preview1.19506.1
Visual Studio Version: 16.4.0 Preview 2.0
  HResult=0x80070057
  Message=The path must be absolute.
  Source=Microsoft.Extensions.FileProviders.Physical
  StackTrace:
   at Microsoft.Extensions.FileProviders.PhysicalFileProvider..ctor(String root, ExclusionFilters filters)
   at Microsoft.Extensions.FileProviders.PhysicalFileProvider..ctor(String root)
   at Microsoft.AspNetCore.Builder.BlazorHostingApplicationBuilderExtensions.UseClientSideBlazorFiles(IApplicationBuilder app, String clientAssemblyFilePath)
   at Microsoft.AspNetCore.Builder.BlazorHostingApplicationBuilderExtensions.UseClientSideBlazorFiles[TClientApp](IApplicationBuilder app)
   at UNITS.Server.Startup.Configure(IApplicationBuilder app, IWebHostEnvironment env) in S:\herdo-opensource\HoU UNITS\src\UNITS\Server\Startup.cs:line 255
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor, Boolean wrapExceptions)
   at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at Microsoft.AspNetCore.Hosting.ConfigureBuilder.Invoke(Object instance, IApplicationBuilder builder)
   at Microsoft.AspNetCore.Hosting.ConfigureBuilder.<>c__DisplayClass4_0.<Build>b__0(IApplicationBuilder builder)
   at Microsoft.AspNetCore.Hosting.ConventionBasedStartup.Configure(IApplicationBuilder app)
   at Microsoft.AspNetCore.Mvc.Filters.MiddlewareFilterBuilderStartupFilter.<>c__DisplayClass0_0.<Configure>g__MiddlewareFilterBuilder|0(IApplicationBuilder builder)
   at Microsoft.ApplicationInsights.AspNetCore.ApplicationInsightsStartupFilter.<>c__DisplayClass2_0.<Configure>b__0(IApplicationBuilder app)
   at Microsoft.AspNetCore.HostFilteringStartupFilter.<>c__DisplayClass0_0.<Configure>b__0(IApplicationBuilder app)
   at Microsoft.AspNetCore.Hosting.WebHost.BuildApplication()
   at Microsoft.AspNetCore.Hosting.WebHost.<StartAsync>d__27.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
   at Microsoft.AspNetCore.Hosting.WebHostExtensions.<RunAsync>d__5.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Microsoft.AspNetCore.Hosting.WebHostExtensions.<RunAsync>d__5.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
   at Microsoft.AspNetCore.Hosting.WebHostExtensions.<RunAsync>d__4.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
   at Microsoft.AspNetCore.Hosting.WebHostExtensions.Run(IWebHost host)
   at UNITS.Server.Program.Main(String[] args) in S:\herdo-opensource\HoU UNITS\src\UNITS\Server\Program.cs:line 16

This is related to webassembly, which we will be focusing on in the coming months

@javiercn So should I make a new issue for the blazor-wasm milestone - referencing this issue, and/or #10747? I don't seem to find any other issue reporting this problem for that milestone.


[question] Add filter to the every column

I have to add filter for each column like this: Filter for each column in Backgrid

But I can't understand how should I extend Backgrid.Header to achieve this. Can you point me to the right direction?

@terales This feature is part of which plugin?

@irraju, this feature wasn't implemented in project.

@terales did you ever build an extension to handle this?
Currently experimenting with a few things and curious how others may have handled this feature.

@phillipsnick no, i didn't. We walked away from tables on that project :smile:


api or code snippet to intergate map monitor in any html page

hi
need to know if you have any option in api or code snippet to intergate map monitor for specific user in any html page (view only not control or manage device)

Unfortunately I don't have any example for you.


Redis is not able to connect to newer IP of redis pod in case of restart

When I am restarting redis pods, the redis is not able to connect to newer pods IP. I am getting below errors. The older IP's are getting cached.
ERROR-Unable to unfreeze entry: [freeSubscribeConnectionsAmount=0, freeSubscribeConnectionsCounter=value:51:queue:0, freeConnectionsAmount=0, freeConnectionsCounter=value:66:queue:0, freezeReason=MANAGER,
I am passing the cluster address at the time of connecting and using 3.18.1 redisson version.

Please update to 3.23.2 or the latest version

@mrniko Please can you confirm under which fix in 3.23.2 version this issue is fixed ?


docs(READEME.md): change image size

Type of change:

[ ] Bugfix
[ ] New feature provided
[x] Improve performance
[ ] Backport patches

What this PR does / why we need it:

The image is to flat, so I change the changed the ratio of height and width.
solved #796
Pre-submission checklist:

[x] Did you explain what problem does this PR solve? Or what new features have been added?
[ ] Have you added corresponding test cases?
[x] Have you modified the corresponding document?
[x] Is this PR backward compatible? If it is not backward compatible, please discuss on the mailing list first

cc @gxthrj


gaes = (gaes - gaes.mean()) / gaes.std()

What does this formula mean in ppo? Thanks

I can't remember clearly, but I think it is for stability.
Similar question and answer is found in the link below.
Hope it answers your question!
https://datascience.stackexchange.com/questions/20098/why-do-we-normalize-the-discounted-rewards-when-doing-policy-gradient-reinforcem


Ethier sign addresses

Intro
Extend the ethier CLI to sign allowances for a given list of addresses
Example usage
echo -e '0x7CB4297274c620E6EFEF72Fd0A2ED4CFd15fdAb1\n0x8C647f0AbA9E801417E3Ac4af26D5502328f86CB\n\n' > addresses && cat addresses | go run ./ethier sign addresses

Example with mnemonic
echo -e '0x7CB4297274c620E6EFEF72Fd0A2ED4CFd15fdAb1\n0x8C647f0AbA9E801417E3Ac4af26D5502328f86CB\n\n' > addresses && echo -e 'window model trial cotton logic title item smile tunnel segment depend dose tape hope treat reason final refuse base season tourist recycle obey toss\nSUPERSECRET_PASS\n42' > mnemonic && cat addresses | go run ./ethier sign addresses --mnemonic ./mnemonic

Edit 01: add mnemonic example
Todo

[x] Set up the CLI  command sign addresses
[x] Sign addresses using a newly generated signer
[x] Output of address/signature pairs as JSON
[x] Provide a signer via mnemoics from a file using the --mnemonics=<path> flag

Sorry, I totally forgot to add a comment to inform you about the changes that I made.
In the last iteration, I removed signing using mnemonics from the CL again because a proper account management system will be introduced later. The code is therefore mostly just defining the ethier sign command with the basic signing logic using a randomly generated signer.