Downloads after NVD init don't seem to be using proxy settings correctly

Describe the bug
We have a corporate proxy in place and I am providing the settings including proxy user and proxy pass through the JAVA_TOOL_OPTIONS as described in the documentation.
This works for the NVD updates, but as soon as I get to the the point where it wants to init the retireJS repo or download the publishedSupressions.xml, then I receive a 407 error from the proxy.
Version of dependency-check used
The problem occurs using version 10.0.1
Log file
[INFO] Checking for updates
[INFO] Skipping the NVD API Update as it was completed within the last 240 minutes
[ERROR] Failed to initialize the RetireJS repo
org.owasp.dependencycheck.data.update.exception.UpdateException: Failed to initialize the RetireJS repo
        at org.owasp.dependencycheck.data.update.RetireJSDataSource.initializeRetireJsRepo(RetireJSDataSource.java:152)
        at org.owasp.dependencycheck.data.update.RetireJSDataSource.update(RetireJSDataSource.java:95)
        at org.owasp.dependencycheck.Engine.doUpdates(Engine.java:906)
        at org.owasp.dependencycheck.Engine.initializeAndUpdateDatabase(Engine.java:711)
        at org.owasp.dependencycheck.Engine.analyzeDependencies(Engine.java:637)
        at org.owasp.dependencycheck.App.runScan(App.java:262)
        at org.owasp.dependencycheck.App.run(App.java:194)
        at org.owasp.dependencycheck.App.main(App.java:89)
Caused by: org.owasp.dependencycheck.utils.DownloadFailedException: Download failed, unable to copy 'https://raw.githubusercontent.com/Retirejs/retire.js/master/repository/jsrepository.json' to 'C:\actions-runner\_work\NSDT\dependency-check\data\jsrepository.json'; Error downloading file https://raw.githubusercontent.com/Retirejs/retire.js/master/repository/jsrepository.json; unable to connect.
        at org.owasp.dependencycheck.utils.Downloader.fetchFile(Downloader.java:152)
        at org.owasp.dependencycheck.utils.Downloader.fetchFile(Downloader.java:100)
        at org.owasp.dependencycheck.data.update.RetireJSDataSource.initializeRetireJsRepo(RetireJSDataSource.java:150)
        ... 7 common frames omitted
Caused by: org.owasp.dependencycheck.utils.DownloadFailedException: Error downloading file https://raw.githubusercontent.com/Retirejs/retire.js/master/repository/jsrepository.json; unable to connect.
        at org.owasp.dependencycheck.utils.HttpResourceConnection.obtainConnection(HttpResourceConnection.java:267)
        at org.owasp.dependencycheck.utils.HttpResourceConnection.fetch(HttpResourceConnection.java:163)
        at org.owasp.dependencycheck.utils.Downloader.fetchFile(Downloader.java:138)
        ... 9 common frames omitted
Caused by: java.io.IOException: Unable to tunnel through proxy. Proxy returns "HTTP/1.1 407 Proxy Authentication Required"
        at java.base/sun.net.www.protocol.http.HttpURLConnection.doTunneling0(HttpURLConnection.java:2271)
        at java.base/sun.net.www.protocol.http.HttpURLConnection.doTunneling(HttpURLConnection.java:2143)
        at java.base/sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185)
        at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:141)
        at org.owasp.dependencycheck.utils.HttpResourceConnection.obtainConnection(HttpResourceConnection.java:206)
        ... 11 common frames omitted
[WARN] Failed to update hosted suppressions file, results may contain false positives already resolved by the DependencyCheck project
org.owasp.dependencycheck.data.update.exception.UpdateException: Failed to update the hosted suppressions file
        at org.owasp.dependencycheck.data.update.HostedSuppressionsDataSource.fetchHostedSuppressions(HostedSuppressionsDataSource.java:156)
        at org.owasp.dependencycheck.data.update.HostedSuppressionsDataSource.update(HostedSuppressionsDataSource.java:87)
        at org.owasp.dependencycheck.Engine.doUpdates(Engine.java:906)
        at org.owasp.dependencycheck.Engine.initializeAndUpdateDatabase(Engine.java:711)
        at org.owasp.dependencycheck.Engine.analyzeDependencies(Engine.java:637)
        at org.owasp.dependencycheck.App.runScan(App.java:262)
        at org.owasp.dependencycheck.App.run(App.java:194)
        at org.owasp.dependencycheck.App.main(App.java:89)
Caused by: org.owasp.dependencycheck.utils.DownloadFailedException: Download failed, unable to copy 'https://jeremylong.github.io/DependencyCheck/suppressions/publishedSuppressions.xml' to 'C:\actions-runner\_work\NSDT\dependency-check\data\publishedSuppressions.xml'; Error downloading file https://jeremylong.github.io/DependencyCheck/suppressions/publishedSuppressions.xml; unable to connect.
        at org.owasp.dependencycheck.utils.Downloader.fetchFile(Downloader.java:152)
        at org.owasp.dependencycheck.utils.Downloader.fetchFile(Downloader.java:83)
        at org.owasp.dependencycheck.data.update.HostedSuppressionsDataSource.fetchHostedSuppressions(HostedSuppressionsDataSource.java:154)
        ... 7 common frames omitted
Caused by: org.owasp.dependencycheck.utils.DownloadFailedException: Error downloading file https://jeremylong.github.io/DependencyCheck/suppressions/publishedSuppressions.xml; unable to connect.
        at org.owasp.dependencycheck.utils.HttpResourceConnection.obtainConnection(HttpResourceConnection.java:267)
        at org.owasp.dependencycheck.utils.HttpResourceConnection.fetch(HttpResourceConnection.java:163)
        at org.owasp.dependencycheck.utils.Downloader.fetchFile(Downloader.java:138)
        ... 9 common frames omitted
Caused by: java.io.IOException: Unable to tunnel through proxy. Proxy returns "HTTP/1.1 407 Proxy Authentication Required"
        at java.base/sun.net.www.protocol.http.HttpURLConnection.doTunneling0(HttpURLConnection.java:2271)
        at java.base/sun.net.www.protocol.http.HttpURLConnection.doTunneling(HttpURLConnection.java:2143)
        at java.base/sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185)
        at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:141)
        at org.owasp.dependencycheck.utils.HttpResourceConnection.obtainConnection(HttpResourceConnection.java:206)
        ... 11 common frames omitted
[INFO] Updating CISA Known Exploited Vulnerability list: https://www.cisa.gov/sites/default/files/feeds/known_exploited_vulnerabilities.json
[ERROR] org.owasp.dependencycheck.utils.DownloadFailedException: Error downloading file https://www.cisa.gov/sites/default/files/feeds/known_exploited_vulnerabilities.json; unable to connect.
org.owasp.dependencycheck.data.update.exception.UpdateException: org.owasp.dependencycheck.utils.DownloadFailedException: Error downloading file https://www.cisa.gov/sites/default/files/feeds/known_exploited_vulnerabilities.json; unable to connect.
        at org.owasp.dependencycheck.data.update.KnownExploitedDataSource.update(KnownExploitedDataSource.java:93)
        at org.owasp.dependencycheck.Engine.doUpdates(Engine.java:906)
        at org.owasp.dependencycheck.Engine.initializeAndUpdateDatabase(Engine.java:711)
        at org.owasp.dependencycheck.Engine.analyzeDependencies(Engine.java:637)
        at org.owasp.dependencycheck.App.runScan(App.java:262)
        at org.owasp.dependencycheck.App.run(App.java:194)
        at org.owasp.dependencycheck.App.main(App.java:89)
Caused by: org.owasp.dependencycheck.utils.DownloadFailedException: Error downloading file https://www.cisa.gov/sites/default/files/feeds/known_exploited_vulnerabilities.json; unable to connect.
        at org.owasp.dependencycheck.utils.HttpResourceConnection.obtainConnection(HttpResourceConnection.java:267)
        at org.owasp.dependencycheck.utils.HttpResourceConnection.fetch(HttpResourceConnection.java:163)
        at org.owasp.dependencycheck.data.update.KnownExploitedDataSource.update(KnownExploitedDataSource.java:80)
        ... 6 common frames omitted
Caused by: java.io.IOException: Unable to tunnel through proxy. Proxy returns "HTTP/1.1 407 Proxy Authentication Required"
        at java.base/sun.net.www.protocol.http.HttpURLConnection.doTunneling0(HttpURLConnection.java:2271)
        at java.base/sun.net.www.protocol.http.HttpURLConnection.doTunneling(HttpURLConnection.java:2143)
        at java.base/sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185)
        at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:141)
        at org.owasp.dependencycheck.utils.HttpResourceConnection.obtainConnection(HttpResourceConnection.java:206)
        ... 8 common frames omitted
[WARN] Unable to update 1 or more Cached Web DataSource, using local data instead. Results may not include recent vulnerabilities.
[ERROR] Unable to continue dependency-check analysis.
[ERROR] One or more fatal errors occurred
[ERROR] org.owasp.dependencycheck.utils.DownloadFailedException: Error downloading file https://www.cisa.gov/sites/default/files/feeds/known_exploited_vulnerabilities.json; unable to connect.
[ERROR] No documents exist

To Reproduce
Steps to reproduce the behavior:

Run dependency check with proxy parameters provided through JAVA_TOOLS_OPTIONS
Wait for the NVD to finish
See error

Expected behavior
The process should download additional resources without error using the same proxy config.
Additional context
I have successfully downloaded the files manually, so it is not the proxy that blocks these specific URLs

The correct env var is JAVA_TOOL_OPTIONS not JAVA_TOOLS_OPTIONS. Are you sure you've configured it correctly?
Your log above shows the below, so it never tried to call NVD.
[INFO] Checking for updates
[INFO] Skipping the NVD API Update as it was completed within the last 240 minutes

Well the error implies it is talking to your proxy, hence getting Proxy returns "HTTP/1.1 407 Proxy Authentication Required" so folks would probably need to see the specific arguments you are sending (redacted) and how you are sending them (Gradle? Maven? Standalone? did it used to work but no longer does? can you force it to update NVD and show a log which shows NVD working but other data sources not?).
Depending on your proxy configuration, there are many reasons this could happen, e.g the proxy happens to whitelist or allow through the NVD API host, but not cisa.gov or github.io etc. Would need to see the args and a more complete log showing the NVD API working to tell.

What is the specific value of JAVA_TOOL_OPTIONS you are using and how are you exposing it to the CLI? What commands are you using? How can someone else replicate your problem reliably without having to guess what you are doing and replicate your precise proxy or runtime environment (which is not disclosed)?
While you may be right that a core proxy functionality is fundamentally broken, given the very wide user base of ODC it seems more likely that you’re doing something wrong, there is something specific to your environment that is wrong - or the documentation is misleading.
If you don’t explain precisely what you’re doing or include logs that show what you say is happening (NVD API working), we can’t easily get anywhere.

Does this work for you (of course updating the proxy?
curl --proxy "http://user:pwd@<IP_ADDRESS>:1234" "https://raw.githubusercontent.com/Retirejs/retire.js/master/repository/jsrepository.json"

As I already stated before, the NVD downloads went fine with these settings. If I drop the --disableRetireJS or --disableKnownExploited option then I would see the error log entries from my first post.

Yes, you stated/told, but you didn't show via logs. I wouldn't ask to see if I didn't think it was useful to eliminate problems or assumptions about how ODC works, this is why I asked:

can you force it to update NVD and show a log which shows NVD working but other data sources not?).

Anyway, good luck to you.

...I had to switch to another proxy url to make it work with curl

Sounds like an environment issue? possibly one of the proxies blocks access to some resources? I know some users have had to work with the networking team to allow the connections on the proxy.

You can also host these files inside your network - see https://jeremylong.github.io/DependencyCheck/data/index.html

Yes, correctly setting the JAVA_TOOL_OPTIONS should be used for all requests. The legacy options were left there to make it easier for upgrades.

Could it also be that your network team has different rules for nvd.nist.gov (seen by them as a trusted source; anonymous proxy access allowed) versus github (seen by them as a risky site, so people should authenticate on the proxy (and maybe even be part of certain user-groups) before being granted access to information stored there)
After all it's not uncommon for enterprise proxies to have various categories of sites with different access grants from the internal network.

Managed to reproduce locally with a customized squid-proxy docker container at localhost. I'll dive deeper into it.

Authenticating proxy even fails when using the legacy configuration parameters. Only when I force the setting for jdk.http.auth.tunneling.disabledSchemes early on (by making it part of settings initialisation) the legacy parameters work. The JAVA_TOOL_OPTIONS does not work for those cases still, which is logical, as the https.proxyPassword and https.proxyUser are system properties that are not consulted by the JVM (fully in line with the documentation of the proxy properties on ), they appear to be useful extensions to the set by Apache HTTPClient.
@jeremylong I propose to try and swap out our custom HTTP client code customizing the standard JVM classes for the various resource downloads by the Apache HTTPClient. Need to see how that is best done, but expect I would be able to get that done somewhere this week; would be a good additional candidate for the 11.0.0 release I think

100% agree. I've been using the HTTPClient in the open-vulnerability-client. If you can work on this - that would be great.

Think I was a bit too optimistic regarding the amount of work to get rid of all others and consistently use HTTPClient within a weeks time. Nevertheless things are moving forward piece by piece.


[CI] Add pytest-testmon to reduce testing time

Problem (Why?)
only to run needed tests, not the whole cases.
Solution (What/How?)

[x] add pytest-testmon

Other changes (bug fixes, small refactors)
nope
Notes
nope

Codecov Report

Merging #87 (dcf07e7) into main (534a0f4) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main      #87   +/-   ##
=======================================
  Coverage   96.30%   96.30%
=======================================
  Files          31       31
  Lines        2218     2218
=======================================
  Hits         2136     2136
  Misses         82       82

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


High level enemies with Relic hunter start

When starting a new game and selecting the Guild master starting scenario, the skeletons in the starter ruin are lvl 18 rather than lvl 1 as they should be.
Installed through Wabbajack and only extra mod installed outside of mod pack is Skyrim save system overhaul

It will be a while before this will be fixed.


Slider has issues with Start/From parameters

Example: Start=6, From=9
The slider allows selecting a range below the Start point going to 0.
Also, the left offset for the lower range selector is not pointing to the right place.

@ShotaMegre The from/to properties is a visual properties only for positioning the handlebars on the slider. If you want to specify the beginning/ending values, you should use start/end properties instead.

Thanks for the reply, though I have to clarify.
If you have set Start=6 (for example, any number greater than 0), the slider allows you to pull the lower range selector to 0.
Therefore it's a bug. See the screenshot.

Thanks for the clarification. I will fix it soon. @ShotaMegre

@MuhammadMabrouk i see bug not fix ? Do you have solution ?


Camera navigation via keyboard missing input events with tilemaps

Godot version:
3.3
OS/device including version:
AMD Ryzen 3600
5700 XT
Arch Linux 5.11.16-arch1-1
Issue description:
With Godot 3.2.3, the tilemaps I'd set up performed poorly when zoomed out.  Now, with 3.3, it's  smooth which is great, but now it seems to be missing input events occasionally.
Most of the time using WASD to pan the camera works fine, but sometimes it keeps moving in a direction once I release key press, and sometimes it doesn't start moving immediately.  Outputting the events, it looks like the key press (or key release) isn't registered.  I don't recall noticing this in 3.2.3, but I did not test extensively.
Steps to reproduce:
Load and run the attached test project, full screen, zoomed out (should be by default), and try panning around with WASD.  Hopefully you observe that some inputs are being skipped -- that the panning isn't always moving exactly in line with the keys you're pressing
I tested also on a Windows machine with a weaker CPU and GPU, and the issue doesn't seem to be (as) apparent.
Minimal reproduction project:
tilemap_camera.zip

Can you reproduce this if you call Input.set_use_accumulated_input(false) in the main scene script's _ready() function?

I noticed I'm using _physics_process instead of _process for camera inputs.  Tested these permutations:

_phsyics_process & accumulate=default
_physics_process & accumulated=false
_process & accumulate=default
_process & accumulate=false

While the problem is evident using all, it seemed a bit harder to trigger with Input.set_use_accumulated_input(false).  This isn't a rigorous experiment though so it may also be making no difference.

I should add, most of the time (most inputs) it doesn't happen, but happens often enough for me to notice.

I've had a very similar issue after updating to Godot 3.3, also on Arch Linux. I fixed it by disabling vsync in Project Settings. Let me know if it helps.

I have no issue with 3.2.3, only with 3.3. Also, it seems to be input in general, not tied to the camera.
Here's my reproduction project:
test.zip
Try holding down right, then repeatedly tapping left. Sometimes the input isn't registered immediately.
After adding Input.set_use_accumulated_input(false), it seems to help, but doesn't fix it entirely. Only disabling vsync solves it. This is standalone desktop executable, no issue with HTML5 export.

@jusw85 Can you test this on all older 3.3 RCs and betas to determine when the regression started? You can download them here and here (3.2.4 was renamed to 3.3 during the development process).

@jusw85 Can you test this on all older 3.3 RCs and betas to determine when the regression started? You can download them here and here (3.2.4 was renamed to 3.3 during the development process).

Issue appears in beta 1 Godot_v3.2.4-beta1_x11.64.zip
No issue with 3.2.2 Godot_v3.2.2-stable_x11.64.zip

Just thought I'd mention that I've just tried 3.3.2 and the issue is still present.

In my opinion the title should be changed, because I have had a frustrating time with inputs in general, like @jusw85, and I think this is all related.
Under circumstances I cannot determine, making it difficult to reproduce, Godot may ignore inputs, period. With how my project works, this sometimes causes analog-input-based values to become stuck, which is game-breaking. It may be fixed simply by closing and restarting the game. I think Godot might also be reporting duplicate "is_action_just_pressed" events, but I'm not sure yet.
I've had enough and I am going to try disabling Vsync and capping the framerate at 144 to see if it helps ("debug/settings/fps/force_fps" is not only for debug mode, is it?). If that doesn't work, I intend to try going back to Godot 3.2.2, because the symptoms are unacceptable. :|

("debug/settings/fps/force_fps" is not only for debug mode, is it?)

It works in exported projects too.

It works in exported projects too.

Good to know in general, thanks.
I will spend some time with 3.2.2, 3.2.3, and 3.2.4 to see if I can corroborate @jusw85's observation.

bisect from 3.2.3-stable to 3.3-stable.
This is the commit which introduces the problem: f725d9cb733e103c91d889777b78c5a646095658
To clarify, I'm not missing any inputs. All my inputs are registered. Instead, my (keyboard) inputs are delayed and experiencing input lag. It doesn't happen consistently. Sometimes its very noticeable, other times barely noticeable. Disabling vsync in the project fixes the issue for me. I haven't tried other inputs such as mouse clicks. I estimate the input lag to be around 100ms - 500ms (definitely <1second)
I'm using Openbox on Arch Linux with the picom compositor with vsync enabled (on the compositor)

I have booted my project many times in 3.2.3 stable now, and I cannot reproduce the symptom I've reported for 3.2.4 beta 1 and 3.3.2 stable. I have also tried turning off Vsync, which does not prevent the symptom. Turning off "Always on Top" has no effect either, so I don't think it's related to #48312.
Unfortunately I still have not been able to find a clue on what circumstances are causing the problem in the first place. When it happens, sometimes it can be repeated for a period of time each time the game is restarted, while sometimes restarting the game may fix it. When it isn't happening, I can't predict how many times I could restart the game before the problem appears again.
I've tried closing other programs when it happens, and I've tried deliberately acquiring keyboard focus in another window (like right here) to make it appear. Neither works.

For anyone who an reproduce the issue, could you please test with #49720 to see if it helps?
There's a mutex to protect multi-threaded operations for events which I've tried to switch to a spin lock to see if it can avoid some random stalling.

For anyone who can reproduce the issue, could you please test with #49720 to see if it helps?
There's a mutex to protect multi-threaded operations for events which I've tried to switch to a spin lock to see if it can avoid some random stalling.

Doesn't help.
I simulated keypresses using xdotool to demonstrate the issue. It repeats right then left with 0.1s interval.
sleep 1; for i in {0..20}; do xdotool keydown Right; sleep 0.1; xdotool keyup Right; xdotool keydown Left; sleep 0.1; xdotool keyup Left; done

3.2.3
https://user-images.githubusercontent.com/3406652/122643592-2f35c880-d143-11eb-8ff9-997c15037de0.mp4
#49720, 3.3.*
https://user-images.githubusercontent.com/3406652/122643644-80de5300-d143-11eb-84de-2210f84b4bcc.mp4
https://user-images.githubusercontent.com/3406652/122643645-820f8000-d143-11eb-8f40-23f372d33787.mp4

That excellent reproduction from @jusw85 is exactly what I experience with 3.3.2 on Ubuntu Desktop.
Despite all the improvements in 3.3, reliable input is a critical component for any game, so I'll have to stick with 3.2.3 for now.

I did some testing on this issue and I've discovered something that might be of interest when running games with v-sync enabled in Godot 3.3.
When using Linux Mint 20.2, the stated input lag issue is very noticeable and the inputs will randomly delay themselves for almost up to a full second seemingly at random intervals. Regular gameplay becomes much harder, if not totally impossible in some cases because of this.
So I tried using Debian 10.10 (non-free) and found that the issue is still present, but it is much less invasive. It only really becomes noticeable when using the terminal command provided by @jusw85 a couple of times when running on Debian.
Both distros were run on the same machine, but I think it might be best if someone else could also confirm this discrepancy between the two distros on their own computer.

I'm using godot 3.3.3 on ubuntu 21.04 X64 and I have both the default gnome environment and LXDE (openbox) installed.
Interestingly with me the vsync workaround is the oposite: when vsync is turned off I get the input delay, while with vsync turned on input works without problems. This happens both on gdm with xorg (gnome) and openbox with xorg (lxde). I havent tested with wayland. Input.set_use_accumulated_input(false) doesn't seem to make any difference.
I'm testing only keyboard input and I would like to notice that the problem seems to happen most commonly on the keyup events. The keydown events doesnt seem to have any problem, but the keyup events several times have a big delay of about one second between the actual mechanical release of the key and the engine noticing it.
The bug can be tested creating an empty scene, attaching the following script and running it on the editor:
extends Object
func _input(event):
	print(event.as_text())
func _physics_process(delta):
	print(Input.is_action_pressed('ui_right'))

While the scene is running, keep your eyes on the console (preferably a dedicated terminal window, not the built-in console) and do very quick key taps of the 'right arrow' key with a small interval of about half second between each keypress. One will note that sometimes the keydown and keyup events happen right after one other or with a very few number of frames between. But sometimes, the keyup will delayed by a large amount, like one second.

Thanks for all the feedback and test scripts, it's going to be very useful! I'm sorry I didn't look into this issue in the past couple months, but I'll try to find some time soon to see if I can figure out what's going on.

I've recently faced the same problem. In my case it happens only when gamepad is connected. Otherwise I don't face this problem. And v-sync has nothing to do for me.
godot 3.3.2 windows 11

In my case it happens only when gamepad is connected. Otherwise I don't face this problem
Do you mean that, when a gamepad is connected, the problem occurs with gamepad input only? or with any other kind of key input, like if you connect the gamepad and start experiencing the problem also on keyboard input?

I've recently faced the same problem. In my case it happens only when gamepad is connected. Otherwise I don't face this problem. And v-sync has nothing to do for me.
godot 3.3.2 windows 11

Given that it's a different input altogether, vsync doesn't impact it, and it's on a wholly different operating system, I would be surprised if this were indeed the same issue.

@jusw85 I'm able to reproduce the issue using your script, but for me it happens in both 3.2.3 and 3.3.2 whether vsync is on or off. I'm on ArchLinux / Gnome.
Is it always perfect on 3.2.3 like in the video for you, or do you experience similar issues,  just more consistently in 3.3.2? Does it happen in 3.2.3 under any circumstances?

@pouleyKetchoupp
I tested again using 3.2.3 from here and 3.3 from here. I'm on Arch / Openbox + picom (vsync on)
On 3.3, the issue almost always occurs (with vsync on). Sometimes it's less noticeable, sometimes more. This is one of the more noticeable ones. Note the occasional responsiveness issues with the input.
https://user-images.githubusercontent.com/3406652/132810138-adb1e788-13d1-4085-913f-34eb75b5fa18.mp4
On 3.2.3, I don't notice any problems. However, because I added a marker for the midpoint, I observed that sometimes it drifts away from the start (although sometimes it does). This however might be due to timing precision in sleep 0.1 and having an extra frame of right down or 1 frame delay of right up.
https://user-images.githubusercontent.com/3406652/132810600-0fd0ee13-bd18-4788-8a21-e5a3c3f74604.mp4

Just encountered this bug as well. It was definitely due to a controller being plugged in. I had a PS4 controller plugged in, and the arrow keys would not report correctly when being held down.

Fixed for 3.4 by #54313.


Add loading of reg status and derivation mode vocabs on startup to SPARQL store

This PR loads the Registry Status vocabulary and the Vocab Derivation Modes vocabulary on startup to the SPARQL store.
Each of the vocabs are loaded into their own system graph:

urn:system:reg-status
urn:system:vocab-derivation-modes

Usage just like all other loaded data assumes the SPARQL store has default union turned on.
The VocPub system profile has been updated to include sdo:color in API responses. This colour value is used to colour the status marker in the vocab listings.
Some other bits:

Add tbox_cache.nt to .gitignore
This PR includes https://github.com/RDFLib/prez/pull/107 as the fix there is needed here

Will merge this only once https://github.com/RDFLib/prez/pull/107 is merged as it depends on it.


feat: Support spilling for hash aggregation

Which issue does this PR close?
Closes https://github.com/apache/arrow-datafusion/issues/1570
Rationale for this change
For hash aggregation, the sizes of group values and accumulators can become large that may cause out of memory.
What changes are included in this PR?
This PR lets the hash aggregation operator spill large data to local disk using Arrow IPC format. For every input RecordBatch, the memory manager checks whether the new input size meets the memory configuration. If not, spilling happens, and later stream-merge sorts the spilled data to read back. The stream merge sort function was reused from the one in sort operator.
Are these changes tested?
Yes
Are there any user-facing changes?
No

Waiting for https://github.com/apache/arrow-datafusion/pull/7399 for the fmt issues

Thank you @kazuyukitanimura  -- I plan to review this PR tomorrow. I am very excited!

Just FYI, I will be traveling entire next week. I may not be able to respond in time, but I plan to address comments the week after (9/5~)
Thank you in advance.
cc @sunchao @viirya

Thank you @alamb for reviewing. I plan to work on addressing comments a week after next (9/5~) once I am back from my travel.

Performance tests showing it doesn't slow down performance (I don't expect it to and I could help with this)

Regarding the benchmark, what would be the best way to proceed? I read https://github.com/apache/arrow-datafusion/blob/main/benchmarks/README.md
Is bench.sh run tpch a good one for this PR purpose? I am wondering if there is a github action / CI so that we can use the same machine for benchmarking...

Is bench.sh run tpch a good one for this PR purpose? I am wondering if there is a github action / CI so that we can use the same machine for benchmarking...

This is what I recommend.
I have some scripts in https://github.com/alamb/datafusion-benchmarking that I used to compare a branch to main, but they aren't super user friendly at the moment.  I'll run this branch on a gcp machine and report back
Thanks again

Marking as draft to signify this PR has gotten feedback and is waiting to incorporate it before subsequent review

Thank you @yjshen @alamb for reviewing. I addressed all comments. I would appreciate it if you could take another look. Thanks again.

Thanks @kazuyukitanimura  -- I plan to review this PR tomorrow

Thank you @alamb for your feedbacks. I addressed them just now.

The only last thing I want to do prior to approving this PR is to run some benchmarks on this PR to ensure it doesn't lose performance. Otherwise it is ready to go from my perspective

Based on the earlier conversation above, I assume that you can help on this?

Based on the earlier conversation above, I assume that you can help on this?

yes, absolutely. I am in the process of running them and will report back shortly

@yjshen  would you like to review this PR again prior to merging?

QQuery 15

What's the reason behind QQuery 15 +2.36x faster?

On a M2 Max Mac:
--------------------
Benchmark tpch.json
--------------------

┏━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┓
┃ Query        ┃     main ┃ agg-spill-pr ┃       Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━┩
│ QQuery 1     │ 373.61ms │     379.52ms │    no change │
│ QQuery 2     │  54.07ms │      71.85ms │ 1.33x slower │
│ QQuery 3     │ 127.28ms │     125.39ms │    no change │
│ QQuery 4     │  77.72ms │      77.59ms │    no change │
│ QQuery 5     │ 153.95ms │     155.71ms │    no change │
│ QQuery 6     │  74.79ms │      75.29ms │    no change │
│ QQuery 7     │ 215.22ms │     217.05ms │    no change │
│ QQuery 8     │ 181.37ms │     182.10ms │    no change │
│ QQuery 9     │ 255.84ms │     254.07ms │    no change │
│ QQuery 10    │ 201.64ms │     204.14ms │    no change │
│ QQuery 11    │  50.53ms │      51.22ms │    no change │
│ QQuery 12    │ 136.34ms │     133.59ms │    no change │
│ QQuery 13    │ 209.86ms │     213.03ms │    no change │
│ QQuery 14    │ 107.24ms │     110.68ms │    no change │
│ QQuery 15    │  77.42ms │      78.31ms │    no change │
│ QQuery 16    │  46.33ms │      46.21ms │    no change │
│ QQuery 17    │ 165.80ms │     167.61ms │    no change │
│ QQuery 18    │ 308.12ms │     299.10ms │    no change │
│ QQuery 19    │ 206.49ms │     207.36ms │    no change │
│ QQuery 20    │ 120.51ms │     122.26ms │    no change │
│ QQuery 21    │ 319.92ms │     325.50ms │    no change │
│ QQuery 22    │  47.27ms │      46.02ms │    no change │
└──────────────┴──────────┴──────────────┴──────────────┘

--------------------
Benchmark tpch_mem.json
--------------------

┏━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┓
┃ Query        ┃     main ┃ agg-spill-pr ┃       Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━┩
│ QQuery 1     │ 249.33ms │     249.81ms │    no change │
│ QQuery 2     │  34.74ms │      43.41ms │ 1.25x slower │
│ QQuery 3     │  45.14ms │      45.13ms │    no change │
│ QQuery 4     │  35.32ms │      35.42ms │    no change │
│ QQuery 5     │  90.31ms │      89.50ms │    no change │
│ QQuery 6     │   9.60ms │       9.50ms │    no change │
│ QQuery 7     │ 172.73ms │     178.81ms │    no change │
│ QQuery 8     │  69.94ms │      68.29ms │    no change │
│ QQuery 9     │ 131.35ms │     126.52ms │    no change │
│ QQuery 10    │  84.17ms │      86.08ms │    no change │
│ QQuery 11    │  37.16ms │      36.25ms │    no change │
│ QQuery 12    │  57.98ms │      57.18ms │    no change │
│ QQuery 13    │  93.57ms │      92.33ms │    no change │
│ QQuery 14    │  11.52ms │      11.62ms │    no change │
│ QQuery 15    │  13.57ms │      14.29ms │ 1.05x slower │
│ QQuery 16    │  36.73ms │      38.23ms │    no change │
│ QQuery 17    │  74.55ms │      74.09ms │    no change │
│ QQuery 18    │ 248.72ms │     242.23ms │    no change │
│ QQuery 19    │  50.12ms │      50.65ms │    no change │
│ QQuery 20    │  73.09ms │      73.83ms │    no change │
│ QQuery 21    │ 240.83ms │     231.30ms │    no change │
│ QQuery 22    │  27.21ms │      27.18ms │    no change │
└──────────────┴──────────┴──────────────┴──────────────┘

Thank you for all the benchmarks! Glad to see there is no degradation.

Do these benchmarks test the case of spilling?
It would be very nice if we can do some tests like in this PR: duckdb/duckdb#7931, which measures how performance may degrade as memory budget becomes lower.

Thanks, though the intent of the benchmarks were to make sure this PR does not depredate, I agree that it is nice to to measure the performance under different memory pressure. Since this is the first PR for spilling, how about we do it for the future PRs?

Thank. Though the intent of the benchmarks were to make sure this PR does not depredate, I agree that it is nice to to measure the performance under different memory pressure. Since this is the first PR for spilling, how about we do it for the future PRs?

Sure, SGTM. Thanks @kazuyukitanimura for the work !

Thank you all for the reviews! I think addressed all of them. @alamb @yjshen @sunchao @viirya
There are some TODOs I plan to follow up in the future PRs. For any further improvements on this PR, I plan to tack on the next PR.

Thank you all for the reviews! I think addressed all of them. @alamb @yjshen @sunchao @viirya
There are some TODOs I plan to follow up in the future PRs. For any further improvements on this PR, I plan to tack on the next PR.

Thank you @kazuyukitanimura  - this is a really nice piece of technology, as well as a nice example of collaboration. I agree let's merge this and continue working on main / follow on PRs.
I filed https://github.com/apache/arrow-datafusion/issues/7571 to track adding spilling group benchmarks

QQuery 15

What's the reason behind QQuery 15 +2.36x faster?

@Dandandan  I believe the reason is that Q15 is so fast (15 ms!)  it is susceptible to small pturbations, and the difference is largely a measurement error:

│ QQuery 15    │  13.57ms │      14.29ms │ 1.05x slower │

Great, thank you all again.


The --only-deps flag can sometimes install too much

When specifying
mamba create -c conda-forge -c bioconda -n snakemake snakemake-minimal snakemake --only-deps

Mamba wants to install snakemake-minimal as well (because it is a dependency of the snakemake package). However, correct behavior would be not neither install snakemake nor snakemake-minimal. Conda does this correctly.

thanks for this bug report -- and the others!

I have a fix in #256
However this works slightly different from conda. It will hard ignore all specs that are in the current command when using --only-deps.
Conda has some logic to only remove the leaf packages (which is the "bug" you encountered). To be honest we were using conda's logic here and I agree it seems flawed.
The usecase they are saying is that you can use another package to constrain the leaf package, e.g.
conda create -n bla python=2 flask --only-deps
would use python=2 to constrain the flask package. However, I think it's better to do these kind of things in a two step process or have another flag for constraining packages.
Here is the conda-source for the only-deps feature. Judging by the comment conda should also install snakemake-minimal:
https://github.com/conda/conda/blob/2cdd7ab052c2b96c088137f52dc40c613e99ce33/conda/core/solve.py#L866-L894

anyways this should be fixed with #256
however we still need to add back a way to specify constraints / pins when creating a new environment.


Form - Doesn't validate, instead refreshes page

Version
@nuxt/ui: 2.7.0 and edge
nuxt: 3.7.1
Reproduction URL
You don't see the page refreshing here, but the same problem sort of: no errors. While in the example it does show errors on submit if nothing is filled in.
https://stackblitz.com/edit/nuxt-ui-3tkvck
Steps to reproduce
Copy and paste the example, first one here: https://ui.nuxt.com/dev/forms/form#usage. This could be a component which is used in a page. Or you can copy paste the example directly to a page.
What is Expected?
Two validation errors show, that the fields are required.
What is actually happening?
The page refreshes, no errors are shown.
Browser
Safari + Brave
Package manager
Bun

Did you try the latest v3.8.x release? The Form component has been greatly improved since 2.7.0. You will have to change your package to @nuxt/ui to get the latest updates.

@benjamincanac Works on 2.8.1! Thanks 🙇


Setting custom port in "deply/environment.rb" doesn't work

It always connects me to port 22 even if I set the port other than 22.
# For example
set :ssh_options, {
  port: "xxx",
  user: "apps"
}

Hi,
I made some changes. Can you please try the current master?


[NCL-7230] Implement Product Detail page

Hi guys @patrikk0123 and @DnsZhou ,
components/ProductDetailPage/ProductDetailPage.tsx contains bug, would be able to found it? :)


Add Fast-Refresh for Web

[Semver-Status] minor

Important
This PR is based on of the keg-proxy-trafik PR
So this can't be merged in until the PR is merged in
So if you have not reviewed that PR yet, go review that one first

Context

Added webpack fast-refresh
Allows hot-swapping component files without a page reload
Keeps state after code changes in some cases

Goal

Improve developer experience
Keep state between code changes
Remove the need for constant browser refreshing

Updates

repos/keg-core/babel.config.js

Updated to cache the babel config based on env

repos/keg-core/core/configs/webpack.config.js

Added react refresh plugin

repos/keg-core/package.json

Added react refresh plugin to dependencies

Testing
Test 1

This test is just to show docker packages still work with the update

It does not test the added functionality

Run the docker package url => keg evf pack run docker.pkg.github.com/simpleviewinc/keg-packages/tap:add-fast-refresh

Ensure the app works as expected

Test 2

Check out the develop branch for the events-force tap

keg evf && keg br develop

Then pull this PR => keg && keg pr 64
Next, start the events force tap, with a sync for core & components

keg evf start --sync core:install,components:install:start

This command will start the events force tap, sync, install dependencies for core, and sync, install, run rollup for components

Once this command completes, navigate your browser to evf-develop.local.kegdev.xyz

Ensure the app works as expected in the browser

Now open the code for events force tap, and make a change to one of the components and save the change
Ensure it's a component that currently being used on the page ( evf-button is a good one )
In the browser you should see the change without the tab refreshing

In some cases the browser WILL REFRESH

It really depends on the change that was made, and if it can be hot-loaded into the app
If the page does refresh, try making a small change like console.log in a functional component
This typically works with hot-mounting

Next make a change in the keg-components

The same rules above, apply here.
The change should be reflected on the page without a tab refresh
There are cases where the page still does a full refresh, but they should be minimal

Test 3

In another terminal window add a sync for retheme

keg evf sync retheme:install:run

Make a change within the retheme repo, and save it

The same rules above, apply here.
The change should be reflected on the page without a tab refresh
There are cases where the page still does a full refresh, but they should be minimal

Closing for now. May re-open when the plugin becomes more mature


[NWPS-1325] Reverting Padding to 24px, Mourning banner background to …

@AlexKirstenZA @smanivasag
Following changes agreed with Andy.

Update banners padding top & bottom to 24px ( excluding phase)
Mourning banner background colour should set to #000000

@sankar-shunmuga @smanivasag sure no problem, I'll action the changes now :+1:

@sankar-shunmuga @smanivasag nevermind, seems Selva beat me to it :sweat_smile:
Looks good on my end, happy to merge this.


check whether two strings are anagrams or not using JAVA

@all-contributors please add @meghnabajoria  for code, ideas

Beutifully done

Thank you @ephremdeme :) Also, will this contribution count towards Hacktober fest?


dracula-theme: add gtk-4.0 theme

Description of changes
with more and more apps using gtk-4.0 it makes sense to include the gtk-4.0 theme in $out

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[x] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
22.05 Release Notes (or backporting 21.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

any suggestions for a reviewer with merge access?


Add @posthog/plugin-globals (0.0.1)

Changes

Trying out the @posthog/plugin-globals package. No matter what we'll do with meta, we still have globals and will probably want some things to remain globals. This adds support for them.

The plugin-globals project needs to be extended with other globals. Currently only posthog is there.

Checklist

[ ] Tests for new code

Thanks for the contribution!
I personally am okay with writing things like that. I am concerned however that anyone who does not work at PostHog may find the increased cognitive load of additional tsconfig and one more package discouraging from writing plugins with TypeScript or in general. It's also more things to maintain for us. That's why I'd really really like all te definitions to be in the single plugin-scaffold package.

Fair enough re: keeping things simpler. It's possible to do the same thing with the @posthog/plugin-scaffold package: https://github.com/PostHog/plugin-server/pull/274#issuecomment-805967417 -- we'll just have to import it from a different source in posthog & plugin-server to not get the globals, but that shouldn't be too much work, considering it's a mostly one-off affair (a few times).


ROW: clean up in preparation to hide CharRow & AttrRow

Moving things out of CharRow into ROW helps us hide it as an implementation detail.
This is part one of many.
CharRow: Hide ClearCell, use ROW::ClearColumn
CharRow: Hide GetText, use ROW::GetText
CharRowBaseTests: remove dead file (never used!)
CharRow: Move DoubleBytePadded into ROW
CharRow: Move WrapForced into ROW
Char/AttrRow: Hide Reset, use ROW::Reset
Remove RowCellIterator (dead code)
RCI was unused; it was replaced by TextBufferCellIterator shortly after its creation
Move AttrRowTests to ut_textbuffer from ut_host
It had no reliance on the host.

@msftbot make sure @miniksa signs off on this

/azp run

/azp run


Update external dependencies to silence warnings at install

@timleslie haha was writing a comment about the eslint errors, and before I finished you pushed a change fixing them. nice 👍

Good call.

@timleslie I added some commits to fix the errant FilterTests case because that'll actually fail to be run on a case-sensitive FS. Good to go now 👌


Genders being swapped

When I run it shows correct palbox location and pals that I have but the genders are swapped. Male are Female and Female are Male.

Can you provide a sample save

Closing since no additional info has been provided


Fix Ignoring archive files after processing of internal archive

Fix #343

@adamhathcock when are you going to merge this PR?
@codeFather2 could you resolve merge conflicts?

This is really about non-compressed internal archives which is same as https://github.com/adamhathcock/sharpcompress/issues/86 I believe

in this case, the problem was that if there is a file in the child archive, the size of which without compression is longer than the remaining length of the parent archive, then after reading the header of this file, the Reader moved to the end of the parent archive, skipping subsequent files. This is due to the fact that when compressing the file header from the child archive, it was placed before the header of the child archive itself, and then it was duplicated inside. My corrections allowed me to bypass this error, without damage to the usual archives


Issue 208

#208 resolves logic issues in filter_type control flow

Looks like I accidentally swept a few other unresolved code changes into this one. I'll see if I can separate them, but it passes precondition tests.


[Feature Request] Seperation of Image based and text based subs

Allow the ability to set SRT subs to be embeded but pgs and image based subs to be always dumped so that they can later be OCR'd.
^ to avoid transcoding with image based subs

It would also need to be able to copy external srt files and embed them and leave any image based subs if it was grabbed or imported that way.

2891bf79703646c95203767409f19ad715014eca
Should be working, new separate embed-image-subs option. Both embed options are still dependent on some codec being specified

So this is working now, but It is no longer embedding external srt files
Here is an example of what im trying to do :

Mux the MP4 and srt and leave the image based subs, i want to make sure if i receive files that have been split apart they can be muxed back together leaving the image based subs.

autoProcess.txt

nvm this works fine, it was messing up because its case sensitive apparently and ENG > eng fixed it.
Sometimes i see people listing subs as .en.srt and .ENG.srt so maybe making it detect that would be helpful if those moments come along.

It already handles 2 character language codes, though the case sensitivity thing wasn't intended
91a2872bac31edfc1fce4f3375cd59984f3aca0d
Should be fixed that that patch

Excellent, Great work! Myself and others who use this greatly appreciate your hard work.


Add options to activate some commented entries from hosts zero

After reading the source file, I realized that some hosts are commented:
#<maybe-spy>
#<IP_ADDRESS> auto.search.msn.com  # Microsoft uses this server to redirect
                                # mistyped URLs to search engines. They
                                # log all such errors.
#<IP_ADDRESS> sitefinder.verisign.com	# Verisign has joined the game
#<IP_ADDRESS> sitefinder-idn.verisign.com	# of trying to hijack mistyped
					# URLs to their site.
					# May break iOS Game Center.

#<IP_ADDRESS> s0.2mdn.net		# This may interfere with some streaming
				# video on sites such as cbc.ca
#<IP_ADDRESS> ad.doubleclick.net   # This may interefere with www.sears.com
				# and potentially other sites.
<IP_ADDRESS> media.fastclick.net	# Likewise, this may interfere with some
<IP_ADDRESS> cdn.fastclick.net	# sites.
#<IP_ADDRESS> ebay.doubleclick.net		# may interfere with ebay
#<IP_ADDRESS> google-analytics.com			# breaks some sites
#<IP_ADDRESS> ssl.google-analytics.com
#<IP_ADDRESS> www.google-analytics.l.google.com
#<IP_ADDRESS> stat.livejournal.com		# There are reports that this may mess
 					# up CSS on livejournal
#<IP_ADDRESS> stats.surfaid.ihost.com	# This has been known cause
					# problems with NPR.org
#<IP_ADDRESS> www.google-analytics.com		# breaks some sites
#<IP_ADDRESS> ads.imeem.com		# Seems to interfere with the functioning of imeem.com
#</maybe-spy>

I wonder if it would be possible to activate/deactivate some ranges like that:
sudo adblock --with-range maybe-spy

I'm not sure about the syntax but hopefully you get the point :)

Thanks for the suggestion!
However, upon closer inspection, I find only two sections that are partially commented out: maybe-spy and maybe-ads, and each commented line has a specific reason why next to it. I surely wouldn't recommend blocking google-analytics.com, for example, seeing how many sites break when GA is blocked… much better to have a browser switch.
I don't think that option would be worth using for two sections only. Maybe a --aggressive option could uncomment everything, but I feel it's too much work for the usage. The whole point of adblock is, I believe, that you can forget about it most of the time. If you have to switch it off and on through the command-line three times a week, like I have to do with Ghostery at the moment, I think it's going to be too much of a pain.
If one wants to uncomment specific lines, these can be added manually in your standard hosts file, as it is merged with the blocker one  :)
I'll close that issue for the moment, but feel free to continue the discussion if you really believe it's important.


doc: genrest: Optionally use separate index page for all symbols

Add an option --separate-all-index that makes genrest.py generate a
separate index-all.rst index page that lists all symbols, instead of
listing all symbols in index.rst. index-all.rst is linked from
index.rst.
This was motivated by an external project where index.rst becomes the
top-level page, which runs into a Sphinx bottleneck with
sphinx_rtd_theme. See https://github.com/sphinx-doc/sphinx/issues/6909.
Also unclutter the generated documentation a bit by removing some
headings. This makes the navigation menu on the left nicer too.
Piggyback making genrest.py executable, which is handy when running it
manually.

Screenshot with --separate-all-index (though it's not passed in Zephyr):

Unrelated failure, in case it's persistent:
zephyr/CMakeFiles/offsets.dir/arch/arm/core/offsets/offsets.c.obj -L"/opt/sdk/zephyr-sdk-0.10.3/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/8.3.0/thumb/v6-m/nofp" -L/home/buildslave/src/github.com/zephyrproject-rtos/zephyr/sanity-out/bbc_microbit/tests/bluetooth/mesh/bluetooth.mesh.main.microbit/zephyr -lgcc -Wl,--print-memory-usage -mthumb -mcpu=cortex-m0 -Wl,--gc-sections -Wl,--build-id=none -Wl,--sort-common=descending -Wl,--sort-section=alignment -u_OffsetAbsSyms -u_ConfigAbsSyms -nostdlib -static -no-pie -Wl,-X -Wl,-N -Wl,--orphan-handling=warn -Wl,--fatal-warnings -mabi=aapcs -march=armv6s-m && :
Memory region         Used Size  Region Size  %age Used
           FLASH:       94217 B       256 KB     35.94%
            SRAM:       17672 B        16 KB    107.86%
        IDT_LIST:         120 B         2 KB      5.86/opt/sdk/zephyr-sdk-0.10.3/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/8.3.0/../../../../arm-zephyr-eabi/bin/ld: zephyr/zephyr_prebuilt.elf section `datas' will not fit in region `SRAM'
/opt/sdk/zephyr-sdk-0.10.3/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/8.3.0/../../../../arm-zephyr-eabi/bin/ld: section .intList VMA [0000000020004000,0000000020004077] overlaps section datas VMA [0000000020003d00,000000002000437b]
/opt/sdk/zephyr-sdk-0.10.3/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/8.3.0/../../../../arm-zephyr-eabi/bin/ld: region `SRAM' overflowed by 1288 bytes
collect2: error: ld returned 1 exit status
%
ninja: build stopped: subcommand failed.

I'm fine with a separate "all symbols" page for the Zephyr docs, particularly as you've pictured, and stop putting all the symbols on the main page.

Updated CMakeLists.txt to use the new option in Zephyr too. Agree that it turned out pretty nicec after removing the headings, as suggested by @ru-fu.


Game isn't selectable in the 'add game' dialog

I bisected it down to this commit: bdee41af2ce5cee9ec2004dfceaa37ec67af87d6

Fixed by #291, but I should add feature tests for the game modal before closing this.

Tests added in #292


SOLID POD: Improve forward & back navigation

Navigation in solidpod is implemented often by providing a child widget as an argument of the constructor of a widget, so that the user will see the child widget once the work of the current widget is done.
When there are more than one out-going routes in a widget, e.g. in the POD initialisation widget, if the user click SUBMIT, it goes to its child widget, but if the user click LOGOUT, it goes to the child widget of the LogoutPopup, and these two child widgets are not necessarily the same. As a result, when constructing the POD intialisation widget, we may need two child widgets which does not seem to be a good approach (this may contribute to the problems described in #157).
This issue is for us to propose an improved navigation design in solidpod that can naturally support multiple out-going routes in a widget.

This issue is here for discussion and brainstorming to improve the navigation design in solidpod. Move it to Ready.


Irrelevant (for the user) warning messages - change

[route bang set] in "_main.pd/timebase/change.pd" @ (x:22, y:48): Heavy interprets route arguments such as "bang", "list", "float", and "symbol" literally. They cannot be used to filter generic messages as in Pd.
No 'change.pd' subpatch present in patch.

See https://github.com/enzienaudio/heavy/issues/128.


[Question] Performance of ref API

Hi, I am opening a regular issue as I was not sure what to open, feel free to close if inappropriate.
You chose a different API than other library authors on how to register input elements in the form state. Most other libs use some kind of helper like <input {...registerField({ name: 'abc', ...)} />.
I tend to prefer yours because it looks cleaner in the code. Do you see any downsides? Is there a performance tradeoff that you know of? What happens under the hood? Are you bailing out of standard react APIs? Do you have to use DOM elements directly, or use DOM event binders? (which other may not have to, as they use standard props / event binders)
Any insight appreciated, on performance or otherwise.
Thanks for the library 👍

Hey @pandaiolo,
aha I love this question and I am more than happy to answer the question!
 The first approach was just like others spread operator, so no point at all. the second approach was thinking about useContext API from Hooks, oh boy It didn't turn out pretty... and work colleges telling me I am wasting my time and just use X. so I said 'fuck off', let me do my shit, haha and finally I have this idea to use ref and having validation stuff similar to Angular 1 (i think angular 1 dealing with form is excellent)
##Insight:
React hook form is taking a different approach than others library because it's not typically controlled input
<input onChange={this.handler} value={this.state.value />
instead, it's uncontrolled
<input ref={regsiter} /> (basically naked version)
so benefit out of the box is 0 re-render onChange with state update, and you are no longer need to have a Component wrapper around or things like <input {...registerField({ name: 'abc', ...)} /> because it's uncontrolled. the only source true which it's holding is the ref, now ref is really the HTML dom and you can retrieve and validate value without any harm. so what about invalidate inputs, yes I have attached each input with event listeners as a safeguard to revalidate inputs error. so what about GC, you will end up 2000 thousand event listeners end of the day, thanks to mutation observer https://developer.mozilla.org/en-US/docs/Web/API/MutationObserver I can safely remove event listener when the input gets removed. 🍾
##Performance benchmark time:
http://react-forme.now.sh
Hook Form

Formik

ReduxForm

as you clearly see less mount, quick execute and the following link show 1000 fields bit of crazy case:
https://github.com/bluebill1049/react-hook-form-performance-compare
Conclusion, I am not competing with Formik and Redux Form here, I am not a rock star dev like these guys, just play around with my shit as an average dev. it's a completely different approach which some people may find it useful and enjoy using (hopefully) we are using in production at work, so far it's approved its simplicity and value. (devs here love it, even the guy told me I am wasting of my time heheh) 😋
my English is bloody terrible so hope these make sense. Enjoy coding :)
Cheers
Billy

Hey @bluebill1049, thanks a lot for the detailed answer. Really appreciated. You are abstracting the "non-react" way to the consumers of your lib, so I guess it's perfectly fine!
I'll definitely give it a try especially if you say you're using this in production without hiccups.

@bluebill1049, you are a legend mate! Very happy to see the progress of your library, well done 👍

welcome back @afgallo ❤

@bluebill1049 - thanks for the benchmarks. Any chance you could add react-final-form to the mix?

@et sure, I can take a look that as well, the general idea is React Hook Form is uncontrolled component other controlled, so it has the advantage of mounting faster cause it doesn't require extra render/mount (controlled component or render props)
basically this
<ControlInput>
  <input />
</ControlInput>

with uncontrolled, you only do this
  <input />

in turns of re-render, because it's uncontrolled hence the library itself is driving and optimizing when to re-render.

@bluebill1049 - Fantastic response! Thank you for that.

because it's uncontrolled hence the library itself is driving and optimizing when to re-render.

Is this safe? In other words, won't this be susceptible to bugs that are hard to catch?

it's safe :) @et, because it's a form lib hence it's constrained to only trigger a re-render with relevant tasks, eg when a user onBlur, when user change input and etc...
(re-render is trigger by useState)

Story
Ok, a bit of story mode here (if you don't mind). I work for a startup company, which involves working on forms pretty much every day, technologies like Formik and Redux form which you probably heard them all.  The first approach was just like others by using spread operator on input props, so I guess no point at all to do the same stuff again.  Thank you for this awesome library. I love it. I saw it when i was trying to achieve same thing and when i almost coded my approach, i was thinking; "Yeah i think it is good now", and i saw your approach.   For the adding/removing side effects like event listeners, Why you did not use useEffect() for the process. Can't we achieve the same result with it rather than MutationObserver?
Cheers

Thank @cihataykas! Your words made by day! 😸Because all the inputs are essentially uncontrolled component (basically raw DOM element the ref), there won't be any lifecycle with them and useEffect wouldn't work. The only solution for me was to use native DOM methods MutationObserver.

@bluebill1049 Man your idea is cool ❤.  to reach the point, react way has more boilerplate than yours imho. My breakpoint was which one of the deadpoints,  is, to bind a ref in render, we can't use a function which holds useRef and returns it which breaks rules hooks. Or generating references dynamically.

@cihataykas do you mean to share the ref with others usage?
import React, { useRef } from "react"
import useForm from "react-hook-form"

function App() {
  const { register, handleSubmit } = useForm()
  const firstNameRef = useRef()
  const onSubmit = data => alert(JSON.stringify(data))

  return (
    <form onSubmit={handleSubmit(onSubmit)}>
      <input name="firstName" ref={(e) => {
        register(e)
        firstNameRef.current = e // you can still assign to your own ref
      }} />
      <input name="lastName" ref={(e) => {
        // register's first argument is ref, and second is validation rules
        register(e, { required: true })
      }} />

      <button>Submit</button>
    </form>
  );
}

@bluebill1049 Well, not really 😃. My bad that my english is not enough to tell the right topic. Actually i need an insight. And i wonder what you think.  let me tell you more specific:
You said, you tried to do same thing before with useContext or useState. And you did not like it the result. Hence you made your pattern. I'm trying to think another way just for brainstorming.
What if i make it this way:
import useForm from "react-hook-form"

function App() {
 // using a hook rather than regular register()
  const { useRegister, handleSubmit } = useForm()
  const onSubmit = data => alert(JSON.stringify(data))

  return (
    <form onSubmit={handleSubmit(onSubmit)}>
      // The point is, i want to perform the same idea with hooks.
      // I want to store all the field refs in useState.
      // So i can handle the state with react way.
      // below example is might not be an appropriate usage.
      <input name="lastName" ref={useRegister(e, { required: true })} />
      <button>Submit</button>
    </form>
  );
}

What are your insights about it?

@cihataykas I don't think that would work, because when you using ref you are leveraging uncontrolled input features.
if you want to use useEffect you can do the following tho
// the register will have `value` and `onChange` as props

<input {...register} />

@bluebill1049 Exacly. that makes them controlled.
i can get the ref via a custom hook and i make it controlled like this:
https://codesandbox.io/embed/infallible-tdd-hbrb1
So we loose re-rendering optimisation right?
to solve the problem i was thinking that if i use debounce pattern, it might bring a solution but not enough comparing your approach imo.

It's perfectly fine to build a form hook with controlled input. :)
you will have to setState/useState during input onChnage, which is fine to re-render too.
I think react-hook-form leverage both uncontrolled and custom hook, which making form building a bit closer to build native HTML form because you only have to do
<input name="test" ref={register} />

It's perfectly fine to build a form hook with controlled input. :)
you will have to setState/useState during input onChnage, which is fine to re-render too.
I think react-hook-form leverage both uncontrolled and custom hook, which making form building a bit closer to build native HTML form because you only have to do
<input name="test" ref={register} />
I am not a big fan of doing spread operator on the props but doesn't mean it's a bad thing to do. I personally prefer to see each prop on the component.

Definetly. thank you for the perspective. i think i'm satisfied enough 😄

Thanks for this awesome library 😁! I just converted one of my forms from formik because it uses an array of inputs then calculate the sum and it causes a lag whenever it reaches more than 10 rows because it keeps re-rendering for every keystroke. I tried using debounce just to ease the pain of my form 🤣. When I switched to this, I didn't have that problem anymore. Thanks 😄

thank you @JeromeDeLeon and most welcome ❤ going to follow up those issues that you reported.

How Can I test that values are changing? After using jest and  mount from enzyme it returns
 <input className="oldPassword" name="oldPassword" type="password" /> when usually I have value set and I can easily check what is rendered

@michaelbirchler what do you try to test?


NIFI-12556 Refactor components in cipher bundle using current API methods

Summary
Refactor tests and components in cipher bundle to use updated APIs from NIFI-12452.
Additionally, makes use of newer Java APIs, e.g. List.of and Set.of for declaration of relationships and properties.
NIFI-12556
Tracking
Please complete the following tracking steps prior to pull request creation.
Issue Tracking

[x] Apache NiFi Jira issue created

Pull Request Tracking

[x] Pull Request title starts with Apache NiFi Jira issue number, such as NIFI-00000
[x] Pull Request commit message starts with Apache NiFi Jira issue number, as such NIFI-00000

Pull Request Formatting

[x] Pull Request based on current revision of the main branch
[x] Pull Request refers to a feature branch with one commit containing changes

Verification
Please indicate the verification steps performed prior to pull request creation.
Build

[x] Build completed using mvn clean install -P contrib-check

[x] JDK 21

Licensing

[x] New dependencies are compatible with the Apache License 2.0 according to the License Policy
[x] New dependencies are documented in applicable LICENSE and NOTICE files

Documentation

[x] Documentation formatting appears as expected in rendered files

Thank you for the review & merge, @exceptionfactory.


About the yolo-det flow output size.

First of all, thank you for your contribution to the open source community.,(I don't used all the code, only I use two plug-ins, Yolo  and  upsample.)
When I  don't have yolo-det layer, the yolov3 network will have three outputs: "yololayer 82-conv", "layer 94-conv", "layer 106-conv". Their sizes are 211313, 212626, 325252, respectively.
I define the scale directly when I forward.
Cuda Malloc (& buffers [output Index0], 1 * OUTPUT_SIZE0 * sizeof (float));
CUDA Malloc (& buffers [output Index1], 1 * OUTPUT_SIZE1 * sizeof (float);
Cuda Malloc (& buffers [output Index 2], 1 * OUTPUT_SIZE2 * sizeof (float));
Context. enqueue (batchSize, buffers, stream, nullptr);
CudaMemcpyAsync (output 0, buffers [output index 0], batchSize * OUTPUT_SIZE0 * sizeof (float), cudaMemcpyDeviceToHost, stream);
CudaMemcpyAsync (output 1, buffers [output Index1], batchSize * OUTPUT_SIZE1 * sizeof (float), cudaMemcpyDeviceToHost, stream);
CudaMemcpyAsync (output 2, buffers [output Index 2], batchSize * OUTPUT_SIZE2 * sizeof (float), cudaMemcpyDeviceToHost, stream);
Then you can copy their output in output 1, output 2, output 3.   I want to know how to define my output size if the output layer is replaced by yolo-det, or whether it is reflected in your code.
I have seen it for a long time,    and  I can see net. getoutputsize (), but I do not understand it. can you provite some  blog or website about the form of output.
Thank you !

find some code  ,sorry ,  use this code  can get the outputsize ,but i can't sure   the output size is  assert  ,    ASSERT.
Dims dims = mTrtEngine->getBindingDimensions(i);
DataType dtype = mTrtEngine->getBindingDataType(i);
int64_t totalSize = volume(dims) * mTrtBatchSize * getElementSize(dtype);
mTrtBindBufferSize[i] = totalSize;
mTrtCudaBuffer[i] = safeCudaMalloc(totalSize);
if(mTrtEngine->bindingIsInput(i))
mTrtInputCount++;


Memory leak when putting records to kinesis

What platform/OS are you using?
Linux version 3.10.0-693.2.2.el7.x86_64<EMAIL_ADDRESS>(gcc version 4.8.5 20150623 (Red Hat 4.8.5-16)
Which version of the SDK?
What compiler are you using? what version?
c++
What are your CMake arguments?
Can you provide a TRACE level log? (sanitize any sensitive information)
CODE
    void CAwsEnvironment::PackSendRecord()
    {
        if(m_firehoseClient == nullptr || !isWrite){
            return;
        }
        //
        m_sendRecords.clear();
        //
        m_sendRecords.swap(m_pushRecords);
        //LOG_LOG_DEBUG("send.{} push.{}", m_sendRecords.size(), m_pushRecords.size());

    }
    void CAwsEnvironment::DoWrite()
    {
        if(m_firehoseClient == nullptr || !isWrite){
            return;
        }
        if(m_sendRecords.size() <= 0){
            return;
        }
        Aws::Firehose::Model::PutRecordBatchRequest putRecordRequest;
        putRecordRequest.SetDeliveryStreamName(m_streamName);
        putRecordRequest.WithRecords((Aws::Vector<Aws::Firehose::Model::Record>&&)m_sendRecords);
        auto result = m_firehoseClient->PutRecordBatch(putRecordRequest);
        if(result.GetResult().GetFailedPutCount() > 0){
            LOG_LOG_DEBUG("aws do write error {}", result.GetResult().GetFailedPutCount());
        }
        m_cacheLen = 0;
    }

    bool CAwsEnvironment::Init(std::string const &awsKey, std::string const &awsSecret,std::string const &fireHoseStr, const char* logName)
    {
        if(awsKey.empty() || awsSecret.empty() || fireHoseStr.empty() || logName == nullptr){
            m_firehoseClient = nullptr;
            return false;
        }
        SDKOptions options;
        InitAPI(options);

        AWSCredentials credentials(awsKey.c_str(),awsSecret.c_str());

        Aws::Client::ClientConfiguration client_configuration;
        client_configuration.region = "ap-southeast-1";
        client_configuration.scheme = Aws::Http::Scheme::HTTPS;
        client_configuration.connectTimeoutMs = 30000;
        client_configuration.requestTimeoutMs = 120000;
        //Aws::Firehose::FirehoseClient firehoseClient(credentials, client_configuration);
        m_firehoseClient = new Aws::Firehose::FirehoseClient(credentials, client_configuration);
        if(m_firehoseClient == nullptr){
            //std::cout << "init err" <<  std::endl;
            return false;
        }
        if(strlen(logName) + fireHoseStr.length() >= AWS_FIREHOSE_NAME_LENGTH - 1){
            return false;
        }
        sprintf(m_streamName, "%s-%s", fireHoseStr.c_str(), logName);
        LOG_LOG_DEBUG("aws.{}-{} init suc", fireHoseStr.c_str(), logName);
        isWrite = true;
        return true;
    }

init() and PackSendRecord() in main thread  ,DoWrite() in sub thread,run some hours ，there are some memory leak. valgrind log like this
==57866== 1,428,504 bytes in 2,319 blocks are possibly lost in loss record 3,976 of 4,031
==57866==    at 0x4C2B9B5: calloc (vg_replace_malloc.c:711)
==57866==    by 0x311356F9: ??? (in /usr/lib64/libnsspem.so)
==57866==    by 0x311261F1: ??? (in /usr/lib64/libnsspem.so)
==57866==    by 0x3112A5E8: ??? (in /usr/lib64/libnsspem.so)
==57866==    by 0x3112F4BA: ??? (in /usr/lib64/libnsspem.so)
==57866==    by 0x8492841: ??? (in /usr/lib64/libnss3.so)
==57866==    by 0x8494702: PK11_CreateGenericObject (in /usr/lib64/libnss3.so)
==57866==    by 0x67EE66C: ??? (in /usr/lib64/libcurl.so.4.3.0)
==57866==    by 0x67EE728: ??? (in /usr/lib64/libcurl.so.4.3.0)
==57866==    by 0x68272E6: ??? (in /usr/lib64/libcurl.so.4.3.0)
==57866==    by 0x681D95D: ??? (in /usr/lib64/libcurl.so.4.3.0)
==57866==    by 0x67F4E4C: ??? (in /usr/lib64/libcurl.so.4.3.0)
==57866==
==57866== 12,985,984 bytes in 405,812 blocks are possibly lost in loss record 4,029 of 4,031
==57866==    at 0x4C2B9B5: calloc (vg_replace_malloc.c:711)
==57866==    by 0x311356F9: ??? (in /usr/lib64/libnsspem.so)
==57866==    by 0x311263F8: ??? (in /usr/lib64/libnsspem.so)
==57866==    by 0x3112A5E8: ??? (in /usr/lib64/libnsspem.so)
==57866==    by 0x3112F4BA: ??? (in /usr/lib64/libnsspem.so)
==57866==    by 0x8492841: ??? (in /usr/lib64/libnss3.so)
==57866==    by 0x8494702: PK11_CreateGenericObject (in /usr/lib64/libnss3.so)
==57866==    by 0x67EE66C: ??? (in /usr/lib64/libcurl.so.4.3.0)
==57866==    by 0x67EE728: ??? (in /usr/lib64/libcurl.so.4.3.0)
==57866==    by 0x68272E6: ??? (in /usr/lib64/libcurl.so.4.3.0)
==57866==    by 0x681D95D: ??? (in /usr/lib64/libcurl.so.4.3.0)
==57866==    by 0x67F4E4C: ??? (in /usr/lib64/libcurl.so.4.3.0)
==57866==
==57866== 13,060,160 bytes in 408,130 blocks are possibly lost in loss record 4,030 of 4,031
==57866==    at 0x4C2B9B5: calloc (vg_replace_malloc.c:711)
==57866==    by 0x311356F9: ??? (in /usr/lib64/libnsspem.so)
==57866==    by 0x31126363: ??? (in /usr/lib64/libnsspem.so)
==57866==    by 0x3112A5E8: ??? (in /usr/lib64/libnsspem.so)
==57866==    by 0x3112F4BA: ??? (in /usr/lib64/libnsspem.so)
==57866==    by 0x8492841: ??? (in /usr/lib64/libnss3.so)
==57866==    by 0x8494702: PK11_CreateGenericObject (in /usr/lib64/libnss3.so)
==57866==    by 0x67EE66C: ??? (in /usr/lib64/libcurl.so.4.3.0)
==57866==    by 0x67EE728: ??? (in /usr/lib64/libcurl.so.4.3.0)
==57866==    by 0x68272E6: ??? (in /usr/lib64/libcurl.so.4.3.0)
==57866==    by 0x681D95D: ??? (in /usr/lib64/libcurl.so.4.3.0)
==57866==    by 0x67F4E4C: ??? (in /usr/lib64/libcurl.so.4.3.0)

Once I remove this line in DoWrite() function, there is no memory leak anymore:
" m_firehoseClient->PutRecordBatch(putRecordRequest);".
and the valgrind memory leak info I pasted above is gone.
addiction 1:
This is the libc library version and info.
[serveruser@localhost build]$ /lib/libc.so.6
GNU C Library (GNU libc) stable release version 2.17, by Roland McGrath et al.
Copyright (C) 2012 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
Compiled by GNU CC version 4.8.5 20150623 (Red Hat 4.8.5-16).
Compiled on a Linux 3.10.0 system on 2017-08-01.
Available extensions:
The C stubs add-on version 2.1.2.
crypt add-on version 2.1 by Michael Glad and others
GNU Libidn by Simon Josefsson
Native POSIX Threads Library by Ulrich Drepper et al
BIND-8.2.3-T5B
RT using linux kernel aio
libc ABIs: UNIQUE IFUNC
For bug reporting instructions, please see:
http://www.gnu.org/software/libc/bugs.html.

I suggest against using libcurl that building with libnss, which grow memory unbounded. Try build curl with openssl.

Problem solved after upgrading to a newer version of libcurl.
Old version:
curl 7.29.0 (x86_64-redhat-linux-gnu) libcurl/7.29.0
New version:
curl 7.29.0 (x86_64-redhat-linux-gnu) libcurl/7.66.0


WIP: Removal of identity nodes in matrix DDs

This pull request contains the changes which remove identity nodes in matrix DDs.
This includes not constructing a gate DD for the full system size (i.e. a single qubit operation is expanded to n-qubit with identities) as well as ignoring/removing identities in operations such as multiplication.

MQT Core is now live and the DD package has been migrated there.
Please integrate your changes in a new PR to that repository. Since all files were migrated as-is, this should be straight-forward. Sorry for the inconvenience.


Docker container for tutorial

Can start jupyter or nbdiff-web in the container using:
sudo docker pull rsesheffield/rse-conf-2017:1.0.0
sudo docker run -p 65000:65000 --ti rsesheffield/rse-conf-2017:1.0.0 /bin/bash
# then from within the container
jupyter notebook --ip=<IP_ADDRESS> --port=65000 --no-browser
# or
nbdiff-web --ip <IP_ADDRESS> --port 65000

jupyter and nbdiff-web will then present URLs that you can browse to from outside the container (with Gnome Terminal: right-click link then select 'Open Link').
Alternatively, if you run the container without specifying a command then it will just start a Jupyter Notebook server (i.e. does what lines 1&2 above do), but note that if you use this approach then stopping the Notebook server will also stop the container.
sudo docker run -p 65000:65000 rsesheffield/rse-conf-2017:1.0.0

Works a charm 🙌🏼 thanks a lot @willfurnass


Fix vulnerabilities

Two vulnerabilities were found in the project.

One in gulp-delete-lines package, which was not used, so removed it from devDependencies.
One is deeply nested and depended on by many packages. We can use the npm overrides field to manually set this one to a specific version

npm audit report
glob-parent  <5.1.2
Severity: high
glob-parent before 5.1.2 vulnerable to Regular Expression Denial of Service in enclosure regex - https://github.com/advisories/GHSA-ww39-953v-wcq6
fix available via `npm audit fix --force`
Will install<EMAIL_ADDRESS>which is a breaking change
node_modules/chokidar/node_modules/glob-parent
node_modules/glob-stream/node_modules/glob-parent
  chokidar  1.0.0-rc1 - 2.1.8
  Depends on vulnerable versions of glob-parent
  node_modules/chokidar
    glob-watcher  >=3.0.0
    Depends on vulnerable versions of chokidar
    node_modules/glob-watcher
  glob-stream  5.3.0 - 6.1.0
  Depends on vulnerable versions of glob-parent
  node_modules/glob-stream
    vinyl-fs  >=2.4.2
    Depends on vulnerable versions of glob-stream
    node_modules/vinyl-fs
      gulp  >=4.0.0
      Depends on vulnerable versions of glob-watcher
      Depends on vulnerable versions of vinyl-fs
      node_modules/gulp

lodash.template  <4.5.0
Severity: critical
Prototype Pollution in lodash - https://github.com/advisories/GHSA-jf85-cpcp-j695
No fix available
node_modules/gulp-util/node_modules/lodash.template
  gulp-util  >=1.1.0
  Depends on vulnerable versions of lodash.template
  node_modules/gulp-util
    gulp-delete-lines  *
    Depends on vulnerable versions of gulp-util
    node_modules/gulp-delete-lines

9 vulnerabilities (6 high, 3 critical)

how can we test that the override doesn't cause any problems before merging? Just try exporting in our app locally with this branch?

The surface area of the override is limited to the build scripts which lowers the risk on the output.
We can test everything still works by:

Running the build scripts and monitor for thrown errors,
Running the demos after building


decide how questions are gonna work

Questions to be answered
What different types of questions do we want to use
Multiple choice #44
Example

What colour do you get if you mix red and blue?

Requirements

[ ] store list of answer options
[ ] make sure that the construction of options doesn't become complicated due to constructor requirements such as those that might be needed for calculated answers

Suggestions
could have the following trait to be implemented on an answer type:
pub trait IncorrectAnswerOptions: Hash + Sized {
    fn incorrect_options(&self, quantity: usize) -> HashSet<Self>;
}

Simple #43
Example

What year was Python released?

Requirements

[ ] does not need to store answer options
[ ] might need to store information on the format / type of answer

With calculated answers #45
Example

1 + 2 = ?

Requirements

[ ] answer calculation service

What should a question struct look like
struct Question<ANSWER: QuestionAnswer> {
  id: ...,
  answer: ANSWER,
  multiple_choice_options: Vec<ANSWER>
  message: String,
}

trait QuestionAnswer {
  fn value(&self) -> String;
}

split into separate issues #43 #44 #45


Swipe item too sensitive, it is hard to do onClick

I use a Recycler view where user can do click, long click and swipe. I don't have any problem with long click and swipe, but when I try to do a click, four out five times, I end up getting a swipe gesture instead a click.
I tried app:minDistRequestDisallowParent=20dp but I do not notice any improvement. Also I tried setMinFlingVelocity without success.
Any idea about what I'm doing wrong?
Thanks.

app:minDistRequestDisallowParent=20dp and setMinFlingVelocity not effect for click sensitivity. I have to implement by other way to do this.

Maybe it is not to adjust onClick sensitivity but adjust swipe gesture sensitivity.

I have the same problem.

solved this

@tanakasama  please can you tell us how you solved the problem?

Did anyone came up with a solution for this? Really irritating

Yeah, I would be interested in a solution, too.

I am using this library now  : https://github.com/daimajia/AndroidSwipeLayout

I guess people should use https://github.com/daimajia/AndroidSwipeLayout . This repos have many issues with android version and devices. And this is not maintain now.

private float mX = -1,mY = -1;
@Override
public boolean onInterceptTouchEvent(MotionEvent ev) {
    if(isDragLocked()){
        return super.onInterceptTouchEvent(ev);
    }

    mGestureDetector.onTouchEvent(ev);
    mDragHelper.processTouchEvent(ev);

    switch (ev.getAction()) {
        case MotionEvent.ACTION_DOWN: {
            mX = ev.getRawX();
            mY = ev.getRawY();
        }
    }

    if(inMainView(ev)){
        boolean in = isInTouchSlop(ev);
        if(in){
            return false;
        }
    }

    boolean settling = mDragHelper.getViewDragState() == ViewDragHelper.STATE_SETTLING;
    boolean idleAfterScrolled = mDragHelper.getViewDragState() == ViewDragHelper.STATE_IDLE
            && mIsScrolling;

    return settling || idleAfterScrolled;
}

private boolean inMainView(MotionEvent ev){
    float x = ev.getX();
    float y = ev.getY();
    return x >= mMainView.getLeft() && x <= mMainView.getRight() && y >= mMainView.getTop() && y <= mMainView.getBottom();
}

private boolean isInTouchSlop(MotionEvent ev){
    boolean in = false;
    if(getDragEdge() == DRAG_EDGE_LEFT || getDragEdge() == DRAG_EDGE_RIGHT){
        float distanceX = Math.abs(ev.getRawX()-mX);
        if(distanceX < mDragHelper.getTouchSlop()){
            in = true;
        }
    }else{
        float distanceY = Math.abs(ev.getRawY()-mY);
        if(distanceY < mDragHelper.getTouchSlop()){
            in = true;
        }
    }

    return in;
}

This is my changed code.
good luck.

@qihongbo where did u placed this code?

download sources,in SwipeRevealLayout.java

@qihongbo
Thank you it works perfectly!

@qihongbo
Thanks for Your code, this helped me in my project to fix this annoying issue. Beside this problem i was having difficulties with closing, opened item in my recycler view after onClick and notifyDataSetChanged. If someone is having the same problem as i had, You should remove  "state == SwipeRevealLayout.STATE_DRAGGING" from if statment in ViewBinderHelper, bind method.

@qihongbo Thank you. It helped me.

Hi all. I tried to change the code as @qihongbo did. After downloading the source and change the code. I then added the source code as an external library and commented out compile 'com.chauthai.swipereveallayout:swipe-reveal-layout:1.4.0'. Android Studio then complained that No resource identifier found for SwipeRevealLayout in xml. What do I do now? Cheers

I can't find that method getDragEdge() ?


(M3) Consider improving efficiency for ListTile defaults

Consider improving efficiency for M2 and M3 ListTile defaults
In master channel (3.7.0-13.0.pre.80) we have the following default themes for ListTile in M2 and M3.

class _LisTileDefaultsM2 extends ListTileThemeData {
  _LisTileDefaultsM2(this.context, ListTileStyle style)
    : _themeData = Theme.of(context),
      _textTheme = Theme.of(context).textTheme,
      super(
        shape: const Border(),
        style: style,
      );

  final BuildContext context;
  final ThemeData _themeData;
  final TextTheme _textTheme;

  @override
  Color? get tileColor =>  Colors.transparent;

  @override
  TextStyle? get titleTextStyle {
    switch (style!) {
      case ListTileStyle.drawer:
        return _textTheme.bodyLarge;
      case ListTileStyle.list:
        return _textTheme.titleMedium;
    }
  }

  @override
  TextStyle? get subtitleTextStyle => _textTheme.bodyMedium;

  @override
  TextStyle? get leadingAndTrailingTextStyle => _textTheme.bodyMedium;

  @override
  Color? get selectedColor => _themeData.colorScheme.primary;

  @override
  Color? get iconColor {
    switch (_themeData.brightness) {
      case Brightness.light:
        // For the sake of backwards compatibility, the default for unselected
        // tiles is Colors.black45 rather than colorScheme.onSurface.withAlpha(0x73).
        return Colors.black45;
      case Brightness.dark:
        return null; // null, Use current icon theme color
    }
  }
}

class _LisTileDefaultsM3 extends ListTileThemeData {
  const _LisTileDefaultsM3(this.context)
    : super(shape: const RoundedRectangleBorder());

  final BuildContext context;

  @override
  Color? get tileColor => Theme.of(context).colorScheme.surface;

  @override
  TextStyle? get titleTextStyle => Theme.of(context).textTheme.bodyLarge;

  @override
  TextStyle? get subtitleTextStyle => Theme.of(context).textTheme.bodyMedium;

  @override
  TextStyle? get leadingAndTrailingTextStyle => Theme.of(context).textTheme.labelSmall;

  @override
  Color? get selectedColor => Theme.of(context).colorScheme.primary;

  @override
  Color? get iconColor => Theme.of(context).colorScheme.onSurface;
}

Is there any reason why the M3 generated theme default could not do what the M2 handwritten theme does? Meaning, look up the theme of context just once instead of 6 times?
Also M2 theme could do it just once too, and reuse the already traversed inherited theme for Theme.of(context) for the _textTheme it needs as well.
These are of course small optimizations, but if used frequently in the framework for defaults for all widgets, they may add up. Especially since widgets often use their default values.
Maybe consider Theme.of(context) usage review for defaults?
Maybe consider reviewing this for all generated M3 themes and M2 themes that have been rewritten to extend the default null theme, instead of doing it inline in the widget code. This has been observed in other widget defaults as well.

DPT BTW: The rewrite of M2 default value to also extend the all null sub-theme data defaults with none null values is really nice. It makes it much easier to find the default values. Thanks for doing this as part of the M3 implementations as well.

Observation: Challenge with current M3 tileColor default
The new M3 spec default background (Theme.of(context).colorScheme.surface) for ListTile is a poor choice. Past M2 transparent color is a better choice, also for M3.
Rationale:
With the new M3 default, the developer has to determine what elevation a surface has (Card, Popup, Dialog, Drawer, etc) and set the ListTile elevation to same value to get it to fit color wise on the used background, so it gets same elevation tint as the surface it is on.
Or alternatively always create a sub-theme and there define the tileColor color to be transparent again, or set it to transparent on widget level. All three options work, but are extra work. If not done, the ListTile will not match the background color in many use cases.
Considering that ListTile is often used on elevated surfaces like Card, Drawer and PopupMenuButton, that get tint elevation in M3, and one typically want the ListTile to fit seamlessly on such backgrounds. This no longer works out of the box with current defaults and it seems a bit counter productive.
This also creates a design break when migrating an existing app from M2 to M3, they will not fit on backgrounds where they worked previously in M2. Sure, it can easily be fixed by defining a custom sub-theme for ListTile that sets the color back to Colors.transparent that it used by default before. But I'm sure many will stumble on this and wonder about why it has to be this way. Consider reviewing with the Material team if Colors.transparent would still not be a better default for ListTile.tileColor in M3, at least for the Flutter implementation. Maybe in others too.

Flutter doctor
 flutter doctor -v
[!] Flutter (Channel master, 3.7.0-13.0.pre.80, on macOS 13.0.1 22A400 darwin-arm64, locale en-US)
    • Flutter version 3.7.0-13.0.pre.80 on channel master at /Users/rydmike/fvm/versions/master
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision bf2701d2a8 (7 hours ago), 2022-12-27 09:54:34 -0500
    • Engine revision 9d69a91bba
    • Dart version 3.0.0 (build 3.0.0-65.0.dev)
    • DevTools version 2.20.0
    • If those were intentional, you can disregard the above warnings; however it is recommended to use "git" directly to perform
      update checks and upgrades.

[✓] Android toolchain - develop for Android devices (Android SDK version 33.0.0)
    • Android SDK at /Users/rydmike/Library/Android/sdk
    • Platform android-33, build-tools 33.0.0
    • Java binary at: /Applications/Android Studio.app/Contents/jre/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 11.0.13+0-b1751.21-8125866)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 14.2)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Build 14C18
    • CocoaPods version 1.11.3

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2021.3)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.13+0-b1751.21-8125866)

[✓] IntelliJ IDEA Community Edition (version 2022.3.1)
    • IntelliJ at /Applications/IntelliJ IDEA CE.app
    • Flutter plugin version 71.2.6
    • Dart plugin version 223.8214.16

[✓] VS Code (version 1.73.1)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.54.0

[✓] Connected device (2 available)
    • macOS (desktop) • macos  • darwin-arm64   • macOS 13.0.1 22A400 darwin-arm64
    • Chrome (web)    • chrome • web-javascript • Google Chrome 108.0.5359.124

[✓] HTTP Host Availability
    • All required HTTP hosts are available

Back from vacation.
Thanks, @rydmike for reporting. It looks like M3 tileColor affected some internal tests and part 1 PR has been reverted https://github.com/flutter/flutter/pull/117756.
I'll land one PR with all the updates and address this issue in it.

Few more tests, will file the  PR tomorrow.

https://github.com/flutter/flutter/pull/117965 will land with the fix and other M3 updates for ListTile.


Remove Pete Runyon from /board

Pete submitted his resignation-- should be removed from the /board page.

i got this one


fixed projection law gains

Thank you!

Hi Jacob, the plot for the gain sweep (https://github.com/RoboticExplorationLab/non-monotonic-detumbling/blob/main/figs/pdf/gain_sweep_momentum_magnitude_vs_time.pdf) has some issues.

Oh, thanks for pointing that out. The merge must not have been completely
clean. I reran everything on my computer and updated the paper. I must’ve
failed to push the updates to GitHub. I’m away from that computer for a
couple of days but I’ll send you the paper with the updated plots.
On Thu, Apr 25, 2024 at 7:37 AM davinve @.***> wrote:

Thank you!
Hi Jacob, the plot for the gain sweep (
https://github.com/RoboticExplorationLab/non-monotonic-detumbling/blob/main/figs/pdf/gain_sweep_momentum_magnitude_vs_time.pdf)
has some issues.
—
Reply to this email directly, view it on GitHub
https://github.com/RoboticExplorationLab/non-monotonic-detumbling/pull/3#issuecomment-2077211427,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AHAS4ZQVXRE7ICED5ZX33VLY7EBKNAVCNFSM6AAAAABF2SQTXOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANZXGIYTCNBSG4
.
You are receiving this because you modified the open/close state.Message
ID: <RoboticExplorationLab/non-monotonic-detumbling/pull/3/c2077211427@
github.com>


tallyarbiter crash when close vmix

Tallyarbiter crash when close vmix program or is not online and his is set to connect and or reconnect

Which version of Tally Arbiter, and what error message do you get?

At the moment everything works regularly.
I probably had something wrong with it
Thanks

Feel free to reopen if that changes. 😃


Ability to document miscellaneous information

It would be very useful if there was a way to document "extra information", such as how to build CDN urls, for example.
I would suggest this is added much like errors, auth, etc, as an array of optional pages which could be inserted into the docs, i.e:
{ plugin: HapiDocs,
      options: {
        pages: [
          {
            name: 'CDN Urls',
            title: 'Some page title here'
            descriptions: [
              "Some different pararaphs go here",
              "Relating to the subject matter"
            ]
          }
        ],
        ...
      }
}

This will be part of next release and documented by then. Thanks for the suggestion!


DecimalMoneyFormatter How to show comma seperator for amount in thousands

Is there an option for the DecimalMoneyFormatter object to add commas to amounts in the thousands?
eg 3000.00 is displayed as 3,000.00 ?
Thanks

I think this is what you're looking for https://moneyphp.org/en/stable/features/formatting.html#intl-decimal-formatter

Thanks. Turns out I needed to construct the NumberFormatter as a 'NumberFormatter::CURRENCY'. I didn't realise NumberFormatter was a native object, otherwise I would have referred to the PHP docs!


Will I need use PM2 along with sticky-cluster to run forever nodejs

is there any necessity to run my Socket.io nodejs process using PM2?
If my server crash duet any error within code will sticky-cluster auto restart the process?
Can we use PM2 along with sticky-cluster?
I am using 'socket.io-redis' module for clustering with redis Key Value memory cache mechanism to store Socket id of all connected users.

Hi, @JamesStark-wasp, thanks for reaching out! Unfortunately, I'm not an expert of pm2's functionality to give a well thought recommendation. If you're interested in my guess, then I think that you definitely can start sticky-cluster based app with pm2, that way pm2 will restart master process and sticky-cluster will restart child processes.

When I started sticky-cluster based app with PM2  using  pm2 start api.js -i max
command its not working client disconnected from socket.io immediately


fix bibliography problems (refManageR package)

get a MWE working and see where the issue is

fixed; see here


Let's introduce our app to the world

This PR adds responsive styles for content, a little in browser demo, and additional content.

Responsive styles:

@josh I hit you up in Slack, but just leaving a note here for posterity.
I think it'd be rad to do three small things:

Create a switch to go into dark mode (this should be easy as everything is CSS / SVG. All we would need to do is change the .menubar bg color, text color, and svg fill color).
Write a little JS that makes the clock accurate to system time.
Possibly write a little JS that will start the Aware timer on page load (not that anyone will stay on this page for > 1 minute to see it change, but it'd be slick!)

Thoughts?

This is totally badass!

I think it'd be rad to do three small things:

Awesome idea! I love that you reproduced with the menu bar with html and css. We could totally make that work!
❤

:metal:


Snapshots is not working when using native ESM module loader

It seems like snapshots is not working when using the native ESM loader in node when the module being tested is an ESM.
My module is defined as a ESM by defining type as module in package.json:
{
    "name": "foo",
    "version": "1.0.0",
    "type": "module"
}

My module and tests are then written as ESM. I run the tests with the --no-esm flag to use the native ESM module loader in later node.js (12 and 14).
Generating snapshots by the following command works well:
tap --no-esm tests/*.js --snapshot

Snapshots are written to the tap-snapshots folder but when I run the tests without generating the snapshots with the following command I get an error:
tap --no-esm tests/*.js

The error it outputs is (and a bit miss leading since the file it say it does not find does exist):
Snapshot file not found: /pah/to/module/tap-snapshots/my-file.js-TAP.test.js
Run with TAP_SNAPSHOT=1 in the environment to create snapshot files

The real error message is actually eaten by tap. The real problem here is as follow:
Error [ERR_REQUIRE_ESM]: Must use import to load ES Module: /pah/to/module/tap-snapshots/my-file.js-TAP.test.js
require() of ES modules is not supported.
require() of /pah/to/module/tap-snapshots/my-file.js-TAP.test.js  from /pah/to/module/node_modules/tap/lib/snapshot.js is an ES module file as it is a .js file whose nearest parent package.json contains "type": "module" which defines all .js files in that package scope as ES modules.
Instead rename my-file.js-TAP.test.js to end in .cjs, change the requiring code to use import(), or remove "type": "module" from /pah/to/module/package.json.

In other words; when the module is defined as a ESM by setting type in package.json, the loader expects the snapshots to be ESM too due to its .js extension.
Renaming the snapshot file to .cjs as suggested by node does not solve the issue. Tap will then complain about the snapshot not existing.
Only workaround I have found to get this working is to put a package.json in the tap-snapshots folder without a type property. That makes the native ESM loader in node treat the snapshots as a CommonJS module.

This is a known issue, the workaround is to create tap-snapshots/package.json containing {"type": "commonjs"}.  The snapshot files will have a .cjs extension in tap v15 but that switch is a breaking change.

Thanks for your quick reply.
If the solution for this is using a .cjs extension in tap v15 and no work is done on this yet I'm happy to take a stab at a PR for this.

This has already been changed in the v15 branch.  No ETA yet on v15 release.

Given v15 is out by now, can this issue be closed?

I'm still seeing this on V16.0.0 with typescript.
npx tap  --snapshot --ts
test/index.spec.ts 2> TypeError [ERR_UNKNOWN_FILE_EXTENSION]: Unknown file extension ".ts" for /Users/rkesters/code/github/remark-pdf/test/index.spec.ts

@rkesters Different issue. You're describing #807