Performance issue when auto-save triggers in constrained memory situations

We've had some reports lately about performance issues when the auto-save runs and I think I've found more closely what makes it happen.
If you push the game to near the memory limit, by increasing the view distance and flying around loading chunks a lot, then while you may be low FPS due to the memory churn when auto-save kicks in it gets really bad
In my case I started from source in a Cities world and just flew around with ghost + hspeed. Was able to push to the 1 gb default limit pretty easily and keep the memory used hovering constantly near the max. I increased to 2 gb max instead and with ample space from current use to max limit the auto-save was barely noticeable again.
@flo you around with any time to take a look at some point convenient to you? Although I'm not really sure what the solution is, since saving is going to need some memory. During auto-save shave off some view distance dynamically to have enough memory if low?
Maybe related: http://forum.terasology.org/threads/delta-too-great.1374/

I think we should increase the default memory maximum to 2 GB.

That'll help in some causes (and hurt in others - like trying to run on 32-bit), but wouldn't we want to harden the process to degrade more gracefully under pressure? With the right settings you could still put the game under strain at 2 GB.

I found this bug report after trying Terasology for the first time and experiencing similar issues at autosave times.
I had initially set min 768M and max 1G for the RAM. In the core mode I don't have too much problems, but in TTA2 the memory gets quickly up to the max and as soon as the auto-save kicks in, the FPS just collapse and the game gets pretty quickly unplayable (0.5 FPS or less). The problem is made worse by the fact that when one autosave eventually finishes (after as much as 1 minute), another one starts a few seconds/milliseconds after and there we go for another round of extremely degraded performance.
After setting the limit to 2GB, the game was mostly playable for about 5 minutes until it crashed. However the crash may be due to another issue. In case it helps, here are the links to the uploaded logs from the crash reporter: https://zerobin.zertrin.org/?18c5e7dbc0f24dba#R4FZpH1W8jJlkRFdQBQ0zavSTP+8c6M7IYaJjYRgpm8= (link valid one year)

Thanks for the details @zertrin :-)
Snippet from the game-killing error:
orageManager - Saving - Snapshot created: Writing phase starts
23:12:42.878 [main] WARN  o.t.engine.internal.TimeBase - Delta too great (3074), capping to 1000
23:12:55.013 [Saving-0] INFO  o.t.p.internal.SaveTransaction - Save game finished
23:12:55.016 [main] WARN  o.t.engine.internal.TimeBase - Delta too great (3463), capping to 1000
23:13:38.293 [main] INFO  o.t.p.i.ReadWriteStorageManager - Saving - Creating game snapshot
23:13:38.295 [main] INFO  o.t.p.i.ReadWriteStorageManager - Saving - Snapshot created: Writing phase starts
23:13:42.914 [main] WARN  o.t.engine.internal.TimeBase - Delta too great (3187), capping to 1000
23:13:51.470 [main] WARN  o.t.engine.internal.TimeBase - Delta too great (3589), capping to 1000
23:13:55.537 [main] WARN  o.t.engine.internal.TimeBase - Delta too great (3251), capping to 1000
23:13:58.155 [Saving-0] INFO  o.t.p.internal.SaveTransaction - Save game finished
23:14:02.197 [main] ERROR o.terasology.engine.TerasologyEngine - Uncaught exception, attempting clean game shutdown
java.lang.NullPointerException: null
	at org.terasology.logic.delay.DelayedActionSystem.invokeDelayedOperations(DelayedActionSystem.java:74) ~[engine-0.54.4.jar:61, master, 61, pre-alpha]
	at org.terasology.logic.delay.DelayedActionSystem.update(DelayedActionSystem.java:53) ~[engine-0.54.4.jar:61, master, 61, pre-alpha]
	at org.terasology.engine.modes.StateIngame.update(StateIngame.java:178) ~[engine-0.54.4.jar:61, master, 61, pre-alpha]
	at org.terasology.engine.TerasologyEngine.mainLoop(TerasologyEngine.java:414) ~[engine-0.54.4.jar:61, origin/master, 61]
	at org.terasology.engine.TerasologyEngine.run(TerasologyEngine.java:369) ~[engine-0.54.4.jar:61, origin/master, 61]
	at org.terasology.engine.Terasology.main(Terasology.java:154) [Terasology.jar:61, origin/master, 61]
23:14:02.198 [main] INFO  o.terasology.engine.TerasologyEngine - Shutting down Terasology...
23:14:02.761 [main] INFO  o.t.p.i.ReadWriteStorageManager - Saving - Creating game snapshot
23:14:02.765 [main] INFO  o.t.p.i.ReadWriteStorageManager - Saving - Snapshot created: Writing phase starts
23:14:02.767 [main] INFO  o.t.n.internal.NetworkSystemImpl - Client disconnected: Zertrin

Not a super informative error, but yeah it may be something other than the auto save. Three times earlier this error occurs, which looks familiar, but wasn't the same as https://github.com/Terasology/Cities/issues/28 as I thought at first.
23:02:32.688 [main] INFO  o.t.r.world.RenderableWorldImpl - New Viewing Distance: Moderate
23:02:49.041 [main] WARN  o.t.engine.internal.TimeBase - Delta too great (2164), capping to 1000
23:02:52.470 [main] ERROR o.t.e.event.internal.EventSystemImpl - Failed to invoke event
java.lang.NullPointerException: null
	at org.terasology.logic.players.LocalPlayerSystem.onTargetChanged(LocalPlayerSystem.java:243) ~[engine-0.54.4.jar:61, master, 61, pre-alpha]
	at org.terasology.logic.players.LocalPlayerSystemMethodAccess.invoke(Unknown Source) ~[reflectasm-1.11.0.jar:61, master, 61, pre-alpha]
	at org.terasology.entitySystem.event.internal.EventSystemImpl$ByteCodeEventHandlerInfo.invoke(EventSystemImpl.java:502) ~[engine-0.54.4.jar:61, master, 61, pre-alpha]
	at org.terasology.entitySystem.event.internal.EventSystemImpl.sendStandardEvent(EventSystemImpl.java:270) [engine-0.54.4.jar:61, master, 61, pre-alpha]
	at org.terasology.entitySystem.event.internal.EventSystemImpl.send(EventSystemImpl.java:261) [engine-0.54.4.jar:61, master, 61, pre-alpha]
	at org.terasology.entitySystem.entity.internal.BaseEntityRef.send(BaseEntityRef.java:144) [engine-0.54.4.jar:61, master, 61, pre-alpha]
	at org.terasology.input.cameraTarget.PlayerTargetSystem.update(PlayerTargetSystem.java:74) [engine-0.54.4.jar:61, master, 61, pre-alpha]
	at org.terasology.engine.modes.StateIngame.update(StateIngame.java:178) [engine-0.54.4.jar:61, master, 61, pre-alpha]
	at org.terasology.engine.TerasologyEngine.mainLoop(TerasologyEngine.java:414) [engine-0.54.4.jar:61, origin/master, 61]
	at org.terasology.engine.TerasologyEngine.run(TerasologyEngine.java:369) [engine-0.54.4.jar:61, origin/master, 61]
	at org.terasology.engine.Terasology.main(Terasology.java:154) [Terasology.jar:61, origin/master, 61]
23:02:56.656 [main] WARN  o.t.engine.internal.TimeBase - Delta too great (2868), capping to 1000
23:03:07.434 [main] WARN  o.t.engine.internal.TimeBase - Delta too great (1120), capping to 1000

Maybe that's related to the lag? I'm not sure.
Hopefully we can dig deeper in this sometime, but we may need @flo to take a look. Or maybe @msteiger would have some clues.


update python-dateutil for current python versions #18

Update python-dateutil to current version for  #18
Tested locally on python 3.11

+1 for this. It would also resolve some security vulns that are being reported by dependabot due to the old version of urllib3


Improve handlers

Implement the ability to only restart services that require a restart. This will be based on the option we are changing in the ceph.conf, more precisely which section.
@SamYaple, wanna take this one when you get some spare time? :)

More info:

execute config_template
detect what changed in which section and put it in a registered variable
read the json
for each section and each change inject the argument with "injectargs"
once injected verify with the admin socket that the config has been properly injected
we compare the 'current' and the 'default' value, even for config options that disappeared
we need a patch from ceph to have the new version of the admin socket (see: http://www.spinics.net/lists/ceph-devel/msg31593.html)
if we detect that a configuration option wasn't properly injected we restart the approprieate daemon by looking at the section

For OSDs:

we need to restart them one by one on a machine
between each osd restart we need to verify that pgs are healthy
the restarting sequence should be serialized, which means serializing a task which is not possible at the moment (feature request here: https://github.com/ansible/ansible/issues/12170)

This issue will likely take some time to get resolved as we will need to wait for both Ansible (ETA unknown) and Ceph (relatively soon) need some code changes.

First version PR is here: #922

@SamYaple @mattt416 I added more thoughts here.

New PR here: https://github.com/ceph/ceph-ansible/pull/995


Quantization and Binarization Implementations for Linear Layers

Following code from Quantization Repo, integrated quantization into the nanoGPT repo.
Added a new QuantizedLinear layer that replaced linear layers and simulates quantization aware training. Also, added a binarization layer.

Awesome! This trains and converges!
We should try what the output from quantized inference looks like with the sample.py.
When trying to run sample.py on the created checkpoints, it encounters unexpected keys in the state_dict:

Could you take a look into how to test with the sample.py (feel free to modify if needed)?

Unfortunately, I didn't think ahead to split these changes into multiple PRs. I will try to summarize all the changes as best as possible.
Updates:
Added two options for quantization: "affine_quant" and "stochastic_quant"
Added QuantizedEmbedding to position_encoding_variations.py, which can be used for word embeddings and position embeddings
Added quantize_helper function which is used for quantization of the outputs of matrix multiplications and linear layers
train.py arguments:
"quantization_linear_method": quantization method used for all quantized linear layers
"quantize_wte": whether the word embedding weights are quantized (using QuantizedEmbedding)
"quantize_wpe": whether the position embedding weights are quantized (using QuantizedEmbedding)
"quantization_embedding_method": method of quantization used for embeddings
"quantize_attn_all": this boolean argument will quantize everything in the attn layer, including outputs and linear layers
"quantize_c_attn_q": whether the query weights are quantized (using QuantizedLinear)
"quantize_c_attn_k": whether the key weights are quantized (using QuantizedLinear)
"quantize_c_attn_v": whether the value weights are quantized (using QuantizedLinear)
"quantize_q": whether the query output is quantized (using quantize_helper)
"quantize_k": whether the key output is quantized (using quantize_helper)
"quantize_v": whether the value output is quantized (using quantize_helper)
"quantize_q_k_mult": whether the output of the query key matrix multiplication is quantized (using quantize_helper)
"quantize_softmax_v_mult": whether the output of the softmax value matrix multiplication is quantized (using quantize_helper)
"quantize_softmax": whether the output of the softmax variant is quantized (using quantize_helper)
"quantize_attn_proj": whether the output projection is quantized (using QuantizedLinear)
"quantize_mlp_all": this boolean argument will quantize everything in the mlp layer, including activation outputs and linear layers
"quantize_mlp_up": whether the mlp_up weights are quantized (using QuantizedLinear)
"quantize_mlp_down": whether the mlp_down weights are quantized (using QuantizedLinear)
"quantize_activation": whether the output of the activation function is quantized (using quantize_helper)
"quantization_activation_method": method of quantization for the activation output
Example code for running train.py:
python3 train.py --device="cuda" --dataset="shakespeare_char" --out_dir="out_shakespeare_char" --linear_variant="quantized_linear" --quantize_attn_all --quantize_mlp_all --quantize_wte --quantize_wpe
Example code for running sample.py:
python3 sample.py --out_dir="out_shakespeare_char" --device="cuda" --quant_weights_file="shakespeare" --visualize_weights_dir="shakespeare_weights"

Will start diving into the code. one thing we'll have to separate out the quantization for the attention output projection -- experiment with this show that it might need it's own unique precision (possibly higher) in order to operate well.


Use git_repository instead of http_archive

github.com/repo/archive/commit zip are created on the fly and can change, breaking the checksum. Either recommend use git_repository or upload real releases with fixed .zip files.

Bazel recommends against using git_repository.


requestHandler doesn't get called when typing a " "

Hi @benogle,
@joefitzgerald said I should send this one your way.
requestHandler should be called when a " " is typed.
ex scenario:
If a "." or a " " is typed a new list of suggestions is fetched from another process which is cache and as the user continues to type the list is filtered based on the prefix.

@benogle what are your thoughts on this? I know you had an opinion on triggering on backspace. I can see how triggering on space could be useful. @acormier uses the example of:
using<spacehere> he would like to show a list of namespaces that someone could choose, which I think is a valid scenario.
Similarly, I could write a provider for golang that would provide imports suggestions on import<spacehere.
This is offset by my thoughts that a carriage return should not trigger autocomplete. What do you think of this:

Addition / removal of whitespace on a single line should be allowed to trigger activation
Insertion / deletion of carriage return / line feed should not be allowed to trigger activation
Insertion / deletion of tab should not be allowed to trigger activation
Any other rules here?

This would imply removing the trim() we currently do on newText and oldText and doing something a little more nuanced. Thoughts?

This is key for even the basics, like function( hello,<space>NOPE
+1!

(On 2.6.0 - this seems to be resolved and or much better, quick tests).

@underscorediscovery do you expect the default provider to show on space? Or your own custom provider? I disagree with the former, and agree with the ability for the latter.

I think it will vary by language, but , space/s and ( space/s are common and frustrating to use without them. You can't type an argument list without crunching them with no spaces, because the completion just keeps getting pulled away.
I don't see a benefit in that in any provider, but I can see how on a blank line (say when using spaces as tabs) it would be a bit weird to pop up constantly. For custom providers this should be ok, because they're rejecting if there's no state.
What i've been "wanting" when implementing a more complex custom provider, is that it be configurable where possible if not good for defaults. Like if it was a simple whitelist "trigger on [',",(,.]", it could help because we can return nothing to dismiss the list, and return the same state to keep it open.
Aside for the completion not showing at all (trigger), the inverse of that (the dismissal) is out of our control, making it impossible to cater to certain aspects of the language, but the majority of the pain points are not language specific that I've run into - though they will potentially be coder preference based (like typing function calls with func( arg0 ) instead of func(arg0).

Like if it was a simple whitelist "trigger on [',",(,.]"

We can just trigger on nearly everything, and allow the provider to make the decision. I dont think we need another option. There is now a prefixRegex that has all the trigger chars.

That's great, that's plenty configurable I think!

That's great!  Thanks.


⚠ Bofrost.at has degraded performance

In 36504f4, Bofrost.at (https://www.bofrost.at) experienced degraded performance:

HTTP code: 200
Response time: 1336 ms

Resolved: Bofrost.at performance has improved in 302f322.


Industrial Revolution 3

Interested in contributing?
Use FactorioLab Export (Source/README) to export the mod set yourself, fork this repository, and create a pull request!
Have you checked whether this mod or set of mods has already been requested?
Check here
If you find the mod you're looking for, add a "thumbs-up" reaction or a comment to the issue.
What mod set are you requesting?
Industrial Revolution 3
**Is this mod set compatible with the latest version of Factorio?"
Yes.
What mods are included in this mod set?
Industrial Revolution 3 3.0.10
Industrial Revolution 3: Assets 1 >= 1.0.7
Industrial Revolution 3: Assets 2 >= 1.0.4
Industrial Revolution 3: Assets 3 >= 1.0.1
Industrial Revolution 3: Assets 4 >= 1.0.1

This was added yesterday, is there a problem with the data set?

Nope, i did refresh the page just, i didnt see it, maybe network problem. thx for the update, you can close the issue.


⚠ Barchessa Loredan has degraded performance

In 7b4d4b2, Barchessa Loredan ($SITE_BRKLRD) experienced degraded performance:

HTTP code: 200
Response time: 6208 ms

Resolved: Barchessa Loredan performance has improved in d1f85b3.


In HIL mode, motor will be disarmed after 10s even if disable the auto disarmed function on the QGC

In HIL mode, motor will be disarmed after 10s even if I disable the auto disarmed function on the QGC

px4 version is 1.11.0

What are you using for HIL? Can you share the log with this happening? https://logs.px4.io/

Thank you for your reply. I used Airsim v1.3.1 for HIL.  The log is https://logs.px4.io/plot_app?log=28740df9-a979-40b0-82aa-bf7cf7cd6eb3
This issue maybe start with v1.11beta2

My hardware is Cube Black and Durandal. They are both have the issue.

@kcx064 Can you try and build Airsim from source? I think there was a issue with the v1.3.1 release of Airsim and was fixed afterwards

@kcx064 Can you try and build Airsim from source? I think this is most likely a Airsim issue since there was some issue with the v1.3.1 release of Airsim and was fixed afterwards

Thank you! But PX4 v1.10.1 does not have the issue when I used Airsim v1.3.1

@kcx064 Yes, and that is why something had to be fixed on the Airsim side

@kcx064 Yes, and that is why something had to be fixed on the Airsim side

It is to say PX4 v1.11 makes some modification for HITL mode which makes Airsim can not work properly in HITL mode, so Airsim need to be fixed?

Thank you for your reply. I used Airsim v1.3.1 for HIL. The log is https://logs.px4.io/plot_app?log=28740df9-a979-40b0-82aa-bf7cf7cd6eb3
This issue maybe start with v1.11beta2

"Disarmed by lockdown"

You're hitting this. https://github.com/PX4/Firmware/blob/af84f449c74b1b108bf23f6b8fdf63cb95c1b39a/src/modules/commander/Commander.cpp#L1681-L1692
In HIL lockdown is enabled for safety. Let me see about a quick fix.
@Jaeyoung-Lim we should talk about getting automated HIL tests in CI. One helpful step in that direction would be if we make it as trivial to launch HITL (gazebo or jmavsim) as SITL gazebo.

@Jaeyoung-Lim we should talk about getting automated HIL tests in CI. One helpful step in that direction would be if we make it as trivial to launch HITL (gazebo or jmavsim) as SITL gazebo.

Okay, makes sense. Do we want to hook this up to github actions?

The lockdown -> auto disarm was introduced in https://github.com/PX4/Firmware/pull/14766.

@Jaeyoung-Lim we should talk about getting automated HIL tests in CI. One helpful step in that direction would be if we make it as trivial to launch HITL (gazebo or jmavsim) as SITL gazebo.

@dagar @Jaeyoung-Lim,
@bperseghetti is already working on those. And he is facing the exact same problem as above. @bperseghetti do you mind syncing with @dagar on a fix for this? Thanks!

@Jaeyoung-Lim we should talk about getting automated HIL tests in CI. One helpful step in that direction would be if we make it as trivial to launch HITL (gazebo or jmavsim) as SITL gazebo.

Okay, makes sense. Do we want to hook this up to github actions?

Let's get the pieces in place to do it as a developer first, then figure out how to make it live in github actions or Jenkins (attaching to hardware is a "detail").

Is sitl_gazebo HIL good enough to start using regularly? How can we make it trivial to launch directly and specify the serial port? What about included airframes that correspond to sitl_gazebo models? I don't think they're in the NuttX ROMFS.
Scripting the process.

flashing, setting the right airfarme, connecting
launch mission/mavsdk test and monitor
transfer ulg over mavlink

We can keep discussing outside of this issue.

@kcx064 can you try https://github.com/PX4/Firmware/pull/15690?

@kcx064 can you try #15690?

It's my pleasure!

@kcx064 can you try #15690?

Thank you!  I have try your branch,  it work properly in hil mode!

@dagar @TSC21 @Jaeyoung-Lim I had noticed this issue with HITL using gazebo last Monday. I think there is a lot of ways we can automate HITL testing. I had "ugly fixed" it by just commenting out the autodisarm and was looking through the rest of the HITL pipeline to see more about avoiding edge cases and how well it can "represent" the actual system before issuing an actual fix for that specific bug. I pulled and tested the bug for HITL/gazebo and it works as well.


Bug: Installing certain packages fails!?

Branch : Both main and develop
There is something missing in the installation pipeline.
The debug output shows that package is installed,
however the unzipped content is not there
in installation_dir
$ gbpacman -i readline
[DEBUG]  gbpacman | 22 : Pinging...https://packages.msys2.org/search?q=readline
[DEBUG]  gbpacman | 116 : Skipping downloading for package : mingw-w64-readline
[DEBUG]  gbpacman | 118 : mingw-w64-clang-i686-readline-8.2.001-6-any.pkg.tar.zst is already downloaded.
[DEBUG]  gbpacman | 59 : Unzipping with zstd...
[WARNING]  gbpacman | 65 : zstd: E:\test\lib\site-packages\gbpacman\temp\mingw-w64-clang-i686-readline-8.2.001-6-any.pkg.tar already exists; not overwritten

[DEBUG]  gbpacman | 31 : unzipping E:\tst\lib\site-packages\gbpacman\temp\mingw-w64-clang-i686-readline-8.2.001-6-any.pkg.tar
[]
[DEBUG]  gbpacman | 144 : E:\test\lib\site-packages\gbpacman\lib\api.py
[DEBUG]  gbpacman | 145 : []
[DEBUG]  gbpacman | 159 : List of downloaded files written to E:\test\lib\site-packages\gbpacman\store\mingw-w64-clang-i686-readline-8.txt
[INFO]  gbpacman | 57 : Please add 'E:\test\lib\site-packages\gbpacman\installation\usr\bin' to path

oh!
https://github.com/PrashanthaTP/gbpacman/blob/10443fab328fcf33cfd1e65855fb867f31cbbbbc/gbpacman/utils/files.py#L37
Here the target dir is being updated while passing as argument
but we should have reused the variable which was set here
https://github.com/PrashanthaTP/gbpacman/blob/10443fab328fcf33cfd1e65855fb867f31cbbbbc/gbpacman/utils/files.py#L22


Potential Speedup with number of threads

@HolyWu
I commented out:
#if num_streams > vs.core.num_threads:
#    raise vs.Error("rife: setting num_streams greater than `core.num_threads` is useless")

and then set the number of streams to 24. It gave me 125-135 Frames per second, instead of 115 Frames per second with 12 streams.
If I set it to 25/26 streams, it's getting very slow like 55fps, because of the new cuda fallback policy, which literally would end the program instead of working slowly.
My hardware:
I5-10500H Laptop with 6Cores / 12 threads.
RTX 3060 with 6144mb
The video I have tested, just the first minute:
720p anime video.
Suggestion:
I will try to look up with the help ai coding tools, if we can slowly increase the number of streams up until the point, where the vram is full with a 10% percent margin. So we might get the biggest boost. I do not think we should set a default num_stream number or at least set it to 1, because, the numofstreams value may change with the resoultion of the video.
A 4k video is 9 times bigger than a 720p video, which means, you cannot have "numofstreams=24", which would not fit into the memory. A dynamic approach to this would be better.
I will close this issue, since this is just a documentation for me and if you think, this is important for you.

Something is not working as it worked on the day I posted. It really is not improving performance setting the number more than thread available. Must look at:
-Spectre, Meltdown, Downfall.

@HolyWu
Another test:
18:32 with script3.py/script4.py/script5.py and 3 bat files.
First it gets split into three parts, then it interpolates, then it merges.
This was a naive approach by the way, if all videos would have been equally split. It might have been a little bit faster.
Num_streams was 5 for each script.
18:43 with script2.py with numstream 12.
Just scaling and merging audio and video together.
18:35 with script3.py/script4.py/script5.py and 3 bat files.
First it gets split into three parts, then it interpolates, then it merges.
This was equally split.
Num_streams was 5 for each script.
So this means, even if cuda is utilized 100% it does not get any faster in any meaninful way.
The Video: 720p with 23:42 runtime 23.974 FPS upscaled x3.
The Hardware:  I5-10500H Laptop with 6 Cores / 12 threads. Nvidia RTX 3060 with 6144mb.
So the good news is, that it does not get any faster. This is pretty much the limit on how fast it goes.


Android - Attempt to edit a comment crashes the app - Reported by @adeel0202

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

 Have you informed them of the workaround?
Platform:

Where is this issue occurring?

Web
iOS
Android
Desktop App
Mobile Web

Version Number: 1.2.7-0
Reproducible in staging?: Y
Reproducible in production?: N
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Notes/Photos/Videos: Any additional supporting documentation
https://user-images.githubusercontent.com/43995119/192384528-b33630a7-6079-41c8-9282-ee3d4b055e1b.mp4
Expensify/Expensify Issue URL:
Issue reported by: @adeel0202
Slack conversation: https://expensify.slack.com/archives/C01GTK53T8Q/p1664224827873869
View all open jobs on GitHub

:wave: Friendly reminder that deploy blockers are time-sensitive ⏱ issues! Check out the open StagingDeployCash deploy checklist to see the list of PRs included in this release, then work quickly to do one of the following:

Identify the pull request that introduced this issue and revert it.
Find someone who can quickly fix the issue.
Fix the issue yourself.

Hi @marcaaron, I think I'm eligible for the reporting bonus here.

Hi @marcaaron, I think I'm eligible for the reporting bonus here.

bump @marcaaron

I'm not entirely sure. You will have to ask in Slack about this.
Thanks so much for reporting!

Sorry for the ping, @dylanexpensify Can you please pay @adeel0202 out a reporting bonus for this issue? Thank you very much!

Thank you @mountiny :)

@dylanexpensify Can you additionally also pay out $250 to @aimane-chnaif for their help with finding the root cause for this deploy blocker and acting so fast for urgent issue? Thank you very much
Slack thread for context https://expensify.slack.com/archives/C01GTK53T8Q/p1664224827873869

Dylan is OOO, asking someone to handle the payments, thank you for patience

Stepping in to help move this to completion.
Upwork job - https://www.upwork.com/jobs/~01dfc0368aadcc8d4f
@aimane-chnaif and @adeel0202 please accept my job invite in Upwork, you both should've received an email.

Accepted, thanks.

@adeel0202 is paid ✅
Just waiting on @aimane-chnaif next ⏳

Both paid, closing.


Sys_error("export/debug/windows/obj/include//MainMenuState.h: Permission denied")

Describe your problem here.
I make psych engine modded and got this error:
Sys_error("export/debug/windows/obj/include//MainMenuState.h: Permission denied")
I turn off Avast and try again, but this no help
Are you modding a build from source or with Lua?
Source
What is your build target?
Windows x86/x32
Did you edit anything in this build? If so, mention or summarize your changes.
Yes, i edited Healthicon.hx and add winning icons and edited main menu,

It usually happens when the build bugs when it tries to compile the game, it's rare, it also happened to me, the only thing I did was download the source code again and compile it before making any changes.
I suggest that every time you make a big change you compile it.

It usually happens when the build bugs when it tries to compile the game, it's rare, it also happened to me, the only thing I did was download the source code again and compile it before making any changes. I suggest that every time you make a big change you compile it.

Thx u its works


Update dependencies to K8S v1.13.0-beta.1

What this PR does / why we need it:
This PR updates vendor files to K8S v1.13.0-beta.1.
Which issue this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged): fixes #
Special notes for your reviewer:
Release note:

Tested this and it works.


Publish explicit dependency version requirements

Using OpenCensus for Go with gRPC depends on recent versions of gRPC-Go. However, this dependency is never made explicit anywhere (such as a Gopkg.toml or go.mod file). I understand this is a bit in flux at the moment with the vgo proposal, but I think we should discuss how we make these version dependencies explicit, so I'm opening this issue for the purposes of that discussion.
Personally, I would welcome simply defining a Gopkg.toml and generate a Gopkg.lock and check those in right now. It would help users of golang/dep to get started with OpenCensus, and we could also include a fix for  https://github.com/census-instrumentation/opencensus-go/issues/675 in the dependency declaration.
Later on, as vgo is merged into the standard tool, we can move to defining a go.mod file.

I am happy with this approach as long as we don't vendor library dependencies.
/cc @odeke-em @ramonza for input.


gorm get datas by method order show error

table column

// order column

// have error
pq: column "createtime" does not exist
but i input column >>>> createTime

Maybe need double quote.
https://stackoverflow.com/questions/20878932/are-postgresql-column-names-case-sensitive
https://www.postgresql.org/docs/current/sql-syntax-lexical.html#SQL-SYNTAX-IDENTIFIERS
.Order(`"createTime desc"`)

thanks


Added Eloghosa to list

Can you please update your code by pulling the latest code from the original repo?

git pull upstream main

git push origin main


SWIG Required

Install fails on fresh yosemite. Needs to have SWIG added as dep.

Whoops, yeah, I overlooked build-only dependencies being skipped when using bottles.  It should be ok now.


Featuresupport update for Nereids

Proposed changes
Issue Number: close #xxx
Problem summary
support update unique-key table for Nereids and fix bugs:
failed to insert into a table with mv.
failed to insert into a table with not all the columns.
failed to insert into a table with complex select.
insert into a table with too many data.
Checklist(Required)

[ ] Does it affect the original behavior
[ ] Has unit tests been added
[ ] Has document been added or modified
[ ] Does it need to update dependencies
[ ] Is this PR support rollback (If NO, please explain WHY)

Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

run buildall

run buildall

run buildall

run buildall

run buildall

run buildall


How can I add 360 image without marzipano tool?

To add an image to an existing tour generated by the tool, edit the generated data.js file.
If you want to display a single preexisting panorama, it may be simpler to adapt one of the demos, e.g. cube-single-res or cube-multi-res.


Deploying without ssh

Hi! I know this template is in active development (I'm so excited for it!) but I already started using it this morning and I could not figure out how to set up the ssh keys.
This is not a complaint - I'm sure it's easy to figure out if your not as impatient as I am lol
Anyways, I wanted to leave this issue so that others who may have problems too can do what I did if they so want to.
This template is using peaceiris/gh-pages GitHub Action and perusing the repo I noticed that you can use a personal token instead of ssh and it works just fine.
I followed the instructions here https://github.com/peaceiris/actions-gh-pages/blob/master/README.md#-personal_token
Thanks for the awesome tool! fastpages and fast_template are amazing!

Did you try following the instructions on the README of this repo ?
https://developer.github.com/v3/guides/managing-deploy-keys/#deploy-keys
Did you set your deploy key to have write access?

By the way you should not use a Personal Access Token, as that is not as secure as a SSH deploy key.  If someone gets a hold of your PAT they can access all of your repos.  an SSH deploy key is scoped only to one repo, so that is why these instructions are here.
Closing this issue for now, feel free to make any more comments or ask any questions.  Please let us know how you are getting along and if you have any feedback

GitHub might have started to supporting to trigger the GitHub Pages build event by GITHUB_TOKEN on public repositories. (We can already use the token on private repositories.)
Be careful, there is no official announcement about this by GitHub.
- name: Deploy
  uses: peaceiris/actions-gh-pages@v3
  with:
    github_token: ${{ secrets.GITHUB_TOKEN }}
    publish_dir: ./_site

Thanks @peaceiris I will check into this!  🙇


Add util to update GCS metadata based on file suffix/type

In case of artifacts with website stored on GCS, it might be important to adjust metadata/content-type. We currently do this for the NotebookArtifact HTML report. This issue is to extend that to any objects within the artifact staging area. Should support:

HTML files
styles/css
javascript

This would great! Even if we just add support for future artifacts. The specific case I'm currently interested in is uploading a HTML file using articat dump.


Metrics Feedback Survey

Description
A feedback survey will be opened up to OSS maintainers and technical oversight members.
The feedback survey should stay open for at least two weeks, then responses will be aggregated into refinement steps.

The survey is now open here:
https://forms.gle/Cr2iSBUDK3nbEtSy6

The survey is now being disseminated by the CNCF TAG Security and the FNIOS TOC to project maintainers and advisors.


Where's SpotifyAPI.Local.....?

I just download by NuGet and I realized I did not found the folder :( Can someone help me?

It's deprecated and removed in version > 3
See #254


minor formatting improvements for CHANGES

for consistency with crate/crate#5187

Codecov Report

Merging #409 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #409   +/-   ##
=======================================
  Coverage   38.41%   38.41%
=======================================
  Files          20       20
  Lines        1658     1658
=======================================
  Hits          637      637
  Misses       1021     1021

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update bdc6118...c12fbe9. Read the comment docs.


Feature request: start background service after reboot and even if the program is killed

If the service is on going there is a way to start automatically the service after the reboot of the device?
If the service is on going there is a way to not stop it even if the main program is killed?
Have you implemented these features? (on Android)

Android has been refactored.

plugin no longer handles http; calls js callback (just like ios)
much improved battery efficiency.
no more foreground service

http://christocracy.github.io/cordova-background-geolocation/


Defining button bit masks as enum is a bad idea

engine/input.hpp defines the bit masks for each button in an enum:
  enum button {
    DPAD_LEFT = 1,
    DPAD_RIGHT = 2,
    DPAD_UP = 4,
    DPAD_DOWN = 8,
    A = 16,
    B = 32,
    X = 64,
    Y = 128,
    MENU = 256,
    HOME = 512,
    JOYSTICK = 1024
  };

  extern uint32_t buttons;

The type of an enum is undefined - it may be signed or unsigned. Usually is it signed. If you try to take the bitwise or of buttons and a button mask defined in the enum, this gives a compiler warning.
I think this should be exactly equivalent (in particular, it should use no memory):
  namespace button {
    static const uint32_t DPAD_LEFT = (1<<0);
    static const uint32_t DPAD_RIGHT = (1<<1);
    static const uint32_t DPAD_UP = (1<<2);
    static const uint32_t DPAD_DOWN = (1<<3);
    static const uint32_t A = (1<<4);
    static const uint32_t B = (1<<5);
    static const uint32_t X = (1<<6);
    static const uint32_t Y = (1<<7);
    static const uint32_t MENU = (1<<8);
    static const uint32_t HOME = (1<<9);
    static const uint32_t JOYSTICK = (1<<10);
  }

  extern uint32_t buttons;

Would enum button : unsigned int { ... }; not read better, or not valid here?

I believe typed enums are a C++11 feature, so that should be okay.

Note there is a difference between doing it with enum vs namespace: The enum also puts the names into the parent scope. So with enum you can do both blit::DPAD_LEFT and blit::buttons::DPAD_LEFT but with namespace you can only do blit::buttons::DPAD_LEFT.

A quick test of applying an unsigned int type to the enum suggests that it still compiles and functions as expected. I've pushed it for now, pending further discussion.


Training hangs at the end of the first epoch in image classification task.

Dear author:
When I training the Uniformer model with 8 GPUs, I start the code with the following run.sh:
work_path=$(dirname $0)
PYTHONPATH=$PYTHONPATH:../../ \
python -m torch.distributed.launch --nproc_per_node=8 --master_port=22335 --use_env main.py \
    --model uniformer_base \
    --batch-size 64 \
    --num_workers 8 \
    --drop-path 0.3 \
    --epoch 300 \
    --dist-eval \
    --output_dir ${work_path}/ckpt \
    2>&1 | tee -a ${work_path}/log.txt

And the logs are (I have deleted the display of model details):
/home/data/user/local/anaconda3/envs/uniformer/lib/python3.9/site-packages/torch/distributed/launch.py:163: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
  logger.warn(
The module torch.distributed.launch is deprecated and going to be removed in future.Migrate to torch.distributed.run
INFO:torch.distributed.launcher.api:Starting elastic_operator with launch configs:
  entrypoint       : main.py
  min_nodes        : 1
  max_nodes        : 1
  nproc_per_node   : 8
  run_id           : none
  rdzv_backend     : static
  rdzv_endpoint    : <IP_ADDRESS>:22335
  rdzv_configs     : {'rank': 0, 'timeout': 900}
  max_restarts     : 3
  monitor_interval : 5
  log_dir          : None
  metrics_cfg      : {}

INFO:torch.distributed.elastic.agent.server.local_elastic_agent:log directory set to: /tmp/torchelastic_yejsfquq/none_6nw5du9x
INFO:torch.distributed.elastic.agent.server.api:[default] starting workers for entrypoint: python
INFO:torch.distributed.elastic.agent.server.api:[default] Rendezvous'ing worker group
/home/data/user/local/anaconda3/envs/uniformer/lib/python3.9/site-packages/torch/distributed/elastic/utils/store.py:52: FutureWarning: This is an experimental API and will be changed in future.
  warnings.warn(
INFO:torch.distributed.elastic.agent.server.api:[default] Rendezvous complete for workers. Result:
  restart_count=0
  master_addr=<IP_ADDRESS>
  master_port=22335
  group_rank=0
  group_world_size=1
  local_ranks=[0, 1, 2, 3, 4, 5, 6, 7]
  role_ranks=[0, 1, 2, 3, 4, 5, 6, 7]
  global_ranks=[0, 1, 2, 3, 4, 5, 6, 7]
  role_world_sizes=[8, 8, 8, 8, 8, 8, 8, 8]
  global_world_sizes=[8, 8, 8, 8, 8, 8, 8, 8]

INFO:torch.distributed.elastic.agent.server.api:[default] Starting worker group
INFO:torch.distributed.elastic.multiprocessing:Setting worker0 reply file to: /tmp/torchelastic_yejsfquq/none_6nw5du9x/attempt_0/0/error.json
INFO:torch.distributed.elastic.multiprocessing:Setting worker1 reply file to: /tmp/torchelastic_yejsfquq/none_6nw5du9x/attempt_0/1/error.json
INFO:torch.distributed.elastic.multiprocessing:Setting worker2 reply file to: /tmp/torchelastic_yejsfquq/none_6nw5du9x/attempt_0/2/error.json
INFO:torch.distributed.elastic.multiprocessing:Setting worker3 reply file to: /tmp/torchelastic_yejsfquq/none_6nw5du9x/attempt_0/3/error.json
INFO:torch.distributed.elastic.multiprocessing:Setting worker4 reply file to: /tmp/torchelastic_yejsfquq/none_6nw5du9x/attempt_0/4/error.json
INFO:torch.distributed.elastic.multiprocessing:Setting worker5 reply file to: /tmp/torchelastic_yejsfquq/none_6nw5du9x/attempt_0/5/error.json
INFO:torch.distributed.elastic.multiprocessing:Setting worker6 reply file to: /tmp/torchelastic_yejsfquq/none_6nw5du9x/attempt_0/6/error.json
INFO:torch.distributed.elastic.multiprocessing:Setting worker7 reply file to: /tmp/torchelastic_yejsfquq/none_6nw5du9x/attempt_0/7/error.json
Please update your PyTorchVideo to latest master
Please update your PyTorchVideo to latest master
Please update your PyTorchVideo to latest master
Please update your PyTorchVideo to latest master
Please update your PyTorchVideo to latest master
Please update your PyTorchVideo to latest master
Please update your PyTorchVideo to latest master
Please update your PyTorchVideo to latest master
| distributed init (rank 3): env://
| distributed init (rank 6): env://
| distributed init (rank 1): env://
| distributed init (rank 0): env://
| distributed init (rank 5): env://
| distributed init (rank 2): env://
| distributed init (rank 7): env://
| distributed init (rank 4): env://
[W ProcessGroupNCCL.cpp:1569] Rank 0 using best-guess GPU 0 to perform barrier as devices used by this process are currently unknown. This can potentially cause a hang if this rank to GPU mapping is incorrect.Specify device_ids in barrier() to force use of a particular device.
[W ProcessGroupNCCL.cpp:1569] Rank 4 using best-guess GPU 4 to perform barrier as devices used by this process are currently unknown. This can potentially cause a hang if this rank to GPU mapping is incorrect.Specify device_ids in barrier() to force use of a particular device.
[W ProcessGroupNCCL.cpp:1569] Rank 5 using best-guess GPU 5 to perform barrier as devices used by this process are currently unknown. This can potentially cause a hang if this rank to GPU mapping is incorrect.Specify device_ids in barrier() to force use of a particular device.
[W ProcessGroupNCCL.cpp:1569] Rank 3 using best-guess GPU 3 to perform barrier as devices used by this process are currently unknown. This can potentially cause a hang if this rank to GPU mapping is incorrect.Specify device_ids in barrier() to force use of a particular device.
[W ProcessGroupNCCL.cpp:1569] Rank 2 using best-guess GPU 2 to perform barrier as devices used by this process are currently unknown. This can potentially cause a hang if this rank to GPU mapping is incorrect.Specify device_ids in barrier() to force use of a particular device.
[W ProcessGroupNCCL.cpp:1569] Rank 6 using best-guess GPU 6 to perform barrier as devices used by this process are currently unknown. This can potentially cause a hang if this rank to GPU mapping is incorrect.Specify device_ids in barrier() to force use of a particular device.
[W ProcessGroupNCCL.cpp:1569] Rank 1 using best-guess GPU 1 to perform barrier as devices used by this process are currently unknown. This can potentially cause a hang if this rank to GPU mapping is incorrect.Specify device_ids in barrier() to force use of a particular device.
[W ProcessGroupNCCL.cpp:1569] Rank 7 using best-guess GPU 7 to perform barrier as devices used by this process are currently unknown. This can potentially cause a hang if this rank to GPU mapping is incorrect.Specify device_ids in barrier() to force use of a particular device.
Warning: Enabling distributed evaluation with an eval dataset not divisible by process number. This will slightly alter validation results as extra duplicate entries are added to achieve equal num of samples per-process.
Creating model: uniformer_base
number of params: 49468752
Start training for 300 epochs
[W reducer.cpp:1158] Warning: find_unused_parameters=True was specified in DDP constructor, but did not find any unused parameters in the forward pass. This flag results in an extra traversal of the autograd graph every iteration,  which can adversely affect performance. If your model indeed never has any unused parameters in the forward pass, consider turning this flag off. Note that this warning may be a false positive if your model has flow control causing later iterations to have unused parameters. (function operator())
[W reducer.cpp:1158] Warning: find_unused_parameters=True was specified in DDP constructor, but did not find any unused parameters in the forward pass. This flag results in an extra traversal of the autograd graph every iteration,  which can adversely affect performance. If your model indeed never has any unused parameters in the forward pass, consider turning this flag off. Note that this warning may be a false positive if your model has flow control causing later iterations to have unused parameters. (function operator())
[W reducer.cpp:1158] Warning: find_unused_parameters=True was specified in DDP constructor, but did not find any unused parameters in the forward pass. This flag results in an extra traversal of the autograd graph every iteration,  which can adversely affect performance. If your model indeed never has any unused parameters in the forward pass, consider turning this flag off. Note that this warning may be a false positive if your model has flow control causing later iterations to have unused parameters. (function operator())
[W reducer.cpp:1158] Warning: find_unused_parameters=True was specified in DDP constructor, but did not find any unused parameters in the forward pass. This flag results in an extra traversal of the autograd graph every iteration,  which can adversely affect performance. If your model indeed never has any unused parameters in the forward pass, consider turning this flag off. Note that this warning may be a false positive if your model has flow control causing later iterations to have unused parameters. (function operator())
[W reducer.cpp:1158] Warning: find_unused_parameters=True was specified in DDP constructor, but did not find any unused parameters in the forward pass. This flag results in an extra traversal of the autograd graph every iteration,  which can adversely affect performance. If your model indeed never has any unused parameters in the forward pass, consider turning this flag off. Note that this warning may be a false positive if your model has flow control causing later iterations to have unused parameters. (function operator())
[W reducer.cpp:1158] Warning: find_unused_parameters=True was specified in DDP constructor, but did not find any unused parameters in the forward pass. This flag results in an extra traversal of the autograd graph every iteration,  which can adversely affect performance. If your model indeed never has any unused parameters in the forward pass, consider turning this flag off. Note that this warning may be a false positive if your model has flow control causing later iterations to have unused parameters. (function operator())
[W reducer.cpp:1158] Warning: find_unused_parameters=True was specified in DDP constructor, but did not find any unused parameters in the forward pass. This flag results in an extra traversal of the autograd graph every iteration,  which can adversely affect performance. If your model indeed never has any unused parameters in the forward pass, consider turning this flag off. Note that this warning may be a false positive if your model has flow control causing later iterations to have unused parameters. (function operator())
[W reducer.cpp:1158] Warning: find_unused_parameters=True was specified in DDP constructor, but did not find any unused parameters in the forward pass. This flag results in an extra traversal of the autograd graph every iteration,  which can adversely affect performance. If your model indeed never has any unused parameters in the forward pass, consider turning this flag off. Note that this warning may be a false positive if your model has flow control causing later iterations to have unused parameters. (function operator())
Epoch: [0]  [0/9]  eta: 0:01:24  lr: 0.000001  loss: 6.0915 (6.0915)  time: 9.3436  data: 3.3803  max mem: 11905
Epoch: [0]  [8/9]  eta: 0:00:02  lr: 0.000001  loss: 6.0828 (6.0922)  time: 2.0037  data: 0.3758  max mem: 12141
Epoch: [0] Total time: 0:00:18 (2.0372 s / it)
Averaged stats: lr: 0.000001  loss: 6.0828 (6.0883)
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)

The displayed logs are very messy and maddening, and I am totally no clue whether the code runs correctly. These warnings only occur when distributed training.  I will appreciate it if you can give me some advice.

I also met these warnings. But it does not affect the performance, just ignore it~

You can simply check the stdout.log. It will not show the warnings.

Thanks for your kindly response.

However, the training really hang :(
[E ProcessGroupNCCL.cpp:325] Some NCCL operations have failed or timed out. Due to the asynchronous nature of CUDA kernels, subsequent GPU operations might run on corrupted/incom
plete data. To avoid this inconsistency, we are taking the entire process down.
terminate called after throwing an instance of 'std::runtime_error'
  what():  [Rank 1] Watchdog caught collective operation timeout: WorkNCCL(OpType=BROADCAST, Timeout(ms)=1800000) ran for 1802691 milliseconds before timing out.
[E ProcessGroupNCCL.cpp:325] Some NCCL operations have failed or timed out. Due to the asynchronous nature of CUDA kernels, subsequent GPU operations might run on corrupted/incom
plete data. To avoid this inconsistency, we are taking the entire process down.
terminate called after throwing an instance of 'std::runtime_error'
  what():  [Rank 6] Watchdog caught collective operation timeout: WorkNCCL(OpType=BROADCAST, Timeout(ms)=1800000) ran for 1802719 milliseconds before timing out.
[E ProcessGroupNCCL.cpp:325] Some NCCL operations have failed or timed out. Due to the asynchronous nature of CUDA kernels, subsequent GPU operations might run on corrupted/incom
plete data. To avoid this inconsistency, we are taking the entire process down.
terminate called after throwing an instance of 'std::runtime_error'
  what():  [Rank 4] Watchdog caught collective operation timeout: WorkNCCL(OpType=BROADCAST, Timeout(ms)=1800000) ran for 1802704 milliseconds before timing out.
[E ProcessGroupNCCL.cpp:325] Some NCCL operations have failed or timed out. Due to the asynchronous nature of CUDA kernels, subsequent GPU operations might run on corrupted/incom
plete data. To avoid this inconsistency, we are taking the entire process down.
terminate called after throwing an instance of 'std::runtime_error'
  what():  [Rank 5] Watchdog caught collective operation timeout: WorkNCCL(OpType=BROADCAST, Timeout(ms)=1800000) ran for 1802712 milliseconds before timing out.
[E ProcessGroupNCCL.cpp:325] Some NCCL operations have failed or timed out. Due to the asynchronous nature of CUDA kernels, subsequent GPU operations might run on corrupted/incom
plete data. To avoid this inconsistency, we are taking the entire process down.
terminate called after throwing an instance of 'std::runtime_error'
  what():  [Rank 3] Watchdog caught collective operation timeout: WorkNCCL(OpType=BROADCAST, Timeout(ms)=1800000) ran for 1802694 milliseconds before timing out.
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: -6) local_rank: 0 (pid: 74124) of binary: /home/data/user/local/anaconda3/envs/uniformer/bin/python
ERROR:torch.distributed.elastic.agent.server.local_elastic_agent:[default] Worker group failed

It seems that you should increase share memory?

See here. You can try to use a smaller batch size if you train UniFormer-B.

Thanks for your advice first.
However, it might not be the reason of the shared memory. My server has a shared memory of more than 30G. I have watched the usage of shared memory when training, but only less than 500M is used throughout the whole stage.
I have also tried many different possible solutions from the Internet but never worked.
The training also usually to be stuck at the end of the first epoch, and before or after the end of the last batch:
Epoch: [0]  [1234/1237]  eta: 0:00:01  lr: 0.000001  loss: 4.8091 (5.1428)  time: 0.6466  data: 0.0003  max mem: 12501
Epoch: [0]  [1235/1237]  eta: 0:00:01  lr: 0.000001  loss: 4.8209 (5.1427)  time: 0.6463  data: 0.0003  max mem: 12501

or
Epoch: [0]  [1234/1237]  eta: 0:00:01  lr: 0.000001  loss: 4.8091 (5.1428)  time: 0.6466  data: 0.0003  max mem: 12501
Epoch: [0]  [1235/1237]  eta: 0:00:01  lr: 0.000001  loss: 4.8209 (5.1427)  time: 0.6463  data: 0.0003  max mem: 12501
Epoch: [0]  [1236/1237]  eta: 0:00:00  lr: 0.000001  loss: 4.8091 (5.1423)  time: 0.6451  data: 0.0003  max mem: 12501
Epoch: [0] Total time: 0:13:13 (0.6412 s / it)

I have debugged that for several days but still failed to run the training.

About the messy warnings:
It might be the reason of the version of PyTorch. I reinstall my virtual environment of PyTorch 1.7.0, and all warnings and errors are disappeared. However, instead of being interrupted, the training is always stuck at the end of the first epoch.

There is another strange case worth noticing.
When I use a smaller dataset (about 1000 images) to debug, anything goes well.
And when I add
debug += 1
if debug > 1000:
     break

at Line33
https://github.com/Sense-X/UniFormer/blob/e8024703bffb89cb7c7d09e0d774a0d2a9f96c25/image_classification/engine.py#L30-L33
anything goes well.
I really don't know what's wrong with my training.

It looks so strange. I never meet this bug....
Since my code is forked from DeiT, maybe you can try it and open an issue if you meet the same problem. I just add my model and adopt the same training settings.

Maybe you can check the CUDA, CUDNN, Pytorch, Torchvision versions....
Recently, I have met some strange bugs when using mismatched torch and torchvision.
I suggest you try DeiT first and check the log~~

It's really strange, and I have never met similar questions before. I have debugged the code for several days but still failed to run the network.

I have tried Pytorch 1.7 or Pytorch 1.9 for Cuda 11. Considering the DeiT is an earlier work, using Pytorch 1.9 or Cuda 11 might not be a good idea? Did you remember your versions of CUDA and CUDNN?

Come on! Try DeiT and see if you will meet the same bugs.
If so, maybe you need to create a new environment. If not, you can simply copy the model code and run it!

In my environment, torch1.6-1.10 is okay for DeiT.
Now I use torch1.9 + CUDA11.1 + cudnn8.0.4.
I use torch1.6 + CUDA10.1 + cudnn7.6.5 before.

Hi! Have you solved your problem?

Thanks a lot. I had solved the problem by following your kind advice.


Increase width of ə˞

Consider increasing the width of rhotacized ə˞. Here’s the string kənˈsɪdə˞d from IPA tests rendered in Roboto-Regular and Roboto-Italic:

Compare to Adobe Source Sans Pro:

What version of Roboto are you using to test this? The latest release should be fine (and thus the Android tag is inappropriate).

The version that Google publishes on GitHub. If that repo isn’t kept current, perhaps it would be best to delete it? But where to find the actual source?

Those aren't published fonts, they're the sources we have to use for the actual published hinted fonts (which are at https://github.com/google/roboto/releases) since afaik there's no way to store hinting data in a UFO.
I think there's been confusion about this before, so maybe I'll rename that "hinted" directory.


Add default users

Coverage remained the same at 99.385% when pulling bb53a4c81b05a2cec7de87bef074cbfc9ee7ffb2 on development into 401c671230a940ca8e79137e55fdbf7ee02161f5 on master.


I want to know how to prepare my own data?

I  know for example spk2utt utt2spk  wav.scp and so on ,I want to know i need to create rttm and segments file?is segments file created by SAD?

You need segments, reco2dur, wav.scp, utt2spk and spk2utt. The segments can be generated by SAD when you have two-speaker conversations in separate channels.
Our training data generated by running run_prepare_shared.sh are “simulated” mixtures, where segments are generated according to its random simulation.
rttm is required for scoring you should prepare it for test data.

sorry ,I still have some places that don’t understand. I only have some two-speaker conversations video, for example , it is named speech.wav which contains two speakers.
I want to know that  the segments is created directly  from speech.wav depending on sad? or i need to create segments which only contain one speakers by manual?

If you want to prepare your own "training" data, you need to create segments as reference labels.
It should be created manually when you only have two-speaker mixtures in monaural recordings.

However, we have not yet released such a pre-trained model. We have no easy-to-use tools to perform inference only.


Adjust calculation for floating label height to support scalable text on iOS

Fixes https://github.com/flutter/flutter/issues/14583

Actually not quite ready yet.


BUG: Memory leak in function sf_error due to new reference is not decreased (static analyzer report)

Local variable scipy_special is declared and assigned to NULL.
https://github.com/scipy/scipy/blob/8627df31ab07d66b438c7e5d5880e3f4d435f248/scipy/special/sf_error.c#L55

A new reference is returned and assigned to scipy_special.
https://github.com/scipy/scipy/blob/8627df31ab07d66b438c7e5d5880e3f4d435f248/scipy/special/sf_error.c#L93

Assume the scipy_special is not NULL.
https://github.com/scipy/scipy/blob/8627df31ab07d66b438c7e5d5880e3f4d435f248/scipy/special/sf_error.c#L94

The PyObject is used then but its refcnt is never decreased before the exit of the function. (all other usages)
https://github.com/scipy/scipy/blob/8627df31ab07d66b438c7e5d5880e3f4d435f248/scipy/special/sf_error.c#L101
https://github.com/scipy/scipy/blob/8627df31ab07d66b438c7e5d5880e3f4d435f248/scipy/special/sf_error.c#L105

Internal Report ID: 1917e6

Thank you for the test case to trigger this defect.


Normalization breaks array parameters with unnamed keys in query string

During the normalization, arrays in the query string are collapsed into a single value if the keys are unnamed.
Example:
$url = "http://www.example.com/?foo[]=bar&foo[]=baz";
$un = new Normalizer($url);
echo "Expected: http://www.example.com/?foo%5B%5D=bar&foo%5B%5D=baz\n";
echo "Actual  : ".$un->normalize()."\n\n";

// Ouptut:
// Expected: http://www.example.com/?foo%5B%5D=bar&foo%5B%5D=baz
// Actual  : http://www.example.com/?foo%5B%5D=baz

The problem arises from the custom parseStr implementation which doesn't take that case into account:
   /**
     * Replacement for PHP's parse_string which does not deal with spaces or dots in key names
     *
     * @param string $string URL query string
     * @return array key value pairs
     */
     private function parseStr( $string ) {
        $params = array();

        $pairs = explode( '&', $string );

        foreach ( $pairs as $pair ) {
            $var = explode( '=', $pair, 2 );
            $params[$var[0]] = ( isset( $var[1] ) ? $var[1] : '' );
        }

        return $params;
    }

Using PHP internal parse_str fixes this issue but will break if keys in the query have a whitespace or a dot in it (those will be replaced by an underscore), see http://php.net/manual/de/function.parse-str.php#76978
Which, I guess, was the original intention for using a custom function for query string parsing :)
Anyway, this should definitely be fixed although I'm not sure if there's an easy way to avoid using parse_str and keep up with any edge cases at the same time...
Cheers
Pascal

Hey Glen,
did you have a moment to look into this or did I just miss your comment?
Cheers
Pascal

Hey @paslandau -- I did reply to you here:
https://github.com/glenscott/url-normalizer/issues/17#issuecomment-65086836
Did you have any other questions that I haven't answered?
Glen

Hey @glenscott ,
I was referring to

(I'll answer your other queries in separate comments later)

:) You've "only" explained the overriding behavior, but did not go into the other questions:

removing trailing "&"
ordering of parameters

Further I was wondering if you'd be open to make some of those points configurable. Reason being is, I would like to use the normalizer to make my web crawler more efficient (by avoiding duplicate URLs). I'm mostly scraping known websites (so I know they would override duplicate parameters etc.) and it would be great to make the normalizer aware of that.
Cheers
Pascal

Hi @paslandau , apologies for not finishing my reply :)
With regards to your first point, I've been re-reading the RFC 3986 specification, and I've found a paragraph that suggests there's a bug in my implementation:

the URI "http://example.com/?" cannot be assumed to be equivalent to "http://example.com/"

https://tools.ietf.org/html/rfc3986#section-6.2.3
In any case, yes, I think making removing trailing ? and & should be a optional (default is do not remove them).
There is nothing in the specification about reordering query string parameters, but again I think an option to re-order them would be good.

Hey @glenscott
great, looking forward to it :)

Hey @paslandau -- I've added options to remove empty delimiters and reorder query parameters.  Please take a look and let me know if it's working well for you.

Hi @paslandau  -- have you had a chance to look through the updated code yet?

Hi @glenscott ,
finally came around to test your fixes (on current dev-master branch) - sorry for the delay.
Ordering looks pretty good, trimming the delimiter as well. Thx!
The only thing I noticed is that a trailing ampersand will be removed regardless of the setting fo $remove_empty_delimiters. Same holds true for multiple ampersands without key/values pairs.
Code to reproduce:
            $url = "http://example.com/?foo=baz&";
            $remove_empty_delimiters = false;
            $sort_query_params = false;

            $normalizer = new Normalizer($url,$remove_empty_delimiters,$sort_query_params);
            echo $normalizer->normalize();
            // Output: http://example.com/?foo=baz
            // Expected: http://example.com/?foo=baz&

            $url = "http://example.com/?foo=baz&&&bar";
            $remove_empty_delimiters = false;
            $sort_query_params = false;

            $normalizer = new Normalizer($url,$remove_empty_delimiters,$sort_query_params);
            echo $normalizer->normalize();
            // Output: http://example.com/?foo=baz&bar
            // Expected: http://example.com/?foo=baz&&&bar

My full test suite:
    public function test_normalizeUrl(){

        $tests = [
            "no-value" =>
                [
                    "base" => "http://example.com/?foo[]",
                    "expected" => "http://example.com/?foo%5B%5D"

                ],
            "value" =>
                [
                    "base" => "http://example.com/?foo[]=bar",
                    "expected" => "http://example.com/?foo%5B%5D=bar"

                ],
            "2-no-value" =>
                [
                    "base" => "http://example.com/?foo[]&foo[]",
                    "expected" => "http://example.com/?foo%5B%5D&foo%5B%5D"
                ],
            "2-value" =>
                [
                    "base" => "http://example.com/?foo[]=bar&foo[]=baz",
                    "expected" => "http://example.com/?foo%5B%5D=bar&foo%5B%5D=baz"
                ],
            "value-no-value" =>
                [
                    "base" => "http://example.com/?foo[]=bar&foo[]",
                    "expected" => "http://example.com/?foo%5B%5D=bar&foo%5B%5D"
                ],
            "whitespace-key" =>
                [
                    "base" => "http://example.com/?f oo[]=bar&f oo[]",
                    "expected" => "http://example.com/?f%20oo%5B%5D=bar&f%20oo%5B%5D"
                ],
            "dot-key" =>
                [
                    "base" => "http://example.com/?f.oo[]=bar&f.oo[]",
                    "expected" => "http://example.com/?f.oo%5B%5D=bar&f.oo%5B%5D"
                ],
            "dot-key-whitespace-key" =>
                [
                    "base" => "http://example.com/?f.oo[]=bar&f oo[]",
                    "expected" => "http://example.com/?f.oo%5B%5D=bar&f%20oo%5B%5D"
                ],
            "dot-name-key-whitespace-name-key" =>
                [
                    "base" => "http://example.com/?f.oo[foo]=bar&f oo[2]",
                    "expected" => "http://example.com/?f.oo%5Bfoo%5D=bar&f%20oo%5B2%5D"
                ],
            "nested-arrays" =>
                [
                    "base" => "http://example.com/?f.oo[foo.1][foo.2][foo.3]=foo-val&f.oo[foo.1][foo.2][foo.4]=foo-val2",
                    "expected" => "http://example.com/?f.oo%5Bfoo.1%5D%5Bfoo.2%5D%5Bfoo.3%5D=foo-val&f.oo%5Bfoo.1%5D%5Bfoo.2%5D%5Bfoo.4%5D=foo-val2"
                ],
            "ordering-key-true" =>
                [
                    "base" => "http://example.com/?b_foo=a&a_foo=a",
                    "orderParams" => true,
                    "expected" => "http://example.com/?a_foo=a&b_foo=a"
                ],
            "ordering-values-true" =>
                [
                    "base" => "http://example.com/?foo=b&foo=a",
                    "orderParams" => true,
                    "expected" => "http://example.com/?foo=b&foo=a"
                ],
            "ordering-key-false" =>
                [
                    "base" => "http://example.com/?b_foo=a&a_foo=a",
                    "orderParams" => false,
                    "expected" => "http://example.com/?b_foo=a&a_foo=a"
                ],
            "ordering-values-false" =>
                [
                    "base" => "http://example.com/?foo=b&foo=a",
                    "orderParams" => false,
                    "expected" => "http://example.com/?foo=b&foo=a"
                ],
            //This test fails --- multiple ampersands should not be removed
            "multiple-ampersand" =>
                [
                    "base" => "http://example.com/?foo=baz&&&bar=baz",
                    "removeTrailing" => false,
                    "expected" => "http://example.com/?foo=baz&&&bar=baz"
                ],
            "multiple-ampersand-remove" =>
                [
                    "base" => "http://example.com/?foo=baz&&&bar=baz",
                    "removeTrailing" => true,
                    "expected" => "http://example.com/?foo=baz&bar=baz"
                ],
            "trailing-ampersand-remove" =>
                [
                    "base" => "http://example.com/?foo=baz&",
                    "removeTrailing" => true,
                    "expected" => "http://example.com/?foo=baz"
                ],
            "trailing-question-mark-remove" =>
                [
                    "base" => "http://example.com/?",
                    "removeTrailing" => true,
                    "expected" => "http://example.com/"
                ],
            "trailing-question-mark" =>
                [
                    "base" => "http://example.com/?",
                    "removeTrailing" => false,
                    "expected" => "http://example.com/?"
                ],
            "nested-arrays-order" =>
                [
                    "base" => "http://example.com/?f.oo[foo.1][foo.2][foo.4]=foo-val2&f.oo[foo.1][foo.2][foo.3]=foo-val",
                    "orderParams" => true,
                    "expected" => "http://example.com/?f.oo%5Bfoo.1%5D%5Bfoo.2%5D%5Bfoo.3%5D=foo-val&f.oo%5Bfoo.1%5D%5Bfoo.2%5D%5Bfoo.4%5D=foo-val2"
                ],
            //This test fails --- the trailing ampersand should not be removed
            "trailing-ampersand" =>
                [
                    "base" => "http://example.com/?foo=baz&",
                    "removeTrailing" => false,
                    "expected" => "http://example.com/?foo=baz&"
                ],
            // fails due to trailing ampersand being removed
//            "crazy-stuff" =>
//                [
//                    "base" => "http://example.com/?f.o  o[foo]=bar&f oo[2]&",
//                    "expected" => "http://example.com/?f.o%20%20o%5Bfoo%5D=bar&f%20oo%5B2%5D&"
//                ],
            //we cannot assume that values are overriden!
//            "overriding-values" =>
//                [
//                    "base" => "http://example.com/?foo=bar&foo=baz",
//                    "expected" => "http://example.com/?foo=baz"
//                ],
        ];

        foreach($tests as $name => $data){

            $removeTrailingDelimiter = array_key_exists("removeTrailing",$data) ? $data["removeTrailing"] : false;
            $sortQueryParams = array_key_exists("orderParams",$data) ? $data["orderParams"] : false;
            $url = $data["base"];
            $normalizer = new Normalizer($url,$removeTrailingDelimiter,$sortQueryParams);
            $res = $normalizer->normalize();
            $msg = [
                "Error in test $name:",
                "Input    : ".$data["base"],
                "Excpected: ".json_encode($data["expected"]),
                "Actual   : ".json_encode($res),
            ];
            $msg = implode("\n",$msg);
            $this->assertEquals($data["expected"],$res,$msg);
        }

Cheers
Pascal

Thanks for the feedback, Pascal.
Preserving empty query string separators will introduce an additional amount of complexity to the underlying code, so I'm wary of adding it.  I'm also skeptical about its usefulness, unless you can persuade me otherwise :)
The feature I added was specifically for delimiters (e.g. ?) and not separators (e.g. & in a query string) which I think will address most people's needs.
Let me know your thoughts.

Hey @glenscott
to be blunt, I don't really see a use-case for preserving multiple ampersands so I don't mind the current behavior. I just wasn't aware that 'delimiter' only inlcudes '?' but not '&' - but I feel it won't hurt to mention that in the docu / doc comment ;)
Apart from that: Do you plan to release a new version soon? The current stable (1.3) doesn't include the the delimiter-removal option :)
Cheers
Pascal

Hey @paslandau
I'll update the docs, but in the meantime v1.3.0 should have the delimited removal option unless I am mistaken?  https://github.com/glenscott/url-normalizer/releases/tag/v1.3.0
Glen

Hey @glenscott
you're right, it was the $sort_query_params option that was missing in v1.3 but is present in dev-master.
I need both for my tests to run through :) So the question about the next version release remains ;)
Cheers
Pascal

Done https://github.com/glenscott/url-normalizer/releases/tag/1.4.0

Great, thx!


Implement actions artifacts

Implement action artifacts server api.
This change is used for supporting https://github.com/actions/upload-artifact and https://github.com/actions/download-artifact in gitea actions. It can run sample workflow from doc https://docs.github.com/en/actions/using-workflows/storing-workflow-data-as-artifacts. The api design is inspired by https://github.com/nektos/act/blob/master/pkg/artifacts/server.go and includes some changes from gitea internal structs and methods.
Actions artifacts contains two parts:

Gitea server api and storage (this pr implement basic design without some complex cases supports)
Runner communicate with gitea server api (in comming)

Now this pr follows https://github.com/go-gitea/gitea/pull/21937. I will work on this pr when https://github.com/go-gitea/gitea/pull/21937 updated until merged.

Should this use the package registry to benefit from some of the features (deduplication, auto-cleanup, ...)?


DLSS for RE4

"Hello Praydog! Thank you for your work that you do for us gamers. To be inside our favorite games is probably what every game lover dreams about. I've played many RE releases with your projects, but after installing your mod on RE4, I can't get over the potato-level graphics, everything is very blurry and soapy, lacking that crispness, when you want to see clearly and have a sharp picture. I have an RTX 3090 graphics card, 64GB RAM. I've read a lot about how people used your DEV files to install DLSS libraries, and as a result, they got clearer images and an increase in FPS. Now, I can't find those files of yours anymore, and what I found doesn't work, RE4 won't start. I understand that you have very little time, but could you give me some advice on how to improve the picture quality in VR? I would be very grateful to you. Thank you very much.
my email<EMAIL_ADDRESS>

By the way, please add motion control too :)

https://github.com/praydog/REFramework/actions
Download pd-upscaler builds. Necessary instructions for RE4 upscaler installation and dependency downloads are on its NexusMods page.


Categorical color palettes

If passing two HEX colors as the palette, the returned thumbnail or tiles interpolate the values between those two colors.
Is there a way to use the color maps as categorical colors? e.g., split the min/max range into the number of colors in the palette and assign those values directly to the given colors.

For example, if I use the palette ["#000", "#f00"], this right now interpolates the values between those colors. This is great most of the time, but what if I want the colors to be either "#000" or "#f00" and nothing else?
Current resulting thumbnail:

Example tile result:

But I actually want something like:

The naming "scheme": "discrete" is inherited from the mapnik source.  We could make aliases so categorical could b used instead of discrete.


Status of a code review comment

Regarding: https://docs.microsoft.com/en-us/azure/devops/repos/git/pullrequest?view=azure-devops#how-does-the-code-review-work
The possible statuses for a comment are: Active, Pending, Resolved, Won't Fix and Closed.  I think it would be great to add some documentation giving examples of when you'd use each status (personally, I'm not clear on when to use "Closed").

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: eea32d3a-53e9-3e21-d419-6e8c8c055ef0
Version Independent ID: 4942fd13-34c2-a2ef-5e2d-fff913a397ed
Content: Create a pull request - Azure Repos
Content Source: docs/repos/git/pullrequest.md
Product: devops
Technology: devops-code-git
GitHub Login: @apawast
Microsoft Alias: sdanie

Yeah, I found this issue from a web search for explanation of what the statuses mean.
Documentation would be nice. Progressively-disclosed documentation in-app would be even better!

This briefly explains it. Although, adding details about which statuses block autocompletion and which don't would be an improvement.


added Ganglia relation class

Merged with trivial modification (reduced indentation of the "return None" line).


update hasura to v1.0.0-beta.3

Description of PR
Update hasura to v1.0.0-beta.3
What component(s) does this PR affect?

[ ] Back-End (Database, Microservices, Containers, etc)
[X] Front-End (Flask, API, etc)
[ ] Docs
[ ] Build System

Does the PR require changes on other components? If yes, please mark the components:

[ ] Back-End (Database, Microservices, Containers, etc)
[ ] Front-End (Flask, API, etc)
[ ] Docs
[ ] Build System

Related Issue

Resolves #
Solution

Type

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] Docs update
[ ] None of the above

Checklist:

[ ] I have read the contributing guide and my code conforms to the guidelines.
[ ] This change requires a change in the documentation.
[ ] I have updated the documentation accordingly.

Do not merge until go-ahead from @slowr


RSOP does not contain consolidated LcmConfig data

When running the Build script, the RSOP output only contains the LcmConfig data that is included in the YAML file of the node itself. The data from the DscBaseline.yml file is missing:
https://github.com/dsccommunity/DscWorkshop/blob/935d168cad1546c4bf37db23f0735380d2d9866d/DSC/DscConfigData/Roles/DscBaseline.yml#L38-L54
Example of DSCFile01.yml
  MaxLcmRuntime: 00:30:00
LcmConfig:
  ConfigurationRepositoryWeb:
    Server:
      ConfigurationNames: DSCFile01
PSDscAllowDomainUser: true

Would be great if these config items would also be included in the RSOP data.

Right, this has bothered me quite a few times as well. I am looking into it...

Closing this issue and tracking the fix in gaelcolas/datum#95.
Thanks for reporting this, @ykuijs.


Verifiy what happens to billing pakages after the TFT amount decreased

while renting a node yo get offer packages depends on the TFT amoun balance (silver, gold, ...)
its not clear enough to the user that this packages are hourly calculated which means in the next hout if the amount of TFT decreased the user will be billed with the default price.
this must be clear in the package information for the user as user expect the discount for a month not hourly.

https://github.com/threefoldtech/tfgrid-sdk-ts/issues/1879

https://github.com/threefoldtech/tfgrid-sdk-ts/issues/1880

Verified,

Devnet
2.3.0-alpha5.
TC2453 - Discount package

I tried with a 4000 TFT balance and had a discount on the node while renting it.

Then I transferred all the balance, and the node value was brought back to its original value.

And after that, I increased the balance more (7000 TFT) and had a higher discount than the first time.

"its not clear enough to the user that this packages are hourly calculated which means in the next hout if the amount of TFT decreased the user will be billed with the default price."
Very good point.
Two things to add here:
First, we could then add a clear statement on this fact in the manual, here. I will create an issue in info_grid.
Second, there is a proposition to add a calculator to take into account hourly calculated discount, see #773. This would make it easy for users to check the discount available.

The two issues, https://github.com/threefoldtech/tfgrid-sdk-ts/issues/1879 and https://github.com/threefoldtech/tfgrid-sdk-ts/issues/1880 were verified and closed.
And the PR is merged.


release: fitsio-sys v0.5.3

This is a release PR for version 0.5.3.
Use a merge commit.
To trigger builds initially, close and then immediately re-open this PR once.

coverage: 76.716%. remained the same when pulling e90821244f1e0c28d544cb6eb24fa10f55aac19f on release/fitsio-sys/0.5.3 into c3d02a7815d9473f79282f943602dc0db48eec3f on main.


Add support for stored-procedures

Stored Procedure calls are not being tested in the spec suite. This is a popular feature, and should be tested.

@talevy is there a way to do this with the current framework, or will this require integration testing against a different database (MySQL, PostgreSQL, etc)?

@untergeek to properly test verify these work, integration testing against specific venders would have to be done I believe.

Putting in a request for SQL Server 2014

I do not know that we can effectively do integration testing against SQL Server, as there are licensing constraints.  However, stored procedure testing should demonstrate proof of concept for any supported JDBC database.  This is what I'm working on right now.

@talevy are you using the plugin with a stored procedure call? Do you ever use the --debug switch? We're having issues with using a stored proc and logstash debug mode which causes this plugin to attempt to log the count by trying to execute something like the following which isn't valid SQL
SELECT TOP (1) count(*) AS [COUNT] FROM ({ call [dbo].[stored_proc_name] (1)}) AS [T1]

@mikeholczer, I have not done much work with this plugin and stored procedures so I cannot attest to any expected behaviors yet

I had a support case where one of my customer wanted to issue alter session in oracle and I had to check if it can be done.
For my testing (I used MySQL) I declared a variable in first input clause and in 2nd input clause  I referenced it.
But the variable came out null in elasticsearch.
I guess it is due to simple fact that each input1 clause becomes a new session.
However using stored procedure worked for me.
But having multiple select statement such as this ends up in only getting data from 1st input statement, which I suppose is the limitation.
delimiter '$';
create procedure all_pet() begin select name from pet; select name from pet2; end$

Confirmed stored procedures cannot be run with --debug mode enabled for logstash. Please resolve. Thanks

Hi, still have the problem today, which is sad because I cannot see the output of the query.

Using Logstash 6.6 and the Mysql .jar, I was able to call a stored procedure in MySQL, but ONLY if I turned off paging of the output.  This wouldn't work for large result set that would overload the HEAP.
I get the dilemma - using the current process to page the output, you need to get a count of total records to parse them into controllable blocks and it would seem communication is 1 way - hence the issue.  Is there any way to take the incoming data stream and cache it someplace so the local jdbc can pull it in without overloading?

Still the issue is open ?


Custom mapping not working

I can't get the Custom Mapping to work with version 1.4.4 of play-swagger.
I tried both YML and JSON for the "swagger-custom-mappings.json". Both seem to do nothing, whatever i define in those files.
Im using following "swagger-custom-mappings.json":
[{
  "type": "java\\.sql\\.Date",
  "specAsParameter": [{
    "type": "string",
    "format": "date"
  }],
  "specAsProperty": {
    "type": "string",
    "format": "date"
  }
}]

but nothing changes in the generated swagger.json. The relevant part looks like this:
"date_of_birth" : {
  "format" : "date",
  "type" : "java.sql.date",
  "x-nullable" : true
},

I also tried:

java\.sql\.date
integer

but nothing seems to change in the generated target/swagger/swagger.json file. I generate the swagger.json via:
sbt swagger

When i change the "swagger-custom-mappings.json" in a way that makes it invalid the "sbt swagger" command also crashes with an exception, so i am sure that it is actually loaded! It just seems to do nothing, whatever i change in it.
Please also see my project in github. Relevant files:

https://github.com/funkrusher/fk-framework-play-slick/blob/main/fk_server/conf/swagger-custom-mappings.json
https://github.com/funkrusher/fk-framework-play-slick/blob/main/fk_server/conf/swagger.json
https://github.com/funkrusher/fk-framework-play-slick/blob/main/build.sbt
https://github.com/funkrusher/fk-framework-play-slick/tree/main/fk_server/public/swagger-ui

Thanks for your help :)

I solved this problem for me, by using the latest version of this project (1.4.4).
see:

https://github.com/play-swagger/play-swagger/issues/36

With this version 1.4.4 and the following configs i successfully created a Swagger-V3 swagger.json file that contained my custom-mappings:

plugins.sbt:
addSbtPlugin("io.github.play-swagger" % "sbt-play-swagger"     % "1.4.4")

build,sbt:
...
    swaggerV3 := true,
    swaggerPrettyJson := true,
    swaggerDomainNameSpaces := Seq("library.dtos"),
...

swagger-custom-mappings.yml
---
  - type: java\.sql\.Date
    specAsParameter:
      - type: string
        format: date
    specAsProperty:
      type: string
      format: date

After i executed "sbt swagger" i got a good looking swagger.json that contained the resolved custom-mappings:
...
 "date_of_birth" : {
    "nullable" : true,
    "type" : "string",
    "format" : "date"
  },
...


Import issue with typescript - Declaration or statement expected

Hi. I am using this library in my typescript project
"react-native": "0.67.3"
"react-native-blob-util": "^0.16.0"
I am getting an error while importing this library
Parse errors in imported module 'react-native-blob-util': Declaration or statement expected.
Can someone please suggest what am I missing here

Does it state any more information about where it is expecting a declaration or statement?

@RonRadtke
No nothing else just this error only in the import statement. Rest all works fine.
I've added it to import/ignore in .eslintrc.js as workaround for now
'import/ignore': ['node_modules/react-native-blob-util/index\\.js$'],


Run linting against all possible versions of project generation

Eventually it would be nice to run tests as well but a low hanging fruit is just linting so we can catch issues related to if/else cookiecutter blocks.
The idea here being that we generate X number of projects in Github Actions. Just generating the code in separate folders. Then run the linters against those folders.
This will help us catch issues where linting is fine, unless you answer yes/no to a certain step and then injected code has a linting problem.
Newly bootstrapped projects should not have CI/CD errors on day 1.

This was implemented in https://github.com/thinknimble/tn-spa-bootstrapper/pull/115


Remove email addresses from API output

(This was discussed last year, for some reason I couldn't find the issue, so here is a new one)
The courses output from the API contains also email addresses.
For privacy reasons, etc. I would recommend to remove them from the API output and only show on the web page at the course detail page.

Agree to remove email from API result.

If it is a matter of under an hour, let's do it now, otherwise postpone.

Note: This can be reviewed when it's available in the test instance
Possible urls to review:

https://test-dhcr.clarin-dariah.eu/api/v2/courses/index
https://test-dhcr.clarin-dariah.eu/api/v2/courses/view/6
https://test-dhcr.clarin-dariah.eu/api/v2/courses/index?recent

@PixlTracer @IvdL22
Can you review this? Urls can be found above.

hi @patrickakk
I've checked the following URLs:

https://test-dhcr.clarin-dariah.eu/api/v2/courses/index - everything looks fine, no email (field) detected
https://test-dhcr.clarin-dariah.eu/api/v2/courses/view/6 - ok too
https://test-dhcr.clarin-dariah.eu/api/v2/courses/index?recent - ok too!

I'll set the label to done.
thank you!

Implemented in 2024-11


feat: add C implementation for math/base/special/factorial2

This commit if applied adds the C implementation to @stdlib/math/base/special/factorial2 package
Description
This pull request:

adds the native C implementation to @stdlib/math/base/special/factorial2

Related Issues

Does this pull request have any related issues?

This pull request:

resolves #1892

Questions

Any questions for reviewers of this pull request?

No.
Other

Any other information relevant to this pull request? This may include screenshots, references, and/or implementation notes.

No.
Checklist

Please ensure the following tasks are completed before submitting this pull request.

[x] Read, understood, and followed the contributing guidelines.

@stdlib-js/reviewers

Working on fixing the C errors


Says Steam isnt Running, when it is!

so yea im using linux and running sam.game.exe and it says Steam isnt running even though i have steam open!  can somebody help me please!

SAM isn't intended to function on Linux. You may be able to get it working using WINE via using https://github.com/xomachine/SteamForwarder but I haven't tried myself or heard any results from anyone who has tried it.


I've uploaded Donald Trump speeches and transcripts

I've uploaded some of his speeches and annotation (Text, and JSON at the word level). thought of sharing.
https://www.kaggle.com/etaifour/trump-speeches-audio-and-word-transcription

have you tried training this model with this data? how was the result?


Searchable, text you typed is put as value when you select option

Describe the bug
When searchable is turned on, and you type for example "ca" and select the option "cars" ca is put as value in the select.
this looks like to be the model value of the input. But when you change the "modelvalue" attribute on the input using @selected the value stays "ca"

OR is it something i'm doing wrong and do I need to change something else to see the selected option value inside the input?

You could use :clear-on-select="true" to clear/reset input when you select or remove an option.
Hope this could solve it in your scenario.

Thanks that worked :)


Fix an incorrect Ruby Gapic call.

Right now, Artman builds all fail. @eoogbe says this is the culprit. :-)

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@38dfc26). Click here to learn what that means.
The diff coverage is 100%.

@@            Coverage Diff            @@
##             master    #1039   +/-   ##
=========================================
  Coverage          ?   85.31%
  Complexity        ?     3452
=========================================
  Files             ?      359
  Lines             ?    13737
  Branches          ?     1568
=========================================
  Hits              ?    11720
  Misses            ?     1563
  Partials          ?      454

Impacted Files
Coverage Δ
Complexity Δ

...nsformer/ruby/RubyGapicSurfaceTestTransformer.java
98.18% <100%> (ø)
8 <ø> (?)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 38dfc26...a7fcae8. Read the comment docs.


deployment fails as database is not created

The postgresql dbs (template0 etc) created by the zammad installer has SQL_ASCII encoding. Therefore the creation of the zammad db fails, as it seems to require UTF-8. This is probably a upstream bug, as
yum install postgresql postgresql-server postgresql-contrib
postgresql-setup initdb

creates the dbs with UTF-8 encoding. This finally results in cpu hogging and a 502 from nginx as the application server can not serve.
If time allows I check the ruby installer scripts tomorrow and report upstream.
A quick workaround is likely to deploy postgresql beforehand. But I think the default zammad installer has several issues anyways (empty password, world readable config files).

https://github.com/ansible/ansible/issues/42114


Move keybase.io proof to a device key

I'm an early adopter, therefore my PGP key is part of my sigchain. I'm preparing to move to a new PGP key and retire my current one, so out of an abundance of caution, I've revoked and re-verified most of my Keybase identity proofs, which were previously connected directly to the PGP key. These proofs are now connected directly to various Keybase device keys. (See https://keybase.io/dhess/graph)
The only remaining identity that's attached to my current PGP key is my keybase @dhess ID. Is there a way to move this identity to a Keybase device key?
This question is somewhat related to #4662, in that I'm trying to avoid that situation.

Nothing that you need to do, many of my early proofs are with my PGP key, that I've since rotated.

Thanks. So if I revoke the current key, it'll remain in the graph just like my revoked device key nodes do?

Yes.