The AWS Access Key Id you provided does not exist in our records

I've been running Riak CS in a docker container as a local alternative to S3 for awhile, it's unspeakably awesome. After it's been up for a few days and restated I start being frustrated with this error.
The AWS Access Key Id you provided does not exist in our records.

It's very inconsistent. Sometimes it happens more often, sometimes it doesn't at all, it's almost feels like when the Riak CS "warms up" this happens less often. Sometimes I get it a lot with Transmit, then when I run some random command with s3cmd, like ls, it works and starts working with Transmit. Sometimes it works with Transmit, but not with the AWS SDK.
I googled this up, the only suggestion on how to solve was found here:

Can you try raising the pb_backlog to 128 in your riak app.config on each
node. It's likely those disconnect errors are left over from the stampede
of connections from the CS connection pool on startup. For one reason or
another the resets don't come through and the hanging disconnected socket
isn't discovered until the next send attempt.

But, when I uncomment the pb_backlog and set any value (10, 20, 64, 128), Riak refuses to start, it just eats all CPU and hangs.
I run everything on a single node, it's doesn't get under pressure, it receives a few files during unit testing, thats it. I remember from the past that it's recommended to have at least 3 nodes, but that feels counter productive in my case.
Is this a bug or is this something that can be fixed by tuning the configuration?

Are there any logs of Riak CS or Riak?

I've started a new instance a few hours ago, it behaves good right now, can't reproduce this at the moment. Will share the logs when it happens again.

Got it… Same thing happens again, sometimes works, sometimes doesn't. Right now it failed to display the bucket contents, after a few dozen retries it seems to be showing the bucket contents finally.
/var/log/riak-cs/error.log
/var/log/riak-cs/crash.log
/var/log/riak-cs/console.log
/var/log/riak/error.log – empty
/var/log/riak/crash.log – empty
/var/log/riak/console.log

Also I just notice, that originally the AWS SDK was failing with Access Denied message, not it's failing with The AWS Access Key Id you provided does not exist in our records..

When was trying to upload the file it failed at first, then tried downloading a file, this also failed, but uploading began functioning, downloading still doesn't work.
cat /var/log/rial-cs/error.log
2015-01-24 21:01:00.143 [error] <0.17982.3>@riak_cs_block_server:handle_cast:194 Put <<"mybucket">> <<"user/pianorow.gif">> UUID <<144,20,214,161,186,232,77,68,129,80,82,95,66,193,152,20>> block 0 failed: {error,timeout}
2015-01-24 21:01:00.146 [error] <0.17982.3> gen_server <0.17982.3> terminated with reason: no match of right hand value {error,timeout} in riak_cs_block_server:handle_cast/2 line 198
2015-01-24 21:01:00.146 [error] <0.17982.3> CRASH REPORT Process <0.17982.3> with 1 neighbours exited with reason: no match of right hand value {error,timeout} in riak_cs_block_server:handle_cast/2 line 198 in gen_server:terminate/6 line 747
2015-01-24 21:01:00.146 [error] <0.431.0> Supervisor riak_cs_put_fsm_sup had child undefined started with {riak_cs_put_fsm,start_link,undefined} at <0.17979.3> exit with reason no match of right hand value {error,timeout} in riak_cs_block_server:handle_cast/2 line 198 in context child_terminated
2015-01-24 21:01:00.147 [error] <0.123.0> Lager event handler error_logger_lager_h exited with reason {'EXIT',{{badmatch,["/buckets/mybucket/objects/user%2Fpianorow.gif",{error,{exit,{{{badmatch,{error,timeout}},[{riak_cs_block_server,handle_cast,2,[{file,"src/riak_cs_block_server.erl"},{line,198}]},{gen_server,handle_msg,5,[{file,"gen_server.erl"},{line,607}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,227}]}]},{gen_fsm,sync_send_event,[<0.17979.3>,{finalize,undefined},infinity]}},[{gen_fsm,sync_send_event,3,[{file,"gen_fsm.erl"},{line,214}]},{riak_cs_wm_object,...},...]}},...]},...}}
2015-01-24 21:01:00.156 [error] <0.17981.3> gen_fsm <0.17981.3> in state waiting_update_command terminated with reason: no match of right hand value {error,timeout} in riak_cs_block_server:handle_cast/2 line 198
2015-01-24 21:01:00.156 [error] <0.17981.3> CRASH REPORT Process <0.17981.3> with 0 neighbours exited with reason: no match of right hand value {error,timeout} in riak_cs_block_server:handle_cast/2 line 198 in gen_fsm:terminate/7 line 611
2015-01-24 21:01:20.159 [error] <0.18056.3>@riak_cs_block_server:handle_cast:194 Put <<"mybucket">> <<"user/pianorow.gif">> UUID <<29,221,45,219,49,91,67,25,167,208,37,250,208,111,0,77>> block 0 failed: {error,timeout}
2015-01-24 21:01:20.159 [error] <0.18056.3> gen_server <0.18056.3> terminated with reason: no match of right hand value {error,timeout} in riak_cs_block_server:handle_cast/2 line 198
2015-01-24 21:01:20.160 [error] <0.18056.3> CRASH REPORT Process <0.18056.3> with 1 neighbours exited with reason: no match of right hand value {error,timeout} in riak_cs_block_server:handle_cast/2 line 198 in gen_server:terminate/6 line 747
2015-01-24 21:01:20.160 [error] <0.431.0> Supervisor riak_cs_put_fsm_sup had child undefined started with {riak_cs_put_fsm,start_link,undefined} at <0.18054.3> exit with reason no match of right hand value {error,timeout} in riak_cs_block_server:handle_cast/2 line 198 in context child_terminated
2015-01-24 21:01:20.160 [error] <0.123.0> Lager event handler error_logger_lager_h exited with reason {'EXIT',{{badmatch,["/buckets/mybucket/objects/user%2Fpianorow.gif",{error,{exit,{{{badmatch,{error,timeout}},[{riak_cs_block_server,handle_cast,2,[{file,"src/riak_cs_block_server.erl"},{line,198}]},{gen_server,handle_msg,5,[{file,"gen_server.erl"},{line,607}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,227}]}]},{gen_fsm,sync_send_event,[<0.18054.3>,{finalize,undefined},infinity]}},[{gen_fsm,sync_send_event,3,[{file,"gen_fsm.erl"},{line,214}]},{riak_cs_wm_object,...},...]}},...]},...}}
2015-01-24 21:01:20.264 [error] <0.18055.3> gen_fsm <0.18055.3> in state waiting_update_command terminated with reason: no match of right hand value {error,timeout} in riak_cs_block_server:handle_cast/2 line 198
2015-01-24 21:01:20.264 [error] <0.18055.3> CRASH REPORT Process <0.18055.3> with 0 neighbours exited with reason: no match of right hand value {error,timeout} in riak_cs_block_server:handle_cast/2 line 198 in gen_fsm:terminate/7 line 611
2015-01-24 21:01:40.182 [error] <0.18131.3>@riak_cs_block_server:handle_cast:194 Put <<"mybucket">> <<"user/pianorow.gif">> UUID <<59,193,218,56,87,231,76,199,136,255,168,93,62,126,20,95>> block 0 failed: {error,timeout}
2015-01-24 21:01:40.182 [error] <0.18131.3> gen_server <0.18131.3> terminated with reason: no match of right hand value {error,timeout} in riak_cs_block_server:handle_cast/2 line 198
2015-01-24 21:01:40.182 [error] <0.18131.3> CRASH REPORT Process <0.18131.3> with 1 neighbours exited with reason: no match of right hand value {error,timeout} in riak_cs_block_server:handle_cast/2 line 198 in gen_server:terminate/6 line 747
2015-01-24 21:01:40.184 [error] <0.123.0> Lager event handler error_logger_lager_h exited with reason {'EXIT',{{badmatch,["/buckets/mybucket/objects/user%2Fpianorow.gif",{error,{exit,{{{badmatch,{error,timeout}},[{riak_cs_block_server,handle_cast,2,[{file,"src/riak_cs_block_server.erl"},{line,198}]},{gen_server,handle_msg,5,[{file,"gen_server.erl"},{line,607}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,227}]}]},{gen_fsm,sync_send_event,[<0.18129.3>,{finalize,undefined},infinity]}},[{gen_fsm,sync_send_event,3,[{file,"gen_fsm.erl"},{line,214}]},{riak_cs_wm_object,...},...]}},...]},...}}
2015-01-24 21:01:40.184 [error] <0.431.0> Supervisor riak_cs_put_fsm_sup had child undefined started with {riak_cs_put_fsm,start_link,undefined} at <0.18129.3> exit with reason no match of right hand value {error,timeout} in riak_cs_block_server:handle_cast/2 line 198 in context child_terminated
2015-01-24 21:01:40.288 [error] <0.18130.3> gen_fsm <0.18130.3> in state waiting_update_command terminated with reason: no match of right hand value {error,timeout} in riak_cs_block_server:handle_cast/2 line 198
2015-01-24 21:01:40.288 [error] <0.18130.3> CRASH REPORT Process <0.18130.3> with 0 neighbours exited with reason: no match of right hand value {error,timeout} in riak_cs_block_server:handle_cast/2 line 198 in gen_fsm:terminate/7 line 611
2015-01-24 21:02:22.682 [error] <0.18537.3>@riak_cs_get_fsm:waiting_chunks:315 riak_cs_get_fsm: Cannot get S3 <<"mybucket">> <<"user/0/temporary/d314a6e985301e2d10150f7fc089fd2d.jpeg">> block# {<<128,173,23,110,130,29,71,116,161,59,47,158,164,113,64,105>>,0}: {error,notfound}
2015-01-24 21:02:22.683 [error] <0.18504.3> CRASH REPORT Process <0.18504.3> with 0 neighbours exited with reason: {normal,{gen_fsm,sync_send_event,[<0.18537.3>,get_next_chunk,infinity]}} in gen_fsm:sync_send_event/3 line 214
2015-01-24 21:02:22.724 [error] <0.18543.3>@riak_cs_get_fsm:waiting_chunks:315 riak_cs_get_fsm: Cannot get S3 <<"mybucket">> <<"user/0/temporary/d314a6e985301e2d10150f7fc089fd2d.jpeg">> block# {<<128,173,23,110,130,29,71,116,161,59,47,158,164,113,64,105>>,0}: {error,notfound}
2015-01-24 21:02:22.726 [error] <0.13137.0> CRASH REPORT Process <0.13137.0> with 0 neighbours exited with reason: {normal,{gen_fsm,sync_send_event,[<0.18543.3>,get_next_chunk,infinity]}} in gen_fsm:sync_send_event/3 line 214
2015-01-24 21:02:22.751 [error] <0.18549.3>@riak_cs_get_fsm:waiting_chunks:315 riak_cs_get_fsm: Cannot get S3 <<"mybucket">> <<"user/0/temporary/d314a6e985301e2d10150f7fc089fd2d.jpeg">> block# {<<128,173,23,110,130,29,71,116,161,59,47,158,164,113,64,105>>,0}: {error,notfound}
2015-01-24 21:02:22.753 [error] <0.17973.3> CRASH REPORT Process <0.17973.3> with 0 neighbours exited with reason: {normal,{gen_fsm,sync_send_event,[<0.18549.3>,get_next_chunk,infinity]}} in gen_fsm:sync_send_event/3 line 214
2015-01-24 21:02:35.410 [error] <0.18331.3>@riak_cs_block_server:handle_cast:194 Put <<"mybucket">> <<"user/pianorow.gif">> UUID <<207,0,62,171,80,92,74,165,132,87,60,152,188,119,223,189>> block 0 failed: {error,timeout}
2015-01-24 21:02:35.410 [error] <0.18331.3> gen_server <0.18331.3> terminated with reason: no match of right hand value {error,timeout} in riak_cs_block_server:handle_cast/2 line 198
2015-01-24 21:02:35.411 [error] <0.18331.3> CRASH REPORT Process <0.18331.3> with 1 neighbours exited with reason: no match of right hand value {error,timeout} in riak_cs_block_server:handle_cast/2 line 198 in gen_server:terminate/6 line 747
2015-01-24 21:02:35.411 [error] <0.123.0> Lager event handler error_logger_lager_h exited with reason {'EXIT',{{badmatch,["/buckets/mybucket/objects/user%2Fpianorow.gif",{error,{exit,{{{badmatch,{error,timeout}},[{riak_cs_block_server,handle_cast,2,[{file,"src/riak_cs_block_server.erl"},{line,198}]},{gen_server,handle_msg,5,[{file,"gen_server.erl"},{line,607}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,227}]}]},{gen_fsm,sync_send_event,[<0.18329.3>,{finalize,undefined},infinity]}},[{gen_fsm,sync_send_event,3,[{file,"gen_fsm.erl"},{line,214}]},{riak_cs_wm_object,...},...]}},...]},...}}
2015-01-24 21:02:53.450 [error] <0.18663.3>@riak_cs_get_fsm:waiting_chunks:315 riak_cs_get_fsm: Cannot get S3 <<"mybucket">> <<"user/0/temporary/d314a6e985301e2d10150f7fc089fd2d.jpeg">> block# {<<128,173,23,110,130,29,71,116,161,59,47,158,164,113,64,105>>,0}: {error,notfound}
2015-01-24 21:02:53.451 [error] <0.18051.3> CRASH REPORT Process <0.18051.3> with 0 neighbours exited with reason: {normal,{gen_fsm,sync_send_event,[<0.18663.3>,get_next_chunk,infinity]}} in gen_fsm:sync_send_event/3 line 214
2015-01-24 21:02:53.467 [error] <0.18669.3>@riak_cs_get_fsm:waiting_chunks:315 riak_cs_get_fsm: Cannot get S3 <<"mybucket">> <<"user/0/temporary/d314a6e985301e2d10150f7fc089fd2d.jpeg">> block# {<<128,173,23,110,130,29,71,116,161,59,47,158,164,113,64,105>>,0}: {error,notfound}
2015-01-24 21:02:53.468 [error] <0.18510.3> CRASH REPORT Process <0.18510.3> with 0 neighbours exited with reason: {normal,{gen_fsm,sync_send_event,[<0.18669.3>,get_next_chunk,infinity]}} in gen_fsm:sync_send_event/3 line 214
2015-01-24 21:02:53.489 [error] <0.18675.3>@riak_cs_get_fsm:waiting_chunks:315 riak_cs_get_fsm: Cannot get S3 <<"mybucket">> <<"user/0/temporary/d314a6e985301e2d10150f7fc089fd2d.jpeg">> block# {<<128,173,23,110,130,29,71,116,161,59,47,158,164,113,64,105>>,0}: {error,notfound}
2015-01-24 21:02:53.490 [error] <0.18513.3> CRASH REPORT Process <0.18513.3> with 0 neighbours exited with reason: {normal,{gen_fsm,sync_send_event,[<0.18675.3>,get_next_chunk,infinity]}} in gen_fsm:sync_send_event/3 line 214
2015-01-24 21:02:55.422 [error] <0.18407.3>@riak_cs_block_server:handle_cast:194 Put <<"mybucket">> <<"user/pianorow.gif">> UUID <<97,92,249,197,242,177,67,151,180,124,117,86,71,173,35,213>> block 0 failed: {error,timeout}
2015-01-24 21:02:55.423 [error] <0.18407.3> gen_server <0.18407.3> terminated with reason: no match of right hand value {error,timeout} in riak_cs_block_server:handle_cast/2 line 198
2015-01-24 21:02:55.424 [error] <0.18407.3> CRASH REPORT Process <0.18407.3> with 1 neighbours exited with reason: no match of right hand value {error,timeout} in riak_cs_block_server:handle_cast/2 line 198 in gen_server:terminate/6 line 747
2015-01-24 21:02:55.424 [error] <0.123.0> Lager event handler error_logger_lager_h exited with reason {'EXIT',{{badmatch,["/buckets/mybucket/objects/user%2Fpianorow.gif",{error,{exit,{{{badmatch,{error,timeout}},[{riak_cs_block_server,handle_cast,2,[{file,"src/riak_cs_block_server.erl"},{line,198}]},{gen_server,handle_msg,5,[{file,"gen_server.erl"},{line,607}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,227}]}]},{gen_fsm,sync_send_event,[<0.18405.3>,{finalize,undefined},infinity]}},[{gen_fsm,sync_send_event,3,[{file,"gen_fsm.erl"},{line,214}]},{riak_cs_wm_object,...},...]}},...]},...}}
2015-01-24 21:02:55.424 [error] <0.431.0> Supervisor riak_cs_put_fsm_sup had child undefined started with {riak_cs_put_fsm,start_link,undefined} at <0.18405.3> exit with reason no match of right hand value {error,timeout} in riak_cs_block_server:handle_cast/2 line 198 in context child_terminated
2015-01-24 21:02:55.428 [error] <0.18406.3> gen_fsm <0.18406.3> in state waiting_update_command terminated with reason: no match of right hand value {error,timeout} in riak_cs_block_server:handle_cast/2 line 198
2015-01-24 21:02:55.429 [error] <0.18406.3> CRASH REPORT Process <0.18406.3> with 0 neighbours exited with reason: no match of right hand value {error,timeout} in riak_cs_block_server:handle_cast/2 line 198 in gen_fsm:terminate/7 line 611
2015-01-24 21:03:49.016 [error] <0.18888.3>@riak_cs_get_fsm:waiting_chunks:315 riak_cs_get_fsm: Cannot get S3 <<"mybucket">> <<"user/0/temporary/d6a8ea5f7c0b61121236989de8d68c3f.jpeg">> block# {<<221,53,209,168,57,141,79,89,134,24,57,209,22,93,115,179>>,0}: {error,notfound}
2015-01-24 21:03:49.017 [error] <0.17915.3> CRASH REPORT Process <0.17915.3> with 0 neighbours exited with reason: {normal,{gen_fsm,sync_send_event,[<0.18888.3>,get_next_chunk,infinity]}} in gen_fsm:sync_send_event/3 line 214
2015-01-24 21:03:49.057 [error] <0.18894.3>@riak_cs_get_fsm:waiting_chunks:315 riak_cs_get_fsm: Cannot get S3 <<"mybucket">> <<"user/0/temporary/d6a8ea5f7c0b61121236989de8d68c3f.jpeg">> block# {<<221,53,209,168,57,141,79,89,134,24,57,209,22,93,115,179>>,0}: {error,notfound}
2015-01-24 21:03:49.057 [error] <0.18877.3> CRASH REPORT Process <0.18877.3> with 0 neighbours exited with reason: {normal,{gen_fsm,sync_send_event,[<0.18894.3>,get_next_chunk,infinity]}} in gen_fsm:sync_send_event/3 line 214
2015-01-24 21:03:49.088 [error] <0.18900.3>@riak_cs_get_fsm:waiting_chunks:315 riak_cs_get_fsm: Cannot get S3 <<"mybucket">> <<"user/0/temporary/d6a8ea5f7c0b61121236989de8d68c3f.jpeg">> block# {<<221,53,209,168,57,141,79,89,134,24,57,209,22,93,115,179>>,0}: {error,notfound}
2015-01-24 21:03:49.089 [error] <0.18548.3> CRASH REPORT Process <0.18548.3> with 0 neighbours exited with reason: {normal,{gen_fsm,sync_send_event,[<0.18900.3>,get_next_chunk,infinity]}} in gen_fsm:sync_send_event/3 line 214

@ianbytchek I don't understand what is the cause, just explain what log says.

Two lines in CS console.log https://gist.github.com/ianbytchek/ee3e6c7ea05a1e8b2541#file-console-log-L24 indicates user fetch (getting user object from riak by riak-cs) failed by notfound.
The log just above says block (content of S3 object) retrieval failed  by timeout (default 60sec).

It may be nice to try without docker?

Thanks @shino, it might be the case of some problem between CS and Riak then. Using the same user I can list all buckets, but can't list the specific bucket – one of the common problems.
The problem seems to be happening when the VM gets unexpectedly switched off, maybe Riak gets cut off in the middle of doing something?
I used to run this on my hardware not that long ago, this never happened, so it would be pointless trying this without docker. My setup is only one node for local development, it's not critical that it fails, I simply start the new container, and it's not what the real life setup would be like. But if you want to chase this I can share the docker file, you can build the image and try this yourself. Doing this in a virtual machine and killing it should give the same result, the instance must contain objects and must be in operation for awhile.

The problem seems to be happening when the VM gets unexpectedly switched off, maybe Riak gets cut off in the middle of doing something?

Network connectivity issue may cause the errors.
I will close this issue because it seems not related to riak cs software itself.
If you want to discuss and/or seek workaround, I guess riak-users mailing list is
better place than here.
@ianbytchek  Thanks for your interests and reporting :smile:

@shino cool :smile:


inefficient solution with recursive calls

creates the fizzbuzz solution with recursive function. prints the solution with another recursive function. Wastes a lot of stack.

haha interesting! Thanks :)


Preventing division by zero

In case airspeed is zero, this is division by zero. As true and indicated airspeeds are affine, the check for true_airspeed>7.0f should make sure that also indicated airspeed is non-zero.
Additionally saves some computation time, if airspeed can not be updated.

Has been solved in https://github.com/PX4/Firmware/pull/4615


Docs: Docs is down, just returns a 404

What is the improvement or update you wish to see?
Docs back up
Is there any context that might help us understand?
Docs is down
Does the docs page already exist? Please link to it.
https://turbo.build/repo/docs

They're back up. Closing this


Task-plugin

(I thought, there already was a threat with this issue, but I can't find it.)
When I use a new TestVault and only have tasks plugin and yours installed, I get this from your plugin:

And I get this from Obsidians native pdf export.

muell-export-pdf-task-obsidian-native-pdf.pdf
muell-export-pdf-task-obsidian-better-export-pdf.pdf
muell-export-pdf-task.md
Was confirmed here:
https://forum.obsidian.md/t/pdf-export-of-tasks-not-all-tasks-are-in-the-exported-pdf-file/80375/2
I think, I better test with an isolated vault.

The problem is still caused by the inability to obtain the completion time of tasks rendering. I have not yet found a good way to obtain the completion time of third-party plugin rendering. I have added a detection method, although it is not perfect, it only mitigates the problem. I have tested it with 2000 tasks and it worked properly.
better-export-pdf.zip

The problem is still caused by the inability to obtain the completion time of tasks rendering. I have not yet found a good way to obtain the completion time of third-party plugin rendering. I have added a detection method, although it is not perfect, it only mitigates the problem. I have tested it with 2000 tasks and it worked properly.
better-export-pdf.zip

I made a new testVault with minimal plugins, changed the files to those you provided in the post above and it looks well:

Noting that the latter half of the task appears while the first half is blank, it may not be caused by the issue of slow rendering. If you see all the tasks in the preview interface, at this point, the rendering is already complete. The next step is to call the interface to export the PDF, rendering will no longer affect the final result. The problem should be in the stage of invoking the underlying interface to print the PDF. It may be related to #162 , but I have never found the reason.

Can you try adjusting the page size and margins to see if there are different results?

Good idea.
Test with my productive vault:
New issue: In all exports, regardless of the vault, there are no headers/footers anymore.
Correct:
15_35_A0_muell-export-pdf-task.pdf
15_38_A1_muell-export-pdf-task.pdf
Not correct:
15_39_A2_muell-export-pdf-task.pdf
15_25_A3_muell-export-pdf-task.pdf
15_41_A4_muell-export-pdf-task.pdf
Your preview is always ok. One should believe, as long as there is no page break, the export of the pdf file works correct... but then comes A5, which works again.
15_43_A5_muell-export-pdf-task.pdf
A6 has 2 empty sites:
15_43_A6_muell-export-pdf-task.pdf

Test with my testVault:
A0, A1 works and even A2, A3, A4, A5 and A6 which have page breaks.
15_43_A0_muell-export-pdf-task.pdf
15_49_A1_muell-export-pdf-task.pdf
15_49_A2_muell-export-pdf-task.pdf
15_50_A3_muell-export-pdf-task.pdf
15_50_A4_muell-export-pdf-task.pdf
15_50_A5_muell-export-pdf-task.pdf
15_53_A6_muell-export-pdf-task.pdf
In my TestVault, every output works. There must be something wrong with my productive vault.
BTW: In your data.json, there is a typo:
pageSise -> pageSize
data.json

Thank you very much for your thorough testing. You could try disabling all plugins in the productive vault and only keeping a few essential ones enabled, then systematically check which plugin is causing the issue.


Twilio inconsistent message order

Rasa Open Source version
3.2.1
Rasa SDK version
3.2.1
Python version
3.9
What operating system are you using?
Linux, Cloud (AWS / GCP / AZURE)
What happened?
When deploying my bot (based mostly on custom actions) on AWS and using the Twilio connector, sometimes I get messages in the wrong order. This happens when:

I have 2 utter_message calls in the same action
Use a FollowupAction with another utter_message

Is there a way to wait out messages or avoid this behavior without compromising scalability?
Command / Request
No response
Relevant log output
No response

The normal Twilio connector does not wait. However that would be possible. Doing so without compromising scalability might make it more complex. Twilio provides callback for a change in message status. You could track these events to find out if you have undelivered messeges to a phone number.
Better solution is to use the (relatively new) WhatsApp Cloud API and write a custome connector for that (and maybe contribute to Rasa) https://developers.facebook.com/docs/whatsapp/cloud-api/


[Feature] Remove old ui.

Purpose of the pull request
close #9896

Brief change log

Verify this pull request

This pull request is code cleanup without any test coverage.
(or)
This pull request is already covered by existing tests, such as (please describe tests).
(or)
This change added tests and can be verified as follows:

Codecov Report

Merging #9909 (ec37f7b) into dev (693b8a9) will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##                dev    #9909   +/-   ##
=========================================
  Coverage     40.32%   40.32%
  Complexity     4532     4532
=========================================
  Files           835      835
  Lines         33747    33747
  Branches       3726     3726
=========================================
  Hits          13607    13607
  Misses        18853    18853
  Partials       1287     1287

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 693b8a9...ec37f7b. Read the comment docs.

I restart the failed CI


[WIP] Add imported attribute for Source

TPINVTRY-632
WIP:

[ ] https://github.com/ManageIQ/sources-api/pull/102

NOTE: after generation client there is also something related  to GraphQL and tenant attribute
@miq-bot assign @agrare

I think we just didn't generate the client after some changes to api-common.


Error when creating new mindmap

Hi, would love to explore your app but I can't create a new mindmap. Is this an easy fix?

I'm sorry to hear you are having issues @dominiczy. I haven't had a problem with my own account using Chrome or Firefox but when I checked the database and couldn't find a mind map collection for your user. Seems like it wasn't auto-created for you on registration so I manually created the collection instead. Give it a shot and let me know if it works!

And looking into it a little further the cause appears to originate from an upgrade to Appwrite itself. The update for system events that trigger the function for handling user registrations didn't migrate correctly and I just had to re-add them to the function settings. Should be all good now :)

Thanks @benweier ! However unfortunately it still doesn't seem to work. I've hard refreshed, anything else I should do?

Alright, I'll do a bit more of a deep dive. I may have set up it up wrong

Okay my bad, I didn't set the attributes and collection-level permissions correctly. Please try again @dominiczy

Ok that now works. However in the mindmap I can only add one node


Add log file for CNI executions

Without the access to kubelet and / or container runtime logs, right now it's not possible to know when CNI executions are made into cilium-cni. It would be nice to export these logs into a specific log file (/run/cilium/cilium-cni.log), with log rotation, which would store all the CNI executions including log messages in there. When implementing this, take into account that cilium-cni binary can be executed concurrently so storing into a single file has to handle concurrent writes.

New to the project, can I try out this ticket? Thanks!

@vishjain sure, let us now if you need any help with it. We are in the #development channel of http://slack.cilium.io

/assign

seems like no one is working on this, let me give it a crack.


Detached View error when recieving any input from hardware keyboard

Steps to Reproduce

Connect a hardware keyboard to the phone. In my case I am using a bluetooth barcode scanner that interfaces like a keyboard.
Run any flutter application with multiple pages.
Navigate to a page
Use the scanner.
The error message D/FlutterView( 5766): FlutterView.send called on a detached view, channel=flutter/textinput will appear in the terminal and the Navigator will have popped to the original page.

What is a detached view?
Logs

[   +2 ms] 🔥  To hot reload changes while running, press "r". To hot restart (and rebuild state), press "R".
[   +1 ms] An Observatory debugger and profiler on Pixel 2 is available at: http://<IP_ADDRESS>:8114/
[        ] For a more detailed help message, press "h". To quit, press "q".
[+4368 ms] I/flutter ( 3848): Navigating to scan-setup
[+7526 ms] D/FlutterView( 3848): FlutterView.send called on a detached view, channel=flutter/textinput
[ +144 ms] I/FlutterActivityDelegate( 3848): onResume setting current activity to this
[+1527 ms] I/zygote64( 3848): Do partial code cache collection, code=23KB, data=28KB
[        ] I/zygote64( 3848): After code cache collection, code=23KB, data=28KB
[   +2 ms] I/zygote64( 3848): Increasing code cache capacity to 128KB

Analyzing app...

   info • Unused import: '../models/Session.dart' • lib/components/ItemExpansionTile.dart:2:8
   info • Unused import: '../main.dart' • lib/components/ItemExpansionTile.dart:3:8
   info • Unused import: '../pages/SessionDetailsPage.dart' • lib/components/ItemExpansionTile.dart:4:8
   info • Unused import: 'package:intl/intl.dart' • lib/components/ItemExpansionTile.dart:5:8
   info • Unused import: '../components/ScannerFocusedTextField.dart' • lib/pages/ScanPage.dart:12:8
   info • The method '_addItem' isn't used • lib/pages/ScanPage.dart:50:8

6 issues found. (ran in 3.5s)

flutter doctor -v
[✓] Flutter (Channel beta, v0.5.1, on Mac OS X 10.12.6 16G1408, locale en-US)
    • Flutter version 0.5.1 at /Users/d0r6/flutter
    • Framework revision c7ea3ca377 (7 weeks ago), 2018-05-29 21:07:33 +0200
    • Engine revision 1ed25ca7b7
    • Dart version 2.0.0-dev.58.0.flutter-f981f09760

[✓] Android toolchain - develop for Android devices (Android SDK 28.0.1)
    • Android SDK at /Users/d0r6/Library/Android/sdk
    • Android NDK location not configured (optional; useful for native profiling support)
    • Platform android-28, build-tools 28.0.1
    • Java binary at: /Applications/Android Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 1.8.0_152-release-1024-b01)
    • All Android licenses accepted.

[✓] iOS toolchain - develop for iOS devices (Xcode 9.0.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 9.0.1, Build version 9A1004
    • ios-deploy 1.9.2
    • CocoaPods version 1.5.3

[✓] Android Studio (version 3.1)
    • Android Studio at /Applications/Android Studio.app/Contents
    ✗ Flutter plugin not installed; this adds Flutter specific functionality.
    ✗ Dart plugin not installed; this adds Dart specific functionality.
    • Java version OpenJDK Runtime Environment (build 1.8.0_152-release-1024-b01)

[!] IntelliJ IDEA Community Edition (version 2018.1.6)
    • IntelliJ at /Applications/IntelliJ IDEA CE.app
    ✗ Flutter plugin not installed; this adds Flutter specific functionality.
    ✗ Dart plugin not installed; this adds Dart specific functionality.
    • For information about installing plugins, see
      https://flutter.io/intellij-setup/#installing-the-plugins

[!] IntelliJ IDEA Ultimate Edition (version 2018.1.4)
    • IntelliJ at /Applications/IntelliJ IDEA.app
    ✗ Flutter plugin not installed; this adds Flutter specific functionality.
    ✗ Dart plugin not installed; this adds Dart specific functionality.
    • For information about installing plugins, see
      https://flutter.io/intellij-setup/#installing-the-plugins

[!] VS Code (version 1.25.0)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension not installed; install from
      https://marketplace.visualstudio.com/items?itemName=Dart-Code.flutter

[✓] Connected devices (1 available)
    • Pixel 2 • HT7AM1A02980 • android-arm64 • Android 8.1.0 (API 27)

! Doctor found issues in 3 categories.

Without additional information, we are unfortunately not sure how to resolve this issue. We are therefore reluctantly going to close this bug for now. Please don't hesitate to comment on the bug if you have any more information for us; we will reopen it right away!
Thanks for your contribution.


Create PR to fix broken travis build due to corrupted .travis.yml

Coverage remained the same at 98.897% when pulling 6b0deab079c420bdecb91645aa6b1dfbe0d6a2a5 on fix/broken-travis-build into 8dfe25c723c776ec5bec73dd7db82507e44e4369 on master.

No longer needed


Configuring GIT_LOCK_TIMEOUT with helm

Hello all,
I've been using deis 2.8 and installed it with helmc. I used to generate my manifests and configure Object Storage this way:
STORAGE_TYPE=s3 AWS_BUILDER_BUCKET=some_bucket AWS_DATABASE_BUCKET=some_bucket AWS_REGISTRY_BUCKET=some_bucket S3_REGION=eu-west-1 helmc generate -x manifests workflow-$DEIS_VERSION

Then I edited the file ~/.helmc/workspace/charts/workflow-$DEIS_VERSION/manifests/deis-builder-deployment.yaml and set GIT_LOCK_TIMEOUT to 60.
If I don't do this, I can't push big app that take a while to compile (push rejected, ssh connection closed).
Now that deis uses helm (and not helmc), I see how I can configure the Object Storage (it's documented) however I have no idea how to edit the GIT_LOCK_TIMEOUT.
Regards,
Robin

The easiest way would be to fetch the chart locally and edit the builder deployment manifest directly like you would with helm-classic. To do that, run
helm fetch deis/workflow --untar
vim workflow/charts/builder/templates/deis-builder-deployment.yaml
helm install ./workflow --namespace deis

Hope this helps!

Thank you very much, I was confused by how helm differs from helmc (with the server side aspect). I ran helm init --client-only and got less confused. I can now do what you suggest and everything should work fine :+1:


PositionMode.LENGTH incorrect

w = Wire([Line(v1, v2).edge() for v1,v2 in itertools.pairwise([Vector(a,0) for a in [(i/10)**2 for i in range(0, 11)]])]).close()
s = w.positions(range(11), PositionMode.LENGTH)

These should be uniformly spaced
The overall length of this Wire is 1, yet it fits sample points at distances of up to 10

Are you sure this is working right? I am having similar problems. I think the issue is :
class Mixin1D:
...
        law.D0(param, tangent, normal, binormal)
        pnt = curve.Value(param)

It is not clear to me what the law is actually doing here, and I am having some trouble getting results that I expect.


Consent Management link doesn't work

Is your suggestion related to a missing or misleading document? Please describe.
Consent Management link doesn't work in Consent Management with Single-Sign-On Page.

Once this link is clicked, the user is directed to https://is.docs.wso2.com/en/latest/learn/user-portal/#consent-management this page which doesn't exist.

Docs version - 5.11.0

This issue is being closed due to extended inactivity. Please feel free to reopen it if further attention is needed. Thank you for helping us keep the issue list relevant and focused!


A11y Improvements - Robust

Overview
This branch aims to improve on the "robust" category of WCAG standards for the app. Each of the areas of improvement are listed in the connected issue.
This also adjusts the version of black that pre-commit uses to match the version used in github actions.

Connects #1024

Notes
The original issues for each category are listed in #1024

4.1.1 Parsing

There are a few errors that come up when using an html validator such as the one provided by the W3C, that weren't directly resolved in this branch

The autocomplete attribute on the search radio buttons raises errors in that validator, but I believe we can keep them there. This github comment notes that it used to be required to set that attribute to "off" to prevent some browsers from persisting the choice across pages. The metro app handles that itself in a few different ways. I have a couple more resources on this topic if interested.
There are a number of errors that exist because of dependencies we're using. Some of these have solutions in updated versions, while others do not. I'll open issues for the ones that would benefit from an update.

Many data attributes were adjusted to be more specific, and the functions that use them have been adjusted as well.
Moment.js has been removed because it wasn’t effecting anything about the datepickers on either chrome or firefox, but was causing a validation error.

Dependencies that cause validation errors

May be resolved with an update:

mozilla pdf.js
recaptcha
leaflet
datatables

Cannot be resolved with an update:

select2
google maps
google translate
google programmable search engine
google tag analytics

Testing Instructions

Get to the site
Asses if the points covered in the issue have been properly addressed.

Firefox has an add-on available called HTML Validator that can be used to test pages on localhost

Merging this into main instead of #1009 since this has everything that pr does as well as resolved merge conflicts.


React Structure Implementation

Added Landing Page with action buttons for login
Added Dashboard Page
Added place holder for Calendar in Dashboard
Added Group List view with button to create a new group
Added Navigation bar with user icon and notification icon
Added general app layout

Code reviewed and approved.


Use new API for getting newest grades

The new API also tells what the grade is for, this is a nice addition

Possible solution for #19 as well


Default value in initial filter state

It would be nice to have an option when defining the filters to give them a default value. The use case we have, is that the default view of the list should be to only show the clients of the current user, but with an option to deactivate this filter to view all clients. Currently this is not possible, since the permanent filter cannot be interacted with by the user, and we cannot set a default value in a regular filter, that is applied on first/initial visit to the page.
The idea is to be able to specify what the initial state of the filters should look like, including values, on first visit or after reset, instead of the current behaviour of setting all filters to blank.

Already done in #639, please use the master branch.


chore: increased async test timeout to 10s from 5s

Description
The tests are failing sometimes, we don't know all the reasons the tests might be failing but recently we figured that some tests are failing because they are taking more time than the async timeout that was set to 5s. This PR aims to resolve these issues, currently, all the tests in the development server are passing so no way to understand which tests cases are taking more time than intended.
It was running fine in ubuntu but was having issues in mac and windows.

I think conventional commits have prefix for test but don't remember if it is test or tests`

Yeah it is test shall I update the pt title ?

@Souvikns yes please, or just update it in the commit message when you will squash and merge


how to apply validation for file and image

Versions:

VueJs: #.#.#
Vee-Validate:

Description:
Steps To Reproduce:
i am unable to understand for upload image and file validation in documentation please describe details if possible to  give example in jsfiddle  thank you

Sorry for taking a long time to respond to this issue, if you or anyone else is stuck on similar issue here is an example:
https://jsfiddle.net/logaretm/bxdyhfto/1/
Docs will receive a a rewrite soon.

is reject modifier disabled on<EMAIL_ADDRESS>


Make Fleet helm chart values configurable

Issue: rancher/fleet#1717
Problem
Currently Rancher installs Fleet with fixed values (see https://github.com/rancher/rancher/blob/9c5a25675676b680898f933681abde88f53dba95/pkg/controllers/dashboard/fleetcharts/controller.go#L89-L100), making the solution less flexible than Fleet standalone cases.
Solution
These changes allow Rancher users to specify additional arbitrary values for the Fleet chart, so that their Fleet installation can be customized flexibly, by adding a new fleet entry to the rancher-config ConfigMap.
This is particularly useful as a catch-all measure when assisting customers to introduce Fleet-specific experimental tuning settings, debug settings, etc. even if they are not yet covered by the Rancher UI.
Testing
A new unit tests have been added for the chart and relevant code.
Engineering Testing
Manual Testing

I built the rancher image pushed it to a local registry.
Built the chart as well and installed it in a local k3d cluster, setting rancherImage to point to the local registry.
Complete the installation and ensure regular installation keeps working.

Checked rancher-config ConfigMap is created and only contains priorityClassName entry.
Checked the fleet chart release that was installed in the cattle-fleet-system namespace.

Use helm upgrade --reuse-values combined with --set fleet.debug=true to update the configuration:

Check that rancher-config now contains a fleet entry with the desired contents.
The fleet chart release was updated as well and now is installed with debug: true.

Automated Testing

Test types added/modified:

Unit
Chart unit tests

Summary: existing tests keep working, and new ones have been added
QA Testing Considerations

Regressions Considerations
No expected regressions

@CamrynCarter are you OK in having this backported to 2.8?
On one hand, this is not a bug fix.
On the other hand, this buys Support/Operations people the possibility to set, for example, Fleet performance tuning parameters in case of Fleet performance problems. I see it as a way to provide users a workaround solution without necessarily waiting for a code fix. I have seen several such cases during the year, SURE-3805 being one internal example. We don't know when we will need it, but I believe we will at some point.
From a technical perspective engineers agree the patch is relatively small and risk is low (although of course we can't guarantee it's absolutely zero).
PS. if this is accepted, a related Fleet patch will likely follow to propagate parameters, if needed, to Fleet agents.

@moio So to double verify, this PR would allow for workarounds to existing and upcoming bugs? Any examples of existing bugs this would help with?

@CamrynCarter that's correct - this PR is the Rancher part of what is needed to provide workarounds (a corresponding Fleet part is also needed, that will be discussed as part of a separate Fleet pull request).
Example cases which required or still require code fixes that could have been avoided include SURE-6645 (5 reported cases), SURE-3805 (2 reported cases),SURE-6125 (2 reported cases).

Thanks @moio! In that case, I think this is something we should backport. @leflambeur are you okay with that as well?


C:\Users\virgo\AppData\Local\Microsoft\PowerToys\PowerToys Run\Logs\<IP_ADDRESS>\2021-06-06.txt

Microsoft PowerToys version
<IP_ADDRESS>
Running as admin

[ ] Yes

Area(s) with issue?
General, Settings
Steps to reproduce
Version: <IP_ADDRESS>
OS Version: Microsoft Windows NT 10.0.19042.0
IntPtr Length: 8
x64: True
Date: 06/06/2021 08:51:14
Exception:
System.IO.IOException: There is not enough space on the disk. : 'C:\Users\virgo\AppData\Local\Microsoft\PowerToys\PowerToys Run\Cache\Image.cache'
at System.IO.FileStream.WriteCore(ReadOnlySpan1 source)    at System.IO.FileStream.FlushWriteBuffer(Boolean calledFromFinalizer)    at System.IO.FileStream.Dispose(Boolean disposing)    at System.IO.Stream.Close()    at System.IO.Stream.Dispose()    at Wox.Infrastructure.Storage.BinaryStorage1.Save(T data)
at Wox.Infrastructure.Storage.BinaryStorage`1.TryLoad(T defaultData)
at Wox.Infrastructure.Image.ImageLoader.Initialize(Theme theme)
at PowerLauncher.App.<>c__DisplayClass14_0.b__0()
at Wox.Infrastructure.Stopwatch.Normal(String message, Action action)
at PowerLauncher.App.OnStartup(Object sender, StartupEventArgs e)
at System.Windows.Application.OnStartup(StartupEventArgs e)
at System.Windows.Application.<.ctor>b__1_0(Object unused)
at System.Windows.Threading.ExceptionWrapper.InternalRealCall(Delegate callback, Object args, Int32 numArgs)
at System.Windows.Threading.ExceptionWrapper.TryCatchWhen(Object source, Delegate callback, Object args, Int32 numArgs, Delegate catchHandler)
✔ Expected Behavior
There is not enough space on the disk. That's the issue
❌ Actual Behavior
Powertoys cant run
Why? look at this
Version: <IP_ADDRESS>
OS Version: Microsoft Windows NT 10.0.19042.0
IntPtr Length: 8
x64: True
Date: 06/06/2021 08:51:14
Exception:
System.IO.IOException: There is not enough space on the disk. : 'C:\Users\virgo\AppData\Local\Microsoft\PowerToys\PowerToys Run\Cache\Image.cache'
at System.IO.FileStream.WriteCore(ReadOnlySpan1 source)    at System.IO.FileStream.FlushWriteBuffer(Boolean calledFromFinalizer)    at System.IO.FileStream.Dispose(Boolean disposing)    at System.IO.Stream.Close()    at System.IO.Stream.Dispose()    at Wox.Infrastructure.Storage.BinaryStorage1.Save(T data)
at Wox.Infrastructure.Storage.BinaryStorage`1.TryLoad(T defaultData)
at Wox.Infrastructure.Image.ImageLoader.Initialize(Theme theme)
at PowerLauncher.App.<>c__DisplayClass14_0.b__0()
at Wox.Infrastructure.Stopwatch.Normal(String message, Action action)
at PowerLauncher.App.OnStartup(Object sender, StartupEventArgs e)
at System.Windows.Application.OnStartup(StartupEventArgs e)
at System.Windows.Application.<.ctor>b__1_0(Object unused)
at System.Windows.Threading.ExceptionWrapper.InternalRealCall(Delegate callback, Object args, Int32 numArgs)
at System.Windows.Threading.ExceptionWrapper.TryCatchWhen(Object source, Delegate callback, Object args, Int32 numArgs, Delegate catchHandler)
Other Software
<IP_ADDRESS>

@marioegghead2
Just to clarify, you want to see a message There is not enough space on the disk in the heading?
Or cache takes a lot of space?

yes

@marioegghead2
It is still unclear what you mean. Does cache takes a lot of space or you want to see the message in the heading? Maybe both

@mykhailopylyp you think we somehow maxed the disk out via Image.cache?


Added Numeric filter

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (fe13c0b) 100.00% compared to head (7db6e53) 100.00%.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@             Coverage Diff             @@
##                main        #6   +/-   ##
===========================================
  Coverage     100.00%   100.00%
- Complexity        71        82   +11
===========================================
  Files             10        12    +2
  Lines            212       231   +19
===========================================
+ Hits             212       231   +19

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Foldable for resolving props

Implement Foldable so that we can resolve props easily. Useful for monomorphization, since we need to resolve props that show up in if-else conditions, for one.

Should already be implemented, in subst.rs:163


[SPARK-24948][SHS][BACKPORT-2.2] Delegate check access permissions to the file system

What changes were proposed in this pull request?
In SparkHadoopUtil. checkAccessPermission,  we consider only basic permissions in order to check whether a user can access a file or not. This is not a complete check, as it ignores ACLs and other policies a file system may apply in its internal. So this can result in returning wrongly that a user cannot access a file (despite he actually can).
The PR proposes to delegate to the filesystem the check whether a file is accessible or not, in order to return the right result. A caching layer is added for performance reasons.
How was this patch tested?
added UT

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1902/
Test PASSed.

Test build #94360 has started for PR 22022 at commit 16b7b40.

Test build #94360 has finished for PR 22022 at commit 16b7b40.

This patch fails to build.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94360/
Test FAILed.

cc @jerryshao @mridulm

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1903/
Test PASSed.

Test build #94361 has started for PR 22022 at commit 657d364.

Test build #94361 has finished for PR 22022 at commit 657d364.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94361/
Test FAILed.

retest this please

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1909/
Test PASSed.

Test build #94369 has started for PR 22022 at commit 657d364.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1913/
Test PASSed.

Test build #94374 has started for PR 22022 at commit 16233d1.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1914/
Test PASSed.

Test build #94369 has finished for PR 22022 at commit 657d364.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94369/
Test FAILed.

Test build #94374 has finished for PR 22022 at commit 16233d1.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94374/
Test PASSed.

Test build #94401 has finished for PR 22022 at commit 16233d1.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged to branch 2.2, please close this PR @mgaido91

Thanks @jerryshao , closing.


HBASE-27205 Fix tests that rely on EnvironmentEdgeManager in branch-2.4

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
1m 18s
Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s
No case conflicting files found.

+1 :green_heart:
hbaseanti
0m  0s
Patch does not have any anti-patterns.

+1 :green_heart:
@author
0m  0s
The patch does not contain any @author tags.

_ branch-2.4 Compile Tests _

+1 :green_heart:
mvninstall
2m 11s
branch-2.4 passed

+1 :green_heart:
compile
2m  9s
branch-2.4 passed

+1 :green_heart:
checkstyle
0m 31s
branch-2.4 passed

+1 :green_heart:
spotless
0m 42s
branch has no errors when running spotless:check.

+1 :green_heart:
spotbugs
1m 16s
branch-2.4 passed

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
1m 53s
the patch passed

+1 :green_heart:
compile
2m  7s
the patch passed

+1 :green_heart:
javac
2m  7s
the patch passed

+1 :green_heart:
checkstyle
0m 30s
the patch passed

+1 :green_heart:
whitespace
0m  0s
The patch has no whitespace issues.

+1 :green_heart:
hadoopcheck
10m 25s
Patch does not cause any errors with Hadoop 2.10.0 or 3.1.2 3.2.1.

+1 :green_heart:
spotless
0m 39s
patch has no errors when running spotless:check.

+1 :green_heart:
spotbugs
1m 20s
the patch passed

_ Other Tests _

+1 :green_heart:
asflicense
0m 11s
The patch does not generate ASF License warnings.

30m  3s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4625/1/artifact/yetus-general-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/4625

Optional Tests
dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile

uname
Linux cfa1f8e985bf 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
branch-2.4 / 4b3464531a

Default Java
AdoptOpenJDK-1.8.0_282-b08

Max. process+thread count
65 (vs. ulimit of 12500)

modules
C: hbase-server U: hbase-server

Console output
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4625/1/console

versions
git=2.17.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
1m 13s
Docker mode activated.

-0 :warning:
yetus
0m  4s
Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck

_ Prechecks _

_ branch-2.4 Compile Tests _

+1 :green_heart:
mvninstall
2m 36s
branch-2.4 passed

+1 :green_heart:
compile
0m 42s
branch-2.4 passed

+1 :green_heart:
shadedjars
4m 11s
branch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 27s
branch-2.4 passed

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
2m 21s
the patch passed

+1 :green_heart:
compile
1m  5s
the patch passed

+1 :green_heart:
javac
1m  5s
the patch passed

+1 :green_heart:
shadedjars
4m 58s
patch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 32s
the patch passed

_ Other Tests _

+1 :green_heart:
unit
175m 13s
hbase-server in the patch passed.

195m  3s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4625/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/4625

Optional Tests
javac javadoc unit shadedjars compile

uname
Linux 08c0f1acb9e6 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
branch-2.4 / 4b3464531a

Default Java
AdoptOpenJDK-11.0.10+9

Test Results
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4625/1/testReport/

Max. process+thread count
2315 (vs. ulimit of 12500)

modules
C: hbase-server U: hbase-server

Console output
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4625/1/console

versions
git=2.17.1 maven=3.6.3

Powered by
Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

All tests passed. One of 3 hooks failed mvninstall but it looked like a transient failure to read a license file.


Add coverage test for forcetorque plugin

@traversaro Do you have any advice for the coverage test? Thank you in advance!

I never setup code coverage integration in our projects, but let's try. For example, an example codebase that integrates coverage is https://github.com/temken/template_cpp_cmake_libphysica .
Let's decompose the practice of doing coverage in some steps:

We just need to add coverage to one CI job, so probably for coverage we can concentrate on the apt job

To collect coverage data, you need to compile all your code with the --coverage flag, and in Debug mode (to avoid that different lines are merged together or re-ordered by the optimizers). Enabling this option ensure that whenever we run the tests, a .gcno file is also generated, that contains information on which lines have been tested by the test (see https://gcc.gnu.org/onlinedocs/gcc-12.1.0/gcc/Instrumentation-Options.html for more nfo)

Once you have all the .gcno files, you need something to collect them in a structured form, such as gcov or lcov

Then, you need someway to get the files by gcov/lcov and display them. Possible ways to do so are services like codecov (https://about.codecov.io/) or simple github actions like https://github.com/threeal/gcovr-action

Again, I do not have a lot of experience on this, so feel free to explore yourself also in other directions.

Then, you need someway to get the files by gcov/lcov and display them. Possible ways to do so are services like codecov (https://about.codecov.io/) or simple github actions like https://github.com/threeal/gcovr-action

For the moment we decided to use github actions like https://github.com/threeal/gcovr-action because to use other services like codecov (https://about.codecov.io/) we should configure them, but we don't have the required permissions

Ok! If you want to test any system just ask and I can add it to the repo.

If possible, we would like to try codecov. Thanks a lot!

If possible, we would like to try codecov. Thanks a lot!

Ok! I received this instructions:
Step 1: add repository token as [repository secret](https://github.com/robotology/gz-yarp-plugins/settings/secrets/actions)

Admin required to access repo settings > secrets and variable > actions

CODECOV_TOKEN=<..>

Step 2: add Codecov to your [GitHub Actions workflow yaml file](https://github.com/robotology/gz-yarp-plugins/tree/main/.github/workflows)

After tests run, this will upload your coverage report to Codecov:

- name: Upload coverage reports to Codecov
  uses: codecov/codecov-action@v3
  env:
    CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}

After you committed your changes and ran the repo's CI/CD pipeline. In your pull request, you should see two status checks and PR comment.
codecov patch and project

Once merged to the default branch, subsequent pull requests will have checks and report comment. Additionally, you'll find your repo coverage dashboard here.

I already completed Step 1, can you give me the mail you used to register to codecov so I can add you?

I can't find you, did you try to log in codecov at least once?

Yes I did

Yes I did

Ok, now I see you, you should have the access enabled.


Color space transform now accepts env. variable

The Color space transform is now accepting env. variable for the src and dst color space names.

Accept env. variables for the color space names
Add unit tests
Fix a typo and improve a unit test macro

Well, I attempted to resolve the merge conflict but that apparently broke the build. It looks like you had tried that before as well?

Refer to pull request #479 to have the right/final code change.


diff broken: considers two files or directories as delete+add rather than modify

I'm not sure when this happened (some time before v1.0.0), but I noticed that mal diff is basically broken. Given two directories:
/tmp/old
/tmp/old/lottie-player.min.js
/tmp/new
/tmp/new/lottie-player.min.js

If I run mal diff /tmp/old /tmp/new, it sees it as one file deleted and another one added, rather than a single file that changed:
% m diff /tmp/old /tmp/new

Deleted: ../../private/tmp/old/lottie-player.min.js [⚠ MEDIUM]
-------------------------------------------------------------------------------------------------------------------------------------
RISK  KEY                             DESCRIPTION                             EVIDENCE
-------------------------------------------------------------------------------------------------------------------------------------
-LOW  data/encoding/json/decode       Decodes JSON messages                   JSON.parse
-LOW  data/encoding/json/encode       encodes JSON                            JSON.stringify
-LOW  impact/words/plugin             references a 'plugin'                   function installPlugin
                                                                              getExpressionsPlugin
                                                                              plugins
                                                                              return expressionsPlugin
                                                                              setExpressionsPlugin
-LOW  net/url/embedded                contains embedded HTTPS URLs            https://www.jsdelivr.com/using-sri-with-dynamic-files
-LOW  net/url/parse                   Handles URL strings                     new URL
-MED  exec/remote_commands/code_eval  evaluate code dynamically using eval()  eval("
-MED  net/download/download           download files                          download_
-MED  os/time/clock/sleep             uses setInterval to wait                setInterval(
-------------------------------------------------------------------------------------------------------------------------------------

Added: ../../private/tmp/new/lottie-player.min.js [🚨 CRITICAL]
-----------------------------------------------------------------------------------------------------------------------------------------
RISK   KEY                               DESCRIPTION                                            EVIDENCE
-----------------------------------------------------------------------------------------------------------------------------------------
+LOW   c2/addr/url/unusual               Contains HTTP hostname with unusual top-level domain   https://api.mantlescan.xyz/
                                                                                https://mantlescan.xyz/
                                                                                https://openchain.xyz/
+LOW   credential/ssl/private_key        References private keys                                privateKey
+LOW   crypto/aes                        Supports AES (Advanced Encryption Standard)            AES
+LOW   crypto/ed25519                    Elliptic curve algorithm used by TLS and SSH           ed25519
+LOW   data/encoding/base64              Supports base64 encoded strings                        base64
+LOW   data/encoding/json/decode         Decodes JSON messages                                  JSON.parse
+LOW   data/encoding/json/encode         encodes JSON                                           JSON.stringify
+LOW   fs/file/open                      opens files                                            open(
+LOW   fs/mount                          mounts file systems                                    -o
                                                                                mount
+LOW   impact/words/password             references a 'password'                                PasswordBasedCipher
                                                                                to countless passwords
+LOW   impact/words/plugin               references a 'plugin'                                  plugin_relativeTime
                                                                                plugin_updateLocale
                                                                                plugins
+LOW   net/resolve/hostport/parse        Network address and service translation                getaddrinfo
+LOW   net/socket/socket/listen          listen on a socket                                     accept
                                                                                socket
+LOW   net/socket/socket/send            send a message to a socket                             _send
+LOW   net/url/embedded                  contains embedded HTTPS URLs                           https://abitype.dev
                                                                                https://andromeda-explorer.metis.io/api
                                                                                https://andromeda.metis.io/?owner=1088
                                                                                https://api-era.zksync.network/api
                                                                                https://api-moonbeam.moonscan.io/api
                                                                                https://api-moonriver.moonscan.io/api
                                                                                https://api-optimistic.etherscan.io/api
                                                                                https://api-zkevm.polygonscan.com/api
                                                                                …
+LOW   net/url/parse                     Handles URL strings                                    new URL
+LOW   os/env/get                        Retrieve environment variable values                   env.DEBUG
                                                                                env.MODE
                                                                                env.NEXT
                                                                                env.NODE
+LOW   os/fd/read                        reads from a file handle                               e.read()
+LOW   os/fd/write                       writes to a file handle                                a.write(o)
                                                                                decoder.write(n)
                                                                                decoder.write(t)
                                                                                e.write(t)
                                                                                i.write(e)
                                                                                t.write(o)
                                                                                this.write(e)
+MED   anti/static/obfuscation/generic/  converts hex data to ASCII                             toString("hex");
       hex_conversion
+MED   c2/addr/ip                        hardcoded IP address                                   <IP_ADDRESS>
                                                                                <IP_ADDRESS>3
                                                                                <IP_ADDRESS>
                                                                                <IP_ADDRESS>
                                                                                <IP_ADDRESS>
                                                                                146.288.257.686
                                                                                <IP_ADDRESS>
                                                                                <IP_ADDRESS>
                                                                                …
+MED   credential/keychain/keychain      May access the macOS keychain                          keychain
+MED   data/embedded/embedded/base64/    Contains base64 url                                    odHRwOi8v::$http
       url
+MED   discover/system/platform          get system identification                              process.platform
                                                                                process.versions
+MED   exec/remote_commands/code_eval    evaluate code dynamically using exec()                 exec(e))return
                                                                                exec(e),e
                                                                                exec(h)
                                                                                exec(l),null
                                                                                exec(o))
                                                                                exec(r))
                                                                                exec(t)
+MED   exfil/stealer/browser             Uses HTTP, archives, and references multiple browsers  .config
                                                                                Brave
                                                                                Chrome
                                                                                Discord
                                                                                Firefox
                                                                                Opera
                                                                                POST
                                                                                Safari
                                                                                …
+MED   fs/path/relative                  references and possibly executes relative path         ./aes
                                                                                ./blowfish
                                                                                ./cipher-core
                                                                                ./core
                                                                                ./evpkdf
                                                                                ./format-hex
                                                                                ./hmac
                                                                                ./lib-typedarrays
                                                                                …
+MED   impact/words/agent                references an 'agent'                                  useragent
+MED   impact/words/heartbeat            references a 'heartbeat'                               heartBeatTimeout
                                                                                heartbeat_pulse
                                                                                lastHeartbeatResponse
                                                                                updateLastHeartbeat
+MED   net/download/download             download files                                         Downloads
                                                                                downloads-view
                                                                                mobile-download-links
+MED   net/http/http/form/upload         upload content via HTTP form                           POST
                                                                                application/json
                                                                                application/x-www-form-urlencoded
+MED   net/http/http/post                submits content to websites                            Content-Type
                                                                                HTTP
                                                                                POST
                                                                                http
+MED   net/http/websocket                supports web sockets                                   WalletLinkWebSocket
                                                                                WebSocket:gV
                                                                                WebSocket:typeof
                                                                                WebSocketClass:h
                                                                                WebSocketClass:l
                                                                                clearWebSocket
                                                                                webSocket:e
                                                                                webSocket:r
                                                                                …
+MED   net/url/encode                    encodes URL, likely to pass GET variables              urlencode
+MED   net/url/request                   requests resources via URL                             requests.get(e)
+CRIT  exfil/stealer/wallet              makes HTTPS connections and references multiple        BraveWallet
                                         wallets by name                                        Coinbas
                                                                                Ronin
                                                                                http
-----------------------------------------------------------------------------------------------------------------------------------------

The same happens if I use specify the files by path name:
mal diff /tmp/old/lottie-player.min.js /tmp/new/lottie-player.min.js
Deleted: ../../../private/tmp/old/lottie-player.min.js [⚠ MEDIUM]
-------------------------------------------------------------------------------------------------------------------------------------
RISK  KEY                             DESCRIPTION                             EVIDENCE
-------------------------------------------------------------------------------------------------------------------------------------
-LOW  data/encoding/json/decode       Decodes JSON messages                   JSON.parse
-LOW  data/encoding/json/encode       encodes JSON                            JSON.stringify
-LOW  impact/words/plugin             references a 'plugin'                   function installPlugin
                                                                              getExpressionsPlugin
                                                                              plugins
                                                                              return expressionsPlugin
                                                                              setExpressionsPlugin
-LOW  net/url/embedded                contains embedded HTTPS URLs            https://www.jsdelivr.com/using-sri-with-dynamic-files
-LOW  net/url/parse                   Handles URL strings                     new URL
-MED  exec/remote_commands/code_eval  evaluate code dynamically using eval()  eval("
-MED  net/download/download           download files                          download_
-MED  os/time/clock/sleep             uses setInterval to wait                setInterval(
-------------------------------------------------------------------------------------------------------------------------------------

Added: ../../../private/tmp/new/lottie-player.min.js [🚨 CRITICAL]
--------------------------------------------------------------------------

Here is the output of v0.10.0, showing the expected behavior (except that the filename is "."):
go run . --diff /tmp/old/lottie-player.min.js /tmp/new/lottie-player.min.js
Changed: . [⚠ MEDIUM → 🚨 CRITICAL]

+++ ADDED: 24 behavior(s) +++
----------------------------------------------------------------------------------------------------------------------------
RISK   KEY                       DESCRIPTION                                            EVIDENCE
----------------------------------------------------------------------------------------------------------------------------
+LOW   crypto/aes                Supports AES (Advanced Encryption Standard)            AES
+LOW   crypto/ed25519            Elliptic curve algorithm used by TLS and SSH           ed25519
+LOW   encoding/base64           Supports base64 encoded strings                        base64
+LOW   env/get                   Retrieve environment variable values                   env.DEBUG
                                                                                env.MODE
                                                                                env.NEXT
                                                                                env.NODE
+LOW   fs/mount                  mounts file systems                                    -o
                                                                                mount
+LOW   net/hostport/parse        Network address and service translation                getaddrinfo
+LOW   net/socket/listen         listen on a socket                                     accept
                                                                                socket
+LOW   net/socket/send           send a message to a socket                             _send
+LOW   ref/site/url/unusual      Contains HTTP hostname with unusual top-level domain   https://api.mantlescan.xyz/
                                                                                https://mantlescan.xyz/
                                                                                https://openchain.xyz/
+LOW   ref/words/password        references a 'password'                                PasswordBasedCipher
                                                                                to countless passwords
+LOW   secrets/private_key       References private keys                                privateKey
+MED   combo/stealer/browser     Uses HTTP, archives, and references multiple browsers  .config
                                                                                Brave
                                                                                Chrome
                                                                                Firefox
                                                                                POST
                                                                                Safari
                                                                                http
                                                                                zip
                                                                                …
+MED   data/embedded/base64/url  Contains base64 url                                    odHRwOi8v::$http
+MED   kernel/uname/get          get system identification                              process.platform
                                                                                process.versions
+MED   net/http/form/upload      upload content via HTTP form                           "application/x-www-form-urlencoded
+MED   net/http/post             Able to submit content via HTTP POST                   HTTP
                                                                                POST
                                                                                http
+MED   net/url/encode            encodes URL, likely to pass GET variables              urlencode
+MED   net/url/request           requests resources via URL                             requests.get(e)
+MED   ref/ip                    hardcoded IP address                                   <IP_ADDRESS>
                                                                                <IP_ADDRESS>3
                                                                                <IP_ADDRESS>
                                                                                <IP_ADDRESS>
                                                                                <IP_ADDRESS>
                                                                                146.288.257.686
                                                                                <IP_ADDRESS>
                                                                                <IP_ADDRESS>
                                                                                …
+MED   ref/path/relative         references and possibly executes relative path         ./aes
                                                                                ./blowfish
                                                                                ./cipher-core
                                                                                ./core
                                                                                ./evpkdf
                                                                                ./format-hex
                                                                                ./hmac
                                                                                ./lib-typedarrays
                                                                                …
+MED   ref/words/agent           references an 'agent'                                  useragent
+MED   secrets/keychain          May access the macOS keychain                          keychain
+HIGH  ref/site/unusual          unusual http hostname                                  https://api.mantlescan.xyz/
                                                                                https://mantlescan.xyz/
                                                                                https://openchain.xyz/
+CRIT  combo/stealer/wallet      makes HTTPS connections and references multiple        BraveWallet
                                 wallets                                                Coinbas
                                                                                Ronin
                                                                                http
----------------------------------------------------------------------------------------------------------------------------

However we fix this, we need to add a test as our diff code is really difficult to understand and fragile.

@egibs - any chance you can help with this? I'm confident you can fix this far better and faster than I can.

It looks like there is at least one example where diff get's things right:

m diff ../bincapz-samples/linux/clean/ls.x86_64 ../bincapz-samples/macOS/clean/ls                                                                            695ms  Sat Nov  2 10:45:37 2024
Changed: ../bincapz-samples/macOS/clean/ls [⚠ MEDIUM → ✅ LOW]

+++ ADDED: 1 behavior(s) +++
---------------------------------------------------------------------------
RISK  KEY                    DESCRIPTION                    EVIDENCE
---------------------------------------------------------------------------
+LOW  fs/directory/traverse  traverse filesystem hierarchy  _fts_children
                                                            _fts_close
                                                            _fts_open
                                                            _fts_read
                                                            _fts_set
---------------------------------------------------------------------------

--- REMOVED: 3 behavior(s) ---
-------------------------------------------------------------------------------------------------------------------------------
RISK  KEY                           DESCRIPTION                          EVIDENCE
-------------------------------------------------------------------------------------------------------------------------------
-LOW  discover/system/hostname/get  get computer host name               gethostname
-LOW  net/url/embedded              contains embedded HTTPS URLs         https://gnu.org/licenses/gpl.html
                                                                         https://translationproject.org/team/
                                                                         https://wiki.xiph.org/MIME_Types_and_File_Extensions
                                                                         https://www.gnu.org/software/coreutils/
-MED  process/name/set              get or set the current process name  __progname
-------------------------------------------------------------------------------------------------------------------------------

Some weirdness: if I use a relative path, diff works:
% cd /tmp
% mal diff old new
├─ 🛑 Changed: new/lottie-player.min.js [MEDIUM → CRITICAL]
│     ▲ anti-static [NONE → MEDIUM]
++       🟡 obfuscation/generic/hex_conversion — converts hex data to ASCII: toString("hex");
│     ▲ command & control [NONE → MEDIUM]
++       🟡 addr/ip — hardcoded IP address:
++           <IP_ADDRESS>, <IP_ADDRESS>3, <IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS>, 146.288.257.686, <IP_ADDRESS>, <IP_ADDRESS>, …

If I specify absolute paths, it reverts to the deleted+added bug:
% cd /tmp
% mal diff /tmp/old /tmp/new
├─ 🟡 Deleted: ../../private/tmp/old/lottie-player.min.js [MEDIUM]
│     ≡ data [LOW]
│       🟢 encoding/json_decode — Decodes JSON messages: JSON.parse
│       🟢 encoding/json_encode — encodes JSON: JSON.stringify
│     ≡ execution [MEDIUM]
│       🟢 plugin — references a 'plugin':
│           function installPlugin, getExpressionsPlugin, plugins, return expressionsPlugin, setExpressionsPlugin
│       🟡 remote_commands/code_eval — evaluate code dynamically using eval(): eval("
│     ≡ networking [MEDIUM]
│       🟡 download — download files: download_
│       🟢 url/embedded — contains embedded HTTPS URLs: https://www.jsdelivr.com/using-sri-with-dynamic-files
│       🟢 url/parse — Handles URL strings: new URL
│     ≡ operating-system [MEDIUM]
│       🟡 time/clock_sleep — uses setInterval to wait: setInterval(
│
├─ 🛑 Added: ../../private/tmp/new/lottie-player.min.js [CRITICAL]

Interesting. I'll look into this first thing tomorrow.


Merge devel into gmm

Codecov Report

Merging #487 into GMM_GPLVM will increase coverage by 0.62%.
The diff coverage is 28.91%.

@@              Coverage Diff              @@
##           GMM_GPLVM     #487      +/-   ##
=============================================
+ Coverage      48.96%   49.59%   +0.62%
=============================================
  Files            191      190       -1
  Lines          18465    18301     -164
  Branches        2554     2536      -18
=============================================
+ Hits            9042     9076      +34
+ Misses          8990     8798     -192
+ Partials         433      427       -6

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 408e83c...627c878. Read the comment docs.


SIM118 Use key in vd instead of key in vd.keys() assumes __iter__ is over keys()

Example code:
class ValuesDict:
   """Dict-like container that iterates over the values."""
    def __init__(self, value):
        self.value = value

    def __iter__(self):
        yield from self.values()

    def keys(self):
        return self.value.keys()

    def values(self):
        return self.value.values()

d = {"a": 1, "b": 2, "c": 3}
vd = ValuesDict(d)

for key in vd.keys():
    print(key)

This is the original expected output:
$ python row.py
a
b
c

Applying fixes for SIM118:
$ ruff --select=SIM118 --fix row.py
Found 1 error(s) (1 fixed, 0 remaining).

$ git diff
diff --git a/row.py b/row.py
index 1e7536d..7751a3b 100644
--- a/row.py
+++ b/row.py
@@ -18,5 +18,5 @@ class ValuesDict:
 d = {"a": 1, "b": 2, "c": 3}
 vd = ValuesDict(d)

-for key in vd.keys():
+for key in vd:
     print(key)

This changes the code behavior:
$ python row.py
1
2
3

Yes I think we currently have many lints that assume a conventional implementation of some __dunder__ methods. I agree that we should categorize the applicability of such rules, I just opened #1997 for that :)

Yeah this is probably impossible to avoid completely right now. But we should categorize fixes as described in the issue and consider requiring opt-in for those that could be "dangerous".


Handling case where rqlite node is not available

In my previous PR, I preserved original behavior for when we can't connect to a node: rqlite-java prints a message that it couldn't connect to a node, then returns null.  I'd like to change it to print nothing and throw an exception that the client code can handle.  Is that okay?

That is fine with me, it seems more in line with proper Java style.


[ai-did-you-mean-this] Rename Required Service Parameter "sessionId" to "correlationId"

Rename required service parameter "sessionId" to "correlationId". Required by pending changes to the Aladdin prod endpoint.

This checklist is used to make sure that common guidelines for a pull request are followed.

About Extension Publish
There is a pipeline to automatically build, upload and publish extension wheels.
Once your PR is merged into master branch, a new PR will be created to update src/index.json automatically.
The precondition is to put your code inside this repo and upgrade the version in the PR but do not modify src/index.json.

If this PR is for a new extension or change to an existing extension, use the following to try out the changes in this PR:
docker run -it microsoft/azure-cli:latest
export EXT=<NAME>
pip install --upgrade --target ~/.azure/cliextensions/$EXT "git+https://github.com/christopher-o-toole/azure-cli-extensions.git@thoth-extension-required-service-parameter-change#subdirectory=src/$EXT&egg=$EXT"

ai-did-you-mean-this


Stop emails from sandbox

Context
The sandbox sends a lot of emails, many of them related to mass-generated applications. This is noisy for providers and for us.
Changes proposed in this pull request

Stop delivering emails from sandbox
Add an "Emails" tab to applications in the sandbox where providers can view the email history for an application.

Guidance to review

Are emails correctly stopped
Does the log work?

Link to Trello card
https://trello.com/c/9MUQ9jrv/2292-stop-emails-from-sandbox
Things to check

[X] This code doesn't rely on migrations in the same Pull Request
[X] If this code includes a migration adding or changing columns, it also backfills existing records for consistency
[X] API release notes have been updated if necessary
[X] New environment variables have been added to the Azure config

broken with master — will fix with rebase

Transient failure on Azure. This should be ready for another look


set_bipolar_reference fails for a specific montage

In the following example, I would like to use mne.set_bipolar_reference to define a bipolar montage/reference which is sometimes called "longitudinal anterior-posterior bipolar montage". From the list of electrodes (see fig below), new channels are to be created as follow :
F3 - C3
C3 - P3
P3 - O1
F4 - C4
C4 - P4
P4 - O2
F3 - FC5
FC5 - T7
... and so on. These new channels would be created by the function set_bipolar_reference by providing the right list of channels as anode and cathode. However, in this montage, F3 appears two times in anode. In set_bipolar_reference, when an is F3, the channel F3 is replaced with F3 - C3. Later, when we need F3 to create F3 - FC5, the channel F3 is gone and the function raises an error.

import os.path as op
from mne.datasets import testing
from mne import set_bipolar_reference
from mne.io import read_raw_eeglab
base_dir = op.join(testing.data_path(download=False), 'EEGLAB')
raw_fname = op.join(base_dir, 'test_raw.set')
montage = op.join(base_dir, 'test_chans.locs')

event_id = {'rt': 1, 'square': 2}
raw = read_raw_eeglab(input_fname=raw_fname, montage=montage,
    event_id=event_id, preload=True)
anode = ['F3', 'C3', 'P3', 'F4', 'C4', 'P4', 'F3', 'FC5', 'T7', 'P7',
         'PO7', 'F4', 'FC6', 'T8', 'P8', 'PO8']
cathode = ['C3', 'P3', 'O1', 'C4', 'P4', 'O2', 'FC5', 'T7', 'P7', 'PO7',
           'O1', 'FC6', 'T8', 'P8', 'PO8', 'O2']
raw = set_bipolar_reference(raw, anode, cathode)

If you want, I can suggest a PR to solve this issue.

@wmvanvliet I think you worked on this a bit, can you comment?

Looks like we need a replace_channels=False option. PR welcome!

We need this for some sEEG montages too ...

@jbschiratti can you open a WIP PR so you can get help ?

@jbschiratti can I close this?

Sure! Sorry, I should have closed it before.


Main2

Merge back temporary main branch after https://github.com/GoogleCloudPlatform/spanner-schema-diff-tool/pull/8
Checklist:

[x] how
[ ] this
[x] is

Closing this to make a new cleaner PR


Breaking changes plan for v2

There is no ETA for v2 yet, as there are not much important changes were made or planned to be, but at least we can track the things which can be improved or changed:

[ ] Make override: true input by default
[ ] Make profile: minimal input by default
[ ] Replace target input with targets, which will allow passing multiple targets to install
[x] Support for rust-toolchain file (#21) (not sure yet if breaking change) not a breaking change, yay!

In case you have an idea, feel free to add it in here as an issue comment.

Couldn't a targets input be provided, explicitly overriding targef? That would be back-compatible. Alternatively, it could be additive.

@jhpratt yeah, it is pretty much possible, but this feature does not look that important, so it is easier to postpone it till the breaking release.
Until then, simple run step can be used as a workaround, as this Action guarantees that there always will be a rustup in PATH:
- uses: actions-rs/toolchain@v1
  with: // skipped for clarity
- run: rustup target add my-new-target

targets would be great to have for embedded libraries that target multiple chip families!

another +1 for multiple targets here

Having a toolchains option similar to targets would be useful for some use cases as well. The advantage being the ability to auto-install targets when you know multiple toolchains will be used later on.

I'd also like to second the need for multiple targets. This is a situation that arises regularly for people wanting to use Rust as part of R packages on Windows, because R by default builds 32-bit and 64-bit binaries at the same time. (You can't separate it out into two builds, it always cross-compiles.)
How about adding a keyword targets that accepts more than one input, without removing target? In the future, in a breaking changes release, you could then remove target. Or not. I don't see any major problem with having both.

Oh, and I forgot to say: Adding an entire step just to add a target, and only on Windows, is annoying and clutters the build script and output. So I'd very much prefer not having to do that, even if it solves the problem in principle.

[ ] Make override: true input by default

I just want to say that I hugely support this!

[ ] Make override: true input by default

I just want to say that I hugely support this!


Downgrade rocm 5.2 to 5.1 to fix #46

the 2 AMD GPUs are detected and 2 instance of each tests gets spawn
but they endup using the NVIDIA GPU not the AMD ones

Order of indexes matters ?
git-lfs-linux-amd64-v2.13.3.tar.gz: OK
Looking in indexes: https://pypi.org/simple, https://download.pytorch.org/whl/rocm5.1/
Collecting torch
  Using cached torch-1.13.1-cp310-cp310-manylinux1_x86_64.whl (887.5 MB)
Collecting torchvision
  Using cached torchvision-0.14.1-cp310-cp310-manylinux1_x86_64.whl (24.2 MB)
Collecting torchaudio
  Using cached torchaudio-0.13.1-cp310-cp310-manylinux1_x86_64.whl (4.2 MB)
Collecting torchtext
  Using cached torchtext-0.14.1-cp310-cp310-manylinux1_x86_64.whl (2.0 MB)
Collecting gym
  Using cached gym-0.25.2-py3-none-any.whl
Collecting psutil
  Using cached psutil-5.9.4-cp36-abi3-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (280 kB)
Requirement already satisfied: typing-extensions in ./venv/torchbench/lib/python3.10/site-packages (from torch->-r /tmp/milabench/19268/code/torchbench/requirements-bench.txt (line 1)) (4.4.0)
Collecting nvidia-cudnn-cu11==<IP_ADDRESS>
  Using cached nvidia_cudnn_cu11-<IP_ADDRESS>-2-py3-none-manylinux1_x86_64.whl (557.1 MB)
Collecting nvidia-cuda-nvrtc-cu11==11.7.99
  Using cached nvidia_cuda_nvrtc_cu11-11.7.99-2-py3-none-manylinux1_x86_64.whl (21.0 MB)
Collecting nvidia-cuda-runtime-cu11==11.7.99
  Using cached nvidia_cuda_runtime_cu11-11.7.99-py3-none-manylinux1_x86_64.whl (849 kB)
Collecting nvidia-cublas-cu11==<IP_ADDRESS>
  Using cached nvidia_cublas_cu11-<IP_ADDRESS>-py3-none-manylinux1_x86_64.whl (317.1 MB)
Requirement already satisfied: setuptools in ./venv/torchbench/lib/python3.10/site-packages (from nvidia-cublas-cu11==<IP_ADDRESS>->torch->-r /tmp/milabench/19268/code/torchbench/requirements-bench.txt (line 1)) (65.6.3)

learning_to_paint.D1 >   File "/tmp/milabench/32336/code/torchbench/torchbenchmark/util/framework/huggingface/model_factory.py", line 4, in <module>
learning_to_paint.D1 >     import transformers
learning_to_paint.D1 > ModuleNotFoundError: No module named 'transformers'
learning_to_paint.D1 > Warning: Could not find dependent module cv2 for Model LearningToPaint, skip it
learning_to_paint.D1 > Unable to find model matching LearningToPaint.
learning_to_paint.D1 >! An error occurred

learning_to_paint.D1 >! Traceback (most recent call last):
learning_to_paint.D1 >!
  File "/tmp/milabench/32336/venv/torchbench/lib/python3.10/site-packages/voir/forward.py", line 82, in __call__
    yield ov.phases.run_script(priority=-1000)
learning_to_paint.D1 >!
  File "/tmp/milabench/32336/venv/torchbench/lib/python3.10/site-packages/voir/phase.py", line 183, in _step_one
    next_phase = gen.throw(ph.exception)
learning_to_paint.D1 >!
  File "/tmp/milabench/32336/milabench/milabench/instruments.py", line 389, in profile_gpu
    yield ov.phases.run_script
learning_to_paint.D1 >!
  File "/tmp/milabench/32336/venv/torchbench/lib/python3.10/site-packages/voir/overseer.py", line 65, in run
    result = func()
learning_to_paint.D1 >!
  File "/tmp/milabench/32336/venv/torchbench/lib/python3.10/site-packages/voir/utils.py", line 56, in <lambda>
    return lambda: exec(code, glb, glb)
learning_to_paint.D1 >!
  File "/tmp/milabench/32336/code/torchbench/run.py", line 170, in <module>
    exit(-1)
learning_to_paint.D1 >!
  File "/tmp/milabench/32336/conda/lib/python3.10/_sitebuiltins.py", line 26, in __call__
    raise SystemExit(code)
learning_to_paint.D1 >!
SystemExit: -1
learning_to_paint.D1 >!
learning_to_paint.D1 End: 2023-02-01 12:46:07.921393

learning_to_paint.D0 End: 2023-02-01 12:46:08.021645