mse_loss

🚀 Feature
Implement torch.nn.functional.mse_loss
Motivation
NeMo text-to-image model.

Hello Team! I am Kaeun, one of the new contributors at Thunder who is having a lot of fun with these tasks. I am wondering if it would be ok for me to handle this issue. It seems like I have to update torch/__init__.py with something like:
@torchsymbol(torch.nn.functional.mse_loss, id="torch.nn.functional.mse_loss", is_method=False)
def mse_loss(a: TensorLike, b: TensorLike, /) -> TensorLike:

Would it be ok if I take care of this? Appreciate your help and support! Best, Kaeun

Absolutely! Anyone is welcome to submit a PR to address this issue. Yes, this PR would start by updating torch/__init__.py.  A few additional notes:

Make sure to capture the additional arguments to the function, too
The decomposition of this function may be a little tricky
The torch executor can be updated to run the operation if it's called without having to execute the decomposition

Hi @mruberry! I am almost done with the implementation regarding mse_loss. However, I have some questions  and would appreciate your help! (this will help me a lot :) )

I am using the following script to quickly confirm the forward and backward pass of mse_loss.

import torch
import thunder

reduction = "none"

def mse(input, target):
    output = torch.nn.functional.mse_loss(input, target, reduction=reduction)
    return output

input = torch.randn(3, 5, requires_grad=True)
target = torch.randn(3, 5)

cfn = thunder.jit(mse)
actual_loss = cfn(input, target)

grad_jfn = thunder.core.transforms.grad(cfn)
actual_grad, = grad_jfn(input, target)

expected_loss = torch.nn.functional.mse_loss(input, target, reduction = reduction)
go = torch.ones_like(expected_loss)
expected_grad, = torch.autograd.grad(torch.nn.functional.mse_loss(input, target, reduction=reduction), input, go)

print("Max error in loss:", (actual_loss - expected_loss).abs().max().item())
print("Max error in logits grad:", (actual_grad - expected_grad).abs().max().item())

I am not sure if this is the proper way to calculate the gradient. Would you be able to check this for me? Also, just to check, does the above script call mse_loss with decomposition?

Regarding the backward pass that will be added in torch/__init__.py, I am trying to mimic what has been done with _cross_entropy_grad. However, I am having a hard time understanding the main purpose of _cross_entorpy_grad and cross_entropy_backward. cross_entropy_backward simply returns a TensorProxy of the gradient that has been computed through get_grad(fwd). Why would we want to separate the two?
When taking a look at the implementation of log_softmax_backward, it explicitly calculates the derivative of log_softmax and does not do any get_grad or put_grad. Why is there a difference between the cross_entropy's backward pass and log_softmax's backward pass?
For the mse_loss, I am assuming I should be doing something like the cross_entropy's implementation. Would that be a correct approach? (having mse_loss_backward and _mse_loss_grad)

Referring to the third bullet point in your comment, I am also currently working on implementing mse_loss in torchex.py. How would I call the mse_loss without having to execute the decomposition (as you have mentioned)? I am wondering if there is a way to check if I have implemented the non-decomposed mse_loss in torchex.py properly.

Hi @mruberry! I am almost done with the implementation regarding mse_loss. However, I have some questions and would appreciate your help! (this will help me a lot :) )

Happy to help!

I am using the following script to quickly confirm the forward and backward pass of mse_loss.

import torch
import thunder

reduction = "none"

def mse(input, target):
    output = torch.nn.functional.mse_loss(input, target, reduction=reduction)
    return output

input = torch.randn(3, 5, requires_grad=True)
target = torch.randn(3, 5)

cfn = thunder.jit(mse)
actual_loss = cfn(input, target)

grad_jfn = thunder.core.transforms.grad(cfn)
actual_grad, = grad_jfn(input, target)

expected_loss = torch.nn.functional.mse_loss(input, target, reduction = reduction)
go = torch.ones_like(expected_loss)
expected_grad, = torch.autograd.grad(torch.nn.functional.mse_loss(input, target, reduction=reduction), input, go)

print("Max error in loss:", (actual_loss - expected_loss).abs().max().item())
print("Max error in logits grad:", (actual_grad - expected_grad).abs().max().item())

I am not sure if this is the proper way to calculate the gradient. Would you be able to check this for me? Also, just to check, does the above script call mse_loss with decomposition (the one in torch/__init__.py not in torchex.py)?

You should be able to do the following, which might be simpler:
actual_loss = cfn(inp, target)
actual_loss.sum().backward()
thunder_grad = inp.grad
inp.grad = None

expected_loss = fn(inp, target)
expected_loss.sum().backward()
pytorch_grad = inp.grad

assert_close(thunder_grad, pytorch_grad)

@IvanYashchuk can correct me if I'm mistaken about this. Take a look at the assert_close utility for comparing tensors.
Your question about the decomposition is great. I'm not sure because it depends on the details the mse_loss implementation and any updates to the torchexecutor. You can see how the program is actually being executed by printing the execution trace of the program, which will show whether torch.nn.functional.mse_loss is being called directly, or a decomposition is being called instead.
Note for @jjsjann123 @t-vi @IvanYashchuk, we should probably reintroduce a developer option for the jit to force executors like torch to execute only primitives. This would be straightforward to do.
One way to test the decomposition locally is to not register a direct binding of mse_loss with the torch executor and check that the execution trace decomposes as expected. Then you can add the direct binding for the loss and verify it's working as expected, too. In the future this should be easier (once the above developer option is available).

Regarding the backward pass that will be added in torch/__init__.py, I am trying to mimic what has been done with _cross_entropy_grad. However, I am having a hard time understanding the main purpose of _cross_entorpy_grad and cross_entropy_backward. cross_entropy_backward simply returns a TensorProxy of the gradient that has been computed through get_grad(fwd). Why would we want to separate the two?
When taking a look at the implementation of log_softmax_backward, it explicitly calculates the derivative of log_softmax and does not do any get_grad or put_grad. Why is there a difference between the cross_entropy's backward pass and log_softmax's backward pass?

This is a great question! First, if you have a decomposition for mse_loss, and a grad formula is defined for every element of that decomposition, then you will also define an (implicit) grad formula for mse_loss, and shouldn't have to add a custom grad formula at all.
Second, the current state of grad formulas in thunder can be confusing! @IvanYashchuk can help direct you here. It would be helpful if you submitting a draft PR, so we can look at the code in more detail before making a recommendation.

For the mse_loss, I am assuming I should be doing something like the cross_entropy's implementation. Would that be a proper approach? (having mse_loss_backward and _mse_loss_grad)

Let's take a look at a draft PR, maybe even one where grad support isn't considered to start, and then we can discusss!

Referring to the third bullet point in your comment, I am also currently working on implementing mse_loss in torchex.py. How would I call the mse_loss without having to execute the decomposition (as you have mentioned)? I am wondering if there is a way to check if I have implemented the non-decomposed mse_loss in torchex.py properly.

The easiest way to check is to inspect the execution trace and verify that torch.nn.functional.mse_loss is called directly. An example of a "direct" lowering to torch is the dropout operation. See here:
https://github.com/Lightning-AI/lightning-thunder/blob/649c3d71de9af13a3247694a4e23289221d0ff72/thunder/executors/torchex.py#L1208
and here:
https://github.com/Lightning-AI/lightning-thunder/blob/649c3d71de9af13a3247694a4e23289221d0ff72/thunder/executors/torchex.py#L1548
The torch executor has some helper functions that make this straightforward. The _register_torch_operation function tells thunder how to call torch operations (like dropout), and the _register_implementation function tells thunder that it can call PyTorch's dropout to execute thunder.torch.dropout.

I hope you understand that I am in the process of learning how things work in thunder and would appreciate your help :)

These are great questions, and it's great you're asking them. I hope these responses are helpful. Let us know if you have additional questions!

Hi @mruberry!
Thanks so much for the detailed explanation!😄 It definitely helped me to further understand how trace execution can help my testing during the implementation and how gradients are calculated in general.
I have submitted a draft PR that at least passes tests/test_ops.py. Also, using the script that you have provided above, it seems to calculate the gradient and forward pass correctly. (+ I have performed traces = thunder.last_traces(cfn) and confirmed that it is calling the decomposition of mse_loss) It'll be awesome if you can take a look so we can further discuss about the implementation.
Currently, I am facing issues with tests/test_grad.py. Specifically, there is some discrepancy when running test_vjp_correctness_mse_loss_torch_cpu_float64 which I suspect is due to some implementation in torchex.py (strange how the current implementation's output is different to torch.ops.aten.mse_loss_backward's output).
I do have another question regarding your comment:

One way to test the decomposition locally is to not register a direct binding of mse_loss with the torch executor and check that the execution trace decomposes as expected.

Does this mean that I should not have something like:
mse_loss = _register_torch_operation("mse_loss", module=torch.nn.functional)

in torchex.py? And just run when doing
actual_loss = cfn(input, target)
actual_loss.sum().backward()
thunder_grad = input.grad
traces = thunder.last_traces(cfn)

I should be able to see the decomposition of mse_loss?

Then you can add the direct binding for the loss and verify it's working as expected, too. In the future this should be easier (once the above developer option is available).

This sounds like, once I add _register_torch_operation for mse_loss, I should be able to see torch.nn.functional.mse_loss in traces = thunder.last_traces(cfn). However, I am only seeing the decomposed version of mse_loss. Did I misunderstand something? Please let me know!
I am actually having a lot of fun and I really appreciate your help :)

Hi @mruberry!
Thanks so much for the detailed explanation!😄 It definitely helped me to further understand how trace execution can help my testing during the implementation and how gradients are calculated in general. I have submitted a draft PR that at least passes tests/test_ops.py.

Awesome! I'm glad that was helpful.

Also, using the script that you have provided above, it seems to calculate the gradient and forward pass correctly. (+ I have performed traces = thunder.last_traces(cfn) and confirmed that it is calling the decomposition of mse_loss) It'll be awesome if you can take a look so we can further discuss about the implementation.

I look forward to reviewing the PR in more detail.

Currently, I am facing issues with tests/test_grad.py. Specifically, there is some discrepancy when running test_vjp_correctness_mse_loss_torch_cpu_float64 which I suspect is due to some implementation in torchex.py (strange how the current implementation's grad output is different to torch.ops.aten.mse_loss_backward's output).

Interesting! Let's discuss more with @IvanYashchuk on the PR itself.

I do have another question regarding your comment:

One way to test the decomposition locally is to not register a direct binding of mse_loss with the torch executor and check that the execution trace decomposes as expected.

Does this mean that I should not have something like:
mse_loss = _register_torch_operation("mse_loss", module=torch.nn.functional)

in torchex.py? And just run when doing
actual_loss = cfn(input, target)
actual_loss.sum().backward()
thunder_grad = input.grad
traces = thunder.last_traces(cfn)

I should be able to see the decomposition of mse_loss?

Yes, I think that's correct. Without the direct binding of mse_loss registered in the torch executor it should have to execute it by decomposing it into other operations.

Then you can add the direct binding for the loss and verify it's working as expected, too. In the future this should be easier (once the above developer option is available).

This sounds like, once I add _register_torch_operation for mse_loss, I should be able to see torch.nn.functional.mse_loss in traces = thunder.last_traces(cfn). However, I am only seeing the decomposed version of mse_loss. Did I misunderstand something? Please let me know!

 The decomposition will still appear below it, but it will be commented out.

I am learning a lot from your comments and having a lot of fun! Thank you so much :)

You're very welcome; I'm glad you're having fun.


Merge release 0.22 into gold/2021

Merges main containing 0.22.0 tag into gold/2021

This PR looks good to me. Whether it can still be picked up for the release is up to @xaleryb

I'm going to fix the problem with wheels before merging this

@diptorupd  conda linux tests end with code 0 but in fact they didn't even start because of the following problem:
python -c 'import dpctl; print(" ".join([dev.backend.name+":"+dev.device_type.name for dev in dpctl.get_devices() if dev.device_type.name in ["cpu","gpu"]]))'
      from ._sycl_device import SyclDevice, SyclDeviceCreationError
  ImportError: dpctl/libDPCTLSyclInterface.so.0: undefined symbol: _ZN4sycl3_V17handler19getMaxWorkGroups_v2Ev

Speaking of wheels there are several tests failing:
FAILED tests/dpjit_tests/parfors/test_builtin_ops.py::test_built_in_operators[parfor_ipow-int32-device-100] - Failed: Failed to compile.
FAILED tests/dpjit_tests/parfors/test_builtin_ops.py::test_built_in_operators[parfor_ipow-int32-device-shape1] - Failed: Failed to compile.
FAILED tests/dpjit_tests/parfors/test_builtin_ops.py::test_built_in_operators[parfor_ipow-int32-host-100] - Failed: Failed to compile.
FAILED tests/dpjit_tests/parfors/test_builtin_ops.py::test_built_in_operators[parfor_ipow-int32-host-shape1] - Failed: Failed to compile.
FAILED tests/dpjit_tests/parfors/test_builtin_ops.py::test_built_in_operators[parfor_ipow-int32-shared-100] - Failed: Failed to compile.
FAILED tests/dpjit_tests/parfors/test_builtin_ops.py::test_built_in_operators[parfor_ipow-int32-shared-shape1] - Failed: Failed to compile.
FAILED tests/dpjit_tests/parfors/test_builtin_ops.py::test_built_in_operators[parfor_ipow-int64-device-100] - Failed: Failed to compile.
FAILED tests/dpjit_tests/parfors/test_builtin_ops.py::test_built_in_operators[parfor_ipow-int64-device-shape1] - Failed: Failed to compile.
FAILED tests/dpjit_tests/parfors/test_builtin_ops.py::test_built_in_operators[parfor_ipow-int64-host-100] - Failed: Failed to compile.
FAILED tests/dpjit_tests/parfors/test_builtin_ops.py::test_built_in_operators[parfor_ipow-int64-host-shape1] - Failed: Failed to compile.
FAILED tests/dpjit_tests/parfors/test_builtin_ops.py::test_built_in_operators[parfor_ipow-int64-shared-100] - Failed: Failed to compile.
FAILED tests/dpjit_tests/parfors/test_builtin_ops.py::test_built_in_operators[parfor_ipow-int64-shared-shape1] - Failed: Failed to compile.


[BUG] Cant play more than 10 mins without it freezing my pc.

Acknowledgement of preliminary instructions

[X] I have read the preliminary instructions, and I am certain that my problem has not already been addressed.

What problem did you encounter?
Every time i load into a game i either crash after 30 seconds or after 10 minutes, i have tried numerous ways of trying to fix this but nothing works.

when my game freezes all background noises stay playing but my windows key and alt-F4 don't work, and the only way to back out of the freeze is to turn off my pc manually. It is almost like the game is running normally but a new frame appears every 2-4 seconds.

Any support would be greatly appreciated, Thank you :)

Does it happen without? https://github.com/pizzaboxer/bloxstrap/wiki/Switching-between-Roblox-and-Bloxstrap

This might be a dupe of https://github.com/pizzaboxer/bloxstrap/issues/835 as multiple people are experiencing the same issue.
Check https://github.com/pizzaboxer/bloxstrap/issues/828 as well.

@genstarhimms, this might sound like not the problem but PC specs please?? Also, what version of windows??

Issue has been awaiting clarification for over a month, closing as stale. If you have anything further to add, just respond and I'll reopen.


Revert "Revert "Attempt to make CI faster""

Reverts helium/oracles#809
Worked here https://github.com/helium/oracles/actions/runs/9085930210

so this will run 11 postgres docker containers simultaneously?

yes 1 per job in the matrix


Generate "Tables" when creating Excel sheets

Currently we simply load a CSV into an Excel workbook using the xlsxwriter Python module. However, Excel has the concept of Tables and for convenience we can already create those in the script (instead of manually creating them afterwards)

The commit above also revealed that there was one cell with the value =brussels,brussels which was recognized as formulas. The script was adapted to not interprete cells as formulas.


Hosted Agent- Azure Container Instances -gtVersion 2.172.0

Having issue with software on Hosted Agent?
I configure the hosted container instance agent and get an error
"##[error]No agent found in pool aci-agent which satisfies the specified demands: Agent.Version -gtVersion 2.172.0"
Agent Platform:
Azure DevOps Services

Hi @cheahengteong there seems to be demand in your pipeline - agent version should be greater than 2.172.0.
You would probably need to update your agent, or at least make sure that auto-update is enabled for your agent pool.

Hi @cheahengteong do you have any updates?

@anatolybolshakov yes

@cheahengteong is this issue still actual for you? Have the steps above helped you to resolve it?

You can refer to the https://medium.com/@cloudlabs01/running-azure-self-hosted-agent-in-azure-container-instance-aci-ad1fa338d769

@ceteongvanness thanks! As I see there are instructions to set up self-hosted agent with azure container instance, although the error here seems to be related to demand required by some pipeline (agent version should be greater than 2.172.0). Seems like it was either configured with version less than 2.172.0, or there is some other possible reason).
@cheahengteong I'm closing this at the moment due to inactivity - please let us know if you have any questions.
For Azure DevOps related questions - you can also open a ticket on https://developercommunity.visualstudio.com/search?space=21 to get support


⚠ Bofrost.at has degraded performance

In a88a4f1, Bofrost.at (https://www.bofrost.at) experienced degraded performance:

HTTP code: 200
Response time: 1588 ms

Resolved: Bofrost.at performance has improved in e50bc90 after 20 minutes.


How can I add a button to exit Viro back into my app?

Here is a code sample to accomplish this -> https://gist.github.com/achuvm/4f0d414aa615d84fad0761beb57d723e
This is an example of adding your own button to a 360 degree VR experience (should also work with the ViroARSceneNavigator) in order to exit the Viro experience.
Drop this file in place of your index.{ios/android}.js file in a new React Viro project (react-viro init ViroSample) Don't forget to add YOUR_API_KEY and replace YOUR_PROJECT_NAME with your project name if you're not using the Viro Testbed App.


feature request : periodical folder scan

I have this workflow :
1/ request from Petio
2/ radarr + lidarr + sonarr
3/ download by qbittorrent
4/ to avoid unwanted new versions downloads from radarr + sonarr
-i use qbittorrent categories to move downloaded files to new destination folders and keep seeding
-once moved i delete movie/tv from lidarr/sonarr, so i have a short list of things there
5/ sometimes when a movie/tv series are too big
-i transcode them to new folder
-than delete the torrent and original file from qbittorrent
So my files move from folder to folder like this :
radarr : initial download to -> /partage/downloads/radarr
radarr create renamed mirror to file -> /partage/media/movies
post import qbittorrent category : move to -> /partage/videos/2d
then i delete movie from radarr :
-mirror file deleted in : /partage/media/movies
-but movie is present in the qbittorrent category folder and seeding : /partage/videos/2d
if movie is too big :
-transcoding to new file in transcode directroy : /transcode/videos/2d
-delete the torrent and file from qbittorrent : deleted in /partage/videos/2d
So my movie can be in 3 different folders :
/partage/media/movies -> renamed version from radarr
/partage/videos/2d -> seeding in qbitttorrent category folder
/transcode/videos/2d -> transcoded file
So to always have the movie in plex i have all 3 froders attached to the movie plex library.
When my file is downloaded via radarr , autoscan detects it and refrech Plex.
Then if i move the file to other directory (qbittorent category folder), then delete movie from radarr :
autoscan delete movie from Plex.
So in my workflow, autoscan deleted the movie from plex because i deleted it from radarr :
/partage/media/movies
But then my movie file exists in one of two other folders :
/partage/videos/2d -> seeding in qbittoreent, but deleted from radarr because i don't want to download new versions
/transcode/videos/2d -> deleted from radarr, than transcoded to transcode folder, then deleted from qbittorrent
It would be nice to say to autosacn to periodically scan files newer than last scan timestamp in list of folders,
then notify plex the file exists in the new folder (moved from one folder to an other).
There is no webhooks, only periodical folder scan + plex notification.
Would be nice to have this feature please,
Best regards :)

I'm curious why you want to see Autoscan implement this feature, given that it is already an option inside Plex?


Do not resolve view in provider's register() method.

You currently extend the view directly in the register() method. This should never be done at this point, because it may cause conflicts with other packages.
Either move this code to the boot() method, or - my preferred method, as the code will only be run when needed - wrap it in something like this:
$app->resolving('view', function($view) {
    // your stuff here...
});

Read my preferred way of minifying HTML in Laravel 5/5.1+ apps here: Using Gulp to Minify Laravel Blade Templates


add critical threshold for expired token

@craigtracey I modify the code as you comment, please review again.

@craigtracey I update the code again, which include:
-- fix double spaces in these two lines, and lines should be < 80 chars
-- change the variable name
and it did work.
root@controller2:/etc/sensu/plugins# ./check-keystone-expired-tokens.py -c 100
CRITICAL: expired tokens not being flushed. current count: 273
root@controller2:/etc/sensu/plugins# echo $?
2
root@controller2:/etc/sensu/plugins# ./check-keystone-expired-tokens.py -w 100
WARNING: expired tokens not being flushed. current count: 312
root@controller2:/etc/sensu/plugins# echo $?
1
root@controller2:/etc/sensu/plugins# ./check-keystone-expired-tokens.py
root@controller2:/etc/sensu/plugins# echo $?
0

please review again.


Nuitka error when using MSVC through prompt launchers

I'm using Python 3.9, and MSVC on windows
I installed nuitka with:
pip install --upgrade nuitka
Then I activated the x64 native MSVC command prompt and tried to compile this code:
import code
code.interact()
But I got this:
C:\py>nuitka c.py
Nuitka:INFO: Starting Python compilation.
Nuitka:INFO: Completed Python level compilation and optimization.
Nuitka:INFO: Generating source code for C backend compiler.
Nuitka:INFO: Running data composer tool for optimal constant value handling.
Nuitka:INFO: Running C level backend compilation via Scons.
Nuitka-Scons:INFO: Backend C compiler: cl (cl).
KeyError: 'WindowsSDKVersion':
File "C:\python\lib\SITE-P~1\nuitka\build\BACKEN~1.SCO", line 401:
scons_details_logger.info("Using Windows SDK %r." % env["WindowsSDKVersion"])
File "C:\python\lib\SITE-P~1\nuitka\build\INLINE~1\lib\scons-3.1.2\SCons\Environment.py",
line 405:
return self._dict[key]

That is probably an indication of you not using the right kind of MSVC. This one doesn't have a Windows SDK, and therefore won't work. I can make the error nicer, but that not going to change much.

That is probably an indication of you not using the right kind of MSVC. This one doesn't have a Windows SDK, and therefore won't work. I can make the error nicer, but that not going to change much.

Also, don't activate any MSVC prompt, Nuitka finds the correct one, and if it does not, well it's not installed. Chances are you picking one that doesn't match your Python arch, etc.
I am including code that makes this key default to "None" in which case this should be output.
    if not env["WindowsSDKVersion"]:
        scons_logger.sysexit(
            "Error, the Windows SDK must be installed in Visual Studio."
        )

Specifically "build tools" are known to not contain the right thing. Not sure though how to recognize it.

Also, don't activate any MSVC prompt, Nuitka finds the correct one, and if it does not, well it's not installed. Chances are you picking one that doesn't match your Python arch, etc.
I am including code that makes this key default to "None" in which case this should be output.
    if not env["WindowsSDKVersion"]:
        scons_logger.sysexit(
            "Error, the Windows SDK must be installed in Visual Studio."
        )

Specifically "build tools" are known to not contain the right thing. Not sure though how to recognize it.

You can see this on https://nuitka.net/doc/factory.html and it will be in the hotfix coming this weekend as of course, a KeyError is not a good user experience.

You can see this on https://nuitka.net/doc/factory.html and it will be in the hotfix coming this weekend as of course, a KeyError is not a good user experience.

Ok, seeing that you had previous had a working setup, I got more suspicious, and it seems that the WindowsSDK version is in fact only set, when Scons detects it, but not otherwise. That is of course a pity, because in the future, we want to be able to depend on it, but there is the environment itself set from outside shell, so this is about repairing stuff.

Ok, seeing that you had previous had a working setup, I got more suspicious, and it seems that the WindowsSDK version is in fact only set, when Scons detects it, but not otherwise. That is of course a pity, because in the future, we want to be able to depend on it, but there is the environment itself set from outside shell, so this is about repairing stuff.

So, factory was updated with this, sorry for my confusion, this seems important to fix.

So, factory was updated with this, sorry for my confusion, this seems important to fix.

Since you were talking about seconds to re-link in the other issue, one thing on my mind is that I saw it taking a second to run these batch scripts in scons, in case one is not running from the prompt.
Usually that's not too important, but I can see how it would be really nice for my CI it least, if that second could be taken away, esp. now that even clcache is non-forking, it's more likely to be closer to zero overhead. I just don't know how caching could work reliably, or how to make a cmd batch to run. Maybe you got more insights than me there @Keithcat1

Since you were talking about seconds to re-link in the other issue, one thing on my mind is that I saw it taking a second to run these batch scripts in scons, in case one is not running from the prompt.
Usually that's not too important, but I can see how it would be really nice for my CI it least, if that second could be taken away, esp. now that even clcache is non-forking, it's more likely to be closer to zero overhead. I just don't know how caching could work reliably, or how to make a cmd batch to run. Maybe you got more insights than me there @Keithcat1

I just ran this from a normal shell with no build tools command prompt activated and it still gave me GCC:
C:\py>nuitka c.py --msvc 2019
Nuitka:INFO: Starting Python compilation.
Nuitka:INFO: Completed Python level compilation and optimization.
Nuitka:INFO: Generating source code for C backend compiler.
Nuitka:INFO: Running data composer tool for optimal constant value handling.
Nuitka:INFO: Running C level backend compilation via Scons.
Nuitka-Scons:INFO: Backend C compiler: C:\Users\Keith\AppData\Local\Nuitka\Nuitka\gcc\x86_6
4\10.2.0-11.0.0-8.0.0-r5\mingw64\bin\gcc.exe (gcc.exe).
Nuitka-Scons:INFO: Compiled 11 C files using ccache.
Nuitka-Scons:INFO: Cached C files (using ccache) with result 'cache hit': 11
Nuitka:INFO: Keeping build directory 'c.build'.
Nuitka:INFO: Successfully created 'c.exe'.
BTW, could I give the --msvc option without an argument to have Nuitka automaticallyfind the right MSVC?

I just ran this from a normal shell with no build tools command prompt activated and it still gave me GCC:
C:\py>nuitka c.py --msvc 2019
Nuitka:INFO: Starting Python compilation.
Nuitka:INFO: Completed Python level compilation and optimization.
Nuitka:INFO: Generating source code for C backend compiler.
Nuitka:INFO: Running data composer tool for optimal constant value handling.
Nuitka:INFO: Running C level backend compilation via Scons.
Nuitka-Scons:INFO: Backend C compiler: C:\Users\Keith\AppData\Local\Nuitka\Nuitka\gcc\x86_6
4\10.2.0-11.0.0-8.0.0-r5\mingw64\bin\gcc.exe (gcc.exe).
Nuitka-Scons:INFO: Compiled 11 C files using ccache.
Nuitka-Scons:INFO: Cached C files (using ccache) with result 'cache hit': 11
Nuitka:INFO: Keeping build directory 'c.build'.
Nuitka:INFO: Successfully created 'c.exe'.
BTW, could I give the --msvc option without an argument to have Nuitka automaticallyfind the right MSVC?

If you picked the wrong MSVC, it would deactivate it. However, I think what is happening with your option there, is that is not how it works, I bet it's more like 14.2 or 14.3 or something.

If you picked the wrong MSVC, it would deactivate it. However, I think what is happening with your option there, is that is not how it works, I bet it's more like 14.2 or 14.3 or something.

This is what it did for me:
scons: warning: VC version 2019.0 not installed.  C/C++ compilers are most likely not set correctly.
 Installed versions are: ['14.2', '14.1']
File "C:\Users\kayha\repos\Py2C\nuitka\build\SconsUtils.py", line 101, in createEnvironment

scons: warning: VC version 2019.0 not installed.  C/C++ compilers are most likely not set correctly.
 Installed versions are: ['14.2', '14.1']
File "C:\Users\kayha\repos\Py2C\nuitka\build\SconsUtils.py", line 101, in createEnvironment
Nuitka-Scons:INFO: Backend C compiler: C:\Users\kayha\AppData\Local\Nuitka\Nuitka\gcc\x86_64\10.2.0-11.0.0-8.0.0-r5\mingw64\bin\gcc.exe (gcc.exe).

That are 2019 and 2017 versions I think. Arguably I could add symbolic names for them to be accepted, it kind of makes sense, doesn't it, but then I don't really want to enter MS versioning messes.
However, you not getting that is probably due to having a pre-selected compiler from the prompt, and nothing is changing that, and honestly, I think it would be really hard to change it, as the one in PATH is probably simply going to win.

This is what it did for me:
scons: warning: VC version 2019.0 not installed.  C/C++ compilers are most likely not set correctly.
 Installed versions are: ['14.2', '14.1']
File "C:\Users\kayha\repos\Py2C\nuitka\build\SconsUtils.py", line 101, in createEnvironment

scons: warning: VC version 2019.0 not installed.  C/C++ compilers are most likely not set correctly.
 Installed versions are: ['14.2', '14.1']
File "C:\Users\kayha\repos\Py2C\nuitka\build\SconsUtils.py", line 101, in createEnvironment
Nuitka-Scons:INFO: Backend C compiler: C:\Users\kayha\AppData\Local\Nuitka\Nuitka\gcc\x86_64\10.2.0-11.0.0-8.0.0-r5\mingw64\bin\gcc.exe (gcc.exe).

That are 2019 and 2017 versions I think. Arguably I could add symbolic names for them to be accepted, it kind of makes sense, doesn't it, but then I don't really want to enter MS versioning messes.
However, you not getting that is probably due to having a pre-selected compiler from the prompt, and nothing is changing that, and honestly, I think it would be really hard to change it, as the one in PATH is probably simply going to win.

Noting the duplicate however, with me not using --mingw the MSVC detection is run twice. There is code to disable for --mingw that code from scons, that I have now moved to where it is running when the fallback compiler environment.

Noting the duplicate however, with me not using --mingw the MSVC detection is run twice. There is code to disable for --mingw that code from scons, that I have now moved to where it is running when the fallback compiler environment.

This code is suspicious:
    # If we are on Windows, and MinGW is not enforced, lets see if we can
    # find "cl.exe", and if we do, disable automatic scan.
    if (
        os.name == "nt"
        and not mingw_mode
        and (
            getExecutablePath("cl", env=None) is not None
            or getExecutablePath("gcc", env=None) is not None
        )
    ):
        args["MSVC_USE_SCRIPT"] = False

I will have to checkout what MSVC_USE_SCRIPT would mean in your case. I guess, it might make the overriding from prompt work and the reason I disable it, is because there was a warning giving. This is very old stuff.

This code is suspicious:
    # If we are on Windows, and MinGW is not enforced, lets see if we can
    # find "cl.exe", and if we do, disable automatic scan.
    if (
        os.name == "nt"
        and not mingw_mode
        and (
            getExecutablePath("cl", env=None) is not None
            or getExecutablePath("gcc", env=None) is not None
        )
    ):
        args["MSVC_USE_SCRIPT"] = False

I will have to checkout what MSVC_USE_SCRIPT would mean in your case. I guess, it might make the overriding from prompt work and the reason I disable it, is because there was a warning giving. This is very old stuff.

How are you getting Nuitka to print that?
When I run this command, it still gives me GCC:
C:\py>nuitka --msvc=14.2 c.py
Nuitka:INFO: Starting Python compilation.
Nuitka:INFO: Completed Python level compilation and optimization.
Nuitka:INFO: Generating source code for C backend compiler.
Nuitka:INFO: Running data composer tool for optimal constant value handling.
Nuitka:INFO: Running C level backend compilation via Scons.
Nuitka-Scons:INFO: Backend C compiler: C:\Users\Keith\AppData\Local\Nuitka\Nuitka\gcc\x86_6
4\10.2.0-11.0.0-8.0.0-r5\mingw64\bin\gcc.exe (gcc.exe).
Nuitka-Scons:INFO: Compiled 11 C files using ccache.
Nuitka-Scons:INFO: Cached C files (using ccache) with result 'cache hit': 11
Nuitka:INFO: Keeping build directory 'c.build'.
Nuitka:INFO: Successfully created 'c.exe'.
I upgraded my nuitka to the latest source and now it automatically uses CL from the visual studio prompt again.

How are you getting Nuitka to print that?
When I run this command, it still gives me GCC:
C:\py>nuitka --msvc=14.2 c.py
Nuitka:INFO: Starting Python compilation.
Nuitka:INFO: Completed Python level compilation and optimization.
Nuitka:INFO: Generating source code for C backend compiler.
Nuitka:INFO: Running data composer tool for optimal constant value handling.
Nuitka:INFO: Running C level backend compilation via Scons.
Nuitka-Scons:INFO: Backend C compiler: C:\Users\Keith\AppData\Local\Nuitka\Nuitka\gcc\x86_6
4\10.2.0-11.0.0-8.0.0-r5\mingw64\bin\gcc.exe (gcc.exe).
Nuitka-Scons:INFO: Compiled 11 C files using ccache.
Nuitka-Scons:INFO: Cached C files (using ccache) with result 'cache hit': 11
Nuitka:INFO: Keeping build directory 'c.build'.
Nuitka:INFO: Successfully created 'c.exe'.
I upgraded my nuitka to the latest source and now it automatically uses CL from the visual studio prompt again.

@Keithcat1 your last comment is self-contradicting, is the last sentence the current state, i.e. what you describe is not happening anymore?

@Keithcat1 your last comment is self-contradicting, is the last sentence the current state, i.e. what you describe is not happening anymore?

What I am thinking is that you might have this, when I use the wrong prompt, in this case native x64 with Python 32 bits.
C:\Users\kayha\repos\Py2C>c:\Python27_32\python.exe bin\nuitka tests\basics\Asserts.py
Nuitka:INFO: Starting Python compilation.
Nuitka:INFO: Completed Python level compilation and optimization.
Nuitka:INFO: Generating source code for C backend compiler.
Nuitka:INFO: Running data composer tool for optimal constant value handling.
Nuitka:INFO: Running C level backend compilation via Scons.
Nuitka-Scons:INFO: Mismatch between Python binary ('c:\\Python27_32\\python.exe' -> 'pei-i386') and C compiler (None -> 'pei-x86-64') arches, ignored!
Nuitka will use gcc from MinGW64 of winlibs to compile on Windows.

The mismatch is only a warning, because e.g. in the Github CI, there is always a compiler in the PATH, a 32 bit one, while the Python's used are 64 bits, so it's normal, albeit a gcc there. I wonder why it says None now, that's slightly ugly, but in principle that is the prime reason a cl.exe from a prompt would be ignored.

What I am thinking is that you might have this, when I use the wrong prompt, in this case native x64 with Python 32 bits.
C:\Users\kayha\repos\Py2C>c:\Python27_32\python.exe bin\nuitka tests\basics\Asserts.py
Nuitka:INFO: Starting Python compilation.
Nuitka:INFO: Completed Python level compilation and optimization.
Nuitka:INFO: Generating source code for C backend compiler.
Nuitka:INFO: Running data composer tool for optimal constant value handling.
Nuitka:INFO: Running C level backend compilation via Scons.
Nuitka-Scons:INFO: Mismatch between Python binary ('c:\\Python27_32\\python.exe' -> 'pei-i386') and C compiler (None -> 'pei-x86-64') arches, ignored!
Nuitka will use gcc from MinGW64 of winlibs to compile on Windows.

The mismatch is only a warning, because e.g. in the Github CI, there is always a compiler in the PATH, a 32 bit one, while the Python's used are 64 bits, so it's normal, albeit a gcc there. I wonder why it says None now, that's slightly ugly, but in principle that is the prime reason a cl.exe from a prompt would be ignored.

Ok, I will need to reverse logging and clearing of selected compiler, now it tells me this:
Nuitka-Scons:INFO: Mismatch between Python binary ('c:\\Python27_32\\python.exe' -> 'pei-i386') and C compiler ('C:\\Program Files (x86)\\Microsoft Visual Studio\\2017\\Community\\VC\\Tools\\MSVC\\14.16.27023\\bin\\HostX64\\x64\\cl.exe' -> 'pei-x86-64') arches, ignored!

Seems I picked a 2017 prompt there too.

Ok, I will need to reverse logging and clearing of selected compiler, now it tells me this:
Nuitka-Scons:INFO: Mismatch between Python binary ('c:\\Python27_32\\python.exe' -> 'pei-i386') and C compiler ('C:\\Program Files (x86)\\Microsoft Visual Studio\\2017\\Community\\VC\\Tools\\MSVC\\14.16.27023\\bin\\HostX64\\x64\\cl.exe' -> 'pei-x86-64') arches, ignored!

Seems I picked a 2017 prompt there too.

And now, with --msvc=14.2 I can make it switch away from the wrong 14.1 of the prompt. Something that is maybe a bit uninformative as that it says only cl and not the full path, which I think I did in other places, but I don't want to immediately go there.

And now, with --msvc=14.2 I can make it switch away from the wrong 14.1 of the prompt. Something that is maybe a bit uninformative as that it says only cl and not the full path, which I think I did in other places, but I don't want to immediately go there.

I am also making a check if you specified the MSVC version for the fallback to be used. Using it would be against what you said.

I am also making a check if you specified the MSVC version for the fallback to be used. Using it would be against what you said.

So, on factory it should now work to override a selected prompt with --msvc-version. The error outputs should contain compiler paths rather than None (a gcc too old error had the same issue) and there is no fallback to gcc being done when --msvc-version is specified. In a way, that is a way of enforcing MSVC now.
Please try it out, and let me know if that works as expected for you.

So, on factory it should now work to override a selected prompt with --msvc-version. The error outputs should contain compiler paths rather than None (a gcc too old error had the same issue) and there is no fallback to gcc being done when --msvc-version is specified. In a way, that is a way of enforcing MSVC now.
Please try it out, and let me know if that works as expected for you.

While these other changes are not, the fix for build tools to work was in <IP_ADDRESS> and I think that should work now, so I am closing this, these other things are just general improvements.


Can't load DICOM data

Hello,
I'm trying to load DICOM MRI data in medInria but for every file I get this error:
E: can't load data dictionary
DEBUG - mer. août 1 11:39:02 2018 - Unrecognized component type:  0
WARN  - mer. août 1 11:39:02 2018 - Reader was unable to read:  "/path/to/file"

This issue was observed on both Mac and linux, on the 3.0.0 version

Solved by https://github.com/medInria/medInria-public/pull/352

This issue can be closed (i don't have the rights to close it)

It's Ok, I close it.


Extend the clean and shutdown flags to BenchmarkConfig.py

What this PR does and why we need it:
Extension to https://github.com/bazelbuild/bazel-bench/pull/164. This allows to specify clean and shutdown flags in the config (yaml) file.

@joeleba sorry for another ping - can you take a look please?


A11y_OSvisualupdate_MiniControGallery_GenericControls_Screenreader : Using NVDA the screen reader users are unable activate the control while navigation using "Insert + arrow keys".

GitHubTags: #A11y_OSvisualupdate, #A11yMAS, #A11yTCS, #Win10, #DesktopApp, #WindowsUILibrary, #MAS1.3.1, #MAS2.1.1, #NVDA
Environment Details:
Application Name: OS visual update
Windows Version: Windows10
Screen Reader: Narrator, NVDA 2019.2
Steps to Reproduce:

Launch Mini Control Gallery App from the installer location.
Screen with all generic controls would appear on left side of the screen.
Verify that all the elements of "MiniControlGallery-Generic Controls" are accessible and MAS Compliant.

Actual:
When screen reader users navigate through the controls on the screen using NVDA with the “Insert + arrow keys”, then they are unable to activate controls but when they navigate using the “Tab” key they are able to activate it.
Similar issue is observed in the below scenario:

MiniControlGallery - Calender and Color Picker Controls -> Launch Mini Control Gallery App from the installer location. -> Screen with Calender and Color Picker controls would appear on right side of the screen.

Expected:
The screen reader users should be able to activate the control even when they are navigating using the “Insert + arrow keys”.
User Impact:
The screen reader users are not able to interact with the controls using NVDA then they will miss out the information that is able for the keyboard users.
Attachment:
A11y_OSvisualupdate_MiniControGallery_GenericControls_Screenreader  Using NVDA the screen reader users are unable activate the control while navigation using Insert + arrow ke.pptx

I don't know enough about NVDA, what patterns or properties are different such that Narrator works but NVDA doesn't?


Dijkastra’s Shortest Path Algorithm Under Graph Algorithms

Name:
[Dijkastra’s Shortest Path Algorithm]
About:
Dijkstra's Shortest Path Algorithm is a popular algorithm used to find the shortest path from a source node to all other nodes in a graph with non-negative edge weights. It works by maintaining a priority queue (min-heap) of nodes based on their current shortest distance from the source. Starting from the source, the algorithm iteratively selects the node with the smallest known distance, updates the distances to its neighbors, and repeats until all nodes are processed. The result is the shortest path from the source to every other node in the graph.
@AlgoGenesis
I want to write a C program for Dijkstra's Algorithm

@pankaj-bind I would be happy to work on it. Please assign me the issue.


Update index.html

added my profile  ;-)

@HGlab01  contribute to https://github.com/Ishaan28malik/Get-Dev-Resources instead.


 Need to:

Tidy up the code
Wrap all duplicate logic in functions
Have a more cohesive process with a logical chain of events
Figure out whether you're putting client-side logic in script.js or renderer.js

Dealt with in merge of version-1-refactoring branch.


[Tests] Fix RemoteMirror test to work when DYLD_LIBRARY_PATH is set.

Changed the RemoteMirror test to work around DYLD_LIBRARY_PATH when it's given an absolute path; when it's called from CI, we really want it to load the libswiftRemoteMirror.dylib that we just built, not the system one, which is what happens for some of the builders.
rdar://82124292

@swift-ci Please test

Build failed
Swift Test OS X Platform
Git Sha - d39e06919579ca30d12ece6ec29a2733db9bcd78

@swift-ci Please smoke test

@swift-ci Please test

@swift-ci Please test Windows platform

@swift-ci Please smoke test Windows platform

@swift-ci Please smoke test

@swift-ci Please test

Build failed
Swift Test OS X Platform
Git Sha - d39e06919579ca30d12ece6ec29a2733db9bcd78

Build failed
Swift Test Linux Platform
Git Sha - d39e06919579ca30d12ece6ec29a2733db9bcd78

@swift-ci Please test

@swift-ci Please smoke test

Build failed
Swift Test OS X Platform
Git Sha - d39e06919579ca30d12ece6ec29a2733db9bcd78

@swift-ci Please test

@swift-ci Please smoke test


stream_manipulator_3d: 0.1.7-0 in 'indigo/distribution.yaml' [bloom]

Increasing version of package(s) in repository stream_manipulator_3d to 0.1.7-0:

upstream repository: https://github.com/3DVision-Stack/stream-manipulator-3D.git
release repository: https://github.com/3DVision-Stack/stream-manipulator-3D-release.git
distro file: indigo/distribution.yaml
bloom version: 0.5.21
previous version for package: 0.1.6-0

rqt_stream_manipulator_3d
* Updates & Bugfixes
  Bugfix: CropBox not updating Translation properly
  Add Pause/Resume functionality:
  - Pause button will block processing and all plugins, it also
  unsubscribe from input_topic.
  - Resume will resubscribe and resume all normal functionality.
  - While stream_manipulator_3d is paused you cannot modify configuration
  or plugin specific settings
  Add Saving/Loading configuration functionality:
  - Saving config to rosparams and then dumpng to yaml file
  - Loading yaml file into rosparams, then update configuration from them.
* Add two more filter plugins
  * Add Statistical Outlier Removal Plugin
  Wrapping  of  pcl::StatisticalOutlierRemoval  filter.  StatisticalOutlierRemoval
  uses  point  neighborhood  statistics  to filter  outlier  data.  The  algorithm
  iterates through  the entire  input twice:  During the  first iteration  it will
  compute the average distance that each point has to its nearest k neighbors. The
  value of k  can be set using  nr_k_neighbors param. Next, the  mean and standard
  deviation of all  these distances are computed in order  to determine a distance
  threshold. The distance threshold will be equal to: mean + stddev_mult * stddev.
  The multiplier  for the  standard deviation can  be set  using stddev_multiplier
  parameter. During the next iteration the  points will be classified as inlier or
  outlier if  their average  neighbor distance  is below  or above  this threshold
  respectively. Outliers will then be removed from the stream. The neighbors found
  for each query point will be found amongst ALL input points.
  * Add Radius Outlier Removal Plugin
  Wrapping  of  pcl::RadiusOutlierRemoval   filter.  RadiusOutlierRemoval  filters
  points in a cloud  based on the number of neighbors  they have. Iterates through
  the entire  input once, and  for each point,  retrieves the number  of neighbors
  within a certain radius.  The point will be considered an outlier  if it has too
  few neighbors, as determined by  k_neighbors_threshold parameter. The radius can
  be changed using  radius_search param. The neighbors found for  each query point
  will be found amongst ALL points of input stream.
* Contributors: Federico Spinelli

stream_manipulator_3d
* Updates & Bugfixes
  Bugfix: CropBox not updating Translation properly
  Add Pause/Resume functionality:
  - Pause button will block processing and all plugins, it also
  unsubscribe from input_topic.
  - Resume will resubscribe and resume all normal functionality.
  - While stream_manipulator_3d is paused you cannot modify configuration
  or plugin specific settings
  Add Saving/Loading configuration functionality:
  - Saving config to rosparams and then dumpng to yaml file
  - Loading yaml file into rosparams, then update configuration from them.
* Add two more filter plugins
  * Add Statistical Outlier Removal Plugin
  Wrapping  of  pcl::StatisticalOutlierRemoval  filter.  StatisticalOutlierRemoval
  uses  point  neighborhood  statistics  to filter  outlier  data.  The  algorithm
  iterates through  the entire  input twice:  During the  first iteration  it will
  compute the average distance that each point has to its nearest k neighbors. The
  value of k  can be set using  nr_k_neighbors param. Next, the  mean and standard
  deviation of all  these distances are computed in order  to determine a distance
  threshold. The distance threshold will be equal to: mean + stddev_mult * stddev.
  The multiplier  for the  standard deviation can  be set  using stddev_multiplier
  parameter. During the next iteration the  points will be classified as inlier or
  outlier if  their average  neighbor distance  is below  or above  this threshold
  respectively. Outliers will then be removed from the stream. The neighbors found
  for each query point will be found amongst ALL input points.
  * Add Radius Outlier Removal Plugin
  Wrapping  of  pcl::RadiusOutlierRemoval   filter.  RadiusOutlierRemoval  filters
  points in a cloud  based on the number of neighbors  they have. Iterates through
  the entire  input once, and  for each point,  retrieves the number  of neighbors
  within a certain radius.  The point will be considered an outlier  if it has too
  few neighbors, as determined by  k_neighbors_threshold parameter. The radius can
  be changed using  radius_search param. The neighbors found for  each query point
  will be found amongst ALL points of input stream.
* Contributors: Federico Spinelli

We're in a  quiet period holding indigo releases in preparation for a sync. If it's important for this to be in the sync please comment on this thread titled http://discourse.ros.org/t/proposed-indigo-sync-aug-1-2016/381


Command CompileSwift failed with a nonzero exit code

Seems failure on compiling library.
Local Build & Archive works perfectly.
Any clue for it? ;(
with Clean Build, Release Config
w/o Clean Build, Default Config

It was Xcode Version Issues. Editting Xcode Version on Stack resolves this issue!


Bumping Boto3

Issue #, if available:
It generated an error when building the lambci python3.8 Docker image.
Why is this change necessary?
Bumps to parity with aws-cli and gets rid of build errors on lambci python3.8 Docker image
How does it address the issue?
Eliminates the build error.
What side effects does this change have?
It bumps boto3 from 1.13.x to 1.14.x.
Did you change a dependency in requirements/base.txt?
Yes.
If so, did you run make update-reproducible-reqs
No. For security reasons I would feel better if a core maintainer updated the hashcodes.
Checklist:

[ ] Write Design Document (Do I need to write a design document?)
[ ] Write unit tests
[ ] Write/update functional tests
[ ] Write/update integration tests
[ ] make pr passes
[ ] Write documentation

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
Yes.

Please see my comment here: https://github.com/lambci/docker-lambda/issues/296#issuecomment-661104076.
This is only a short term fix. Docker-lambda (and any other tool) should be installing python tools into isolated venv, if they use pip. Otherwise, this will happen again in the future with your function code or AWS CLI.

#2130 takes Jacob's feedback and replaces this PR - thank you for the contribution!


Incorrect Rounding Applied When resizing

There are certain situation when rounding isn't applied correctly.
Given the snippet below, you can see that rounding of the final two values in the array incorrectly rounds down.
It seems to have something to do with the dtype being used to store the value under the hood.  if t_fxp is changed such that the total number of bits <32, then the rounding gets applied correctly.
I have also noticed that this issue is only occuring when the new array is created from an existing FXP array.  ie using .resize, or initializing using another FXP array as the initial value.   The rounding is correct if initializing from an array of floats.
It also looks like the rounding is correct for FXP initialized from a scalar float.
from fxpmath import Fxp, Config
import numpy as np

cfg=Config(dtype_notation="Q",rounding="around")
t_fxp = Fxp(0.0,1,n_int=16,n_frac=15,config=cfg)   # Rounding fails if sign+n_int+n_frac >= 32
t_int = Fxp(0.0,1,n_int=13,n_frac=0,config=cfg)

arr = [-5,0,14.8,7961.625]
fullprec        = Fxp(arr      , like=t_fxp )
rounded_direct  = Fxp(arr      , like=t_int )    # Works
rounded         = Fxp(fullprec , like=t_int )    # Doesn't Work
print(repr(fullprec))
print(repr(rounded_direct))
print(repr(rounded))

scalar_full         = Fxp(7961.625     , like=t_fxp )
scalar_round_direct = Fxp(7961.625     , like=t_int )            #<- Works
scalar_round        = Fxp(scalar_full  , like=t_int )            #<- Also Works
print(repr(scalar_full))
print(repr(scalar_round_direct))
print(repr(scalar_round))

The above will print out
Q17.15([-5.00000000e+00  0.00000000e+00  1.47999878e+01  7.96162500e+03])
Q14.0([-5.000e+00  0.000e+00  1.500e+01  7.962e+03])
Q14.0([-5.000e+00  0.000e+00  1.400e+01  7.961e+03])
Q17.15(7961.625)
Q14.0(7962.0)
Q14.0(7962.0)

Hello @DanChianucci
Thanks for report the issue. Which version are you using?
I ran your example with v0.4.6 and it is working.

 I tested it in windows and I could replicate the problem.
When fullprec is created, a internal raw value has 32 bits. In this case, numpy rounding functions don't work, so rounding process is by-passed (maybe I should add a warning message), although even, precision of float (32 bits) make you lose information to be rounded.
How the value passed to rounded doesn't have a resolution greater than 32 bits, it could be fixed internally (I will include that in next release).
Just like a workaround (well... here work around is not a "happy" word, hehe), you can try:
rounded         = Fxp(fullprec() , like=t_int ) #note () after fullprec to pass float values instead Fxp

Please, try new version 0.4.7 and check if now it is working in Windows.

I have already test in a PC with Windows 64 bits, and could replicate your conditions (uint equivalent to uint32). The problem was a mistake of mine to not include a change from dev branch.
The comparison in lines you mentioned are necessary to deal with this problem, forcing to work numpy with python objects instead of uint. The problem I found is that when input value has raw value as object type.
You should get it fixed doing including following code line between line 809 and 810 of objects.py:
val = val.astype(original_vdtype)

this results in:
            else:
                val = val.astype(original_vdtype) # <----- added line
                val_dtype = np.int64 if self.signed else np.uint64

If you see line 840 to 842, this change is already implemented for complex values.
Please, let me know if you try this!
Version 0.4.7 should have this change :( !!
I've run test again, and it pass ok!
def test_issue_60_v0_4_6():
    cfg=Config(dtype_notation="Q",rounding="around")

    t_fxp = Fxp(0.0,1,n_int=16,n_frac=15,config=cfg)
    t_int = Fxp(0.0,1,n_int=13,n_frac=0,config=cfg)

    arr = [-5,0,14.8,7961.625]
    fullprec        = Fxp(arr      , like=t_fxp )
    rounded_direct  = Fxp(arr      , like=t_int )
    rounded         = Fxp(fullprec , like=t_int )
    assert np.all(rounded_direct == rounded)

    scalar_full         = Fxp(7961.625     , like=t_fxp )
    scalar_round_direct = Fxp(7961.625     , like=t_int )
    scalar_round        = Fxp(scalar_full  , like=t_int )

    assert scalar_round_direct == scalar_round

Yep! That seems to have fixed the issue

I close the issue considering it solved in version 0.4.8.
Thanks!


Feat(Algebra/Group/Subgroup/Pointwise): Additive Subgroup.pointwise_smul_def

Add additive version of Subgroup.pointwise_smul_def.

Thanks! :tada:
bors merge


Feature/support proxy for freshclam

PR for #19
Test Case (with ajilaag/clamav-rest image):
/etc/clamav # ./test.sh
Replaced proxy server #=> ensuring sed works with new delimiter when testing
Replaced proxy port #=> ensuring sed works with new delimiter when testing
Replaced proxy user and pass #=> ensuring sed works with new delimiter when testing

/etc/clamav # cat freshclam.conf | grep "HTTP*"
# The HTTPProxyServer may be prefixed with [scheme]:// to specify which kind
#   http://     HTTP Proxy. Default when no scheme or proxy type is specified.
#   https://    HTTPS Proxy. (Added in 7.52.0 for OpenSSL, GnuTLS and NSS)
HTTPProxyServer https://proxy-test.com
HTTPProxyPort 54321
HTTPProxyUsername TESTUSER
HTTPProxyPassword TEST/PASS
#HTTPUserAgent SomeUserAgentIdString

Usage:
If $PROXY_SERVER is not set in an environment variable, then none of this will run
If  $PROXY_SERVER is set then replace the value in freshclam.conf

Since port, username and password are not required, only apply them to freshclam.conf if $PROXY_PORT, $PROXY_USERNAME and $PROXY_PASSWORD are set in an environment variable

@rhermanek would you mind merging this for us? We're hoping to avoid maintaining our own image.


Add API functions for EDS parser

ssia.

Idea: add test function to the test report API, #69
This way you can output test reports for conformance tests.


Typo fixed："Iteratating" -> "Iterating"

Fixs typo in lin 304

/assign @bobsky

Hi @JoeWrightss Thank you for your contribution.  Files under /blog are reviewed by a separate group of reviewers.  If you feel strongly about the changes you can contact the blog owners team located in https://github.com/kubernetes/website/blob/master/content/en/blog/OWNERS.   Since this PR cannot move forward through our standard review process I am closing this PR.


RPC docs seem to be missing for 25.0

See https://bitcoincore.org/en/doc/

@stickies-v wanna follow up here like #919?


iis 10 on win server 2016

im setting new empty ( just the site visual studio create for me ) web site on the iis
and receive error 500.19
any help

Could be any of the reasons. Did you install AspNetCoreModule on IIS for example? (It comes with Windows Server Hosting Bundle which you can find at https://www.microsoft.com/net/download/core#/runtime)

We periodically close 'discussion' issues that have not been updated in a long period of time.
We apologize if this causes any inconvenience. We ask that if you are still encountering an issue, please log a new issue with updated information and we will investigate.


Add a "set -x" to the dind-sidecar test script.

Changes
I was bitten by this test not having one, when the "docker build"
failed but we continued on executing anyway. I did a cursory look
at the other tests, and they either already have this or are so
short/simple that it doesn't matter.
Ref https://github.com/tektoncd/pipeline/pull/3628
/kind cleanup
Submitter Checklist
These are the criteria that every PR should meet, please check them off as you
review them:

[X] Includes tests (if functionality changed/added)
[X] Includes docs (if user facing)
[X] Commit messages follow commit message best practices
[X] Release notes block has been filled in or deleted (only if no user facing changes)

See the contribution guide for more details.
Double check this list of stuff that's easy to miss:

If you are adding a new binary/image to the cmd dir, please update
the release Task to build and release this image.

Reviewer Notes
If API changes are included, additive changes must be approved by at least two OWNERS and backwards incompatible changes must be approved by more than 50% of the OWNERS, and they must first be added in a backwards compatible way.
Release Notes
NONE

/test pull-tekton-pipeline-unit-tests

/test tekton-pipeline-unit-tests

/lgtm

/lgtm


RANGER-2128: Implementation of Ranger Spark SQL plugin

pre-work
Basic concepts and introductions can be found in spark-authorizer's documentations.
additionals
https://github.com/apache/spark/pull/17724 exposed a new experimental develop api SparkSessionExetensions, which is able to add user supplied extensions to SparkSession object during instantiation via program api or the spark property named spark.sql.extensions.
This PR uses spark.sql.extensions and other necessary ranger-hive-plugin settings to enable Ranger security support for Spark SQL with hive as external catalog.
spark.sql.extensions=
org.apache.ranger.authorization.spark.authorizer.RangerSparkSQLExtension

@boscodurai ship it?

keen to see this in action! do you have any jar and setup notes for this?

@yaooqinn can u create apache review?

@tooptoop4 review request created.

Hi!
This work is very interesting!
Any progress on the merge?

@yaooqinn ^ Can you fix the conflicts?

@yaooqinn any thoughts on resolving the conflicts?

What does this PR do/accomplish that isn't already possible with the existing Hive support? We're currently running Spark Thriftserver (3.2.x) with the kyuubi plugin against Ranger where in Ranger we've defined the service as a Hive service and everything with regards to authentication and authorization seems to be working as expected.
The only thing that I've observed that doesn't work is the auto-complete when creating policies via the Ranger UI, I assume this is a slight dialect difference in the response from the Spark Thriftserver vs a real HiveServer2.

What does this PR do/accomplish that isn't already possible with the existing Hive support? We're currently running Spark Thriftserver (3.2.x) with the kyuubi plugin against Ranger where in Ranger we've defined the service as a Hive service and everything with regards to authentication and authorization seems to be working as expected.
The only thing that I've observed that doesn't work is the auto-complete when creating policies via the Ranger UI, I assume this is a slight dialect difference in the response from the Spark Thriftserver vs a real HiveServer2 since the query being run by Ranger (show databases like "*") returns the databases just fine when I run it myself.

Thanks @simonvanderveldt. I think than we can include Kyuubi plugin in Ranger for Spark.
If you are familiar with Kyuubi than Can you please raise demo PR?

+1

will this work with spark-submit cluster mode without passing keytab?

I am going to close this in favor of the kyuubi spark authz plugin


How can I train up models?

as title.

please see also https://github.com/taizan/PaintsChainer/issues/13


Add program change events

Adds NoteEvent::ProgramChange.

Thanks! I'll add functions for outputting parameter changes from the process function later, if I don't forget.


Desktop configuration dosen't work in IDEA

Issue details
When running 'Desktop' run configuration in IntelliJ IDEA the error below appears.
Error:com.android.tools.idea.gradle.util.Projects.lastGradleSyncFailed(Lcom/intellij/openapi/project/Project;)Z

Reproduction steps/code
Following the setup instructions, I've installed

Android SDK API 26 Platform

With sdkmanager I've installed:

Are Android emulator and Android NDK necessary?
Enabled the org.robovm.idea-2.3.1-plugin-dist.jar plugin by install form disk option.
Following the instructions for creating a libgdx project with the following settings as in the article:

Then going to instructions about Gradle and IntelliJ IDEA.
I imported the project and set up a "Desktop" configuration as described (working directory pointing to /Users/erik/Documents/Dev/repos/mygame/android/assets) and when running it, the error mentioned above appears.
My project can be seen here. (I've tried to include the IntelliJ project files...)
Version of LibGDX and/or relevant dependencies
libgdx: 1.9.6 (in build.gradle)
JDK: 1.8.0_141
org.robovm.idea-2.3.1-plugin-dist.jar enabled in Preferences -> Plugins -> Install plugin from disk...
IntelliJ IDEA IntelliJ IDEA 2017.2
Build #IU-172.3317.76, built on July 15, 2017
JRE: 1.8.0_152-release-915-b5 x86_64
JVM: OpenJDK 64-Bit Server VM by JetBrains s.r.o
Mac OS X 10.12.5

Stacktrace
The error message is:
Error:com.android.tools.idea.gradle.util.Projects.lastGradleSyncFailed(Lcom/intellij/openapi/project/Project;)Z

Please select the affected platforms

[ ] Android
[ ] iOS (robovm)
[ ] iOS (MOE)
[ ] HTML/GWT
[ ] Windows
[ ] Linux
[x] MacOS

Uninstall your robovm plugin, or upgrade it to the latest snapshot

It worked, thanks. Btw is the Android NDK necessary for building for Android?

Nope


x11 windows  display is out of screen for pi4 hardware res 1.4

hi all,
sorry to bother you again,
I am playing  pi4 hardware res 1.4, meta-rasberrypi  commit 1014d41ef459af807f95cb7ff9105f077bc6daa
I am using a 13.3" LCD Lilliput TK1330-NP/C/T screen   (mode :https://lilliputdirect.com/TK1330-NP-C-T-OF    ). before the pi4 res 1.2 and older meta-rasberrypi worked well.
but for the pi4 hardware res 1.4, for DVI port as input, the menu display position seems OK, but the display is not clear and some color points for display.
for HDMI port as input, the menu top position is a little out of the screen. we need resolution 1280X720. I tried a few resolutions, eg 1920x1080 .. all of them are out of the screen at the top side.
please the attached picture, at the top, I should show 3 -- but only 2 --,  the top is out of the screen.

see more picture:   please waiting booting is not show all

my resolution settings, I think it's corrent, because DVI as input port is OK, why HDMI is bad?
my 18.5' and 10 '' different company's monitor work well.
any help? can I use the older driver? if OK, what files do I need to replace by the older driver?
Thank you very much.
root@raspberrypi4-64:~# xrandr
Screen 0: minimum 320 x 200, current 1280 x 720, maximum 7680 x 7680
HDMI-1 connected primary 1280x720+0+0 (normal left inverted right x axis y axim
1280x720_60.00  59.86*+
1920x1080     60.02
1680x1050     59.88
1680x945      60.02
1400x1050     59.95
1600x900      60.00
1280x1024     60.02
1440x900      59.90
1280x960      60.00
1360x768      60.02
1280x800      59.91
1152x864      59.97
1280x768      59.99
1280x720      60.00
1024x768      60.00
1024x576      59.97
800x600       60.32
640x480       59.94
cat /etc/X11/xorg.conf.d/10-monitor.conf
Section "Monitor"
Identifier "HDMI-1"
            Modeline "1280x720_60.00"   74.50  1280 1344 1472 1664  720 72c

EndSection
Section "Screen"
Identifier "Screen0"
Device "HDMI-1"
Monitor "HDMI-1"
DefaultDepth 24
SubSection "Display"
Depth 24
Modes "1280x720_60.00" "1024x768"
EndSubSection
EndSection

Hi all,
we added a adapter , it works.
like the following connection:
PI4 --------micro HDMI ----------HDMI ------ HDMI to DVI adapter ------- DVI input ------- 13" display


Automated documentation for available operators

Is your feature request related to a problem? Please describe.
I've received feedback that the CHANGELOG is difficult to parse when users need to learn about available operators and their import statements. We used to maintain a list of these operators on docs.astronomer.io, but with the frequency of astronomer-providers releases it was becoming difficult to maintain and risked becoming out-of-date.
Describe the solution you'd like
We should have an automated solution for documenting available providers on https://astronomer-providers.readthedocs.io/en/stable/ so that we:

Don't have to double-document operators.
Don't force users to visit the CHANGELOG as a source of truth.

Could we regularly parse the CHANGELOG for new table rows, then upload those entries to a second doc? Or generate a doc directly from source code.
Describe alternatives you've considered
We've tried documenting these operators on docs.astronomer.io, but at this point it doesn't make sense for us to manually update a table with dozens of operators.

Below is a grep command to bring the list of all operators and sensors in our repo, may be we can use it to autogenerate list of operators and sensors present on our repo.
(conda_venv) phanikv@Phanis-MBP providers % grep --exclude=\*.rst  -r -e '^class [a-zA-Z]\+[Sensor,Operator]Async([a-zA-Z]\+)'| cut -d ' ' -f 2 | cut -d "(" -f 1 | sort
AzureDataFactoryPipelineRunStatusSensorAsync
AzureDataFactoryRunPipelineOperatorAsync
BatchOperatorAsync
BigQueryCheckOperatorAsync
BigQueryGetDataOperatorAsync
BigQueryIntervalCheckOperatorAsync
BigQueryTableExistenceSensorAsync
BigQueryValueCheckOperatorAsync
DatabricksRunNowOperatorAsync
DatabricksSubmitRunOperatorAsync
DataprocSubmitJobOperatorAsync
EmrContainerOperatorAsync
EmrContainerSensorAsync
EmrJobFlowSensorAsync
EmrStepSensorAsync
ExternalTaskSensorAsync
FileSensorAsync
GCSObjectExistenceSensorAsync
GCSObjectUpdateSensorAsync
GCSObjectsWithPrefixExistenceSensorAsync
GCSUploadSessionCompleteSensorAsync
GKEStartPodOperatorAsync
HivePartitionSensorAsync
HttpSensorAsync
KubernetesPodOperatorAsync
LivyOperatorAsync
NamedHivePartitionSensorAsync
RedshiftClusterSensorAsync
RedshiftPauseClusterOperatorAsync
RedshiftResumeClusterOperatorAsync
RedshiftSQLOperatorAsync
SnowflakeOperatorAsync

@phanikumv We will need some way to publish this to our Sphinx docs.
Some options that we discussed with the team y'day were:

Sphinx parsing itself
ast parse + sphinx
And the one you have here i.e. Bash + Pre-commit Hook.

cc @pankajastro @pankajkoti @bharanidharan14

@jwitz Until this is done we now also have an entry on the Astronomer Registry.
Operators: https://registry.astronomer.io/providers/astronomer-providers?type=Operators
Sensors: https://registry.astronomer.io/providers/astronomer-providers?type=Sensors

Wrote a python script to generate list of operators and sensors using ast.parse.
Working next on how to generate document from this Python list of operators and sensors.

hi @jwitz the docs are available now with the latest release here: https://astronomer-providers.readthedocs.io/en/stable/providers/operators_and_sensors_list.html


Cannot Create Dynamic Link

Hey,
Here is my code.
`
    $url = 'https://vienot.page.link';

    Firebase::project('app')->auth();
    try {
        $dynamicLinkInstance = Firebase::dynamicLinks()->createDynamicLink($url);
    } catch (CreateDynamicLink\FailedToCreateDynamicLink $e) {
        echo $e->getMessage(); exit;
    }

`
Here is the error:
Invalid Dynamic Link domain: '' or Domain Uri Prefix: ''. Expecting exactly one. Dynamic Link Domain isPresent = false,
Domain URI prefix isPresent = false,
[https://firebase.google.com/docs/dynamic-links/rest#create_a_short_link_from_parameters]
I dont have any idea. I can use cloud messaging easily but can not use dynamic links.
Everything ready at my Firebase panel by the way.
Is there any example code?

It seems like you haven't configured the dynamic-links domain in https://github.com/kreait/laravel-firebase/blob/main/config/firebase.php#L102 (either directly in the config file or via environment variable).
You need to configure the dynamic-links domain with your https://vienot.page.link/ and use the URL call the createDynamicLink method with the URL you want to shorten.
In your .env file:
FIREBASE_DYNAMIC_LINKS_DEFAULT_DOMAIN=https://vienot.page.link/

In your code:
$url = 'https://the.url/you/want-to-shorten';

try {
    $dynamicLinkInstance = Firebase::dynamicLinks()->createDynamicLink($url);
} catch (CreateDynamicLink\FailedToCreateDynamicLink $e) {
    echo $e->getMessage(); exit;
}

Oh, it looks great but the problem is still going.
Invalid Dynamic Link domain: '' or Domain Uri Prefix: 'https://vienot.page.link/'. Expecting exactly one. Dynamic Link
Domain isPresent = false, Domain URI prefix isPresent = false,
[https://firebase.google.com/docs/dynamic-links/rest#create_a_short_link_from_parameters]
I've added .env file like you provided.
Here is the code:
`
    $url = 'https://vienot.app/12342';
    Firebase::project('app')->auth();
    $dynamicLinkInstance = Firebase::dynamicLinks();
    try {
        $link = $dynamicLinkInstance->createDynamicLink($url);
    } catch (CreateDynamicLink\FailedToCreateDynamicLink $e) {
        echo $e->getMessage(); exit;
    }

`

Glad it worked out!


Good project. Nice job. Hope to add support for multiple k lines on the same chart

eg:
okex's btcusdt and binance's btcusdt on the same chart for comparing.

This idea seems feasible, but I need some time.

One way to do this is to support a custom dataset like:
[
  {
    'timestamp': 0000,
    'close1': 5,
    'close2': 10
  },
  {
    'timestamp': 0001,
    'close1': 6,
    'close2': 11
  },
  ...
]

I need more time.

Custom technical indicator support this.


NativeEngine may be doing more copying than is necessary

In a number of places, NativeEngine.cpp is using bgfx::copy to create locally owned copies of data from Babylon.js (this line, for example). This is probably redundant, as Babylon.js is already keeping copies to deal with device loss and other problems, so it should be possible to have the NativeEngine operate on refs alone so long as it's given access to the permanently-stored versions of data on the JavaScript side, not temporary copies. Fixing will require changes on both managed and native sides.

It's unclear for me if the typed array will still be available when the function exists.
https://github.com/BabylonJS/BabylonNative/blob/2759935c851538afcdbde8fef748b861abdfdf92/Library/Source/NativeEngine.cpp#L460
does it create a new typedarray or keeps a reference on an existing one?

Not sure. We need to find out. Typed arrays in JavaScript are intended to be backed by a native array.

Does it make sense to close this issue?

Closing as we have to copy now due to security restrictions.


examples/c: add socket filter example

This is a simplified example I port from linux samples/bpf directory, many code snippet came from there.
In this example, I opened a SOCK_RAW socket in lo interface, then attach it to the bpf program for monitoring.
I use BPF_MAP_TYPE_RINGBUF to carry information back to user space program, and the user space program will print formatted information to stdout, for example:
interface:lo    protocol: UDP   <IP_ADDRESS>:51845(src) -> <IP_ADDRESS>:53(dst)
interface:lo    protocol: UDP   <IP_ADDRESS>:41552(src) -> <IP_ADDRESS>:53(dst)

Currently, most of the IPv4 protocols defined in uapi/linux/in.h are included, please check ipproto_mapping for the supported protocols.
Another thing to be done is that we need to add another linux uapi header into libbpf submodule :  uapi/linux/ip.h, because we need to retrieve protocol, saddr and daddr from struct ip_hdr.
I will temporarily mark this PR as draft, please give me some suggestions for this, thank you!

In general, have you thought about doing the same thing in tc hook instead?  tc is a more popular hook for bpf prog.  Also, it should be faster (kernel does not always have to clone the skb) and has more bpf features/helpers.  For example, it can directly read skb->data.  The tc hook has much more helper to find useful things also.  Just name a few, when the tc-bpf prog intercepted an interesting src/dst ip, it can find the sk and print more information about this sk.  e.g. the sock cookie and what tcp state it is in...etc.

In general, have you thought about doing the same thing in tc hook instead? tc is a more popular hook for bpf prog. Also, it should be faster (kernel does not always have to clone the skb) and has more bpf features/helpers. For example, it can directly read skb->data. The tc hook has much more helper to find useful things also. Just name a few, when the tc-bpf prog intercepted an interesting src/dst ip, it can find the sk and print more information about this sk. e.g. the sock cookie and what tcp state it is in...etc.

Thanks a bunch for all of these useful information!
Actually I haven't tried this before, but I think we can cover as many bpf program types as possible:
https://github.com/libbpf/libbpf/blob/4eb6485c08867edaa5a0a81c64ddb23580420340/src/libbpf.c#L9002-L9081
Maybe I can add another example for tc bpf program later, what kind of implementation do you think, is suitable for a tc bpf program?

Sorry for delay, looks good. skeleton__attach() isn't necessary (socket programs are not auto-attachable, so it has no effect, I'll drop it and force-push locally, no need to resubmit).
But also (I'm not a big BPF networking expert), at least for some use cases, skb->data and skb->data_end are probably a better approach than more heavyweight bpf_skb_load_bytes(). Have you used data/data_end special pointers before? Do they work in this case? If yes, maybe let's use them at least for some accesses to improve the example and demonstrate both ways of accessing data?

Sorry for delay, looks good. skeleton__attach() isn't necessary (socket programs are not auto-attachable, so it has no effect, I'll drop it and force-push locally, no need to resubmit).
But also (I'm not a big BPF networking expert), at least for some use cases, skb->data and skb->data_end are probably a better approach than more heavyweight bpf_skb_load_bytes(). Have you used data/data_end special pointers before? Do they work in this case? If yes, maybe let's use them at least for some accesses to improve the example and demonstrate both ways of accessing data?

After tracking down linux source code, I thought skb->data and skb->data_end are not accessible in BPF_PROG_TYPE_SOCKET_FILTER, reference:
https://github.com/torvalds/linux/blob/072e51356cd5a4a1c12c1020bc054c99b98333df/net/core/filter.c#L8365-L8392
I will try to use this in other examples, thanks a lot for your suggestions!

Very well could be, as I said not a big BPF networking expert here :)

Very well could be, as I said not a big BPF networking expert here :)

It's OK, you looks like an expert to me!
Still thanks for the information anyway.


Handling of wrong paths

Currently the library does not handle gracefully when a wrong file path is passed to pyedflib.EdfReader() since it raises a generic 'malloc error' (see trace below):

Traceback (most recent call last):
f = pyedflib.EdfReader(data_EDF)
File "pyedflib_extensions_pyedflib.pyx", line 91, in pyedflib._extensions._pyedflib.CyEdfReader.init (pyedflib_extensions_pyedflib.c:2674)
File "pyedflib_extensions_pyedflib.pyx", line 125, in pyedflib._extensions._pyedflib.CyEdfReader.open (pyedflib_extensions_pyedflib.c:3287)
File "pyedflib_extensions_pyedflib.pyx", line 101, in pyedflib._extensions._pyedflib.CyEdfReader.check_open_ok (pyedflib_extensions_pyedflib.c:2846)
IOError: malloc error

Handling of wrong path is fixed in the current master

Thank you!


Variables in config

Paths, queries and headers should allow using simple variables e.g.
in main config:
"auth" = "auth-token-12345"
in request
{
  "headers" : {
     "Auth-Token" : "${auth}"
  }
}

Can we use similar mechanism to "response file" instead? We would not have a variable, but we would have file that could be reused in multiple requests.
Additionally we could override/attach content of the file using old "headers" syntax like this:
"headers file": "commonHeaders.json"
"headers" : {
     "Auth-Token" : "auth-token-12345"
  }```
This syntax could be also added for "queries" as "queries file".

For now I've added support for "headers file" and "queries file". ( 568768701a3121abf2afb55041f5f0b68101a2ac ). If it is not sufficient please reopen this issue.

Available for testing in 2.0.0-SNAPSHOT


TS-4987: Don't prefer loopback over API set origin server address.

Linux build failed! See https://ci.trafficserver.apache.org/job/Github-Linux/946/ for details.

FreeBSD build successful! See https://ci.trafficserver.apache.org/job/Github-FreeBSD/1055/ for details.

Linux build successful! See https://ci.trafficserver.apache.org/job/Github-Linux/947/ for details.

Looking back, it seems this is basically handled (but better) with #1079 ?

@zwoop this is different; you need to check whether the API set an address before worrying about whether it is loopback.

I see. The first portion of this patch is important, whereas the second portion is done (better?) in #1079.

WIth regard to #1079, this doesn't intersect with that change. This maintains the existing loopback only check, it just does it better by dropping the pointless yet fragile string comparisons.


refactor(lodash): remove find usage

This removes all lodash/find usages. We need to be careful with undefined collections being loop over since Lodash used to deal with that for us.
Note that we already use Array.prototype.find in the codebase and that we document using a polyfill to support browsers.

@samouss Do you mean that it would be safer to create our own util?

@francoischalifour Yep. @algolia/instantsearch-for-websites opinions?

Yes, I'd go for polyfill here too for the reasons discussed by @samouss. This is likely going to break implementations since we didn't use it visibly before

That would make sense to me.
When you say "polyfill", you mean an actual polyfill or a wrapper function?

We shouldn't polyfill, but instead use our own util function which uses indexOf. The real polyfill on MDN seems more advanced, but not things we really need: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/find#Polyfill (it also has a note about not polyfilling)

I created a find util function that uses the native find method if available, otherwise an implementation with filter.
I chose to keep the native one when available because it's optimized to return once the item is found (it doesn't go through all items).
If you're concerned about the reduce implementation which goes through all the items, we can refactor it with a basic loop.
Once this is merged, we can remove the find mention in the Browser support section of the documentation.


quasar test -u gives error after clean install

I'm getting this error after initialising a new project with quasar and running a unit test

Oops, forgot about that. Just replace or delete the test case file in there.

Fixed. I think some more thinking needs to be put for testing with vue-router though. Need some help here.

I haven't written lots of tests yet but I'm determined to have my current project properly covered with tests
Will share once I've got something decent running

Great. Looking forward to it!


Sync Single Echo Client demo tasks (#2181)

Port fix of TCP echo client demo race condition made in #2181  to release-candidate
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

/cbmc run checks

/cbmc run checks


Value to set MJPG isn't magic and can be gotten programatically

Line 121 of recorders/video_capture.py has a comment saying:
Set a magic number, will enable MJPEG but is badly documentated
This number can be generated using cv2.VideoWriter_fourcc('M', 'J', 'P', 'G')

Using that would improve readability, thanks for the suggestion!


Release/2.1.3

Pull Request Test Coverage Report for Build 64

0 of 0 (NaN%) changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 100.0%

Totals

Change from base Build 63:
0.0%

Covered Lines:
110

Relevant Lines:
110

💛  - Coveralls


fix: 'requirements' in ansibleplaybookv2 should be optional

这个 PR 实现什么功能/修复什么问题:

是否需要 backport 到之前的 release 分支:

/cc @yousong
/area ansibleserver

/cc @yousong

Codecov Report

Merging #6503 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #6503   +/-   ##
=======================================
  Coverage    7.17%    7.17%
=======================================
  Files         646      646
  Lines      108945   108945
=======================================
  Hits         7819     7819
  Misses     100526   100526
  Partials      600      600

Flag
Coverage Δ

#aFlag
7.17% <ø> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 899c31d...77e5801. Read the comment docs.

/lgtm
/approve


mission-center: 0.3.1 -> 0.3.2

Description of changes
Changelog: https://gitlab.com/mission-center-devs/mission-center/-/releases/v0.3.2
cc @9p4
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.11 Release Notes (or backporting 23.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 254004 run on x86_64-linux 1

1 package built:

mission-center


Can't compose zero-copy choice(a, b).repeated() (bug?)

I recently started using Chumsky trunk so I could help tease out bugs and missing parts of the API, and I came upon something I did a lot in 0.9.2 that doesn't seem to work any more (8ad71861 specifically). The following code only prints  _:
#[test]
fn repro() {
	let parser = primitive::choice((
		primitive::just::<_, _, extra::Default>(" ").map_with_state(|s, _, _| {
			println!("{s}?");
		}),
		primitive::just::<_, _, extra::Default>("_").map_with_state(|s, _, _| {
			println!("{s}!");
		}),
	)).repeated().slice().map_with_state(|s, sp, builder| {
		println!("{s}!");
	});

	let _ = parser.parse(" _");
}

I wanted to pass a tree builder as state, but it seems those map calls never happen. I sense this isn't supposed to be the case given the recommendation to "map the output before it goes through repeated" given here.
If this is intentional, do I have an alternative? If so I'll submit a PR to write an example or some documentation illustrating how to do what I want without allocating, since I doubt I'm the only one. The closest I could find was in the map_with_state part of the doc comment for foldl_with_state but it doesn't compile since map_with_state doesn't take 2 arguments. Or am I missing something obvious?
Thanks.

Hey. It seems like you're hitting the new GAT-powered optimiser that chumsky uses internally. The behaviour you're observing is correct, even if not immediately intuitive. You can see more details here about what's going on.
TL;DR: The .slice() call tells chumsky that you don't care about the output value of the parser it's called on, and so chumsky automatically skips anything needed to produce that output value. Although this has surprising results for impure map functions like yours, it's also one of the big things that makes chumsky 1.0 so fast.

Oh, good, that's what I was speculating was happening. That means I can work it around immediately. I'll do some cooking; if I end up depending on any really ubiquitous patterns I'll suggest them to see if they belong upstream.
Cheers for the help, and for managing to wrangle the type system into the shape of this crate.


Snimple snupples

Will contribute if merged.
Is the project off Crowdin?

Will contribute if merged.
Is the project off Crowdin?

Thanks for your help. 😘
I switched from Crowdin to Hosted Weblate. After some time I will delete in on Crowdin.


Ansible role to provision microk8s (single node and multi-node)

Summary

Ability to form microk8s clusters using ansible officially supported by canonical would be great
Why is this important?

Microk8s is great at being a single-node offering and can also cater for multi-node offerings, however achieving a multi-node offering "out of the box" is not simply and repeatable without automation. Ansible is a great tool to connect to remote machines and run commands such as microk8s join to form a cohesive microk8s cluster.
It would be ideal to be able to cater for the following:

single node setup (basic microk8s install)
multi-node setup (microk8s install whereby all targets are considered worker and control-plane)
multi-node setup (control plane and worker split)

Are you interested in contributing to this feature?

Currently i don't have much time to contribute to this but i do feel there would be benefits to getting a design together which can then be tested (happy to assist in testing if needed).
For a great starting point (something i have been using previously): https://github.com/istvano/ansible_role_microk8s

Hey @jack1902 ,
Thank you for the feedback!
We've heard similar requests about Ansible roles and are discussing this topic as a team. I'll keep you updated on any related news. We are open to and appreciative of any contributions.

Any updates for this?

There are no plans to have an officially supported Ansible role. You will have to use one of the community driven projects.


Fix ECDSA test

Solved by https://github.com/ArkEcosystem/ark-js/commit/adecdc5044e10f06086b88d38ca44f34b422083c.


Package Scope
etc
Description
I see that suspensive recently added a new feature to seamlessly use jotai's async (suspend) behavior. In a similar vein, I'd like to suggest adding a wrapping component for relay that actively uses suspensive.
Possible Solution
AS-IS
https://relay.dev/docs/api-reference/use-fragment/
function UserComponent(props: Props) {
  const data = useFragment(
    graphql`
      fragment UserComponent_user on User {
        name
        profile_picture(scale: 2) {
          uri
        }
      }
    `,
    props.user,
  );

  return (
    <>

{data.name}

    </>
  );
}

TO-BE
function App() {
  return (
    <Suspense fallback={/* fallback component */}>
      <RelayFragment {...fragmentOptions}>
        {(data) => (
          <>

{data.name}

          </>
        )}
      </RelayFragment>
    </Suspense>
  );
}

etc.
No response

@2-NOW I assigned you to do this. Open draft pull request to make it

I close this as not planned yet. Reopen you can do it please


LogbackServiceProvider has been compiled by a more recent version of the Java Runtime

logback dependency should be 1.3.x because
<maven.compiler.target>1.8</maven.compiler.target>
is set in pom.xml
(class file version 55.0), this version of the Java Runtime only recognizes class file versions up to 52.0

At least logback should be scope=provided to be able to take other slf4j implementation.

https://github.com/obscure1910/flow-landscape/blob/bugfix/logback_version/pom.xml#L138

Fixed in version 1.1.8


update maintainers file for parsing

this updates the MAINTAINERS file to the new toml format,
so that it can be parsed and collected in the docker/opensource
repository.
see docker/opensource#35 and docker/docker#18321

ping @dmcgowan @jfrazelle :smile:

@thaJeztah curious, is all of this case insensitive? I notice this uses "people" while the opensource one uses "People".  I am not 100% sure how to judge whether this is correct but it can be updated either way, so LGTM

its good
On Fri, Dec 4, 2015 at 4:42 PM, Derek McGowan<EMAIL_ADDRESS>wrote:

@thaJeztah https://github.com/thaJeztah curious, is all of this case
insensitive? I notice this uses "people" while the opensource one uses
"People". I am not 100% sure how to judge whether this is correct but it
can be updated either way, so LGTM
—
Reply to this email directly or view it on GitHub
https://github.com/docker/spdystream/pull/62#issuecomment-162119055.

hm, good eyes! I used a template based on Jess's first PR, so, lol
thanks for the merge!


Use gsutil to upload test archives.

gsutil appears to be twice as fast as google-github-actions/upload-cloud-storage. We also gsutil to download artifacts and it never fails.
b/266771047

Updated description with a bug number.

I really want to get this in before nightlies start, going to merge...


Default TTL?

The docs don't explicity say what's the default TTL and looking at the code all I can find is None. Does None imply here that it's kept as long as it's not stale?

Hi!
ttl is simply a limitation for the response to live in our storages; if it is None, which is the normal behavior, it can remain in our storages until the controller decides to remove it.
You are right, we definitely need to mention that in the docs.

Understood, thank you!

Hey, I created a trivial PR for this in #250.

I think this can be closed. I forgot to use the "Closes X" in the PR.