Add AttnGAN submodule as a text-to-image implementation

Main change
The AttnGAN submodule at 'tasks/computer-vision/image-generation/text-to-image/AttnGAN' is set to track the mlflow branch of my AttnGAN repo. This is an image generation task (#12).
Other changes
Other proposed changes in this pull request are:

Add python and vscode extra files to .gitignore. The list of these files are from the GitHub gitignore repository.
Add conda.yaml file. One can prepare the ai-platform environment simply by conda env create -f conda.yaml if they are using conda. The list of requirements is almost identical to that in requirements.txt.
Add SETUP.md, a quickstart guide to use either conda or pip to prepare the ai-platform environment, and to export local environments.

Status of AttnGAN
AttnGAN is implemented for the birds dataset (but some testing is still required). It will download, train, and generate images using a single command through mlflow.
One main task remains to be done before this task is ready for general use: either a guide needs to be created for using AttnGAN with new datasets, or an automated configuration file generator can be implemented. Using AttnGAN on a new dataset involves creating configuration files for training and evaluation that includes model hyperparameters.
Submodules
Justification
I have opted to add a submodule reference instead of adding the actual files mainly because it is still a work-in-progress. Git submodules might be the preferred way to add tasks (other than examples), to keep tasks independent, reduce the repository size, and allow users of ai-platform to only download the tasks they need.
Notes
To download ai-platform with all the submodules, one can use:
git clone https://github.com/produvia/ai-platform.git --recurse-submodules

It is possible to have a separate submodule url for each task. It is also worth noting that a submodule is in a detached HEAD state, and is only a reference to specific commits in the submodule's repo. This means several things:

It is possible to have two independent submodules that come from the same repo but are of different versions.
Updates to a submodule repo will not be automatically pushed to the main (ai-platform) repo. These changes must be pushed manually by an ai-platform contributor. For example, one can use:

git submodule update --remote # Updates all submodules.

or (https://stackoverflow.com/a/39279179/9954163)
git submodule update --remote <path to the submodule> # Update a specific submodule.

to sync the submodule(s) with their repo(s) to the branch specified in <reporoot>/.gitmodules (default: master). To push the changes to ai-platform, one can use:
git add .
git commit -m "Change"
git push

A user can also pull only the submodules that they want (https://stackoverflow.com/a/31471460/9954163):
git clone https://github.com/mlaradji/ai-platform.git # This shouldn't download any submodules
cd ai-platform
git submodule update --init -- tasks/computer-vision/image-generation/text-to-image/AttnGAN/ # This pulls the ai-platform version of AttnGAN

Hi @mlaradji, We are so exiteted to have your contribution.

You have proposed some nice changes into repository and we should be open to consider them:

conda is being used by mlflow as default so I think we should have it as mlflow recommends. i.e. one conda.yaml for each MLflow project.
submodules seems a good alternative if size of repository goes beyond control.

I noticed mlflow branch in original repo but pull request was made from master branch so you might need to send another pull request with mlflow implementation. Or why not mlflow have in master? as long as ai-platform is concerned, we need mlflow as default.

I tried running repository but it seems you might need to update MLproject file. we should have only one command and all above steps should run in order. But if seems it's not working as expected resulting it into error.

2019/07/02 15:51:33 INFO mlflow.projects: === Running command 'source activate mlflow-697830836e46982e343d4e3caec0e2856607f468 && python src/download.py birds --raw --metadata --overwrite python src/pretrain_DAMSM.py --cfg cfg/birds/DAMSM/bird.yml --gpu 0 python src/main.py --cfg cfg/birds/bird_attn2.yml --gpu 0 python src/main.py --cfg cfg/birds/eval_bird.yml --gpu 0' in run with ID 'cf5748ed64c5410c8bca1d08134e0394' ===
usage: download.py [-h] [--all] [--raw] [--metadata] [--damsme] [--models]
                   [--overwrite] [--base-dir BASE_DIR]
                   dataset
download.py: error: unrecognized arguments: python src/pretrain_DAMSM.py --cfg cfg/birds/DAMSM/bird.yml --gpu 0 python src/main.py --cfg cfg/birds/bird_attn2.yml --gpu 0 python src/main.py --cfg cfg/birds/eval_bird.yml --gpu 0

Most severe: While running commands individually, I got issue on 2nd python script and it failed with follwoing error.

src/pretrain_DAMSM.py:97: UserWarning: torch.nn.utils.clip_grad_norm is now deprecated in favor of torch.nn.utils.clip_grad_norm_.
  cfg.TRAIN.RNN_GRAD_CLIP)
Traceback (most recent call last):
  File "src/pretrain_DAMSM.py", line 274, in <module>
    dataset.ixtoword, image_dir)
  File "src/pretrain_DAMSM.py", line 103, in train
    s_cur_loss0 = s_total_loss0[0] / UPDATE_INTERVAL
IndexError: invalid index of a 0-dim tensor. Use tensor.item() to convert a 0-dim tensor to a Python number
(mlflow-697830836e46982e343d4e3caec0e2856607f468) user@dell:~/Documents/AttnGAN$

Thank you for your detailed response.

I do think that each task should have its own conda.yaml, as you mentioned. The conda.yaml in the root directory that I want to add is not meant to be used for tasks, but it's rather an alternative to the requirements.txt file for users who would rather use conda than pip to manage their dependencies.
I have indeed forgotten to update the AttnGAN submodule to the mlflow branch, and it should now be fixed. If it gives too much trouble later on then I can certainly just merge the mlflow branch into master in https://github.com/mlaradji/AttnGAN as you have suggested.
I am currently working on getting AttnGAN useable, and I will repost in this pull request once I believe I have a working version.

I am slightly unclear on one thing. You have mentioned that a task should have one endpoint; that seems to me to be a good idea as it would simplify higher-level usage of tasks. Does that mean that each task should only work for a specific dataset? Or would the task look for data in a default directory?

Hi @mlaradji, thank you very much for your quick action and clear explanation.
Regarding last question, I wanted say that if we see MLflow multi step workflows (documentation and code examples) for how multiple steps can communicate with each other, we would find that previous entry points are defined as dependencies, where output from the previous step is used as input in next step and at last main entry point is called, which has only one command to execute. But when I am trying to execute AttnGAN implementation, it fails with error pasted on above comment as it's trying to run all 4 python scripts as single command where first script is treating rest of the scripts as arguments.
Even if we try to execute all 4 script using unix && operator, it might not the way how MLflow expects multi step should function.
Please try to explore more on MLflow muti step example and implement it in expected way. Otherwise try try to run with mlflow run ... command and if it's working, we can close this pull request and come back later when we have more experience with MLflow api's.
Thanks once again for your contribution.

Main change
The AttnGAN submodule at 'tasks/computer-vision/image-generation/text-to-image/AttnGAN' is set to track the mlflow branch of my AttnGAN repo. This is an image generation task (#12).
Other changes
Other proposed changes in this pull request are:

Add python and vscode extra files to .gitignore. The list of these files are from the GitHub gitignore repository.
Add conda.yaml file. One can prepare the ai-platform environment simply by conda env create -f conda.yaml if they are using conda. The list of requirements is almost identical to that in requirements.txt.
Add SETUP.md, a quickstart guide to use either conda or pip to prepare the ai-platform environment, and to export local environments.

Status of AttnGAN
AttnGAN is implemented for the birds dataset (but some testing is still required). It will download, train, and generate images using a single command through mlflow.
One main task remains to be done before this task is ready for general use: either a guide needs to be created for using AttnGAN with new datasets, or an automated configuration file generator can be implemented. Using AttnGAN on a new dataset involves creating configuration files for training and evaluation that includes model hyperparameters.
Submodules
Justification
I have opted to add a submodule reference instead of adding the actual files mainly because it is still a work-in-progress. Git submodules might be the preferred way to add tasks (other than examples), to keep tasks independent, reduce the repository size, and allow users of ai-platform to only download the tasks they need.
Notes
To download ai-platform with all the submodules, one can use:
git clone https://github.com/produvia/ai-platform.git --recurse-submodules

It is possible to have a separate submodule url for each task. It is also worth noting that a submodule is in a detached HEAD state, and is only a reference to specific commits in the submodule's repo. This means several things:

It is possible to have two independent submodules that come from the same repo but are of different versions.
Updates to a submodule repo will not be automatically pushed to the main (ai-platform) repo. These changes must be pushed manually by an ai-platform contributor. For example, one can use:

git submodule update --remote # Updates all submodules.

or (https://stackoverflow.com/a/39279179/9954163)
git submodule update --remote <path to the submodule> # Update a specific submodule.

to sync the submodule(s) with their repo(s) to the branch specified in <reporoot>/.gitmodules (default: master). To push the changes to ai-platform, one can use:
git add .
git commit -m "Change"
git push

A user can also pull only the submodules that they want (https://stackoverflow.com/a/31471460/9954163):
git clone https://github.com/mlaradji/ai-platform.git # This shouldn't download any submodules
cd ai-platform
git submodule update --init -- tasks/computer-vision/image-generation/text-to-image/AttnGAN/ # This pulls the ai-platform version of AttnGAN

Hi @mlaradji, would it be possible to add submodule usage in README page. Describing how to access AttnGAN repository and also how to refresh in future.
Thanks!

Our vision for ai-platform is to make automate AI tasks. This requires us to focus on granularity of software components.
We use MLflow to manage machine learning models. With MLflow multistep workflows, each software component of the system is treated as an individual step.
On our ai-platform, each task should be able to run by itself. We should also be able to combine tasks together to create systems (or "chains of tasks").
For example, consider the following two AI tasks: object detection (Issue #18) and anomaly detection (Issue #8). We should be able to run object detection model by itself. We should also be able to chain these tasks together, such that anomaly detection is performed first, then object detection is performed second, where the output of the former model becomes the input of the latter.

Hi @mlaradji, thank you for making changes.
Once you fix this issue, we can pull the task. Also please check AttnGAN readme.
I belivieve, there is a typographical error mlflow run . -e birds is mentioned as mlflow . -e birds
mlflow run . -e birds
2019/07/07 13:38:58 INFO mlflow.projects: === Created directory /tmp/tmp0g3j2_0l for downloading remote URIs passed to arguments of type 'path' ===
2019/07/07 13:38:58 INFO mlflow.projects: === Running command 'source activate mlflow-697830836e46982e343d4e3caec0e2856607f468 && python src/download.py birds --raw --metadata --overwrite python src/pretrain_DAMSM.py --cfg cfg/birds/DAMSM/bird.yml --gpu 0 python src/main.py --cfg cfg/birds/bird_attn2.yml --gpu 0 python src/main.py --cfg cfg/birds/eval_bird.yml --gpu 0' in run with ID '6704d08ed3514f0eaf08059401ceb674' ===
usage: download.py [-h] [--all] [--raw] [--metadata] [--damsme] [--models]
                   [--overwrite] [--base-dir BASE_DIR]
                   dataset
download.py: error: unrecognized arguments: python src/pretrain_DAMSM.py --cfg cfg/birds/DAMSM/bird.yml --gpu 0 python src/main.py --cfg cfg/birds/bird_attn2.yml --gpu 0 python src/main.py --cfg cfg/birds/eval_bird.yml --gpu 0
2019/07/07 13:39:01 ERROR mlflow.cli: === Run (ID '6704d08ed3514f0eaf08059401ceb674') failed ===

Our vision for ai-platform is to make automate AI tasks. This requires us to focus on granularity of software components.
We use MLflow to manage machine learning models. With MLflow multistep workflows, each software component of the system is treated as an individual step.
On our ai-platform, each task should be able to run by itself. We should also be able to combine tasks together to create systems (or "chains of tasks").
For example, consider the following two AI tasks: object detection (Issue #18) and anomaly detection (Issue #8). We should be able to run object detection model by itself. We should also be able to chain these tasks together, such that anomaly detection is performed first, then object detection is performed second, where the output of the former model becomes the input of the latter.

thank you @slavakurilyak , your comment has improved our understanding of mflow multistep workflow.


DND: implement the first version

Checklist

[x] npm test passes
[x] tests and/or benchmarks are included
[ ] documentation is changed or added

There are some conflicts exists, @FeelyChau May I ask you to rebase again?

There are some conflicts exists, @FeelyChau May I ask you to rebase again?

merged

Congrats to @FeelyChau with the first contribution 🥇


chore(master): release 1.3.1

:robot: I have created a release beep boop
1.3.1 (2024-12-31)
Miscellaneous Chores

master: release 1.3.1 (eb9c43f)

This PR was generated with Release Please. See documentation.

:robot: Release is at https://github.com/unione-repo/unione-php/releases/tag/v1.3.1 :sunflower:


Connector password encrypted twice on Update request

What happened?
I created a Connector with a password field.
I then created an update request and the password field took the encrypted password and encrypted it again now showing a different password, meaning the saved password would be overwritten in this update.
What did you expect to happen?
The password should only be encrypted if it is a new entry or has been altered from its previous state.
What else do we need to know?
On version 2.4.0 (bleeding edge)

Also just to mention the encrypted password also changes on a promotion

Hey @aindriu-aiven,
I would like to pick this up!

Hey @Sadaf-A thanks for offering to pick this up, actually @muralibasani is already looking at it, I should have assigned it apologies. We are currently testing before a planned release though so we should have a few more issues coming up shortly!


Jumping on right click (context menu)

Steps to reproduce
Open terminal and click right mouse button, it jumps for split of second (there is 100ms delay in code)

The issue is still present when adding terminal to body.

Final fix is in version 1.16.1


Add support for EXCLUDING attachment extensions

Great plugin.
Being able to configure attachment extensions that need to be cleaned is a nice feature. From the perspective of my usage, I would like to clean up all useless extensions (configure ".*" directly), but exclude specified suffix names, such as .mdenc (which is used by another encryption plug-in and should not be cleaned up).
Thank you!

Hm, so if I understand you correctly, you'd like inverse matching.
F.ex if you have the following files:
file.md
file.jpg
file.png
file.pdf
file.mdenc

And you set your filter to be:
.md
.mdenc

You want it to all files except .mdand .mdenc to removed?
So, the remaining files would be:
file.md
file.mdenc

Yes, that's basically what I expected. But it would be more reasonable not to clean up the .md file for the plugin.
I think .md does not need to be configured here, otherwise the user may delete them by mistake.

Thanks for the feedback.
The only reason why the markdown files would be cleaned up is if they are empty so they wouldn't be removed by mistake :)
I'll have a look at how to implement it and get back to you when it's ready.

Hey @situfang, just a heads up that I've implemented this and it will be part of v0.9.0 releasing soon.
This issue will be closed automatically when the PR has been merged.


Running Prettier wipes out all existing marks

Do you want to request a feature or report a bug?
Report a bug
What is the current/expected behavior?
After running Prettier all user created marks in the current buffer are deleted
What version of vim-prettier are you using - (output of :PrettierVersion) ?
1.0.0-alpha
What version of prettier are you using - (output of :PrettierCliVersion) ?
2.0.5
What is your prettier executable path - (output of :PrettierCliPath) ?
Absolute path to the project's node_modules/.bin/prettier
Did this work in previous versions of vim-prettier and/or prettier ?
Haven't tested
So to reproduce:

add a mark via ma
run Prettier (ensure this resulted in some changes)
trying to go to the mark via 'a does not work

I experience the same but only in lines that are prettified. Unmodified lines will remain the marks.

I'm also experiencing this. If Prettier encounters an error, it will:
If using :Prettier, replace the buffer with the error. Discarding the contents of the file you were working on.
If using :PrettierAsync, empty the buffer. Discarding the contents of the file you were working on.
This makes the extension completely unusable in its current state.

Same problem

Cab you folks do a plug update and check if this PR https://github.com/prettier/vim-prettier/pull/300 fixed this issue for you ?

It seems to work fine now!
Very nice! Thank you so much 🙏  🎉


ignore this pr

Codecov Report

Merging #2176 into develop will increase coverage by <.01%.
The diff coverage is n/a.

@@             Coverage Diff             @@
##           develop    #2176      +/-   ##
===========================================
+ Coverage    69.48%   69.49%   +<.01%
===========================================
  Files          685      685
  Lines        50520    50520
===========================================
+ Hits         35105    35109       +4
+ Misses       15415    15411       -4

Impacted Files
Coverage Δ

src/ripple/app/consensus/RCLConsensus.cpp
63.4% <ø> (ø)
:arrow_up:

src/ripple/app/ledger/PendingSaves.h
93.54% <0%> (-6.46%)
:arrow_down:

src/ripple/protocol/impl/Serializer.cpp
70.3% <0%> (+0.34%)
:arrow_up:

src/ripple/core/impl/JobQueue.cpp
86.79% <0%> (+0.47%)
:arrow_up:

src/ripple/server/impl/BaseWSPeer.h
71.15% <0%> (+0.64%)
:arrow_up:

src/ripple/basics/DecayingSample.h
86.11% <0%> (+8.33%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 87742a5...f898f84. Read the comment docs.


TelemetryConfiguration.Active is marked Obsolete in Assembly Microsoft.ApplicationInsights, Version=<IP_ADDRESS>5

TelemetryConfiguration.Active is marked Obsolete in Assembly Microsoft.ApplicationInsights, Version=<IP_ADDRESS>5
You may wish to update the example that uses it.
See https://github.com/microsoft/ApplicationInsights-dotnet/issues/1152

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 2f4c906d-7abe-eab7-331a-120230344583
Version Independent ID: ab24ade5-5e4c-b430-c3f8-2a00073e9e81
Content: Telemetry sampling in Azure Application Insights - Azure Monitor
Content Source: articles/azure-monitor/app/sampling.md
Service: azure-monitor
Sub-service: application-insights
GitHub Login: @AaronMaxwell
Microsoft Alias: aaronmax

.Active is referenced in the "ASP.NET applications" section. This is still applicable here.
The .Active singleton is only obsolete for "ASP.NET Core applications".

@TimothyMothra  - Thank you for providing your insights.
@imercerwillow - I hope @TimothyMothra has answered your question.
At this moment, Since there is no doc update required, We will now proceed to close this thread. If there are further questions regarding this matter, please tag me in your reply. We will gladly continue the discussion and we will reopen the issue.


Give "hanging" comments a details element to reside in

Comments at the end of a security.txt file cannot be associated with a directive. Currently, they are just dumped onto the end. This could have minor security implications if the comment was Submit vulnerability information from the last few days in one place: https://evil.example.com.

https://github.com/joker314/securitytxt-extension/commit/01cbc29bad294edd496d5062c160d52b16ef2e7b


Regtest AdvancedReOrgTests dataset for Orchard and Sapling

Recreate a freshed version of the dataset

https://github.com/zingolabs/darksidewalletd-datasets/pull/12


auto-upload while offline constantly shows error toast

In airplane mode with Auto Upload turned on I get the "Not all photos were synced" error again and again after saving a new observation with a photo. It also seems to slow down the UI, almost to the point of locking it up.

Wrong repo again.


Add variant filtering options to the script

Add the ability to filter by a table of variants to the subsetting script.
Also fix the error strings for when a file exists.

potentially a change to be made confirming that your requested sample subset it present in the full sample set

The way I subset samples is that rather than specifying a list of samples, you instead just give a number and you get that many people from the original VDS


building:min_level and building:max_level

min_height works fine (for ex Tower Bridge: https://www.openstreetmap.org/way/367652753) for building suspended or cantilevered
Are the keys building:min_level and building:max_level also used?
It doesn't looks ok here for example:

Are the keys building:min_level and building:max_level also used?

Please notice that the correct tags are building:min_level=* and building:levels=*, and that the tags used on that building are max_level and min_level
Judging on the wiki max_level is used for indoor tagging, not 3d tagging.

oh, yes, sorry. with the correct key+values, it works fine.


added a new (unlockable) beret in the botanist's loadout.

About the PR
I have added a "Botanical Beret" available in botanists' loadout for players with 15 hours of botany playtime.
Why / Balance

I felt that it was strange that there was nothing to unlock with botany playtime despite how much depth there is to the job - and i wanted to change that.
Media

Requirements

[X] I have read and I am following the Pull Request Guidelines. I understand that not doing so may get my pr closed at maintainer’s discretion
[X] I have added screenshots/videos to this PR showcasing its changes ingame, or this PR does not require an ingame showcase

Changelog
:cl:

add: A new beret is now available in the botanist loadout, requiring 15 hours of botany time.

the flower should be more visible and prominent imo, especially on the icon sprite

Why would a cosmetic beret require playtime unlocking? Treadmills are for games where you need to maximize CCUs to appease a board of investors, SS14 decidedly isn't that.

a sense of pride and accomplishment

Hey, I brought this up for discussion with the maintainers and it was agreed not to include this item. The main reasons for this were that we do not need a beret for every single job and that a beret is a little random as an unlockable loadout item, since it does not have much to do with botany. A botany-related item, for example a golden scythe, was suggested instead. Thank you for your contribution.


Error about Java

Error: Could not find or load main class LaunchSupport
Caused by: java.lang.ClassNotFoundException: LaunchSupport
Error: Could not find or load main class LaunchSupport
Caused by: java.lang.ClassNotFoundException: LaunchSupport
Error: Could not find or load main class LaunchSupport
Caused by: java.lang.ClassNotFoundException: LaunchSupport

Failed to find a supported JDK.  Please refer to the Ghidra Installation Guide's Troubleshooting section.
Press any key to continue . . .

I tried almost everything to make it work, I included in path and tried so much stuff but still the same error.

It worked


Yolo on rv1106

I try several Yolo on rv1106 board:
Yolov5 - work fine
Yolov8 and Yolov10 - error:
load lable ./model/coco_80_labels_list.txt
E RKNN: failed to decode config data!
Segmentation fault (core dumped)

Please confirm that Yolov8/10 dont work on rv1106/03 board.

我也遇到这个问题了，请问是库的版本不对么？

rv1106/03 can support yolov8, but not yolov10


Assignment 3 - Mahmoud

This is the updated cleaner version of the assignment.

Great work! The code works very well!


[gvar] Move low-level routines to a common place

Some of the fonttools code for compiling and decompiling variation data structures (tuples, packed deltas, etc.) could probably be moved from the gvar table into otTables or another base library. After that, implementing cvar would be easier.
Generally, looking at my old gvar code, I’m not terribly happy with it. While the TTX output is easy to read for humans, it also hides a fair bunch of internals. For example, one does not see which tuples are shared versus embedded. I think the code in ttLib should become more “assembler” than “compiler”, and the higher-level stuff should move to varLib.

Ideally, the ttx format (not just for gvar but overall) might be a bit simplified and perhaps hide some of the gory internals while ttLib could expose more lower-level stuff.
When researching this curious Apple TT behavior I tried to set the bit 6 of the flags field of a glyf entry using ttLib, one that is officially "reserved" but is mentioned in an old AAT-related document, and failed, because ttLib had no mechanism to access the flags bitmask. Ultimately, I had to revert to a hex editor and calculate the offsets manually. :)

I would be sad to see the highlevel API to gvar go.  That said, the code can move to varLib.structs or something, and called from gvar / cvar.

There’s now a common place in TupleVariation.py, which seems good enough for now.


✨ Adding line numbers for rest of Token-Permissions (and by extension, Packaging)

Packaging)

Please check if the PR fulfills these requirements

[x] Tests for the changes have been added (for bug fixes / features)
[x] PR title follows the guidelines defined in  https://github.com/ossf/scorecard/blob/main/CONTRIBUTING.md#pr-process

What kind of change does this PR introduce? (Bug fix, feature, docs update, ...)
Feature

What is the current behavior? (You can also link to an open issue here)
When the Token-Permission check makes logs for packaging workflow, it does not set the line number. Same thing for the Packaging check.
Closes https://github.com/ossf/scorecard/issues/886

What is the new behavior (if this is a feature change)?
All logs from the Token-Permission and Packaging checks will include line numbers. This PR also makes the check for packaging workflow more precise. Instead of doing a string search for an action name, it will parse the workflow file and check that a uses field on a step matches the action name.

Does this PR introduce a breaking change? (What changes might users need to make in their application due to this PR?)
No

Other information:

Thanks @chrismcgehee , this is really cool!
I think this PR closes https://github.com/ossf/scorecard/issues/1100, is that right?
closes https://github.com/ossf/scorecard/issues/1100

I think this PR closes #1100, is that right?

Maybe it does; I'll comment on that issue.

Nit suggestion: this is a good corner case to test for. We should probably have a test which tests that we handle these parsing issues gracefully.

I'm all for testing corner cases, but I don't think this case is a good fit for our existing unit tests. If a workflow file does not conform to Github's specifications for a workflow file, I think it should be indeterminate what our code does; in this case it will still run the token-permissions check on this file, but it would treat is as a run step and ignore the uses. I think that behavior is fine, and I also think it would be fine if our code reported an error on this file. What we need to do is create a larger test that checks multiple files and continues checking if 1 of the files has some sort of error.


Fix logits processor when prompt_logprobs is not None

Hello,
I am currently developing an LLM toolkit based on vLLM. However, I encountered a bug related to logits processor #2141 developed by @noamgat within vllm detailed at #2800 . The primary issue with this bug is that when prompt_logprobs in SamplingParams is set to a value other than None, the logits for the first iteration include the prompt. This evidently occurs while calculating the logprobs for the tokens corresponding to the prompt. However, this leads to an AssertionError at this assert statement.
I have fixed this bug when prompt_logprobs is not None and conducted tests with different values for prompt_logprobs. With these changes, the error does not occur in either scenario.
In detail, when len(sampling_metadata.prompt_lens) > 0, it means this is the first iteration; whereas sum(sampling_metadata.prompt_lens) == logits.shape[0] implies that the logits include the prompt.
Thank you for your review. I look forward to your fix for this issue, which will assist us in completing our project development.
Wishing you a happy day.

I found that I cited an incorrect pull request. Logits processor is introduced here: #1469
@simon-mo may be able to review this pr.

Hello @huyiwen. I found your PR after also encountering the AssertionError and working on an independent fix in https://github.com/vllm-project/vllm/pull/3867. I was trying to actually process the prompt logits with logits_processors, but decided that it would be easier to just skip them like in your PR 😅 So I'd like to help get this merged in if I can.
I think there is another case to handle in your PR:
if there are multiple requests batched together where only a subset have prompt_logprobs, then we need to check each request independently to decide to skip instead of skipping for all prompts in the batch (only the request with prompt_logprobs will have rows in logits for each prompt token).

Script to reproduce
import torch
from vllm import LLM, SamplingParams

model_path = 'facebook/opt-125m'

llm = LLM(model=model_path)

prompts = [
    "Hello, My name is",
]

def pick_ith(token_ids, logits):
    # skip lower ids that have special tokens
    logits[len(token_ids)+10] = torch.finfo(torch.float16).max
    return logits

sampling_params = SamplingParams(
    temperature=0.5,
    logits_processors=[pick_ith],
    prompt_logprobs=3,
)

prompt = "Hello, My name is"

llm._add_request(
    prompt = prompt,
    sampling_params = sampling_params,
    prompt_token_ids = None
)
llm._add_request(
    prompt = prompt,
    sampling_params = SamplingParams(), # default sampling_params without prompt_logprobs
    prompt_token_ids = None
)

outputs = llm._run_engine(False)

# Print the outputs.
for output in outputs:
    prompt = output.prompt
    generated_text = output.outputs[0].text
    print(f"Prompt: {prompt!r}, Generated text: {generated_text!r}")

It seems _apply_logits_processors has been moved to a different file. I've opened a new PR to accommodate it: #3899.


CAMEL-13395 Threadsafe camel-api-component-maven-plugin

See https://issues.apache.org/jira/browse/CAMEL-13395

LGTM


Question about best practices for sane-stack repo structure

Is including both ember and sails apps And the residual sane files in the same repo the current best practice, just for the example, both, or negligible?
I ask because I would suspect any project of significant size or contributors could warrant separate repos, but it's unclear to me where fig/docker-compose/package.json/.sane-cli/.nvmrc/etc would go. Git submodules for the client and server?? :fearful:
Thanks!
Max

I like each application to be in separate repos, and I avoid submodules like the plague.  What I do generally is:
> sane new blah
> cd blah
> mv client ../blah-client
> mv server ../blah-server
> ln -s ../blah-client client
> ln -s ../blah-server server
> git init
> cd client && git init
> cd ../server && git init
# set up remotes and push, etc

You will end up with 3 repos: "blah", "blah-client", "blah-server".  To onboard other developers, simply pull all 3 repos into the same dir, like:
> cd ~/projects/
> git clone https://github.com/me/blah
> git clone https://github.com/me/blah-client
> git clone https://github.com/me/blah-server

add more client and server applications to taste.

I agree with @kriswill in a real world situation, I would split them. For the sake of keeping this example in one place, they're together here

Cool. I like the symlink solution--thanks!
On Mon, Oct 5, 2015 at 2:53 PM Martin Genev<EMAIL_ADDRESS>wrote:

I agree with @kriswill https://github.com/kriswill in a real world
situation, I would split them. For the sake of keeping this example in one
place, they're together here
—
Reply to this email directly or view it on GitHub
https://github.com/mgenev/how-to-sane/issues/93#issuecomment-145630758.


which should I select? mash_serovar or serovar?

Hi,
first of all, your bioinformatic tool is working very well! However, I am wondering about which column I should use as final serovar result? In rare cases, the serovar in "mash_serovar" column didn't correspond with the serovar in "serovar" column. Can I add a filter, maybe drafted from the cgmlst_distance number, for isolates with a low level of confidence.
Thank you very much,
A+

Thank you very much,
A+

Hi Jean,
That's great to hear that the tool is working well for you!
I would recommend using the serovar column value as it is an aggregate of the O/H-antigen prediction and the cgMLST (or Mash) prediction. If you are determining the cgMLST and Mash predictions then only the cgMLST prediction is used for refining the antigen-based prediction.
The Mash prediction should be considered more experimental and suitable for a very quick analysis (BLAST searching all cgMLST alleles against a Salmonella genome can take a while). I've also noticed that Mash is more sensitive to contaminant contigs in assemblies, which can result in fewer matching kmer "sketches". But if your Mash distance is very close to 0 (>900 matching sketches), then you should get similar results as with cgMLST.
Yes, cgmlst_distance can definitely be used to determine the level of confidence in a prediction. Internally, sistr_cmd uses a cgMLST distance threshold of 0.1 (90% similarity) to a set of reference genomes (currently 7511 curated public genomes) for confidentally calling serovar from cgMLST data. Although you could get an accurate serovar prediction with a higher distance threshold (e.g. 0.3 or 70% similarity), that threshold may not hold for all serovars, especially closely related serovars (e.g. Dublin vs Enteritidis).
Also, it would be okay if the cgMLST distance to the closest reference genome were high if the antigen data is complete and the antigen-based serovar call is unambiguous.

Hi Peter,
I'm glad you took time to respond as quickly.
A+
2016-09-07 19:15 GMT-04:00 peterk87<EMAIL_ADDRESS>
Hi Jean,
That's great to hear that the tool is working well for you!
I would recommend using the serovar column value as it is an aggregate of
the O/H-antigen prediction and the cgMLST (or Mash) prediction. If you are
determining the cgMLST and Mash predictions then only the cgMLST prediction
is used for refining the antigen-based prediction.
The Mash prediction should be considered more experimental and suitable
for a very quick analysis (BLAST searching all cgMLST alleles against a
Salmonella genome can take a while). I've also noticed that Mash is
more sensitive to contaminant contigs in assemblies, which can result in
fewer matching kmer "sketches". But if your Mash distance is very close to
0 (>900 matching sketches), then you should get similar results as with
cgMLST.
Yes, cgmlst_distance can definitely be used to determine the level of
confidence in a prediction. Internally, sistr_cmd uses a cgMLST distance
threshold of 0.1 (90% similarity) to a set of reference genomes (currently
7511 curated public genomes) for confidentally calling serovar from cgMLST
data. Although you could get an accurate serovar prediction with a higher
distance threshold (e.g. 0.3 or 70% similarity), that threshold may not
hold for all serovars, especially closely related serovars (e.g. Dublin vs
Enteritidis).
Also, it would be okay if the cgMLST distance to the closest reference
genome were high if the antigen data is complete and the antigen-based
serovar call is unambiguous.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/peterk87/sistr_cmd/issues/10#issuecomment-245448094,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AJ9PjJyZTzgmi78KGkQP_aN666H9fhsfks5qn0V4gaJpZM4J3P62
.

--
Jean-Guillaume Emond-R, M.Sc.
Professionnel de recherche,
Institut de Biologie Intégrative et des Systèmes,
Université Laval


feat(v2): complete Chinese code translations

Motivation
Complete code translations in zh-Hans and zh-Hant that are not yet translated.
I'm not confident enough about zh-Hant translations since Taiwan/Hong Kong people have slightly different way of speaking 🤷♂
Have you read the Contributing Guidelines on pull requests?
Yes

The translation for zh-Hans looks good to me! For zh-Hant, I checked with my dad and he says it is correct :)

The translation for zh-Hans looks good to me! For zh-Hant, I checked with my dad and he says it is correct :)

Thanks! That was very helpful :P

thanks :)


Brand Coloring

Change header and footer color to match brand colors based on node selection.
Intended to minimize confusion on which node you are using.
bump version to <IP_ADDRESS>

closed because git hub files from other branches were pulled in when not needed resubmitted as #720


Added $type parameter to fontFaceVariableCustom mixin

Added $type parameter to the fontFaceVariableCustom mixin, so you are able to use it for "full" and not only "wghtOnly".
This issue is also reported here: #419
SassError: No argument named $type.

Codecov Report

Merging #617 (66a208a) into main (230439b) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main     #617   +/-   ##
=======================================
  Coverage   93.23%   93.23%
=======================================
  Files          16       16
  Lines         473      473
  Branches       63       63
=======================================
  Hits          441      441
  Misses         32       32

Impacted Files
Coverage Δ

scripts/templates/scss.ts
100.00% <0.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

Sorry for the delay in reviewing this, I'm currently travelling. I think it looks good and closes #419.
I'll merge this and rebuild the repo when I find the chance!

Thanks for the PR. Sorry for taking so long! I'll publish new packages once #623 is merged!


Bad error when downloading bogus versions

1 brian@ip-10-235-20-183:/mnt/dev/hello⟫ rustup update 1.0.0-alpha
info: syncing channel updates for '1.0.0-alpha'
info: downloading component 'rust'
error: could not download file from 'https://static.rust-lang.org/dist/rust-1.0.0-alpha.tar.gz.sha256' to '/home/brian/dev/.multirust/tmp/3ujwchag_1tpheuu_file
info: caused by: Status: 404 Not Found

This should say something more like "no release found for X".

This should be easy. Just detect the 404 error and chain a custom error onto it.

I missed this at the time but saw it pop up in "This Week in Rust." I'd like to give it a shot, if possible.


version 2.1.3 raise rediscluster.exceptions.MovedError when connect redis cluster

I use newest version redis-py-cluster==2.1.3 connect redis cluster with code
redis_store = RedisCluster(
    startup_nodes=startup_nodes,
    decode_responses=True,
    skip_full_coverage_check=True,
    readonly_mode=True,
    password=REDIS_PASSWORD
)
redis_store.get("foo")

raise rediscluster.exceptions.MovedError.
And I change version to redis-py-cluster==1.3.6 by using 'StrictRedisCluster' without this problem.
Is there a bug for new version?
How to fix it.

@Vector-Cross Is this exception raised back to you? Or are you just getting the log message written back to you?
Most likley is that your redis-cluster is broken or not in a valid cluster state somehow. This usually happens when you use skip_full_coverage_check=True as that would validate if your cluster has everything covered properly. Check your cluster first.

@Vector-Cross Is this exception raised back to you? Or are you just getting the log message written back to you? Most likley is that your redis-cluster is broken or not in a valid cluster state somehow. This usually happens when you use skip_full_coverage_check=True as that would validate if your cluster has everything covered properly. Check your cluster first.

Is the exception raised back. I'm not sure my redis cluster is everthing ok. But I do use skip_full_coverage_check=False once and  the exception raised in the first minute. Another thing is the exception not raised when I use the old version.

@Vector-Cross Can you paste the entire stack trace maybe? It is impossible to track down any error when i do not know where it comes from. If i look in the _execute_command method that you are using, there is no possible way that a MovedError is raised back to you as a user. It has to either be in the cluster discovery function or just before the try/except part.
Older versions have nothing todo with anything.
Also try to remove the readonly_mode=True, maybe your client is hitting a replica node that is not working as expected.

@Vector-Cross Can you paste the entire stack trace maybe? It is impossible to track down any error when i do not know where it comes from. If i look in the _execute_command method that you are using, there is no possible way that a MovedError is raised back to you as a user. It has to either be in the cluster discovery function or just before the try/except part. Older versions have nothing todo with anything. Also try to remove the readonly_mode=True, maybe your client is hitting a replica node that is not working as expected.

thanks for your reply. I'll see, maybe it is my enviroment problem or others. It's complicated due to using python-alpine as docker container. let me double check my code.


variable-argument functions

This is just an initial implementation of variable-argument functions. I'm expecting to need to make changes, but I wanted to demonstrate the initial approach and get feedback before going further.
With this implementation, there are two types of functions supported: regular (func(a,b,c) {}) and variadic (func(...args) {}). Currently, if a function is variadic, it must have exactly one argument. The VM is responsible for consolidating passed parameters when it processes an OpCall instruction on a CompiledFunction or Closure that is variadic. It does this by consolidating the arguments on the stack into an array, which is then stored on the stack in place of the original first argument. If no arguments are passed to a variadic function call, an empty array is added to the stack as the first argument.

Thanks for the PR. I will take a look!

@mccolljr I will invite you to a Slack channel if you send me an email<EMAIL_ADDRESS>

The other concern I have is of performance. I modified the two fib tail call functions in cmd/bench. The variadic implementation is quite slow in both cases. I'm assuming it adds a significant overhead when the arguments are consolidated into an array and pushed to the stack.
The modifications and results:
fib := func(...x) {
	if x[0] == 0 {
		return 0 + x[1]
	} else if x[0] == 1 {
		return 1 + x[1]
	}

	return fib(x[0]-1, fib(x[0]-2, x[1]))
}

fib := func(...x) {
	if x[0] == 0 {
		return x[1]
	} else if x[0] == 1 {
		return x[2]
	}

	return fib(x[0]-1, x[2], x[1]+x[2])
}

Results:
BEFORE:
-------------------------------------
fibonacci(35) (tail-call #1)
-------------------------------------
Result:  9227465
Go:      79.25467ms
Parser:  23.245µs
Compile: 99.047µs
VM:      3.026240441s
-------------------------------------
fibonacci(35) (tail-call #2)
-------------------------------------
Result:  9227465
Go:      442ns
Parser:  28.094µs
Compile: 85.925µs
VM:      16.416µs

AFTER:
-------------------------------------
fibonacci(35) (tail-call #1)
-------------------------------------
Result:  9227465
Go:      74.706679ms
Parser:  59.182µs
Compile: 106.193µs
VM:      8.298983466s
-------------------------------------
fibonacci(35) (tail-call #2)
-------------------------------------
Result:  9227465
Go:      282ns
Parser:  36.099µs
Compile: 107.335µs
VM:      42.433µs

@earncef that performance hit is pretty big... but I have to wonder how likely it is that tail-call recursion is going to be used in conjunction with variadic functions. I tried to keep changes to the core structure of the language to a minimum, which meant that the only place I could get information about the callee was in at the vm level, when an OpCall instruction is processed. That's why the process of consolidating arguments into an array for variadic functions happens at the vm level. I'm open to suggestions or a nudge in the right direction but I'm not sure what the best way to improve the situation is. Maybe there's some special handling that can be done in the tail call check to update indexes in the argument array rather than pushing the arguments back on the stack? I'l look into it when I have some time later today, but advice would be appreciated.


Auto-update go-utils to latest version

:robot: Beep boop I am a bot
This is an automatically created PR to change keptn/go-utils to version 086f16f7523467d8e4a38906631bd3058b4050f8.
Please consult https://github.com/keptn/go-utils/actions?query=workflow%3A"Auto+PR+to+keptn%2Fkeptn" for more information.

Will be handled in https://github.com/keptn/keptn/pull/3977


UI: scale VM not able to use same custom offering

ISSUE TYPE

Improvement Request

COMPONENT NAME

UI

CLOUDSTACK VERSION

<IP_ADDRESS>

CONFIGURATION

OS / ENVIRONMENT

SUMMARY

You should be able to use the same compute offering for scaling a VM when the offering is a custom offering with variable cpu and memory values. Via api call scaleVirtualMachine it is possible to use the same serviceofferingid with other values via details parameter.
STEPS TO REPRODUCE

Create a custom compute offering. Create a new VM using this offering. Stop VM and click on the scale VM icon.

EXPECTED RESULTS

You should be able to edit cpu an memory of the VM by using the same custom offering

ACTUAL RESULTS

Custom offering is not visible in UI.

Hi @sbrueseke
Could you please let me know what type of compute offering you created, was it fixed, custom constrained, custom unconstrained

I was able to edit cpu and memory of the vm when i used custom constrained offering

@kiranchavala I am using a Custom constrained.

@sbrueseke I'm not able to reproduce this. Am I missing anything?
https://user-images.githubusercontent.com/153340/229446067-a114a726-9af5-4de1-a31a-7908cda2742d.mp4

@shwstppr thank you very much! Interesting so see. Are you able to use tags for the disk offerings your compute offerings are using? In my test environment I have a primary storage which is using tag local-ssd and a shared storage using the tag shared-linstor. Maybe this is the difference.

@sbrueseke I'll check that

I think I found the root cause of this. It is not an UI related issue, it is an api related one.
When using command listServiceOfferings and parameter virtualmachineid and you provide an id of a VM using a custom offering with diskofferingstrictness=true, the api will not include this offering in the response and so the UI is not able to provide this offering.
My humble opinion is that this is not correct behavior and you should be able to scale cpu and ram even if the used offer in uses strictness for the disk. Since scaleVirtualMachine is able to this using the same offering, listServiceOfferings should also include the used custom offering in the response.

@DaanHoogland since this is not an UI related issue, should I edit this issue and correct subject and post or should we close this one and I open a new one with the correct explanation?

@DaanHoogland since this is not an UI related issue, should I edit this issue and correct subject and post or should we close this one and I open a new one with the correct explanation?

completely your choice @sbrueseke . If you think discussion so far helps whoever solves it and doesn´t obfuscate the "real" issue, edit this one, otherwise open a new one. €0,02

Fixed in #7398


Please relist WatsonTcp 1.0.4 and compatible WatsonCluster on NuGet

In light of recent changes committed, can you please relist WatsonTcp 1.0.4 on NuGet? It can still be accessed by direct reference e.g. https://www.nuget.org/packages/WatsonTcp/1.0.4 but doesn't show up in NuGet Package Manager. Also, please republish WatsonCluster to restore compatibility with WatsonTcp 1.0.4, if you have not already. Thanks!

Done, thank you for the reminder!


Make it clear that utils configuration has a impact on variant generation

As commented here: https://github.com/saadeghi/daisyui/discussions/828#discussioncomment-2930866, while customizing DaisyUI, I caught an unexpected "issue" where variants like alert-info were not being generated. The thing is, for this class to be generated properly, one has to have the utils config set to true. The problem here is that the docs do not make this sufficiently clear IMHO. It doesn't help much that while having utils set to false, the btn-info variant is generated while alert-info is not - which can cause confusion.
Now, I'm not sure if the best approach is to document this behavior or if variants like btn-info should be bound to the same constraints as other components.

It is specified on the docs here:
https://daisyui.com/docs/config/

utils
Boolean (default: true)
If it's true, responsive and utility classes will be added

And you can see all the class names considered as utils right there in the link
Let me know if you have any questions

@saadeghi I've read the docs, especially this section and I think that the confusion lies on how the framework is dealing with all the "variants". So, I'm looking at the component pages here and btn-info is categorized as "modifier" while alert-info is categorized as "responsive". Is this by design?
It seems weird that similar class names are categorized in different ways from component to component. If I may ask, what's the reasoning behind this convention? Seems very confusing.

I might need to make it more clear but here's how it is:
btn is the component class.
Every other class name that we use alongside that, would be a modifier
Modifier classes using just change the color or size and some of these modifier classes are responsive modifier classes.
It might be confusing at first but these are the best names I found to categorize the class names. Part of it is inspired by Tailwind CSS structure and part of it by BEM.
Let me know if you have any questions

I might need to make it more clear but here's how it is: btn is the component class. Every other class name that we use alongside that, would be a modifier Modifier classes using just change the color or size and some of these modifier classes are responsive modifier classes. It might be confusing at first but these are the best names I found to categorize the class names. Part of it is inspired by Tailwind CSS structure and part of it by BEM.
Let me know if you have any questions

Cool, what hasn't clicked for me yet is why alert-info is considered a responsive modifier while btn-info is just a "modifier".
I'm asking because there's clearly a differentiation here because one is generated with utils: true and the other is not.

I'm gonna give some examples of what I'm talking about:

class
type

btn
component

btn-primary
modifier

btn-lg
responsive

btn-square
responsive

alert
component

alert-info
responsive

badge
component

badge-info
responsive

card
component

card-normal
responsive

carousel
component

carousel-vertical
modifier

stats
component

stats-vertical
responsive

tooltip
component

tooltip-primary
modifier

radio
modifier

radio-primary
modifier

radio-lg
modifier

input
component

range
component

range-lg
responsive

We have main components mostly defined as a "component" class. However, radio is listed as a "modifier" while input, which is in the same "category" (forms) is a "component". Variant classes like *-primary|info|success range from "modifier" to "responsive". Sizing classes *-xs|sm|lg range from "responsive" to "modifier" and other classes that change the behavior of a component like stats-vertical or card-normal are either "responsive" or "modifier".
If I can make a suggestion I'd standardize this making all components consistently be "component", sizing utilities be called "sizing", variants like primary and info be called "variant" and everything else that can change the behavior a "modifier".

Ping @saadeghi.

Sorry I missed this issue since it wasn't open. I will check it soon.

class name categorizations for alert, button, radio and badge has been reworked in daisyUI  2.24.0
Some class names were in the wrong categories.

Now all the class names that modify colors are modifier
All class names that modify sizes are responsive

Let me know if you have any questions.

Great work @saadeghi!!! ❤🖤💙🤍💛🧡💚💜🤎


Reactive options is behaving weird?

Hello, I have a simple example where I want to disable the query when an input is undefined or an empty string. Here's my code:
// api.js
import { reactive } from "vue";
import { useQuery } from "vue-query";

const fetcher = async (id) => {
  const res = await fetch("https://jsonplaceholder.typicode.com/todos/" + id);
  const data = await res.json();
  return data;
};

export function useGetTodo(id) {
  const key = reactive(["todo", id]);
  const options = reactive({
    enabled: !!id
  });

  return useQuery(key, () => fetcher(id.value), options);
}

// Component.vue
<template>
  <input type="text" v-model="id" />

Loading...

{{ JSON.stringify(data) }}

</template>

First thing I expect here is that on first load, since the input is empty, the query will not run. The options is inside the reactive property and to my surprise, if I am missing something, it is still running.
You can check it here https://codesandbox.io/s/vue-query-reactive-options-forked-nslhg

So I tried logging const id = ref(""); with
console.log(!!id)

and it actually returns true. To make it return false, it needs to do console.log(!!id.value).
How do we go about this?

So i would go with something like:
export function useGetTodo(id) {
  const enabled = computed(() => Boolean(id.value));
  const key = reactive(["todo", { id }]);
  const options = reactive({
    enabled
  });

  return useQuery(key, () => fetcher(id.value), options);
}

Notice computed property and key that includes id as an object, not directly in array.
The reason for this is that ref is an object actually, not a value. So doing !!{} will always return true.
Also reactive on arrays have some caveats, so passing id in the object will work better.
More concise:
export function useGetTodo(id) {
  const options = reactive({
    queryKey: ["todo", { id }],
    queryFn: () => fetcher(id.value),
    enabled: computed(() => Boolean(id.value))
  });

  return useQuery(options);
}

So i would go with something like:
export function useGetTodo(id) {
  const enabled = computed(() => Boolean(id.value));
  const key = reactive(["todo", { id }]);
  const options = reactive({
    enabled
  });

  return useQuery(key, () => fetcher(id.value), options);
}

Notice computed property and key that includes id as an object, not directly in array.
The reason for this is that ref is an object actually, not a value. So doing !!{} will always return true.
Also reactive on arrays have some caveats, so passing id in the object will work better.
More concise:
export function useGetTodo(id) {
  const options = reactive({
    queryKey: ["todo", { id }],
    queryFn: () => fetcher(id.value),
    enabled: computed(() => Boolean(id.value))
  });

  return useQuery(options);
}

Oh I like the query object instead of parameters thing! That worked. Vue reactivity is weird in times like this lol thank you

Oh btw, this


Out of bounds read in oss-detect-cryptography

Using oss-detect-cryptography gives an error (warning w/ stack trace) when analyzing certain files, like below.
root@412946d6c067:/usr/src/app# oss-detect-cryptography<EMAIL_ADDRESS>
   ____   _____ _____    _____           _            _
  / __ \ / ____/ ____|  / ____|         | |          | |
 | |  | | (___| (___   | |  __  __ _  __| | __ _  ___| |_
 | |  | |\___ \\___ \  | | |_ |/ _` |/ _` |/ _` |/ _ \ __|
 | |__| |____) |___) | | |__| | (_| | (_| | (_| |  __/ |_
  \____/|_____/_____/   \_____|\__,_|\__,_|\__, |\___|\__|
                                            __/ |
                                           |___/
OSS Gadget - oss-detect-cryptography 0.1.307+9be4e1e55f - github.com/Microsoft/OSSGadget
WARN  - Error analyzing /tmp/gyeeinxi.udf/npm-left-pad<EMAIL_ADDRESS>One or more errors occurred. (Index and length must refer to a location within the string. (Parameter 'length'))
WARN  -    at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
   at System.Threading.Tasks.Task.Wait(TimeSpan timeout)
   at Microsoft.CST.OpenSource.DetectCryptographyTool.AnalyzeDirectory(String directory) in /home/vsts/work/1/s/src/oss-detect-cryptography/DetectCryptographyTool.cs:line 549
Summary Results:
Cryptographic Implementations:
  No implementations found.

Cryptographic Library References:
  No library references found.

Other Cryptographic Characteristics:
  No additional characteristics found

I think this is a surfacing a bug in devskim that doesn't arise from the default rules. I did some preliminary debugging but was having issues with symbols.
Hope to address this week.

The problem appears to be with using Application Inspector rules in the DevSkim engine.  If I comment out the lines that include the Application Inspector rules there are no issues.  It would be difficult to identify exactly which Application Inspector rule causes the issue - but it is not relevant anyway since the rules are intentionally compatible.
For now I have a fix that should stop the crash in in DevSkim - and I'll move up the priority of replacing the DevSkim engine with the App Inspector engine.

Fixed in devskim. I'll update the dependency and confirm tomorrow.


Crash when lambda connected to signal attempts to use freed instance

Godot version
v4.1.2.rc.custom_build [6884be6b1]
System information
Godot v4.1.2.rc (6884be6b1) - Debian GNU/Linux trixie/sid trixie - Vulkan (Mobile) - dedicated NVIDIA GeForce GTX 1060 (nvidia) - Intel(R) Core(TM) i5-8400 CPU @ 2.80GHz (6 Threads)
Issue description
When:

A lambda is connected to a signal
The lambda references some node
That node gets freed
The signal is then triggered

The game will crash. If running in the editor, no feedback is produced in the output or debugger.
I initially saw this on v4.1.stable.official [970459615], so compiled the engine from the current 4.1 tag to get a backtrace with debug symbols:
Godot Engine v4.1.2.rc.custom_build.6884be6b1 - https://godotengine.org
Vulkan API 1.3.224 - Forward+ - Using Vulkan Device #1: NVIDIA - NVIDIA GeForce GTX 1060

Connecting
Freeing
Emitting
ERROR: Can't get method on CallableCustom "<anonymous lambda>(lambda)".
   at: get_method (core/variant/callable.cpp:386)

================================================================
handle_crash: Program crashed with signal 11
Engine version: Godot Engine v4.1.2.rc.custom_build (6884be6b173d22c758806e2207d64f080e9ea7c1)
Dumping the backtrace. Please include this when reporting the bug to the project developer.

-- END OF BACKTRACE --
================================================================

Expectation
If the lambda is replaced by a regular callable (eg. do_boop.bind(...)), then the editor will provide helpful error messages and debugging information:

I would expect something similar when a lambda is being used.
Steps to reproduce
This is the script from the attached MRP. The key points appear to be:

The signal is defined in a persistent location (ie. doesn't itself get free'd)
A lambda is used in the connect call
The crash is triggered on the emit() call

extends Node2D

signal boop

func _ready():
	var level : Node2D = $Level
	print("Connecting")

	# Will show runtime errors in editor
	#boop.connect(do_boop.bind(level))

	# Will crash with no editor feedback
	boop.connect(func(): do_boop(level))

	print("Freeing")
	level.queue_free()
	await get_tree().create_timer(0.1).timeout
	print("Emitting")
	boop.emit()
	print("Done")

func do_boop(level):
	level.position = Vector2.ZERO

(Strangely, if running in the editor, the Emitting text does not appear in the output tab, though it does appear in STDERR)
Minimal reproduction project
Self-contained scene which reliably reproduces the crash:
lambda_crash.zip

Strangely, if running in the editor, the Emitting text does not appear in the output tab, though it does appear in STDERR

This might be threading related and to the problems with printing from threads
This seems related to #79591


Building enzyme with browserify fails after<EMAIL_ADDRESS>

After enzyme 2.8.1 I cannot build enzyme with browserify. Enzyme 2.8.2 does not fix this issue either.
With 2.8.0 and ^2.7.1 everything works ok.
I put all my setup and errors in gist: https://gist.github.com/tiriana/26a0394d8f9e26e28a1726bc33b13317
Let me know if you rather have this info in the very ticket instead.

There is same problem in webpack too

The fix here is to mark "react-test-renderer/shallow" as an external dependency.
See https://github.com/airbnb/enzyme/blob/master/docs/guides/browserify.md. Follow these instructions, except for you'll need this line:
b.external("react-test-renderer/shallow");

The documentation should be updated.

I don't think this is relevant any more now that we're on v3.


Add documentation about the audit message requirements

Hi @msmock
I had that documentation about ATNA audit logging as an extension of #24, in my repository.
I guess it might be helpful to others, it's a good summary of all the requirements.
A good review would be nice.

Rebased on top of #24

Rebased on top of main and fixed a few issues.


Add istioctl analyze warning about fragile EnvoyFilter patch order

(This is used to request new product features, please visit https://discuss.istio.io for questions on using Istio)
Describe the feature request
An EnvoyFilter with an insert position relative to another filter (e.g., INSERT_BEFORE) can be
fragile because the default order of filter evaluation is based on the creation time of the filters.
If the insertion point for a filter is not found, the filter will not be added to the chain and nothing will
appear in the error log to indicate that it was skipped.
Documentation describing this subtlety has been added in https://github.com/istio/istio.io/pull/10935.
It would be good to also make istioctl analyze produce a warning message for this kind of
fragile configuration.
Describe alternatives you've considered
Affected product area (please put an X in all that apply)
[ ] Docs
[ ] Installation
[ ] Networking
[ ] Performance and Scalability
[ ] Extensions and Telemetry
[ ] Security
[ ] Test and Release
[x] User Experience
[ ] Developer Infrastructure
Affected features (please put an X in all that apply)
[ ] Multi Cluster
[ ] Virtual Machine
[ ] Multi Control Plane
Additional context

Reopening since the fix was prematurely merged. See comment here: https://github.com/istio/istio/pull/38294#discussion_r847393883

@frankbu @AndreaM12345 @ericvn Can this be closed now? This is mentioned as something fixed in the release note of 1.14.
cc @lei-tang @GregHanson


Query parameter array types are not parsed properly, returning a warning.

If you have a swagger document that has, among other things, the following:
/test:
  get:
    parameters:
      - name: testing
        in: query
        required: true
        schema:
          type: array
          items:
            type: string

And attempt to curl it using something like the following:
curl localhost:3000/test?testing%5B0%5D=123&testing%5B1%5D=234

( unescaped, that's testing[0]=123&testing[1]=234 )

You'll see a warning logged,
[
    {
        "message": "wrong parameter test in query",
        "error": [
            {
                "code": "INVALID_TYPE",
                "params": [
                    "array",
                    "string"
                ],
                "message": "Expected type array but found type string"
            }
        ]
    }
]

This happens here:
https://github.com/isa-group/oas-tools/blob/2.1.7/src/middleware/oas-validator.js#L169-L177
When the code enters this point, it attempts to JSON.parse(req.query.testing).
req.params.testing at this point is already an array -- attempting to call JSON.parse with an array fails so so it goes into the catch block, and at that that point it calls String on it, winding up with  "123,234", which fails validation because it's a string, not an array.
It might make sense to call into Array.isArray() here and if the provided value is an array, don't attempt to call JSON.parse it.
I'm not sure how many people are passing complicated data structures through querystrings -- I guess one could include JSON documents there (I for one would not)... given this possible use case, it might make sense to still parse for non-strings and non-arrays & to iterate the array (if it is one), and call JSON.parse(item) with a fallback to String(item) as well.

Hello,
Same problem here.
/users/{ids}:     get:       x-swagger-router-controller: users       operationId: get       parameters:         - name: ids           in: path           description: ids           required: true           schema:             type: array             items:               type: integer
And this error in the logs :

2019-11-26T09:21:20.609Z debug: Register: GET - /users/:ids
2019-11-26T09:21:20.610Z debug:   GET - /users/{ids}
2019-11-26T09:21:20.610Z debug:     OAS-doc has x-swagger-router-controller property users
2019-11-26T09:21:20.628Z debug:       Controller for GET - /users/{ids}: OK
2019-11-26T09:21:56.632Z debug: Checking security...
2019-11-26T09:21:56.633Z info: Requested method-url pair: get - /users/12,13,14
2019-11-26T09:21:56.634Z debug: OASValidator  -res.locals.requestedSpecPath: /users/{ids}
2019-11-26T09:21:56.637Z error: [{"message":"Wrong parameter ids in params. ","error":[{"code":"INVALID_TYPE","params":["array","string"],"message":"Expected type array but found type string","path":"#/"}]}]

Can I do anything to help fix this problem ?

Same issue, it works if you supply something like localhost:/endpoint/[1,2,3] rather than localhost:/endpoint/1,2,3. This I believe is erroneous.
https://swagger.io/docs/specification/serialization/

Parameter serialization is now supported in version v3.0.0.


React to default hosting configuration changes

Updates for IISIntegration after https://github.com/aspnet/Hosting/pull/730
cc @Tratcher @davidfowl @BrennanConroy
These can be checked in first.

:shipit:


FrontEndTest Crashes

Help

do you have a specific error code you can show us

not really it doesn't say Details on the stopped working messege

oh I think I know why. you're on windows 7, and we're doing some unguarded live tile checks. you're not missing out on much though, the entire application is just testing live tiles

Im on windows 8.1

oh it's probably not finding the application then

How do I make it find the app then

Um this MainWindow is awkward

It's hardcoded to one specific version of the app. Literally the exact same issue as the Double-Decker-Bar ._.

Oh RIP

...which, by the way, isn't actually an "issue" at all.

we'll fix it later

k then


Facing unexpected behavior while requesting a resource !

I am facing unexpected behavior while requesting my resource.
I m using scopes, clients and users from database. I have implemented IClientStore, IScopeStore, IProfileService and IResourceOwnerPasswordValidator
I am able to get access token from authorization server.
when I request resource I am getting "No endpoint found - try /api/todo."
Logs
dbug: Microsoft.AspNetCore.Server.Kestrel, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=adb9793829ddae60[1]
Connection id "0HKS5BJVAIFU3" started.
info: Microsoft.AspNetCore.Hosting.Internal.WebHost[1]
Request starting HTTP/1.1 POST http://<IP_ADDRESS>:86/Api/Account/Login application/x-www-form-urlencoded 39
info: Microsoft.AspNetCore.Authentication.JwtBearer.JwtBearerMiddleware[2]
Successfully validated the token.
info: Microsoft.AspNetCore.Authentication.JwtBearer.JwtBearerMiddleware[3]
HttpContext.User merged via AutomaticAuthentication from authenticationScheme: Bearer.
dbug: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware[1]
POST requests are not supported
dbug: Microsoft.AspNetCore.Routing.Tree.TreeRouter[1]
Request successfully matched the route with name '' and template 'api/Account/Login'.
dbug: Microsoft.AspNetCore.Mvc.Internal.MvcRouteHandler[1]
Executing action DiagniaWeb.Controllers.Api.AccountController.Login (DiagniaWeb)
info: Microsoft.AspNetCore.Authorization.DefaultAuthorizationService[2]
Authorization failed for user: xyz.
warn: Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker[1]
Authorization failed for the request at filter 'Microsoft.AspNetCore.Mvc.Authorization.AuthorizeFilter'.

Solved it.
My api was not setting the name : O


Problem with the SSL CA cert

it works fine in windows, but when running from a notebook using linux, I get this erros
duckdb.sql(''' create or replace view lineitem as select * from 'azure://tpch/lineitem/*.parquet';''')

Error: Invalid Error: Fail to get a new connection for: https://dddddddd.blob.core.windows.net./ Problem with the SSL CA cert (path? access rights?)

I'm able to replicate this under both WSL and Ubuntu, with the following:
INSTALL azure;
LOAD azure;
SET azure_storage_connection_string='DefaultEndpointsProtocol=https;AccountName=azuresdkdocs;AccountKey=redacted;EndpointSuffix=core.windows.net';
SELECT count(*) FROM 'azure://development/testing_of_duckdb/file.snappy.parquet';

Partial workaround for users on linux running into this: installing curl seems to fix the issue. The problem here is that libcurl is statically linked into the extension and there are certificates missing or in the wrong path. Installing curl may resolve this issue for some environments but a more thorough solution is required.
People at ArcticDB seem to be running into the same issue here: https://github.com/man-group/ArcticDB/issues/514. There's a PR up as we speak. They have actually already gone through the work of getting a PR in at the azure sdk for setting the path, it will be available at the 10th of november through vcpkg. We should be able to make use of their hard work by updating the azure sdk by then and exposing the path through duckdb.

This one seems to be affecting my workflows as well, see discussion at https://github.com/microsoft/PlanetaryComputer/discussions/278#discussioncomment-7229515
Would love to be able to use this azure extension, it'll make working with a few GeoParquet datasets a lot easier. Thanks for all your great work on this so far!

I'm using the python:3.10.13-bullseye image and I got the same issue. I already had the latest version of libcurl4-openssl-dev but I tried to also install libcurl4-gnutls-dev but still got
Error: Invalid Error: Fail to get a new connection for: https://stsynussp.blob.core.windows.net/. Problem with the SSL CA cert (path? access rights?)

thanks for reporting @cholmes and @deanm0000. This is definitely something that will need fixing

Same also happens to me using Ubuntu, there were no problem using Windows instead

Are the binaries are built on an RHEL distribution ?: man-group/ArcticDB#514 (Azure/azure-sdk-for-cpp#4738)
The following "fixes" the error for me on Ubuntu 22.04, but I don't know if there are security implications:
mkdir -p /etc/pki/tls/certs
ln -s /etc/ssl/certs/ca-certificates.crt /etc/pki/tls/certs/ca-bundle.crt

This worked for me :)
Thanks @kmatt

People at ArcticDB seem to be running into the same issue here: man-group/ArcticDB#514. There's a PR up as we speak. They have actually already gone through the work of getting a PR in at the azure sdk for setting the path, it will be available at the 10th of november through vcpkg. We should be able to make use of their hard work by updating the azure sdk by then and exposing the path through duckdb.

The PR mentioned above appears to be released now:
https://github.com/Azure/azure-sdk-for-cpp/pull/4982

@daviewales Thanks for the ping, latest vcpkg release now has this version of azure sdk as well, I will try to find some time to look into this issue in the near future!

Hello,
just for info I also got this annoying issue so I have make a PR#35 to make possible:

to configure the path
try to find the right path directly

hope it will help :)

Is this fix now released in latest Azure extension?

@brianwyka not yet!
however, once this job has succeeded, you can use the nightly build of azure, which does contain these fixes with: force install azure from 'http://nightly-extensions.duckdb.org';

The azure website page of the doc is not up to date so here you go:

Name
Description
Type
Default

azure_transport_option_type
Underlying adapter to use in the Azure SDK. Valid values are: default or curl.
VARCHAR
default

Setting azure_transport_option_type explicitly to curl with have the following effect:

On Linux, this may solve certificates issue (Error: Invalid Error: Fail to get a new connection for: https://<storage account name>.blob.core.windows.net/. Problem with the SSL CA cert (path? access rights?)) because when specifying the extension will try to find the bundle certificate in various paths (that is not done by curl by default and might be wrong due to static linking see issue).
On Windows, this replaces the default adapter (WinHTTP) allowing you to use all curl capabilities (for example using a socks proxies).
On all operating systems, it will honor the following environment variables:

CURL_CA_INFO: Path to a PEM encoded file containing the certificate authorities sent to libcurl. Note that this option is known to only work on Linux and might throw if set on other platforms.
CURL_CA_PATH: Path to a directory which holds PEM encoded file, containing the certificate authorities sent to libcurl.

@brianwyka not yet!
however, once this job has succeeded, you can use the nightly build of azure, which does contain these fixes with: force install azure from 'http://nightly-extensions.duckdb.org';

Thanks for the great work!
I am not sure this fixed it for me. I performed duckdb.sql("force install azure from 'http://nightly-extensions.duckdb.org';"). I still had to do a cert simlink for it to work on WSL ubuntu 22.04.

Hi @luuk-codebeez
Just to be sure, did you set the the variable ?
SET azure_transport_option_type = 'curl';

Hi @luuk-codebeez
Just to be sure, did you set the the variable ?
SET azure_transport_option_type = 'curl';

Woops that worked with the nightly build

@luuk-codebeez Great to hear!
I just deployed the azure nightly binaries, so from now on with force install azure you will get the updated extension with this feature in it.
Also, this has now been added to the docs https://duckdb.org/docs/extensions/azure.html.
Thanks a lot for the effort here @quentingodeau!

Would it make sense to make SET azure_transport_option_type = 'curl'; the default when running on Linux?

I thought about that but didn't do it, here my opinion/experience on this, when you change the default behavior of something it's really complex to rollbacks this changes.
When you change the default you have to keep the change so if tomorrow the azure SDK evolve to handle our use case by for example adding an AZURE_SSL_BUNDLE_PATH environment variable, then all user that are used to the SDK will expect that the env variable is handle exactly as it is describe in the azure doc. So in this case I will have to then handle this case with keeping the other parameters that I have allowed. I agree that it may not look like a great deal like this but then you have more and more small things like this pills up and making the code more and more complex for newcomers that do not have the full history.
But once again it's only an opinion ^^


turkish menu fix

close https://github.com/vim/vim/pull/10724

@chrisbra

you don't need to ping me, when creating a PR. Please don't change the maintainer. unlesss you intend to take over the file maintainership or the mentioned maintainer indicates he will do so.

@chrisbra

Thanks. Please change the Commit Author.

@chrisbra

closing for the reasons mentioned here https://github.com/vim/vim/pull/14703#issuecomment-2092337585


Fix typo in MultiQubitSystemMeasurements.ipynb

Hello!
I fixed a typo in MultiQubitSystemMeasurements.ipynb.

@delight1019 please read the following Contributor License Agreement(CLA). If you agree with the CLA, please reply with the following information.
@microsoft-github-policy-service agree [company="{your company}"]

Options:

(default - no company specified) I have sole ownership of intellectual property rights to my Submissions and I am not making Submissions in the course of work for my employer.
(when company given) I am making Submissions in the course of work for my employer (or my employer has intellectual property rights in my Submissions by contract or applicable law). I have permission from my employer to make Submissions and enter into this Agreement on behalf of my employer. By signing below, the defined term “You” includes me and my employer.

Contributor License Agreement

@microsoft-github-policy-service agree

I just realized that the workbook for task 7 has the same issue, since task descriptions are duplicated in both notebooks. Can you do the same fix there as part of this pull request? (This is your 4th one so you shouldn't need a separate one for Hacktoberfest :-))

Oops, I missed the workbook :( Thank you!!


Operator-sdk scorecard life-cycle

Type of question
Does the operator-sdk scorecard have any type of life-cycle, ie pre install, install, post install?  If so, can the external plugin be called in different life-cycle steps?
Question
What did you do?
Running basic external plugin example to call bash script.
What did you expect to see?
Assuming the external plugin is running AFTER scorecard installs operator and operator example CR and BEFORE the operator and namespace are deleted from the cluster.  Is it possible to run the external plugin BEFORE an operator is installed?  For example checking resources in a cluster.
What did you see instead? Under which circumstances?
NA
Environment

operator-sdk version 0.14.0

operator-sdk version
operator-sdk version: "v0.14.0", commit: "4fb185b9d45682473b4951c9788f845ba2fa5b12", go version: "go1.13.3 linux/amd64"

Kubernetes version information:

Client Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.6", GitCommit:"96fac5cd13a5dc064f7d9f4f23030a6aeface6cc", GitTreeState:"clean", BuildDate:"2019-08-19T11:13:49Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"14+", GitVersion:"v1.14.6+dea7fb9", GitCommit:"dea7fb9", GitTreeState:"clean", BuildDate:"2019-11-19T17:18:37Z", GoVersion:"go1.12.12", Compiler:"gc", Platform:"linux/amd64"}

Kubernetes cluster kind:

Additional context
Add any other context about the question here.

based on comments from #2398 closing as this was really related to external plugin capabilities


Update Advanced Install instructions for Ubuntu to use correct username

Bug Report
System Information
N/A
I confirm:

[x] that I have searched for an existing bug report for this issue.
[x] that I am using the latest available version of AMP.
[x] that my operating system is up-to-date.

Symptoms

On the Ubuntu page for installing AMP (https://cubecoders.com/AMPInstall), there is inconsistent information regarding the name of the user that should be created to run AMP.
The instructions refer to the user as AMP when explaining how to create it:
sudo useradd -d /home/AMP -m AMP -s /bin/bash
sudo passwd AMP

However, the instructions later assume that the user name is amp, when explaining how to switch to it to run ampinstmgr commands.
As Linux is case-sensitive for user names, the instructions should be consistent. For consistency with how getamp.sh works and how the AMP systemd services work (and with the Debian instructions on the website), I suggest that the user should be amp and the instructions for user creation should read:
sudo useradd -d /home/amp -m amp -s /bin/bash
sudo passwd amp

Reproduction

See above.

The documentation just hasn't been updated. Just needs to be updated. It should be "amp" now instead of "AMP"

Isn't that what I said? lol

sure, but I put this in #Development in discord for him to read. it'll be taken care of when he gets a chance.

Gotcha. Lodging it here was the only way I could think of to make sure it was noticed. Will use #Development for little issues like this in future

Fixed 👍

Thanks


[Q] tsduck.tables.model.xml - Illegal instruction

Hello Lelegard
I'm trying to install TSDuck on a Okdo Rock 4C+ (similar to Raspberry 4). I'm using Debian 11 64bit.
The scenario is:
uname -a
Linux rock-4c-plus 5.10.110-1-rockchip #27b5bf1db SMP Mon Apr 3 11:17:08 UTC 2023 aarch64 GNU/Linux
Linux is Debian 11 64bits.
First of all I executed the script install-prerequisites.sh without problems; Next I execute only "make" command and after some time it show this error message:
....
[CXX] utestXML.cpp
[LD] /home/a2t/Descargas/tsduck/bin/release-aarch64-rock-4c-plus/utest
[LD] /home/a2t/Descargas/tsduck/bin/release-aarch64-rock-4c-plus/utest_static
[DEP] tsprofiling.cpp
[DEP] tsmux.cpp
[DEP] setpath.cpp
[CXX] tsmux.cpp
[LD] /home/a2t/Descargas/tsduck/bin/release-aarch64-rock-4c-plus/tsmux
[CXX] tsprofiling.cpp
[LD] /home/a2t/Descargas/tsduck/bin/release-aarch64-rock-4c-plus/tsprofiling
[GEN] tsduck.tables.model.xml
make[2]: *** [Makefile:67: /home/a2t/Descargas/tsduck/bin/release-aarch64-rock-4c-plus/tsduck.tables.model.xml] Illegal instruction
make[1]: *** [Makefile:38: default] Error 2
make: *** [Makefile:61: default] Error 2
What's wrong? I searched in the issues but I found nothing; thank in advance.

Sorry, I just founded a similar case #1203

Hi @jof-cw
Also involving @tthil as he reported a similar issue first.
The file tsduck.tables.model.xml is generated using the TSDuck tool tsxml. So, the build starts with the libraries, then the applications and, finally, the model file using the previously generated application. If there is a problem with the generated code of the application, building the final configuration file will fail.
If two unrelated people have the same "illegal instruction" issue, this is not an accident and there must be something to investigate.
On my side, I did tests on a Mac M1 using macOS and Linux Ubuntu. The M1 is an Arm64 chip but more advanced in terms of Arm features. The M1 uses Apple Firestorm cores at Arm8.5-A level while the Rockchip RK3399-T uses a mix of Cortex A72 and A53 at Armv8.0-A level. I wonder if this difference has an impact on the binary.
Specifically, I customized the CRC32 and crypto code to use the accelerated Arm64 instructions, when available. The compiled code contains the two versions, with and without acceleration. The presence of the special instructions is checked at runtime and the corresponding code path is taken only when the instructions are present. So, in theory, it should work in all cases. However, I wonder if there could be some hidden issue.
More details on Arm64 features here: https://github.com/lelegard/arm-cpusysregs/blob/main/docs/features.md

Hello Lelegard; thank you for your response.
I done the same as @tthil with the source zip file and the compilation was done without problems.
I tried to compile the tool that you told me (arm-cpusysregs) but the compilator showed a lot of erros related with some variables; if you need a feedback I would open a new issue on that place.
Best regards; Jesus

I done the same as @tthil with the source zip file and the compilation was done without problems.

What did you do exactly the first time? Which zip file did you download? Is it a zip of the current state of the repo or the zip of the latest release?
Cloning the repo or getting an archive of the current state of the repo shall produce exactly the same result. If one operation consistently pulls garbage, there is a problem that we need to investigate.

I tried to compile the tool that you told me (arm-cpusysregs) but the compilator showed a lot of erros related with some variables; if you need a feedback I would open a new issue on that place.

This was just a pointer to additional information on the various Arm features. If you are only interested in TSDuck, this project is useless to you. It is only useful to study some technical aspects of the Arm64 internals. That being said, you should not get compilation errors on an Arm64 Linux system, unless you use an old compiler.

Hello Lelegard

What did you do exactly the first time? Which zip file did you download? Is it a zip of the current state of the repo or the zip of the latest release?

The first time I cloned the repository with git clone <...>

This was just a pointer to additional information on the various Arm features. If you are only interested in TSDuck, this project is useless to you. It is only useful to study some technical aspects of the Arm64 internals. That being said, you should not get compilation errors on an Arm64 Linux system, unless you use an old compiler.

I am interested in TSDuck only; but for your interest, this Debian distro use gcc version 10.2.1 20210110 (Debian 10.2.1-6), maybe this is old.
Thank you so much; Jesus

The first time I cloned the repository with git clone <...>

This is worrying. Especially because two independent users got the same problem around the same time frame. If you still have the cloned repo, could you make a recursive diff with the unzipped archive? They should be identical. But they are probably not since one works and the other doesn't.

I just cloned the repo and downloaded the master zip. The contents are identical, just as expected:
$ git clone --depth 1 https://github.com/tsduck/tsduck tsduck-git-clone
$ curl https://codeload.github.com/tsduck/tsduck/zip/refs/heads/master -o tsduck-master.zip
$ unzip tsduck-master.zip
$ diff -Naur -x .git tsduck-git-clone tsduck-master
$

What did you do differently?

This is worrying. Especially because two independent users got the same problem around the same time frame.

I was right to worry, there is an actual issue (that I just fixed). This is due to the lack of flexibility of GCC in terms of code generation for Arm64. This is a bit tricky and deserves some explanations.
A few months ago, I drastically improved the performances of CRC32 and crypto on Arm64 by using the corresponding specialized instructions (CRC, AES, SHA-1/256/512). These instructions are optional in the Arm architecture. So, the code anticipates this: the level of features in the current Arm CPU is checked. If the corresponding instructions are present, we use the accelerated version of the algorithm. Otherwise, we use the standard portable implementation. This is dynamically decided in one single binary.
So far, so good, we have a universal binary which can run on all Arm64 CPU's and gets the best of the CPU on which it executes. The problem is the GCC compiler. If we want to compile the asm() code with the specialized instructions (or the equivalent intrinsics), we must specify a minimum level of architecture, for instance -march=armv8.2-a+crypto+sha2+sha3. Otherwise, GCC refuses to compile the specialized parts of the code.
However, this has undesired side effects. Using -march=armv8.2-a+..., GCC feels free to generates post-Armv8.0 instructions here and there. When you use an Armv8.0 CPU such as the Arm Cortex A72, the CPU core in the BCM2711 SoC as used in the Raspberry Pi 4, it fails with "illegal instruction".
I only had Armv8.4 and Armv8.5 CPU's to test and the problem is invisible at these levels of architecture. Trying on a Raspberry Pi 4, I was able to determine that the code crashes on an atomic add instruction (ldaddab), which is part of Arm FEAT_LSE, which an Armv8.1 feature. This explains the problem.
I would expect the compiler to accept optional features (crc, crypto, sha2, etc) without being obliged to raise the level of architecture. Ideally, some pragma should allow to change the level of architecture and features in specific portions of source code which are called in a specific context. Unfortunately, GCC is rather limited in that matter. The Arm code generation is still too influenced by the embedded world where you recompile everything for each target. Too rigid, not flexible enough.
I consequently modified the structure of the source code wherever specialized instructions are used. The accelerated parts of the code - which are called only when the corresponding feature is implemented in the current CPU - were moved in separate modules. Each of these specialized modules is built with a customized -march option (see src/libtsduck/Makefile). The rest of the code is compiled with the default Arm64 level of architecture.
Now, the same binary, same installer, works at all level of architectures. To view the level of features of the current platform, use the following command.
On a Raspberry Pi 4:
$ tsversion --version=acceleration
CRC32: yes, AES: no, SHA-1: no, SHA-256: no, SHA-512: no

On a MacBook M1:
$ tsversion --version=acceleration
CRC32: yes, AES: yes, SHA-1: yes, SHA-256: yes, SHA-512: yes

So, you may now clone the repo and rebuild.
BTW, cloning the repo or downloading a zip does not make any difference. The source code and the binaries are identical and exhibit the same issue. I suspect that you both downloaded the zip from the previous release, 3.33, instead of a zip of the current state of the repo. Version 3.33 did not implement the specialized instruction optimization and did not have the problem.

Hello Lelegard
Sorry for my long silence; I was working out of office.
I just done your commands and now it works fine.
Thank you so much.

.

Hello Lelegard
I apologize for my long silence; I was working out of office all this time.
I tried the commands you told me and now it works fine. Thank you so much for your work and patience :)


make backward compatible for the url change

related to #8712

Thanks, Seagnchan. Tested in my VM and it works just fine.


Service don't work

Thanks a lot for this repo !
I want to automate the start but the service don't work.
I'm on a raspberry pi 4/64 bits and record on the SD card directly (/home/pi/RECORD/)
I don't know why but when I launch in background, the application dosen't record. (I try to use & or do an sh script or ....)
The only way to make this happend is to use 'screen' to launch in a background terminal

What does your service file look like? The service file is located at: /etc/systemd/system/ndi_recorder.service
By default, the service file records to this path: /media/root/primary/

Service file is like :
[Unit]
Description=NDI Recorder
After=network-online.target

[Service]
User=root
ExecStart=/opt/ndi_recorder/bin/ndi_recorder -p /home/pi/RECORD
Restart=always
RestartSec=5

[Install]
WantedBy=multi-user.target

ndi_recorder running well, I can acces with browser, start record but no recorded file ...

Strange...I will do some testing of my own and see if I can figure out what the problem is.


Add isTestRunner utility

The isTestRunner function can be used to determine if a story is rendering in the context of the test runner.
import { isTestRunner } from '@storybook/test-runner/is-test-runner';
export const MyStory = () => (

Is this story running in the test runner?

{isTestRunner() ? 'Yes' : 'No'}

);

The result of isTestRunner() will be true in the following scenarios:

In the browser, when the story is rendered while running the test runner
In node, if you prepend your Storybook script with STORYBOOK_TEST_RUNNER=true

📦 Published PR as canary version: 0.8.0--canary.198.b923816.0

:sparkles: Test out this PR locally via:
npm install<EMAIL_ADDRESS># or
yarn add<EMAIL_ADDRESS>

Codecov Report
Base: 76.53% // Head: 76.63% // Increases project coverage by +0.09% :tada:

Coverage data is based on head (b923816) compared to base (aa45827).
Patch coverage: 80.00% of modified lines in pull request are covered.

Additional details and impacted files
@@            Coverage Diff             @@
##             next     #198      +/-   ##
==========================================
+ Coverage   76.53%   76.63%   +0.09%
==========================================
  Files          11       12       +1
  Lines         179      184       +5
  Branches       37       38       +1
==========================================
+ Hits          137      141       +4
- Misses         42       43       +1

Impacted Files
Coverage Δ

.storybook/preview.js
80.00% <80.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


[Waiting Reviews]: Fixes the parameters descriptions

This PR is related to both #1364 and #1328, it fixes the mismatch in the function's documentation that listed many more parameters than the jax version really accepts.
As of now it's still not fixing the function's signature in the documentation, but already removes all the descriptions of the parameters that are not present in the jax version of the function.
Is there interest in further development of this PR?

So, right now I fixed the signature but there are two problems:

When the argument's names from numpy to jax the parser will say the function takes no argument

The code is still not compatible with python 2.

If there's interest in this PR I will fix this bugs.

I think this is a worthwhile improvement! Can you give an example of a function where problem 1 shows up?

Numpy's not_equal has arguments x1 and x2, but jax's not_equal has arguments x, y.

Should we just change the jax functions to have the same parameter names as numpy?

Well, I think it would be a really good idea, want me to do it in this PR? Or open another one?

Separate one is generally better (makes it easier to review, etc.), but it's not a big deal in this case so either way.

Oks, I'm on it. Numpy docstrings are full of inconsistencies (like some having a tab at the beggining of each line, and others not)
Also where seems to be a special case, because it is jitted, so it's parameters names become really weird, still thinking about how to deal with it.

So, I believe there are some 170 functions left to fix the documentation, which is mostly manual work, and that the function that updates the documentation is pretty much in its final form, feedback is as always appreciated

@gnecula

Hey @gnecula and/or @skye and/or @jekbradbury, can someone take a look at this please? I wanted to finish it so I can move on to other PRs (like the reformatting PR or the seperable conv PR)

Hey @gnecula and/or @skye and/or @jekbradbury, can someone take a look at this please? I wanted to finish it so I can move on to other PRs (like the reformatting PR or the seperable conv PR)

One more thing, can you please squash the 17 commits. I could do it myself during merge, but I think it is best if you massage the description.

I guess that's kind of it (I seem to have failed to squash all the commits, still getting the hang of git, sorry)