Tdarr Qsv didn't work for me  (I714700-UHD770)

✅ Have you read and understood the above guidelines?
yes
📜 What is the name of the script you are using?
tdarr script
📂 What was the exact command used to execute the script?
bash -c "$(wget -qLO - https://github.com/community-scripts/ProxmoxVE/raw/main/ct/tdarr.sh)"
📝 Provide a clear and concise description of the issue.
The script installs correctly, but within tdarr I do not have acceleration. I use default settings, advanced, privileged and unprivileged..  nothing...
⚙ What settings are you using?

[x] Default Settings
[x] Advanced Settings

🖥 Which Linux distribution are you using?
Debian 12
🔄 Steps to reproduce the issue.
Install default, or advanced...
I Entering the FFmpeg Help terminal I enter "-init_hw_device"
ffmpeg version 5.0.1-static https://johnvansickle.com/ffmpeg/ Copyright (c) 2000-2022 the FFmpeg developers
built with gcc 8 (Debian 8.3.0-6)
configuration: --enable-gpl --enable-version3 --enable-static --disable-debug --disable-ffplay --disable-indev=sndio --disable-outdev=sndio --cc=gcc --enable- fontconfig --enable-frei0r --enable-gnutls --enable-gmp --enable-libgme --enable-gray --enable-libaom --enable-libfribidi --enable-libass --enable-libvmaf --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librubberband --enable-libsoxr --enable-libspeex - -enable-libsrt --enable-libvorbis --enable-libopus --enable-libtheora --enable-libvidstab --enable-libvo-amrwbenc --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libdav1d --enable-libxvid --enable -libzvbi --enable-libzimg
libavutil 57. 17,100 / 57. 17,100
libavcodec 59. 18,100 / 59. 18,100
libavformat 59. 16,100 / 59. 16,100
libavdevice 59. 4,100 / 59. 4,100
libavfilter 8. 24,100 / 8. 24,100
libswscale 6.4.100 / 6.4.100
libswresample 4.3.100 / 4.3.100
libpostproc 56.3,100 / 56.3,100
Unrecognized option 'init_hw_device'.
Error splitting the argument list: Option not found

In the shell in proxmox:
root@proxmox:~# ls -l /dev/dri
total 0
drw-rw---- 2 root root 80 Dec 22 10:58 by-path
crw-rw---- 1 root video 226, 0 Dec 22 10:58 card0
crw-rw---- 1 root render 226, 128 Dec 22 10:58 renderD128
shell in tdarr
root@tdarr:~# ls -l /dev/dri
total 0
drw-rw---- 2 root root 80 Dec 22 10:58 by-path
crw-rw---- 1 root video 226, 0 Dec 22 10:58 card0
crw-rw---- 1 root render 226, 128 Dec 22 10:58 renderD128
On another Lxc I have Jellyfin running with acceleration, I have tried to install several Intel drivers, but nothing.. I would need to update ffmpeg, which I see is version 5, but I don't know how to do it. Am I doing something wrong?
❌ Paste the full error output (if available).
No errors..
🖼 Additional context (optional).
No response

Show pct config of LXC as default unprivilegeted install

``  GNU nano 7.2                                                         /etc/pve/lxc/100.conf
#

Tdarr LXC

>

#
arch: amd64
cores: 2
dev0: /dev/dri/card0,gid=44
dev1: /dev/dri/renderD128,gid=104
features: keyctl=1,nesting=1
hostname: tdarr
memory: 2048
net0: name=eth0,bridge=vmbr0,hwaddr=BC:24:11:64:46:5F,ip=dhcp,type=veth
onboot: 1
ostype: debian
rootfs: AlmacenNano:100/vm-100-disk-0.raw,size=4G
swap: 512
tags: arr;community-script
unprivileged: 1

?

I was able to get mine working, I am happy to chat more directly if you need on the discord. I am using the same username. However, here's a small checklist:

You are sure you are using a QSV enabled intel processor with intel UHD gfx enabled.
You have installed any drivers needed for this processor depending on your OS
The /dev/dri/ are passed into the container via proxmox
The permissions are properly set to the user that is running tdarr
The user running tdarr is a part of the correct group
The linux container that holds tdarr has necessary VAAPI support drivers/libs installed (you can test this by installing vainfo and running vainfo, this should output a huge list of available encoding/decoding profiles).
The tdarr instance is set to only use the hardware encoding profiles you support (nvenc for nvidia cards, vaapi for quicksync on linux, and QSV for quicksync on windows).

Thank you very much, I finally got the Docker version working. I will try this script later, it may be a permissions problem, or the version of the "non free" drivers.
Regards.

After you run the script to setup a Tdarr lxc, to make use of QSV you should install jellyfin-ffmpeg, then update your tdarr server and node configuration files to point to jellyfin-ffmpeg.
Run,
apt update && apt install jellyfin-ffmpeg-y


Minor grammatical error fix

I have added a minor grammatical update that appears to have been an oversight.

thanks!


Add LocalTime adapter for Java and Kotlin

PR related with issue: https://github.com/apollographql/apollo-kotlin/issues/4828
Added JavaLocalTimeAdapter and KotlinxLocalTimeAdapter

This looks good!
The CI failed because we track API changes - could you run ./gradlew apiDump and push it? 🙏


0x666EE13f8b21ffC3Af675A56e1aE5Ddebda841e5

Name = jupwifhat
Symbol = JIF
CA= 0x666EE13f8b21ffC3Af675A56e1aE5Ddebda841e5

same comment as your other PR, please put logo in the folder not as a comment


Add a new SUSHI sapce with new voting calculation

@bartjman Hi! Can you use sushipowah as the key, folder name and css class for the old space config, and use sushi for the new config?


Nvidia Jetson Nano support

Feature request
Please add support for nvidia jetson nano so we can use it with hyperion

What problem does this feature solve?
This will add support for armv8 devices including nvidia jetson nano
What does the proposed API look like?
no api changes is needed
How should this be implemented in your opinion?
kernel level
Are you willing to work on this yourself?
no, i dont.

You could try to install Ubuntu 18.04 and install Hyperion (nightly) via our APT server.


fix scope for ScalaTest dependency

Moves ScalaTest to the test scope (ScalaTest is 7MB in size on its own).
If ScalaTest is not used for testing, this will actually lead to issues apart from bloat (for example with µTest, testOnly does no longer work, as the command is intercepted by ScalaTest). In this case, one can simply exclude the ScalaTest dependency; but this tiny PR fixes the underlying issue.
Thanks for this nice utility library!

Hi @amesgen ,
I'm truly sorry about the late response, I completely missed all the PRs in this repo!!!
Thank you for sending this PR!


Removed --dev option from composer install

Dev packages are installed by default. There is no need for this option.
See https://travis-ci.org/yiisoft/yii-base-web/jobs/546238056#L257

Merged. Thanks!


move blinding.py to own file and add secure_secret_path method

As much as I like this clever bip32 path encoding technique, I think secure_secret_path is more straightforward.
There isn't a great use case for "the bip32 path to your blinded key is encode_to_bip32path(sha256("correct horse battery staple")). At that point, you probably just want to reveal the full account map. Or if for some reason you just wanted to reveal a path but not the full account map, you could at that point reveal the bip32 path (no "correct horse battery staple" needed).

@jimmysong ok to merge?


GIF exporter: previous frames do not dispose on gifs with trnasparency

Hi! I'm trying to save animated gif composed from few bitmaps with transparency and it doesn't remove previous frame from animation. How can it be fixed?
Here is the result file:

My code:
`
var outputFrames = new List();
for (inti = 0; i <= animationLength; i++)
{
CanvasRenderTarget offscreen = await Export(settings, vp, i);
                IBuffer buff = offscreen.GetPixelBytes().AsBuffer();

                SoftwareBitmap bm = SoftwareBitmap.CreateCopyFromBuffer(buff, BitmapPixelFormat.Bgra8, (int)offscreen.SizeInPixels.Width, (int)offscreen.SizeInPixels.Height);
                SoftwareBitmapImageSource bs = new SoftwareBitmapImageSource(bm);

                outputFrames.Add(bs);
            }

            using (var gifRenderer = new GifRenderer())
            {
                gifRenderer.Duration = 100;
                gifRenderer.NumberOfAnimationLoops = 10000;
                gifRenderer.UseGlobalPalette = false;
                gifRenderer.ApplyDithering = false;
                gifRenderer.Sources = outputFrames.ToArray();

                var buffer = await gifRenderer.RenderAsync();
                using (var stream = await fileToSave.OpenAsync(FileAccessMode.ReadWrite))
                {
                    await stream.WriteAsync(buffer);
                }
            }

`

Hi,
Is it possible that you are using transparency in your frames? This is the result I would expect in case when you are only setting the pixels where your content (circle/ellipse in your case) appears, and leaving everything else unset - transparent.
You can either make sure that the API you are exporting the frames from has a background color, or you can blend in the background color with Lumia Imaging SDK before rendering.

Hi David! Yes I use transparency. When I don't everything is fine.
But I need to save animation With transparency. So I asked if there is any way to dispose previous frames?

No, not easily, that I'm aware off. I will investigate a bit and get back to you, but I think you will need to paint over the area in the previous frame with a background color, and then paint over your next frame.
I realize this is not ideal for your scenario...

In photoshop and Gimp there are option "dispose frame"
like here:
https://s3.amazonaws.com/satisfaction-production/s3_images/683741/disposal_inline.
and it allowed me to make:

may be there is a way to erase area from previous frame with transparent color?


labels on this repo?

What happened?

What were you attempting to do?

Attempting to find non-language family buildpacks on github by searching repos with the cnb and all-applications.

What did you expect to happen?

I expected this repo to show up.
Am I in correct or should this repo have the cnb and all-applications labels?

Thanks for pointing this out! I added the labels to the repo. Sorry About the slow response :)


Log number of bytes in/out for each request type on validators

This will help us understand which request is consuming the most network bandwidth.

@gdanezis @lxfind I just spoke with @bmwill on this, and based on our assessment, there isn't a way to record the bytes as metrics per route using the GRPC infra that we currently have. I don't see much value in having the bytes recorded among all routes but not filterable. I experimented with a different approach here where we record the memory usage of the struct that we send as requests / responses. However, that approach would give us at best a very rough estimation, and is both resource intensive computationally and hard to maintain.
Brandon's plans for our next generation networking stack enable tracking this kind of data in a much simpler way. I would propose delaying this until we have migrated away from GRPC, which may be sometime next month.


DJ SQL AST

Create a DJ SQL AST. 👀

This is done with https://github.com/DataJunction/dj/pull/242


Consider moving from Google Sheets to CSV

Google Sheets has no concept of pull requests, so we can't let people propose changes. While it's a convenient back-end, it's not very open. Also, it's slow. Without a loading indicator, people just assume that pages are blank and give up.
On the other hand, moving off of Tabletop.js is going to be a non-trivial change, and doing so may be counter to the spirit of the original project.

For my next use of the census, I want to crowdsource a bunch of reviews during a national day of action. We'll be asking people to score the quality of the websites used to apply for a handful of important social services.
I was considering using Google Forms as the input.
Pull requests make sense if there is only going to be occasional outside updates.

Hmm, good point—Google Forms certainly does lower the bar for submissions.
OTOH, it decouples the submission process from the maintenance process, which is to say there is an uncrossable chasm between "person who suggests new datasets" and "person who wants to help to maintain the data."
Now, I'm just thinking out loud here, but given that the crowdsourced data goes into a hopper for review anyway (IIRC, you created a worksheet just for submissions), I'm not sure that these two approaches are mutually exclusive. That is, submissions need to be reviewed and moved over to the worksheet that powers the site anyway. Perhaps, instead of moving them between Google Sheets worksheets, site operators would be moving them from a Google Sheets worksheet to an Excel worksheet (or whatever they're using to manage their Git-versioned CSV file)?
It seems like both of these approaches—letting people enter things into Google Forms and versioning data in Git—are both about the same thing: lowering the bar to contributing to a census. But I think they're about lowering the bar for very different types of users (non-devs vs. devs, editors vs. submitters), whose needs are perhaps incompatible?

I'm happy to use either version. For the next census, I want to use whatever this becomes.
To continue the discussion though, I still think Google Docs works. We don't need a chasm, the spreadsheet containing the submissions can be the same sheet powering the webpage. Two changes required. The first is we'd need to abstract out the questions. The second is that if there are more than one submission for a place and dataset, we use the most recent one.
You are right that there are different audiences, with either lots of input or limited. For yours, you have already done the work and there will be limited updates. For ours, the work will be done by lots and lots of volunteers submitting lots of updates.
We could make it overly complicated and have a flag that points to the datasource, either csv or spreadsheet. Actually Google Sheets lets you just add csv on to the end of the url and access it that way ...

I still think Google Docs works. We don't need a chasm, the spreadsheet containing the submissions can be the same sheet powering the webpage.

I think the chasm is between submitting information and updating information. I only had maybe a dozen datasets inventoried and submitted by people, but I had many dozen corrected by people. That is, I would inventory a state, conclude that a given dataset didn't exist, and then somebody would want to update that record to say "yes, that does exist, and here's where it is." I was surprised to find that happening, but on reflection, I think it's because people prefer to fix something that is broken than create something new. Anyway, the point is that Google Forms is great for submitting new things, but not great for correcting existing things.

We could make it overly complicated and have a flag that points to the datasource, either csv or spreadsheet. Actually Google Sheets lets you just add csv on to the end of the url and access it that way ...

Huh. This is...pretty clever, actually! It abstracts away some of these problems. Folks can host their CSV right on the site, or they can host it on Sheets. The more I think about it, the more I like it.
For the record, I should list a few of the other reasons why I favor of leaving Sheets (or, as you suggest, abstracting it away):

The Sheets API is not reliable. It has gone down before for many hours at a time, they could change it at any time (breaking any site using it), and Google rate-limits it, which would break a site just as it's getting a lot of attention.
It's slow. It just takes a long, long time—on the order of multiple seconds—for data to load, mostly because the Sheets API's response is so bloated.
Versioning of data is important. When adding or editing records in Sheets, I'm acutely aware that everything that I'm doing is changing the site in real-time, when what I'd really like is atomic updates (e.g., when I finish working on an inventoried datapoint, then I push out the changes).

I'm going to see if I can gin up some other folks who might be interested in using this package who have opinions about this. It'd be nice to get some other folks to weigh in here. This is an important architectural decision, and I have no particular reason to think that my position is a good one. :)

I think we chose/stuck with google sheets not as much for the public/private interface but because not everyone on the team knew the requisite amount of git and github to update a static file. However it looks like github has rolled out some nice in-browser csv editing tools that might have broader appeal.


Implementing counter of remaining bombs

Want to add some things to your repository, one of it counter of remaining bombs.
Can u check code (for correct usage, code-style) and approve to add these features
Reference (upper-left corner):

Could you please also attach a screenshot? I'm curious how it looks like?

Could you please also attach a screenshot? I'm curious how it looks like?

im thinking about how to do it right:

leave RestartButton at center, and place counter lefty from it
or make how i make it with Row, because in next iteration i'm also want to add a stopwatch

It would be nice to keep the reset button centered. Both counters can have weight=1 and horizontally centered text.

done, please check @arkivanov

It would be nice to keep the reset button centered. Both counters can have weight=1 and horizontally centered text.

not saw, this message, think to update for it, wait please:c

@arkivanov not so perfect, but its start looks like that

Thanks! I will add proper assets (images) a bit later and then we can update the design.

I've added digit icons, see #13.

I've added digit icons, see #13.

Yea, i've checked it out, it looks awesome.
When I tried this repo in the game and started implementing counter - I thought that it was good to add some fonts to stylise the app more as classic. Which approach is better (subjective, how do you think?):

to use images
or fonts
in some cases or for some things?

I'd also like to use images in this case because we use only [0-9-]:)

Since this project resembles the original Win9x Minesweeper design, it would be good to use the digit images. We can convert numbers to strings padded with zeros (taking special care for negative numbers), and then just through chars and render digit images.

Since this project resembles the original Win9x Minesweeper design, it would be good to use the digit images. We can convert numbers to strings padded with zeros (taking special care for negative numbers), and then just iterate through chars and render digit images.

Yea, I understand how to work in many ways, now trying to understand how implement it here as good as possible, first sketch looks like..

(trying to understand why Compose Images, increase their sizes like a square-shaped thing, even using .height instead of .size modifiers, but I think that I try to solve it myself)

I think we need to set width and height explicitly for every image. It should be the same as the actual size of the images.


Lock to Radon 3 0 1

Codecov Report

Merging #47 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master      #47   +/-   ##
=======================================
  Coverage   53.69%   53.69%
=======================================
  Files           5        5
  Lines         149      149
=======================================
  Hits           80       80
  Misses         69       69

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update fa2c5bb...e8e70f0. Read the comment docs.

@tisnik Your image is available in the registry: docker pull quay.io/openshiftio/rhel-fabric8-analytics-f8a-api-gateway:SNAPSHOT-PR-47


Adding TFv2 support for Faster-RCNN

Description
This PR introduces TensorFlow v2 support for the Faster-RCNN object detection model.
Includes:

TensorFlowV2FasterRCNN estimator
Tests for estimator in test_tensorflow_v2_faster_rcnn.py
Example notebook "estimator_tf2_faster_rcnn.ipynb" detailing installation and usage

Fixes # (issue)
#1616
Type of change
Please check all relevant options.

[x] Improvement (non-breaking)
[ ] Bug fix (non-breaking)
[ ] New feature (non-breaking)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update

Testing
Please describe the tests that you ran to verify your changes. Consider listing any relevant details of your test configuration.

[x] Test object detection
[x] Test compute gradients and losses
[x] Test errors thrown on invalid parameters (defences/clipping)

Test Configuration:

OS
Python version
ART version or commit number
TensorFlow / Keras / PyTorch / MXNet version

Checklist

[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[x] I have commented my code
[x] I have made corresponding changes to the documentation
[x] My changes generate no new warnings
[x] I have added tests that prove my fix is effective or that my feature works
[x] New and existing unit tests pass locally with my changes

Codecov Report

Merging #1951 (e6ee797) into dev_1.13.0 (c366b1d) will decrease coverage by 32.90%.
The diff coverage is 0.00%.

@@               Coverage Diff               @@
##           dev_1.13.0    #1951       +/-   ##
===============================================
- Coverage       54.87%   21.96%   -32.91%
===============================================
  Files             251      258        +7
  Lines           23464    24029      +565
  Branches         3942     4365      +423
===============================================
- Hits            12875     5278     -7597
- Misses           9829    18659     +8830
+ Partials          760       92      -668

Impacted Files
Coverage Δ

...tors/object_detection/tensorflow_v2_faster_rcnn.py
0.00% <0.00%> (ø)

art/evaluations/evaluation.py
0.00% <0.00%> (-100.00%)
:arrow_down:

art/evaluations/security_curve/__init__.py
0.00% <0.00%> (-100.00%)
:arrow_down:

art/attacks/poisoning/perturbations/__init__.py
0.00% <0.00%> (-100.00%)
:arrow_down:

...rmation/natural_corruptions/contrast/tensorflow.py
0.00% <0.00%> (-100.00%)
:arrow_down:

...mation/natural_corruptions/zoom_blur/tensorflow.py
0.00% <0.00%> (-93.55%)
:arrow_down:

...ation/natural_corruptions/shot_noise/tensorflow.py
0.00% <0.00%> (-90.00%)
:arrow_down:

...n/natural_corruptions/gaussian_noise/tensorflow.py
0.00% <0.00%> (-90.00%)
:arrow_down:

...cks/poisoning/perturbations/image_perturbations.py
0.00% <0.00%> (-89.75%)
:arrow_down:

art/evaluations/security_curve/security_curve.py
0.00% <0.00%> (-89.66%)
:arrow_down:

... and 149 more

Please create a similar file as https://github.com/Trusted-AI/adversarial-robustness-toolbox/blob/main/.github/workflows/ci-pytorch-object-detectors.yml to run the new tests.
Please update notebooks/README.md with the new notebook


fix(doc): Typo in aws-sns-lambda README.md

Fix typo and reference SnsToLambdaProps correctly.
Issue #, if available: N/A
Description of changes: Minor typo in the docs.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

AWS CodeBuild CI Report

CodeBuild project: codebuildgithubautobuildPro-fkVQbXRiQi6A
Commit ID: 3673faee176c7e8766fed34f857da0816006b9e0
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Thanks @pvbouwel !


Background and redesign

There should be a cool background image or even an interactive background. The color of the input containers will be adjusted accordingly.

Was closed with commit 7f41820


Update merchants.yml

https://www.reddit.com/r/Monero/comments/4ybnxw/good_news_you_can_now_buy_monero_xmr_with_gbp_uk/

closing this PR because it is a duplicate. See stefioan PR from 3 minutes prior


SHIP Proposal Process

Moving and codifying the enhancement proposal process in
shipwright-io/build so that it can be shared across all Shipwright
projects.
Relevant community documentaiton updated with links to the SHIP
guidelines.
Removed references to OKD and simplified the ship template.

/lgtm


Add GitHub Actions CI

This should add GitHub Actions CI for the Linux and macOS builds. Since TravisCI is no longer free for us, this should hopefully allow us to retire that support.

Here's a run from a test-branch: https://github.com/kusma/rocket/actions/runs/4251418879


Support latest version of OpenAi.rb

see here: https://github.com/alexrudall/ruby-openai/blob/main/CHANGELOG.md
The newest version has a breaking change:
Endpoints now return JSON rather than HTTParty objects. You will need to update your code to handle this change, changing JSON.parse(response.body)["key"] and response.parsed_response["key"] to just response["key"].

@bborn Did you mean to close this Issue?

@andreibondarev yeah, I thought it was broken but then realized it was something on my end.


BUG: Pinned-Dependencies assumes that Dockerfile commands can be parsed as sh

Describe the bug
Pinned-Dependencies parses commands in Dockerfiles using the sh (https://github.com/mvdan/sh) parser. However, Dockerfiles can use any shell they want using the SHELL statement.
In particular, this means that Dockerfiles designed for Windows targets (and Powershell) will often fail to parse.
Reproduction steps
Steps to reproduce the behavior:

Create a Dockerfile that uses a non-default SHELL which doesn't parse as sh (an example here)
Pinned-Dependencies might fail to parse it
Pinned-Dependencies: internal error: error parsing shell code: src/runtime-tools/win64/Dockerfile:1:18: & can only immediately follow a statement

Expected behavior
Pinned-Dependencies should not assume that all Dockerfiles contain sh-compatible commands, especially if a SHELL statement is present.

Just a heads-up that we've hit this in the Zeek project as well, in this Windows Dockerfile. In our case the complaint is:
Error: check runtime error: Pinned-Dependencies: internal error: error parsing shell code: ci/windows/Dockerfile:1:124: (( can only be used to open an arithmetic cmd

Similar for Powershell
https://github.com/dotnet/runtime/blob/79c021d65c280020246d1035b0e87ae36f2d36a9/eng/docker/libraries-sdk.windows.Dockerfile#L15
RUN & .\dotnet-install.ps1 -Channel $env:_DOTNET_INSTALL_CHANNEL -Quality daily -InstallDir 'C:/Program Files/dotnet'

In Powershell, I believe a leading ampersand is the "call operator": https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_operators?view=powershell-7.3&viewFallbackFrom=powershell-7.1#call-operator-
But I guess the fix is the same as for the top post -- don't assume the command is shell.

@afmarcum could you help set expectations on this? Perhaps it's changed "Pinned-Dependencies" to skip lines it can't parse.
Would love to be able to get a scorecard for dotnet/runtime, it's one of the most active repos on Github by their measures.

@danmoseley I believe this was fixed in #3515, which hasn't entered a release yet.
Running Scorecard with that PR, we get the following output for Pinned-Dependencies for dotnet/runtime:
{
  "details": [
    "Info: Possibly incomplete results: error parsing shell code: & can only immediately follow a statement: eng/docker/libraries-sdk.windows.Dockerfile:14",
    "Info: Possibly incomplete results: error parsing shell code: & can only immediately follow a statement: eng/docker/libraries-sdk.windows.Dockerfile:23",
    "Info: Possibly incomplete results: error parsing shell code: \"fi\" can only be used to end an if: eng/testing/RunnerTemplate.sh:0",
    # ... unpinned GitHub Actions ...
    # ... unpinned containers ...
    "Warn: containerImage not pinned by hash: eng/docker/libraries-sdk.windows.Dockerfile:4",
    # ... more unpinned things ...
    "Info:   0 out of   8 GitHub-owned GitHubAction dependencies pinned",
    "Info:   0 out of   1 third-party GitHubAction dependencies pinned",
    "Info:   0 out of  18 containerImage dependencies pinned",
    "Info:   0 out of   2 downloadThenRun dependencies pinned",
    "Info:   0 out of   2 npmCommand dependencies pinned"
  ],
  "score": 0,
  "reason": "dependency not pinned by hash detected -- score normalized to 0",
  "name": "Pinned-Dependencies",
  "documentation": {
    "url": "https://github.com/ossf/scorecard/blob/main/docs/checks.md#pinned-dependencies",
    "short": "Determines if the project has declared and pinned the dependencies of its build process."
  }
}

Note that the parsing errors on the Dockerfile do not stop Scorecard from detecting that it is unpinned.


External map resources

Feature Description
As we expand our library of available maps, we will need some external storage to avoid storing several large geoJSON files inside all Evidence projects.
This could be something like an S3 bucket where we maintain a collection of geoJSON files that can be imported by our map components. We could then accept contributions for new maps the community would like supported in Evidence.

A lot of the heavy lifting is already done by leaflet:
https://leafletjs.com
Including support for a lot of different base map providers:
https://leafletjs.com/plugins.html#basemap-providers

@nathan-gs great point - we're big fans of leaflet. We've got a simple marker map component set up in Evidence Labs using leaflet, and I think there's good potential to build more fulsome mapping features with leaflet:
https://labs.evidence.dev/leaflet-map/
There's also a set of choropleth maps we've so far been using ECharts for (https://labs.evidence.dev/maps)
Are there any map types/features you would want to see added to Evidence?

@hughess is this now resolved?

yes!


Accepting arguments over a UNIX pipe

I have a command like this, which will update data item 1, with some user-provided JSON
tool update-data 1 '{"name": "A New Name"}'

Sometimes, the JSON the user has to provide can get quite long, so it makes sense to provide it via a file. I'd like to do
tool update-data 1 < data.json

or
cat data.json | tool update-data 1

The way I get around this right now is by explicitly handling whether or not any values were provided to the optional second argument, and if not, trying to pull from stdin, like so:
    if data == "":
        if select.select([sys.stdin,],[],[],0.0)[0]:
            for line in sys.stdin:
                if not line:
                    break
                data += line
        else:
            click.echo("No data provided.")
            return

I'm not sure if it's out of scope for click to transparently accept stdin or a string to a last argument.

Yeah I think this is out of scope... You probably want to write a custom converter that takes either a filename, JSON, or - for stdin. Click's File can deal with half of it.


flutter run  : appear:Could not download bcprov-jdk15on.jar

Steps to Reproduce

...flutter create my_first_app
...flutter  run

Logs

Could not resolve all artifacts for configuration ':classpath'.
   > Could not download bcprov-jdk15on.jar (org.bouncycastle:bcprov-jdk15on:1.56)
      > Could not get resource 'https://jcenter.bintray.com/org/bouncycastle/bcprov-jdk15on/1.56/bcprov-jdk15on-1.56.jar'.
         > Could not GET 'https://jcenter.bintray.com/org/bouncycastle/bcprov-jdk15on/1.56/bcprov-jdk15on-1.56.jar'.
> Connect to d29vzk4ow07wi7.cloudfront.net:443 [d29vzk4ow07wi7.cloudfront.net/<IP_ADDRESS>, d29vzk4ow07wi7.cloudfront.net/<IP_ADDRESS>, d29vzk4ow07wi7.cloudfront.net/<IP_ADDRESS>, d29vzk4ow07wi7.cloudfront.net/<IP_ADDRESS>] failed: Read timed out
   > Could not download fastutil.jar (it.unimi.dsi:fastutil:7.2.0)
      > Could not get resource 'https://jcenter.bintray.com/it/unimi/dsi/fastutil/7.2.0/fastutil-7.2.0.jar'.
         > Could not GET 'https://jcenter.bintray.com/it/unimi/dsi/fastutil/7.2.0/fastutil-7.2.0.jar'.
            > Connect to d29vzk4ow07wi7.cloudfront.net:443 [d29vzk4ow07wi7.cloudfront.net/<IP_ADDRESS>, d29vzk4ow07wi7.cloudfront.net/<IP_ADDRESS>, d29vzk4ow07wi7.cloudfront.net/<IP_ADDRESS>, d29vzk4ow07wi7.cloudfront.net/<IP_ADDRESS>] failed: Read timed out
   > Could not download kotlin-reflect.jar (org.jetbrains.kotlin:kotlin-reflect:1.2.0)
      > Could not get resource 'https://jcenter.bintray.com/org/jetbrains/kotlin/kotlin-reflect/1.2.0/kotlin-reflect-1.2.0.jar'.
         > Could not GET 'https://jcenter.bintray.com/org/jetbrains/kotlin/kotlin-reflect/1.2.0/kotlin-reflect-1.2.0.jar'.
            > Connect to d29vzk4ow07wi7.cloudfront.net:443 [d29vzk4ow07wi7.cloudfront.net/<IP_ADDRESS>, d29vzk4ow07wi7.cloudfront.net/<IP_ADDRESS>, d29vzk4ow07wi7.cloudfront.net/<IP_ADDRESS>, d29vzk4ow07wi7.cloudfront.net/<IP_ADDRESS>] failed: Read timed out
> Could not download auto-value.jar (com.google.auto.value:auto-value:1.5.2)
      > Could not get resource 'https://jcenter.bintray.com/com/google/auto/value/auto-value/1.5.2/auto-value-1.5.2.jar'.
         > Could not GET 'https://jcenter.bintray.com/com/google/auto/value/auto-value/1.5.2/auto-value-1.5.2.jar'.
            > Connect to d29vzk4ow07wi7.cloudfront.net:443 [d29vzk4ow07wi7.cloudfront.net/<IP_ADDRESS>, d29vzk4ow07wi7.cloudfront.net/<IP_ADDRESS>, d29vzk4ow07wi7.cloudfront.net/<IP_ADDRESS>, d29vzk4ow07wi7.cloudfront.net/<IP_ADDRESS>] failed: Read timed out
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
BUILD FAILED in 2m 4s
  Command: E:\flutter\second_app\android\gradlew.bat app:properties

Doctor summary (to see all details, run flutter doctor -v):
[√] Flutter (Channel beta, v1.0.0, on Microsoft Windows [Version 10.0.17134.407], locale zh-CN)
[√] Android toolchain - develop for Android devices (Android SDK 28.0.3)
[√] Android Studio (version 3.2)
    X Flutter plugin not installed; this adds Flutter specific functionality.
    X Dart plugin not installed; this adds Dart specific functionality.
[√] Connected device (1 available)
• No issues found!

OS:windows10 ,
Flutter 1.0.0 • channel beta • https://github.com/flutter/flutter.git
Framework • revision 5391447fae (8 days ago) • 2018-11-29 19:41:26 -0800
Engine • revision 7375a0f414
Tools • Dart 2.1.0 (build 2.1.0-dev.9.4 f9ebf21297)

This problem has plagued me for a long time. I hope you can help me.

try to add mavenCentral() ,but no effect

Please add the output of flutter doctor -v.

Do you see this also with a newly created app (flutter create foo) or only with an existing project?

C:\Windows\system32>flutter doctor -v
[√] Flutter (Channel beta, v1.0.0, on Microsoft Windows [Version 10.0.17134.407], locale zh-CN)
• Flutter version 1.0.0 at D:\fluttersdk\flutter
• Framework revision 5391447fae (10 days ago), 2018-11-29 19:41:26 -0800
• Engine revision 7375a0f414
• Dart version 2.1.0 (build 2.1.0-dev.9.4 f9ebf21297)
[√] Android toolchain - develop for Android devices (Android SDK 28.0.3)
• Android SDK at D:\androidSdk
• Android NDK location not configured (optional; useful for native profiling support)
• Platform android-28, build-tools 28.0.3
• ANDROID_HOME = D:\androidSdk
• Java binary at: D:\AndroidStudio\jre\bin\java
• Java version OpenJDK Runtime Environment (build 1.8.0_152-release-1136-b06)
• All Android licenses accepted.
[√] Android Studio (version 3.2)
• Android Studio at D:\AndroidStudio
X Flutter plugin not installed; this adds Flutter specific functionality.
X Dart plugin not installed; this adds Dart specific functionality.
• Java version OpenJDK Runtime Environment (build 1.8.0_152-release-1136-b06)
[√] Connected device (1 available)
• OPPO A33 • af4cc1 • android-arm64 • Android 5.1.1 (API 22)

After creating a new project, there will be no such problems, but before I tried to create a few, there are such problems. I guess it should be a network problem because I am in China.

@zhuiye
Thanks for the update.
Could be https://github.com/flutter/flutter/wiki/Workarounds-for-common-issues#generated-project-files-outdated


Display radar sites on the map and allow radar site selection by double clicking labels

As a user, I should be able to view all of the radar sites as labels on the map. Double clicking a radar's label should switch my current radar site to the site that was clicked.

Pull request #97 has been opened. This adds the "Radar Sites" layer, which you can enable from the Layer Manager.

This will be in the next release. You can preview it here, and on any future builds of the develop branch:
https://github.com/dpaulat/supercell-wx/actions/runs/6938893074


fixed a problem where pinch-to-zoom should scale around the pinch point

Currently when you pinch-to-zoom the Android implementation, it always scale around the top left corner instead of the pinch point.
This PR sets the pivot point for scaling to where the pinch started.

@oscarkuo Thanks for the PR - it's a great start. I noticed though that if the scroll offset is (0,0) then after you zoom the viewport isn't correctly aligned with the content (so you either get extra white space, or content is cut off and can't be scrolled to). One solution might be to do an animated realignment when ACTION_UP fires. Want to have a go at this? Or I may be able to take a look when I get some more time 😄

@chrisfisher thanks for the feedback, by "scroll offset", do you mean if you use the scrollTo method then there will be extra whitespace?
I'll give it a go since I can do this during work time :)

@oscarkuo The scroll offset is (0,0) when you're in the top left corner (i.e. the starting point). If you pinch to zoom here in the example then you either get extra white space (zooming in) or content is cut off (zooming out).


Wiki: Add few missing dependencies and other useful stuff for building

some-username-here1/ALVR@db764b75db1bd048203ae0787d909edb03943c74: Add a few missing dependency packages for Ubuntu (Ubuntu renamed to "Debian / Ubuntu / Pop_OS") under "Server build"; Add subsection under "Client build" for installing android developer packages for various Linux flavors from upstream and adding Linux directory examples for environment variables.

The different directories listed for Arch and Gentoo for the ANDROID_SDK_ROOT environment variable is based off of each wiki's respective pages, 1 and 2, so apologies if I got the directory wrong. The packages for all mentioned flavors were also looked up beforehand, comparing to and from, but if anyone who has those flavors would like to make changes or fixes to the pull, feel free!

I'm handing off the review to @m00nwtchr since I'm not qualified enough.

I have updated the rest of the wiki a bit with the text formatting and fixed the weird "Basic-" substring that appeared everywhere as a prefix to any "installation" mentioned, anything else that needs to be changed?

There is an issue with titles with the wiki sync workflow. It seems it gets confused when the name of the page does not correspond with the topmost title within the page. Can you please go back in history and search what where the original filenames and titles of the pages? Also can you correct this in a separate PR? So here we can only discuss the building from source page.

The wiki has been fixed by renaming the Installation page. Can you please rebase this PR?

Undo the fixes for the other pages and placed into its own branch/PR, then rebase?

The fix for other pages was already done. Rebase this PR just for the build from source page.

Remove the 'openxr-client' Branch from the building page, too? It has been merged a few days ago

Sorry for the late response, but how are these changes?

You don't need to put newlines between bullet point. I'm not sure if this breaks the rendering of the lists. Anyway please remove those spaces if you can do that soon or I'll merge and fix it later.

Done!

I think it's good. thanks for your patience!


More info

The backend for the Demo is a separate Java Springboot application (with API documentation)....

Done


background Sync does not work on production envirnoment

Hi,
i am using below skeleton for PWA
https://codelabs.developers.google.com/codelabs/workbox-indexeddb/
Library Affected:
//storage.googleapis.com/workbox-cdn/releases/3.6.3/workbox-sw.js
Browser & Platform:
E.g. Version 72.0.3626.96 (Official Build) (64-bit) for chrome and server  Ubuntu 12.04.4 LTS and local environment is win 8.1.
Issue or Feature Request Description:
i have below codebase which works as expected in local but does not work in prod environment
meaning index db "workbox-sync-" is not getting created
case sceanrio:

run npm run start
cltr+c to stop webserver
enter details on page and click on submit

Working screenshot

Not Working screenshot

my sw.js
var startDate = new Date();

self.addEventListener("install", function(event) {
  self.skipWaiting();
});

self.addEventListener("activate", function(event) {
  if (self.clients && clients.claim) {
    clients.claim();
  }
});

self.addEventListener("sync", function(event) {
  console.log("firing: sync" + event.tag);
  //workbox-background-sync:
  if (event.tag == "workbox-background-sync:dashboardr-queue") {
    console.log("sync event fired");
    event.waitUntil(fetchGeneric());
  }
});

function fetchGeneric() {
  console.log("firing: doSomeStuff()");
  var endDate = new Date();
  var seconds = (endDate.getTime() - startDate.getTime()) / 1000;
  console.log(seconds);
}

importScripts(
  "//storage.googleapis.com/workbox-cdn/releases/3.6.3/workbox-sw.js"
);

if (workbox) {
  console.log(`Yay! Workbox is loaded 🎉`);
  workbox.setConfig({ debug: true });
  workbox.core.setLogLevel(workbox.core.LOG_LEVELS.debug);
  const showNotification = () => {
    self.registration.showNotification("Background sync success!", {
      body: "🎉`🎉`🎉`"
    });
  };
  //const bgSyncPlugin = new workbox.backgroundSync.Plugin('dashboardr-queue');
  const bgSyncPlugin = new workbox.backgroundSync.Plugin(
    "dashboardr-queue",
    {
      maxRetentionTime: 24 * 60 // Retry for max of 24 Hours
    },
    {
      callbacks: {
        queueDidReplay: storableRequest => {
          console.log(storableRequest);
          // ...
        },
        requestWillEnqueue: storableRequest => {
          console.log(storableRequest);
          // ...
        },
        requestWillReplay: storableRequest => {
          console.log(storableRequest);
        }
        // other types of callbacks could go here
      }
    }
  );

  const networkWithBackgroundSync = new workbox.strategies.NetworkFirst({
    plugins: [bgSyncPlugin]
  });

  workbox.routing.registerRoute(
    /\/api\/add/,
    networkWithBackgroundSync,
    "POST"
  );

  workbox.precaching.precacheAndRoute([]);
} else {
  console.log(`Boo! Workbox didn't load 😬`);
}

and my main.js
if ("serviceWorker" in navigator) {
  window.addEventListener("load", () => {
    navigator.serviceWorker
      .register("/sw.js")
      .then(registration => {
        console.log(`Service Worker registed! Scope: ${registration.scope}`);
        document
          .getElementById("add-event-button")
          .addEventListener("click", () => {
            registration.sync
              .register("data-fetch")
              .then(() => {
                console.log("sync registered");
                startDate = new Date();
              })
              .catch(function(error) {
                console.log("Unable to fetch image.");
              });
          });
      })
      .catch(err => {
        console.log(`Service Worker registration failed: ${err}`);
      });
  });
}

const container = document.getElementById("container");
const offlineMessage = document.getElementById("offline");
const noDataMessage = document.getElementById("no-data");
const dataSavedMessage = document.getElementById("data-saved");
const saveErrorMessage = document.getElementById("save-error");
const addEventButton = document.getElementById("add-event-button");

addEventButton.addEventListener("click", addAndPostEvent);

Notification.requestPermission();

const dbPromise = createIndexedDB();

loadContentNetworkFirst();

function loadContentNetworkFirst() {
  getServerData()
    .then(dataFromNetwork => {
      updateUI(dataFromNetwork);
      saveEventDataLocally(dataFromNetwork)
        .then(() => {
          setLastUpdated(new Date());
          messageDataSaved();
        })
        .catch(err => {
          messageSaveError();
          console.warn(err);
        });
    })
    .catch(err => {
      console.log("Network requests have failed, this is expected if offline");
      getLocalEventData().then(offlineData => {
        if (!offlineData.length) {
          messageNoData();
        } else {
          messageOffline();
          updateUI(offlineData);
        }
      });
    });
}

/* Network functions */

function getServerData() {
  return fetch("api/getAll").then(response => {
    if (!response.ok) {
      throw Error(response.statusText);
    }
    return response.json();
  });
}

function addAndPostEvent(e) {
  e.preventDefault();
  var id_val = Date.now();
  const data = {
    id: id_val,
    first_name: document.getElementById("first_name").value,
    last_name: document.getElementById("last_name").value,
    company: document.getElementById("company").value,
    phone: document.getElementById("phone").value,
    email: document.getElementById("email").value,
    street: document.getElementById("street").value,
    city: document.getElementById("city").value,
    state: document.getElementById("state").value,
    country: document.getElementById("country").value,
    lead_source: document.getElementById("lead_source").value,
    marketing_rep_name: document.getElementById("marketing_rep_name").value
  };
  updateUI([data]);
  saveEventDataLocally([data]);
  const headers = new Headers({ "Content-Type": "application/json" });
  const body = JSON.stringify(data);
  return fetch("api/add", {
    method: "POST",
    headers: headers,
    body: body
  }).catch(error => console.error("Error:", error));
}

/* UI functions */

function updateUI(events) {
  events.forEach(event => {
    const item = `

${event.first_name} ${event.last_name}

${event.company}

${event.email}

${event.phone}

${event.street}, ${event.city}, ${event.state}, ${
      event.country
    }

Lead Source: ${event.lead_source}

Marketing Rep: ${event.marketing_rep_name}

`;
    container.insertAdjacentHTML("beforeend", item);
  });
}

function messageOffline() {
  // alert user that data may not be current
  const lastUpdated = getLastUpdated();
  if (lastUpdated) {
    offlineMessage.textContent += " Last fetched server data: " + lastUpdated;
  }
  offlineMessage.style.display = "block";
}

function messageNoData() {
  // alert user that there is no data available
  noDataMessage.style.display = "block";
}

function messageDataSaved() {
  // alert user that data has been saved for offline
  const lastUpdated = getLastUpdated();
  if (lastUpdated) {
    dataSavedMessage.textContent += " on " + lastUpdated;
  }
  dataSavedMessage.style.display = "block";
}

function messageSaveError() {
  // alert user that data couldn't be saved offline
  saveErrorMessage.style.display = "block";
}

/* Storage functions */

function getLastUpdated() {
  return localStorage.getItem("lastUpdated");
}

function setLastUpdated(date) {
  localStorage.setItem("lastUpdated", date);
}

function createIndexedDB() {
  if (!("indexedDB" in window)) {
    return null;
  }
  return idb.open("dashboardr", 1, function(upgradeDb) {
    if (!upgradeDb.objectStoreNames.contains("events")) {
      const eventsOS = upgradeDb.createObjectStore("events", { keyPath: "id" });
    }
  });
}

function saveEventDataLocally(events) {
  if (!("indexedDB" in window)) {
    return null;
  }
  return dbPromise.then(db => {
    const tx = db.transaction("events", "readwrite");
    const store = tx.objectStore("events");
    console.log(tx);
    console.log(store);
    return Promise.all(events.map(event => store.put(event))).catch(() => {
      tx.abort();
      throw Error("Events were not added to the store");
    });
  });
}

function getLocalEventData() {
  if (!("indexedDB" in window)) {
    return null;
  }
  return dbPromise.then(db => {
    const tx = db.transaction("events", "readonly");
    const store = tx.objectStore("events");
    return store.getAll();
  });
}

@matt212 have you tried testing your background sync code in the workbox v4 release candidate? We did a fairly major rewrite of the workbox-background-sync API in v4, so I think the first step is to test it in v4 and then go from there.
Pull request https://github.com/GoogleChrome/workbox/pull/1710 has a description of what's changed, let me know if you have any issues upgrading.

@philipwalton  , thanks for update , i'll commence the v4 changes in my existing code and provide with an update accordingly ! :+1:

@philipwalton ,
i am getting below error when i am using v4

any pointers on same
below is my updated
sw.js
importScripts("./js/workbox/workbox-sw.js");
if (workbox) {
  const queue = new workbox.backgroundSync.Queue("dashboardr-queue");

  const networkWithBackgroundSync = new workbox.strategies.NetworkOnly({
    plugins: [
      {
        fetchDidFail: async ({ request }) => {
          await queue.addRequest(request);
        }
      }
    ]
  });

  const addEventRoute = new workbox.routing.Route(
    ({ url }) => url.pathname === "/api/add",
    networkWithBackgroundSync,
    "POST"
  );

  workbox.routing.registerRoute(addEventRoute);
  workbox.precaching.precacheAndRoute([]);
} else {
  console.log(`Boo! Workbox didn't load 😬`);
}

I think I see the issue. The addRequest method has been removed in favor of push/pop/shift/unshift variants.
Try changing your plugin code to this:
fetchDidFail: async ({ request }) => {
  await queue.pushRequest({ request });
}

hey @philipwalton ,
thanks now background Sync is getting created but not records in it
Please find below screenshot

and code snippets for the same .
my updated sw.js
var startDate = new Date();

// Do your operations

self.addEventListener("install", function(event) {
  self.skipWaiting();
});

self.addEventListener("activate", function(event) {
  if (self.clients && clients.claim) {
    clients.claim();
  }
});

self.addEventListener("sync", function(event) {
  console.log("firing: sync" + event.tag);
  if (event.tag == "workbox-background-sync:dashboardr-queue") {
    console.log("sync event fired");
    event.waitUntil(fetchGeneric());
  }
});

function fetchGeneric() {
  console.log("firing: doSomeStuff()");
  var endDate = new Date();
  var seconds = (endDate.getTime() - startDate.getTime()) / 1000;
  console.log(seconds);
}

importScripts("./js/workbox/workbox-sw.js");
if (workbox) {
  const queue = new workbox.backgroundSync.Queue("dashboardr-queue");

  const networkWithBackgroundSync = new workbox.strategies.NetworkOnly({
    plugins: [
      {
        fetchDidFail: async ({ request }) => {
          console.log(request);
          await queue.pushRequest({ request });
        }
      }
    ]
  });

  const addEventRoute = new workbox.routing.Route(
    ({ url }) => url.pathname === "/api/add",
    networkWithBackgroundSync,
    "POST"
  );

  workbox.routing.registerRoute(addEventRoute);
  workbox.precaching.precacheAndRoute([]);
} else {
  console.log(`Boo! Workbox didn't load 😬`);
}

I am also experiencing a case where background sync works locally but does not work when deployed using workbox 4.  Even when I see that there are records sitting in indexedDB waiting to be synced and I go to the dev tools to force a sync event, no background sync happens with my deployed PWA.

@matt212  are you able to get it fixed ? I am also facing same issue.
-Ramesh

@philipwalton any update on this?

@firrae can you describe what problem you're experiencing? I think this particular issue thread is a bit scattered—it spans multiple versions of Workbox, and many people have reported the problem going away (which suggests it's a Chrome issue more than a Workbox issue).
Without a clear way to reproduce the problem, it's hard for me to know how to help, so I think opening a new issue with a repro would be best (if you can).

@philipwalton I agree this is something that has gotten scattered but it was the only issue that happened on to have the (hopefully) temporary solution that has allowed our early adaptors to sync their offline work (@klcantrell's answer update). Though we haven't had a verification with our client yet that it continues to update after that as the device has yet to sync automatically over multiple attempts. It is not in our hands and we cannot replicate it locally so far. Our client also doesn't use the device solely for our application.
We verified a number of things:

The device is on Chrome 78 on Android
Closing all tabs and opening a new one to the app does cause a sync of their previous work
When they are attempting to sync they have an internet connection (they have video called us to show us it happening)
Device is a Samsung Galaxy Tab A (I believe 2015 but I can double check with our account rep for them)

Their workflow is as follows:

log into our application to get the latest updates and cache the necessary info
travel to their worksite, going offline in the process
do their work
return to their main office
reconnect to wifi
wait upwards of 10 minutes to not have it sync (we have waited longer periods in the past as well)

Looking through the code it seems we are ending up with it hitting the fallback of sync whenever it restarts, but as I said above we do not see this on other physical devices. And having done multiple video calls with them we don't believe its a setting on their device (we had them update Chrome from an older version to 78 as the initial attempt at fixing it).

Closing this as stale


Selenium Grid

Integrate Selenium Grid into the project. It should be possible to define if Selenium Grid should be used or not and automatically register each device Appium server in the grid

Done with #33


Feature request: Download specific courses in a row

Would it be possible to implement a way with the "-c" option to specify multiple courses to download in a row?
So e.g.  ine.py -c <Course ID1>, <Course ID2>
Thanks

It is easy to implement but will have to change a lot of code, currently, you can only download a category with multiple processes with -p argument, if that works for you.
I can add it to the backlog but can't commit when it'll be implemented.


[Feature][Agent] Support CloudEvents Source

Description
CloudEvents is a specification for describing event data in common formats to provide interoperability across services, platforms and systems.
https://github.com/cloudevents/spec
Use case
Kafka Example:
https://github.com/cloudevents/sdk-java/tree/master/examples/kafka
Are you willing to submit PR?

[ ] Yes, I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

Motivation
Add the collecting CloudEvents data function to agent module and provide the necessary management ability for manager module.
Changes

An agent module plugin with Reader and Source added
Design

This PR following the document Agent Plugin

read CloudEvents data in reader
create CloudEvents reader and add it to source
add Enum type in manager class

Implementation

Add the ability of collecting data from CloudEvents

CloudEventsReader: read data from CloudEvents by Debezium
CloudEventsSource: implement split method

Add the management ability for manager module

add type in TaskTypeEnum

Add Unit test for CloudEventsReader

CloudEventsReaderTest: test CloudEventsReader's functional integrity


[logging] Add metric for total bytes sent via logging

Overview
Through hardening, we wanted to add a metric for keeping track of bytes sent through the logging system.  This adds that metric, and in addition deletes an empty file that somehow was still around?
Related Issue
https://github.com/libra/libra/issues/6196

/land


Less output from optional chaining and null coalescing operator

TypeScript Version:  3.8.0-dev.20191023
Search Terms
null coalescing, optional chaining, strict equality operator, loose equality operator, double equal, triple equal
Suggestion
Currently
declare var foo: any;
foo ?? 1;
foo?.bar;

compiles to
var _a;
(foo !== null && foo !== void 0 ? foo : 1);
(_a = foo) === null || _a === void 0 ? void 0 : _a.bar;

This results in redundant !== null && foo !== void 0 and (_a = foo) === null || _a === void 0 output, strict equality/inequality checks cannot be optimized further by existing JavaScript minifiers (UglifyJS, Terser, Babel, etc.)
The suggestion is to use == null standard equality check instead of === null and === void 0 strict equality checks to produce less output.
Use Cases
Since there aren't supposed to be runtime helpers (#33933) for ?? and ?. operators, this may appreciably reduce the footprint, depending on the source code.
Examples
declare var foo: any;
foo ?? 1;
foo?.bar;

compiles to
var _a;
(foo != null ? foo : 1);
(_a = foo) == null ? void 0 : _a.bar;

Checklist
My suggestion meets these guidelines:

[x] This wouldn't be a breaking change in existing TypeScript/JavaScript code
[x] This wouldn't change the runtime behavior of existing JavaScript code
[x] This could be implemented without emitting different JS based on the types of the expressions
[x] This isn't a runtime feature (e.g. library functionality, non-ECMAScript syntax with JavaScript output, etc.)
[x] This feature would agree with the rest of TypeScript's Design Goals.

Is not valid to check with == null because, in a browser:
document.all == null // is true
document.all === null || document.all === undefined // is false

The reason that document.all has this weird behavior is that the developers of the current browsers want tests for its existence to fail, so that the code doing the testing will use a newer, better alternative. For historical reasons, there's lots of old code out there like this.
if (document.all) {
  // assume we're on old IE and use document.all
} else {
  // use better alternatives here
}

When browsers added support for document.all in order to "unbreak" websites that used it without checking for its existence, they didn't want to regress the behavior of websites that were smart enough to check for it and avoid using it. So, they designed it to pretend not to exist.
Given that, I think it would actually be better for TS to use the simpler == null test both for this and for optional chaining (?.). Hopefully no one would write new code that does such old testing, but if they did, the simpler == null test would give the more desirable behavior.

For those who still want this to happen: I wrote a little transformer that hooks into the compilation and changes the output to the compact version using loose equality comparison: https://www.npmjs.com/package/ts-transform-compact-nullish-compare

For those who still want this to happen: I wrote a little transformer that hooks into the compilation and changes the output to the compact version using loose equality comparison: https://www.npmjs.com/package/ts-transform-compact-nullish-compare


add gcp env check for testing

DO NOT MERGE JUST FOR TESTING

****************************************************************

libgrpc.so

     VM SIZE                                                                                FILE SIZE
 ++++++++++++++ GROWING                                                                        ++++++++++++++

 -------------- SHRINKING                                                                      --------------
  -0.0%     -32 [None]                                                                         -3.94Ki  -0.0%
      -0.0%      -8 [Unmapped]                                                                     -3.95Ki  -0.0%
      -2.2%     -24 [None]                                                                               0  [ = ]
      [DEL]      -8 g_polling_mu                                                                         0  [ = ]
 -24.2%    -896 src/core/lib/security/credentials/google_default/google_default_credentials.cc    -896 -24.2%
     -39.6%    -696 grpc_google_default_credentials_create                                            -696 -39.6%
      [DEL]    -204 on_compute_engine_detection_http_response                                         -204  [DEL]
      [DEL]      -5 destroy_pollset                                                                     -5  [DEL]

  -0.1%    -928 TOTAL                                                                          -4.81Ki  -0.1%

****************************************************************

libgrpc++.so

     VM SIZE        FILE SIZE
 ++++++++++++++  ++++++++++++++

  [ = ]       0        0  [ = ]

[trickle] No significant performance differences

[microbenchmarks] No significant performance differences

****************************************************************

libgrpc.so

     VM SIZE                                                                                FILE SIZE
 ++++++++++++++ GROWING                                                                        ++++++++++++++

 -------------- SHRINKING                                                                      --------------
  -0.0%     -32 [None]                                                                         -3.94Ki  -0.0%
      -0.0%      -8 [Unmapped]                                                                     -3.95Ki  -0.0%
      -2.2%     -24 [None]                                                                               0  [ = ]
      [DEL]      -8 g_polling_mu                                                                         0  [ = ]
 -24.2%    -896 src/core/lib/security/credentials/google_default/google_default_credentials.cc    -896 -24.2%
     -39.6%    -696 grpc_google_default_credentials_create                                            -696 -39.6%
      [DEL]    -204 on_compute_engine_detection_http_response                                         -204  [DEL]
      [DEL]      -5 destroy_pollset                                                                     -5  [DEL]

  -0.1%    -928 TOTAL                                                                          -4.81Ki  -0.1%

****************************************************************

libgrpc++.so

     VM SIZE        FILE SIZE
 ++++++++++++++  ++++++++++++++

  [ = ]       0        0  [ = ]

[trickle] No significant performance differences

[microbenchmarks] No significant performance differences

****************************************************************

libgrpc.so

     VM SIZE                                                                                FILE SIZE
 ++++++++++++++ GROWING                                                                        ++++++++++++++

 -------------- SHRINKING                                                                      --------------
  -0.0%     -32 [None]                                                                         -3.94Ki  -0.0%
      -0.0%      -8 [Unmapped]                                                                     -3.95Ki  -0.0%
      -2.2%     -24 [None]                                                                               0  [ = ]
      [DEL]      -8 g_polling_mu                                                                         0  [ = ]
 -24.2%    -896 src/core/lib/security/credentials/google_default/google_default_credentials.cc    -896 -24.2%
     -39.6%    -696 grpc_google_default_credentials_create                                            -696 -39.6%
      [DEL]    -204 on_compute_engine_detection_http_response                                         -204  [DEL]
      [DEL]      -5 destroy_pollset                                                                     -5  [DEL]

  -0.1%    -928 TOTAL                                                                          -4.81Ki  -0.1%

****************************************************************

libgrpc++.so

     VM SIZE        FILE SIZE
 ++++++++++++++  ++++++++++++++

  [ = ]       0        0  [ = ]

[trickle] No significant performance differences

[microbenchmarks] No significant performance differences

****************************************************************

libgrpc.so

     VM SIZE        FILE SIZE
 ++++++++++++++  ++++++++++++++

  [ = ]       0        0  [ = ]

****************************************************************

libgrpc++.so

     VM SIZE        FILE SIZE
 ++++++++++++++  ++++++++++++++

  [ = ]       0        0  [ = ]

[trickle] No significant performance differences

[microbenchmarks] No significant performance differences


Use new CI image

use CI image 2.4.0
add new flags for GCC 13

Coverage remained the same at 100.0% when pulling f93d1e771fd800dcc6125824c7aa1a9db8844b75 on new_ci into 48a102c2c5ef7c2552ea291f5527a8481c98a7a7 on develop.

@falbrechtskirchinger PTAL - 32 bit support is now also enabled.

Welp. I hoped you'd merge one of my 32bit PRs into this one first before I'd do a final review.
The addition of readability-simplify-boolean-expr triggers a bunch of errors on my system with a recent Clang 15 build. Specifically in HEDLEY_(UN)LIKELY.

Oh, sorry. But readability-simplify-boolean-expr is excluded. In this PR.

Oh, sorry. But readability-simplify-boolean-expr is excluded. In this PR.

Oh, good. I just quickly glanced at it and misinterpreted the plus sign from the diff. I'll see if there's anything else.

I wished Clang-Tidy was more flexible and one could define a list of sources that you care about and treat warnings outside these files as out of scope.  But not if it's the consequence of piling up calls from different libraries...

I wished Clang-Tidy was more flexible and one could define a list of sources that you care about and treat warnings outside these files as out of scope.  But not if it's the consequence of piling up calls from different libraries...

Agreed.
Everything else looks mostly good. The bool() cast could probably have been removed entirely. std::conditional doesn't do that, I'm sure.


⚠ Secret Website 3 has degraded performance

In 5eee2d5, Secret Website 3 ($SECRET_WEBSITE_3) experienced degraded performance:

HTTP code: 200
Response time: 2559 ms

Resolved: Secret Website 3 performance has improved in b07efdc.


Error when trying to add a staticwebapp.config.json

I have a angular application that I'm deploying with the AzureStaticWebApp build task. When I add a "staticwebapp.config.json" to the  angular app, the build task says it finds the file and then proceeds to just exit out after. If I remove the config file the build works fine.
Here's my staticwebapp.config.json
{
  "navigationFallback": {
    "rewrite": "/index.html",
    "exclude": ["*.{css,scss,js,png,gif,ico,jpg,svg"]
  }
}

Here is the error from the build pipeline

Verbose logging enabled
Build timeout not specified, defaulting to 15 minutes
App Directory Location: '/dist/dispatch-ui' was found.
No Api directory specified. Azure Functions will not be created.
Looking for event info
Event info parsed from action options.
Skipping step to build /working_dir/dist/dispatch-ui with Oryx
Didn't find Oryx manifest file under location: /ss-oryx/app-manifest/oryx-manifest.toml
Found staticwebapp.config.json file: 'dist/dispatch-ui/assets/staticwebapp.config.json'
Determined default file to be: index.html
Using 'staticwebapp.config.json' file for configuration information, 'routes.json' will be ignored.

For further information, please visit the Azure Static Web Apps documentation at https://docs.microsoft.com/en-us/azure/static-web-apps/
If you believe this behavior is unexpected, please raise a GitHub issue at https://github.com/azure/static-web-apps/issues/
Exiting
##[debug]Exit code 1 received from tool '/usr/bin/bash'
##[debug]STDIO streams have closed for tool '/usr/bin/bash'
##[debug]task result: Failed
##[error]Error: The process '/usr/bin/bash' failed with exit code 1
##[debug]Processed: ##vso[task.issue type=error;]Error: The process '/usr/bin/bash' failed with exit code 1
##[debug]Processed: ##vso[task.complete result=Failed;]Error: The process '/usr/bin/bash' failed with exit code 1
Finishing: AzureStaticWebApp

Here is my build YAML

# Node.js with Angular
# Build a Node.js project that uses Angular.
# Add steps that analyze code, save build artifacts, deploy, and more:
# https://docs.microsoft.com/azure/devops/pipelines/languages/javascript

trigger:
  - main

pool:
  vmImage: ubuntu-latest
steps:
- task: NodeTool@0
  inputs:
    versionSpec: '16.x'
  displayName: 'Install Node.js'

- script: |
    npm install -g @angular/cli
    npm install
    ng build -c $(BuildConfiguration)
  displayName: 'npm install and build'

- task: AzureStaticWebApp@0
  inputs:
      app_location: "/dist/dispatch-ui"
      skip_app_build: true
      output_location: ''
      config_file_location: '/src'
      azure_static_web_apps_api_token: $(deployment_token)

Expected behavior
The build succeeds and deploys my staticwebapp.config.json file.

Hi, I believe there's a missing curly bracket } at the end of "exclude": ["*.{css,scss,js,png,gif,ico,jpg,svg"]; it should be "exclude": ["*.{css,scss,js,png,gif,ico,jpg,svg}"]. Let us know if that fixes things.
On our end we'll make an improvement in logging these errors to the build pipeline.


rate limit based on the served objects instead of the requested objects count

PR Description
Update objects requests rate limiting based on the served objects count instead of the requested objects count:
BeaconBlockByRange, BeaconBlockByRoot, BlobSidecarsByRange and BlobSidecarsByRoot are all rate limited based on the requested objects count (The rate limiter is triggered when the request is received). In most case the real served objects count is lower (For blocks request, some blocks could not be returned and BlobSidecars we estimate the requested objects count by assuming that every block will contains MAX_BLOB_SIDECARS_BY_BLOCK Blob Sidecars which is not always/rarely the case).
The proposed solution is to add an object count adjustment to the rate limiter:

popBlockRequests and popBlobSidecarsRequests will update the rate limiter with the max estimated objects counts.
adjustBlockRequests and adjustBlobSidecarsRequests will adjust the initially estimated objects count and replace it with the served objects count (only if the adjustment happens during the same rate limiting time window).
cancelBlockRequests and cancelBlobSidecarRequests will reset the initially estimated objects if the request is cancelled for some reason (mainly errors when serving the requests)

Fixed Issue(s)

#6788
Documentation

[x] I thought about documentation and added the doc-change-required label to this PR if updates are required.

Changelog

[x] I thought about adding a changelog entry, and added one if I deemed necessary.

My first feeling on this: time especially in seconds couldn't be unique identifier, so I'd change
Pair<UInt64, Long> popObjectRequests(long objectCount)
to
Optional<UUID> popObjectRequests(objectCount)
if it's provided, we are allowed, and we next pass this uuid along with updated count in
adjustRequestObjects(long returnedObjectCount, long initialObjectCount, UUID requestId)
For UUID we could use AtomicInteger or AtomicLong if we turn on paranoid, we probably don't need real UUID.
NavigableMap<UInt64, Long> requestCount is changed to NavigableMap<UInt64, Map<UUID, count>> requestCount
Something like this.
Also I think we need to always toggle some number, say 5 or 10 to avoid DoSing with empty requests. Or did I miss total request tracking?

My miss, I see a sum now.
So what I don't like:

Pair return
Usage of time reference in client class

So I see 2 approaches:

The one I already suggested and we could skip requestObjects in this case I see
Wrap this time in something like RequestSlot, I don't like the word slot, confusing with BeaconChain slot, maybe another word... but the idea is to pass the same thing but wrap it to isolate logic from the client. So you change return type to Optional<RequestSlot> which stores the same value. When it's provided, you are allowed to pop. And it doesn't look like some strange Uint64 passing to client. You could keep almost the same code but it will be more elegant on the client side.

@zilm13 Here is a summary of what I changed:

Removed the Pair usage: Replaced it with an Optional<ObjectsRequestResponse> which is empty is the request couldn't be fulfilled and contains the requests detail otherwise
Wrapped the request details in ObjectsRequestResponse which contains the time in seconds, the objects count and a UUID
Used a UUID as id for the requests. this could be replaced by an AtomicInteger if required (I guess we won't have that much requests per second). I used the UUID because I find it easier
Updated the RateTracker data structure from NavigableMap<UInt64, Long> to NavigableMap<ObjectRequestsEntryKey, Long> (ObjectRequestsEntryKey is a combination of the UUID and the time in seconds). I think this is more readable, and performant compared to using 2 separate maps.
The  popObjectRequests now returns an Optional<ObjectsRequestResponse> which can be used later to adjust the counter by calling adjustObjectRequests


Inconsistency with Kubernetes required version

The documentation mentions that Kube 1.14+ is required to use Flagger while the charts mentions 1.11+.
I tested the Canary feature of Flagger on an old 1.12 cluster and did not face any issue.
Maybe the doc is not in sync? Or is there some advanced Flagger feature than won't work on this older cluster version?
Thanks!

If you're using Flagger with NGINX then it needs Kubernetes 1.14+ due to the ingress API being networking.k8s.io/v1beta1. For all the other integrations, ti should work on  Kubernetes 1.11+.

Ah, I am using Istio, that's probably why I did not face any issue, but this version requirement made me wonder enough to ask. Maybe it will be good to add a note on the documentation?
Anyway, thanks for the swift reply :) And feel free to close this issue.


np allocators need to either handle raw python types as dtypes or fail more clearly

Reporting a bug

[x] I have tried using the latest released version of Numba (most recent is
visible in the change log (https://github.com/numba/numba/blob/main/CHANGE_LOG).
[x] I have included a self contained code sample to reproduce the problem.
i.e. it's possible to run as 'python bug.py'.

This:
from numba import njit
import numpy as np

@njit
def foo():
    np.empty(4, dtype=int)

foo()

is entirely legal in NumPy, but in Numba raises a generic TypingError about there being no matching template. The issue is that the dtype is set as a Python int which gets interpreted as a Function(<class 'int'>) i.e. the int class ctor. This is reflected in the error message:
No implementation of function Function(<built-in function empty>) found for signature:

 >>> empty(Literal[int](4), dtype=Function(<class 'int'>))

Options:

Catch this case as it's probably relatively common and report in the error message how the user can change their code to get it to work.
Actually fix it in Numba, probably just spot the e.g. Function(<class 'int'>) and translate to a suitable type, e.g. types.intp.

Depends on: https://github.com/numba/numba/pull/7999

From the triage meeting: Option 1 may be the preferred option, since there is some ambiguity on what types like int and float may actually mean (and vary from platform to platform)


[Bug]: investigate and fix pipeline failure

Investigation shows that the pipeline fails for a connection timeout with grid tests,
But this only happens near the end of execution.

Further investigation shows that the failure happens after the "properties" tests. Which are known to change global engine configuration and are thus huge culprits.
NOTE: INVESTIGATE ADDING A PARAMETER TO ISOLATE PROPERTY CONFIGURATION SCOPE PER CLASS.

Indepth investigation in the pipeline execution and Git History shows that I am the culprit. I changed one value in the platform tests class and one value in the web tests class.
I will revert those values and open a PR.


Guides folder and README.md

Guides page for coding tutorials

Hello, thank you for your pull request. I will now be reviewing your contributions. I will let you know if there are any changes required.

I am going to convert your PR to a draft, once its done you may change it back.

Hello, thank you for your pull request. I will now be reviewing your contributions. I will let you know if there are any changes required.

alright, still gonna work on stuff tommorow


[Investigation] Mismatch between trend report and filtered distribution

Summary
Investigate the following and write up making trend months match up with the issued at dates.
Why
Consistency.  Banks having confidence in our nunbers
Details
There is a mismatch between trend report monthlys and the values you get if you filter the distributions.   We think the trend report might be looking at when the line items were created rather than at the issued at.   This would throw things off for any edited distributions, as well as ones that were entered late (or early) across month boundaries.
Please confirm that that is what is going on,  and either fix it (so that trend report should be by issued_at), or write it up -- or even just comment on this issue indicating that it has been confirmed.
If you fix it, please make the same change for the donations trend and the purchases trend reports.
Criteria for completion

[ ] report back on why the difference is occurring

Bonus Round

[ ] add an issue for the fix
or
[ ] fix it!  Alll Trend reports work based on the issued on date rather than the entered date
[ ] tests to support that.

I'd like to help with this!

please do!

I can confirm that the mismatch comes from the Distributions page filtering by distribution.issued_at while the Distributions Trend Report looks at line_item.created_at.
I'll work on a fix for this.


Implement str.isnumeric

Thanks for the PR. It's currently failing flake8 https://dev.azure.com/numba/numba/_build/results?buildId=3164&view=logs&jobId=687448c2-4012-5c9e-43b3-3520c9635a30&taskId=dba03d4a-57a7-5e59-a47a-79fe530f297a&lineStart=64&lineEnd=65&colStart=1&colEnd=1

Please could you resolve the conflicts by merging master into this branch, thanks.

Conflicts resolved. Do you have anymore comment or recommendations related to this PR?

Сhange the label, please.

@Rubtsowa thanks for the ping on this PR, I have updated the label accordingly.

Please could you resolve the conflicts by merging master into this branch. Thanks!

@stuartarchibald, I merged conflicts.

@stuartarchibald, I resolved conflicts.

Please check this PR! If everything is correct, please merge.


HTTP inputs should be copied by reference while constructing Request objects

First make a TestRequest with artisan make:request TestRequest. Then put this code in your routes.php file:
$router->get('/test', function()
{
  /** @var \Illuminate\Http\Request $request */
  $request = App::make('request');
  $request->merge(['request' => 'works']);
  /** @var \App\Http\Requests\TestRequest $testRequest */
  $testRequest = App::make(\App\Http\Requests\TestRequest::class);
  $testRequest->merge(['testRequest' => 'does not work']);
  dd(Input::all());
});

Hitting localhost/test?q=1, the response is not what I was expecting at first:
array:2 [
"q" => "1"
"request" => "works"
]
Is it a bug?
My expectation was that all objects inherited from Request are holding reference to the same 'singleton' data buffer, so by changing this buffer through $one->merge(), $others also observe the same change. But this expectation seems to be not met.

When you use the Input facade, you're always resolving the request service from the IoC container. Since $testRequest is another instance, these results make sense.

I know. It sounds like in the current implementation every request object has its own 'copy' of all the http inputs. But why it is not the case that every instance has 'reference' to those?
I was thinking of using both laravel 5 resquests and laracasts/commander CommanderTrait together, but then this merge() behavior makes calling execute() with default params not to do what I first thought it should do.


samples: smp_svr requires CONFIG_MULTITHREADING=y

Add an explicit CONFIG_MULTITHREADING=y to prj.conf
of this sample to make the setting dependency explicit.
Verified by running
./zephyr/scripts/twister --inline-logs -v -N -M \   --integration --overflow-as-errors --retry-failed 2 \   -T zephyr/samples/subsys/mgmt/mcumgr/smp_svr on:

Original condition. Verified all tests passed
and generated .config files already indicated
CONFIG_MULTITHREADING=y.

Amended prj.conf with CONFIG_MULTITHREADING=n.
Verified the resulting build failures were consistent
with a single-threaded environment (no k_sleep() definition).

Amended prj.conf with CONFIG_MULTITHREADING=y.
Verified all tests passed and generated .config
files indicated CONFIG_MULTITHREADING=y.

Amended prj.conf with CONFIG_MULTITHREADING=y and local
mcuboot module with the proposed zephyr port reorganization.
Verified all tests passed.

Fix #47895
Signed-off-by: Gregory SHUE<EMAIL_ADDRESS>

Hi @gregshue ,
I'm curious about why do we treat this sample as something special? Most of the samples / tests in Zephyr require to have MULTITHREADING enabled, and only few can work without it (so we explicitly disable MULTITHREADING to test them).
If some platform has MULTITHREADING disabled by default than that means that we need to explicitly enable it for all other tests and not only for this one.
I'm not saying that what we do here is wrong, but I'm wondering why do we do it.

@evgeniy-paltsev

I'm curious about why do we treat this sample as something special?

Good question.  The help description in zephyr/kernel/Kconfig and the associated comments in bootloader/mcuboot/boot/zephyr/Kconfig show that disabling MULTITHREADING is a valid configuration with limited support.  This configuration is particularly relevant for first stage boot code, which should be immutable coming from the board factory (i.e., fit within the special, permanently lockable subrange of flash designed for boot code). The SOC imposes a severe size restriction on boot code size, which may not allow for multithreaded execution.  Boot code should also run as quickly as possible, so multithreading is often not optimal.
If I understand Kconfig behavior correctly, the real question is why did someone chose to default MULTITHREADING to n for all NRF SOCs only when the MCUBoot module is in the build tree and independent of whether the MCUBoot application/service is enable.  I have seen that MCUBoot design must support USB device options for recovery communication, and this Zephyr USB subsystem/driver requires MULTHREADING=y, so the MULTITHREADING default value may need to be conditional. For this case the default should be dependent on building in the MCUBoot application/service.  So far I haven't found an exposed Kconfig to control this.

@de-nordic @nvlsianpu PR rebased to pass Pull Request Assigner check.  Please re-approve.

I am fine with the change here, but I am also wondering whether that dependency, on multithreading, shouldn't be selected when mcumgr is enabled.

hmm, so are we going to go through each sample and subsystem that requires the default and start adding this dependency? This is implicit and this is a sample, I do not think we should be putting this type of Kconfigs in the samples. Also, why would this be set differently, if someone else depends on multithreading=n and does that, it will generate kconfig warnings?

hmm, so are we going to go through each sample and subsystem that requires the default and start adding this dependency?

This looks reasonable given that:

The mcumgr itself as it depends on workqueues, transports and other subsystems, which use primitives like semaphores, and are only available when multithreading is enabled.

as @de-nordic noted above.

as @de-nordic noted above.

but isnt this the case with almost everything in zephyr?

but isnt this the case with almost everything in zephyr?

Unfortunately I have not found if we specify whether components explicitly set all their immediate dependencies or rely upon the project configuration to set everything.
It seems the expected use of Kconfig is that variation points are by default disabled, explicitly enabled and explicitly blocked.  Following that pattern components need to either support both values of MULTITHREADING or explicitly force the conditions they depend on, right?

Hey trying to understand where we are here since the PR is approved but the conversation still open: @nashif should we merge this or you want to suggest an alternative approach?
There's few other subsystem with depends on MULTITHREADING as well so that may be an option as well and I think makes more sense than having it in the sample if the underlying code requires it.

AFAICT, depends on MULTITHREADING only makes sense outside of a top level "project".
It seems to me that the only agent that should enforce multithreading to 'n' is the top level "project" (where prj.conf lives).  If the project doesn't care, then the selected feature sets/subsystems/drivers may cause multithreading to be enabled.  Otherwise, multithreading would be disabled.
The relevant question is: should MULITHREADING be enabled by each subsystem that requires it, or should it be enabled only by the top-level "project"?

@nashif

Still not convinced we should set an option that is the default in sample applications.

Nothing prevents a module from setting the default of MULTITHREADING differently. See how this is already being done in $workspace/bootloader/mcuboot/boot/zephyr/Kconfig.
This is fundamental to Kconfig, as is config settings implicitly defaulting to 'n'.
The issue with this and other code that directly depends on multithreading, is the expecting this dependence to be set elsewhere rather than forcing it to be set when needed. It needs to set its direct dependencies. Not doing so is a bug.

the real question is why did someone chose to default MULTITHREADING to n for all NRF SOCs only when the MCUBoot module is in the build tree and independent of whether the MCUBoot application/service is enable.

Answer are: we (Nordic) want to limit the size of the bootloader as much as possible, lot of efforts went here. As you can notice, even for nRF the multithreading in the MCUboot is not disabled by default for all configuration. When we comes to the application it is obvious that we in most cases need multithreading.

Answer are: we (Nordic) want to limit the size of the bootloader as much as possible, lot of efforts went here.

That is a great answer, and is good justification.  It proves users may have solid reasoning for changing the default of MULTITHREADING. This means that code that directly depends on multithreading needs to express that either though a Kconfig select or through a CONFIG_MULTITHREADING=y.

Isn't the issue you are going to fix here caused by the mcuboot port refactoring you are working on?

"Caused"? No.  Revealed? yes.  Downstream users may have a variety of reasons for changing the default of MULTITHREADING. This is a perfectly valid configuration in a our extensible system.
NOTE: It turns out the mcuboot port refactoring erroneously changed the MULTITHREADING default when the mcuboot "app" was NOT being built into the image.  That scoping has since been fixed in that PR, but the issue still exists - an implicit direct dependence on multithreading.

Can this be fixed in the refactoring in state-of-art manner?

No, because this component is where the bug exists.  Working around it reduces overall code quality, not improves it. This is actual a wide-spread issue, but not universal.
Note that some components already have a select MULTITHREADING in their Kconfig logic (e.g., $workspace/zephyr/drivers/kscan/Kconfig.it8xxx2, $workspace/zephyr/drivers/kscan/Kconfig.xec).  Some samples already explicitly set CONFIG_MULTITHREADING=y (e.g., $workspace/zephyr/samples/basic/minimal/mt.conf).  Other samples set CONFIG_MULTITHREADING=n (e.g., $workspace/zephyr/samples/arch/mpu/pmu_test/prj-single.conf).

Discussion in the dev-review meeting summary:

Module Kconfigs are treated as extensions of the Zephyr repository Kconfig.
At least some of the participants recognized that modules changing defaults provided by the Zephyr repository is a present risk.
There is no known way in the Kconfig standard to prevent modules from changing the default value of Kconfig variables provided by the Zephyr repository.
Having each component set the Kconfig values it directly requires is a big burden, and akin to "throwing the baby out with the bathwater". There is great value in relying upon Kconfig defaults.
No general solution to the problem was identified.
Mitigating the risk is being postponed until we know what functionality is required in this area.

Since the problem is much larger than what is addressed by this PR and this risk was revealed by an error in refactored mcuboot Kconfig that has now been addressed, this PR is being closed by the author without merging.


Not stable Transaction verifing

#include <iostream>

#include  "ark/arkCrypto.h"

#include <iostream>
using namespace std;

int main()
{

 string passphrase = "any passphrase";

	Address addr = Address::fromPassphrase("any other passphrase", 0);
	string recipientId = addr.toString();

	size_t j = 0;
	size_t i;
	for (i = 0; i < 10; ++i) {
		Transaction transaction = Builder::buildTransfer(recipientId, 24, "vendor field", passphrase);
		string signature = transaction.sign(passphrase.c_str());
		cout << "iter"<< i <<  " transaction.verify: " << transaction.verify() << endl;
		if (transaction.verify()) {
			j++;
		}
		Serializer serialiser(transaction);
		string serialised = serialiser.serialize();
		cout << "iter" << i << " serialised: " << serialised << endl;
		Deserializer deserialiser(serialised);
		transaction = deserialiser.deserialize();
		cout << "iter" << i << " transaction.verify: " << transaction.verify() << endl << endl;
	}

	cout << "success verify " << j << " out of " << i << " attempts";
}

// output 4 of 10 was success

I am able to reproduce this and looking into it.

The first one has insufficient padding in r and the second one has insufficient padding in s, meaning they are negative numbers in ASN.1 IntVar format. These test vectors are detected as non-canonical signatures by Haskoin's canonical DER parser, but since I moved to this secp256k1 library, the vectors are considered valid by the default strict parser.
@xenog: https://github.com/bitcoin-core/secp256k1/issues/341#issue-113167198

I presume this expectation was due to BIP66.
BIP66 is stricter than canonical DER and needed to be because most underlying ECDSA implementations used in Bitcoin land will not just parse but also pass signatures which are validly encoded invalid signatures (e.g. OpenSSL decodes like an unsigned value always). So if BIP66 had limited itself to only restricting things to canonical DER the network consensus would still depend on crazy implementation specific behavior in underlying signature libraries.
gmaxwell: https://github.com/bitcoin-core/secp256k1/issues/341#issuecomment-150887778

@officefish @ciband
Status update:
This issue is resolved in the AIP11 branch.
#include <cstdint>

#include "identities/address.hpp"
#include "identities/keys.hpp"

#include "transactions/serializer.hpp"
#include "transactions/deserializer.hpp"
#include "transactions/transaction.hpp"

#include "transactions/builders/transfer.hpp"

#include "utils/hex.hpp"

using namespace Ark::Crypto::identities;
using namespace Ark::Crypto::transactions;

int main() {
    const auto passphrase = "this is a top secret passphrase";
    const auto senderPublicKey = Keys::fromPassphrase(passphrase).publicKey;

    const auto anotherPassphrase = "this is a top secret passphrase too";
    const auto recipientId = Address::fromPassphrase(anotherPassphrase, 30).toString();

    size_t j = 0UL;
    size_t i;
    for (i = 0UL; i < 1000UL; ++i) {
        auto transaction = builder::Transfer()
                .network(30)
                .type(0U)
                .nonce(1ULL)
                .fee(10000000)
                .senderPublicKey(senderPublicKey.data())
                .vendorField("vendorField")
                .amount(24ULL)
                .expiration(0)
                .recipientId(recipientId)
                .sign(passphrase)
                .build();

        auto verified = transaction.verify();
        printf("\niter %zu transaction.verify: %d\n", i, verified);
        if (verified) {
            j++;
        }

        const auto serialized = Serializer::serialize(transaction.data);
        printf("iter %zu serialized: %s\n", i, BytesToHex(serialized).c_str());

        memset(&transaction.data, 0, sizeof(transaction.data));

        Deserializer::deserialize(&transaction.data, serialized);
        printf("iter %zu transaction.verify: %d\n", i, transaction.verify());
    }

    printf("\nsuccess verify %zu out of %zu\n\n", j, i);
}
}

output:

iter 0 transaction.verify: 1
iter 0 serialized: ff021e0100000000000100000000000000034151a3ec46b5670a682b0a63394f863587d1bc97483b1b6c70eb58e7f0aed19280969800000000000b76656e646f724669656c641800000000000000000000001ee04f5be8d462a0a0509a3e7b9853e1d9724bb16c3045022100cd3816e5eef9eb8b5cc136970ac8e3ba73cad82ee77439c26ddd40074b6194ab02203d629a42006e939ce39b2435db346cb6a3d27d2025188f51e5d63bdb9aa0514b
iter 0 transaction.verify: 1

...

iter 999 transaction.verify: 1
iter 999 serialized: ff021e0100000000000100000000000000034151a3ec46b5670a682b0a63394f863587d1bc97483b1b6c70eb58e7f0aed19280969800000000000b76656e646f724669656c641800000000000000000000001ee04f5be8d462a0a0509a3e7b9853e1d9724bb16c3045022100cd3816e5eef9eb8b5cc136970ac8e3ba73cad82ee77439c26ddd40074b6194ab02203d629a42006e939ce39b2435db346cb6a3d27d2025188f51e5d63bdb9aa0514b
iter 999 transaction.verify: 1

success verify 1000 out of 1000

Thank you again for finding this, @officefish 👍


Create mentonegirls.txt

Added Mentone Girls' Grammar

@mggs-lin First stage of verification passed, official domain for Mentone Girls' Grammar is mentonegirls.vic.edu.au as far as I verified. This is just an informational note. Pull request is still in the review. I appreciate your patience while waiting for the final decision.

@mggs-lin Thank you for your patience while waiting for a review and processing your request. Pull request merged.
Please start requesting the licenses in about an hour, to let the changes to propagate through our system.


Create a Java parser wrapper

Implement a Java wrapper of the Go parser, by using the C shared object resulting from the compilation here: https://github.com/asyncapi/parser/blob/master/compile.sh.
It should be fairly straightforward to code but we need to make sure we understand how to compile C for different platforms and architectures.
References

https://gist.github.com/steeve/6905542#gistcomment-1988684
https://www3.ntu.edu.sg/home/ehchua/programming/java/JavaNativeInterface.html

I think it is easier to implement a separate Java parser rather than wrap Go parser.

Sure, but then we'll have to maintain 2. If we keep growing, eventually we'll invest in creating a separate one. In the meantime, it's the easiest solution to provide Java support.

Is it still planned to implement this java wrapper around the AsyncAPI parser?

It's still planned but we don't have a date yet.

Does it really make sense in 2021?

We did implement it in Java for our needs. We only took the schema and used standard java libraries to perform the validation of async api spec against the schema. Of course a Java Library providing model, parsing and validation would be awesome, but I think unless it comes with a model, there is no much benefit.

The problem is that not everything in AsyncAPI document can be validated with AsyncAPI JSON Schema. So we need a parser. Models could be generated with https://github.com/asyncapi/modelina maybe 🤔
Most important though is that it cannot just be my initiative. The best would be to have other companies work on it under AsyncAPI umbrella. I mean best if it is a work of folks that will actually use it in production. @gexclaude 😉

I see. Currently I no longer work for the customer where I've put AsyncAPI in place. At the moment I am not involved in projects where we can make use of AsyncAPI, therefore interest / benefit is not given right now. Maybe situation change and I'll be back contributing ;-)

We would definitely appreciate a Java library for parsing AsyncAPI spec files.  We currently have some tooling for parsing the OpenAPI spec that uses the Java parser for OpenAPI.  I can see us extending out tooling for AsyncAPI specs if the Java library were available.

@fmvilas @derberg I am interested in working on this project. let me know if i can work on the new parser : )

@AayushSaini101 please 🙏 That would be amazing.

@jonaslagoni had some ideas on how to semi-automate the generation of parsers in multiple languages. Is this documented somewhere, Jonas? Also, there's a Building Blocks Working Group that's currently stale but could be restarted by folks interested in pushing it forward.

You can checkout some of the previous working group meetings on the AsyncAPI youtube, and then checkout https://github.com/jonaslagoni/asyncapi-python-parser which is how I did it for python, still need some love though


Temporarily redact recordedBy and identifiedBy fields

These fields need to be blanked in short term due to DOB issues.  The longer term fix should be for Axiell to update their DwC update scripts.

Currently redacted.  @mswoodburn - can you confirm with Axiell where we are with the fix please.

Now unredacted by Josh, incorrect fix done by Axiell which removes all dates, but safe to publish again at least.

Closed


fix(api): order Projects and AnalysisTemplates by name

Fixes: #782

@krancour I actually checked on those as well, figured they were already sorted in the frontend, and then decided to not do it. But the CLI benefiting from it is actually a good point, and I will follow up.