Adding encoding and bits_per_sample options to save

The current release version's "soundfile" backend's save function changes the encoding of the audio file based on the dtype of the provided Tensor. For example, if the dtype is "float32", then it will be saved as 32bit floating point PCM. This behavior was taken from SciPy's scipy.io.wavefile.write function. However it was pointed out that this is inconvenient for torchaudio users. Because most torchaudio's functionality works on float32 Tensor yet, the common audio formats typically retains only 16 bit, such as 16 bit signed integer PCM.
To resolve the inconvenience while keeping the functionality to support different encodings, we would like to add;

Add encoding and bits_per_sample parameters to save function.
~For non-compressed format (such as "wav"), it defaults to 16-bit signed integer PCM. (This is BC-breaking behavior if users were dumping Tensor object without converting to the matching dtype)~

See #1226 for the corresponding changes for "sox_io" backend. (but for "soundfile" backend the expected changes are much simpler)
Steps

Add encoding and bits_per_sample options to save function of soundfile backend. Refer to the #1226 for the specification (valid values, fallback values etc). Note that sound file does not support all the formats libsox does. (wav and flac are the ones that should be covered and match the behavior of "sox_io" backend as much as possible)
Update the logic that determines "subtype" argument so that subtype is determined by format, encoding and bits_per_sample parameters. Note To learn how PySoundFile internally expresses audio format, see here
Update the test

Update the mocked test that checks what parameters are given to the underlying soundfile module. (Input parameter should be changed from dtype to encoding and bits_per_sample so that the logic added in step 2 is tested)
Fix the reset of the test which will brake because for wav format the function will now default to 16bit PCM.

Build and test
Refer to CONTRIBUTING for the development setup.
To run the tests;
pytest test/torchaudio_unittest/backend/soundfile/save_test.py

Assigning @prabhat00155 temporarily

Thanks @mthrok, I can work on this.

@prabhat00155 Thanks.
Following the feedback in #1226, we drop the requirement 2 (defaulting to 16 bit signed PCM).
Keep the original logic for the case neither encoding or bits_per_sample are provided. Let me know if you need help.

This issue could be closed now; PR merged.


bugfix install pip on centos6 using scl

Fix the travis-ci builds ;)

done!

Why didn't you just use the SCL pip package?


fix: bump sc4s/splunk versions

ci: fix permissions
fix: update sc4s/splunk versions

:tada: This PR is included in version 1.9.4 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


AF crashes when a joystick is removed

Fixed in this commit:
https://github.com/MarkOates/allegro_flare/commit/61b098fa007703b7161e784a77c4f3e32c971da9


Add support for respective capture options on set-enabled APIs

Will tackle "mute instead of unpublish" for screenshare in a separate PR.
Making the functionality available seems like higher priority for now than the internal handling of it.

Should this be a minor or patch version? given the new options.


The SVG icons cover the entire button

I'm sorry to notice late. I could not test the previous solution correctly (#37 ). In the current situation the SVG icons cover the entire button. I think the width of the button and the icon should be different. (.v-button.floating-action, .v-button.floating-action .v-icon)

I'm sorry to notice late.

No Problem!

I could not test the previous solution correctly (#37 ). In the current situation the SVG icons cover the entire button. I think the width of the button and the icon should be different. (.v-button.floating-action, .v-button.floating-action .v-icon)

Could you please provide some example code?

...
Button cookButton = new Button(new ThemeResource("image/cook.svg")); //(size: 150px x 150px)
cookButton.addStyleNames(MaterialTheme.BUTTON_FLOATING_ACTION, MaterialTheme.BUTTON_DANGER, MaterialTheme.BUTTON_ICON_ONLY);
layout.addComponent(cookButton);
...


mac search shortcut symbol on windows

im on windows but the symbol is a mac only symbol

thanks, will fix it

@ludfjig it's fixed and the fix will be included in the next release of Hextra. thanks for the feedback to make Hextra better 💪


Allow easy replacement of the logo in Foomedical

Currently the logo on Foomedical is difficult to change and requires a hard coded SVG.  Update it to support swapping with a PNG similar to app shell

Took a quick look at this:
Looks like we have 2 separate headers in this example: one for landing page, and one for the main app once authenticated. That makes replacing the logo in non-icky a bit more painful since we have to pass it in two places, and through an entire component tree in one of those cases.
Potential solution:
Add a context provider to the AppShell and a useLogo hook which each of these headers can call independently to get the logo as a ReactNode. Definitely not a light touch solution but might be useful in general to have such a hook for apps where you might use the logo in several places


fix: Use new tauri-plugin-os instead of tauri::api::os

The tauri::api::os module does not exist anymore on Tauri v2. We need to to use tauri-plugin-os instead.

Check here: https://github.com/ahonn/keygen-rs/tree/master/packages/tauri-plugin-keygen-rs2

Oh so the master branch is now targeted at Tauri v2 ?

The master branch now contains two version of Tauri plugins. You can see them under https://github.com/ahonn/keygen-rs/tree/master/packages.


Add modal with contextual information and searches

Modal should include:

Text (below)
Three default searches that lead to different cities with pre-set layers selected (Deb will select)
Location search box

Modal will be skipped if the url includes a center=lat,lng query parameter.
The job of the modal is to set a center=lat,lng query parameter

Intro paragraph:
Trees provide countless economic, social, and environmental benefits. Discover optimal tree planting locations based on environmental and demographic data. Using the sidebar on the left of the page, set the priorities that connect to your planting goals, choose the zip codes you would like to see, and view maps displaying possible planting areas.
Search links:
Search Box
Text: Enter location to view maps
Los Angeles:
Explore planting locations related to environmental justice
Philadelphia:
Find planting locations where the most people live
Chicago:
Discover planting locations with low housing vacancy rates

@jwalgran I'm not seeing the prioritize plantings link when viewing a map in Firefox. I do see it in Chrome and Safari.


Step 1 of 10: Code doesn't run inside the Focus Menu

I copy/pasted the Code in Step 1 of 10 in this tutorial, and the Focus Window complains there is an error. I think you would want to fix it. Thanks.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 1a3686c2-4c73-5686-0da9-abc38c74aeec
Version Independent ID: ca995f80-9375-e9ca-b855-6e6893581e23
Content: Explore C# 6 - Try the new features in C# 6 interactively, using your browser
Content Source: docs/csharp/tutorials/exploration/csharp-6.yml
Product: dotnet-csharp
GitHub Login: @BillWagner
Microsoft Alias: wiwagn

Thank you for letting us know @citrusit. I can repro that https://docs.microsoft.com/en-us/dotnet/csharp/tutorials/exploration/csharp-6?tutorial-step=1.
@BillWagner @LadyNaggaga @jonsequitur is this related to the try.net changes? If I remove the using System statement, I get an OOM exception.

@rchande

Let me know if we should transfer this issue to the try.net repo.

Closing this since a new issue was created under dotnet/try to track this problem:
https://github.com/dotnet/try/issues/174

this has still not been fixed. if you remove the import statement, and change Program to programs or something else it works.

i don't know how to fix it in focus mode, but hopefully you guys circle back to this problem.


Translate May 8 covid report into Japanese

Happy weekend @nextstrain/japanese,
The situation report from 2020-05-08 is ready to be translated to Japanese. This week's report is short and sweet and focuses on Asia. We tried to tie the trees in with other epidemiological info; I'm curious to hear what you think of it. As with other regional reports (and really all reports) if you don't have time or are not able to translate this report, totally fine. Just let me know.
We've been publishing weekly reports on covid for two months now! As such, we're taking time this next week to re-evaluate narratives and figure out how we can make these reports as useful as possible going forward. I'd welcome your feedback on the situation reports -- what you find useful, what you don't, how often you think reports should be published, how these reports help (or don't help) Japanese speakers.
Branch to translate on: sit-rep_ja
File to translate: https://github.com/nextstrain/translations/blob/sit-rep_ja/narratives/translations_ncov_sit-rep_ja_2020-05-08.md You can translate directly in this file, but please remember to push often as other people may be working on the translation!
Link to preview: https://nextstrain.org/community/narratives/nextstrain/translations@sit-rep_ja/ncov/sit-rep/ja/2020-05-08
Twitter post to translate:
“Want to learn more about outbreaks in Asia? This week's situation report uses #SARSCoV2 genomes to track the recent spread of #COVID19 in Asia. The report is now available in Japanese. Thank you @, @, and @ for the rapid translation."
Thanks so much for your help -- it's always a pleasure to work with you. See below for standard translation stuff.
Best,
Cassia
Logistics
Please update the translations board as you work on the translation: https://github.com/orgs/nextstrain/projects/7
Please assign yourself to the translation if you’re working on it this week.
When a translation is ready for publication -- at minimum meaning two people have worked on it and both agree that it’s ready -- please create a pull request to merge to master and tag me and the issue in it!
Discussion board for your team is here: https://github.com/orgs/nextstrain/teams/japanese
Standard rules for translation:
Note that the file to translate is in Markdown. In general, it is important not to change any formatting or links. (Although it is okay to update Wikipedia or CDC links to the appropriate language version.) Comments to help with translation are noted in comment symbols like <! -- Comment -->.
Please add your name under translators: with new line, tab, and dash, formatted just like authors are. Please also add a link to your professional page (can use Twitter) under translatorLinks:.
Note: You may know of other great resources available in Japanese; please don’t add them to the situation report. We need to keep content the same across all translations, so we can be confident in the accuracy of what we publish. Thanks for understanding.
At minimum, I require two sets of eyes on every translation; one of those people should have a background in phylogenetics. Although if you have more, that’s fine too.
Feel free to use a live, online editor like hackmd (sign in with GitHub) to translate together.

Hello folks.
I'll translate from now on. This time the amount is small text, so I'll finish everything until the afternoon.

@satotake @fzhang12 The translation is done. Please verification.
@cassiawag For me, living in Japan, this report is about "Southeast Asia". Should this be translated as "Asia"?

@satotake @fzhang12 The translation is done. Please verification.
@cassiawag For me, living in Japan, this report is about "Southeast Asia". Should this be translated as "Asia"? If Nextstrain have a plans for Northeast Asia such as "China, South Korea, Japan" next week or soon, I would like to translate it as "Southeast Asia" in this report. If Nextstrain don't have any plans, "Asia" may be fine.

@satotake @fzhang12 The translation is done. Please verification.
@cassiawag For me, living in Japan, this report is about "Southeast Asia". Should this be translated as "Asia"? If Nextstrain have a plans for Northeast Asia such as "China, Taiwan, South Korea, Japan, etc" next week or soon, I would like to translate it as "Southeast Asia" in this report. If Nextstrain don't have any plans, "Asia" may be fine.

@satotake @fzhang12 The translation is done. Please verification.
@cassiawag For me, living in Japan, this report is about "Southeast Asia and India". Should this be translated as "Asia"? If Nextstrain have a plans for Northeast Asia such as "China, Taiwan, South Korea, Japan, etc" next week or soon, I would like to translate it as "Southeast Asia" or "Southeast Asia and India" in this report. If Nextstrain don't have any plans, "Asia" may be fine.

Hi @tomoyuki-nezu, I'm not sure where we're planning on focusing next, we've been deciding on a week by week basis, typically focusing on areas with new sequencing that tells a story we haven't told yet. In the report, I'd appreciate it if you just saw Asia. But for the Twitter post Southeast Asia and India is just fine.

@tomoyuki-nezu done.
Just editing a few of parts. Please check it
Thank you

@satotake Thank you!!!
@cassiawag I see it. Then, in translation use Asia, and when I tweet, will use "especially Southeast Asia and India" etc. Thanks.
@cassiawag Please review it.

Is this ready to publish? Or does it need @fzhang12's review?

I think it is better to confirm by @fzhang12

@cassiawag However, if @fzhang12 do not contact by the next morning in your time zone, publish it and update it later if necessary.

Deeply sorry for being so postponed this week. I had to work on other things during the weekend. The work is as perfect as usual.
@cassiawag I think @tomoyuki-nezu and @satotake have gained enough experience on this field. I am not so stable with my schedule these days. I am totally convinced that timely translation could be pushed without my proofreading.
I will create the pull request.
BTW, Can someone translate the tweet?

@fzhang12 Thank you for your reply despite being so busy. It's based on GitHub, so if you have time, I'd be happy if you could comment or PR :)
@cassiawag How about such a tweet?
今週の＃COVID19 状況レポートではアジアに焦点を当て、インドでの感染拡大の起源、 シンガポールにおける感染拡大の背景、そして地域全体への国際的な広がりについて報告します。迅速な翻訳を行って頂いた @tommy_nezy さん、Takeshi Sato さんと @fengjun_zhang さんに感謝します。

@tomoyuki-nezu Probably I didnt write it clear that I have already proofread this week's report.
I did not comment because both of you did prefect job :)

LGTM!
Thanks @cassiawag

Published and tweeted: https://nextstrain.org/narratives/ncov/sit-rep/ja/2020-05-08
Thanks all! I'll take any feedback that you have on these reports.


Fix the svc name of webhook to avoid breaking istio

This addresses the issue https://github.com/kedacore/charts/issues/500
Only webhook's svc name is http, others are fine
Checklist

[ ] When introducing a new scaler, I agree with the scaling governance policy
[ ] I have verified that my change is according to the deprecations & breaking changes policy
[ ] Tests have been added
[ ] Changelog has been updated and is aligned with our changelog requirements
[ ] A PR is opened to update our Helm chart (repo) (if applicable, ie. when deployment manifests are modified)
[ ] A PR is opened to update the documentation on (repo) (if applicable)
[x] Commits are signed with Developer Certificate of Origin (DCO - learn more)

Fixes #

Relates to #

/skip-e2e

❤


Move API types into separate files

What this PR does / why we need it:
Moves our API types into separate types_certificates and types_issuer files
Which issue this PR fixes:
This is to ease implementation of #788
Special notes for your reviewer:
This should be a simple no-op change
Release note:
NONE

/cc @kragniz


Switch Icarus back to master

Signed-off-by: Cary R<EMAIL_ADDRESS>See if the latest Icarus builds correctly.

Hmm, I thought I had said I was struggling trying to understand why this was not working, but it's certainly not in this PR. I would like to eventually get back to this, but given I have a part back from fabrication to look at creating long days and then trying to get a paper written for a conference deadline in a couple weeks along with my normal Icarus work in the evening. I'm overloaded right now so do not have time to dig in and figure out what is wrong.

Done in #176


To-do List Auth added

Level-4 Completed

Level-4 Completed

level 4 requires you to implement a policy system like opaL (readme level-4). while your submitted meets level 3 criteria.

Sorry I mistakenly wrote level 4

no problem


Question about Train and Test Scripts

Hello Youness,@CVxTz
Why don't we use mask when we are training (e.g. in baseline_aug.py)  train_data = list(zip(sorted(glob('../input/DRIVE/training/images/*.tif')),                           sorted(glob('../input/DRIVE/training/1st_manual/*.gif'))))
and why do we use annotations when are predicting?(e.g. in baseline_aug_predict.py) val_data = list(zip(sorted(glob('../input/DRIVE/test/images/*.tif')),                           sorted(glob('../input/DRIVE/test/2nd_manual/*.gif')),                         sorted(glob('../input/DRIVE/test/mask/*.gif'))))

The mask is mostly useful to do a more accurate evaluation, we want to focus our evaluation metric on the central region of the image (where the vessels are) and ignore the corners that are all black.

, we want to focus our evaluation metric on the central region of the image (where the vessels are) and ignore the corners that are all black.

Thanks for prompt response. I understand, but actually my question is something else.

Why did NOT we use a mask in training?
Why did we use annotations in prediction? (in prediction shouldn't we only need image and mask?)

We don't need accurate evaluation during training so we don't use the mask.
The model only sees the input image, the annotation are only used for the evaluation ( done in the same prediction script)


support iptables tcp and udp tproxy

I'm submitting a...

[ ] bug report

[X] feature request

[X] question

blinksocks version: [3.0.0] Binary

os: [Debian 9.3 ]

How Can I use iptables nat table  redirect and TPROXY in mangle table to transparent transfer traffic
to blinksocks
blinksocks server is using Websocket
Thank you

Other information (e.g. detailed explanation, stacktraces, related issues, suggestions how to fix)

I have no idea about this, but I found a document of tproxy which could help:
http://lxr.linux.no/linux+v3.10/Documentation/networking/tproxy.txt#L44

This is TCP. I also need to implement UDP support via TPROXY like ss-redir does.
I don't know weather this can be done via Node.js or not.
Thank you.


Autosave option

Because with the autoupdate option we have a very fluent writing workflow, it could be nice to add an "Auto save" option in Editor Settings.
With this option, we just have to think about writing, not saving.

@NicolasGeraud  Hmm, when you close any tab or application, it will save document(s) by default. Does it mean what you want?

No,
I use intellij in my daily work, and it saves my work in background. No more Ctrl+S .
Actually I write a long document, and I have to not forget to save my file if a crash occurs or a battery failure.

You dont need to use ctrl+s in afx. But jvm crash or battery failure causes to data lost. We are imitating intellij idea' saving behaviour. Are you sure intellij saves your data if crash or battery failure occurs?

@NicolasGeraud what do you think about AsciidocFX's saving behaviours ?

@rahmanusta I think the suggestion by @NicolasGeraud perfectly makes sense. You are right in the sense that the sudden crash causes the data lost in any way, but data edited 10seconds ago will be saved if autosaved 10 seconds ago, and it's very unlikely to save the data frequently manually without the writer consumes his performance just to save the data. I think most of the modern editor has an auto-save feature as the default, not as an option.
What do you lose by the simple implementation of such an important feature? In fact, I am another guy who googled to make sure this nice editor has the auto-save feature, and now I'm a bit of surprised have found it to be ignored.
Thanks.

Hi @kenokabe, I'm open for contribution and pull requests

@rahmanusta Hi, thanks a lot to have reopened the issue. Let me correct my prior mention:

most of the modern editors have an auto-save feature as the default, not as an option.

For Microsoft Officce Suite. it should be true:
https://support.office.com/en-us/article/what-is-autosave-6d6bd723-ebfd-4e40-b5f6-ae6e8088f7a5

What is AutoSave?
"Save early, save often" is now a thing of the past. Now there's AutoSave which saves every few seconds so that you don't have to.
AutoSave is enabled when a file is stored on OneDrive, OneDrive for Business, or SharePoint Online. It automatically saves your changes to the cloud as you are working. And, if other people are working on the same file, AutoSave lets them see your changes in a matter of seconds.

For Microsoft VisualStudio Code which I heavily use, it is an option:
https://code.visualstudio.com/docs/editor/codebasics#_save-auto-save

Save / Auto Save
By default, VS Code requires an explicit action to save your changes to disk, Ctrl+S.
However, it's easy to turn on Auto Save, which will save your changes after a configured delay or when focus leaves the editor. With this option turned on, there is no need to explicitly save the file. The easiest way to turn on Auto Save is with the File > Auto Save toggle which turns on and off save after a delay.
For more control over Auto Save, open User or Workspace settings and find the associated settings:
files.autoSave: Can have the values:

off - to disable auto save.
afterDelay - to save files after a configured delay (default 1000 ms).
onFocusChange - to save files when focus moves out of the editor of the dirty file.
onWindowChange - to save files when the focus moves out of the VS Code window.
files.autoSaveDelay: Configures the delay in milliseconds when files.autoSave is configured to afterDelay. The default is 1000 ms.

I personally use VSCode auto-save  onFocusChange because the auto-format of the code is also depends on the save. Eitherway, nowadays looks like auto-save on the spot or every few seconds seems an established manner, and since I probably keep using this nice editor, I hope this one also behaves in that manner to avoid the data loss.  I wish I could help this issue, but unfortunately I'm JavaScript coder.

Thanks for the details. I think this can be implemented, but I don't know when.. Btw, onFocusChange  and onWindowChange  is default behaviour of AsciidocFX.

@rahmanusta
Thanks for this nice editor again. I think this editor is the best in this AsciiDoc world.

Btw, onFocusChange and onWindowChange is default behaviour of AsciidocFX.

Yesterday, I start writing on this editor and found that include does not reflect the preview for the included another file with a change. It may be saved onFocusChnage (of tab), but the multiple file's changes do not integrate well. Just let you know. At that time, I thought it must be saved manually.

We long time auto-saving files in AsciidocFX


Added support for service.version

See #132. For the reference, I did not have a look at the implementation in #132.

@eyalkoren I will add support for it in the agent once this is released.

@eyalkoren Would you please have a look?

ping

@tobiasstadler sorry, we are quite overloaded atm, I hope we have a chance to get to this and additional ecs-logging stuff soon enough 🙏

Thank You!


Get started - API and TFS version mapping

Section API and TFS version mapping still says vNext instead of 2020 and doesnt list a build number.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: a0ab71a0-a2df-fa37-83fa-080dece952f6
Version Independent ID: f74b7d51-aede-ed93-437e-601654d8fa7e
Content: Get started with the REST APIs for Azure DevOps Services and Team Foundation Server - Azure DevOps Services REST API
Content Source: index.md
Product: devops
Technology: devops-ecosystem
GitHub Login: @elbatk
Microsoft Alias: elbatk

@michha thanks for reporting this has been fixed.


Small styling improvements concerning Create Proof Template

Filters in create proof template are visible now, deleted log button, minor styling adjustments
Signed-off-by: Annika Krause<EMAIL_ADDRESS>RM: @steve-237

The added code is ok but there is an error on the create proof template page. Please let me know if you also get that error on your side.

Yes, I also get this error. This part of the code is not 100% bug free, but it works and nobody had the time to fix it so far :)


Fix issue #77

Coverage increased (+0.05%) to 87.505% when pulling ff0db81335bde10d81a27798af05ab4a2ea39533 on issue_77 into 624d3e7a6abef1f87720a6ced47a23d13ccd0400 on master.


Can't build latest version

Windows 11, MSys 2, rustc 1.69.0:
error[E0433]: failed to resolve: use of undeclared type `AnsiEscaped`
   --> src\render\tree\node\display\mod.rs:122:31
    |
122 |             let out = <str as AnsiEscaped>::truncate(&ln, window_width);
    |                               ^^^^^^^^^^^ use of undeclared type `AnsiEscaped`

error[E0433]: failed to resolve: use of undeclared type `AnsiEscaped`
   --> src\render\tree\node\display\mod.rs:146:31
    |
146 |             let out = <str as AnsiEscaped>::truncate(&ln, window_width);
    |                               ^^^^^^^^^^^ use of undeclared type `AnsiEscaped`

error[E0554]: `#![feature]` may not be used on the stable release channel
 --> src/main.rs:1:30
  |
1 | #![cfg_attr(windows, feature(windows_by_handle))]
  |                              ^^^^^^^^^^^^^^^^^

Ahh yes a contributor ran clippy but failed to check the windows build. The github workflow also only checks the linux build which is something I need to address. Regardless, the issue has been resolved and you should know be able to manually build the HEAD of the master branch. Let me know if you have further issues!

Yup, the errors are gone, but it's still failing with:
error[E0554]: `#![feature]` may not be used on the stable release channel
 --> src/main.rs:1:30
  |
1 | #![cfg_attr(windows, feature(windows_by_handle))]
  |

Is it expected that erdtree can only be compiled by the unstable channel?

Ahh yes unfortunately compiling for Windows does require nightly in order to support the detection of hardlinks via this API. Without it hardlinks that are descended from the same root would be double counted towards total memory usage.


checkov: 3.2.260 -> 3.2.266

Diff: https://github.com/bridgecrewio/checkov/compare/refs/tags/3.2.260...3.2.266
Changelog: https://github.com/bridgecrewio/checkov/releases/tag/3.2.266

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

Result of nixpkgs-review pr 348808 run on x86_64-linux 1

2 packages built:

checkov
checkov.dist


Set OTHER_SWIFT_FLAGS as a string

Xcode prefers it this way. There really is no rhyme or reason to this, is there?

Current dependencies on/for this PR:

main

PR #153

PR #157

PR #158

PR #159

PR #160

PR #165

PR #185

PR #186   (this PR)


bundle: filesystem metadata format

@stevvooe and I caught-up in person about our digest discussion and the need for serialize file-system metadata. If you want to read my attempt it is found here: https://github.com/opencontainers/specs/issues/5#issuecomment-114208979
Problem: a rootfs for a container bundle sitting on-disk may not reflect the exact intended state of the bundle when it was copied to its current location. Possible causes might include: running on filesystems with varying levels of metadata support (nfs w/o xattrs), accidental property changes (chown -R), or purposeful changes (xattrs added to enforce local policies).
Obviously the files contents will be identical so that isn't a concern.
Solution: If we hope to create a stable digest of the bundle in the face of these likely scenarios we should store the intended filesystem metadata into a file itself. This can be done in a variety of ways and this issue is a place to discuss pros/cons. As a piece of prior-art @vbatts has implemented https://github.com/vbatts/tar-split and we have the linux package managers with tools to verify and restore filesystem metadata from a database with rpm -a --setperms and rpm -V.

After some thought, we need a format that does the following:

A manifest enumerates files in a bundle.

Provides a content hash.
Provides a path.
Provides a file type.
Provides standard file mode.
Provides xattr.

Geared towards multiple OS support.
Not infinitely extendable but should be easy to add new field.

Bundle contents attributes can be reset to contents of file manifest.

Bundle is scanned and any differences from manifest are rectified.
Unames/Gnames/Uid/Gid can be mapped during "reset".

Bundle contents can be verified against manifest.

Content hash can be checked.
Attributes can be checked.

Certain attributes can be checked against machine-local mapping (uid/gid, etc.).

Manifest can optionally be signed.

Meeting the above three use cases with this format puts us above the test for Cost problems with the ideas from #5. Requirement 2 above has a common use case, by avoiding placing unreasonable requirements on transports. Requirement 3 above gives us the functionality of #5 with the extra benefits of Requirement 2.
@shykes @philips @crosbymichael

@stevvooe I am having a hard time parsing this sentence: "Meeting the above three use cases with this format puts us above the test for Cost problems with the ideas from #5."
I agree with all three needs overall despite my confusion above.

@philips That is a poor sentence where I've shoved in a lot of meaning.
What I'm saying is that, given the goals of #5 (cryptographic verification), the cost of scanning a bundle is not warranted. Given these new goals, the cost is warranted and it makes bundles portable over different transports. Basically, we have a solid reason for filesystem scanning that could also be used as a signable target.

@stevvooe Ack. So the next step is a .proto file?

No better way to get started than with a straw man:
syntax = "proto3";

package ocf.bundle;

// BundleManifest specifies the entries in a container bundle, keyed and
// sorted by path.
message BundleManifest {

	message Entry {
		// path specifies the path from the bundle root
		string path = 1;

		// NOTE(stevvooe): Need to define clear precedence for user/group/uid/gid precedence.

		string user = 2;
		string group = 3;

		uint32 uid = 4;
		uint32 gid = 5;

		// mode defines the file mode and permissions. We've used the same
		// bit-packing from Go's os package,
		// http://golang.org/pkg/os/#FileMode, since they've done the work of
		// creating a cross-platform layout.
		uint32 mode = 6;

		// NOTE(stevvooe): Beyond here, we start defining type specific fields.

		// digest specifies the content digest of the target file. Only valid for
		// regular files. The string is formatted as <alg>:<digest hex bytes>.
		string digest = 7;

		// target defines the target of a hard or soft link, relative to the
		// bundle root.
		string target = 8;

		// specifies major and minor device numbers for charactor and block devices.
		string major = 9;
		string minor = 10;

		message XAttr {
			string name = 1;
			string value = 2;
		}

		// xattr provides storage for extended attributes for the target resource.
		repeated XAttr xattr = 11;

		message ADS {
			string name = 1;
			bytes value = 2;
		}

		// ads stores one or more alternate data streams for the given resource.
		repeated ADS ads = 12;
	}

	repeated Entry entries = 1;
}

@stevvooe looks pretty good. Two questions:

What is an ADS?
Should digest be repeated so we can deprecate old hashes and upgrade to new ones over time?

What is an ADS?

This is NTFS equivalent of extended attributes (sort of), known as "Alternate Data Streams". The semantics are slightly different, so I've pulled it out into a separate type. Notice the use of type  bytes for the value, instead of string. I'd like to get some feedback from a Windows expert to see if this is sufficient.

Should digest be repeated so we can deprecate old hashes and upgrade to new ones over time?

In this case, I don't see why not.
I've updated the comment in-line.

@philips We may also want to define an exclusion operator to the manifest specification, since it operates at the bundle level.
enum Op {
	// EXCLUDE specifies that the matched files should be explicitly exlcuded
	// from the manifest. They may be still part of the bundle.
	EXCLUDE = 0;

	// INCLUDE specifies that the resource should be include in the manifest.
	// This has the act of "pinning" the resource. For example, if the resouce
	// is later matched by an exclude statement, it will still be included.
	INCLUDE = 1;
}

message PathSpec {
	Op operation = 1 [default=EXCLUDE];

	// path specifies a path relative to the bundle root. If the path is a
	// directory, the entire tree will be excluded.
	string path = 2;

	// pattern specifies a glob to match resources and apply the operation.
	string pattern = 3;
}

// path specifies the bundle paths covered by the manifest. Specifications are
// order by precedence. For a given path, only the first matching
// specification applies. During processing, it is important to fully process
// all resources, even if a directory is excluded, since child resources may
// first match an inclusion.
repeated PathSpec pathSpec;

Benefits:

Very explicit for recalculation
Allows us to catch files that may have showed up and don't belong

Cost:

More processing time
Subtle include/exclusion behavior to minimally specify file sets
Can contribute to instability of file -- equivalent manifests may have different specs

Another possibility is to allow this to be specified on the command line when first building the manifest. That doesn't allow us to catch "extra" files, but that may not be that important and likely doesn't warrant the extra complexity.

Okay, so after reading this, I think this makes a bit more sense to me . . .  I think the area where I'm having trouble is understanding the scope vis a vis the goals...
I think what I've read so far is well thought out and reasoned, so props to everyone , heh.  Forgive me ahead of time, but I wanted to share some thoughts.  Discard if you don't think they are useful.
There seems a basic consensus -- in the limited info I've read  on the container crypto goals --that everyone would probably be on board with the following ideas /in principle/ , certainly as options:
Basic Themes

Containers may need to be secured against A-MITM in-transit (integrity)
Containers may need to be secured against A-MITM during distribution (integrity)
We need a way of validating and identifying who is the author of a container (authentication, non-repudiation, integrity)
Containers do NOT need their own transport layer, since we are assuming TLS / OpenSSH will take care of that .
<--- Additional reasons go here --->

Just as general thought, first I'm going to put these here just as problems I have run into myself with design of cryptography...  You guys may know this already but writing this down helps me organize my thoughts.
I'll be back with specifics tomorrow or Saturday.
Statements in general I've found true in crypto engineering.  May be useful in this context.

Never reinvent the wheel in crypto -- this is the 'once upon a time' for systems that get owned

If the full description & implementation of yr system can withstand the scrutiny of a 16-year old Norwegian hacker named Jon , such that it cannot be cracked in 8 lines of Perl, then you are off to a decent start. (unlike DeCSS)

Always aim for zero-knowledge even though it's near impossible to do right... The line of thinking provides insight regardless.

Never put new / novel cryptography and/or standards directly into production -- if necessary , only do so if it's backed by scientific research and a rock-solid implementation

Don't do an implementation from scratch -- always use an open-source well-tested library like NaCl, PyCrypto, etc.

Side channel attacks are not theoretical (Heartbleed, WEP, Lucky13, etc), so pay attention to the leaky system borders where side channels live.

Work at the highest level possible, in primitives/concepts rather than in methods

Use nonces, timestamps, & digital signatures wherever possible to avoid replay attacks

Correct forward operation is Compress, Encrypt, then MAC/Authenticate  (never the opposite)... The correct reverse operation is MAC/Authenticate, Decrypt, then Decompress.  ("Cryptographic Doom Principle" for side channels)
http://www.thoughtcrime.org/blog/the-cryptographic-doom-principle/

Study existing protocols... always adapt an existing format or protocol instead of writing one from scratch

Scope creep in crypto can be brutal, it goes on forever

- Define what your threat model before you do anything else.  Who are we trying to protect against?
What are we trying to protect (data, secrets, access, etc)?  What is our tolerance in terms of a threat vs it's complexity/likelyhood?
I've got a series of questions but I'm tired at the moment -- I'll post the questions tomorrow: I hope this helps in the time being.
General Questions:

What is our threat model? Who is the attacker?  What are their capabilities/
What does a passive attack look like (if one is possible here)?
What form is an active attack?  What is the weakest link in the existing thinking?
Do we need Digests?  What threat do they address and how?
Do we need Digital Sigs / MACs?   How are Digital Sigs keyed?  What threat do they address?
Do we have a line of thinking for the key distribution model?  WoT / CA etc.  Here it's often there's no choice but that we must piggyback on an existing implementation like OpenPGP (like Ubuntu PPA)
Based on the threat model, what are the appropriate countermeasures (aka crystallographic primitives)?
Do we need to support encryption as a use-case, or can we cut it with just hashes / digital sigs?

Specific Questions (assuming key distribution is solved, heh... like with OpenPGP):

Should we deploy Digital Signatures to secure 'single-file' compressed container images (like rkt does now)?

Do we know if containers will always (or at least almost all of the time) be distributed on the Internets as some sort of compressed single-file archive ( exact format is irrelevant)?

If containers are distributed as a single-file archive, and these are secured properly with digital sigs and/or GPG etc, why is it necessary (or beneficial) to secure the data-at-rest in an uncompressed container?

Has a container format been finalized or standardized beyond the json manifest?  What is certain about the container archive format, and what is up in the air?  What will be up to the implementation, and what is defined by the spec?

What are the  advantages of securing data-at-rest inside the container vs simply securing a container archive image file?  Do we gain security here?  If so, what specific attack does this prevent?

What are the pitfalls in scope creep and/or other impacts of securing data beyond a single-file container archive?

Are there any compromise solutions that lie between these two ideas?

What is the threat model we are considering?

Does someone trick me into downloading a malicious container?

Or does someone poison the ARP table at a starbucks?

Are we protecting against a 'runaway app' inside a container?  A malicious actor on the host system which has done privilege escalation?  Industrial espionage by ceiling cat?

What sort of primitives do we need to achieve stated level of security?

What can we re-use to minimize the effort and make it easy (or certainly at least straightforward) for others to implement this standard?

What can we re-use to avoid implementation , testing, and possible exploit headaches?

this fileset would be a binary packed format, like CrAU?

@vbatts https://github.com/stevvooe/continuity has been opened up to continue this research.

Probably related to #302. Will need to be considered as part of that.

I think the https://github.com/GNOME/ostree format has a lot of advantages.  It was designed from the start to be checksummed.  If implementing anything else, at least study it.

For example:

It intentionally does not include device files, because why would you have devices in container images?
It doesn't include timestamps per file, because immutable containers don't need them.  (And if you do need timestamps, just do what git does and derive them from the commit object timestamp).
xattrs are part of the per-file checksum (Although I think container images shouldn't include xattrs, we should drop setuid binaries and file caps for more secure containers)

@cgwalters We have been researching a number of approaches while working on https://github.com/stevvooe/continuity. The big difference is that continuity does not prescribe a distribution format while keep metadata transport consistent.

It intentionally does not include device files, because why would you have devices in container images?

We've found that having an opinion here will work the system into odd chicken-egg problems. For example, if we rely on runc to create a device, how do we specify the ownership parameters in the archive format? We'd have to call into runc to create the devices, then call back out to the archiver to apply the metadata, then back into runc for runtime.
There are also other filesystem objects, such as sockets and named pipes, that may need to be serialized when migrating a process.

It doesn't include timestamps per file, because immutable containers don't need them.

We've gone back and forth on this requirement. The main issue here is that if you want stable regeneration, you cannot have timestamps in the metadata. However, let's say you want to pick a compilation process mid build and then resume it on another node. Modification times are very important here. When you start examining this, there are a number of applications that would behave in odd manners when all of the timestamps are from the extraction time.
Mostly, we can obviate this need by not trying to regenerate an expanded artifact. IMHO, it imposes challenging requirements on the transport format that don't ultimately serve the user while introducing security problems in the pursuit of hash stability (see: tarsum).

(And if you do need timestamps, just do what git does and derive them from the commit object timestamp).

Interesting. I did not know this. Very cool!

xattrs are part of the per-file checksum (Although I think container images shouldn't include xattrs, we should drop setuid binaries and file caps for more secure containers)

We have this in continuity to some degree. There are lots of applications that cannot work correctly without xattrs, in addition to setups that require setuid.
In the past few weeks of development and experimentation, we've actually found the right model is to have continuity collect as much information as possible, then provide tools to selectively apply metadata and verify the on disk data.

On Fri, Feb 12, 2016, at 03:00 PM, Stephen Day wrote:

We've found that having an opinion here will work the system into odd
chicken-egg problems. For example, if we rely on runc to create a
device, how do we specify the ownership parameters in the archive
format? We'd have to call into runc to create the devices, then call
back out to the archiver to apply the metadata, then back into runc
for runtime.

Any non-privileged container should only see the "API" devices
(/dev/null etc.)  Any privileged container is, well, privileged and
can create the device nodes itself.  Why would you ship pre-created
device nodes in an in an image?

There are also other filesystem objects, such as sockets and named
pipes, that may need to be serialized when migrating a process.

Migration is data, not images.  Use tar or whatever for that.  And
data should be cleanly separated in storage from the image.

It doesn't include timestamps per file, because immutable containers
don't need them.

We've gone back and forth on this requirement. The main issue here is
that if you want stable regeneration, you cannot have timestamps in
the metadata. However, let's say you want to pick a compilation
process mid build and then resume it on another node. Modification
times are very important here. When you start examining this, there
are a number of applications that would behave in odd manners when all
of the timestamps are from the extraction time.

Again, that's a data case, not immutable images.  I think using
container images as a backup format doesn't make sense.  A vast
amount of backup software already exists.  Yes, one needs to cleanly
separate code from data, but that's a fundamental requirement for
upgrades anyways.

@cgwalters I am not sure if you saw it, but I made the following point at the bottom of my comment:

we've actually found the right model is to have continuity collect as much information as possible, then provide tools to selectively apply metadata and verify the on disk data.

This approach is compatible with all of the points identified, while not limiting the capability of containers.
In general, images are data, as well. Indeed, a large number of backup software and distribution channels for filesystem images do already exist. Why not make an archive format that is compatible with all of them? Conversely, why require a backup solution in addition to the ability to snapshot and archive containers? Both are acceptable use cases at either end of a continuum. It would be unfortunate to disallow one based on an arbitrary opinion, even if well-grounded.
Ultimately, deciding what a container or image archive can and cannot do just isn't productive. Shipping metadata is inexpensive and the user can always choose to unpack them or not.

In one view, sure it's all "just files".  But I think there's a strong argument to have separate tools and data formats for different problem domains (source code, binaries, database backups) that share ideas rather than trying to do one format for everything.  git is already good for source code and text, etc.
Don't underestimate the cost of inventing a new file format for things like mirroring, versioning, language bindings for parsers, etc.

Going back to the top of the motivation here:

Problem: a rootfs for a container bundle sitting on-disk may not reflect the exact intended state of the
bundle when it was copied to its current location.

I'd say the correct solution here is for the container runtime to work with the storage layer to ensure immutability.  See http://www.spinics.net/lists/linux-fsdevel/msg75085.html for a proposal there.  It'd require plumbing through from the filesystem to the block level, but I think the end result would be simply better than classic tools like tripwire and IMA, as well as whatever verification is invented here.  (Yes, that proposal doesn't cover xattrs, we'd want a way to freeze specific xattrs too likely)

@cgwalters Is there a windows port for OSTree?

On Thu, Feb 25, 2016 at 01:30:26PM -0800, Colin Walters wrote:

Going back to the top of the motivation here:

Problem: a rootfs for a container bundle sitting on-disk may not
reflect the exact intended state of the bundle when it was copied
to its current location.

I'd say the correct solution here is for the container runtime to
work with the storage layer to ensure immutability.

The O_OBJECT proposal you link is about preserving filesystem content
after it lands on the filesystem, but it looks like @philips initial
concern was about landing it on the filesystem in the first place.
For example, “my FAT-16 filesystem doesn't support POSIX permissions,
so my rootfs/foo/bar seems to have 0777 instead of the source's 0600”.

I struggle to understand a scenario where one would reasonably want to unpack container content onto FAT-16 and expect to run it.  Maybe inspection, but even then, you can do that from userspace easily enough with libarchive or whatever.  If you have a Linux container, you have Linux...hence you have xfs/ext4/etc.

On Thu, Feb 25, 2016 at 03:28:36PM -0800, Colin Walters wrote:

I struggle to understand a scenario where one would reasonably want
to unpack container content onto FAT-16…

A poor choice of example, but @philips was pointing out that not all
filesystems support the same attributes (he pointed out NFS without
xattrs, among other things).  Regardless of the specific examples,
unpacking into a local filesystem (what @philips was talking about
1) and maintaining content after that unpacking (what you were
talking about 2) are two separate things.

Anyways my goal here is to try to ensure sharing of ideas, not necessarily code in this area - OSTree is certainly not going to take over the world as a way to get content from A to B any more than other projects in this area.  Another good project to look at is Clear Linux:  https://lists.clearlinux.org/pipermail/dev/2016-January/000159.html
A good example of a mistake in OSTree - I've come to realize the git-like Merkle tree model was a mistake for binary content, because it's really common with software updates for one "package" to change multiple paths (due to /usr/bin and /usr/share etc.)  For git and source code it's a lot more common to only change one subdirectory.
So the Clear Linux manifest makes sense - there's no streaming, but that's fine because we aren't storing huge amounts of content to tape drives.
Also, OSTree not including the size in the tree metadata was really dumb but that's papered over with static deltas.
Speaking of deltas...that's another area where Docker really lacks, and for OSTree I ended up taking a ton of inspiration from http://dev.chromium.org/chromium-os/chromiumos-design-docs/filesystem-autoupdate.  For more on that see https://ostree.readthedocs.org/en/latest/manual/formats/

Regarding NFS...sure, but how does it help a user/admin to determine after the fact that things are broken?  Basically the system is either going to munch the fscaps on /bin/ping or not, a system that tells you "hey the fscaps are missing" may lead you Google faster but that's about it...
fscap binaries can be pretty easily worked around in an NFS root scenario by copying them into tmpfs or something on boot.     Use containers as a reason to leave behind the continual security issues of setuid, and just have them on the host until someone rewrites PAM and /sbin/unix_chkpwd etc.

On Thu, Feb 25, 2016 at 06:31:09PM -0800, Colin Walters wrote:

Regarding NFS...sure, but how does it help a user/admin to determine
after the fact that things are broken?  Basically the system is
either going to munch the fscaps on /bin/ping or not, a system
that tells you "hey the fscaps are missing" may lead you Google
faster but that's about it...

Agreed if the goal is going image → filesystem → running container,
but I think @philips was concerned with round-tripping from image
files to filesystem bundles (image 1 → filesystem → image 2), since he
links tar-split which is focused on unpacking and repacking tarballs
while preserving the tarball's hash.  Folks that are interested in
round-tripping through the filesystem would be concerned about
mismatches between attributes represented in the filesystem and
attributes represented in the image file, but not about freezing
content once it's on the filesystem.  And folks that want to
round-trip in the face of limited filesystems can write tools that
stash the unsupported attributes elsewhere and pull them back in when
checking for changes, so they can do better than failing fast.
Personally, I don't think round-tripping is particularly useful,
because:

Folks who just want to verify a bundle can fetch the original image
1 (e.g. by caching it locally) so they don't have to regenerate
the original image from the filesystem.

Folks who want to generate a new image that reuses content
addressable objects from an earlier image (e.g. adding a few files
to a stock Debian image to create a new image) can handle that
locally (e.g. with something like Git's staging area to bless
changes they're interested in).  There's no need to address this at
the protocol / file-format level.

 Subject: Re: OCI Bundle Digests Summary
 Date: Thu, 15 Oct 2015 16:52:42 -0700
 Message-ID<EMAIL_ADDRESS>

I am closing this out. The image format work is now part of the OCI Image Format project: https://github.com/opencontainers/image-spec

I have an aside question: if there is a bug Undefined syscall.TIOCGPTN, syscall.TIOCSPTLCK during compilation of cri-o (gollvm related)
 , related to https://github.com/containerd/containerd/tree/master/vendor/github.com/containerd/continuity/fs - where should I open issues/report ?
Ivan

@advancedwebdeveloper source code of that package is in https://github.com/containerd/continuity


Add system-component blacklist documentation

Let me know if you want any changes!

Codecov Report

Merging #252 into master will not change coverage.
The diff coverage is n/a.

@@          Coverage Diff          @@
##           master   #252   +/-   ##
=====================================
  Coverage     100%   100%
=====================================
  Files           5      5
  Lines         183    183
=====================================
  Hits          183    183

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a131f69...767593f. Read the comment docs.

Thanks for the PR! But I don't think that I want to support the blacklist prop as public API. It might work, but I wouldn't rely on it. I think this is an issue that should be fixed upstream in the styled-components project, or if you're defining a lot of custom style props, you can try using the clean-tag package instead


integrate zksync pacakges

with zksyncs launch of the testnet - we can officially start to integrate the contracts into zksync.
making note of this for now

No longer applicable


“Impossible” composed schemas raise no type error but fail at runtime

What version of Effect is running?
2.1.2
What steps can reproduce the bug?
This schema t seems obviously impossible (no information is given on how to transform between foo: bar and x: y), but below is valid at compile time:
const t = S.compose(S.struct({ foo: S.literal('bar') }), S.struct({ x: S.literal('y') }));

// These will throw:
S.decodeSync(t)({ foo: 'bar' })
S.decodeSync(t)({ x: 'y' })
S.encodeSync(t)({ x: 'y' });

What is the expected behavior?
Based on behaviour of Effect schema in other regards, it generally reports a lot of similar errors at compile time. (Though I suppose the composed schema may be a trickier case?)
What do you see instead?
Errors thrown at runtime:
Uncaught Error: ({ foo: "bar" } <-> { x: "y" })
└─ To side transformation failure
   └─ { x: "y" }
      └─ ["x"]
         └─ is missing
Uncaught Error: ({ foo: "bar" } <-> { x: "y" })
└─ From side transformation failure
   └─ { foo: "bar" }
      └─ ["foo"]
         └─ is missing
Uncaught Error: ({ foo: "bar" } <-> { x: "y" })
└─ From side transformation failure
   └─ { foo: "bar" }
      └─ ["foo"]
         └─ is missing

We can add an optional { strict: false } parameter as we did with transformOrFail:
import * as S from "@effect/schema/Schema"

declare const compose: {
  <B, C>(bc: S.Schema<B, C>): <A>(ab: S.Schema<A, B>) => S.Schema<A, C>
  <C, D>(bc: S.Schema<C, D>, options: { strict: false }): <A, B>(ab: S.Schema<A, B>) => S.Schema<A, D>
  <A, B, C>(ab: S.Schema<A, B>, cd: S.Schema<B, C>): S.Schema<A, C>
  <A, B, C, D>(ab: S.Schema<A, B>, cd: S.Schema<C, D>, options: { strict: false }): S.Schema<A, D>
}

// @ts-expect-error
compose(S.struct({ foo: S.literal("bar") }), S.struct({ x: S.literal("y") }))

// ok
compose(S.struct({ foo: S.literal("bar") }), S.struct({ x: S.literal("y") }), { strict: false })

closed this as completed in https://github.com/Effect-TS/effect/pull/1922


[GOBBLIN-1504] add a flag to indicate the start of cluster manager

Dear Gobblin maintainers,
Please accept this PR. I understand that it will not be reviewed until I have checked off all the steps below!
@jack-moseley @Will-Lo  please review
JIRA

[x] My PR addresses the following Gobblin JIRA issues and references them in the PR title. For example, "[GOBBLIN-XXX] My Gobblin PR"

https://issues.apache.org/jira/browse/GOBBLIN-1504

Description

[x] Here are some details about my PR, including screenshots (if applicable):
Adding a few job monitor metrics.
Changed a couple of metrics from counter to meter to be able to measure the rate of change instead of the absolute value.
These metrics are initialized after the GobblinClusterManager starts, so added a flag in GobblinClusterManager to indicate the start, otherwise downstream applications may use uninitialized metrics.

Tests

[x] My PR adds the following unit tests OR does not need testing for this extremely good reason:
trivial changes

Commits

[x] My commits all reference JIRA issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "How to write a good git commit message":

Subject is separated from body by a blank line
Subject is limited to 50 characters
Subject does not end with a period
Subject uses the imperative mood ("add", not "adding")
Body wraps at 72 characters
Body explains "what" and "why", not "how"

Codecov Report

Merging #3350 (861f542) into master (f468bb0) will decrease coverage by 0.00%.
The diff coverage is 11.11%.

@@             Coverage Diff              @@
##             master    #3350      +/-   ##
============================================
- Coverage     46.42%   46.42%   -0.01%
  Complexity    10115    10115
============================================
  Files          2052     2052
  Lines         79626    79639      +13
  Branches       8889     8889
============================================
+ Hits          36970    36971       +1
- Misses        39216    39230      +14
+ Partials       3440     3438       -2

Impacted Files
Coverage Δ

...bblin/runtime/job_monitor/KafkaAvroJobMonitor.java
0.00% <0.00%> (ø)

...e/gobblin/runtime/job_monitor/KafkaJobMonitor.java
0.00% <0.00%> (ø)

...apache/gobblin/runtime/metrics/RuntimeMetrics.java
0.00% <ø> (ø)

.../apache/gobblin/cluster/GobblinClusterManager.java
53.91% <100.00%> (+0.42%)
:arrow_up:

...a/org/apache/gobblin/util/limiter/NoopLimiter.java
40.00% <0.00%> (-20.00%)
:arrow_down:

...lin/util/filesystem/FileSystemInstrumentation.java
92.85% <0.00%> (-7.15%)
:arrow_down:

...lin/restli/throttling/ZookeeperLeaderElection.java
70.00% <0.00%> (-2.23%)
:arrow_down:

.../org/apache/gobblin/cluster/GobblinTaskRunner.java
63.46% <0.00%> (-0.33%)
:arrow_down:

.../apache/gobblin/runtime/api/JobExecutionState.java
80.37% <0.00%> (+0.93%)
:arrow_up:

...lin/elasticsearch/writer/FutureCallbackHolder.java
62.85% <0.00%> (+1.42%)
:arrow_up:

... and 2 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f468bb0...861f542. Read the comment docs.

@sv2000 please review/merge

@autumnust please review/merge

Looks like this PR is mostly about adding new metrics, rather than adding a flag. I suggest to update PR title to reflect this.

moved that code out of this PR


Type "@timestamp"→"@timestamp"

bold with escape characters
https://docs.microsoft.com/en-us/sql/reporting-services/subscriptions/file-share-delivery-in-reporting-services?view=sql-server-2017

@hyoshioka0128 : Thanks for your contribution! The author(s) have been notified to review your proposed change.

@hyoshioka0128 Thank you!


I only can upload sketch once

I only can upload sketch once...
after i uploaded the bootloader...

ok...it is all done...!


Should error title/detail be translated ? Should they be displayed to users ?

Hi,
As the error title is a human readable explanation, what if the user doesn't speak the language of the developer of the API ? so should the errors be translated ?
If yes, can the client use them to display the error to the users, or is it just functional and the client should only rely on the code to display the cause of the error in the appropriate language.
Finally, If the error should be translated (actually even if not !), I think the client should always send the Accept-Language header to tell the server in which language the error should be translated, or even if data can only be accessed to certain language, or if a data field can be translated.

Title and detail can both be translated, and this translation can be governed by Accept-Language. I think the specification alludes to this possibility for title, since it says that the value should always be the same "except for purposes of localization" (which implies that it can be localized). However, the definition of the details member doesn't allude to localization, so I'll put together a PR to fix that soon.
As far as translating data fields goes (i.e. fields in resource objects), that's something you probably shouldn't do. Instead, the different translations should have different JSON API resource objects with different ids. Then, you can (optionally) link the user to the alternate versions (using, e.g., the HTTP Link header or a JSON API relationship). The reason for not translating the data fields stems from the fact that JSON API doesn't have the same separation between a resource and a representation that HTTP does; in JSON API, a resource (object)'s fields are its state.


refactor: remove heredoc related code from StringLiteral formatter

Because all heredoc are StringInterpolation even if it has no interpolation.

ping. It is simple dead code removing.


Not running after pylint installation and configuration

Hi.
I did a clean install of python, after I installed the pip and after the pylint.
All of them running without problems in command line prompt and after installed this extension in my Brackets I inserted the pylint location in the brackets.json configuration file like this:
"pylinter.pylintPath": "C:\\Users\\leand\\AppData\\Local\\Programs\\Python\\Python36-32\\Scripts\\pylint.exe",
"pylinter.outputPattern": "{msg_id} > {msg} [{symbol} @ {line},{column}]",

after I did it no more error messages of extension running fail but when I open a .py file is not showing nothing about pylint.
I run the pylint from command line in the same file and it showed messages, but it still not working in Brackets.
My Brackets version:
sprint 8 version 1.8.0-17108 (alf_localization_release_1.8 3af64fae4)
timestamp version: Wed Nov 02 2016 12:07:29 GMT+0000
Thanks.

Hi @trombini77,
I'm sorry to hear that you're having issues with my extension. I'll do my best to help you resolve it.
Things are working fine on my Windows VM (even after updating the Python and Pylint versions). To help me further test, can you provide a gist of the code that you're running into issues with?
Also, can you share with me the output of the following command?:
C:\Users\leand\AppData\Local\Programs\Python\Python36-32\Scripts\pylint.exe --version

Thanks and happy new year,


'ME' contact missing??

I've just noticed that the contact representing 'ME'  (or 'My Local Profile') isn't being returned in android in my app.  Is this contact some sort of 'special' one that isn't stored with the others?

Fixed in #54

Hey, could someone take a quick look at #55 - i've added support for getting the details of the 'profile' (or 'me') contact in Android.


Only assert events in assert_* functions

Turns out we call this function in some Polkadot benchmarks and it makes them panic. Putting this check only in the explicit functions makes more sense anyway IMO and fixes that.
Failing CI example https://gitlab.parity.io/parity/mirrors/polkadot/-/jobs/2320642

The benchmark is failing here:

Yea, actually the benchmark test is failing.

And the general solution should be to fix the benchmark to switch to block 1 instead of changing this code here.

IMO it is a bit unexpected when System::events() panics instead of returning nothing, but I can also change the bench.

Okay here https://github.com/paritytech/polkadot/pull/6635 @bkchr

@conatus-est @marebranza please look into work chat to prevent your account from being removed from the org.


Can't install

I'm using Windows 11
Despite trying to take down my antivirus(es) the computer is not letting me install UVR5
Antiviruses are McAfee and default Windows. Computer does not let me disable McAffee even for a few minutes, the option does not exist and even if trying with the Task Manager, it says I don't have permission to do that.
I used UVR5 in two other machines (modded Win10 called DeltaOS, and Win11 updated from Win10) with no issues, however I had to move a brand new one that is giving this error nonstop and puts the installer in quarintine over and over again, saying it has a virus.
Is there any .zip portable version of UVR5 and patches? I can't build it from source code.

I'd uninstall McAfee and go for something else, it's not a solution to begin with. Avast is free, and if AV comparatives is anything to go by, second best for Windows.

I'd uninstall McAfee and go for something else, it's not a great av solution to begin with. Avast is free, and if AV comparatives is anything to go by, is at the top for Windows solutions.

I despise Avast with all my soul haha...
I couldn't deactivate McAfee but I could let it "stop filtering" UVR5, It definetely works now


chore(deps): update tungstenite, tokio-tungstenite

Update tungstenite and tokio-tungstenite to 0.18.0

Thanks!

Released as v0.9.0 :rocket:  Thanks again!


compute_success function error

In Julien\Data\head-fixed\_Other\test_folder\by_session_folder\JC316L-2022-12-19-094241\pycontrol\JC316L-2022-12-19-094241.txt, the task is reaching_go_spout_bar_dual_all_reward_dec22, the compute_success function from
https://github.com/juliencarponcy/trialexp/blob/3672774bd1c0a1a861b1b6f3f13065f1f29e11dc/trialexp/process/data_import.py#L735-L745
assume there is a waiting_for_spout event in that task but actually there is none according to the event definition in the task file.

Seems to have fixed in dev


Add overflow ellipsis for one-time-many-results in logs.

More lines in one log will cause problem.

As of the new UI, this should be resolved.


Sticky not working in Semantic-UI React

I'm using v6.0.1 of Sticky. I'm also using React Semantic-UI. The following was working in the previous version of sticky, but now is not working. If disableCompensation is set to true, the sidebar doesn't scroll at all. If set to false, it jumps to the middle of the screen (the height of the compensation, around 500px) when sticky is activated, and thereafter stops moving. Not sure what the issue is. I've tried a number of different settings and style configurations but I can't get it to behave properly.
            <Grid className="Article" style={{ backgroundColor: 'white' }}>
                <Grid.Column width={9} floated="left">
                    {this.renderSections()}
                </Grid.Column>
                <StickyContainer style={{ width: '43%' }}>
                    <Grid.Column width={7} floated="right">
                        <Sticky disableCompensation={true}>
                            {({isSticky}) => {
                                console.log({isSticky});
                                return <Sidebar sidebarImage={this.props.sidebarImage} />
                            }}
                        </Sticky>
                    </Grid.Column>
                </StickyContainer>
            </Grid>

Here is a screen capture of the effect.
http://d.pr/i/R0iZGN
If you need more information, I'd be happy to supply it.

You need to add style prop to sticky component. Then you can pass it to sidebar as style attribute. Like this;
<Sticky disableCompensation={true}>
    {({style, isSticky}) => {
        console.log({isSticky});
        return <Sidebar sidebarImage={this.props.sidebarImage} style={style} />
    }
 </Sticky>

After that, use dynamic style variable as you want.


Music, lobby art and other server customizations need to be moved to a different resource source

Description
Lobby music and art is currently stored on this repo. This means servers pulling the game need to pull all the art and music, and Wizden music/art defaults because the assumed "default" across the growing SS14 space. But we know that both of these resources are really commonly customized by downstreams, especially downstreams who do not have to be addicted to the sweet, delcious copyright safety that we do.
Move music and large art into a server resource source that can be migrated off-repo. This would allow different server owners to easily customize their server lobby feel without needing repo edits.
Consider also migrating Wizden setup off-repo as part of this.

i really dont see any issue here

Yeah its a great idea, I think what also is needed is a default placeholder image that tells the person that just forked this repo "Hey! Add your own lobby images/music/jukebox here:" And then point to how they can do it


Fix stuck with an empty view in case of access to an invalid path

What this PR does / why we need it:
Before
https://user-images.githubusercontent.com/32532742/135503698-07765063-55f3-4ec6-ae13-0ee6efe07340.mp4
After
https://user-images.githubusercontent.com/32532742/135503723-38c07e2d-a433-4376-9afa-1461ef2a32bb.mp4
Which issue(s) this PR fixes:
Fixes #
Does this PR introduce a user-facing change?:

Avoid showing an empty view in case of access to an invalid path

Should we redirect to applications page or show an error page?

Should we redirect to applications page or show an error page?

I think both are good and reasonable but redirecting is easier since we don't need to implement an error page.

redirecting is easier since we don't need to implement an error page

Sure. Let's apply this for now.

/lgtm

Reasonable definitely
/approve


Logo?

Any ideas for a Learnmonkey logo?

for now, leave it as the monkey emoji

oof make the monkey look smart at least


ERROR: Failed to create instance: unexpected error when creating modelInstanceState: maxTokensInPagedKvCache must be large enough to process at least 1 sequence to completion (i.e. must be larger than beam_width * tokensPerBlock * maxBlocksPerSeq)

I referred this official blog to run the llama2 7b chat model
python3 tools/fill_template.py --in_place \
      all_models/inflight_batcher_llm/tensorrt_llm/config.pbtxt \
      decoupled_mode:true,engine_dir:/all_models/inflight_batcher_llm/tensorrt_llm/1,\
max_tokens_in_paged_kv_cache:,batch_scheduler_policy:guaranteed_completion,kv_cache_free_gpu_mem_fraction:0.2,\
max_num_sequences:4

python tools/fill_template.py --in_place \
    all_models/inflight_batcher_llm/preprocessing/config.pbtxt \
    tokenizer_type:llama,tokenizer_dir:meta-llama/Llama-2-7b-chat-hf

python tools/fill_template.py --in_place \
    all_models/inflight_batcher_llm/postprocessing/config.pbtxt \
    tokenizer_type:llama,tokenizer_dir:meta-llama/Llama-2-7b-chat-hf

docker run -it --rm --gpus all --network host --shm-size=1g \
-v $(pwd)/all_models:/all_models \
-v $(pwd)/scripts:/opt/scripts \
nvcr.io/nvidia/tritonserver:23.10-trtllm-python-py3

# Log in to huggingface-cli to get tokenizer
huggingface-cli login --token *****

# Install python dependencies
pip install sentencepiece protobuf

when i ran the final command
python /opt/scripts/launch_triton_server.py --model_repo /all_models/inflight_batcher_llm --world_size 1
I'm getting the below error
[TensorRT-LLM][INFO] Using 693 tokens in paged KV cache.
E1128 16:31:49.479096 227 backend_model.cc:634] ERROR: Failed to create instance: unexpected error when creating modelInstanceState: maxTokensInPagedKvCache must be large enough to process at least 1 sequence to completion (i.e. must be larger than beam_width * tokensPerBlock * maxBlocksPerSeq)
E1128 16:31:49.479182 227 model_lifecycle.cc:621] failed to load 'tensorrt_llm' version 1: Internal: unexpected error when creating modelInstanceState: maxTokensInPagedKvCache must be large enough to process at least 1 sequence to completion (i.e. must be larger than beam_width * tokensPerBlock * maxBlocksPerSeq)
I1128 16:31:49.479201 227 model_lifecycle.cc:756] failed to load 'tensorrt_llm'
E1128 16:31:49.479285 227 model_repository_manager.cc:563] Invalid argument: ensemble 'ensemble' depends on 'tensorrt_llm' which has no loaded version. Model 'tensorrt_llm' loading failed with error: version 1 is at UNAVAILABLE state: Internal: unexpected error when creating modelInstanceState: maxTokensInPagedKvCache must be large enough to process at least 1 sequence to completion (i.e. must be larger than beam_width * tokensPerBlock * maxBlocksPerSeq);
I1128 16:31:49.479354 227 server.cc:592]

You can try setting larger kv_cache_free_gpu_mem_fraction or max_tokens_in_paged_kv_cache.
Or try using smaller beam_width or input_len/output_len when you build engine.

When you setup kv_cache_free_gpu_mem_fraction and max_tokens_in_paged_kv_cache, TRT LLM will choose smaller than and I guess max_tokens_in_paged_kv_cache:500 is not enough. Could you try only setting kv_cache_free_gpu_mem_fraction first.

I also tried just setting the kv_cache_free_gpu_mem_fraction :1
But no luck

working.
fill_template.py is not update the values if running for second time.
I changed the kv_cache_free_gpu_mem_fraction :0.8 in the config file directly.
Changing directly in config.pbtxt fixed the issue

Thanks for updating. Close this bug as it is solved. Feel free to reopen it if needed.

Thanks for updating. Close this bug as it is solved. Feel free to reopen it if needed.

how is it solved? in which commit or pr?

Same error when launching baichuan 1 13B model. I tried to change the value of maxTokensInPagedKvCache in config file to a large number (100000000) but still failed. (beam width is 1, tokens per block is 64, not found the value of max blocks per seq)

There is no bug. The issue is caused by wrong settings. When you setup maxTokensInPagedKvCache and kv_cache_free_gpu_mem_fraction, trt llm will choose smaller one automatically. So, you need to check you set both number well or only setup one of them.

Hi @byshiue, @NarenZen ,
I understand this is a config issue however I am getting the same error  with on 2x A10 GPU machine with total 48GB. Is it supposed to work on A10/this configuration?
[TensorRT-LLM][INFO] Allocate 872415232 bytes for k/v cache.
[TensorRT-LLM][INFO] Using 1664 total tokens in paged KV cache, and 20 blocks per sequence
E0120 10:37:25.134537 24459 backend_model.cc:635] ERROR: Failed to create instance: unexpected error when creating modelInstanceState: maxTokensInPagedKvCache (1664) must be large enough to process at least 1 sequence to completion (i.e. must be larger than beam_width (1) * tokensPerBlock (128) * maxBlocksPerSeq (20))

My configuration from tensorrt_llm/config.pbtxt

kv_cache_free_gpu_mem_fraction : 0.9
max_tokens_in_paged_kv_cache : Commented out as I understand only one is sufficient.
max_attention_window_size:2560
max_batch_size: 8
decoupled: True
batch_scheduler_policy:guaranteed_completion
gpt_model_type:inflight_fused_batching
max_num_sequences:1

Could you please help if I am missing something here?
Thanks.

@amir1m, I'm having the same issue. Granted, I'm doing this on a measly 3080 10GB, but I can't get inference working with Triton. I can run the model without issue using the run.py script provided with TensorRT-LLM, but I keep getting the [TensorRT-LLM][INFO] Allocate 872415232 bytes for k/v cache.  [TensorRT-LLM][INFO] Using 1664 total tokens in paged KV cache, and 20 blocks per sequence E0120 10:37:25.134537 24459 backend_model.cc:635] ERROR: Failed to create instance: unexpected error when creating modelInstanceState: maxTokensInPagedKvCache (1664) must be large enough to process at least 1 sequence to completion (i.e. must be larger than beam_width (1) * tokensPerBlock (128) * maxBlocksPerSeq (20)) you're getting.
Did you ever figure out what you problem was?

python3 tools/fill_template.py is using string.Template.safe_substitute to replace values, meaning that only a value represented as "$foo" in the configuration file will be replaced.
Seems to me that in the case of ./all_models/inflight_batcher_llm/tensorrt_llm/config.pbtxt, it looks to me that everything is hardcoded except the single parameter ${enable_trt_overlap}. If you try to override anything but that, the override is just silently ignored.
That seems bad - is that really what is wanted? Or am I misunderstanding something big?

python3 tools/fill_template.py is using string.Template.safe_substitute to replace values, meaning that only a value represented as "$foo" in the configuration file will be replaced.
Seems to me that once you use fill_template.py once, everything you specified becomes hardcoded. If you try to override anything later using python3 tools/fill_template.py --in_place, the override is just silently ignored.
That seems bad - is that really what is wanted? Or am I misunderstanding something big?

you're correct. the script isn't very robust, but it gets the job done (assuming you configure things right the first time and/or backed up your original config files).
in the devs' defense, i believe there are a lot of moving parts in this space, and we are working with the early stages of TensorRT-LLM and should adjust our expectations as such.
FWIW, i've spent many hours configuring environments with Triton/TensorRT-LLM. i've run into a countless little issues, even if you faithfully follow the TensorRT-LLM documentation. there are many contradictions in the documentation, things that are out of date, pointers to things that are no longer relevant, etc. etc.
i assume NVIDIA will refine the documentation for TensorRT-LLM as their product matures. there are real performance gains to using TensorRT-LLM as a model backend, and businesses everywhere are scrambling to add LLM inference to their software offerings. i have to imagine there will be a lot more adoption soon. in my experience, it is the squeaky wheel gets that gets the grease. when more companies are looking to efficiently leverage their hardware, they'll use TensorRT-LLM+Triton.

Did someone ever figure it out? Currently facing the same issue


Broken links on runtime spec

https://specs.opencontainers.org/runtime-spec/
Links to anything under Runtime and Lifecycle or Configuration are broken.

It looks like the parser is likely choking on something here: https://raw.githubusercontent.com/opencontainers/runtime-spec/v1.0.2/spec.md. We need to add more verbosity to printing what is happening on a local deployment to debug.

The issue seems to be this line:
html = html.replace(link, {% if page.parent %}"{{ site.baseurl }}/{{ page.parent }}/" + {% endif %} basename(link).replace(".md") + "/?v=" + window.version);

Given config.md for the link text, that would replace the first occurrence in the html. The problem is that there are many redundant occurrences of the string, so likely it's replaced elsewhere and missed for the actual link.

We can see more closely what happens here, before and after parsing "config.md" via that means.
Here is before - note config.md

linux: runtime/?v=v1.0.2, config/?v=v1.0.2, config-linux.md, and runtime-linux.md.

solaris: runtime/?v=v1.0.2, config.md, and config-solaris.md.

windows: runtime.md, config.md, and config-windows.md.

vm: runtime.md, config.md, and config-vm.md.

And here is after - note config/?v=v1.0.2 where the replacement happens

linux: runtime/?v=v1.0.2, config/?v=v1.0.2, config-linux.md, and runtime-linux.md.

solaris: runtime/?v=v1.0.2, config/?v=v1.0.2, and config-solaris.md.

windows: runtime.md, config.md, and config-windows.md.

vm: runtime.md, config.md, and config-vm.md.

So the replacement does happen, but since the link text is the same as the link (the markdown name) it's replacing that instead. Suggestions for fixes welcome - I'm certainly not a web developer / frontend person.

@nate-double-u - any idea of how we broke this?

@amye I don’t think anyone did anything it’s a bug in the parser code that I pointed out above. Likely the solution is to do a more scoped replacement to only the link text in question and then replace the entire block.

I can futz around with it today - I wrote this whole template and the code in question so I’m probably best to take a first shot.

Happy to help, If you need anything let me know 🙂

Could definitely use a second set of eyes here! https://github.com/opencontainers/specs.opencontainers.org/pull/4

Fixed by #4. Thanks @vsoch!


Add the option to create a next js appilcation

Please assign this issue to me under hactoberfest. It would be a great oppurtunity for me

Go ahead!

make sure you keep the code clean and you make a new file under commands folder named createNextJs.js and import the command in index.js
write your code similar to other commands structure
the command should be devcli create <project_name> --nextjs [nextjs should be in option with devcli create]
also does this also create backend structure as well?

Yes it will create a backend structure also

Ok then, raise the pr


Cannot update user first or last name on GAM 3.7

Full steps to reproduce the issue:

Set up GAM 3.7.
Add user with:
gam.exe create user<EMAIL_ADDRESS>firstname test lastname testsingle password testtesttest
Run command:
gam update user<EMAIL_ADDRESS>lastname testmarried

Expected outcome: The test user's last name will be be changed from "testsingle" to "testmarried".
Output on GAM 3.65:
gam update user<EMAIL_ADDRESS>lastname testmarried
updating user<EMAIL_ADDRESS>
Actual outcome( on GAM 3.7 ): User's name is not changed and error below is generated.
Traceback (most recent call last):
  File "<string>", line 11132, in <module>
  File "<string>", line 10511, in ProcessGAMCommand
  File "<string>", line 6985, in doUpdateUser
  File "<string>", line 6470, in getUserAttributes
KeyError: u'name'
gam returned -1

Aside from the "firstname" and "lastname" parameters the "gam update user" command seems to work fine for other attributes.
Some things I've tried to resolve the problem:

Different permutations of the above commands, substituting usernames and emails or changing attributes to update.
Compared gam.py from 3.65 to 3.7 to check for possible typos - Ultimately seemed fruitless because of how different the two are.
Completely removing GAM from my Google Apps account and setting it up from scratch.

This is fixed in source now. I'll see about pushing out a 3.71 in a few days to cover some of these issues.

Fantastic. Awesome app and awesome support!


Version the documentation

What type of PR is this:

/kind documentation
What does this PR do / why we need it:
This versions the documentation in order to display both 2.5.0 and 3.0.0
alpha1
In this PR we:

Update the package dependencies, as there was a small bug with
docusaurus displaying versioning
Moves all documentation for 3.0.0-alpha1 into it's own folder
Makes the old 2.5.0 documentation as the most current docs

Which issue(s) this PR fixes:

Fixes https://github.com/redhat-developer/odo/issues/5377
PR acceptance criteria:

[X] Unit test

[X] Integration test

[X] Documentation

How to test changes / Special notes to the reviewer:

/lgtm

Love this little thing:

/approve

Moves all documentation for 3.0.0-alpha1 into it's own folder

@cdrage how is this going to work when someone makes changes in main branch? Where it will show up?


Update to aspnet-prerendering 2.*

Updates needed to server.js file

Closed via https://github.com/MarkPieszak/aspnetcore-Vue-starter/commit/4dca68157a8beab8eb4da5a96e692da5322d43b4


The getter 'flagUri' isn't defined for the type 'CountryCode'.

Hi,
I have got error The getter 'flagUri' isn't defined for the type 'CountryCode'. when try to use custom flag image. Have any missing here?
Thank you.

Hello 👋🏻
You can access the flagUri as a property of a CountryCode object.
Example:
    GestureDetector(
    onTap: () async {
        // Show the country code picker when tapped.
        final code = await countryPicker.showPicker(context: context);
        // Null check
        if (code != null) print(code);
       // Access flagUri property.
      print(code.flagUri);
    },
  /// Some omitted values.

Hello 👋🏻 You can access the flagUri as a property of a CountryCode object. Example:
    GestureDetector(
    onTap: () async {
        // Show the country code picker when tapped.
        final code = await countryPicker.showPicker(context: context);
        // Null check
        if (code != null) print(code);
       // Access flagUri property.
      print(code.flagUri);
    },
  /// Some omitted values.

Thank you so much. I got it. I have installed wrong version :d

Now I got error Null check operator used on a null value. Can you please help? @fernan542 Thank you

That error means that the code is null or the user didn't select from the picker.
In order to solve that, you need to check if the code is not null before you use it or any of the property.
        // Check if not null first.
        if (code != null) {
          // Here you're confident that code is not null,
          // you can access its properties.
          print(code.flagUri);
        }

Another thing, in order to help you more, I suggest you paste your code and I'll check it. ☺
Thanks!

Thank you @fernan542
This is my code to call the picker:
Future<void> _onFlagPressed() async {
    const countryPicker = FlCountryCodePicker();
    final code = await countryPicker.showPicker(context: context);
    if (code == null) return;
    setState(() {
      _countryCode = code;
    });
  }

And the picker dialog showing up, then I get the error:

What version are you using? Can you show the full error log at your console?

@fernan542 This my version: phone_number: ^2.0.0
Error log here:

Have you tried adding the localizations in your MaterialApp?
First, import the package and add an alias to prevent other import errors.
import  'package:fl_country_code_picker/fl_country_code_picker.dart' as flc;

Then add the following values at your app's MaterialApp:
MaterialApp(
	title: 'Your App',
	// Supported locales at the moment.
	// Cannot find your locale? Please make a request.
	supportedLocales: flc.supportedLocales.map((e) => Locale(e)),
	localizationsDelegates: const [
		// Package's localization delegate.
		CountryLocalizations.delegate,
		GlobalWidgetsLocalizations.delegate,
		GlobalMaterialLocalizations.delegate,
		GlobalCupertinoLocalizations.delegate,
	],
	// ... some omitted values
);

ok got it. It's working now. Thank you so much @fernan542

You're always welcome.
Thank you for using my package!


Update index.js

Adding description:("Auto-generated by YouTube.") in search query will help in better search result.

It cannot find any results if youtube music is not available in the track's region.

You should use your fork if it can give you better results.


[google_mlkit_text_recognition] Japanese text are not recognized.

Describe your issue. If applicable, add screenshots to help explain your problem.
I am running this code.
final textRecognizer = TextRecognizer(script: TextRecognitionScript.japanese);
    final inputImage = _inputImageFromCameraImage(camera, controller, image);
    if (inputImage == null) {
      return '';
    }
final RecognizedText recognizedText =
        await textRecognizer.processImage(inputImage);

However, Japanese texts are recognized gibberish like this.
ChTHLtt,ZO-6tb
O
OPIZEND22 Kevin Sidharta
2024 Legend Story Studios

Steps to reproduce.
Run the code above to read Japanese text.
What is the expected result?
Be able to read Japanese text.
Did you try our example app?
No
Is it reproducible in the example app?
No
Reproducible in which OS?
iOS
Flutter/Dart Version?
% flutter doctor
Doctor summary (to see all details, run flutter doctor -v):
[✓] Flutter (Channel stable, 3.24.3, on macOS 15.1.1 24B91 darwin-arm64, locale ja-JP)
[✓] Android toolchain - develop for Android devices (Android SDK version 34.0.0)
[✓] Xcode - develop for iOS and macOS (Xcode 16.2)
[✓] Chrome - develop for the web
[✓] Android Studio (version 2024.1)
[✓] VS Code (version 1.96.0)
[✓] Connected device (8 available)
[✓] Network resources
• No issues found!

I was not adding pod 'GoogleMLKit/TextRecognitionJapanese', '~> 7.0.0' to the Podfile.


EFS to integrate with AWS Managed Microsoft AD

Is your feature request related to a problem? Please describe.
My customer wants to use EFS Simple File Manager however they want it to be integrated with AD. The documentation says that it supports Cognito. Could it also be made to work with AWS Managed Active Directory
Describe the feature you'd like
EFS Simple File Manager to be integrated with AWS Managed AD for user authentication
Additional context
Add any other context or screenshots about the feature request here.

Related Issue:
https://github.com/aws-samples/simple-file-manager-for-amazon-efs/issues/152


west.yml: Update ST HAL to pull GCC 12 warning fixes

This commit updates the ST HAL (hal_st) to pull in the fixes for the
warnings observed when building with the GCC 12.
Signed-off-by: Stephanos Ioannidis<EMAIL_ADDRESS>

https://github.com/zephyrproject-rtos/hal_st/pull/11 has been merged. Ready for review and merging.


Upload file through github repository other than main branch

Background & motivation
I wish to upload files from a Github repository, I don't wish to see the main branch(which is default branch) in my case.
Read the doc, couldn't find anything. Any idea how this can be done ?
Desired behaviour
Should be able to upload files via any branch of a  Github repository using plan.upload_files
How important is this to you?
Critical; Kurtosis is unusable for me without it.
What area of the product does this pertain to?
CLI: the Command Line Interface

If you want to run a non-main branch, tag or commit use the following syntax kurtosis run github.com/package-author/package-repo@tag-branch-commit
There's a tool tip on this doc here: https://docs.kurtosis.com/advanced-concepts/packages/#runnable-packages
LMK if this answers your question!

Thanks! I am referring here to uploading files via plan.upload_files which is not allowing me to do so.


Remove deprecated ADDRESS environment variable

Looks like you can also remove a small bit of code from the top of etc/deploy/gen_pachd_tls.sh, otherwise LGTM

Good catch; fixed

Strange, I seem to have pulled in some unrelated changes. I'm not sure how, digging in now.

I think your merge commit doesn't indicate it's from master, so github has the wrong base commit to diff against.


Warning: [vuex] state field "route" was overridden by a module with the same name at "route"

I use the following packages:
"vue-router": "^4.0.0-0",
"vuex": "^4.0.0-0",
"vuex-persistedstate": "^4.0.0-beta.3",
"vuex-router-sync": "^6.0.0-rc.1"

When I reload a page - and I believe it only happens when the state i persisted - I get the warning message
[vuex] state field "route" was overridden by a module with the same name at "route"

In my opinion this shouldn't be happening, because I didn't define any modules in my store. changing the module name to e.g. "RouteModule" as written in the docs throws the same warning, except then with "RouteModule" instead of "route".
Is there a way to fix this?

Is vuex-router-sync going to support vue3? I believe this issue is related to that, but I might be wrong.

Is vuex-router-sync going to support vue3? I believe this issue is related to that, but I might be wrong.

FYI: I'm having this issue on vue2

@Unkrass what you can do...
When you invoke the vuex-persistedstate plugin...
e.g:
import Vue from "vue";
import Vuex from "vuex";
import createPersistedState from "vuex-persistedstate";

Vue.use(Vuex);

export default new Vuex.Store({
  plugins: [
    createPersistedState({
      // This omits the plugins/vue-router-sync state.route from persisting
      paths: ["foo", "bar", "zee"],
    }),
  ],
  state: {
    foo: {},
    bar: {},
    zee: {}
  }
});

The above is a simple contrived example... But what is important is passing to the createPersistedState options the paths and explicitly only setting the paths that are in the vuex state. When you do this the vuex-persistedstate plugin won't persist the automatically added state.route that vuex-router-sync injects.
This will make the warning not appear.
Hope this helps.


Bool#to_unsafe can be improved

I'm very sure Bool's to_unsafe can be optimized/improved. Every boolean in Crystal is represented as a one bit integer in LLVM (i1) and that can only be 1 (true) or 0 (false). So to_unsafe should become a special instance variable or method which has access to this internal LLVM integer. Currently to_unsafe is implemented using branching (if true, return 1, if false, return 0) but that's very inefficent in contrast to that the result of to_unsafe is already present in LLVM and doesn't even have to be computed.
And that would also make it possible to optimize other methods like Number#sign. Currently sign also uses branching but that can be avoided with for example (x > 0).to_unsafe - (x < 0).to_unsafe which would be faster than the current sign implementation when to_unsafe is optmized, I think.

How does it actually end up in the generated code? LLVM is pretty smart, and at least in C++, a similar bool conditional is converted to a mov, and the sign example is converted to around 6 branchless assembler instructions.
On the other hand, a version like what you propose generates one fewer instruction, but it also involves subtractions which take up more cycles.
TL;DR: Don't try to outsmart the optimizer.

Don't worry, LLVM is smart.
lib LibC
  fun exit(code : Int32)
end

struct Int32
  def self.new(value)
    value.to_i32
  end
end

struct Bool
  def to_unsafe : Int32
    Int32.new(self ? 1 : 0)
  end
end

a = true.to_unsafe
LibC.exit a

crystal build tmp.cr --prelude=empty --release --emit=llvm-ir

...
define void @__crystal_main(i32 %argc, i8** %argv) local_unnamed_addr !dbg !4 {
alloca:
  store i8** %argv, i8*** @ARGV_UNSAFE, align 8
  tail call void @exit(i32 1), !dbg !9
  ret void, !dbg !9
}
...

lib LibC
  fun exit(code : Int32)
end

struct Number
  def sign
    self < 0 ? -1 : (self == 0 ? 0 : 1)
  end
end

LibC.exit -100.sign

define void @__crystal_main(i32 %argc, i8** %argv) local_unnamed_addr !dbg !4 {
entry:
  store i8** %argv, i8*** @ARGV_UNSAFE, align 8
  tail call void @exit(i32 -1), !dbg !9
  ret void, !dbg !9
}

this -100 is hardcoded
for value
LibC.exit ARGV_UNSAFE.value.value.sign
define void @__crystal_main(i32 %argc, i8** %argv) local_unnamed_addr !dbg !4 {
entry:
  store i8** %argv, i8*** @ARGV_UNSAFE, align 8
  %0 = load i8*, i8** %argv, align 8, !dbg !9
  %1 = load i8, i8* %0, align 1, !dbg !9
  %2 = icmp ne i8 %1, 0, !dbg !12
  %3 = zext i1 %2 to i32, !dbg !12
  tail call void @exit(i32 %3), !dbg !17
  ret void, !dbg !17
}

Before making such claims, please verify the generated assembly (not just LLVM IR code) for code compiled with --release which turns on aggressive LLVM optimizations, that does a very good job in such situations.

@r00ster91 do you agree we can close this?

Yes, thanks for the explanation.


Update licenses_license_ext.sql

Comment on Columns to provide API Documentation Descriptions
Closes #636
All queries:
- [ ] Release notes added or updated in CHANGES.md
- [ ] Query runs without errors
- [ ] Query output is correct
- [ ] Query logic is clear and well documented
- [ ] Query is readable and properly indented with spaces (not tabs)
- [ ] Table and column names are in all-lowercase
- [ ] Quotation marks are used only where necessary

Report queries:
- [ ] Query has complete user documentation
    - [ ] Purpose of report
    - [ ] Sample output
    - [ ] Query instructions

Derived tables:
- [ ] Query begins with user documentation in comment lines
- [ ] File name is listed in `runlist.txt` after dependencies
- [ ] All columns have indexes
- [ ] Table is vacuumed and analyzed

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Improve map-layers objects creation from JSON.

Updated return type for BaseLayerSettings.fromJSON, MaplayerSettings.fromJSON, MapSubLayerSettings.fromJSON to no longer return undefined.
DisplayStyleState.changeBaseMapProps has been removed, DisplayStyleState.backgroundMapbase property should be used instead.

@Mergifyio update

@mdastous-bentley GitHub is bad at triggering / keeping track of Azure DevOps builds. Please push some trivial change to a comment or something to force it to start those builds.


2.x Eureka2 home page

Currently Eureka2 hasn't any home page, but instead provides two Karyon webadmin UI pages (registry insight and server status). These pages are strictly Eureka2 specific, as unlike in Eureka1 where there was always local full registry copy, in Eureka2 there might be multitude of different client subscriptions, dynamically changing over time. The latter may be tracked but only within a framework, which is out of scope of Eureka2 project itself (it is a good fit for Karyon).
I propose to convert the two Karyon webadmin UI pages into Eureka2 WEB app, and leave to Karyon admin the standard UI tabs that are there.

archiving as eureka2 work is going through some larger internal changes.