Refactored credentials options (from funcs to interfaces and types)

I hereby agree to the terms of the CLA available at: https://yandex.ru/legal/cla/?lang=en

Pull request type

Please check the type of change your PR introduces:

[ ] Bugfix
[ ] Feature
[ ] Code style update (formatting, renaming)
[x] Refactoring (no functional changes, no api changes)
[ ] Build related changes
[ ] Documentation content changes
[ ] Other (please describe):

What is the current behavior?

Issue Number: N/A
What is the new behavior?

Other information

Codecov Report
Attention: 18 lines in your changes are missing coverage. Please review.

Comparison is base (184cadd) 66.03% compared to head (3bd2a22) 65.83%.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #859      +/-   ##
==========================================
- Coverage   66.03%   65.83%   -0.21%
==========================================
  Files         251      248       -3
  Lines       23854    23755      -99
==========================================
- Hits        15752    15638     -114
- Misses       7266     7279      +13
- Partials      836      838       +2

Flag
Coverage Δ

52.79% <40.00%> (-0.31%)
:arrow_down:

go-1.17.x
39.07% <62.79%> (-25.40%)
:arrow_down:

go-1.20.x
65.85% <76.00%> (+0.01%)
:arrow_up:

go-1.21.x
65.70% <76.00%> (-0.11%)
:arrow_down:

integration
52.79% <40.00%> (-0.31%)
:arrow_down:

macOS
39.28% <62.79%> (+0.04%)
:arrow_up:

ubuntu
39.30% <62.79%> (+0.07%)
:arrow_up:

unit
39.38% <62.79%> (+0.03%)
:arrow_up:

windows
39.31% <62.79%> (+0.04%)
:arrow_up:

ydb-22.5
?

ydb-23.1
52.45% <40.00%> (-0.32%)
:arrow_down:

ydb-23.2
52.33% <40.00%> (-0.51%)
:arrow_down:

ydb-23.3
52.66% <40.00%> (-0.12%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Files
Coverage Δ

internal/credentials/access_token.go
100.00% <100.00%> (ø)

internal/credentials/anonymous.go
100.00% <100.00%> (ø)

credentials/credentials.go
95.91% <93.75%> (-4.09%)
:arrow_down:

internal/balancer/balancer.go
88.11% <80.00%> (+2.57%)
:arrow_up:

internal/credentials/static.go
50.53% <0.00%> (-4.12%)
:arrow_down:

... and 15 files with indirect coverage changes

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Is this repository still being maintained? I would like to know if it can still be used with the latest version of Expo.

"I encountered the same issue as described in this  https://stackoverflow.com/questions/75531223/expo-updates-failed-to-download-new-update

Yep, working perfectly on latest sdk as well. The next code needs to be tweaked here and there which took me a while, envs, and the commands had to be tweaked to fit my usecase.
If nothing specific, you can go ahead without it as well.

Updated the readme with a note about warranty and support for this repo.


Disable user management features auth mode is UAA.

Similar to LDAP auth mode.
The following criterias should be met:

Admin can not add user
User cannot register himself.
User cannot reset/update his password or profile.

fixed in #3797
Further work on UI will be tracked by #3803


Fix detached container outside window

Fix for #1801

@pdyck are you no longer interested in pursuing this PR?


No "ease-in-out" timing function?

Hello and thanks for this great little plugin!
Is there no way do declare a Vivus.EASE_IN_OUT to use the native ease-in-out timing-function?
If I enter it, it falls back to linear (as Vivus can't interpret it obviously).
As Vivus Instant has it, I wonder why the JS translation can't refer to that native function?

Hello!
Vivus and Vivus-instant run differently. Vivus is in JavaScript, this is
why you have to provide a JS function. Vivus-instant generates CSS code to
run the animation. This is why you cannot use the native CSS ease
transition with Vivus

ok, thanks for the information!
is there no way to implement it in JS for a further release? 99% of animations I do rely on this behaviour …

I'm afraid this won't be implemented. Vivus (JavaScript library) offers
controls to set the progress of an animation and speed which would be more
complex to implement in CSS. I recommend you to translate the 'ease'
animation function in JS.


AlarmClock is not working under macOS Catalina (64Bit)

I assume the code is not maintained, anyhow I think this is a good place to document the issue, show others that it exists and make cooperation possible.
AlarmClock was build for the 32Bit architecture and since macOS High Catalina 32-Bit apps are not supported anymore.
See https://support.apple.com/en-us/HT208436
Maybe one could just "rebuild" the app for the other architecture, maybe code has to be changed.

I started looking at this a couple months ago and found Alarm Clock is using a lot of very old frameworks most deprecated a while ago including QTKit (deprecated in 10.9).  So this isn't a simple rebuild.  Here might be a good place to start:  https://developer.apple.com/library/archive/technotes/tn2300/_index.html
This is far outside my wheelhouse and I don't have the time to figure it out.
However, I've since moved to Horo from the App Store.

Turns out someone ported it.   https://github.com/bfabiszewski/AlarmClock

Is there a compiled .app anywhere?

I don’t see a binary in the other repo. It wouldn’t hurt to ask there to
see if a binary can be built and a release made.
If you’re willing to install xcode that repo is a simple rebuild.
However, I believe this can be closed and the issue can be open on the
other project.
On Mon, Oct 14, 2019 at 5:07 AM studioetc<EMAIL_ADDRESS>wrote:

Is there a compiled .app anywhere?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/robbiehanson/AlarmClock/issues/2?email_source=notifications&email_token=AAIMU3QZW4LQSID5SM7VFDDQOQZGJA5CNFSM4I7KNVBKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEBD2QJI#issuecomment-541567013,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAIMU3XF32A3BZYSPWL4L4LQOQZGJANCNFSM4I7KNVBA
.

--
Derek Schrock

I failed to look in the releases - it's compiled there. I've downloaded - I'll report back on how it works.

I downloaded the app from the repo linked above, and it looks great.. except it doesn't seem to support Catalina Music files. Catalina shifted away from using XML files for iTunes and is now using a proprietary format for their Music library, although there's supposed to be a third-party API to sync like the XML used to. Has anyone gotten this working?
I also have avoided upgrading to Catalina for a long time because of this app, and I had to when my old computer died. I absolutely need this app. Hoping someone got this working!

Has anyone gotten this working?

@ksmall16 Do you only want to change the sound in general (that's what I always did, I need the most soul crushing sound to wake up) or do you want to select it on a per-alarm basis?
For the former you can just replace the default alarm and/or timer sound by

Right clicking the app -> show package contents
Navigate to Contents/Resources
Replace defaultAlarm.m4a and/or defaultTimer.m4a with your sound of choice. But it has to be in AAC which wasn't the case before. I converted my wave file that I've been using to AAC (via Music app) but now the sound is longer for some reason so it doesn't loop as fast.

2called-chaos Thank-you so much for that thought.  I downloaded 2.5 version that is here somewhere but could never get anything but default alarm to make sound.  So started using Horo (which is no where near as good as Alarm Clock was).  And I really miss my Timers saying "your timer has finished" - I don't respond to bells.  Simply found the song I want in library - converted a copy to m4a and put it in the package, named it defaultAlarm and it looks like its working.   I'd like to have access to all my music to choose per alarm but this is much better than nothing. Thank-you, thank-you, thank-you.
P.S. I too delayed changing to Catalina because of this app.  I'd happily pay to have a version that works as it used to.

When I download alarm.clock.app.zip it tells me I have insufficient permission. What's up with that?


Feature request: Mighty Plug Pro support

Title says it all -- any chance this is in the works?

Yes, it's in the works. I can't give an exact estimate, but I guess there'll be an update in 2 to 3 weeks.

+1 to this, I would absolutely love it if Mighty Plug Pro support was added.


wideint SIMD

There are operations like: https://github.com/cppalliance/decimal/blob/develop/include/boost/decimal/detail/wide-integer/uintwide_t.hpp#L888 which are ripe for packing into AVX2 / ARM NEON instructions instead of repeated calculations.

I think ADX instructions may be the right move for this on x64. They are designed for multi-precision like applications with chained add or mul.


chore: release

This pull request prepares the following release:

Repository
Branch
Update

instantsearch
chore/release-1698335948126
minor

Release Summary
This is going to be published with the following command:
yarn lerna publish from-package --yes
Merging Instructions
When merging this pull request, you need to Squash and merge and make sure that the title starts with chore: release.

See details
After that, a commit chore: release will be added and you or your CI can run shipjs trigger to trigger the release based on the commit.

This pull request is automatically generated by Ship.js.

Releasing on Monday.


hns UI plugin resourcePool bug

currently, if the user doesn't have permission to get namespaces in the cluster the UI still requires filling resources quantities even if user selects ResourcePool.
The problem generates from the wrong logic in the create-sns.tsx component with the following statement :
 const [snsdata, loaded] = useK8sWatchResource<K8sResourceKind>(snsResource);
  if (loaded) {
    fatherResourcePool = false;
    if (snsdata.metadata?.labels['dana.hns.io/resourcepool'] === 'true') {
      fatherResourcePool = true;
    }
  }

/kind bug


Capturing the entity ID of the startsUsing source / target?

Question
What would you like to ask?
I would like to implement triggers for e11s Prismatic Deception. There are several images shown in ACT log, and the key is to know the ID of the image that is actually casting. Here is a working trigger condition for this scenario as implemented in Triggernometry:
${_ffxiventity[${func:format(System.Int32,"X"):${numeric:hex2dec(${id})-1}}].casttargetid (NOT EQUALS “0”)
NetRegexes.startsUsing seems to give names only, but not ID's. This is not enough to distinguish between the images because they have the same name.
await window.callOverlayHandler( call: "getCombatants", names: [...] ) gives all possible ID but does not give cast bar information. Any suggestions?

(See also #2329.  I think this is something that I don't want to add to cactbot proper because it feels a little bit over the line for me as something you couldn't figure out directly.  It'd be better for a user trigger or something "unofficial", if that makes sense.)
NetRegexes.startsUsing has a sourceId field: https://github.com/quisquous/cactbot/blob/main/resources/netregexes.ts#L115
You'll need to do a getCombatants to get the entity locations during startsUsing.

Got it. Just started practicing writing/debugging a trigger. Thanks for your help!


API Renames

For #6014

[ ] Pull request represents a single change (i.e. not fixing disparate/unrelated things in a single PR).
[ ] Title summarizes what is changing.
[ ] Has a news entry file (remember to thank yourself!).
[ ] Appropriate comments and documentation strings in the code.
[ ] Has sufficient logging.
[ ] Has telemetry for enhancements.
[ ] Unit tests & system/integration tests are added/updated.
[ ] Test plan is updated as appropriate.
[ ] package-lock.json has been regenerated by running npm install (if dependencies have changed).

As usual for this will push at EOD.

Bleh, looks like the proposed API that I copied from notebook/dev actually has type errors in it. I believe that I'll need to wait until notebook/dev merge to main and then update this PR again.

The proposed API had some stuff in the test section that were types in the non-proposed API, but in the 1.57 types that were not published yet. Rather than mess with our build I just remove the Test class specific types from our proposed API copy.

Codecov Report

Merging #6015 (c92087e) into main (eeeaa6c) will decrease coverage by 3%.
The diff coverage is 7%.

@@           Coverage Diff           @@
##            main   #6015     +/-   ##
=======================================
- Coverage     72%     68%     -4%
=======================================
  Files        403     403
  Lines      27124   27124
  Branches    3973    3973
=======================================
- Hits       19590   18561   -1029
- Misses      5895    7028   +1133
+ Partials    1639    1535    -104

Impacted Files
Coverage Δ

src/client/common/application/notebook.ts
80% <0%> (-7%)
:arrow_down:

src/client/common/application/types.ts
100% <ø> (ø)

...lient/datascience/jupyter/kernels/cellExecution.ts
6% <2%> (-65%)
:arrow_down:

...t/datascience/notebook/vscodeNotebookController.ts
23% <100%> (-54%)
:arrow_down:

.../datascience/jupyter/kernels/cellExecutionQueue.ts
10% <0%> (-79%)
:arrow_down:

...ience/variablesView/variableViewMessageListener.ts
22% <0%> (-78%)
:arrow_down:

...ent/common/application/webviewViews/webviewView.ts
14% <0%> (-72%)
:arrow_down:

src/client/datascience/jupyter/kernels/kernel.ts
9% <0%> (-67%)
:arrow_down:

...c/client/datascience/variablesView/variableView.ts
18% <0%> (-58%)
:arrow_down:

...ent/datascience/jupyter/kernels/kernelExecution.ts
12% <0%> (-55%)
:arrow_down:

... and 63 more

Breaking changes so CI failures are expected. Merging.


Should support different soft delete strategies

Soft delete only works on 'deleted' column.
Should be able to detect other columns such as 'deleted_at'
Should support datetime and timestamp column types as well.

Each tables schema should record what the soft delete column is.
The queries should automatically adjust based on whether it is a date/timestamp or boolean

Closed by #26


In metadata, every field is treated as a string

id_no,level,area,season,system:time_start
L3_LBN_PHE_s1,3,JL,1,1420113601
resulting in:
raise ee_exception.EEException(json_content['error']['message'])
EEException: Value for property 'system:time_start' must be a number.

Fixed


Issue using TSDoc comments in .tsx files

I always get both ts and tslint errors when I try adding a TSDoc block comment in a .tsx file. Is it even supported to use TSDoc comments in .tsx files? I can't find anyone commenting on the issue on Google.

Are you able to share a branch and some instructions for reproing these errors? I haven't seen that problem before. It might be something about your configuration.

It was a silly mistake on my part, sorry! I'll be closing the issue.


[CARBONDATA-4247][CARBONDATA-4241] Fix Wrong timestamp value query results for data before 1900 years with Spark 3.1

Why is this PR needed?

Spark 3.1, will store timestamp value as julian micros and rebase timestamp value from JulianToGregorianMicros during query.
-> Since carbon parse and formats timestamp value with SimpleDateFormatter, query gives incorrect results, when rebased with JulianToGregorianMicros by spark.

CARBONDATA-4241 -> Global sort load and compaction fails on table having timestamp column

What changes were proposed in this PR?

Use Java Instant to parse new timestamp values. 2. For old stores and query with Spark 3.1, Rebase the timestamp value from Julian to Gregorian Micros
If timestamp value is of type Instant, then convert value to java timestamp.

Does this PR introduce any user interface change?

No

Is any new testcase added?

No (Existing testcase is sufficient)

Build Success with Spark 2.3.4, Please check CI http://<IP_ADDRESS>:12602/job/ApacheCarbonPRBuilder2.3/5729/

Build Success with Spark 2.4.5, Please check CI http://<IP_ADDRESS>:12602/job/ApacheCarbon_PR_Builder_2.4.5/3987/

Build Success with Spark 3.1, Please check CI http://<IP_ADDRESS>:12602/job/ApacheCarbon_PR_Builder_3.1/133/

Build Failed  with Spark 2.4.5, Please check CI http://<IP_ADDRESS>:12602/job/ApacheCarbon_PR_Builder_2.4.5/3990/

Build Failed  with Spark 2.3.4, Please check CI http://<IP_ADDRESS>:12602/job/ApacheCarbonPRBuilder2.3/5733/

Build Failed  with Spark 3.1, Please check CI http://<IP_ADDRESS>:12602/job/ApacheCarbon_PR_Builder_3.1/136/

retest this please

Build Failed  with Spark 2.3.4, Please check CI http://<IP_ADDRESS>:12602/job/ApacheCarbonPRBuilder2.3/5734/

Build Success with Spark 2.4.5, Please check CI http://<IP_ADDRESS>:12602/job/ApacheCarbon_PR_Builder_2.4.5/3992/

Build Success with Spark 3.1, Please check CI http://<IP_ADDRESS>:12602/job/ApacheCarbon_PR_Builder_3.1/139/

retest this please

Build Failed  with Spark 2.3.4, Please check CI http://<IP_ADDRESS>:12602/job/ApacheCarbonPRBuilder2.3/5735/

Build Failed  with Spark 2.4.5, Please check CI http://<IP_ADDRESS>:12602/job/ApacheCarbon_PR_Builder_2.4.5/3993/

Build Success with Spark 3.1, Please check CI http://<IP_ADDRESS>:12602/job/ApacheCarbon_PR_Builder_3.1/140/

retest this please

Build Success with Spark 2.3.4, Please check CI http://<IP_ADDRESS>:12602/job/ApacheCarbonPRBuilder2.3/5741/

Build Success with Spark 3.1, Please check CI http://<IP_ADDRESS>:12602/job/ApacheCarbon_PR_Builder_3.1/146/

Build Success with Spark 2.4.5, Please check CI http://<IP_ADDRESS>:12602/job/ApacheCarbon_PR_Builder_2.4.5/4000/

Build Failed  with Spark 2.3.4, Please check CI http://<IP_ADDRESS>:12602/job/ApacheCarbonPRBuilder2.3/5751/

Build Failed  with Spark 3.1, Please check CI http://<IP_ADDRESS>:12602/job/ApacheCarbon_PR_Builder_3.1/156/

@akashrn5 handled all comments. Please check

Build Success with Spark 2.4.5, Please check CI http://<IP_ADDRESS>:12602/job/ApacheCarbon_PR_Builder_2.4.5/4011/

Build Failed  with Spark 3.1, Please check CI http://<IP_ADDRESS>:12602/job/ApacheCarbon_PR_Builder_3.1/157/

Build Failed  with Spark 2.3.4, Please check CI http://<IP_ADDRESS>:12602/job/ApacheCarbonPRBuilder2.3/5752/

@Indhumathi27, Is the Alter table test case failure random? Even @ShreelekhyaG had the same issue in one of her PR's

retest this please

Build Failed  with Spark 3.1, Please check CI http://<IP_ADDRESS>:12602/job/ApacheCarbon_PR_Builder_3.1/168/

Build Success with Spark 2.3.4, Please check CI http://<IP_ADDRESS>:12602/job/ApacheCarbonPRBuilder2.3/5762/

retest this please

Build Success with Spark 2.4.5, Please check CI http://<IP_ADDRESS>:12602/job/ApacheCarbon_PR_Builder_2.4.5/4024/

Build Failed  with Spark 3.1, Please check CI http://<IP_ADDRESS>:12602/job/ApacheCarbon_PR_Builder_3.1/176/

Build Success with Spark 2.3.4, Please check CI http://<IP_ADDRESS>:12602/job/ApacheCarbonPRBuilder2.3/5772/

Build Success with Spark 2.4.5, Please check CI http://<IP_ADDRESS>:12602/job/ApacheCarbon_PR_Builder_2.4.5/4030/

Build Failed  with Spark 2.3.4, Please check CI http://<IP_ADDRESS>:12602/job/ApacheCarbonPRBuilder2.3/5777/

Build Failed  with Spark 2.4.5, Please check CI http://<IP_ADDRESS>:12602/job/ApacheCarbon_PR_Builder_2.4.5/4038/

Build Failed  with Spark 3.1, Please check CI http://<IP_ADDRESS>:12602/job/ApacheCarbon_PR_Builder_3.1/182/

retest this please

Build Failed  with Spark 3.1, Please check CI http://<IP_ADDRESS>:12602/job/ApacheCarbon_PR_Builder_3.1/189/

retest this please

Build Failed  with Spark 2.4.5, Please check CI http://<IP_ADDRESS>:12602/job/ApacheCarbon_PR_Builder_2.4.5/4042/

Build Failed  with Spark 2.3.4, Please check CI http://<IP_ADDRESS>:12602/job/ApacheCarbonPRBuilder2.3/5783/

retest this please

Build Success with Spark 2.3.4, Please check CI http://<IP_ADDRESS>:12602/job/ApacheCarbonPRBuilder2.3/5788/

Build Success with Spark 2.4.5, Please check CI http://<IP_ADDRESS>:12602/job/ApacheCarbon_PR_Builder_2.4.5/4045/

Build Success with Spark 3.1, Please check CI http://<IP_ADDRESS>:12602/job/ApacheCarbon_PR_Builder_3.1/193/

LGTM

LGTM


Fix error message when exporting a write-only property without a setter

This is the fix regarding the issue #42355.
When exporting a get-only property, the error message that gets printed mentions this wrongly:
"modules/mono/csharp_script.cpp:2618 - Write-only property (without getter) cannot be exported[..]"

It changes the message to without setter.

The other message is also wrong.

Do you think that maybe if I change both messages to:
    ERR_PRINT("Cannot export a property without a getter: '" + MEMBER_FULL_QUALIFIED_NAME(p_member) + "'.");

and
    ERR_PRINT("Cannot export a property without a setter: '" + MEMBER_FULL_QUALIFIED_NAME(p_member) + "'.");

It would be cool?

Thanks! And congrats for your first merged contribution :tada:

Cherry-picked for 3.2.4.


Chore: fix GitHub actions macOS

Added a brew workaround due to this issue.
Also added root .gitignore.

Any success here?

It's definitely Github action related.

did you tried with           brew upgrade python3 || brew link --overwrite<EMAIL_ADDRESS>

did you tried with brew upgrade python3 || brew link --overwrite<EMAIL_ADDRESS>
I got through that brew hump. However, I also increased the timeout time to 20 minutes, which should be enough to cover the integration tests.


Add yawn-specific savefile manipulation module

This PR is in preparation for my upcoming PR to the main game repo which allows for players to import and export their save files to JSON.
I have avoided edits to base files as much as possible according to Yawn's contribution guidelines, except in a small few places where it couldn't be avoided (renaming cargo config to toml and the default-features compiler config) and otherwise put everything in the new crate::yw module that can house all our new custom stuff from now on.
Salient features include a new DM function rustg_savefile_to_json(string) exposed by a new DM file rust_g_yw.dm and an LR(1) recursive parser that parses text format DM files obtained with savefile/proc/ExportText() into a struct that can be serialized with Serde to whatever is needed.

Sorry. Accidentally made a PR on the upstream instead of the downstream. My bad!


Fix background HMR by @tyn1998

As per this comment https://github.com/lxieyang/chrome-extension-boilerplate-react/issues/40#issuecomment-1032384937 I extracted the fix for the HMR, and did some minor updates

Popup and Greetings are now tsx files
Added a button to the Popup with a counter as an example
Added an example for listening to tab changes in the background

chrome.tabs.onUpdated.addListener(function (tabId, changeInfo, tab) {
  if (changeInfo.status == 'complete') {
    console.log('Tab finished loading', tabId, tab);
    // do your things
  }
});

Updated the webpack config and README

TODO: Add instructions as explained in https://github.com/lxieyang/chrome-extension-boilerplate-react/issues/40#issuecomment-1032384937

expecting for merger

it seems the package author is not active

good job

@lxieyang I know things come up (I've been there an OSS maintainer in the past). If you're not able to remain active, would you mind considering opening up shared responsibility to contributors who appear interested in helping to maintain the repo?


[18.09] backport update containerd to v1.1.4

backport of https://github.com/moby/moby/pull/37925 for 18.09
git checkout -b 18.09_backport_bump_containerd_1.1.4 ce-engine/18.09
git cherry-pick -s -S -x b3c3c7a5a310ba340f1a86a4a708de34b9602e74
git push -u origin

cherry-pick was clean
Fixes a potential content store bug, backported from 1.2

v1.1.3 release notes: https://github.com/containerd/containerd/releases/tag/v1.1.3
v1.1.4 release notes: https://github.com/containerd/containerd/releases/tag/v1.1.4

Full diff: https://github.com/containerd/containerd/compare/v1.1.2...v1.1.4
changes;

https://github.com/containerd/containerd/pull/2473 [release/1.1] Fix options ordering in proto api txt files
https://github.com/containerd/containerd/pull/2488 [release/1.1] native: set '/' permission to 0755
https://github.com/containerd/containerd/pull/2472 [release/1.1] Move ContainerFlags to "commands" package
https://github.com/containerd/containerd/pull/2539 Update cri plugin to v1.0.5
https://github.com/containerd/containerd/pull/2556 [release/1.1] Cherrypick "Set gid 0 when no group is specified" and docs update
https://github.com/containerd/containerd/pull/2600 [release/1.1] Cherrypick: Support >= 128 layers in overlayfs snapshots
https://github.com/containerd/containerd/pull/2637 [release/1.1] Backport: With-helper for supplemental gid support
https://github.com/containerd/containerd/pull/2645 [release/1.1] Backport: Support uid in WithAdditionalGIDs
https://github.com/containerd/containerd/pull/2657 [release/1.1] Update cri to f117382467baf182382c44332bfbf488effc34bb.
https://github.com/containerd/containerd/pull/2654 [release/1.1] Backported: check exists on content commit + testcase
https://github.com/containerd/containerd/pull/2668 [release/1.1] Backport: Add flag to ctr for running with NoNewPrivileges: false
https://github.com/containerd/containerd/pull/2677 [release/1.1] Update continuity

Janky failure on a flaky test (TestSwarmClusterRotateUnlockKey) https://github.com/moby/moby/issues/33041
PowerPC; https://jenkins.dockerproject.org/job/Docker-PRs-powerpc/11602/console is being looked into in https://github.com/moby/moby/pull/37958
20:32:07 FAIL: docker_cli_swarm_test.go:340: DockerSwarmSuite.TestSwarmContainerEndpointOptions
20:32:07
20:32:07 [d0b1c9e2e0caa] waiting for daemon to start
20:32:07 [d0b1c9e2e0caa] daemon started
20:32:07
20:32:07 docker_cli_swarm_test.go:348:
20:32:07     c.Assert(err, checker.IsNil, check.Commentf("%s", out))
20:32:07 ... value *exec.ExitError = &exec.ExitError{ProcessState:(*os.ProcessState)(0xc423b5b4e0), Stderr:[]uint8(nil)} ("exit status 125")
20:32:07 ... 7sfg7muxzslvj1tax7sm3arxp
20:32:07
20:32:07
20:32:07 [d0b1c9e2e0caa] exiting daemon

WindowsRS1 https://jenkins.dockerproject.org/job/Docker-PRs-WoW-RS1/22460/console failure looks like a known issue; see https://github.com/moby/moby/pull/37820
20:35:01 FAIL: docker_cli_start_test.go:189: DockerSuite.TestStartReturnCorrectExitCode
20:35:01
20:35:01 docker_cli_start_test.go:198:
20:35:01     c.Assert(exitCode, checker.Equals, 12)
20:35:01 ... obtained int = 125
20:35:01 ... expected int = 12
20:35:01

z failure looks the same as PowerPC;  https://jenkins.dockerproject.org/job/Docker-PRs-s390x/11471/console
20:15:12 FAIL: docker_cli_swarm_test.go:340: DockerSwarmSuite.TestSwarmContainerEndpointOptions
20:15:12
20:15:12 [d50ebe9d931c7] waiting for daemon to start
20:15:12 [d50ebe9d931c7] daemon started
20:15:12
20:15:12 docker_cli_swarm_test.go:348:
20:15:12     c.Assert(err, checker.IsNil, check.Commentf("%s", out))
20:15:12 ... value *exec.ExitError = &exec.ExitError{ProcessState:(*os.ProcessState)(0xc420980a60), Stderr:[]uint8(nil)} ("exit status 125")
20:15:12 ... w1wj9dphvuwankcx2h5bv5rc5
20:15:12
20:15:12
20:15:12 [d50ebe9d931c7] exiting daemon

Included in [18.09 backport] Update containerd to v1.2.1-rc.0 #121

this fix is already included in #121


Document mail setup

This is probably the wrong repo for this issue. But I did not know where else to repot it.
I installed a OpenSlides4 instanz with docker-compose on my server. Everything looks good. But when I try to reset my password, I get an error message, that the server is not configured correctly. The backend logs:
backend_1          | General send mail exception: [Errno -3] Temporary failure in name resolution

Can someone explain to me, and maybe document somewhere, what I have to do to setup mails?
I have a running mailserver on outside the docker-compose network.

Thank you very much. This worked.
In my case, it was not necessary to set the uplink setting. It seems, that in docker-compose, each host can access the internet.
I only had to set EMAIL_HOST. All other settings seems to be working with there default values. I also set DEFAULT_FROM_EMAIL, but I don't think, this is necessary.
@normanjaeckel Maybe this should be explained in the readme?

Sure, I can do this

In my case, it was not necessary to set the uplink setting. It seems, that in docker-compose, each host can access the internet.

In my case it was indeed necessary to set that setting, so it might be a good idea to also add it to the documentation

@ostcar @peb-adr @gsiv Can someone of you verify whether it is necessary to add the backend to the uplink network. At the moment the backend is only in the frontend and in the data network so it can only be reached via the proxy.

Only Caddy should be in the Uplink network. Can you explain why you think it is necessary for mail support to change this?

Trial and error. I could not get emails to work, when I tried sending one by using "forget password" or sending invitations, it failed with the message [Errno 99] Cannot assign requested address in the logs. As soon as I added the uplink network to backend it worked (and that was the only thing that I changed).
FYI:
Docker version 20.10.14
Docker Compose version v2.2.3

You are right. I misunderstood the flag "internal" in the docker-compose-file. This is strange, because it worked for me. Maybe I have an older compose-version that ignores the internal flag?
How do we solve this? I do not like to add the backend to the upstream repo. Do we need some sort of mail proxy? @peb-adr what do you think?

I also can confirm this ... After adding uplink network to backend it worked  for me aswell
I use the default v4 compose but deactivated https in the proxy. That way I can use may existing traefik setup to do the https by adding the traefik configuration using  labels

See #205


Add CI on GitHub Actions

This pull request introduces CI on GitHub Actions.
This includes the minimum supported platforms and versions.
(Ruby 2.5 has been EOL on 2021-03-31)
It may be possible to add older Rubies (2.3, 2.4, and 2.5), but the CI configuration might be more complex.
So I recommend only active Rubies.
Note that this pull request doesn't touch the existing CI configurations like TravisCI.
I can remove such old CI configurations with this PR, or you can remove them later.
Ref: https://github.com/ybiquitous/rainbow/pull/1

It makes sense to remove the TravisCI configuration as well 👍

As an update: The Rubinius sections in Gemfile are now removed. The lowest required Ruby version is now at 2.4.

@olleolleolle Thanks! I've merged master into this PR branch.

"This branch cannot be rebased due to conflicts" GitHub says. Perhaps you can rebase it on your side and force-push?

@olleolleolle Rebase and force-push done!

I've also removed the Travis and Appveyor configs as well.

Thanks for the patience. My fork PR ybiquitous/rainbow#1 is now all green!
This PR is now ready to review. 👍🏼


install and integrate safety check

https://pypi.org/project/safety/

pi0 (python 3.9.x)
pip install safety installed in directory /home/pi/.local/bin on linux, needed to update path in profile file
sudo vi /etc/profile
add /home/pi/.local/bin to paths and save file
re-open shell.

pi0 report:
| package                    | installed | affected                 | ID       |
+============================+===========+==========================+==========+
| reportlab                  | 3.5.59    | >=0                      | 39642    |
| pillow                     | 8.1.2     | <8.2.0                   | 40594    |
| pillow                     | 8.1.2     | <8.2.0                   | 40596    |
| pillow                     | 8.1.2     | <8.2.0                   | 40595    |
| pillow                     | 8.1.2     | <8.2.0                   | 40593    |
| pillow                     | 8.1.2     | <8.2.0                   | 40592    |
| pillow                     | 8.1.2     | <8.3.0                   | 40965    |
| pillow                     | 8.1.2     | >=5.2.0,<8.3.2           | 41271    |
| pillow                     | 8.1.2     | >=7.1.0,<8.3.2           | 41277    |
| lxml                       | 4.6.3     | <4.6.5                   | 43366
reportlab updated to 3.6.5
pillow updated to 9.0.0
lxml updated to 4.7.1
reportlab is still flagged after updates but I don't think it is being used by this application.

dev machine (3.10.x):
| package                    | installed | affected                 | ID       |
+============================+===========+==========================+==========+
| werkzeug                   | 2.0.1     | <2.0.2                   | 42050    |
| jupyterlab                 | 3.0.10    | <3.1.0                   | 41038    |
| jupyterlab                 | 3.0.10    | <3.1.0b2                 | 40969    |
| jupyterlab                 | 3.0.10    | >=3.0.0a0,<3.0.17        | 41707    |
| jupyter-server             | 1.4.1     | <1.10.2                  | 41065    |
| jupyter-server             | 1.4.1     | <1.12.0                  | 42779    |
| jupyter-server             | 1.4.1     | <1.6.2                   | 41836    |
| flask-socketio             | 2.9.6     | <4.2.0                   | 37309    |
| eventlet                   | 0.25.2    | <0.31.0                  | 40399    |
| eventlet                   | 0.25.2    | <0.31.0                  | 40416    |
| babel                      | 2.9.0     | <2.9.1                   | 42203
upgrades:
werkzueg = 2.0.2
jupyterlab = 3.2.8
jupyter-server = 1.13.4
flask-socketio = 5.1.1
eventlet = 0.33.0
babel = 2.9.1
report is clean after these updates

pi4 (python 3.7.x), needed to install via pip3 install safety
report:
+============================+===========+==========================+==========+
| package                    | installed | affected                 | ID       |
+============================+===========+==========================+==========+
| typed-ast                  | 1.3.1     | ==1.3.1                  | 37768    |
| typed-ast                  | 1.3.1     | ==1.3.1                  | 37767    |
| reportlab                  | 3.5.13    | <=3.5.26                 | 37572    |
| reportlab                  | 3.5.13    | >=0                      | 39642    |
| pylint                     | 2.2.2     | <2.5.0                   | 38224    |
| pylint                     | 2.2.2     | <2.7.0                   | 39621    |
| pycrypto                   | 2.6.1     | <=2.6.1                  | 35015    |
| pillow                     | 5.4.1     | <6.2.2                   | 37779    |
| pillow                     | 5.4.1     | <6.2.2                   | 37781    |
| pillow                     | 5.4.1     | <6.2.2                   | 37782    |
| pillow                     | 5.4.1     | <6.2.2                   | 37780    |
| pillow                     | 5.4.1     | <6.2.3                   | 38450    |
| pillow                     | 5.4.1     | <6.2.3                   | 38449    |
| pillow                     | 5.4.1     | <7.0.0                   | 38451    |
| pillow                     | 5.4.1     | <8.0.1                   | 40264    |
| pillow                     | 5.4.1     | <8.1.0                   | 40265    |
| pillow                     | 5.4.1     | <8.1.0                   | 40270    |
| pillow                     | 5.4.1     | <8.1.0                   | 40271    |
| pillow                     | 5.4.1     | <8.1.1                   | 40275    |
| pillow                     | 5.4.1     | <8.1.1                   | 40274    |
| pillow                     | 5.4.1     | <8.1.1                   | 40272    |
| pillow                     | 5.4.1     | <8.1.1                   | 40266    |
| pillow                     | 5.4.1     | <8.1.1                   | 40273    |
| pillow                     | 5.4.1     | <8.1.2                   | 40263    |
| pillow                     | 5.4.1     | <8.1.2                   | 40268    |
| pillow                     | 5.4.1     | <8.1.2                   | 40267    |
| pillow                     | 5.4.1     | <8.2.0                   | 40594    |
| pillow                     | 5.4.1     | <8.2.0                   | 40596    |
| pillow                     | 5.4.1     | <8.2.0                   | 40595    |
| pillow                     | 5.4.1     | <8.2.0                   | 40593    |
| pillow                     | 5.4.1     | <8.2.0                   | 40592    |
| pillow                     | 5.4.1     | <8.3.0                   | 40965    |
| pillow                     | 5.4.1     | <=7.0.0                  | 38452    |
| pillow                     | 5.4.1     | >=5.2.0,<8.3.2           | 41271    |
| lxml                       | 4.3.2     | <4.4.0                   | 39195    |
| lxml                       | 4.3.2     | <4.6.2                   | 39194    |
| lxml                       | 4.3.2     | <4.6.3                   | 40072    |
| lxml                       | 4.3.2     | <4.6.5                   | 43366    |
| cryptography               | 2.6.1     | <3.3                     | 39252    |
| cryptography               | 2.6.1     | <3.3.2                   | 39606    |
| cryptography               | 2.6.1     | <=3.2                    | 38932
upgrades:
typed-ast = 1.5.1
reportlab = 3.6.5
pylint = 2.12.2
pycrypto = 2.6.1 already latest installed
pillow = 9.0.0
lxml = 4.7.1
cryptography = 36.0.1
pycrypto and reportlab still flagged after these updates, neither are used in this application.

updated azure yml file to include safety check, cannot test this yet due to pipeline being down.


Please invite me to the GitHub Community Organization

Please invite me to the GitHub Community Organisation.

Name:
Jason Liao (jasontwuk)

Discord Username(if exists):
Jasontwuk#2693

Additional Context:
I watched Eddie’s “Complete Guide to Open Source - How to Contribute” video on Youtube yesterday and think it would be awesome to join this community to contribute and learn with you guys. Thank you. :nerd_face:

Invitation sent for the GitHub Organisation. Welcome to the community 🎉
Don't forget after accepting to make it public so it appears on your GitHub profile for everyone else to see, you can do this by finding your name in the GitHub organisation list and change the dropdown to public https://github.com/orgs/EddieJaoudeCommunity/people
Tips for practising:
Customise your GitHub profile, here is a live stream on it https://www.youtube.com/watch?v=cT6GXCuS0Zo
Practice repo, instructions on how to add your name to the README in the README https://github.com/EddieJaoudeCommunity/hacktoberfest-practice
Remember contributing to open source is not just about code, its about collaboration, communication and adding value
I hope that helps


Fix the macOS version for Monterey

This brings back functionality, but it still consumes one full cpu core, so it isn't ready for merging.

Seems to be working correctly now.

I don't know if this breaks the versions before Monterey.

Did they break the API in Monterey?
LGTM anyway, let me know when you want this to be merged (as it's still marked as fraft

Did they break the API in Monterey?
Kinda, they changed some threading related behavior I think.

let me know when you want this to be merged

I just tested it on an older macOS version, and it appears to be broken. I guess I could check on which version it is and use the old method on older macOS.

It's weird to me that such a fundamental thing would break on macOS. I think that it must be that we're doing something wrong, that probably introduces additional bugs. Can we try to find a solution that works on both versions?

It's weird to me that such a fundamental thing would break on macOS.

Yeah it's indeed weird, even some Xcode provided bluetooth tools seem to be broken.

Can we try to find a solution that works on both versions?

Yeah I guess that's a good idea, I will setup a High Sierra VM (to make testing a bit faster) and see test a few more things.

I will setup a High Sierra VM

This didn't work, couldn't get bluetooth working in a macOS VM.
However I did discover a solution that works on both versions, but wakes the cpu every second on Monterey (which should only be needed on High Sierra)
std::unique_lock<std::mutex> lk(macOSBluetoothConnector->disconnectionMutex);
while (macOSBluetoothConnector->running) {
    [[NSRunLoop currentRunLoop] runUntilDate:[NSDate dateWithTimeIntervalSinceNow:.1]];
    macOSBluetoothConnector->disconnectionConditionVariable.wait_for(lk, std::chrono::milliseconds(1000), [&]() {
        return !macOSBluetoothConnector->running;
    });
}

lk.unlock();

I guess it is a decent solution for both versions.

Not quite sure that I understand the macOS concurrency model, but it sounds like it works this way. Merging :)


mypy: Add annotations to api/integrations/perforce/git_p4.py

Automated message from Dropbox CLA bot
@AZtheAsian, it looks like you've already signed the Dropbox CLA. Thanks!

Woah, this is a lot of work @AZtheAsian!  I wish you'd asked about this script before working on it, though, since it is that it's actually a 2-3 year old copy of a third-party tool:
https://github.com/git/git/blob/master/git-p4.py
with a small patch (08973733a7142dee1b3aa54af31847f74d6be39f).
So there's probably not much benefit to annotating our 2-year-old fork of this tool; instead the path forward for this is that we should probably submit our small patch upstream (as an option) so that we're not maintaining a fork of this 3000 line script.

@timabbott how should we do this?


se hicieron los tests unitarios de OpcionIncorrecta, OpcionGrupoUno y…

… OpcionGrupoDos

Codecov Report

Merging #22 into develop will increase coverage by 1.12%.
The diff coverage is n/a.

@@              Coverage Diff              @@
##             develop      #22      +/-   ##
=============================================
+ Coverage      54.32%   55.45%   +1.12%
- Complexity       132      138       +6
=============================================
  Files             52       52
  Lines            532      532
  Branches           7        7
=============================================
+ Hits             289      295       +6
+ Misses           241      235       -6
  Partials           2        2

Impacted Files
Coverage Δ
Complexity Δ

.../edu/fiuba/algo3/modelo/Opcion/OpcionGrupoDos.java
100.00% <0.00%> (+33.33%)
5.00% <0.00%> (+2.00%)

.../edu/fiuba/algo3/modelo/Opcion/OpcionGrupoUno.java
100.00% <0.00%> (+33.33%)
5.00% <0.00%> (+2.00%)

...du/fiuba/algo3/modelo/Opcion/OpcionIncorrecta.java
100.00% <0.00%> (+33.33%)
5.00% <0.00%> (+2.00%)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c7527ff...0f65040. Read the comment docs.


什么时间开源呀

You know, paper from Alibaba, so this question doesn't really have much significance.

有道理，开源，但是得让别人追不上了再开源，先吃一波流量引入到自家程序，别人没法超越了，再开源，换个项目关注吧

https://github.com/HumanAIGC/AnimateAnyone

我基于faceid plus做了个多人模型，不确定会不会放出来，等安排吧

Probably 6 to 4096 months, they choose it by batch size, if they train with higher batchsize, they release too late :smiling_face_with_tear:

+1 期待啊

我基于faceid plus做了个多人模型，不确定会不会放出来，等安排吧

您好，这个工作有具体思路或者开源代码参考吗？

Thank you for your interest in our work. This project is solely for academic research. We are currently finalizing the model and conducting the last checks. All code and models will be released in approximately 2 months. Please stay tuned! :)

We have released the code and checkpoint. So, this issue I closed.


Add npm script to start postgress container

Add a script in package.json to start the postgress container by running docker-compose up.
Expected Behavior
A script in package.json to start the database before starting the other scripts.
Current Behavior
If you run the dev:api or dev:sdk scripts in the package.json you get an error
Possible Solution
Add "dev:db": "docker-compose up", to the package.json
Steps to Reproduce (for bugs)

nxpm-stack init --name sandbox
cd sandbox
code .
Run the dev:sdk or dev:api npm scripts.

Context
Prevented me getting started while following the introduction here: https://www.youtube.com/watch?v=NvcRukwY_kM
Your Environment
MacOS Big Sur

NPM version: 6.14.8
Node version: v12.18.2
nxpm/cli version: 4.18.0
yarn version: 1.22.5
Operating System and version (desktop or mobile): MacOS Big Sur

Output from yarn list @nxpm/cli:
yarn list v1.22.5
warning Filtering by arguments is deprecated. Please use the pattern option instead.
└─<EMAIL_ADDRESS>✨  Done in 0.96s.

Thanks for the suggestion, I just added it.

Hey there @jonparker, I just published version 4.19.0 which has this feature!

Thanks @beeman


A website that might be useful

I find a website on which there are some state of the art objects classification results. I hope it will be useful. The link is:
http://rodrigob.github.io/are_we_there_yet/build/classification_datasets_results.html

Also: https://paperswithcode.com/sota
This project could as well become a wikipedia page, like https://en.wikipedia.org/wiki/List_of_datasets_for_machine_learning_research

Thanks for the recommendation. There're a lot of excellent jobs done in the community to collect STOA from papers and I am also thinking what this project can contribute without duplicating the efforts from the others. Thanks for the suggestion. I will think about the wiki page.


Test Photo meta data module

We need to have a better sense of the robustness of the meta data module we are using, exif-js.
It seems to sometimes obtain NaN for the geolocation fields.
Compare the results using exif-js with online meta data tools.

Here are some photos that may cause the NaN geolocation issue.


Proposed: Facebook oEmbed Support

Looks like Facebook finally got around to adding support for oEmbed to their platform, however looks like auto discover does not work. Should support be added to the core oEmbed providers list? Looks like they have two different end-points for the most part https://developers.facebook.com/docs/plugins/oembed-endpoints. My initial pass at them seems to have resulted in the following configuration. It's allot but I think it's the simplest it can be given the url's they support and don't support?
Oembed:
  providers:
    "http://www.facebook.com/*/posts/*":
      https: "https://www.facebook.com/plugins/post/oembed.json/"
    "http://www.facebook.com/*/activity/*":
      https: "https://www.facebook.com/plugins/post/oembed.json/"
    "http://www.facebook.com/*/posts/*":
      https: "https://www.facebook.com/plugins/post/oembed.json/"
    "http://www.facebook.com/*/photos/*":
      https: "https://www.facebook.com/plugins/post/oembed.json/"
    "http://www.facebook.com/*/media/*":
      https: "https://www.facebook.com/plugins/post/oembed.json/"
    "http://www.facebook.com/*/questions/*":
      https: "https://www.facebook.com/plugins/post/oembed.json/"
    "http://www.facebook.com/notes/*":
      https: "https://www.facebook.com/plugins/post/oembed.json/"
    "http://www.facebook.com/photo.php?fbid=*":
      https: "https://www.facebook.com/plugins/post/oembed.json/"
    "http://www.facebook.com/permalink.php?story_fbid=*":
      https: "https://www.facebook.com/plugins/post/oembed.json/"
    "http://www.facebook.com/*/videos/*":
      https: "https://www.facebook.com/plugins/video/oembed.json/"
    "http://www.facebook.com/video.php?id=*":
      https: "https://www.facebook.com/plugins/video/oembed.json/"
    "http://www.facebook.com/video.php?v=*":
      https: "https://www.facebook.com/plugins/video/oembed.json/"

Technically you could get away with the below but url's that are not supported just fail with a 100x100 object being returned which results in just a link being displayed.
---
Name: mysite_oembed
After: framework#Oembed
---
Oembed:
  providers:
    "http://www.facebook.com/*/videos/*":
      https: "https://www.facebook.com/plugins/video/oembed.json/"
    "http://www.facebook.com/video.php?id=*":
      https: "https://www.facebook.com/plugins/video/oembed.json/"
    "http://www.facebook.com/video.php?v=*":
      https: "https://www.facebook.com/plugins/video/oembed.json/"
    "http://www.facebook.com/*":
      https: "https://www.facebook.com/plugins/post/oembed.json/"

Is there a better oembed lib we can "outsource" this work to?

i found the cms works with facebook videos using the video's page url, except the autodiscover json+oembed tag matching is using the wrong match. Oembed.php line 120 uses $match[1] but for facebook it should be $match[2]. So I just swapped lines 120-122 with 123-125 and it works.
It doesn't return the thumbnail image to display in the CMS editor though. These can be manually assembled using https://graph.facebook.com/VIDEO_ID/picture.
The only other issue is the editor only accepts http but facebook is only https, fortunately the CMS resolves/renders it anyway.

Can you use named parameters for the regexp instead?

i think the $matches are just the wrong way around, should be no big deal to swap them.

bump!

I've commented on https://github.com/silverstripe/silverstripe-framework/pull/5105

complete


Error when running iterative_pruning.py

> python3 train.py --architecture=resnet50 --dataset=cifar10 --model-saving-path=./resnet50_cifar10/
> python3 iterative_pruning.py --architecture=resnet50 --source-dataset=cifar10 --target-dataset=cifar100 --init-path=resnet50_cifar10/2 --model-saving-path=resnet50_prune

Note. --init_path : Path to model with winning ticket initialization
It seems that init-path is necessary to run iterative_pruning.py (in README.md)

Hi authwork,
I think the error is due to incorrect layer_index for masks array. I have pushed an update that indexes the masks array correctly.
Unfortunately, I am away from office and will not be able to verify the solution till January 2020.  Could you please run the code with the latest commit and check?
Thanks,
Varun

After successfully running train.py, this is the error when I run iterative_pruning.py. Please help.
python iterative_pruning.py --architecture=resnet50 --source-dataset=cifar10 --target-dataset=cifar100 --model-saving-path=../model_save/prune --init-path=../model_save/train/90
Using cuda:0 device.
Files already downloaded and verified
Iterative Pruning started
Running pruning iteration 0
Running pruning iteration 1
Fraction of weights pruned = 4737256/23686272 = 0.20000006754967603
Traceback (most recent call last):
File "iterative_pruning.py", line 238, in
prune_iteratively(model, dataloader, args.architecture, args.optimizer, device, args.model_saving_path, args.init_path, args.random, False)
File "iterative_pruning.py", line 182, in prune_iteratively
params.data.mul_(masks[layer_index].to(device))
RuntimeError: output with shape [64, 64, 1, 1] doesn't match the broadcast shape [64, 64, 1, 64]

The problem goes away with the latest iterative_pruning.py update.

Thank you @cmleecm for verifying.
Closing this issue.


Programmatically parse sentence using homeassistant/debug agent?

Hello, I was wondering if there happen to exist a way to accomplish the same as the Developer Tools -> Assist -> Sentence Parser, but programmatically? I see that testing functionality exists in this repo itself, but I am wanting to do it in the same manner that exists within HA, so I can see which/how my actual entities match, without performing the actual actions against them.
When submitting the sentence on the page, I can see the following in the debugger:
Object { type: "conversation/agent/homeassistant/debug", sentences: (1) […], language: "en", device_id: undefined, id: 35 }
device_id: undefined
id: 35
language: "en"
sentences: Array [ "Turn off bedroom light" ]
type: "conversation/agent/homeassistant/debug"

When trying to look up 'conversation/agent/homeassistant/debug' via github I don't see anyone else having used it via any sort of non-internal api call, or in the developer docs searching for 'agent', 'conversation', 'intent', etc I can't find anything relating to this specific method of debugging, or how I might be able to submit a sentence to it and receive back the same data.
Is there any way currently I can accomplish the same result as the Assist page sentence parser outside of on the page itself?
Thanks,
-MH

The code is there, but what exactly are you trying to achieve?

Just as a fun side project, I am working on a voice assistant that can try more than one avenue to accomplish a goal, using ollama tools/function calling. If the call to conversation.process ends up without a match, it will try pulling info via community hassio cli to try and locate whatever it was, etc and try again.
I have a lot of entities added via mqtt, and most of their names are things like downstairs_desk_lamp, or upstairs_bathroom_light_1_c, so I am using the llm to compare that with the existing entities to figure out the right one, and it would be much more efficient and helpful to be able to call a live endpoint in a debugging manner that will not actually change any state, but simply return back if it matched with what I was expecting it to match with. While I am greatful that it exists, having to constantly go to Developer Tools -> Assist, and manually enter the resulting phrase that the llm came up with to see if it works as expected is a pain.
I was testing things at first using the above mentioned hass-cli since I could easily just run, for example: hass-cli call service conversation.process --arguments text="Turn of  upstairs lights".
Typing that into the Sentence Parser gives the following (because I manually renamed the HA name of that particular item, so I knew it matched), which is exactly what I am hoping to get, without actually turning off any lights while I am testing things:
intent:
  name: HassTurnOff
slots:
  name: bedroom light
details:
  name:
    name: name
    value: bedroom light
    text: bedroom light
targets:
  light.upstairs_bedroom_light_1:
    matched: true
match: true
sentence_template: <turn> off (<area> <name>|<name> [in <area>])
unmatched_slots: {}
source: builtin

Hopefully that all makes sense. What it boils down to, is just trying to find a way to get the same info as above, without having to type it in each time.

Ok, I got it figured out. It had to be accessed via websocket.
In case anyone else ends up looking for it in the future, here is an excerpt from my testing script:

Calling code
 def debug_intent(self, sentences):
    """Debug conversation intents for given sentences using Home Assistant's debug functionality.

    Args:
        sentences (list or str): List of sentences or single sentence to analyze

    Returns:
        dict: Debug information including matched intents and entities
    """
    import websockets
    import json
    import asyncio

    if isinstance(sentences, str):
      sentences = [sentences]
    elif not isinstance(sentences, list):
      raise ValueError("sentences must be a string or list of strings")

    async def connect_and_debug():
      uri = f"{self._base_url.replace('http', 'ws')}/api/websocket"
      msg_id = 1  # Start with message ID 1

      async with websockets.connect(uri) as websocket:
        # Auth phase 1: Wait for auth required message
        auth_required = json.loads(await websocket.recv())
        if auth_required['type'] != 'auth_required':
          raise Exception("First message should be auth_required")

        # Auth phase 2: Send auth token
        await websocket.send(json.dumps({
          "type": "auth",
          "access_token": HA_TOKEN,
        }))

        # Auth phase 3: Wait for auth_ok
        auth_ok = json.loads(await websocket.recv())
        log.debug(auth_ok)

        if auth_ok['type'] != 'auth_ok':
          raise Exception("Error in authentication")

        # Send debug request
        debug_request = {
          "id": msg_id + 1,
          "type": "conversation/agent/homeassistant/debug",
          "sentences": sentences,
          "language": "en"
        }
        log.debug(f"Sending debug request: {json.dumps(debug_request, indent=2)}")
        await websocket.send(json.dumps(debug_request))

        # Wait for result
        while True:
          response = json.loads(await websocket.recv())
          log.debug(f"Received response: {json.dumps(response, indent=2)}")

          if response.get("id") == msg_id + 1:
            if not response.get("success", True):
              log.error(f"Error from Home Assistant: {response.get('error', {}).get('message')}")
              return None
            return response.get("result")

    return asyncio.run(connect_and_debug())

Result
2024-12-01 22:00:34,853 - DEBUG - Sending debug request: {
  "id": 2,
  "type": "conversation/agent/homeassistant/debug",
  "sentences": [
    "turn off downstairs desk lamp"
  ],
  "language": "en"
}
2024-12-01 22:00:34,853 - DEBUG - > TEXT '{"id": 2, "type": "conversation/agent/homeassis...mp"], "language": "en"}' [125 bytes]
2024-12-01 22:00:34,974 - DEBUG - < TEXT '{"id":2,"type":"result","success":true,"result"...,"source":"builtin"}]}}' [411 bytes]
2024-12-01 22:00:34,974 - DEBUG - Received response: {
  "id": 2,
  "type": "result",
  "success": true,
  "result": {
    "results": [
      {
        "intent": {
          "name": "HassTurnOff"
        },
        "slots": {
          "name": "downstairs desk lamp"
        },
        "details": {
          "name": {
            "name": "name",
            "value": "downstairs desk lamp",
            "text": "downstairs desk lamp"
          }
        },
        "targets": {
          "light.downstairs_desk_lamp": {
            "matched": true
          }
        },
        "match": true,
        "sentence_template": "<turn> off (<area> <name>|<name> [in <area>])",
        "unmatched_slots": {},
        "source": "builtin"
      }
    ]
  }
}


Fix bug with button Visualize

I couldn't solve it. For now, I just removed the button and let the user just move the slider...


chore(main): release 0.1.13

:robot: I have created a release beep boop
0.1.13 (2023-06-15)
Docs

refine sayings in public file (064b324)

This PR was generated with Release Please. See documentation.

:robot: Release is at https://github.com/cloverdefa/Whosis-Sayings/releases/tag/v0.1.13 :sunflower:


Omni-C

Hi
What should be added for -e option if we use Omni-C data (which uses endonuclease and randomly cuts the DNA)?
Regards

-e DNASE


Error messages are duplicate

In the latest version on master branch, error messages are duplicated if you do this:
property :first_name, validates: { presence: true }
I receive: "last_name"=>["can't be blank", "can't be blank"]
But it is ok if you do this:
property :first_name
validates :first_name, presence: true

I receive: "last_name"=>["can't be blank"]

Same here, using reform 2.1.0 and lotus-validations 0.4.0. I've created a failing test case to confirm that:
https://github.com/gcoderre/reform-duplicate-errors
It seems to be fixed in 2.2.0, though, but I'm not sure what fixed it.

If this is fixed can we close? @acaron any luck?

As of Reform 2.2 all active model / active record related code has moved to the reform-rails gem.
If you're still experiencing this issue, please can you reopen it in the correct gem. Thanks 😺


node.cloneNode(true) causes corruption of document structure

Thanks for your extremely rapid attention and released fix for #65. Unfortunately the fix which has been issued has made the situation worse in some aspects and for the time being I have to return to release 0.8.0.
The following code which was functional under 0.8.0 now no longer returns any result for the query under 0.9.1
var linkedom = require("linkedom");
var text = "<!DOCTYPE html>\n

"
var root = linkedom.parseHTML(text).document.cloneNode(true);
var queried = root.querySelectorAll(".test-component-include");
console.log("queried " + queried.length + " elements");

In addition part of the original problem reported under #65 is still present in that the classList member of the div node in the cloned markup is not populated. And className neither, which is probably what is responsible for the query failure.

Yeah, classList needs to add entries, if available, on setup ... sorry for the mess, I hope now it works as expected.

P.S. your test case now returns 1 element


Microsoft.KeyVault - secrets & keys missing from readme.azureresourceschema.md

The following file is used to generate ARM template schemas for Microsoft.KeyVault: https://github.com/Azure/azure-rest-api-specs/blob/master/specification/keyvault/resource-manager/readme.azureresourceschema.md
The service supports the following API versions for vaults/secrets:

2015-06-01
2016-10-01
2018-02-14-preview
2018-02-14
2019-09-01

The service supports the following API version for vaults/keys:

2019-09-01

This should unblock the following PR:
https://github.com/Azure/azure-resource-manager-schemas/pull/999
This should also address the following issue:
https://github.com/Azure/bicep/issues/1077
This is somewhat related to:
https://github.com/Azure/azure-rest-api-specs/issues/11929

@anthony-c-martin, are you from KV or ARM team? To which team I should assign this issue?

I'm on the ARM team. This issue should be assigned to the KV team.


[Question] Why cant I pass state when I use navigate(-1)

Hello,
Why when I use navigate and pass it -1, the state is not being applied.
This works
<button onClick={ ()=> navigate('/', {state:{id:12345})}>back</button>

But this does not
<button onClick={ ()=> navigate(-1, {state:{id:12345})}>back</button>

What am I doing wrong?
Than you.

which navigate are you using?
if you are using history.navigate, it is implemented this way
https://github.com/reach/router/blob/e203fe612236851a0bd3c77dba693e539a351c62/src/lib/history.js#L75-L78

which navigate are you using?
if you are using history.navigate, it is implemented this way
https://github.com/reach/router/blob/e203fe612236851a0bd3c77dba693e539a351c62/src/lib/history.js#L75-L78

which navigate are you using?
if you are using history.navigate, it is implemented this way
https://github.com/reach/router/blob/e203fe612236851a0bd3c77dba693e539a351c62/src/lib/history.js#L75-L78

So, the state is not being passed if there a number in the place of to?  Is there another way to navigate to the previous page and pass the state?

which navigate are you using?
if you are using history.navigate, it is implemented this way
https://github.com/reach/router/blob/e203fe612236851a0bd3c77dba693e539a351c62/src/lib/history.js#L75-L78

So, the state is not being passed if there a number in the place of to?  Is there another way to navigate to the previous page and pass the state?

you can add a wrapper around it.
maybe using something like this
const { navigate } = history
const dangerouslyGoToLastRoute = async state => {
  try {
    source.history.go(-1)
    const lastRoute = source.history.entries[source.history.index].pathname
    source.history.entries.pop()
    if (lastRoute) await navigate(lastRoute, { replace: true, state })
  } catch (error) {
    console.log(error)
  }
}

you can add a wrapper around it.
maybe using something like this
const { navigate } = history
const dangerouslyGoToLastRoute = async state => {
  try {
    source.history.go(-1)
    const lastRoute = source.history.entries[source.history.index].pathname
    source.history.entries.pop()
    if (lastRoute) await navigate(lastRoute, { replace: true, state })
  } catch (error) {
    console.log(error)
  }
}


Fix parllel UfsIO test

Parallel task were writing to the same file, and thus failing because of that.
Introduces a class level unique id to prevent that.

alluxio-bot, merge this please

alluxio-bot, cherry-pick this to branch-2.4 please

alluxio-bot, cherry-pick this to branch-2.4 please

alluxio-bot, cherry-pick this to branch-2.4 please

alluxio-bot, cherry-pick this to branch-2.4 please

alluxio-bot, cherry-pick this to branch-2.4 please

alluxio-bot, cherry-pick this to branch-2.4 please


 Security scans can get caught up with the contents of the Gemfile.lock even though it is given no heed.
Let's remove these files:
  CHANGELOG.md
- Gemfile
- Gemfile.lock
  LICENSE.md
  README.md
- Rakefile
  lib/redis-client.rb
  lib/redis_client.rb
  lib/redis_client/circuit_breaker.rb
  lib/redis_client/command_builder.rb
  lib/redis_client/config.rb
  lib/redis_client/connection_mixin.rb
  lib/redis_client/decorator.rb
  lib/redis_client/middlewares.rb
  lib/redis_client/pid_cache.rb
  lib/redis_client/pooled.rb
  lib/redis_client/ruby_connection.rb
  lib/redis_client/ruby_connection/buffered_io.rb
  lib/redis_client/ruby_connection/resp3.rb
  lib/redis_client/sentinel_config.rb
  lib/redis_client/url_config.rb
  lib/redis_client/version.rb
- redis-client.gemspec

Updated in 90a61a3.

Grumble. This breaks the JRuby CI. It does ring a bell, I think JRuby doesn't have a proper $PATH for some reason.

Ok, let's try this one.


Support for replaceState

It appears replaceState is confirmed as missing from preact-compat. Would be awesome to consider implementing it.
This is currently breaking a Preact port of a React Hacker News app I work on.

Definitely top of the list, should also be a pretty quick fix. PRs welcome!

Support for replaceState is available in 3.9.0.


Add rclone check function.

This PR wraps Rclone's check command. This is a nice command that checks a source directory (in our case local_path against a target directory (in our case central_path) and returns a list of all files and whether they are:

same across source and target
different across source and target
found in source only
found in target only

It outputs this in a relatively parseable way. This PR introduces:

function to directly wrap the rclone check call
function to parse the output into a dict of paths separated based on the above aases
testing function

This PR includes a test, and is internal so no docs required. This function is aimed to be used as part of the TUI filetree to indicate  differences between local and central filesystems - anything that will case this easier please let me know!

Thanks @b-peri!


Cast "mixed" value as array.

Q
A

Bug fix?
Yes

New feature?
No

BC breaks?
No

Deprecations?
No

Fixed tickets
N/A

Do not forget to add notes about your changes to CHANGELOG.md

What bug? )


Update shapeless to 2.3.12

About this PR
📦 Updates com.chuusai:shapeless from 2.3.10 to 2.3.12
📜 GitHub Release Notes - Version Diff
Usage
✅ Please merge!
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

⚙ Adjust future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "com.chuusai", artifactId = "shapeless" } ]

Or, add this to slow down future updates of this dependency:
dependencyOverrides = [{
  pullRequests = { frequency = "30 days" },
  dependency = { groupId = "com.chuusai", artifactId = "shapeless" }
}]

labels: library-update, early-semver-patch, semver-spec-patch, version-scheme:pvp, commit-count:1

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 78.89%. Comparing base (6d69a24) to head (d4e5a60).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #2904   +/-   ##
=======================================
  Coverage   78.88%   78.89%
=======================================
  Files         467      467
  Lines       15975    15975
  Branches     2556     2592   +36
=======================================
+ Hits        12602    12603    +1
+ Misses       3373     3372    -1

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Adjustments for #814

Handle situation when response is GZIP-ed, but it's empty

Thanks for the PR. However I'm hesitant to bring in an additional library (commons-io) just for reading an inputstream inte a byte-array. I'd rather just have the method copied or implemented manually.

Excellent work! Thanks a lot for helping out, really appreciated. And extra points for writing tests!


ICE: Fix ICE not sending via reflective candidates.

Removed the iceOutboundHandler to slowly start working
towards refactoring the transports.
Relates to #195

Pull Request Test Coverage Report for Build 1012

19 of 63 (30.16%)  changed or added relevant lines in 4 files are covered.
3 unchanged lines in 2 files lost coverage.
Overall coverage decreased (-0.04%) to 50.823%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

internal/stun/stun.go
2
8
25.0%

pkg/ice/candidate.go
0
18
0.0%

pkg/ice/agent.go
2
22
9.09%

Files with Coverage Reduction
New Missed Lines
%

internal/dtls/dtls.go
1
17.24%

pkg/ice/agent.go
2
17.39%

Totals

Change from base Build 1002:
-0.04%

Covered Lines:
3364

Relevant Lines:
6619

💛  - Coveralls


feat/add-tables: Create database tables

should we use user_id as foreign key instead of customer_id from users table.
it makes more sense creating customer and admin from same users table.


Climate slider and cold winter sometimes create an error

When increasing the temperature in blanc NL 2050 scenario and toggle the cold winter, the ETM returns the following error in Inspect> Network> repsonse

The interesting thing is that is does not occur every time

I think this has been fixed now, probably related to https://github.com/quintel/etmodel/issues/3079. I don't get the error when following the steps above.


wrong date

Today is 20.4.2021 for me which is 5 days off from what the script got
here is the log https://pastebin.com/gUS2wqKr

Interesting. Turns out the "Upload Date" in the info I use is really the date the stream was set up. So she probably had it in a pending state since the 15th. Guess I could either just create the date myself or truncate the UTC timestamp from the "startTimestamp" field instead.
I'll probably do one of those two things tomorrow.

Interesting. Turns out the "Upload Date" in the info I use is really the date the stream was set up

ah that makes sense she planned to do the stream at the 15th but rescheduled it to today

I'll probably do one of those two things tomorrow.

okay perfect thanks as always

Fixed in commit 1061a06


do not make local identity outdated

it seems currently local identity flip back to outdated every 15 days unnecessarily, this commit fixes that.
a better solution might be having a dedicated local_updated state which never retry, but that requires much more change and I'll save it for another day.

Nice catch!
I agree with the other state, but yes, definitely a bigger patch.


[iOS] Getting error building with monorepo

Description
Starting from version 0.1.176 trying to build within a monorepo fails. (tested latest as well 0.1.182). I have not tested with use_frameworks which was added support for in that version.
From looking a bit into it seems to be related to this change: https://github.com/Shopify/react-native-skia/pull/1400. This added references to node_modules relative to PODS_ROOT in the podspec file, which is not the same within some monorepos.
Patching the package locally to the correct path for the code i'm working on it works ✅ :
-    "HEADER_SEARCH_PATHS" => '"$(PODS_ROOT)/../../node_modules/@shopify/react-native-skia/cpp/" ' +
-                             '"$(PODS_ROOT)/../../node_modules/@shopify/react-native-skia/cpp/api/" ' +
-                             '"$(PODS_ROOT)/../../node_modules/@shopify/react-native-skia/cpp/rnskia/" ' +
-                             '"$(PODS_ROOT)/../../node_modules/@shopify/react-native-skia/cpp/rnskia/dom/" ' +
-                             '"$(PODS_ROOT)/../../node_modules/@shopify/react-native-skia/cpp/skia/"'
+    "HEADER_SEARCH_PATHS" => '"$(PODS_ROOT)/../../../node_modules/@shopify/react-native-skia/cpp/" ' +
+                             '"$(PODS_ROOT)/../../../node_modules/@shopify/react-native-skia/cpp/api/" ' +
+                             '"$(PODS_ROOT)/../../../node_modules/@shopify/react-native-skia/cpp/rnskia/" ' +
+                             '"$(PODS_ROOT)/../../../node_modules/@shopify/react-native-skia/cpp/rnskia/dom/" ' +
+                             '"$(PODS_ROOT)/../../../node_modules/@shopify/react-native-skia/cpp/skia/"'

I also tested pointing towards $(PODS_ROOT)/Headers/Public/react-native-skia/ locally as that path should be the same with/without monorepo. But did work when testing ❌ .
Seems like there has been a few other reports on errors in the issue related to the mentioned PR as well https://github.com/Shopify/react-native-skia/issues/652, not sure if these are also related to monorepo setups.
Version
>= 0.1.176
Steps to reproduce

Setup a react native monorepo
Add react-native-skia
Try to run/build ios app.

Snack, code example, screenshot, or link to a repository
Build error example:
CompileC /Users/User/Library/Developer/Xcode/DerivedData/app-lsoqweanhrszfrahzbuhzzyhdrfg/Build/Intermediates.noindex/Pods.build/Debug-iphonesimulator/react-native-skia.build/Objects-normal/x86_64/RNSkDomView.o<EMAIL_ADDRESS>normal x86_64 c++ com.apple.compilers.llvm.clang.1_0.compiler (in target 'react-native-skia' from project 'Pods')

Hi @joarkosberg! Thanks for reporting this (og hei fra Oslo!) You mentioned testing out setting it to $(PODS_ROOT)/Headers/Public/react-native-skia/ - not sure if I understood correctly but this was failing as well? It makes sense to use the PODS_ROOT for this - I'll do some testing.

@chrfalch Thanks for taking a look :-)
Yes $(PODS_ROOT)/Headers/Public/react-native-skia/ did not work. I have seen path to $(PODS_ROOT)/Headers/Public/..... been used before in other podspecs, so thought maybe that would work.
I tested with your proposal "HEADER_SEARCH_PATHS" => '"$(PODS_TARGET_SRCROOT)/cpp"/**'
Can confirm it works for me 😊


Some formatting things on dashboard

Context & versions
Sanchonet staging
Steps to reproduce
Go to Dashboard
Actual behavior
See Video please
https://www.loom.com/share/a37f83ac7de0473fbe858e31d186f0e2?sid=21649709-6bcd-4904-8787-6cb5c055474d

fixed in new version


iOS 7.X Tap Background View not response.

on iOS 7.X Tap Background View not response. what? this question contains your examples. :(

i already implementation delegate method:
- (BOOL)emptyDataSetShouldAllowTouch:(UIScrollView *)scrollView
{
    return YES;
}

but on iOS 7.X tap background view is not response.


bugfixRemove redundant split operations

Proposed changes
In explain, we don't need to get all the splits.

run buildall


Explore auto-releasing

Annotated, Signed Git Tags.

I'll have a look at this at some point, but I want to concentrate on some other things for now, and we do currently have a working release process.

I'm unassigning myself as I think there are more important things to do.

GitHub actions is a much nicer way to do this.

I have implemented a GitHub Action to deploy to PyPi for one of my personal projects. While it probably can't be copied exactly it may serve as a decent jumping off point.

I have implemented a GitHub Action to deploy to PyPI for one of my personal projects. While it probably can't be copied exactly it may serve as a decent jumping off point.

I really like the verify stage of this.


Adding results2

Attempt to create a pull request

looks good. Will rather double check by comparing agianst source origin on jfabo-work machine.
Btw, see the "changed files tab" infinity :)


MSDKUI-1892: Fix EmptyRoom layout issues

This commit changes title and subtitle labels to support multiple lines.

Codecov Report

Merging #86 into master will increase coverage by 0.83%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master      #86      +/-   ##
==========================================
+ Coverage   88.38%   89.21%   +0.83%
==========================================
  Files          89       59      -30
  Lines        5493     3524    -1969
==========================================
- Hits         4855     3144    -1711
+ Misses        638      380     -258

Flag
Coverage Δ

#ios
89.21% <ø> (+0.83%)
:arrow_up:

Impacted Files
Coverage Δ

MSDKUI_Demo/IconButton.swift

MSDKUI_Demo/OptionsViewController.swift

MSDKUI_Demo/ManeuversOverviewViewController.swift

MSDKUI_Demo/GuidanceDashboardViewController.swift

MSDKUI_Demo/ExtensionArray.swift

MSDKUI_Demo/RouteViewController.swift

MSDKUI_Demo/ExtensionUITableViewCell.swift

MSDKUI_Demo/ExtensionBundle.swift

MSDKUI_Demo/ExtensionNMAMapMarker.swift

...UI_Demo/GuidanceDashboardTableViewDataSource.swift

... and 20 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ea799b7...603225e. Read the comment docs.


Display images with unknown size in prefetch (#1162)

Images served via scripts usually don't have size header.

It looks like we could just read body.length if there is no content-length header. This will remove the need for a new option.

@bews, mind trying @xPaw's solution? I don't think this scenario grants a dedicated option either, it's rather cryptic, even with proper documentation.

Closing this to keep an history of the proposed changes somewhere, feel free to try the body.length solution in another PR and add a link to this one.

body.length is always 0 for images, so it doesn't work.

Huh? Why would fetched content length be 0?

It's 0 even for websites. I guess this is the reason:
try {
	body = JSON.parse(data);
} catch (e) {
	body = {};
}

So body is {} in most cases.

Try checking before that then.

It just doesnt' work, .length is still 0.
Also, JSON.parse(data); fails every time:
<!DOCTYPE html>
^

SyntaxError: Unexpected token < in JSON at position 0

Right, didn't notice there's a separate variable called data, that should have the length then. That json parsing is for rare cases when data on the page is json, but I don't know what we achieve by parsing it...

So, if answer is html, then original data has length, but if it's an image - it's still 0. Maybe there should be another way to get size in that cases?

That sounds odd, surely there must be a way.

It looks like request.get doesn't download images at all: it just checks headers. Therefore length is 0.

Looked at the code, it aborts the request if content type doesn't match text/html or application/json.

lol, looking but not seeing, here's the reason of this:
if (!(/(text\/html|application\/json)/.test(res.headers["content-type"]))) {
	res.req.abort();
}

soo... should we remove this?

You could certainly try.

Yep, this works now. Should we completely ignore content-length header then?


Update docs header logo to remove text

Summary
On the docs pages we want the paketo logo only with no accompanying text

Duplicate #269. Closing.


DDP training timeout

Bug description
I am using the default configs, code and data to train a model within BioNeMo framework. The timeout occurs at the middle of the training.
What version are you seeing the problem on?
v2.2
How to reproduce the bug
The configs might relate to the training:

trainer:
  devices: 8 # number of GPUs or CPUs
  num_nodes: 1
  accelerator: gpu #gpu or cpu
  precision: 16 #16 or 32
  logger: False # logger is provided by NeMo exp_manager
  enable_checkpointing: False # checkpointing is done by NeMo exp_manager
  replace_sampler_ddp: False # use NeMo Megatron samplers
  max_epochs: null # # use max_steps instead with NeMo Megatron model
  log_every_n_steps: 10  # number of interations between logging
  val_check_interval: 15e4
  limit_val_batches: 50 # number of batches in validation step, use fraction for fraction of data, 0 to disable
  limit_test_batches: 500 # number of batches in test step, use fraction for fraction of data, 0 to disable
  accumulate_grad_batches: 1
  gradient_clip_val: 1.0
  benchmark: False
  max_steps: 500000

### Error messages and logs

Epoch 0:   6%|██                               | 32040/500150 [6:28:43<94:39:17,  1.37it/s, loss=2.6, v_num=95nc, reduced_train_loss=2.590, global_step=3.2e+4, consumed_samples=2.56e+7][E ProcessGroupNCCL.cpp:458] [Rank 0] Watchdog caught collective operation timeout: WorkNCCL(SeqNum=96120, OpType=ALLREDUCE, Timeout(ms)=1800000) ran for 25624887 milliseconds before timing out.
[E ProcessGroupNCCL.cpp:458] [Rank 0] Watchdog caught collective operation timeout: WorkNCCL(SeqNum=96117, OpType=ALLREDUCE, Timeout(ms)=1800000) ran for 25625613 milliseconds before timing out.
[E ProcessGroupNCCL.cpp:458] [Rank 0] Watchdog caught collective operation timeout: WorkNCCL(SeqNum=96125, OpType=ALLREDUCE, Timeout(ms)=1800000) ran for 25625593 milliseconds before timing out.
[E ProcessGroupNCCL.cpp:458] [Rank 0] Watchdog caught collective operation timeout: WorkNCCL(SeqNum=96125, OpType=ALLREDUCE, Timeout(ms)=1800000) ran for 25625593 milliseconds before timing out.
[E ProcessGroupNCCL.cpp:458] [Rank 7] Watchdog caught collective operation timeout: WorkNCCL(SeqNum=64088, OpType=ALLREDUCE, Timeout(ms)=1800000) ran for 25625251 milliseconds before timing out.
[E ProcessGroupNCCL.cpp:458] [Rank 0] Watchdog caught collective operation timeout: WorkNCCL(SeqNum=96117, OpType=ALLREDUCE, Timeout(ms)=1800000) ran for 25625613 milliseconds before timing out.
[E ProcessGroupNCCL.cpp:458] [Rank 2] Watchdog caught collective operation timeout: WorkNCCL(SeqNum=64088, OpType=ALLREDUCE, Timeout(ms)=1800000) ran for 25625251 milliseconds before timing out.
[E ProcessGroupNCCL.cpp:458] [Rank 0] Watchdog caught collective operation timeout: WorkNCCL(SeqNum=96117, OpType=ALLREDUCE, Timeout(ms)=1800000) ran for 25625613 milliseconds before timing out.
[E ProcessGroupNCCL.cpp:458] [Rank 0] Watchdog caught collective operation timeout: WorkNCCL(SeqNum=96120, OpType=ALLREDUCE, Timeout(ms)=1800000) ran for 25624887 milliseconds before timing out.
[E ProcessGroupNCCL.cpp:458] [Rank 3] Watchdog caught collective operation timeout: WorkNCCL(SeqNum=64088, OpType=ALLREDUCE, Timeout(ms)=1800000) ran for 25625251 milliseconds before timing out.
[E ProcessGroupNCCL.cpp:458] [Rank 0] Watchdog caught collective operation timeout: WorkNCCL(SeqNum=96120, OpType=ALLREDUCE, Timeout(ms)=1800000) ran for 25624886 milliseconds before timing out.
[E ProcessGroupNCCL.cpp:458] [Rank 0] Watchdog caught collective operation timeout: WorkNCCL(SeqNum=96120, OpType=ALLREDUCE, Timeout(ms)=1800000) ran for 25624887 milliseconds before timing out.
[E ProcessGroupNCCL.cpp:472] Some NCCL operations have failed or timed out. Due to the asynchronous nature of CUDA kernels, subsequent GPU operations might run on corrupted/incomplete data.
[E ProcessGroupNCCL.cpp:478] To avoid data inconsistency, we are taking the entire process down.
[E ProcessGroupNCCL.cpp:830] [Rank 0] NCCL watchdog thread terminated with exception: [Rank 0] Watchdog caught collective operation timeout: WorkNCCL(SeqNum=96117, OpType=ALLREDUCE, Timeout(ms)=1800000) ran for 25625613 milliseconds before timing out.
terminate called after throwing an instance of 'std::runtime_error'
what():  [Rank 0] NCCL watchdog thread terminated with exception: [Rank 0] Watchdog caught collective operation timeout: WorkNCCL(SeqNum=96117, OpType=ALLREDUCE, Timeout(ms)=1800000) ran for 25625613 milliseconds before timing out.
[E ProcessGroupNCCL.cpp:472] Some NCCL operations have failed or timed out. Due to the asynchronous nature of CUDA kernels, subsequent GPU operations might run on corrupted/incomplete data.
[E ProcessGroupNCCL.cpp:478] To avoid data inconsistency, we are taking the entire process down.
[E ProcessGroupNCCL.cpp:830] [Rank 0] NCCL watchdog thread terminated with exception: [Rank 0] Watchdog caught collective operation timeout: WorkNCCL(SeqNum=96125, OpType=ALLREDUCE, Timeout(ms)=1800000) ran for 25625593 milliseconds before timing out.
terminate called after throwing an instance of 'std::runtime_error'
what():  [Rank 0] NCCL watchdog thread terminated with exception: [Rank 0] Watchdog caught collective operation timeout: WorkNCCL(SeqNum=96125, OpType=ALLREDUCE, Timeout(ms)=1800000) ran for 25625593 milliseconds before timing out.
[E ProcessGroupNCCL.cpp:472] Some NCCL operations have failed or timed out. Due to the asynchronous nature of CUDA kernels, subsequent GPU operations might run on corrupted/incomplete data.
[E ProcessGroupNCCL.cpp:478] To avoid data inconsistency, we are taking the entire process down.
[E ProcessGroupNCCL.cpp:830] [Rank 0] NCCL watchdog thread terminated with exception: [Rank 0] Watchdog caught collective operation timeout: WorkNCCL(SeqNum=96117, OpType=ALLREDUCE, Timeout(ms)=1800000) ran for 25625613 milliseconds before timing out.
terminate called after throwing an instance of 'std::runtime_error'
what():  [Rank 0] NCCL watchdog thread terminated with exception: [Rank 0] Watchdog caught collective operation timeout: WorkNCCL(SeqNum=96117, OpType=ALLREDUCE, Timeout(ms)=1800000) ran for 25625613 milliseconds before timing out.
[E ProcessGroupNCCL.cpp:458] [Rank 4] Watchdog caught collective operation timeout: WorkNCCL(SeqNum=64090, OpType=ALLREDUCE, Timeout(ms)=1800000) ran for 1800741 milliseconds before timing out.
[E ProcessGroupNCCL.cpp:458] [Rank 5] Watchdog caught collective operation timeout: WorkNCCL(SeqNum=64090, OpType=ALLREDUCE, Timeout(ms)=1800000) ran for 1800733 milliseconds before timing out.
[E ProcessGroupNCCL.cpp:458] [Rank 1] Watchdog caught collective operation timeout: WorkNCCL(SeqNum=64090, OpType=ALLREDUCE, Timeout(ms)=1800000) ran for 1800769 milliseconds before timing out.
[E ProcessGroupNCCL.cpp:458] [Rank 6] Watchdog caught collective operation timeout: WorkNCCL(SeqNum=64090, OpType=ALLREDUCE, Timeout(ms)=1800000) ran for 1800781 milliseconds before timing out.
[E ProcessGroupNCCL.cpp:458] [Rank 0] Watchdog caught collective operation timeout: WorkNCCL(SeqNum=64090, OpType=ALLREDUCE, Timeout(ms)=1800000) ran for 1800847 milliseconds before timing out.

 There are several notebooks in the examples folder https://github.com/NVIDIA/BioNeMo/tree/main/examples/service/notebooks but I doubt you are running these. Where is the code and config that you are running?

Hi @awaelchli, the github repo does not have the whole training code, I got it from their docker containers.  If you want to reproduce their code, here is the doc https://docs.nvidia.com/bionemo-framework/latest/quickstart-fw.html
I am afraid it is too much work for you to reproduce it since you have to download and prepare all the data. My problem is kind of tricky that the nccl timeout happens during the training, sometimes earlier sometimes later, but it just out of nowhere. I would like to know how to track down the errors bc I have no clue given the error log. I have tried many solutions but no luck, such as:

increase the timeout to a year
set a bunch of nccl variables.

export NCCL_DEBUG=INFO

export NCCL_P2P_DISABLE=1
export NCCL_P2P_LEVEL=NVL
export NCCL_IB_GID_INDEX=3

Hey @pengzhangzhi
Sorry, lots going on recently, trying to balance priorities. Sorry for missing or delayed replies.
I implemented a system check utility to help with such problems:
Feel free to test it out if you have the time https://github.com/Lightning-AI/pytorch-lightning/pull/19609
The idea of this system check is that it is implemented in raw PyTorch, so if issues arise we would know whether it is in Lightning or not.

Thanks!
Exactly what I need for debugging! Do u have a document for how to use this tool? The link in that pr isn't valid :(
📚 Documentation preview 📚: pytorch-lightning--19609.org.readthedocs.build/en/19609

Yes the docs will only generate once the PR is ready.
The easiest for you to try it right now is to just copy this file
https://github.com/Lightning-AI/pytorch-lightning/blob/feature/system-check/src/lightning/fabric/utilities/system_check.py
locally and just run it as python system_check.py. It doesn't require any dependencies other than torch and psutil.

Thanks!!
Here is log... I can't make sense of it..
Below is the output of `nvidia-smi`. It shows information about the GPUs that are installed on this machine, the driver version, and the maximum supported CUDA version it can run.

Thu Mar 14 17:08:07 2024
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.129.03             Driver Version: 535.129.03   CUDA Version: 12.2     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  NVIDIA A100-SXM4-80GB          Off | 00000000:07:00.0 Off |                    0 |
| N/A   24C    P0              59W / 400W |      7MiB / 81920MiB |      0%      Default |
|                                         |                      |             Disabled |
+-----------------------------------------+----------------------+----------------------+
|   1  NVIDIA A100-SXM4-80GB          Off | 00000000:0F:00.0 Off |                    0 |
| N/A   22C    P0              55W / 400W |      7MiB / 81920MiB |      0%      Default |
|                                         |                      |             Disabled |
+-----------------------------------------+----------------------+----------------------+
|   2  NVIDIA A100-SXM4-80GB          Off | 00000000:47:00.0 Off |                    0 |
| N/A   22C    P0              58W / 400W |      7MiB / 81920MiB |      0%      Default |
|                                         |                      |             Disabled |
+-----------------------------------------+----------------------+----------------------+
|   3  NVIDIA A100-SXM4-80GB          Off | 00000000:4E:00.0 Off |                    0 |
| N/A   22C    P0              58W / 400W |      7MiB / 81920MiB |      0%      Default |
|                                         |                      |             Disabled |
+-----------------------------------------+----------------------+----------------------+
|   4  NVIDIA A100-SXM4-80GB          Off | 00000000:87:00.0 Off |                    0 |
| N/A   29C    P0              59W / 400W |      7MiB / 81920MiB |      0%      Default |
|                                         |                      |             Disabled |
+-----------------------------------------+----------------------+----------------------+
|   5  NVIDIA A100-SXM4-80GB          Off | 00000000:90:00.0 Off |                    0 |
| N/A   27C    P0              59W / 400W |      7MiB / 81920MiB |      0%      Default |
|                                         |                      |             Disabled |
+-----------------------------------------+----------------------+----------------------+
|   6  NVIDIA A100-SXM4-80GB          Off | 00000000:B7:00.0 Off |                    0 |
| N/A   51C    P0             259W / 400W |  80363MiB / 81920MiB |     90%      Default |
|                                         |                      |             Disabled |
+-----------------------------------------+----------------------+----------------------+
|   7  NVIDIA A100-SXM4-80GB          Off | 00000000:BD:00.0 Off |                    0 |
| N/A   50C    P0             213W / 400W |  76741MiB / 81920MiB |     71%      Default |
|                                         |                      |             Disabled |
+-----------------------------------------+----------------------+----------------------+

+---------------------------------------------------------------------------------------+
| Processes:                                                                            |
|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |
|        ID   ID                                                             Usage      |
|=======================================================================================|
+---------------------------------------------------------------------------------------+

The matrix below shows how the GPUs in this machine are connected.
Last error:
Socket recv failed while polling for opId=0x7f8be9d30b00

This output shows that distributed PyTorch won't work on your system. It can't synchronize at the barrier, which is a very basic requirement.
There should be a folder system_check, it might have additional logs from NCCL with warnings. In rare occasions, a driver update or downgrade can help, or reinstalling PyTorch in a fresh environment.

Thanks!!
Since the error is in process 6, I am showing the log of nccl-rank-6 below:
pbg-dgx-1:1243335:1243335 [6] NCCL INFO cudaDriverVersion 12020
pbg-dgx-1:1243335:1243335 [6] NCCL INFO Bootstrap : Using enp226s0:<IP_ADDRESS><0>
pbg-dgx-1:1243335:1244249 [6] NCCL INFO Plugin Path : /opt/hpcx/nccl_rdma_sharp_plugin/lib/libnccl-net.so
pbg-dgx-1:1243335:1244249 [6] NCCL INFO P2P plugin IBext
pbg-dgx-1:1243335:1244249 [6] NCCL INFO NET/IB : No device found.
pbg-dgx-1:1243335:1244249 [6] NCCL INFO NET/IB : No device found.
pbg-dgx-1:1243335:1244249 [6] NCCL INFO NET/Socket : Using [0]enp226s0:<IP_ADDRESS><0> [1]vethe42dbfb:fe80::a0ec:26ff:fe84:f001%vethe42dbfb<0> [2]vethd8013ee:fe80::f8e4:5dff:febd:1409%vethd8013ee<0> [3]vethfa79788:fe80::6813:94ff:fe9a:17e4%vethfa79788<0>
pbg-dgx-1:1243335:1244249 [6] NCCL INFO Using network Socket
pbg-dgx-1:1243335:1244249 [6] NCCL INFO Setting affinity for GPU 6 to ffff0000,00000000,00000000,00000000,ffff0000,00000000,00000000
pbg-dgx-1:1243335:1244249 [6] NCCL INFO NVLS multicast support is not available on dev 6
pbg-dgx-1:1243335:1244249 [6] NCCL INFO Trees [0] 7/-1/-1->6->5 [1] 7/-1/-1->6->5 [2] 7/-1/-1->6->5 [3] 7/-1/-1->6->5 [4] 7/-1/-1->6->5 [5] 7/-1/-1->6->5 [6] 7/-1/-1->6->5 [7] 7/-1/-1->6->5 [8] 7/-1/-1->6->5 [9] 7/-1/-1->6->5 [10] 7/-1/-1->6->5 [11] 7/-1/-1->6->5 [12] 7/-1/-1->6->5 [13] 7/-1/-1->6->5 [14] 7/-1/-1->6->5 [15] 7/-1/-1->6->5 [16] 7/-1/-1->6->5 [17] 7/-1/-1->6->5 [18] 7/-1/-1->6->5 [19] 7/-1/-1->6->5 [20] 7/-1/-1->6->5 [21] 7/-1/-1->6->5 [22] 7/-1/-1->6->5 [23] 7/-1/-1->6->5
pbg-dgx-1:1243335:1244249 [6] NCCL INFO P2P Chunksize set to 524288
pbg-dgx-1:1243335:1244249 [6] NCCL INFO Channel 00/0 : 6[b7000] -> 7[bd000] via P2P/IPC/read
pbg-dgx-1:1243335:1244249 [6] NCCL INFO Channel 01/0 : 6[b7000] -> 7[bd000] via P2P/IPC/read
pbg-dgx-1:1243335:1244249 [6] NCCL INFO Channel 02/0 : 6[b7000] -> 7[bd000] via P2P/IPC/read
pbg-dgx-1:1243335:1244249 [6] NCCL INFO Channel 03/0 : 6[b7000] -> 7[bd000] via P2P/IPC/read
pbg-dgx-1:1243335:1244249 [6] NCCL INFO Channel 04/0 : 6[b7000] -> 7[bd000] via P2P/IPC/read
pbg-dgx-1:1243335:1244249 [6] NCCL INFO Channel 05/0 : 6[b7000] -> 7[bd000] via P2P/IPC/read
pbg-dgx-1:1243335:1244249 [6] NCCL INFO Channel 06/0 : 6[b7000] -> 7[bd000] via P2P/IPC/read

pbg-dgx-1:1243335:1244302 [6] include/alloc.h:178 NCCL WARN Cuda failure 'out of memory'

pbg-dgx-1:1243335:1244302 [6] include/alloc.h:185 NCCL WARN Failed to CUDA calloc 6291456 bytes
pbg-dgx-1:1243335:1244302 [6] NCCL INFO transport/p2p.cc:204 -> 1
pbg-dgx-1:1243335:1244302 [6] NCCL INFO transport/p2p.cc:584 -> 1
pbg-dgx-1:1243335:1244302 [6] NCCL INFO proxy.cc:1303 -> 1
pbg-dgx-1:1243335:1244302 [6] NCCL INFO proxy.cc:1377 -> 1

pbg-dgx-1:1243335:1244302 [6] proxy.cc:1518 NCCL WARN [Proxy Service 6] Failed to execute operation Setup from rank 6, retcode 1

pbg-dgx-1:1243335:1244249 [6] misc/socket.cc:49 NCCL WARN socketProgress: Connection closed by remote peer pbg-dgx-1.egr.duke.edu<50889>
pbg-dgx-1:1243335:1244249 [6] NCCL INFO misc/socket.cc:746 -> 6

pbg-dgx-1:1243335:1244249 [6] proxy.cc:1143 NCCL WARN Socket recv failed while polling for opId=0x7f8be9d30b00
pbg-dgx-1:1243335:1244249 [6] NCCL INFO transport/p2p.cc:386 -> 3
pbg-dgx-1:1243335:1244249 [6] NCCL INFO transport.cc:33 -> 3
pbg-dgx-1:1243335:1244249 [6] NCCL INFO transport.cc:106 -> 3
pbg-dgx-1:1243335:1244249 [6] NCCL INFO init.cc:1032 -> 3
pbg-dgx-1:1243335:1244249 [6] NCCL INFO init.cc:1309 -> 3
pbg-dgx-1:1243335:1244249 [6] NCCL INFO group.cc:64 -> 3 [Async thread]
pbg-dgx-1:1243335:1243335 [6] NCCL INFO group.cc:422 -> 3
pbg-dgx-1:1243335:1243335 [6] NCCL INFO group.cc:106 -> 3
pbg-dgx-1:1243335:1243335 [6] NCCL INFO comm 0x55a0a637e960 rank 6 nranks 8 cudaDev 6 busId b7000 - Abort COMPLETE

FYI, I am using a docker container. It can be reproduced by the following steps.
docker login nvcr.io
Username: $oauthtoken
Password NGc3bWIxM21mbTI0dTBraHE5N2U0NG1saWg6ZTY4MzlhZmUtYTJlZC00NDVmLThjYmEtNjA2ZTMzMzRkZTYy

Pull the Bionemo container:
docker pull nvcr.io/nvidia/clara/bionemo-framework:1.2

Run the container:
CONTAINER="nvcr.io/nvidia/clara/bionemo-framework:1.2"
DEST_PATH="."
CONTAINER_NAME=bionemo
docker run --name $CONTAINER_NAME -itd --rm $CONTAINER bash

To reproduce my error:
You can copy this file feature/system-check/src/lightning/fabric/utilities/system_check.py to the container and run it.

I won't have the bandwidth to help much here. Maybe try disabling plugins: NCCL_NET_PLUGIN=none. And if you are running inside the docker, please make sure that it's picking the correct network interface. Run the system check outside the container in a clean environment to see if it's related to the container or not.

I think the problem I have on  nccl-rank-6 is just OOM based on the log?
pbg-dgx-1:1243335:1244302 [6] include/alloc.h:178 NCCL WARN Cuda failure 'out of memory'

if you ran my system check, that's not possible. It allocates very little memory on the GPU:
https://github.com/Lightning-AI/pytorch-lightning/blob/297e9809d2da7ec2abb0f2e7c5e6c371ae0eaac8/src/lightning/fabric/utilities/system_check.py#L118
If what you show me there is the output of another program, then yes it looks like one rank runs out of memory. If one rank dies, the others will wait and hang forever.

Yeah. I think it is because some of the GPUs are already heavily utilized and triggers the OOM problem as shown in the log. I only ran your program in the container and the host, both of log showing OOM on two utilized GPUs.


[Pico W micropython] Bayesian optimization from scratch

https://machinelearningmastery.com/what-is-bayesian-optimization/ except with Gaussian Processes from scratch also
suggestion by @kir486680

I can make a quick jupyter notebook for this, and you can take a look at it.

@kir486680 do you have a notebook for this?

One of the following resources is probably useful for GPR from scratch:

https://github.com/sp8rks/MaterialsInformatics/blob/main/worked_examples/gaussian_process/coing_gpr_from_scratch_notebook.ipynb (original source of notebook)
https://peterroelants.github.io/posts/gaussian-process-tutorial/
https://towardsdatascience.com/implement-a-gaussian-process-from-scratch-2a074a470bce

For now, sticking with the current solution due to complexity (100's lines) of from-scratch GPR + Bayesian Optimization


Typo + sample code hidden input field

I removed the word 'your' in the paragraph about hidden input fields and added the sample code for adding one.

Thanks!

It was a good exercise for learning to work with Github!