Added recovery code option to 2fa

Details

User can use recovery code instead of two-factor authentication code by switching to recovery code input
Fixed Issues

$ https://github.com/Expensify/App/issues/22335
PROPOSAL: https://github.com/Expensify/App/issues/22335#issuecomment-1628319424
Tests

Have a user whose 2fa is enabled
Sign in with this user
At two-factor authentication step make sure Use recovery code text link is present
Make sure pressing it switches input type to simple text input
Enter recovery code
Make sure recovery code is being validated correctly and you can sign in with correct recovery code

[x] Verify that no errors appear in the JS console

Offline tests

Same as tests
QA Steps

Same as tests

[x] Verify that no errors appear in the JS console

PR Author Checklist

[x] I linked the correct issue in the ### Fixed Issues section above
[x] I wrote clear testing steps that cover the changes made in this PR

[x] I added steps for local testing in the Tests section
[x] I added steps for the expected offline behavior in the Offline steps section
[x] I added steps for Staging and/or Production testing in the QA steps section
[x] I added steps to cover failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[x] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)
[x] I tested this PR with a High Traffic account against the staging or production API to ensure there are no regressions (e.g. long loading states that impact usability).

[x] I included screenshots or videos for tests on all platforms
[x] I ran the tests on all platforms & verified they passed on:

[x] Android / native
[x] Android / Chrome
[x] iOS / native
[x] iOS / Safari
[x] MacOS / Chrome / Safari
[x] MacOS / Desktop

[x] I verified there are no console errors (if there's a console error not related to the PR, report it or open an issue for it to be fixed)
[x] I followed proper code patterns (see Reviewing the code)

[x] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick)
[x] I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
[x] I verified that comments were added to code that is not self explanatory
[x] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[x] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method

[x] If any non-english text was added/modified, I verified the translation was requested/reviewed in #expensify-open-source and it was approved by an internal Expensify engineer. Link to Slack message:

[x] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[x] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is approved by marketing by adding the Waiting for Copy label for a copy review on the original GH to get the correct copy.
[x] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[x] I verified the JSDocs style guidelines (in STYLE.md) were followed

[x] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[x] I followed the guidelines as stated in the Review Guidelines
[x] I tested other components that can be impacted by my changes (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar are working as expected)
[x] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[x] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[x] I verified that if a function's arguments changed that all usages have also been updated correctly
[x] If a new component is created I verified that:

[x] A similar component doesn't exist in the codebase
[x] All props are defined accurately and each prop has a /** comment above it */
[x] The file is named correctly
[x] The component has a clear name that is non-ambiguous and the purpose of the component can be inferred from the name alone
[x] The only data being stored in the state is data necessary for rendering and nothing else
[x] If we are not using the full Onyx data that we loaded, I've added the proper selector in order to ensure the component only re-renders when the data it is using changes
[x] For Class Components, any internal methods passed to components event handlers are bound to this properly so there are no scoping issues (i.e. for onClick={this.submit} the method this.submit should be bound to this in the constructor)
[x] Any internal methods bound to this are necessary to be bound (i.e. avoid this.submit = this.submit.bind(this); if this.submit is never passed to a component event handler like onClick)
[x] All JSX used for rendering exists in the render method
[x] The component has the minimum amount of code necessary for its purpose, and it is broken down into smaller components in order to separate concerns and functions

[x] If any new file was added I verified that:

[x] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[x] If a new CSS style is added I verified that:

[x] A similar style doesn't already exist
[x] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(themeColors.componentBG))

[x] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[x] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[x] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[x] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[x] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.
[x] I have checked off every checkbox in the PR author checklist, including those that don't apply to this PR.

Screenshots/Videos

Web
https://github.com/Expensify/App/assets/59907218/d65c8a15-655e-445f-812e-26107ed9daae

Mobile Web - Chrome
https://github.com/Expensify/App/assets/59907218/2a6ddf40-777b-4fd4-a8ef-5808ae06bd73

Mobile Web - Safari
https://github.com/Expensify/App/assets/59907218/3cb94da5-6021-46ad-a6b3-fab7bcf16c07

Desktop
https://github.com/Expensify/App/assets/59907218/53f21571-0454-45a5-a82f-4a63762a95f4

iOS
https://github.com/Expensify/App/assets/59907218/4fe182fd-2fa9-4a85-83ed-ef46e228c264

Android
https://github.com/Expensify/App/assets/59907218/26e49726-9323-4cfe-894e-beb3ad4b082a

@abdulrahuman5196 Friendly bump

Translations are confirmed, and updated code with the latest main branch and solved conflicts.
Sorry for force push, I have changed my system and had some problems authenticating with github.
It is ready for review now
@abdulrahuman5196

@alitoshmatov We should clear recovery code data as well here https://github.com/Expensify/App/pull/23390/files#diff-873bd2e06c31caa8e7ca519b38cbd825dad372a68ea5f4e4f00bb27f60b9a9c7R185

@MonilBhavsar The recovery code field doesn't have auto-check after entering recovery code. Like we have to press signin after the recovery code has entered in the text input. But both magic code and 2FA code auto hit API after just entering value.
Do we want that? If so in this issue or followup issue?

@alitoshmatov There are multiple useEffects involving clearing/focusing of input based on magicCode and 2FA code status, can we check and include recovery code as well to logics or create useEffects for recovery code in similar way

@alitoshmatov We should clear recovery code data as well here https://github.com/Expensify/App/pull/23390/files#diff-873bd2e06c31caa8e7ca519b38cbd825dad372a68ea5f4e4f00bb27f60b9a9c7R185

Added clearing recovery code there.

@alitoshmatov There are multiple useEffects involving clearing/focusing of input based on magicCode and 2FA code status, can we check and include recovery code as well to logics or create useEffects for recovery code in similar way

I don't think there is a need for handling recovery code in similar cases
@abdulrahuman5196

There are multiple useEffects involving clearing/focusing of input based on magicCode and 2FA code status, can we check and include recovery code as well to logics or create useEffects for recovery code in similar way

I don't think there is a need for handling recovery code in similar cases.

@alitoshmatov  Could you kindly add information on why? Anyways i wasn't able to find any visible issues based on the same, but we need to be sure if we don't want to add similar effects. The reason I think is if something is applicable to 2FA code then the same logic should be applicable for recovery codes as well.

@MonilBhavsar The recovery code field doesn't have auto-check after entering recovery code. Like we have to press signin after the recovery code has entered in the text input. But both magic code and 2FA code auto hit API after just entering value. Do we want that? If so in this issue or followup issue?

By auto check, you mean auto submit? I think it is only for those numeric inputs. We can leave it here

Not sure why this exists, but it looks like it was introduced when refactoring to functional components, and these changes were commited under Guarantee clearing after state change message. As this message suggests, these changes make sure inputs are cleared when user goes from general validation code to 2fa code inputs.
https://github.com/Expensify/App/blob/b43c20f4336ea941a33fb52ce5d6ec0195956af5/src/pages/signin/ValidateCodeForm/BaseValidateCodeForm.js#L119-L131
This is one is here since we store validateCode in onyx state, and it makes sure they are matched.
https://github.com/Expensify/App/blob/b43c20f4336ea941a33fb52ce5d6ec0195956af5/src/pages/signin/ValidateCodeForm/BaseValidateCodeForm.js#L105-L110
And these are for managing focus state of validate code input, not sure why we need this since providing autofocus should work properly, and it seems redundant.
https://github.com/Expensify/App/blob/b43c20f4336ea941a33fb52ce5d6ec0195956af5/src/pages/signin/ValidateCodeForm/BaseValidateCodeForm.js#L91-L103
To conclude, most of these useEffects are here to manage MagicCodeInput. But I think there are some redundant code and this could be optimized. Anyways I think there is no need for similar changes for simple TextInput
@abdulrahuman5196 Let me know what you think.

@abdulrahuman5196 Friendly bump!

@abdulrahuman5196 👋

@abdulrahuman5196 could you please take a look

Hi, yes will take a look into it today. I was focusing more on proposals since we had PR merge freeze.

Resolved merge conflicts

Reviewer Checklist

[x] I have verified the author checklist is complete (all boxes are checked off).
[x] I verified the correct issue is linked in the ### Fixed Issues section above
[x] I verified testing steps are clear and they cover the changes made in this PR

[x] I verified the steps for local testing are in the Tests section
[x] I verified the steps for Staging and/or Production testing are in the QA steps section
[x] I verified the steps cover any possible failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[x] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)

[x] I checked that screenshots or videos are included for tests on all platforms
[ ] I included screenshots or videos for tests on all platforms
[x] I verified tests pass on all platforms & I tested again on:

[x] Android / native
[x] Android / Chrome
[x] iOS / native
[x] iOS / Safari
[x] MacOS / Chrome / Safari
[x] MacOS / Desktop

[ ] If there are any errors in the console that are unrelated to this PR, I either fixed them (preferred) or linked to where I reported them in Slack
[x] I verified proper code patterns were followed (see Reviewing the code)

[x] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick).
[x] I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
[x] I verified that comments were added to code that is not self explanatory
[x] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[x] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method
[x] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[x] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is approved by marketing by adding the Waiting for Copy label for a copy review on the original GH to get the correct copy.
[x] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[x] I verified the JSDocs style guidelines (in STYLE.md) were followed

[x] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[x] I verified that this PR follows the guidelines as stated in the Review Guidelines
[x] I verified other components that can be impacted by these changes have been tested, and I retested again (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar have been tested & I retested again)
[x] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[x] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[x] If a new component is created I verified that:

[x] A similar component doesn't exist in the codebase
[x] All props are defined accurately and each prop has a /** comment above it */
[x] The file is named correctly
[x] The component has a clear name that is non-ambiguous and the purpose of the component can be inferred from the name alone
[x] The only data being stored in the state is data necessary for rendering and nothing else
[x] For Class Components, any internal methods passed to components event handlers are bound to this properly so there are no scoping issues (i.e. for onClick={this.submit} the method this.submit should be bound to this in the constructor)
[x] Any internal methods bound to this are necessary to be bound (i.e. avoid this.submit = this.submit.bind(this); if this.submit is never passed to a component event handler like onClick)
[x] All JSX used for rendering exists in the render method
[x] The component has the minimum amount of code necessary for its purpose, and it is broken down into smaller components in order to separate concerns and functions

[x] If any new file was added I verified that:

[x] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[x] If a new CSS style is added I verified that:

[x] A similar style doesn't already exist
[x] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(themeColors.componentBG)

[x] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[x] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[x] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[x] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[x] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.
[ ] I have checked off every checkbox in the PR reviewer checklist, including those that don't apply to this PR.

Screenshots/Videos

Web

Mobile Web - Chrome

Mobile Web - Safari

Desktop

iOS

Android

Note: I see a slight non-smooth transition on mobile safari alone when switching between inputs. But I don't think its related to this PR since it could be related to re-render on switching input and focus is already not better in safari.
cc: @MonilBhavsar

We also need to change the text above the input that says - Please enter your authenticator code to Please enter your recovery code

@abdulrahuman5196
The message above inputs is managed in SinInPage
https://github.com/Expensify/App/blob/5d5b956dfba80629f17e95a3eeb753dc37ca5580/src/pages/signin/SignInPage.js#L127-L130
I am thinking to shift isUsingRecoveryCode state on level up from BaseValidateCodeForm to SingInPage this way we can keep track of the state and change the message accordingly.
What do you think about this?

I am thinking to shift isUsingRecoveryCode state on level up from BaseValidateCodeForm to SingInPage this way we can keep track of the state and change the message accordingly.

I don't understand this. How do to plan to move a internal state value to its parent component? Currently SignInPage determines the text above input from onyx.

@abdulrahuman5196 I mean, we declare isUsingRecoveryCode state in SignInPage and pass its value, and change function to BaseValidateCodeForm, thus making this state available in both places. We add additional condition using this state to determine the message.
Another way is to declare new state in onyx data and keep track of if user is using recovery code

To be honest, a new Onyx data for maintaining state or user method sounds overkill. Would be great if we can do with state

@abdulrahuman5196 Moved state to SignInPage from BaseValidateCodeForm, thus making welcome text respond when switched to recovery code

@abdulrahuman5196 Moved state to SignInPage from BaseValidateCodeForm, thus making welcome text respond when switched to recovery code

@MonilBhavsar I am not in that much agreement to this approach, but still i agree on the point of onyx being overkill as well. So I am 50-50 on this. Could you kindly give a quick look at the new changes and let me know if this kind of state management is fine?

That looks good to me. We're taking it one component up to pass the correct welcomeText prop, right? @abdulrahuman5196 anything specific you think does not follow good state management practice?

@abdulrahuman5196 Friendly bump

Reviewer Checklist

[x] I have verified the author checklist is complete (all boxes are checked off).
[x] I verified the correct issue is linked in the ### Fixed Issues section above
[x] I verified testing steps are clear and they cover the changes made in this PR

[x] I verified the steps for local testing are in the Tests section
[x] I verified the steps for Staging and/or Production testing are in the QA steps section
[x] I verified the steps cover any possible failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[x] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)

[x] I checked that screenshots or videos are included for tests on all platforms
[ ] I included screenshots or videos for tests on all platforms
[x] I verified tests pass on all platforms & I tested again on:

[x] Android / native
[x] Android / Chrome
[x] iOS / native
[x] iOS / Safari
[x] MacOS / Chrome / Safari
[x] MacOS / Desktop

[ ] If there are any errors in the console that are unrelated to this PR, I either fixed them (preferred) or linked to where I reported them in Slack
[x] I verified proper code patterns were followed (see Reviewing the code)

[x] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick).
[x] I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
[x] I verified that comments were added to code that is not self explanatory
[x] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[x] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method
[x] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[x] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is approved by marketing by adding the Waiting for Copy label for a copy review on the original GH to get the correct copy.
[x] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[x] I verified the JSDocs style guidelines (in STYLE.md) were followed

[x] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[x] I verified that this PR follows the guidelines as stated in the Review Guidelines
[x] I verified other components that can be impacted by these changes have been tested, and I retested again (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar have been tested & I retested again)
[x] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[x] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[x] If a new component is created I verified that:

[x] A similar component doesn't exist in the codebase
[x] All props are defined accurately and each prop has a /** comment above it */
[x] The file is named correctly
[x] The component has a clear name that is non-ambiguous and the purpose of the component can be inferred from the name alone
[x] The only data being stored in the state is data necessary for rendering and nothing else
[x] For Class Components, any internal methods passed to components event handlers are bound to this properly so there are no scoping issues (i.e. for onClick={this.submit} the method this.submit should be bound to this in the constructor)
[x] Any internal methods bound to this are necessary to be bound (i.e. avoid this.submit = this.submit.bind(this); if this.submit is never passed to a component event handler like onClick)
[x] All JSX used for rendering exists in the render method
[x] The component has the minimum amount of code necessary for its purpose, and it is broken down into smaller components in order to separate concerns and functions

[x] If any new file was added I verified that:

[x] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[x] If a new CSS style is added I verified that:

[x] A similar style doesn't already exist
[x] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(themeColors.componentBG)

[x] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[x] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[x] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[x] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[x] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.
[ ] I have checked off every checkbox in the PR reviewer checklist, including those that don't apply to this PR.

Screenshots/Videos

Web

Mobile Web - Chrome

Mobile Web - Safari

Desktop

iOS

Android

https://github.com/Expensify/App/assets/46707890/7ab63a4a-da57-4486-9951-fc0478dd2429
https://github.com/Expensify/App/assets/46707890/53301d7f-0476-497b-ab71-22eca18fc490
https://github.com/Expensify/App/assets/46707890/d4b2d55e-d94d-41c2-a0b2-1259e42e0595
https://github.com/Expensify/App/assets/46707890/1a2a3c1b-b78a-4d80-b0a9-92d009d4f820
https://github.com/Expensify/App/assets/46707890/adccc782-cf80-47bc-8d2f-65dce9f4bcdd

@alitoshmatov could you please merge main in this branch

@MonilBhavsar Done

:hand: This PR was not deployed to staging yet because QA is ongoing. It will be automatically deployed to staging after the next production release.

🚀 Deployed to staging by https://github.com/MonilBhavsar in version: 1.3.73-0 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

🚀 Deployed to production by https://github.com/mountiny in version: 1.3.73-1 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
failure ❌

🕸 web 🕸
success ✅

🚀 Deployed to staging by https://github.com/MonilBhavsar in version: 1.3.74-0 🚀

platform
result

🤖 android 🤖
failure ❌

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

🚀 Deployed to production by https://github.com/chiragsalian in version: 1.3.74-3 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
failure ❌

🕸 web 🕸
success ✅


Write migration guide for azure-spring-cloud-starter

Parent issue: #24351

Already merged to 4.0 branch, closing this issue now.


BlueChip Chain Registry First Push

We are a chain focusing on a decentralized subscription model to enhance content creation.

Looks like you've made a new PR. Can his be closed then?

Yes it can be.
Nothing but the best
Nick Davis,
[cid:886ede93-8ed9-4790-a8dd-5a753858c081]

From: JeremyParish69 @.>
Sent: Saturday, June 22, 2024 2:10 PM
To: cosmos/chain-registry @.>
Cc: Nick Davis @.>; Author @.>
Subject: Re: [cosmos/chain-registry] BlueChip Chain Registry First Push (PR #4730)
Looks like you've made a new PR. Can his be closed then?
—
Reply to this email directly, view it on GitHubhttps://github.com/cosmos/chain-registry/pull/4730#issuecomment-2184158790, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AT2H5ZX2MTZ2O4UN57CBV3TZIXDYVAVCNFSM6AAAAABIN4CYQ6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOBUGE2TQNZZGA.
You are receiving this because you authored the thread.Message ID: @.***>

Is there anything else you will need from me?
Get Outlook for iOShttps://aka.ms/o0ukef

From: Nick Davis @.>
Sent: Saturday, June 22, 2024 2:49:09 PM
To: cosmos/chain-registry @.>
Subject: Re: [cosmos/chain-registry] BlueChip Chain Registry First Push (PR #4730)
Yes it can be.
Nothing but the best
Nick Davis,
[cid:886ede93-8ed9-4790-a8dd-5a753858c081]

From: JeremyParish69 @.>
Sent: Saturday, June 22, 2024 2:10 PM
To: cosmos/chain-registry @.>
Cc: Nick Davis @.>; Author @.>
Subject: Re: [cosmos/chain-registry] BlueChip Chain Registry First Push (PR #4730)
Looks like you've made a new PR. Can his be closed then?
—
Reply to this email directly, view it on GitHubhttps://github.com/cosmos/chain-registry/pull/4730#issuecomment-2184158790, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AT2H5ZX2MTZ2O4UN57CBV3TZIXDYVAVCNFSM6AAAAABIN4CYQ6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOBUGE2TQNZZGA.
You are receiving this because you authored the thread.Message ID: @.***>


Module missing

Hi,
Using python 3.8.6 and Peregrine 1.8.0 (I think...) I get next error when executing the code in the example multiple_coins_multiple_exchanges.py:
ModuleNotFoundError: No module named 'networkx'
Can you please explain how can I install it or if I'm doing something wrong?
Thank you

sudo pip3 install networkx should do it.

It worked.
thanks!


Template was changed

'Store_module' was removed

Coverage decreased (-0.3%) to 66.667% when pulling 3f45fe408741fa93f764bbf3d8e57ba929133ab2 on proggga:patch-2 into 0015c310d887dcc3e7ac558a93dba5039dbf6160 on klen:develop.

Thank you!


Create Template Files for protoc-gen-frisbee

As noted here: https://github.com/loopholelabs/frisbee/pull/78#discussion_r825221129, and here: https://github.com/loopholelabs/frisbee/pull/78#discussion_r825220805
We are doing a lot of manual looping to generate the Frisbee RPC Framework.
We should take some time and write text/template templates to handle this in a more legible format.

Alex is working on this but not onboarded onto Zenhub yet so I can't assign him.


Broken link for Add a data source to use for virtual tables

Hello,
in this section under Point 4 in the table, the first link is broken:
https://docs.microsoft.com/en-us/powerapps/maker/data-platform/create-edit-virtual-entities#add-a-data-source-to-use-for-virtual-tables

Thank you,
Manuela

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 994c6d0a-b4d5-78dc-8151-2b55ef67a977
Version Independent ID: c508384a-3267-2a24-3136-4f5abbb74d19
Content: Create and edit virtual tables with Microsoft Dataverse - Power Apps
Content Source: powerapps-docs/maker/data-platform/create-edit-virtual-entities.md
Service: powerapps
GitHub Login: @Mattp123
Microsoft Alias: matp

@manuelap-msft - thanks for your feedback. The link has been fixed.
Closing issue.


some questions about inference models and submodule

Question 1:
https://bitbucket.org/tamiki/terrain_data_generator.git
This repo link is 404: Repository not found

Question 2:
as the README said, it has Train or generation steps. Is there any pre-trained model in the repo that with input point cloud we can get the gridmap message?

Question 1:
https://bitbucket.org/tamiki/terrain_data_generator.git
This repo link is 404: Repository not found

Facing the same issue

Facing the same issue


[RFF] JSON output logging

Right now parsing result outputs from CME is not really intuitive and requires a lot of crafty bash one-liners.
To address this issue and improve on the tool, how about offering JSON output (with --json) to allow for easily parsing (e.g. extract hosts, users, passwords, shares) with the help of jq and ConvertFrom-JSON.

I really really would this option as well. It does exactly what we need, but I need to enrich the data with our CMDB. To do that I need to parse the data somehow. JSON output would be much appreciated.

Would be a great feature +1

The sponsors version has a log feature where everything is send to a file.

@aress31 @ict-one-nl @SmashRocket @Senne-s @daanwa Do you mean the entire output (so instead of the normal output, it's all JSON), or just an option to have normal STDOUT prints and JSON to a file, or a way to dump CMEDB information as JSON? Could you give some examples and I can see what I can put together to fit the needs.

I want the results of the tool to be incorporated into our tooling. However, the current output shown on STDOUT is somewhat cumbersome to parse. So in my case it would be most handy to have a command line flag that outputs the results in JSON to a file.
Preferably I would not have to use the cmedb cli and I could just provide the JSON output flag to the cme app.

@daanwa Thanks for that information. It might be pretty hard to do, but I can look into it. Can you provide an example of how you would expect the JSON structure to look like with several hosts & modules?

Yup, I'd like to parse the output with something like a python script to write to opensearch and enrich the info with contents of our CMDB :)

That's up to my scripting skills (to ingest data into our SIEM and generate reports). But to do anything useful I need to interact with the data and for that I'd need parsable output and json works fine for that :)

@ict-one-nl What sort of structure would you expect as output? Can you provide an example?

Hmm, doesn't really matter, I can image a header like object for the general stuff and a list of objects for each line outputted by CME containing all the details for that line?

but what's the point ? I don't understand

Exactly. It is mainly aimed at integrating with some of our other tooling or for collecting additional authentication information about certain data points.
@mpgn Perhaps there is already a different way that can do this and I'm just not aware of it? Currently I'm parsing the sqlite databases but I've seen that not all commands/modules populate the db's.

Using the SQLite db quite cumbersome for what's needed in this case. Just a different formatter for the data that's already there would suffice :)

Okay, did it the sqlite way for now:
sqlite3 -json /root/.cme/default/smb.db 'select * from shares inner join computers on computers.hostname = shares.computerid  where read = true and name NOT LIKE "%print%"' > /root/.cme/output.json

So, my 2 cents, if we add a json option it should be used in combinaison with --log file option.
cme <proto> <ip> -u <user> -p <pass> --log filename --json
And the output in the log file 'filename' will be in json. I know @NeffIsBack already work on this, I think this is the best way to do it.

Yeah that would work for me!


Documenting node discover annotations

This PR document nfd annotations to https://kubernetes.io/docs/reference/labels-annotations-taints/ document,
referring to comment here
Fixes: #31612

/kind feature
/sig node

@sftim PTAL

The node-feature-discovery component must be setup for these annotations to show up, we should document this as a precondition.

i agree with this. one way is to mention the setup in a note before the annotations.

I'm not sure these annotations warrants "well-known labels or annotations".

I have added the content on basis of this comment.
Also This addon can be helpful,  if not this page then maybe some other one we can mention it. NFD seems quite interesting and it would be great to have an example to use this addon in the docs.

/assign @marquiz

@sftim @divya-mohan0209 PTAL

I have checked before pushing the commit , idk why the checks are failing? It shows
Build exceeded maximum allowed runtime

The build failures are a known issue, and one we'd love to fix.

If the build fails after roughly 1 hour, it's almost certainly issue #35780

It never happened with me before, thanks for letting me know @sftim, i was checking on my system what could have gone wrong

Hi @sftim @marquiz PTAL! I rebased it again.

Thanks for the review @tengqm . I have updated as per the review.

/lgtm

I'd love to know if we need any more changes in this one. Please let me know in such case.

/retitle Document node discovery annotations

/approve


Documentation for QuBase

It would be great if we could use Docile package for documentation. As we are basing our initial attempt on Julia release version 0.3, we could use Docile as a package, this dependency would not be needed in the future as the next version of Julia has documentation in built, which is a merge of Docile. I have looked through this package which is on similar lines in terms of development depending on Julia 0.3 and using Docile [HttpServer.jl] (https://github.com/JuliaWeb/HttpServer.jl). It would be great to hear some thoughts on this. (I have created a [branch] (https://github.com/amitjamadagni/QuBase.jl/tree/doc) with initial commits which has edits in the REQUIRE as well as added a doc folder with README.md file).

+100 We should definitely use Docile to do internal and external documentation.

I agree as well! Inline documentation rocks.

I haven't looked too much into this, but is there a way to have "internal" and "external" (or "expert" and "user") documentation with Docile? I am thinking of having one set of documents, which cover all the dirty details, like interface descriptions and so on. And a another set, which would basically document exported functions/types for the user.

Are we going to use Lexicon.jl as well, then? I think it only supports HTML/markdown. Not a problem for me, but I know a lot of julia users prefer ReStructured text + Sphinx for documenting.

I guess as Docile/Lexicon are or will be the canonical way to do documentation in Julia, other formats will also be supported if there is enough interest. For us it might be important to have some way of including equations. Is that supported by Lexicon?

@acroy I don't see any specific feature for internal documentation. That would be nice to have, for sure. We might be able to configure Lexicon to save certain parts of the docs differently - I know Docile has separate a @comment macro but I'm not sure what the differences are there.
As for LaTeX support, we'd be far better off with using Sphinx, which I'm not sure will work well with Docile + Lexicon since Sphinx support RST, not markdown.
I suppose this is a more complex issue than I initially thought...

@jrevels : Maybe not. Lexicon and Docile manuals have sections "Manual" and "API", which would do for external and internal documentation. We just have to figure out how it works :-) Also, the Lexicon docs mention MathJax - so I guess the HTML pages can display equations.

There have been discussions of doxygen support for Julia. I guess that could solve the problem to a good extent. But I really dont know how much of it has gone into support for Julia.

@acroy It seems I missed the MathJax support, good catch. I don't know if Lexicon's markdown parser handles nesting MathJax syntax on its own, but it seems that MkDocs has a MathJax extension, so I think we're good either way.
It seems that what Lexicon provides is a way to distinguish between documentation on exported vs. non-exported objects (see the include_internal flag for the save function). AFAIK, that means we wouldn't be able to distinguish between internal and external inline documentation for exported objects. I suppose we could simply say any "internal" documentation on exported objects should simply be left as a normal comment (or the @comment macro, depending on how Lexicon renders it), or be marked off in its own section rendered with standard markdown/not documented inline with the code.
Having different named categories like "Manual"/"API"/etc. is a feature of MkDocs/RTD in general. I don't think we want to equate "Manual"/"API" with "external"/"internal", though, since an API is generally a list of all the user-facing functions/types (apologies if I misinterpreted your wording there).

You are right, we shouldn't use the "API" section for internal documentation. The latter is really for us to keep track of implementation details etc. But in any case it seems we should go with Docile.jl+Lexicon.jl.


System-test is not cleaning up instances properly

system-test failed to clean up test instances with not found errors like: Error: 5 NOT_FOUND: Failed to read: projects/{818418350420}/instances/gcloud-tests-d-inst-c089e871 which causes the test project to run out of node quota.

Is it throwing the error on a test and then as a result, not running one of the after hooks?

Also happens here: https://github.com/googleapis/nodejs-bigtable/issues/1099

https://github.com/googleapis/nodejs-bigtable/issues/1113

https://github.com/googleapis/nodejs-bigtable/issues/975

Looking into this further, it looks like https://github.com/googleapis/nodejs-bigtable/blob/main/system-test/bigtable.ts#L110 should be cleaning up instances, but is not? Maybe we could always delete all existing instances before running tests? Maybe errors prevent the after hook from running?

I think now we rename the tests so that they include a date and we can find another mechanism for cleaning up leftover instances.


Use force layout with angular 2

How can i use force pack layout of bubble chart in wrapper with angular 4?

For general "how to" questions related to the use of D3 in conjunction with Angular, please use StackOverflow. For one, the community is broader there for support purposes and others can not only provide guidance, but also benefit from the suggested answers. The issues for this demo should be used for bug reports or specific improvement suggestions. Thanks for understanding. :smile:


how to upload and read file?

I can not know how to read uploaded file with Moleculer.
Any one can helps me?

Here is an example: https://github.com/ice-services/moleculer-web/blob/master/examples/full/index.js#L209


A lot of deleted video on playlist

Checklist

[x] I'm asking a question

[ ] I've looked through the README and FAQ for similar questions

[ ] I've searched the bugtracker for similar questions including closed ones

Question

WRITE QUESTION HERE
i find that if you download a huge watch later playlist
e.g a watch later list with 5k video but there are a lot of deleted video
it will stop running and can't download the full playlist with options -c -i
video 1 to 1000
video 1001 to 3000 are deleted
video 3001 to 5000 are available
youtube-dl will download 1 to 1000 video and then stop and show FINISHED download from [watch later playlist]
and same problem when you scroll down the playlist in browser also stop at 1000
but it can use options --playlist-reverse -c -i to download the list without problem

options playlist-start 1001 also not work
likely cant get the playlist fully when many Deleted video in playlist


Fixes #36 Complete DynamicValue (previously known as Generic) implementation

This still may have some issues related to multiply nested Schemas but haven't quite figured out whether the issue is with the way I have implemented the generators or whether it is an implementation issue. Either way I can continue to try and sort it out, but wanted to post this for review.
This should also make #63 pretty trivial as it is already more or less implemented by allowing derivation of arbitrary DynamicValues from a Schema (then you can get back to the scala type with Schema.fromGeneric).

@thinkharderdev Really nice work! Just a few minor comments, then good to merge.

Thank you! 🙏


iOS 16

mount_apfs: volume could not be mounted: No such file or directory
AssertMacros:(value = 0x0),  file: seputil.c, line: 2783
seputil: can't open '/mnt1/usr/standalone/firmware/sep-firmware.img4', errno: No such file or directory(2)
seputil.c: failed to load /mnt1/usr/standalone/firmware/sep-firmware.img4 in  mode
mount_apfs: volume could not be mounted: No such file or directory

can you run ls /dev/disk*

pinned potential mountinmg issuer


📅 5/27 @ 1pm PT - transfer issues + comments between repositories while retaining authorship, labels, and milestones

💁🏻 transfer issues + comments between repositories while retaining authorship, labels, and milestones
📅 Wednesday, May 27, 2021
🕐 1:00pm Pacific Time (in your timezone)
🎙 let me know if you'd like to join
🏷 Automation

transfer issues + comments between repositories while retaining authorship, labels, and milestones
This show is a follow up to the idea described at https://github.com/gr2m/helpdesk/issues/12 and the show from May 20th where we created a script to copy a repository including issues, comments, milestones, and labels. With the script from last week we will be able to quickly create repositories we can use to test this show's script.

Outline

[ ] Recap the script created in #16 and use it to create two repositories for testing: source and target

https://github.com/Another-Practice-Organization

https://github.com/gr2m/copy-repository

https://create-user-to-server-token.netlify.app/ (create tokens using credentials of https://github.com/apps/another-practice-org-tokens)

Create test repository for source
GITHUB_TOKENS=$COPY_REPOSITORY_GITHUB_TOKENS SOURCE_REPO=Another-Practice-Organization/template-issues-source npx copy-repository

Create test repository for target
GITHUB_TOKENS=$COPY_REPOSITORY_GITHUB_TOKENS SOURCE_REPO=Another-Practice-Organization/template-issues-target npx copy-repository

Delete test repositories
octoherd-script-delete-repository \
  -T $GITHUB_TOKEN_DELETE_REPO \
  -R "Another-Practice-Organization/template-issues-source-copy-*" "Another-Practice-Organization/template-issues-target-copy-*"

[ ] Transfer all issues source to target
[ ] Make sure the same labels are applied. Create labels that are missing
[ ] Make sure the same milestones are applied. Create Milestones if they are missing

Recording
will be added after the show

Shownotes
will be added after the show
Preparation

[ ]  30 minute announcement tweet
[ ]  start of show tweet
[ ]  comment on issue
[ ]  recording available tweet

now live on https://www.twitch.tv/gregorcodes

🤦🏼  My mic was muted the whole time
here is the video anyway if you are interested: https://www.twitch.tv/videos/1037245669
You can find the code from this episode at https://github.com/gr2m/transfer-issues/
I'm not going to archive this repository on YouTube because it's embarrassing ... ugh


Testing Adding Issue

I like Microbetrace

Great! Always good to get feedback. Bill
From: roxycintron<EMAIL_ADDRESS>Sent: Thursday, September 24, 2020 3:26 PM
To: CDCgov/MicrobeTrace<EMAIL_ADDRESS>Cc: Subscribed<EMAIL_ADDRESS>Subject: [CDCgov/MicrobeTrace] Testing Adding Issue (#280)
I like Microbetrace
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHubhttps://github.com/CDCgov/MicrobeTrace/issues/280, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AHMTDBI2I35LTZBECXQSA7TSHOMK7ANCNFSM4RYV3LBQ.


Problems replicating popups demonstration

I am having trouble replicating the functionality you showcased in your 'popups' demonstration video (https://user-images.githubusercontent.com/17562139/77829950-406e7000-7160-11ea-85aa-0f966feeb237.gif)
Could you please share your .tmux.conf, fzfp wrapper scripts, and any other configuration that makes this possible? For example, opening a file selected inside of the popup window inside of a vim buffer. Not sure how you are performing different send-keys functions on the same popup?
CONFIG PLZ :)

It's zsh completion + fzf + tmux. You can refer to my dotfiles. https://github.com/kevinhwang91/dotfiles/blob/75192e1dfbbd89725e500d2e0bf18585fbcb5c28/zsh/lplug/fzf/fzf-complete-lsof#L1, but my zsh config is complicated for others.

opening a file selected inside of the popup window inside of a vim buffer.

I don't use its feature, but you can use fzf.vim, it supports the tmux popup selecting in vim.

In the video, you get a popup, and then you select a file and sent it to pane 1 and it opens in neovim?... How do you do this?

type nvim <C-t> <C-t> has bind from  fzf-find-widget under zsh/lplug/fzf


ClientTransactionManager naming

Where is the name ClientTransactionManager coming?
It seems to me, it is a PrivateTransactionManager?

This sounds sensible, please submit a PR with your proposed changes


Infer AWS SDK config for a project from AWS references

When a project has a reference to an AWS CloudFormation resource that project can now infer the AWS SDK configuration from the reference CloudFormation. This avoids users having to add the AWS SDK configuration to both the CloudFormation resource and Project resource. It might be possible a user wants a different AWS SDK configuration then was used for CloudFormation. For example use a profile with reduced permissions. If a user still calls WithReference with an AWS SDK configuration it will override any inferred AWS SDK configuration.
Basically this
var awsConfig = builder.AddAWSSDKConfig()
                        .WithProfile("default")
                        .WithRegion(RegionEndpoint.USWest2);

var awsResources = builder.AddAWSCloudFormationTemplate("AspireSampleDevResources", "app-resources.template")
                        .WithReference(awsConfig);

builder.AddProject<Projects.Frontend>("Frontend")
       .WithExternalHttpEndpoints()
       .WithReference(awsResources)
       .WithReference(awsSdkConfig);

Changes to
var awsConfig = builder.AddAWSSDKConfig()
                        .WithProfile("default")
                        .WithRegion(RegionEndpoint.USWest2);

var awsResources = builder.AddAWSCloudFormationTemplate("AspireSampleDevResources", "app-resources.template")
                        .WithReference(awsConfig);

builder.AddProject<Projects.Frontend>("Frontend")
       .WithExternalHttpEndpoints()
       .WithReference(awsResources);

Microsoft Reviewers: Open in CodeFlow

Do we need a doc change to @normj ?

Do we need a doc change to @normj ?

Yeah, I'm out this week. I can send a PR next week.


Update sp-fulltext-database-transact-sql.md

Fixed incorrect descriptions

Learn Build status updates of commit 5954638:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs/relational-databases/system-stored-procedures/sp-fulltext-database-transact-sql.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the learn.microsoft.com contributor guidesPost your question in the Learn support channel

@markingmyname

Can you review this PR?
IMPORTANT: When this content is ready to merge, you must add #sign-off in a comment or the approval may get overlooked.

#label:"aq-pr-triaged"
@MicrosoftDocs/public-repo-pr-review-team

#sign-off


Overcommited enrichments

That's a PR without actual links to YouTube channel, and there may be some message edits
I want to be sure that my approach is okay before the content is submitted

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


RadioButton

1/26-
Kick off meeting held with component group and Ben/Josh/Paul

Some refactors, waiting for some designs on the specs:
@alycrys to add more detail

Pending question: Do we want the icon to be customisable?

dev review needed


[semantic-sil] Move the ownership model eliminator past no return folding.

[semantic-sil] Move the ownership model eliminator past no return folding.
rdar://31521023

@swift-ci test and merge

The linux failure is due to swiftpm failing to load an external resource. Can't be related to this change.

@swift-ci smoke test linux platform

Lets see if that dtd issue was fixed...

@swift-ci smoke test linux platform

@swift-ci Please smoke test Linux platform

Not sure why I did not land this before. I think I thought I did a smoke test and merge


Few coverity fixes

This PR closes some of the most critical issues detected by Coverity.
See https://scan.coverity.com/projects/3971 for the full list (note that this analysis was performed on an older NuttX commit dbcccb2455d759b789d549d25e1fbf489b2d3c83)
The ones that are fixed here have status "Fix Submitted".

Committed upstream: http://sourceforge.net/p/nuttx/git/ci/bcfcb485b98d42d8c67e8948c016f5e766a64b29/


DOC: Adding Total number of PRs and Current ongoing issues (Open & Closed) & Last Commit,stars

What's wrong with the existing documentation
Adding Total number of PRs and Current ongoing issues (Open & Closed) & Last Commit,stars.
File to change
No response
Details of the change
Expected

Type of change
No response
Record

[X] I agree to follow this project's Code of Conduct
[X] I'm a GSSOC'24 contributor
[X] I want to work on this issue

@arghadipmanna101 please assign the above work to me

@arghadipmanna101 please give the labels like gssoc and level1


Race condition when updating an existing group and deployments

When issuing a request to update an existing group and then checking the list of deployments via /v2/deployments the deploymentId that I got back from a successful update of the group is not always present.
I had an assumption that when I received a deploymentId back from an update of a group that the deployment would be present in the api end point for deployments.
This is something that does not happen every time, however is does happen enough that I think it is a racey condition.

Will it eventually appear, or never?

It does eventually appear, this only happens sometimes.

That's just the normal effect of concurrency. The call to the HTTP endpoint returns as soon as the deployment has been accepted, which doesn't mean, that it is already running.

Please reopen, if that is still a problem.

@aquamatthias this is still an issue, I believe #1407 describes the same.

Note: This issue has been migrated to https://jira.mesosphere.com/browse/MARATHON-3680. For more information see https://groups.google.com/forum/#!topic/marathon-framework/khtvf-ifnp8.


Captcha Extension not working?

I need to use the Captcha extension for a project. It's perfect for it. I tried it and it shows up as it should, the lightning invoice appears, I test it, all good.
But when I try to skip the checkbox the user can still sign up and ignore the captcha. Was it supposed to block the user? How do I make it a prerequisite?
Thank you.

@pseudozach please help!


Angular with Oauth2.0

Hi John,
I wanna ask your advice for angular authentication using oauth 2.0. If you have references please tell me.
Thank You

appreciate the question, but this is not the appropriate place. stackoverflow may be better suited.
i have no specific advice for oauth other than do it on the server side. see Troy Hunt's pluralsight videos.


hpa failed to get current CPU utilisation

Newly setup cluster on ubuntu 16.04.2 with kubeadm. Then deployed heapster from instructions for influxdb. The cpu and mem utilisation all shows in the dashboard, but the hpa fails to see it:
$ kubectl get hpa
NAME                  REFERENCE                        TARGETS           MINPODS   MAXPODS   REPLICAS   AGE
xstreamline-cluster   Deployment/xstreamline-cluster   <unknown> / 50%   1         5         1          16m

$ kubectl get ev
...
2m         16m         11        xstreamline-cluster            HorizontalPodAutoscaler                                   Warning   FailedGetResourceMetric        horizontal-pod-autoscaler     unable to get metrics for resource cpu: no metrics returned from heapster
2m         16m         11        xstreamline-cluster            HorizontalPodAutoscaler                                   Warning   FailedComputeMetricsReplicas   horizontal-pod-autoscaler     failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from heapster

$ kubectl logs heapster-434383634-vthjw -n kube-system -f
I0419 08:25:09.401371       1 heapster.go:71] /heapster --source=kubernetes:https://kubernetes.default --sink=influxdb:http://monitoring-influxdb:8086 -v=2
I0419 08:25:09.401489       1 heapster.go:72] Heapster version v1.3.0-beta.1
I0419 08:25:09.401713       1 configs.go:61] Using Kubernetes client with master "https://kubernetes.default" and version v1
I0419 08:25:09.401732       1 configs.go:62] Using kubelet port 10255
I0419 08:25:09.464182       1 influxdb.go:252] created influxdb sink with options: host:monitoring-influxdb:8086 user:root db:k8s
I0419 08:25:09.464205       1 heapster.go:193] Starting with InfluxDB Sink
I0419 08:25:09.464211       1 heapster.go:193] Starting with Metric Sink
I0419 08:25:09.476049       1 heapster.go:105] Starting heapster on port 8082
W0419 08:25:14.381417       1 listers.go:68] can not retrieve list of objects using index : object has no meta: object does not implement the Object interfaces
I0419 08:25:14.381456       1 handlers.go:216] No metrics for pod default/xstreamline-cluster-1605009199-nn379
W0419 08:25:44.392313       1 listers.go:68] can not retrieve list of objects using index : object has no meta: object does not implement the Object interfaces
I0419 08:25:44.392352       1 handlers.go:216] No metrics for pod default/xstreamline-cluster-1605009199-nn379

Running with -v=4
$ kubectl logs heapster-614869780-76zmd -n kube-system -f
I0419 08:31:54.708423       1 heapster.go:71] /heapster --source=kubernetes:https://kubernetes.default --sink=influxdb:http://monitoring-influxdb:8086 -v=4
I0419 08:31:54.708556       1 heapster.go:72] Heapster version v1.3.0-beta.1
I0419 08:31:54.708857       1 configs.go:61] Using Kubernetes client with master "https://kubernetes.default" and version v1
I0419 08:31:54.708874       1 configs.go:62] Using kubelet port 10255
I0419 08:31:54.766858       1 reflector.go:200] Starting reflector *api.Node (1h0m0s) from k8s.io/heapster/metrics/sources/kubelet/kubelet.go:342
I0419 08:31:54.766965       1 reflector.go:249] Listing and watching *api.Node from k8s.io/heapster/metrics/sources/kubelet/kubelet.go:342
I0419 08:31:54.768966       1 influxdb.go:252] created influxdb sink with options: host:monitoring-influxdb:8086 user:root db:k8s
I0419 08:31:54.768985       1 heapster.go:193] Starting with InfluxDB Sink
I0419 08:31:54.769002       1 heapster.go:193] Starting with Metric Sink
I0419 08:31:54.769817       1 reflector.go:200] Starting reflector *api.Pod (1h0m0s) from k8s.io/heapster/metrics/heapster.go:319
I0419 08:31:54.769842       1 reflector.go:200] Starting reflector *api.Node (1h0m0s) from k8s.io/heapster/metrics/heapster.go:327
I0419 08:31:54.770575       1 reflector.go:200] Starting reflector *api.Namespace (1h0m0s) from k8s.io/heapster/metrics/processors/namespace_based_enricher.go:84
I0419 08:31:54.771286       1 reflector.go:200] Starting reflector *api.Node (1h0m0s) from k8s.io/heapster/metrics/processors/node_autoscaling_enricher.go:100
I0419 08:31:54.771912       1 reflector.go:249] Listing and watching *api.Node from k8s.io/heapster/metrics/processors/node_autoscaling_enricher.go:100
I0419 08:31:54.771833       1 reflector.go:249] Listing and watching *api.Pod from k8s.io/heapster/metrics/heapster.go:319
I0419 08:31:54.771844       1 reflector.go:249] Listing and watching *api.Node from k8s.io/heapster/metrics/heapster.go:327
I0419 08:31:54.771797       1 reflector.go:249] Listing and watching *api.Namespace from k8s.io/heapster/metrics/processors/namespace_based_enricher.go:84
I0419 08:31:54.782058       1 heapster.go:105] Starting heapster on port 8082
I0419 08:32:05.000250       1 manager.go:79] Scraping metrics start: 2017-04-19 08:31:00 +0000 UTC, end: 2017-04-19 08:32:00 +0000 UTC
I0419 08:32:05.002454       1 manager.go:98] Querying source: kubelet:<IP_ADDRESS>:10255
I0419 08:32:05.015454       1 manager.go:98] Querying source: kubelet:<IP_ADDRESS>:10255
I0419 08:32:05.017432       1 manager.go:98] Querying source: kubelet:<IP_ADDRESS>:10255
I0419 08:32:05.024460       1 manager.go:98] Querying source: kubelet:<IP_ADDRESS>:10255
I0419 08:32:05.027253       1 kubelet.go:233] successfully obtained stats for 79 containers
I0419 08:32:05.093195       1 kubelet.go:233] successfully obtained stats for 90 containers
I0419 08:32:05.102534       1 kubelet.go:233] successfully obtained stats for 90 containers
I0419 08:32:05.122088       1 kubelet.go:233] successfully obtained stats for 82 containers
I0419 08:32:05.122626       1 manager.go:152] ScrapeMetrics: time: 122.274079ms size: 341
I0419 08:32:05.122639       1 manager.go:154]    scrape  bucket 0: 4
I0419 08:32:05.122674       1 manager.go:154]    scrape  bucket 1: 0
I0419 08:32:05.122693       1 manager.go:154]    scrape  bucket 2: 0
I0419 08:32:05.122710       1 manager.go:154]    scrape  bucket 3: 0
I0419 08:32:05.122726       1 manager.go:154]    scrape  bucket 4: 0
I0419 08:32:05.122742       1 manager.go:154]    scrape  bucket 5: 0
I0419 08:32:05.122759       1 manager.go:154]    scrape  bucket 6: 0
I0419 08:32:05.122776       1 manager.go:154]    scrape  bucket 7: 0
I0419 08:32:05.122792       1 manager.go:154]    scrape  bucket 8: 0
I0419 08:32:05.122808       1 manager.go:154]    scrape  bucket 9: 0
I0419 08:32:05.122818       1 manager.go:154]    scrape  bucket 10: 0
I0419 08:32:05.124888       1 manager.go:113] Pushing data to: Metric Sink
I0419 08:32:05.124888       1 manager.go:113] Pushing data to: InfluxDB Sink
I0419 08:32:05.124926       1 manager.go:116] Data push completed: Metric Sink
I0419 08:32:05.124931       1 manager.go:116] Data push completed: InfluxDB Sink
I0419 08:32:05.145711       1 influxdb.go:215] Created database "k8s" on influxDB server at "monitoring-influxdb:8086"
I0419 08:32:05.289849       1 influxdb.go:169] Exported 2607 data to influxDB in 144.115539ms
W0419 08:32:14.566085       1 listers.go:68] can not retrieve list of objects using index : object has no meta: object does not implement the Object interfaces
I0419 08:32:14.566133       1 handlers.go:216] No metrics for pod default/xstreamline-cluster-1605009199-nn379

needed to add resource limits to all containers in the pod deployment per documentation:
https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/
I do want to point our the the https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/ is not very straight forward on the request limit. I would add in bold font: "request limit needs to be set" before trying the hpa


[feature] Support MyST?

Can it support MyST?

Based on the language server's issue tracker, it appears that MyST is not supported.

https://github.com/swyddfa/esbonio/issues/312


Remove the need for inline assembly trampolines used by Wasmtime

I'm opening this as a loose tracking issue for removing the need to have inline assembly trampolines defined by Wasmtime. Ideally all trampolines necessary could be provided by Cranelift instead of a mixture of what we have today of Rust-defined, inline assembly, and Cranelift-defined trampolines.
Below is a lot of words from https://github.com/bytecodealliance/wasmtime/issues/4535#issuecomment-1197071127 when I first wrote about this:

The stack unwinding in #4431 relies on precisely knowing the stack pointer when we enter WebAssembly along with the frame pointer and last program counter when we exit WebAssembly. This is not generally available in Rust itself so we are relying on handwritten assembly trampolines for these purposes instead.
Entry into WebAssembly
Entry into WebAssembly happens via one of two routes:

A "typed" route using the wasmtime::TypedFunc API or when invoking an core instance's start function (which has a known fixed signature of no inputs and no outputs). In these cases Rust does an indirect call directly to the Cranelift-generated code for the corresponding wasm function.
An "untyped" route which is used by wasmtime::Func::call as well as wasmtime::component::{Func,TypedFunc}::call. In this situation Rust will call a Cranelift-compiled trampoline. The Cranelift trampoline will load arguments from a stack parameter and then make an indirect call to the actual Cranelift-compiled wasm function which is also supplied as an argument.

Today this all records the entry stack pointer via the host_to_wasm_trampoline defined in inline assembly. Concretely Wasmtime will "prepare" an invocation which stores the Cranelift-generated function to call (be it a raw function in case (1) or a trampoline for case (2)) into the VMContext::callee field and then invoke the host_to_wasm_trampoline inline asm symbol.
This entry isn't too relevant to the component model since we're already doing what's necessary for the stack unwinding, recording the sp on entry. Nevertheless I want to describe the situation so I want to describe some oddities here as well:

The actual trampoline used in (2) to load arguments from the stack is not actually always defined by Cranelift. Instead sometimes it's a monomorphized Rust function host_to_wasm_trampoline from the Func::wrap API. This means we unfortunately cannot rely on Cranelift to supply all these trampolines which means we can't rely on the trampolines to do things that Rust itself can't do.
The entry trampoline currently requires the ability to tail-call to the actual callee. This is a technical limitation due to using the exact same trampoline for every single entry point, regardless of signature.

Ideally we would always enter WebAssembly via a Cranelift-compiled trampoline. That would mean we could do anything in the trampoline that Cranelift would do and ideally remove the need to have inline asm for this. We might still need multiple trampolines for untyped entry points and typed entry points, but overall we should ideally be able to do better here.
Exiting WebAssembly
Exiting back to the host happens in a few locations, and this is the focus of this issue where it's missing support in the component model:

Exiting from core wasm will either end up in something defined by Func::wrap or Func::new (roughly). Both of these use a VMHostFunctionContext which internally has two function pointers. One is the VMCallerCheckedAnyfunc which wasm actually calls and the other is the actual host function pointer defined in Rust being invoked. The function pointer contained within the VMCallerCheckedAnyfunc is a trampoline written in inline assembly which spills the fp/pc combo into VMRuntimeLimits. The function pointer to invoke contained within the VMHostFunctionContext has the "system-v ABI" since it receives arguments in native platform registers. For Func::wrap this is a Rust function and for Func::new this is a Cranelift-generated trampoline which spills arguments to the stack and then calls a static address specified at compile time (using Func::new requires Cranelift at runtime).
Exiting from a component will always exits via a lowered host function. Concretely what happens is that a VMComponentContext has an array lowering_anyfuncs: [VMCallerCheckedAnyfunc; component.num_lowerings]. This array is what core wasm actually calls and is exclusively populated by Cranelift-compiled trampolines (via compile_lowered_trampoline). These trampolines are similar to the Cranelift-compiled trampolines for Func::new but call a host function of type signature VMLoweringCallee. This is where fp/pc are not recorded while we exit wasm. There's not clear way to use the same trick as Func::{wrap,new} which have a singular inline asm trampoline for all signatures since the callee to defer to depends on the LoweringIndex.
Finally exiting wasm can also happen via libcalls implemented in Wasmtime. Currently each libcall gets a unique inline-asm-defined trampoline that records the pc/fp combo and then does a direct tail-call to the actual libcall itself.

Proposal to fix this issue
Overall I find the current trampoline story as pretty complicated and also pretty inefficient. There's typically at least one extra indirect call for all of these transitions and additionally there's very little cache-locality. The fix I'm going to propose here isn't a silver bullet though and will only solve some issues, but I think is still worth pursuing.
I think we should add few new pseudo-instructions to Cranelift:

Something to get the current frame pointer
Something to get the current stack pointer
Something to get the return address of the current function
Something to get the address of a label in a function (this may already exist, not sure)

With these tools we can start trying to eventually move all of the trampolines above to Cranelift exclusively and remove both Rust-defined and inline-asm defined trampolines:

For components, and this issue, compile_lowered_trampoline could be updated to use the cranelift instructions to record the pc/fp combo into the VMRuntimeLimits. This would remove the need for any extra trampoline when exiting a component and would solve the issue at hand.
For libcalls we could use the cranelift instructions to manually save fp/pc just before a libcall out to the runtime. This would remove all trampolines related to libcalls.
For Func::new the cranelift-generated trampoline could act similar to compile_lowered_trampoline and store the fp/pc combo to VMRuntimeLimits and avoid the need for two trampolines.
Untyped host-to-wasm trampolines could do the sp-saving internally rather than relying on the external trampoline to do so.

Those are at least the easy ones we could knock out with more Cranelift features. Otherwise there are still a number of places that we are requiring trampolines:

Exit trampolines with Func::wrap could ideally be generated by Cranelift but would still require two indirect calls. One call to get to the trampoline from the original core wasm and then a second call from the trampoline to the host function itself. The main problem here is getting a trampoline. Assuming trampolines are provided by Cranelift then they become available at runtiem when modules are loaded, which means Func::wrap needs to, at some point, dynamically look up a trampoline and find a corresponding one in a previous module's compiled image. This is not trivial.
Entry trampolines to TypedFunc are similarly somewhat nontrivial, but I think surmountable. Today a Store has a registry of untyped trampolines per-function signature, and I think it could also have a registry of typed trampolines per-function signature. This typed trampoline would then be used to enter wasm instead of today's calling the raw wasm function. In this situation the callee would be passed as an argument to the trampoline in the same manner untyped trampolines receive the callee.

Something to get the current frame pointer
Something to get the current stack pointer
Something to get the return address of the current function
Something to get the address of a label in a function (this may already exist, not sure)

@fitzgen added the first three already in #4573; I'm curious about the last one (address of a label though) as the semantics of it and the implications to the compiler pipeline are a bit unclear to me. Is it like a second function entry, where we assume no register state is valid? Or is it assumed to be something like a longjmp target where we'll have some state valid from some other point in the function, so it's more like a special control-flow edge?
In other words, I can see a primitive defined one of several ways:

Define another block as a second entry-point to the function, and allow getting its address. This breaks all sorts of invariants and assumptions throughout the compiler (no domtree root! func args don't dominate all uses!) and I would strongly push back against it, unless there's a very clear need, then we would need to audit a bunch of code.
Define a "gap in the control flow" primitive of some sort: the user can say "I will eventually transfer control to block B by [some mechanism], and register state will be as-if control came directly from block A"; then it's allowed to get the address of block B and follow that contract. This is more like exceptional edges off of an invoke, in LLVM terms. I would want to model it as a control-flow edge somehow as well.

I'm not sure I fully grok the details of what a trampoline would need in this primitive but can you say more about which of the above fits better?

Ah yeah sure I should expand more on that. The idea for getting the address of a label comes from the desire to remove our libcall trampolines right now. Each of the static set of libcalls has its own custom global_asm! trampoline which saves the fp/pc and then tail-calls to the actual libcall itself. Instead we would ideally save the fp/pc within the wasm function itself just before we enter the libcall, putting the work of saving fp/pc in the caller instead of the callee.
Assuming we do this then getting the current frame pointer is easy enough but for the 'last wasm pc' we actually need the address of the instruction after the call instruction itself. Having a label of sorts was my rough idea to do this because at least instruction-wise I want something like lea %dst, $const(%rip) or something like that to be the lowering. I don't think that this maps well to Cranelift abstractions currently though AFAIK (e.g. we don't really want a control-flow edge or to introduce more basic blocks, just "get the address of the instruction after some future call instruction")

Ah, I see! So basically what we need is a "what will the return address be for this call instruction" primitive, is that right?
My first instinct would be to have an instruction that refers to the call instruction, but the problem with that is that it's a forward reference. But we could do the opposite and have the call refer to the "get return address" operator that came earlier. This would work fine with MachBuffer and forward emission order; we create the label first, then bind it just after the call. The CLIF would look something like:
    v1 := get_call_return_address
    ...
    v9 := call_and_provide_return_address fn0(v2, v3, v4, ...), v1

and I can see how to feed it through the pipeline without any problems I think. Does that make sense / fill the need?

Yeah that looks perfect!

So I spun on this for a few hours and stopped here at around ~500 LoC across 25 files... adding a notion of callsite labels  turns out to be fairly cross-cutting and complex, though it is doable. With another ~4 hours or so I could push it through. I am a little apprehensive about the complexity; this is definitely not worth it for a one-off "avoid a single trampoline" tradeoff IMHO; but if it gets us efficiency improvements and you think it's important enough, I can definitely pick it back up later.

Instead we would ideally save the fp/pc within the wasm function itself just before we enter the libcall, putting the work of saving fp/pc in the caller instead of the callee.

Just a thought: does it have to be the exact pc of the call/return site?  Wouldn't a pc anywhere in the calling function be sufficient to provide the correct function name in backtraces?  (For DWARF CFI unwinding we of course need the exact PC, but we're not doing that anymore ...)

I don't think performance is critical here (at least not yet) so this isn't urgent to implement, but I would personally still like to cut down our reliance on inline assembly, especially for entry/exit trampolines that requires a "unityped" trampoline for all function signatures. Requiring these trampolines precludes other possible future features like fancier exception handling things, pinned registers, etc.

does it have to be the exact pc of the call/return site?

While it doesn't have to be 100% precise per-se it also can't just be anywhere in the function. Libcalls can trigger GC operations which need a precise stack map for where we're at in the function, which is the requirement I know of.

I have a somewhat related question - now that PR #3606 has been merged, on AArch64 we have to be careful whenever return addresses are moved from registers to memory, which is what the current inline assembly trampolines do, and what Cranelift-compiled trampolines would continue doing in the future. However, as far as I can tell the values saved by the trampolines do not influence control flow in the sense that they are only used to produce backtraces. Is that correct? If yes, then there is no need to sign them before storing to memory.

They don't influence control for now, but when we get around to implementing the Wasm exceptions proposal, then they will.

https://github.com/bytecodealliance/wasmtime/pull/6262 removes most of the hand-written asm trampolines. All that are left after that PR are the wasm-to-libcall trampolines.

Final ones done in https://github.com/bytecodealliance/wasmtime/pull/8152 now, so closing.


Question for CutMix in loading data

Hello Young,
Thank you for your exceptional work. I've rerun the project, and the performance is indeed impressive.
However, I have some questions about the cutmix part in the file unimatch.py in line 142-145. It seems that the cutmix isn't working. I try to compare the array of img_u_s1 before the line 142 and after the line 145, and I found that they are seems the same array. As I known, the same dataloader loaded the batch with the same data in a iter.
I am still learning about this paper, and I think I might misunderstand the code. Could the you provide some help or guides?
The code in the file unimatch.py in line 142-145 are listed below:
            img_u_s1[cutmix_box1.unsqueeze(1).expand(img_u_s1.shape) == 1] = \
                img_u_s1_mix[cutmix_box1.unsqueeze(1).expand(img_u_s1.shape) == 1]
            img_u_s2[cutmix_box2.unsqueeze(1).expand(img_u_s2.shape) == 1] = \
                img_u_s2_mix[cutmix_box2.unsqueeze(1).expand(img_u_s2.shape) == 1]

Hi,
The img_u_s1 and img_u_s1_mix come from the same dataloader, and the share the same image index (the same raw image). However, they are indeed different views of the same raw image, because we apply random weak augmentations (resize and crop) and random strong augmentation on them. Therefore, performing CutMix on them is also beneficial.
Of course, a more natural choice is to mix two images from two different raw images. Honestly, we previously wrongly thought img_u_s1 and img_u_s1_mix are of different image indexes. After realizing they share the same raw image, we tried to use two different image indexes for CutMix, but did not observe any improvement (comparable results). So you can use our current code for CutMix, or use a more natural and correct way of mixing two images from two different raw images.

Thanks for your reply, your explanation has been very helpful in understanding the code. However, based on my experiments, since the img_u_s1 and img_u_s1_mix come from the same dataloader object, they seem to have the same augmentation (here I subtract the array before and after CutMix, and I always get 0). I thought perhaps using a new trainloader_u_copy as a supplement might solve the problem? Would you be willing to share the code for "use two different image indexes for CutMix" with me? So that I can verify the effectiveness of CutMix. In any case, I think your work is elegant and effective, and thank you once again for your reply.

You can try img_u_s1 - img_u_s1_mix, and will find their difference.
For the code of using two different image indexes for CutMix, you can simply create an auxiliary  dataloader_u_mix:
trainsampler_u_mix = torch.utils.data.distributed.DistributedSampler(trainset_u)
trainloader_u_mix = DataLoader(trainset_u, batch_size=cfg['batch_size'],
                               pin_memory=True, num_workers=1, drop_last=True, sampler=trainsampler_u_mix)

Then, provide different shuffling seeds for dataloader_u_mix and the original dataloader_u (L119 of unimatch.py):
trainloader_u.sampler.set_epoch(epoch)
trainloader_u_mix.sampler.set_epoch(epoch+cfg['epochs'])

loader = zip(trainloader_l, trainloader_u, trainloader_u_mix)

Others are kept the same.

Thank you for your reply. I tried the dataloader_u_mix and I got below 1% improve on some metrics. The changes works, but they seem not important to the whole work. Thank you on your explaining on the CutMix.


YakYak Helper high CPU usage

The YakYak Helper process is using around 10% CPU when the window is active and someone on your Conversations list has an animated gif. It's enough to be a battery draining issue.
YakYak 1.4.2 on macOS 10.12.3

I remember you could turn animated gif off.
View > Conversation List > Show Aniamted..
Does that work ?


jd-cli: init at 1.2.1

Description of changes
Adds jd-cli, a CLI around the jd-core Java decompiler which also powers the jd-gui package.

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
22.11 Release Notes (or backporting 22.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

I put this in pkgs/tools/security because that's where jd-gui lives. Other decompilers like jadx live there, but others like cfr are in pkgs/development/tools. The latter makes more sense to me, but I'd appreciate input from a more experienced maintainer.

Validated build an binary working on amd64.
/jd-cli --version
jd-cli version 1.2.1 - Copyright (C) 2015-2021 Josef Cacek

Looks quite good, thank you! Why do you have most of the definitions outside of stdenv.mkDerivation? Looks a bit wired

Why do you have most of the definitions outside of stdenv.mkDerivation?

It's a pattern I've seen elsewhere in nixpkgs for using a fixed-output derivation for the dependencies from Maven. The src, pname and version are required for both deps and the main derivation.
Would it be better to move deps into the main derivation? I suppose that would make overlays easier to write.

Why do you have most of the definitions outside of stdenv.mkDerivation?

It's a pattern I've seen elsewhere in nixpkgs for using a fixed-output derivation for the dependencies from Maven. The src, pname and version are required for both deps and the main derivation.
Would it be better to move deps into the main derivation? I suppose that would make overlays easier to write.

If this is a Meven specific pattern then I guess it's fine :)

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/prs-ready-for-review/3032/1269


[SPARK-12816][SQL] De-alias type when generating schemas

Call dealias on local types to fix schema generation for abstract type members, such as
type KeyValue = (Int, String)

Add simple test

Can one of the admins verify this patch?

cc @marmbrus

OK to test

Test build #2380 has started for PR 10749 at commit d8e8ce3.

Test build #2380 has finished for PR 10749 at commit d8e8ce3.

This patch fails to build.
This patch merges cleanly.
This patch adds no public classes.

Oops, dealias does not exist in Scala 2.10. I'll see if I can find an alternative

Replaced it with normalize. This method is deprecated in Scala 2.11 but is also available in 2.10. Please retest.

test this please

Test build #49410 has started for PR 10749 at commit c170e3e.

Test build #49410 has finished for PR 10749 at commit c170e3e.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/49410/
Test PASSed.

Merged build finished. Test PASSed.

Thanks, merging to master.


Nkorde/cxx documentation

Pull Request Test Coverage Report for Build 2175

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 47.679%

Totals

Change from base Build 2170:
0.0%

Covered Lines:
2023

Relevant Lines:
4243

💛  - Coveralls


help transformer adding gibberish at the end of the line during decoding

Hi there,
I have trained a transformer that is giving very good and precise results in most of the sentences for my problem (Sanskrit word segmentation). However in about 10% of the sentences it strangely adds a lot of gibberish at the end of the phrase, effectively doubling the length of the phrase. This makes these sentences totally useless. It seems to happen a little bit more frequently with longer sentences; the gibberish usually consists of fragments of the words that occur in the input sentence, but they are presented out of order. It is disturbing since apart from the gibberish the quality of the output is very very high so it seems to be learning well. I observed that there is more gibberish during the beginning of the training, and at some point after training for a couple of days the gibberish again increases.
I am training with transformer_base_single_gpu-parameters, vocab_size is 5k.

Could you please specify the exact version of T2T? An example output would also be great.
I had problems in the decoding process using 1.5.7 (even end-of-sentence marker were missing). 1.5.6 worked and also the latest 1.6.0.

t2t is 1.5.6, tensorflow is 1.5.0.
The undesired output looks like this (first line is how it should look, second line is how it actually looks, the bold part is undesired):
BagavataH- jQAna-darSanam pravartate SrAvastyAm pUrvakEH samyaksambudDEH-mahA-prAtihAryam vidarSitam hitAya prANinAm-iti
BagavataH- jQAna-darSanam pravartate SrAvastyAm pUrvakEH samyaksambudDEH-mahA-prAtihAryam vidarSitam hitAya prANinAm-iti BagavataH- jQAna-darSanam pravartate SrAvastyAm pUrvakEH samyaksambudDEH-mahA-prAtihAryam vidarSitam hitAya prANinAm-iti sambudDEH-mahA-prAtihAryam vidarSitam hitAya prANinAm-iti vidarSitam hitAya prANinAm-iti BagavataH- jQAna-darSanam hitAya prANinAm-iti An-iti BagavataH- jQAna-darSanam hitAya prANinAm-iti vidarSitam hitAya prANinAm-iti vidarSitam hitAya prANinAm-
It affects about 2% of all the sentences. There is a small chance that the system is still on it's way to  convergence and this happens because it hasn't trained enough. At least the overall precision is still slightly improving by about 0,5%/10h. However this improvement does not take place with regard to the gibberish-sentences as far as I can observe.
I found a simple hack to solve the problem. Since in this scenario the number of tokens in the input sequence and the output sequence are expected to be totally identical, I just run a postprocessing script that removes the unwanted tokens. Works. However I wonder whether there can be a more elegant solution to this problem.

the number of tokens in the input sequence and the output sequence are expected to be totally identical

If this is really the case (and the whole task is just about inserting spaces), then it would be better to treat the task as sequence labeling (binary: split vs. no-split) rather than sequence-to-sequence translation. Then the output would be forced to have the same length as the input. You would use just transformer_encoder followed by a softmax (and no decoder).
However, Sanskrit tokenization is complicated because of sandhi, so you cannot expect the same number of tokens on the output (no matter if the tokens are characters or subwords) because the possible tokenizations differ in the number of tokens, for example kurvannapnoti can be tokenized either as kurvan apnoti or as kurvan na apnoti. Even here you could transform the task to sequence labeling, but not binary (you would probably need a grammar, e.g. Sanskrit Heritage Reader to list all possible segmentations in a way that could be treated as multi-class sequence labeling).

Thank you for your reply!
We use a trick on our training data to tackle the problem with the tokens. The training set actually contains dashes where Sandhi has been applied, so the number of tokens remains the same on both sides (tathāgata becomming tathā-gata for example).
I am a little bit skeptical about using sequence labeling in combination with a grammar. Inria is a great resource, however their grammar only covers a fraction of the  language and such an approach will always remain limited. Especially if one wants to segment non-classical material such as Buddhist or Śaiva-Tantras where grammar and orthography can diverge widely.  The current state of the art is seq2seq with a character-based RNN, however I am sure the transformer can do better. Actually with a vocab of 5k and the hack from above it is just 1% short of the state of the art and still converging, so might beat it. I am optimistic that proper training with an even smaller vocab will get things even further. Going down to character-level might yield the best result, yet I am worried about training time since our deadline is not too far away. But I gonna give it a try anyway. A first attempt just gave the impression that it indeed will take a long time to converge.

OK. I am no expert on Sanskrit, I just though it is easy to list all possible segmentations (because the grammar describes a fixed number of rules how to join individual words into a sequence) and the difficult job is to select the correct segmentation (in a given context). Of course, if there are non-classical materials for which the classical list of possible segmentations would not include the correct segmentation, then my approach is not applicable.

The training set actually contains dashes where Sandhi has been applied,

And can you detect this reliably even for non-classical texts? And what do you do with the examples as kurvannapnoti? Do you include double dash and allow an empty word on the output? (I know this is off-topic here, sorry, I just wonder...)
Back on the T2T topic. It would be interesting to explore why the gibberish output starts increasing after a couple of days of training. I have seen gibberish output in standard translation, when the source sentence was too long (longer than max_length, i.e. the maximum sentence length used in training) and I increased the beam search alpha in order to make the translations longer (they were too short). I am not sure if this is relevant for your case though. Anyway, it may be worth to play with --decode_hparams="alpha=$ALPHA,extra_length=$EXTRA_LENGTH" (the defaults are ALPHA=0.6, EXTRA_LENGTH=100), the maximal decoded length is source_length+EXTRA_LENGTH.


How do I include a folder containing images in my project

This is definitely a simple question but I cant seem to find the right answer no matter what. I have a folder containing images for a weather API im building but I cant link to the images, they all appear as broken images.
I am very new to react (started learning this 5 days ago) and I am not sure how to include all these images and then reference them as I need them.
Any help is appreciated.
File structure is setup as such
root
-src
--components
---icon-folder
Node - 8.11.1
npm - 5.6.0

Moving the folder to the public directory worked perfectly, thanks for the quick response!


Pod VM image in additional regions

Is it possible to add the image to the eastus2 region?
 "error": {
    "code": "GalleryImageNotFound",
    "message": "\"The gallery image /CommunityGalleries/cocopodvm-d0e4f35f-5530-4b9c-8596-112487cdea85/Images/podvm_image0/Versions/2024.02.01 is not available in eastus2 region. Please contact image owner to replicate to this region, or change your requested region.\"",
    "target": "imageReference"

cc @surajssd

it is possible to replicate the image to other regions during build time by adding a field called replication_regions to the packer build script here: https://github.com/confidential-containers/cloud-api-adaptor/blob/372e4e72f7fef27b5adb5376e0361f01b901a8a7/azure/image/ubuntu/azure-ubuntu.pkr.hcl#L26C3-L33
Packer docs here: https://developer.hashicorp.com/packer/integrations/hashicorp/azure/latest/components/builder/arm#shared-image-gallery-destination
If you are willing to try the pod vm image build for Azure here is the documentation: https://github.com/confidential-containers/cloud-api-adaptor/blob/main/azure/build-image.md

Thanks @surajssd. I created PR https://github.com/confidential-containers/cloud-api-adaptor/pull/1698 to add the replication region.
Is there more documentation about dependencies and build environment for Option 1 in the image build instructions? I have not been able to successfully build it yet due to multiple errors with missing dependencies and environment variables.

Is there more documentation about dependencies and build environment for Option 1 in the image build instructions? I have not been able to successfully build it yet due to multiple errors with missing dependencies and environment variables.

@cmaf thanks for the feedback, here is the PR that updates the documentation: https://github.com/confidential-containers/cloud-api-adaptor/pull/1699.


Create de.ftl

I hope this is the right way to submit a translation.
I translated the page into german (de.ftl). Note that this is german with swiss grammar, meaning that certain words may be different when comparing to german with germany grammar.

Thanks! It may be a while until I merge this, but it looks good to me.

Thanks! It may be a while until I merge this, but it looks good to me.

Thanks! It may be a while until I merge this, but it looks good to me.

Looks like no merge was actually made lol.


Package and compress formats missing in overview

Expected:
the same visualization as with media type. Since packageFormat and compressFormat are optional, this will be visible only when present.

 Řešit bugy z GitHubu na DCAT-AP Forms


OpenTracing support

http://opentracing.io/ -- Need I say more?

Pull request welcome.

Stay tuned...

Sorry... I forgot to circle back on this.
We have official OpenTracing support here: https://docs.datadoghq.com/tracing/setup/java/#opentracing-api

Thanks @tylerbenson!


"Getting Started" should point to Introduction

The landing page is a sore point right now.

The Getting Started link that is so prominent needs to point to the "Introduction" content.

A link to the API documentation should also be shown prominently on the main page, but not in "getting started"

Getting started link is already fixed, just the api link needs to be improved

This issue can be closed now. The described changes have been made and are already live.

Thanks for the heads up, @dzhavat


⚠ Bofrost.de has degraded performance

In 3df8c0e, Bofrost.de (https://www.bofrost.de) experienced degraded performance:

HTTP code: 200
Response time: 854 ms

Resolved: Bofrost.de performance has improved in 35d582e after 13 minutes.