Frenkield 8595 pdf password

Details
Added password validation form for viewing password-protected PDFs.
Fixed Issues
$ https://github.com/Expensify/App/issues/8595
Tests
Normal PDF with Send button (regression)

Open a chat, click the plus button to the left of the message input, and select "Add attachment"
Select a normal PDF (not password-protected)
Verify that the PDF renders correctly
Click Send button
Verify that the PDF is sent

Normal PDF with enter key (regression)

Open a chat, click the plus button to the left of the message input, and select "Add attachment"
Select a normal PDF (not password-protected)
Verify that the PDF renders correctly
Press enter key on keyboard
Verify that the PDF is sent

Protected PDF with send button

Open a chat, click the plus button to the left of the message input, and select "Add attachment"
Select a password-protected PDF (there's one in a comment in this PR)
Verify that the password form appears
Enter an invalid password and press "Confirm" button
Verify that error message appears
Enter correct password and press "Confirm" button
Verify that the PDF renders correctly
Click Send button
Verify that the PDF is sent

Protected PDF with enter key (for web and desktop)

Open a chat, click the plus button to the left of the message input, and select "Add attachment"
Select a password-protected PDF (there's one in a comment in this PR)
Verify that the password form appears
Enter an invalid password and press enter key
Verify that error message appears
Enter correct password and press enter key
Verify that the PDF renders correctly
Press enter key again
Verify that the PDF is sent

Protected PDF that was already sent

Open a chat where a protected PDF attachment was sent or received
Click on the icon for the PDF attachment
Verify that the password form appears
Enter an invalid password and press enter key
Verify that error message appears
Enter correct password and press enter key
Verify that the PDF renders correctly

Protected PDF without viewing

Open a chat, click the plus button to the left of the message input, and select "Add attachment"
Select a password-protected PDF (there's one in a comment in this PR)
Verify that the password form appears
Click Send button
Verify that the PDF is sent

[ ] Verify that no errors appear in the JS console

PR Review Checklist
Contributor (PR Author) Checklist

[x] I linked the correct issue in the ### Fixed Issues section above
[x] I wrote clear testing steps that cover the changes made in this PR

[x] I added steps for local testing in the Tests section
[ ] I added steps for Staging and/or Production testing in the QA steps section
[x] I added steps to cover failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[ ] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)

[x] I included screenshots or videos for tests on all platforms
[x] I ran the tests on all platforms & verified they passed on:

[x] iOS / native
[x] Android / native
[x] iOS / Safari
[x] Android / Chrome
[x] MacOS / Chrome
[x] MacOS / Desktop

[x] I verified there are no console errors (if there’s a console error not related to the PR, report it or open an issue for it to be fixed)
[x] I followed proper code patterns (see Reviewing the code)

[x] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick)
[x] I verified that comments were added to code that is not self explanatory
[x] I verified that any new or modified comments were clear, correct English, and explained “why” the code was doing something instead of only explaining “what” the code was doing.
[x] I verified any copy / text shown in the product was added in all src/languages/* files
[ ] I verified any copy / text that was added to the app is correct English and approved by marketing by tagging the marketing team on the original GH to get the correct copy.
[x] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named “index.js”. All platform-specific files are named for the platform the code supports as outlined in the README.
[x] I verified the JSDocs style guidelines (in STYLE.md) were followed

[ ] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[x] I followed the guidelines as stated in the Review Guidelines
[ ] I tested other components that can be impacted by my changes (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar are working as expected)
[x] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[x] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[ ] If a new component is created I verified that:

[x] A similar component doesn't exist in the codebase
[x] All props are defined accurately and each prop has a /** comment above it */
[ ] Any functional components have the displayName property
[x] The file is named correctly
[x] The component has a clear name that is non-ambiguous and the purpose of the component can be inferred from the name alone
[x] The only data being stored in the state is data necessary for rendering and nothing else
[x] For Class Components, any internal methods passed to components event handlers are bound to this properly so there are no scoping issues (i.e. for onClick={this.submit} the method this.submit should be bound to this in the constructor)
[x] Any internal methods bound to this are necessary to be bound (i.e. avoid this.submit = this.submit.bind(this); if this.submit is never passed to a component event handler like onClick)
[x] All JSX used for rendering exists in the render method
[x] The component has the minimum amount of code necessary for its purpose and it is

[x] If a new CSS style is added I verified that:

[x] A similar style doesn’t already exist
[ ] The style can’t be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(themeColors.componentBG)

[x] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[ ] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.

PR Reviewer Checklist

[ ] I verified the correct issue is linked in the ### Fixed Issues section above
[ ] I verified testing steps are clear and they cover the changes made in this PR

[ ] I verified the steps for local testing are in the Tests section
[ ] I verified the steps for Staging and/or Production testing are in the QA steps section
[ ] I verified the steps cover any possible failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[ ] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)

[ ] I checked that screenshots or videos are included for tests on all platforms
[ ] I verified tests pass on all platforms & I tested again on:

[ ] iOS / native
[ ] Android / native
[ ] iOS / Safari
[ ] Android / Chrome
[ ] MacOS / Chrome
[ ] MacOS / Desktop

[ ] I verified there are no console errors (if there’s a console error not related to the PR, report it or open an issue for it to be fixed)
[ ] I verified proper code patterns were followed (see Reviewing the code)

[ ] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick).
[ ] I verified that comments were added to code that is not self explanatory
[ ] I verified that any new or modified comments were clear, correct English, and explained “why” the code was doing something instead of only explaining “what” the code was doing.
[ ] I verified any copy / text shown in the product was added in all src/languages/* files
[ ] I verified any copy / text that was added to the app is correct English and approved by marketing by tagging the marketing team on the original GH to get the correct copy.
[ ] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named “index.js”. All platform-specific files are named for the platform the code supports as outlined in the README.
[ ] I verified the JSDocs style guidelines (in STYLE.md) were followed

[ ] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[ ] I verified that this PR follows the guidelines as stated in the Review Guidelines
[ ] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[ ] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[ ] If a new component is created I verified that:

[ ] A similar component doesn't exist in the codebase
[ ] All props are defined accurately and each prop has a /** comment above it */
[ ] Any functional components have the displayName property
[ ] The file is named correctly
[ ] The component has a clear name that is non-ambiguous and the purpose of the component can be inferred from the name alone
[ ] The only data being stored in the state is data necessary for rendering and nothing else
[ ] For Class Components, any internal methods passed to components event handlers are bound to this properly so there are no scoping issues (i.e. for onClick={this.submit} the method this.submit should be bound to this in the constructor)
[ ] Any internal methods bound to this are necessary to be bound (i.e. avoid this.submit = this.submit.bind(this); if this.submit is never passed to a component event handler like onClick)
[ ] All JSX used for rendering exists in the render method
[ ] The component has the minimum amount of code necessary for its purpose and it is broken down into smaller components in order to separate concerns and functions

[ ] If a new CSS style is added I verified that:

[ ] A similar style doesn’t already exist
[ ] The style can’t be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(themeColors.componentBG)

[ ] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[ ] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.

QA Steps

[ ] Verify that no errors appear in the JS console

Screenshots

Web

Mobile Web

Desktop

iOS

Android

Here's a password-protected PDF for testing. The password is 1234
password-protected.pdf

I have to step out for a couple hours. I'll work on the changes later today. Thanks, All!

I will be going OOO so I will be reassigning this for now.

I pushed most of the changes. I'll push the BasePDFView.js change later today.

Hi @Santhosh-Sellavel. I think I have a way of abstracting the code. I'll give it a try and if it works ok I'll push it to a different branch so you can take a look. Then, if you think it's ok, I'll test it and merge it into this pull request.

Sorry, never mind. I'm not able to abstract this using a base component without creating a convoluted mess. Please let me know how you'd like to proceed.

Sorry, never mind. I'm not able to abstract this using a base component without creating a convoluted mess. Please let me know how you'd like to proceed.

I will need to check this update you on the next steps.

@frenkield Can you update screen record videos instead of images for all platforms thanks!

@frenkield Can you update screen record videos instead of images for all platforms thanks!

Sure, no problem. But I'll wait to do that until after decisions are made regarding the refactoring and the press-on-enter handling.

Hi @Santhosh-Sellavel. I reverted all changes to Button.js. The press-on-enter handling is now entirely inside AttachmentModal.js. Please note that the changes to AttachmentModal.js are necessary in order to prevent the message from being sent prematurely when the enter key is pressed.
So I believe there are only 2 blockers remaining:

PDFView refactoring
Videos

@shawnborton Can you design-wise everything looks good!
One more question.
Do we need autofocus to the password field?
I believe that an optional field, shouldn't be focused by default.
cc: @tgolen

@frenkield
Pressing enter does send any attachments can you check?
Without these PR changes, it seems to be working fine.
https://user-images.githubusercontent.com/85645967/171270368-08c3f002-cb20-4f8a-9610-6df46b0256fb.mov

@frenkield Pressing enter does send any attachments can you check? Without these PR changes, it seems to be working fine.

So sorry, @Santhosh-Sellavel, I don't quite understand your question. But yes, pressing the enter does send the PDF attachment.
Please note that the changes in AttachmentModal.js are necessary to prevent prematurely sending the attachement. Without these changes, this is the bad thing that happens:
https://user-images.githubusercontent.com/11221215/171299057-659c43ed-2e23-4112-b66a-4b9c95832e4d.mp4

Updated the comment

Yeah that case handle it please, press enter to send doesn't work breaks other attachments too.

Yeah that case handle it please, press enter to send doesn't work breaks other attachments too.

Ok great. So then I think the code is done.
Besides adding some videos for the testers, is there anything else that needs to be done?

Yeah that case handle it please, press enter to send doesn't work breaks other attachments too.

Ok great. So then I think the code is done.
Besides adding some videos for the testers, is there anything else that needs to be done?

I too had a look at pdf view, I believe there's no further refactoring needed for  the same.
If the mentioned issues are fixed ,and  there are no further breakages, updated the videos.
We are mostly done here, unless if @tgolen or any other CME feels otherwise.
Thanks.

Ok super. I'll add the videos tomorrow.

It looks like we have extra inner padding on the Android mobile screenshot around the password form. Take a look at iOS... it should look like that.
For the highlighted 3 area on this screenshot:

Do we know where that thumbnail is coming from? Can we perhaps provide a custom thumbnail for password protected PDFs?

Thanks, @shawnborton. I'll fix the Android padding issue today.

I fixed the Android padding issue and updated the Android screenshot above.
So these are currently the remaining tasks:

Expensify: decide whether or not to remove autofocus from password input
frenkield: if desired, remove autofocus from password input
frenkield: test on all platforms and create videos

I tend to think we'd want to remove the autofocus, mostly because on mobile, the keyboard layout with the two buttons feels a bit strange.
Is it possible to remove auto-layout but have the keyboard launch on top of the bottom green send button when the password field is focused?

Hi All. Some comments/questions:

I'm still waiting on a decision regarding autofocus on the password field. Once a decision is made I will update the autofocus property (if necessary), retest everything, and create videos for the testers.
I assume it's possible to selectively hide/cover the send button when the keyboard is displayed. However, this will require several hours of additional work. If you want to add this functionality then we'll need to update the proposal and contract accordingly.
I don't know where the preview image comes from. What does the preview image have to do with this pull request?

I think we can go with my recommendation above re: the auto focus
As for the preview image, I am mostly just curious where it comes from and if you had any context. It's related to password protected PDFs.

Thanks, @shawnborton. I'll disable autofocus on the password field.
And so sorry, I don't have any context on the preview image. I haven't looked at that part of the code.

Question: How much time will it take to Load the PDF after password is entered? Does it depend on the file size? Where is this applicable, Native, mWeb, web or all?
What if Pdf is big and takes a couple of seconds to load, it will look bad to wait for that time after user has entered the password and press confim. There is no feedback at all. User will think that nothing happened after pressing confirm and he will press that button again and again. (this is only applicable when above question's answer is yes.)
If Yes, we should think about it and handle it. What do you suggest to solve this problem?

Question: How much time will it take to Load the PDF after password is entered? Does it depend on the file size? Where is this applicable, Native, mWeb, web or all?

Hi @parasharrajat. I'll verify all this but I think it's ok on web/mWeb. The PDF is only loaded once - before the password challenge. On native, however, I think I'll need to display the loading indicator.
I'm investigating this now.

Hi @parasharrajat. Adding pressOnEnter to the confirm button solved most of the enter key issues. So maybe I can revert the changes in AttachmentModal.
However, there's still 1 issue. After successfully viewing a password-protected PDF, the enter key does not close the attachment modal. It currently behaves as follows:
For regular PDF:

Click on regular PDF in list of sent messages
Attachment modal opens and PDF is displayed
Click enter key
Attachment modal closes

For password-protected PDF:

Click on password-protected PDF in list of sent messages
Attachment modal appears and password form is displayed
Enter correct password and press enter
PDF is displayed
Click enter key
Attachment modal does not close

To handle this I think I'll need to explicitly push the focus back to the Modal so that it receives the keyboard event. Here's the code that closes the modal when the enter key is pressed:
https://github.com/Expensify/App/blob/b951e10a8f7dce8ce1f4e915da26cc5f348bce47/src/components/Modal/BaseModal.js#L80-L85
Do you know by any chance if this type of problem has been addressed elsewhere in the code? Or do you have any suggestions about how to easily address this?

However, there's still 1 issue. After successfully viewing a password-protected PDF, the enter key does not close the attachment modal. It currently behaves as follows:

Strange, I don't think it is needed. Enter key behavior may be applied from the lib automatically.
@shawnborton What are your thoughts on Enter key behavior mentioned here ?

Hmm I am a bit confused why the enter key would close the attachment modal in the first place? Is that expected?

No it is not expected as far as I know.

The code is in BaseModal so it seems like close-on-enter is the default behavior for all modals.

Got it. I don't think I would worry too much about the case you outlined above. If the "x" button is in focus and pressing enter does not close the modal, then I think that is a problem. But if simply pressing enter doesn't close it with no clear indication of which piece of UI is in focus, then I think it's fine.

Agree we don't have to worry about this too much.

Ok great. Thanks, @parasharrajat and @shawnborton.

Hi @parasharrajat. I believe everything is done and working correctly. I also added videos for all the platforms.

@shawnborton Could you please review the designs?

For Android, what's happening when the keyboard is in view? It seems like the buttons are squished:

That's not happening on iOS. Can we use the same behavior that iOS is using?

That's not happening on iOS. Can we use the same behavior that iOS is using?

It's mobile web. So it's exactly the same code on iOS and Android. Maybe I can solve this by adding some margin or padding below the confirm button. I'll give it a try.

I think the send button should be above the keyboard on Chrome Mweb.

I think the send button should be above the keyboard on Chrome Mweb.
Note: You can access the storybook server to see usages of a few common components. https://staging.new.expensify.com/docs/index.html

@parasharrajat, storybook appears to be broken. I'll look again later today or tomorrow.
Also, I'd really appreciate it if you could provide the full URL to the compoment you're talking about. It's not at all clear to me where I'm supposed to look. For example, is it this one?
https://staging.new.expensify.com/docs/index.html?path=/story/components-form--default

Oh, that note was not related. I just thought to share it with you. To see the fix button behavior check any RHN(right-hand navigation page). For example, Profile page, etc.

Discussing on the issue.

Uh oh. I merged main and pushed. And now a check is failing:

Hi @parasharrajat. I added the new functionality and updated the videos.

I requested a design review from the design team and I was waiting on that for a few days. Today, I will test it myself and try to help the design team review it easily.
Mostly all of the requested changes are done.

Taking notes on all the things that I noticed and I will share them here asap.

Apologies, I totally missed this! Will make sure to review today.

Okay, design-wise this looks good to me based on the screen recordings in the comment. Nice work!

Ok. Thanks, @shawnborton.

I guess they are following these mockups. IMO, we should enable the eye for all devices.

Yeah, apologies if that got lost at some point but I agree, it should look the same everywhere.

Yeah, apologies if that got lost at some point but I agree, it should look the same everywhere.

So the form should be vertically centered on mobile devices, @shawnborton?

How's this, @shawnborton?
https://user-images.githubusercontent.com/11221215/180365675-29a6308d-7746-4a23-a82f-9ad285d34012.mp4

I think that looks great! Thanks for the quick update.

Thanks, @shawnborton!

Hi @parasharrajat. I updated everything. Can you please test when you have some time?
A couple comments:

Once everything has been tested/verified I'll update the videos.
I ultimately decided to leave the keyboard detection code in PDFView. However, after refactoring, the onAvoidKeyboard property is no longer deeply nested. All the keyboard detection code is now contained in index.android.js and index.js.

Thanks!

Running tests...

I already spent many hours on this code to get everything to render properly on all platforms. The code works correctly and, as far as I can tell, your proposed changes do not work.

If you can't provide a working solution then we'll need to increase the price of the project and add a milestone for this task.
Basically, there are bugs in the code that we are trying to solve here. Also, its common practice to adjust the code during reviews. Also, I think you have already requested a bonus and we agreed on that.
No solution is accepted until it is approved. This is same for all even for us.
A few reasons why we are making these changes as you may be new as a contributor to projects.

We don't use position-based AvoidingView in our app unless there is a reason to do so and padding mode does not work.

There was one bug where the form was only scrollable in the center of the page instead of the full page which has a side-effect on iOS. When user scrolls the page, it looks cropped from vertical sides even though there is enough space above. Instead of throwing the bug at you, I tried to give a solution. I am a reviewer and not an author, you are supposed to implement the full solutions from my suggestions.

Static Vertical offset does not work well with all the devices.

There was unnecessary code.

Author code the PR in any way they like but those are adjusted during review to fit into the project.

There is one more bug on Android. the Send button does not hide and show smoothly. I understand that there is a delay in Keboard events which make it behave that way but still that a bad UX and should be solved in one way or another. But we don't have to fix that in this PR just leave the button visible at all times and rest can be handled in a new issue.

If you are not able to work anymore on this PR, let me know.

Hi @parasharrajat. Sorry for the delay. And thanks again for the code. It works well.
I'll push some updates tomorrow.

PR updated, @parasharrajat.

Please note that I added an extra View component around the "View PDF" label:
https://github.com/Expensify/App/blob/3a04ab7ca68b7dd5f06d91a044f9f9e7a14fdf97/src/components/PDFView/PDFPasswordForm.js#L101-L105
This resolves the iOS issue depicted below where the label animates strangely when the keyboard opens/closes. The problem is also mentioned here: https://github.com/facebook/react-native/issues/11821
https://user-images.githubusercontent.com/11221215/182943484-1103f539-48ba-4927-a2ec-18ddabecf2c1.mov

PR updated, @parasharrajat.
I removed the defunct KEYBOARD_VERTICAL_OFFSET constant.

Personally I think it's a bit strange to have both the Confirm and Send button like that in the same view, so I would prefer to keep the Send button out of view. Are we sure we can't do anything about the screen jumping?

Sure, definitely open to seeing that in action.

Here is what send button fade in/out looks like, @shawnborton and @parasharrajat. Please note that the send button still takes space.
https://user-images.githubusercontent.com/11221215/183314509-6372b742-4c45-4ae7-843d-528e66d3d8b6.mov

Without fade, @parasharrajat and @shawnborton:
https://user-images.githubusercontent.com/11221215/183341879-6f5aaaf2-774f-4614-a473-ae605a9009b6.mov

Hmm the instant doesn't quite feel instant, because you still get a glimpse of the button moving upwards first, and then it disappears.

Can't we just fade out the button before the keyboard comes into view?

Yeah, my thinking was we time it with the :focus event?

But we can do this:
1. On focus on the input fade the button.
2. Show the button after the Keyboard hides.

That doesn't change anything. It's impossible to update the send button state before the keyboard opens on Android.
Here's an option:

Hide the send button when the form appears.
Add a "cancel" button or link to the form that sends the user back to the protected PDF info message. The send button thus reappears.
Apply this change to all platforms since the solution must also work on Android-web.

Given that this is such a low traffic flow, I think I could live with the fade for now rather than adding more complexity to the form itself. I hate to settle like this, but it seems like the easiest path forward.

Who makes the final decision on this, @parasharrajat and @shawnborton?
Is it just you two? Or are there others involved?

The best way to get a final decision on anything related to the New Expensify app is to bring it to the #expensify-open-source room for discussion! That being said, I feel pretty confident that we can just go with the fade out solution for now, especially given that this flow is not commonly used right now.

Ok super.
So I should push the fade solution?

Ok thanks. One question, @parasharrajat:
In all the other components that use animations (via Animated) the duration value for the animation is not in CONST.js. See ReportActionsList.js for example:
Animated.timing(this.state.fadeInAnimation, {
    toValue: 1,
    duration: 100,
    useNativeDriver: true,
}).start();

Should I do the same? Or should I put the duration value in CONST.js?

PR updated, @parasharrajat.

Given the Android performance problems in main (https://github.com/Expensify/App/issues/10345), are you sure you want me to merge @parasharrajat?

Sorry, I don't understand.
So when you test you always run the merged code? You don't test on my working branch?

We should do this change on all platforms to keep the UI consistent. I understand that button is not visible on the iOS...

The send button will thus visibly fade in on iOS after the keyboard closes. I'm very concerned that someone is going to look at this behavior and disapprove.

Yup, let's align the behavior on all mobile platforms please.

Also, I don't think this will be final as we still don't consider this as the best solution.

So who makes the final decision, @parasharrajat and @shawnborton?
And when?
And if whomever decides that this solution is unacceptable, what happens then?

I think we can consider these decisions final for the purposes of this specific project. At a later date, if we feel like this flow is jarring and is a mainflow use case worth solving for, we can revisit with a follow up. But for now, let's proceed as we have been with the addition of aligning the mobile platforms.

You have been a green signal to add the fade effect. That would be last of this PR. There will not be any changes regarding that behaviour in this PR unless it is not working as advertised.

Ok super. Thank you, @parasharrajat and @shawnborton.

PR updated, @parasharrajat.

PR updated, @parasharrajat.
On ios-safari the done/return/go button at the bottom right corner of the keyboard doesn't close the keyboard. It's the same way everywhere else in the application. So there's no bug there.

@parasharrajat, what should I do for this checklist item?

I verified any copy / text that was added to the app is correct English and approved by marketing by tagging the marketing team on the original GH to get the correct copy.

PR updated, @parasharrajat.

PR updated, @parasharrajat and @tgolen.

OK, there it goes! 🎉
Nice work getting through this one @frenkield

It's OK, Melvin. The test is failing because this is an older PR where the comment formats don't match what the test is expecting regarding the reviewer checklist.

🚀 Deployed to staging by @tgolen in version: 1.1.97-2 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
failure ❌

🕸 web 🕸
success ✅

Thanks, @tgolen and @parasharrajat.


CLOUDP-169734: AKO install command

Proposed changes
This PR introduced a new command enabling users to install Atlas Kubernetes Operator using the atlasCLI tool.
Jira ticket: CLOUDP-169734
Checklist

[x] I have signed the MongoDB CLA
[x] I have added tests that prove my fix is effective or that my feature works
[x] I have added any necessary documentation in document requirements section listed in CONTRIBUTING.md (if appropriate)
[ ] I have addressed the @mongodb/docs-cloud-team comments (if appropriate)
[x] I have updated test/README.md (if an e2e test has been added)
[x] I have run make fmt and formatted my code

Looks good! Thanks @helderjs. Please also add doc files to the PR.

@igor-karpukhin, the doc files are there, but still the workflow fails, seems Github still has bugs

@helderjs that means there's a diff between what you committed and what the task is seeing after doing make gen-docs if you run the doc generation is there a diff that needs to be committed on your side?

if you run the doc generation is there a diff that needs to be committed on your side?
@gssbzn no, running the command doesn't generate any change in the existing file


Array function wrappers

Closes #145
Could probably still use a little cleanup, but I think it's getting there. Some places could be improved using string interpolation instead of calling the render*() functions, but the problem is they have some handling of casing, prefix stripping, etc. that would need to be duplicated or refactored in order to get the formatted names as a string. Shouldn't be too bad but not sure whether it's worth cluttering up the existing code for this, let me know if you want me to try it that way.
TODO:

~fill in full list of functions~
~add proxy functions~
~update examples~

Output (mostly unchanged from the issue):
pub const EnumerateInstanceExtensionPropertiesAllocError =
    EnumerateInstanceExtensionPropertiesError || Allocator.Error;
pub fn enumerateInstanceExtensionPropertiesAlloc(
    self: Self,
    p_layer_name: ?[*:0]const u8,
    allocator: Allocator,
) EnumerateInstanceExtensionPropertiesAllocError![]ExtensionProperties {
    var count: u32 = undefined;
    var data: []ExtensionProperties = &.{};
    errdefer allocator.free(data);
    var result = Result.incomplete;
    while (result == .incomplete) {
        _ = try self.enumerateInstanceExtensionProperties(p_layer_name, &count, null);
        data = try allocator.realloc(data, count);
        result = try self.enumerateInstanceExtensionProperties(p_layer_name, &count, data.ptr);
    }
    return if (count == data.len) data else allocator.realloc(data, count);
}

pub fn getPhysicalDeviceQueueFamilyPropertiesAlloc(
    self: Self,
    physical_device: PhysicalDevice,
    allocator: Allocator,
) Allocator.Error![]QueueFamilyProperties {
    var count: u32 = undefined;
    self.getPhysicalDeviceQueueFamilyProperties(physical_device, &count, null);
    const data = try allocator.alloc(QueueFamilyProperties, count);
    errdefer allocator.free(data);
    self.getPhysicalDeviceQueueFamilyProperties(physical_device, &count, data.ptr);
    return if (count == data.len) data else allocator.realloc(data, count);
}

Looks pretty good!

Some places could be improved using string interpolation instead of calling the render*() functions, but the problem is they have some handling of casing, prefix stripping, etc. that would need to be duplicated or refactored in order to get the formatted names as a string.

I've been thinking, a nice refactor would be to write a custom formatter that handles some of the "casing" and namespace stripping operations.

Shouldn't be too bad but not sure whether it's worth cluttering up the existing code for this, let me know if you want me to try it that way.

I think that should be a separate effort, code looks good imo :)

Please also add the Alloc versions to the Proxy wrappers

The "Alloc" in the name should probably come before the vendor suffix, right? E.g. getPhysicalDeviceSurfaceFormatsAllocKHR() and not ...KHRAlloc()? Need to fix that if so.

Turns out there are a lot of these functions - I added all of the core ones I could find and some common extensions, there are a lot more in extensions but most are pretty niche. Hopefully you're still alright with the decision to hardcode the list.
Anyway I went ahead and made the naming change mentioned above. I can revert if you don't like it but then the capitalization will still need a fix as it was formatting as ...KhrAlloc().
Those two things aside, should be just about good to go. There's probably some unnecessary duplicated code which I might have a go at later, along with some misc cleanup, but you can go ahead and merge if you're happy with it as is.

The "Alloc" in the name should probably come before the vendor suffix, right? E.g. getPhysicalDeviceSurfaceFormatsAllocKHR() and not ...KHRAlloc()? Need to fix that if so.

I don't really mind it either way.

Turns out there are a lot of these functions - I added all of the core ones I could find and some common extensions, there are a lot more in extensions but most are pretty niche. Hopefully you're still alright with the decision to hardcode the list.

Ah, I didn't realize there are so many of these either. I guess the main problem is detecting which functions can be wrapped using such an allocator. Maybe it can just be done for all of the functions that can return VK_INCOMPLETE?
Anyway, the changes look fine. I'm fine with merging as is, though autodetecting would be nice. Its up to you if you want to do it.

Maybe it can just be done for all of the functions that can return VK_INCOMPLETE?

There's also vkGetPhysicalDeviceQueueFamilyProperties() (and possibly others) which returns void. Looks like Vulkan-Hpp checks whether one param is the array length of another:
https://github.com/KhronosGroup/Vulkan-Hpp/blob/620cf05ac20e43a729910e343e2ce8229810cc1a/VulkanHppGenerator.cpp#L3187
but I'm not sure whether that one check is enough - there's a lot going on in that file and they have a few different cases for these sorts of functions, and there are some other checks elsewhere that might be necessary to filter out false positives.
So seems like getting autodetection right will be pretty involved, and so if you're okay with the hardcoded list then IMO it's worth sticking with that, at least in the short term. Later I can dig into this some more and hopefully post a followup PR at some point.

That seems like a good plan to me!


"App\Models\User is not notifiable" error

Hi,
thanks a lot for your amazing work.
I would like to understand the given error. According to the documentation in the README file, I should have a Model like this:
class User extends Model {
    use SnoozeNotifiable;

    // ...
}

that sadly fires the error in the issue subject.
If look closer, the error is thrown in lines 46-48 of ScheduledNotification.php:
if (! method_exists($notifiable, 'notify')) {
        throw new SchedulingFailedException(sprintf('%s is not notifiable', get_class($notifiable)));
}

if fact, looking at Traits/SnoozeNotifiable.php I see that only the notifyAt method is declared, but not notify.
The error disappear if I declare the Model as
    use Notifiable, SnoozeNotifiable;

    // ...
}

Anyway I would like to know if there's a mistake somewhere in my code, or "simply" something to add in the README.
version "1.0.5"
Laravel "v6.20.27"

Good catch, docs updated :)


Correction for raw affiliation Univ Rennes, CNRS, CREM - UMR 6211, F-35000 Rennes, France; University of Economics, The University of Danang, Viet Nam

Correction needed for raw affiliation Univ Rennes, CNRS, CREM - UMR 6211, F-35000 Rennes, France; University of Economics, The University of Danang, Viet Nam
raw_affiliation_name: Univ Rennes, CNRS, CREM - UMR 6211, F-35000 Rennes, France; University of Economics, The University of Danang, Viet Nam
new_rors: 03ecpp171;015m7wh34;02feahw73;0047rmf80
previous_rors: 03ecpp171;015m7wh34;02feahw73
works_examples: W4317598146
contact: f68c4e5f7d1776c3e01e023e96d10fab:8412f0467ba735675c78854214b946ee @ univ-rennes.fr

This issue was accepted and ingested by the OpenAlex team on 2024-10-10. The new affiliations should be visible within the next 7 days.


[BUG] SmartThings Button sends same command 3 times

Describe the bug
I have several SmartThings Buttons M/N IM6001-BTP01 that have always worked flawlessly in ST and Home Assistant. Firmware version is 0x00000014. A while back they started repeating the same command 3 times. I checked the output of zha_event and confirmed it
Event 2 fired 7:36:
event_type: zha_event
data:
  device_ieee: 28:6d:97:00:01:04:1e:de
  unique_id: 28:6d:97:00:01:04:1e:de:1:0x0500
  device_id: f59f3b82102504ee70ab649e4300fde9
  endpoint_id: 1
  cluster_id: 1280
  command: button_single
  args:
    press_type: single
    command_id: 0
    args:
      - 1
      - 0
      - 0
      - 0
  params: {}
origin: LOCAL
time_fired: "2022-11-10T13:36:04.166296+00:00"
context:
  id: 01GHGW2EJ6R7HEWYYDDQPQW7B8
  parent_id: null
  user_id: null

Event 1 fired 7:36:
event_type: zha_event
data:
  device_ieee: 28:6d:97:00:01:04:1e:de
  unique_id: 28:6d:97:00:01:04:1e:de:1:0x0500
  device_id: f59f3b82102504ee70ab649e4300fde9
  endpoint_id: 1
  cluster_id: 1280
  command: button_single
  args:
    press_type: single
    command_id: 0
    args:
      - 1
      - 0
      - 0
      - 0
  params: {}
origin: LOCAL
time_fired: "2022-11-10T13:36:01.806008+00:00"
context:
  id: 01GHGW2C8ECN2VY6SNNKB53263
  parent_id: null
  user_id: null

Event 0 fired 7:36:
event_type: zha_event
data:
  device_ieee: 28:6d:97:00:01:04:1e:de
  unique_id: 28:6d:97:00:01:04:1e:de:1:0x0500
  device_id: f59f3b82102504ee70ab649e4300fde9
  endpoint_id: 1
  cluster_id: 1280
  command: button_single
  args:
    press_type: single
    command_id: 0
    args:
      - 1
      - 0
      - 0
      - 0
  params: {}
origin: LOCAL
time_fired: "2022-11-10T13:36:00.020955+00:00"
context:
  id: 01GHGW2AGM97XJKBS0WT4A7ZZM
  parent_id: null
  user_id: null

The same thing happens when I double press or hold. As I mentioned above I have multiple of these buttons around the house that were working perfectly fine in HA and they all started having the issue at the same time so I am guessing it has to do with the driver (Quirk: zhaquirks.samjin.button.SamjinButton) for this device.
To Reproduce
Steps to reproduce the behavior:

Just press the button while watching the zha_event output

Expected behavior
Command should be output just once.
Screenshots
n/a

Device signature
{
  "node_descriptor": "NodeDescriptor(logical_type=<LogicalType.EndDevice: 2>, complex_descriptor_available=0, user_descriptor_available=0, reserved=0, aps_flags=0, frequency_band=<FrequencyBand.Freq2400MHz: 8>, mac_capability_flags=<MACCapabilityFlags.AllocateAddress: 128>, manufacturer_code=4673, maximum_buffer_size=82, maximum_incoming_transfer_size=82, server_mask=11264, maximum_outgoing_transfer_size=82, descriptor_capability_field=<DescriptorCapability.NONE: 0>, *allocate_address=True, *is_alternate_pan_coordinator=False, *is_coordinator=False, *is_end_device=True, *is_full_function_device=False, *is_mains_powered=False, *is_receiver_on_when_idle=False, *is_router=False, *is_security_capable=False)",
  "endpoints": {
    "1": {
      "profile_id": 260,
      "device_type": "0x0402",
      "in_clusters": [
        "0x0000",
        "0x0001",
        "0x0003",
        "0x0020",
        "0x0402",
        "0x0500",
        "0x0b05"
      ],
      "out_clusters": [
        "0x0003",
        "0x0019"
      ]
    }
  },
  "manufacturer": "Samjin",
  "model": "button",
  "class": "zhaquirks.samjin.button.SamjinButton"
}

Diagnostic information
{
  "home_assistant": {
    "installation_type": "Home Assistant OS",
    "version": "2022.11.2",
    "dev": false,
    "hassio": true,
    "virtualenv": false,
    "python_version": "3.10.7",
    "docker": true,
    "arch": "x86_64",
    "timezone": "America/Chicago",
    "os_name": "Linux",
    "os_version": "5.15.74",
    "supervisor": "2022.10.2",
    "host_os": "Home Assistant OS 9.3",
    "docker_version": "20.10.18",
    "chassis": "vm",
    "run_as_root": true
  },
  "custom_components": {
    "favicon": {
      "version": "10.4",
      "requirements": []
    },
    "spotcast": {
      "version": "v3.6.30",
      "requirements": [
        "spotify_token==1.0.0"
      ]
    },
    "garbage_collection": {
      "version": "4.8.3",
      "requirements": [
        "python-dateutil>=2.8.2"
      ]
    },
    "zha_toolkit": {
      "version": "v0.8.23",
      "requirements": [
        "packaging>=20.8"
      ]
    },
    "auto_backup": {
      "version": "1.3.0",
      "requirements": []
    },
    "victorsmartkill": {
      "version": "2022.7.1",
      "requirements": [
        "victor-smart-kill==1.1.0"
      ]
    },
    "nws_alerts": {
      "version": "2.5",
      "requirements": []
    },
    "alexa_media": {
      "version": "4.3.2",
      "requirements": [
        "alexapy==1.26.4",
        "packaging>=20.3",
        "wrapt>=1.12.1"
      ]
    },
    "navien_water_heater": {
      "version": "0.0.7",
      "requirements": []
    },
    "keymaster": {
      "version": "v0.0.82",
      "requirements": []
    },
    "nest_protect": {
      "version": "0.3.8",
      "requirements": []
    },
    "browser_mod": {
      "version": "2.1.2",
      "requirements": []
    },
    "watchman": {
      "version": "0.5.1",
      "requirements": [
        "prettytable==3.0.0"
      ]
    },
    "webrtc": {
      "version": "v2.3.1",
      "requirements": []
    },
    "nodered": {
      "version": "1.1.2",
      "requirements": []
    },
    "hacs": {
      "version": "1.28.3",
      "requirements": [
        "aiogithubapi>=22.2.4"
      ]
    }
  },
  "integration_manifest": {
    "domain": "zha",
    "name": "Zigbee Home Automation",
    "config_flow": true,
    "documentation": "https://www.home-assistant.io/integrations/zha",
    "requirements": [
      "bellows==0.34.2",
      "pyserial==3.5",
      "pyserial-asyncio==0.6",
      "zha-quirks==0.0.85",
      "zigpy-deconz==0.19.0",
      "zigpy==0.51.5",
      "zigpy-xbee==0.16.2",
      "zigpy-zigate==0.10.3",
      "zigpy-znp==0.9.1"
    ],
    "usb": [
      {
        "vid": "10C4",
        "pid": "EA60",
        "description": "*2652*",
        "known_devices": [
          "slae.sh cc2652rb stick"
        ]
      },
      {
        "vid": "1A86",
        "pid": "55D4",
        "description": "*sonoff*plus*",
        "known_devices": [
          "sonoff zigbee dongle plus v2"
        ]
      },
      {
        "vid": "10C4",
        "pid": "EA60",
        "description": "*sonoff*plus*",
        "known_devices": [
          "sonoff zigbee dongle plus"
        ]
      },
      {
        "vid": "10C4",
        "pid": "EA60",
        "description": "*tubeszb*",
        "known_devices": [
          "TubesZB Coordinator"
        ]
      },
      {
        "vid": "1A86",
        "pid": "7523",
        "description": "*tubeszb*",
        "known_devices": [
          "TubesZB Coordinator"
        ]
      },
      {
        "vid": "1A86",
        "pid": "7523",
        "description": "*zigstar*",
        "known_devices": [
          "ZigStar Coordinators"
        ]
      },
      {
        "vid": "1CF1",
        "pid": "0030",
        "description": "*conbee*",
        "known_devices": [
          "Conbee II"
        ]
      },
      {
        "vid": "10C4",
        "pid": "8A2A",
        "description": "*zigbee*",
        "known_devices": [
          "Nortek HUSBZB-1"
        ]
      },
      {
        "vid": "0403",
        "pid": "6015",
        "description": "*zigate*",
        "known_devices": [
          "ZiGate+"
        ]
      },
      {
        "vid": "10C4",
        "pid": "EA60",
        "description": "*zigate*",
        "known_devices": [
          "ZiGate"
        ]
      },
      {
        "vid": "10C4",
        "pid": "8B34",
        "description": "*bv 2010/10*",
        "known_devices": [
          "Bitron Video AV2010/10"
        ]
      }
    ],
    "codeowners": [
      "@dmulcahey",
      "@adminiuga",
      "@puddly"
    ],
    "zeroconf": [
      {
        "type": "_esphomelib._tcp.local.",
        "name": "tube*"
      },
      {
        "type": "_zigate-zigbee-gateway._tcp.local.",
        "name": "*zigate*"
      },
      {
        "type": "_zigstar_gw._tcp.local.",
        "name": "*zigstar*"
      }
    ],
    "dependencies": [
      "file_upload"
    ],
    "after_dependencies": [
      "onboarding",
      "usb",
      "zeroconf"
    ],
    "iot_class": "local_polling",
    "loggers": [
      "aiosqlite",
      "bellows",
      "crccheck",
      "pure_pcapy3",
      "zhaquirks",
      "zigpy",
      "zigpy_deconz",
      "zigpy_xbee",
      "zigpy_zigate",
      "zigpy_znp"
    ],
    "is_built_in": true
  },
  "data": {
    "ieee": "**REDACTED**",
    "nwk": 25971,
    "manufacturer": "Samjin",
    "model": "button",
    "name": "Samjin button",
    "quirk_applied": true,
    "quirk_class": "zhaquirks.samjin.button.SamjinButton",
    "manufacturer_code": 4673,
    "power_source": "Battery or Unknown",
    "lqi": 61,
    "rssi": null,
    "last_seen": "2022-11-10T07:46:53",
    "available": true,
    "device_type": "EndDevice",
    "signature": {
      "node_descriptor": "NodeDescriptor(logical_type=<LogicalType.EndDevice: 2>, complex_descriptor_available=0, user_descriptor_available=0, reserved=0, aps_flags=0, frequency_band=<FrequencyBand.Freq2400MHz: 8>, mac_capability_flags=<MACCapabilityFlags.AllocateAddress: 128>, manufacturer_code=4673, maximum_buffer_size=82, maximum_incoming_transfer_size=82, server_mask=11264, maximum_outgoing_transfer_size=82, descriptor_capability_field=<DescriptorCapability.NONE: 0>, *allocate_address=True, *is_alternate_pan_coordinator=False, *is_coordinator=False, *is_end_device=True, *is_full_function_device=False, *is_mains_powered=False, *is_receiver_on_when_idle=False, *is_router=False, *is_security_capable=False)",
      "endpoints": {
        "1": {
          "profile_id": 260,
          "device_type": "0x0402",
          "in_clusters": [
            "0x0000",
            "0x0001",
            "0x0003",
            "0x0020",
            "0x0402",
            "0x0500",
            "0x0b05"
          ],
          "out_clusters": [
            "0x0003",
            "0x0019"
          ]
        }
      }
    },
    "active_coordinator": false,
    "entities": [
      {
        "entity_id": "binary_sensor.guest_bedroom_bedside_button",
        "name": "Samjin button"
      },
      {
        "entity_id": "button.guest_bedroom_bedside_button_identify",
        "name": "Samjin button"
      },
      {
        "entity_id": "sensor.guest_bedroom_bedside_button_battery",
        "name": "Samjin button"
      },
      {
        "entity_id": "sensor.guest_bedroom_bedside_button_temperature",
        "name": "Samjin button"
      }
    ],
    "neighbors": [],
    "endpoint_names": [
      {
        "name": "IAS_ZONE"
      }
    ],
    "user_given_name": "Guest Bedroom Bedside Button",
    "device_reg_id": "f59f3b82102504ee70ab649e4300fde9",
    "area_id": "remotes",
    "cluster_details": {
      "1": {
        "device_type": {
          "name": "IAS_ZONE",
          "id": 1026
        },
        "profile_id": 260,
        "in_clusters": {
          "0x0000": {
            "endpoint_attribute": "basic",
            "attributes": {
              "0x0004": {
                "attribute_name": "manufacturer",
                "value": "Samjin"
              },
              "0x0005": {
                "attribute_name": "model",
                "value": "button"
              }
            },
            "unsupported_attributes": {}
          },
          "0x0001": {
            "endpoint_attribute": "power",
            "attributes": {
              "0x0020": {
                "attribute_name": "battery_voltage",
                "value": 27
              },
              "0x0021": {
                "attribute_name": "battery_percentage_remaining",
                "value": 193
              }
            },
            "unsupported_attributes": {
              "0x0031": {
                "attribute_name": "battery_size"
              },
              "0x0033": {
                "attribute_name": "battery_quantity"
              }
            }
          },
          "0x0003": {
            "endpoint_attribute": "identify",
            "attributes": {},
            "unsupported_attributes": {}
          },
          "0x0020": {
            "endpoint_attribute": "poll_control",
            "attributes": {
              "0x0000": {
                "attribute_name": "checkin_interval",
                "value": 13200
              }
            },
            "unsupported_attributes": {}
          },
          "0x0402": {
            "endpoint_attribute": "temperature",
            "attributes": {
              "0x0000": {
                "attribute_name": "measured_value",
                "value": 2698
              }
            },
            "unsupported_attributes": {}
          },
          "0x0500": {
            "endpoint_attribute": "ias_zone",
            "attributes": {
              "0x0000": {
                "attribute_name": "zone_state",
                "value": 1
              },
              "0x0001": {
                "attribute_name": "zone_type",
                "value": 32768
              },
              "0x0002": {
                "attribute_name": "zone_status",
                "value": 0
              },
              "0x0010": {
                "attribute_name": "cie_addr",
                "value": "00:12:4b:00:24:c3:15:9f"
              }
            },
            "unsupported_attributes": {}
          },
          "0x0b05": {
            "endpoint_attribute": "diagnostic",
            "attributes": {},
            "unsupported_attributes": {}
          }
        },
        "out_clusters": {
          "0x0003": {
            "endpoint_attribute": "identify",
            "attributes": {},
            "unsupported_attributes": {}
          },
          "0x0019": {
            "endpoint_attribute": "ota",
            "attributes": {},
            "unsupported_attributes": {}
          }
        }
      }
    }
  }
}

Additional logs
Paste any additional debug logs here.
Don't remove the extra line breaks outside the ``` marks.

Additional context
Add any other context about the problem here.

Witch Zigbee radio chip / model / brand is you Zigbee dongle ?
Also can you posting one log from (Z)HA then its happening with debug logging enable in HA ?

@MattWestb I am using a Sonoff Zigbee 3.0 USB Dongle Plus with the TI CC2652P chip. I changed the logging for ZHA to debug, rebooted, pressed the button once and saw the 3 commands on zha_event, turned off debug and rebooted again. I then took the backed up log file (log.1) and pasted it here. Hope this is what you wanted.
home-assistant.log

Im missing the communication with the device that need zigpy: debug also zigpy_znp: debug can being good but you dont need ZHA debug then its only spamming the log.
Also the NWK address of the device is good so not need trying finding it log.

@MattWestb Below is the new log. I pressed the button more than once as I wasn't seeing the event... hopefully that doesn't confuse things. You should find presses in multiples of 3. ID is 0x6573
home-assistant.log

Great logging !!
2022-11-10 15:20:20.320 DEBUG (MainThread) [zigpy.zcl] [0x6573:1:0x0500] Received command 0x00 (TSN 9): status_change_notification(zone_status=<ZoneStatus.Alarm_1: 1>, extended_status=<bitmap8.0: 0>, zone_id=0, delay=0)

Also 15:20:22.091 and 15:20:24.260 the same frame with the same TSN and its one broadcasts to GroupId=0x0000.
@puddly Is not TI filtering group broadcast with the same TSN ?
I dont know if its one direct child or the device is having one no Zigbee 3 parent -> the parent is transmitting the frame 3 times as one ZHA router and not using  passive ack by listening that the neighbors have re sending the frame.
Is IAS Zone using broadcast in one different way as the light is doing ?

@puddly Have you had a chance to look into this? Even with the latest release the issue is still present.

Started seeing this about a month back. Didn't happen before. I have a Aeotec Samjij Button with a TI chip. Other zigbee devices work normally.

I have the same problem with one of my Aeotec buttons (same as SmartThings Button). I even got it replaced and it started again 3 days later. The strange thing is, that one of the buttons still works flawlessly (this one was never replaced).

Mine used to work flawlessly and then started having issues after some HA upgrade.  I just got used to it...

Mine used to work flawlessly and then started having issues after some HA upgrade.  I just got used to it...

I have also looked at other zigbee buttons but none come close to this beautiful design while being this small. It's just frustrating to get a light show at night before going to bed. I might switch to the Aqara buttons, even though they don't look nearly as good as the Aeotec ones.

This has stopped being an issue for me now. Not sure what change but after some updates I noticed it wasn't happening anymore. Best of luck folks.

I have several of these ST buttons that I was not using. Now that I am, I noticed that just one keeps doing this on, off, on business so I suspect that the issue is device or network specific. Maybe a bad router in between or a failing button (does poor signal cause it to try more than once?), etc.

I'm also experiencing this issue. I noticed that removing and replacing the battery on the Aeotec Smart Button (temporarily) solves the issue.
My HomeAssistant is on the latest version; I'm using the HomeAssistant Skyconnect zigbee stick.

Must be my same issue as mine no longer does it. I likely replaced the battery since then. Thank you for letting me know the possible cause.


restore "Copy tuned profile produced by autotune to profile.json for …

Copy tuned profile produced by autotune to profile.json for use with next day of data"
Per suggestion for dev branch from tynbendad.

Confirmed via https://github.com/openaps/oref0/pull/484 that this works, so merging this to this branch, too.


(1.11) Poof! crashes.

Please use the search functionality before reporting an issue. Also take a look at the closed issues!
Issue description: Mekanism Generators crash :/
Steps to reproduce:

In the Minecraft Loading

Version (make sure you are on the latest version before reporting):
Forge: <IP_ADDRESS>1
Mekanism: <IP_ADDRESS>7
Other relevant version:
crash-2017-07-24_22.22.30-client.txt
If a (crash)log is relevant for this issue, link it here: (It's almost always relevant)
[gist/pastebin/etc link here]

You're using an old version. Please update to the latest version (311-fix on curseforge).

Ok thank you ^^


Documentation: improvements

Codecov Report

Merging #1171 into master will decrease coverage by 0.04%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##             master    #1171      +/-   ##
============================================
- Coverage     94.29%   94.25%   -0.05%
  Complexity      455      455
============================================
  Files             2        2
  Lines          6661     6661
  Branches       1792     1791       -1
============================================
- Hits           6281     6278       -3
- Misses          102      105       +3
  Partials        278      278

Impacted Files
Coverage Δ
Complexity Δ

src/main/java/picocli/CommandLine.java
94.08% <0.00%> (-0.05%)
314.00% <0.00%> (ø%)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b8dcc00...ba2e0a1. Read the comment docs.

Merged. Very nice, thank you!
I wasn't aware of this ifdef::env-github feature, good to know!


Document is not clear

The document contains Handle variable at a lot of places. but did not mention what is Handle.
public async Task Post(string handle = null)
in this http method what we need to send as handle?
If we already registered with using above http method then what is the use of put method ?
public async Task Put(string id, DeviceRegistration deviceUpdate)
Can i get full information  asap?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: c9a2c2c5-615f-a3da-f57f-119750ff893f
Version Independent ID: ca22039e-bcd6-a1ad-8f45-9c784c8b7881
Content: Push notifications to specific Android application users using Azure Notification Hubs
Content Source: articles/notification-hubs/notification-hubs-aspnet-backend-gcm-android-push-to-user-google-notification.md
Service: notification-hubs
GitHub Login: @dimazaid
Microsoft Alias: dimazaid

@Tirupathitemburu
Thanks for the feedback! We are currently investigating and will update you shortly.

@Tirupathitemburu The document discusses the handler in detail at this part..
If this information is not clear, can you please reopen the issue and tag me in your reply. Please share with us exactly what is causing trouble.

Hi, what about the process in Xamarin.Android? Where is the documentation to do this? Greetings

Hi, what about the process in Xamarin.Android? Where is the documentation to do this?I have Done the code for WebApi Backend but i want documentation for xamarin android to register user !


Update condition to work with new simulator JSON format

Running xcrun simctl list -j after installing Xcode 11 shows the isAvailable check has a value of true now.

Fixes https://github.com/CocoaPods/fourflusher/issues/21

Oh shoot, yes of course!

@dnkoutso There was already a test in place for Xcode 10.2, which has the isAvailable: true in place already. Not sure how they were passing, but at any rate, added tests with the Xcode 11 setup.

There was already a test in place for Xcode 10.2, which has the isAvailable: true in place already.

Where is it?

In the find_spec.rb starting on line 83. The sample JSON file has both availability and isAvailable included for each device.

@colinhumber bundle exec rubocop -a to fix Rubocop CI error.

@colinhumber ping

Sorry! I’ve been preoccupied at WWDC. I’ll take care of this this morning.

👍 Awesome

Changes pushed

@dnkoutso @colinhumber thank you both.
It may sound precipitated, but as simply installing Xcode 11 immediately causes trouble to many users, I would suggest to not delay a release of fourflusher 2.3.0.

Will release today.

 I should have asked to put this on 2-0-stable branch not master. Anyway will cherry-pick and fix and also add a CHANGELOG entry.

Actually nm it will be 2.3.0.

Released fourflusher 2.3.0 and added missing CHANGELOG entry.

gem update fourflusher or bundle update fourflusher if using Bundler

awesome, thanks a lot for the quick fix and release! :)


The prohibitive cost of on-chain Linux instances

I was wondering if this can be narrowed down to a hypothetical full-node hardware requirements specification and network size handling emulation workloads and memory distributedly, not in full sync, actually capable of providing decentralized "cloud" resources on a Blockchain network.

Since the cost of storing a full Cartesi Machine state within the blockchain is prohibitive, all machine states are represented in the blockchain as cryptographic hashes. The contents of those states and memory represented by those hashes are only known off-chain.

Sorry, I could not understand your question. Please use the issue tracker here only to report bugs. I suggest for you to check the Cartesi Discord to ask questions or start quick discussions.
In any case, in that phrase the cost is prohibitive because it's impractical to store a cartesi machine state (which can be GBs) onchain, but fraud proof algorithms that uses machine-solidity-step contracts from repository doesn't need to access the whole machine state, it just need to access some hashes of relevant parts of the state to prove whoever is challenging.


Support uploading block ids with hash

Possible api - a json list of int block ids that the CLK is a member of:
clks: [
  {"bloomfilter": "732927492", "blocks": [2015, 4]},
  {"bloomfilter": "732927492", "blocks": [2000, 4]},
]

Aha! Link: https://csiro.aha.io/features/ANONLINK-53

Looking at blocking with fresh eyes...


fix issues in scala 2.12 build

somewhere along the line, scala 2.12 support was removed from Alpakka main branch, adding it back to pekko-connectors for 1.0.0 - can be removed later

@pjfanning Looks like the nightly build didn't publish 2.13?
https://repository.apache.org/content/groups/snapshots/org/apache/pekko/pekko-connectors-cassandra_2.12/
https://repository.apache.org/content/groups/snapshots/org/apache/pekko/pekko-connectors-cassandra_2.13/

@nvollmar there is a compile issue in S3 module which https://github.com/apache/incubator-pekko-connectors/pull/65 aims to fix


members: add glrf as member

Hey :wave:
I'm requesting to join the Cilium organization.
I made a few contribution around Hubble and I intend to continue working on it.
https://github.com/pulls?q=is%3Apr+author%3Aglrf+user%3Acilium

Thanks for all of your contributions so far!


ERROR:homeassistant.bootstrap:Error during setup of component logger

Make sure you run the latest version before reporting an issue. Feature requests should go in the forum: https://community.home-assistant.io/c/feature-requests
Home Assistant release (hass --version):
0.18.2
Python release (python3 --version):
Python 3.4.3+
Component/platform:
Linux Ubuntu 14
Description of problem:
INFO:homeassistant.core:Bus:Handling <Event component_loaded[L]: component=recorder>
ERROR:homeassistant.bootstrap:Error during setup of component logger
Traceback (most recent call last):
File "/usr/local/lib/python3.4/dist-packages/homeassistant/bootstrap.py", line 158, in setup_component
if not component.setup(hass, config):
File "/usr/local/lib/python3.4/dist-packages/homeassistant/components/logger.py", line 64, in setup
for key, value in config.get(DOMAIN)[LOGGER_LOGS].items():
AttributeError: 'NoneType' object has no attribute 'items'
INFO:homeassistant.loader:Loaded notify.pushbullet from homeassistant.components.notify.pushbullet
INFO:requests.packages.urllib3.connectionpool:Starting new HTTPS connection (1): api.pushbullet.com
ERROR:homeassistant.components.notify.pushbullet:Wrong API key supplied. Get it at https://www.pushbullet.com/account
ERROR:homeassistant.components.notify:Failed to initialize notification service pushbullet
ERROR:homeassistant.bootstrap:component notify failed to initialize
INFO:homeassistant.core:Bus:Handling <Event component_loaded[L]: component=http>
INFO:homeassistant.core:Bus:Handling <Event component_loaded[L]: component=api>
INFO:homeassistant.core:Bus:Handling <Event component_loaded[L]: component=frontend>
ERROR:homeassistant.bootstrap:Invalid platform config for [light]: required key not provided @ data['platform']. OrderedDict([('Halls', ['scene.kitchen_full_bright', 'light.dinning', 'light.living', 'light.living', 'light.living']), ('Bedroom', ['light.kathir_and_cheliyan', 'light.master', 'light.paadie', 'light.paadie', 'light.paadie']), ('Stairs', ['light.stairs2nd_fl']), ('Basements', ['switch.wemo_insight_5050_led', 'light.basementleft', 'light.basementright', 'light.basementmiddle']), ('Temperature', ['sensor.temperature', 'sensor.weather_summary', 'sun.sun', 'sensor.weather_cloud_coverage', 'sensor.weather_precip_type', 'sensor.weather_precip_probability', 'sensor.weather_precip', 'sensor.weather_precip_intensity'
Expected:
Problem-relevant configuration.yaml entries and steps to reproduce:

Traceback (if applicable):

Additional info:

ohh it was working fine until  yesterday.. all i did pip3 install --upgrade homeassistant.
thats it.
i got error saying scene.yaml line 14 and 19 douplicate key "switch.livingroom_wall_outlet" .. i just remove the line ..after that it let me start but i got those error (above)after that

There are couple of error messages:
ERROR:homeassistant.components.notify.pushbullet:Wrong API key supplied. Get it at https://www.pushbullet.com/account
ERROR:homeassistant.bootstrap:Invalid platform config for [light]: required key not provided @ data['platform'].

The validation of the configuration.yaml file will ensure that the given platform/component will work.


how to set PDO::MYSQL_ATTR_USE_BUFFERED_QUERY in codeigniter

I am using Codeigniter 3.1 Version and trying to use MySQL 5.5. When I am calling Stored Procedure back to back 2 times its giving the error. Anybody know how to fix this ? **** code ****************
public function foo()
{
//test_sp,test1_sp;
echo "called sp1 ";
$sql = "CALL test_sp()";
$query = $this->db->query($sql);
var_dump( $query->result());
    echo "
 called sp2
";
    $sql2 = "CALL test1_sp()";
    $query2 = $this->db->query($sql2);
     var_dump($query2->result());
}

********** out put ***********
Error Number: HY000/2014
Cannot execute queries while other unbuffered queries are active. Consider using PDOStatement::fetchAll(). Alternatively, if your code is only ever going to run against mysql, you may enable query buffering by setting the PDO::MYSQL_ATTR_USE_BUFFERED_QUERY attribute.
CALL test1_sp() Filename: controllers/Test.php Line Number: 27

This is a bug tracker.
Post such questions on our forums.


Consider moving compactor and scan server path cleaners to a utility

The Manager and CompactionCoordinator classes start cleanup threads (here and here) that remove paths in ZooKeeper that are no longer being used. The accumulo-cluster script uses the ZooZap utility to clean ZooKeeper paths when stopping the cluster (here for example).
If we removed the ZK cleaner threads in the Manager and CompactionCoordinator, then it would be relatively easy to determine how many and which compactors and scan servers were started, but not running.

then it would be relatively easy to determine how many and which compactors and scan servers were started, but not running.

Is this for the case of determining how many server processes died?

then it would be relatively easy to determine how many and which compactors and scan servers were started, but not running.

Is this for the case of determining how many server processes died?

Stopped or died. It would make it easier to detect down servers for the monitor or other utilities.

@keith-turner and I discussed this - the idea behind this issue is to use ZooKeeper as a mechanism to understand the intended state of the system. If we know the intended state of the system via paths in ZooKeeper, then we can identify which servers are down by looking for paths that don't have an associated lock. For users using accumulo-cluster, ZooKeeper would be cleaned up automatically when performing a stop operation as accumulo-cluster calls ZooZap. My idea here was that a utility could be created and run periodically by the user to clean up ZooKeeper paths when their intended deployment layout has changed. @keith-turner suggested that it might be too easy for ZooKeeper to get polluted with old paths if, for example, the user doesn't run the utility or in a case where the user is using an orchestration system like Kubernetes that stops and starts pods when needed that end up having different hostnames. @keith-turner suggested allowing the user to specify how many processes by resource group and type that they are intending to run, then we can compare the actual running count against the intended deployment. We might be able to convey this information in a property value (json?).


undefined method `ago' for 1800:Fixnum

Hello!
Rails 4.2.1
Devise 3 5 1
I turn on option ":timeoutable".
And I uncomment option in "config/initializers/devise.rb"
config.timeout_in = 30.minutes
config.expire_auth_token_on_timeout = true
I get this error
undefined method `ago' for 1800:Fixnum
Extracted source (around line #30):
def timedout?(last_access)
return false if remember_exists_and_not_expired?
!timeout_in.nil? && last_access && last_access <= timeout_in.ago
end
  def timeout_in

Rails.root: /home/glafir/RoR/ic-glafir
Application Trace | Framework Trace | Full Trace
devise (3.5.1) lib/devise/models/timeoutable.rb:30:in timedout?' devise (3.5.1) lib/devise/hooks/timeoutable.rb:21:in block in <top (required)>'
...
What can do it?

Can you please provide a sample application that reproduces the error?

We had the same issue, it looks to be a change in the behavior (i.e. removal) of Numeric#ago in ActiveSupport 4.2.
In 4.1.7 you get the following warning:
DEPRECATION WARNING: Calling #ago or #until on a number (e.g. 5.ago) is deprecated and will be removed in the future, use 5.seconds.ago instead. (called from ago at /ruby_gems/2.0/gems/activesupport-4.1.7/lib/active_support/core_ext/numeric/time.rb:66)

And in 4.2 you get the error above.
The simplest solution for us was to change our Devise::Model's initialization call to include .seconds:
new({
  id: id,
  access_token: access_token,
  refresh_token: refresh_token,
  timeout_in: timeout_in,
  scopes: scopes
})

to
new({
  id: id,
  access_token: access_token,
  refresh_token: refresh_token,
  timeout_in: timeout_in.seconds,
  scopes: scopes
})

/cc @paveldruzyak @glafir

@bigtiger thanks

The simplest solution for us was to change our Devise::Model's initialization call to include .seconds:
Sorry, I don`t understand. Where to write it in?

@glafir the basic idea is that the value you're providing for timeout_in is being converted to a Fixnum. In our case it was caused by some DateTime arithmetic, e.g.
timeout_in = auth.credentials.expires_at - Time.now.to_i

Without seeing your code it's hard to say exactly what your fix needs to be. As @lucasmazza said, an example that reproduces the error would be very helpful.

I have done it so
in lib/devise/model/timeoutable in block "def timedout?(last_access)"
!timeout_in.nil? && last_access && last_access <= Time.now - timeout_in.seconds

I had this issue as well after upgrading Rails to 4.2.6 and I re-opened the Numeric class and solved this way(For example: You can add fixnum.rb with the following code into config/initializers folder):
class Numeric
  def ago
    self.seconds.ago
  end
end

I created a PR that fixes the issue -- so maybe you can 👍 it or something to show interest in it being merged.  I don't think we should be needing to monkey patch Numeric when ActiveSupport already has a solution (use #seconds.ago).  https://github.com/plataformatec/devise/pull/4234


How to reduce the priority of my_plugin folder ?

I installed a plugin in the folder my_plugin . This plugin makes the tab button unable to automatically generate the function structure.
There is a conflict between them.
except uninstalling the plugin, is there any good way?

You could directly change the .vimrc file not to prioritize your plugins.


Add hamberguer menu

We should hide the menu and show a hamburger menu on mobile

@DannyFeliz hey man, I'm trying to assign me this issue and I think I can't because I'm not part of the organization. Could you invite me please?

@LeonelSantiago Done, check your email.

Great! Thank you man.


PyTorch Version

I'm trying to convert this model into PyTorch version
Based on the code here, I understand the flow of Modified GDC as:
DW Conv -> BN -> Dropout -> Conv -> Flatten -> BN -> Activation Func -> Output

But from the paper, before the output, there should be linear layer
Can you check if my implementation here in PyTorch is correct? Based on my understanding, the model will create embeddings for facial recognition.

I'm trying to convert this model into PyTorch version
Based on the code here, I understand the flow of Modified GDC as: DW Conv -> BN -> Dropout -> Conv -> Flatten -> BN -> Activation Func -> Output  But from the paper, before the output, there should be linear layer
Can you check if my implementation here in PyTorch is correct? Based on my understanding, the model will create embeddings for facial recognition.

Hi,
Amazing work!
I am not expert in PyTorch, but I do not think line 76 is correct:
self.linear = nn.Linear(flattened_features, num_classes) if num_classes else nn.Identity()
The flow of Modified GDC is correct
DW Conv -> BN -> Dropout -> Conv -> Flatten -> BN -> Activation Func -> Output
The final activation function is simply a linear activation function that leads to my 512-d output. I think you can simply use:
x = x.view(x.size(0), -1)
or
import torch.nn as nn
embedding = nn.BatchNorm1d(emb_shape, momentum=bn_momentum, eps=bn_epsilon, affine=scale)
embedding_fp32 = nn.Identity()(embedding)
And you can optionally use:
#optional
embedding_fp32 = embedding.to(torch.float32)

I see, I misunderstood about the 512 embeddings. Will fix it, thank you.

activation function is simply a linear activation function that leads to my 512-d out

Thanks for your amazing work!
I am very glad that someone is actually implementing the model using PyTorch.
Keep up the good work!

Thank you for your work too!


Next.js dynamic routes 404'ing when using $1 on Now

Let me preface this by saying I don't know for sure whether it's a Next.js or Now bug, but I'm thinking Now.
I have a repository that has a Next.js app in a frontend/ folder, and a now.json file in the root.
In now.json I redirect all Next.js pages from frontend/ to / with { "src": "/(.*)", "dest": "/frontend/$1" }, so they are available in the root of the website.
You can find a reproducible example here. As you can see the main page works perfectly: https://now-next-dynamic-paths-issue.volst1.now.sh
However now comes the interesting part: I'm using a dynamic route, which should be available on https://now-next-dynamic-paths-issue.volst1.now.sh/event/asdf (asdf is just a random parameter). However, this gives a 404!
Maybe this isn't a bug (I think so though), but with now dev this does not happen and everything works fine!

I'm facing the same issue.
I expected that it should work with routes config like that
[
    {"src": "/event/(?<id>[^/]+)", "dest": "/frontend/event/slug=[slug]" },
    {"src": "/(.*)", "dest": "/frontend/$1" }
]

I am facing the same issue, too :(

+1

@SpaceK33z have you solved the problem? I tried specifying each dynamic routes in now.config.json as @eugrdn mentioned, but still got 404 of the client-side script file.

Probably this issue should be moved to next.js or wherever next-server package is situated

@indiejoseph no, but I do have a workaround based on @eugrdn's solution:
[
  { "src": "/event/([^/]+)", "dest": "/frontend/event/[slug]?slug=$1" },
  { "src": "/(.*)", "dest": "/frontend/$1" }
]

Note: /frontend/event/[slug].js is a real file that exists with that name.
@eugrdn: but is it a problem with the next server? With next build && next start this works correctly.

Okay I think this has to do with the @now/next builder, so I'm moving this to https://github.com/zeit/now-builders/issues/811.


Cross Module Dependencies

Modules can query other modules just like an application queries modules:
query({
  uri: "latest.ethereum.web3api.dev",
  query: `mutation { sendTransaction(
    address: "0x...",
    method: "foo(uint256,address)",
    args: [55, 0x...]
    ) { tx { hash } }
  }`
})

And in the client it can filter for these domains and choose to send them to local modules if it has some. This way we don't need to implement it in WASM, and can instead use a module in the client language to do these requests.

const client = new Web3API({
resolvers: [
{
uri: "v1.api.ethereum.*",
resolver: new EthereumProxy(...)
},
(uri: string, query: string): any | undefined => {
if (uri.contains("ethereum")) {
// TODO: call into JS module
return result;
} else {
return undefined;
}
}
]
})

Register Dependencies:
export function register() {
  // TODO: "these are my dependencies"
  _w3_register_dep("v1.api.ethereum.web3api.eth");
}

Working on the resolver idea

This has been completed in prealpha-dev

This has been completed in prealpha-dev


Adding System Diagnostics and Health Check article

Status
Ready/In Progress/In Hold (Reason for hold)
Related Issues
fixes: link to the issue
Description
A few sentences describing the overall goals of the pull request's commits.
Screenshots
Paste here any images that will help the reviewer

Why is that in the integrations and incidents health check article? Should be in a different one.


Differences between train_CSAKD  and train_CSAKD_offline_teacher_student

Hello, I noticed there are two training scripts in the repository: train_CSAKD.py and train_CSAKD_offline_teacher_student.py.
Could you please clarify the differences between these two scripts?
I'm curious to know which training method is utilized in your implementation.
Any insights would be greatly appreciated.
Thank you!

hi,
train_CSAKD.py is the default implementation for our CSAKD, our CSAKD uses the feature map distillation loss to align the feature map between a teacher and student model. Therefore, it is online knowledge distillation training and it archieve more greater results.
This repository needs more maintanence, I will update more detail information in a few day! Thanks!


How to overwrite the artwork of tracks in the RBv6 database?

Is your feature request related to a problem? Please describe.
I would like to overwrite the image path of tracks.
Describe the solution you'd like
A method to overwrite the X metadata of an Y song by it's id
Additional context
I only found that it's possible to read data but not write on

Hey,
as you pointed out, adding new rows to the database is not possible until now.
In principle it is possible, but I’ve noticed some weird behavior that I don’t yet fully understand. If you update the database and open Rekordbox, the changes appear as expected. However, after closing and reopening Rekordbox, most of the time the added items don’t show up anymore, even though they are still in the master.db file.
I am still trying to figure out what causes this, but I’m certain it can be done somehow. But that probably will take some time and a lot of testing, since I don’t want to publish anything that could mess up the library of people.
In your case, however, I think it should be possible, since the ImagePath is just a regular text field, so changing the value should not cause any issues. There is a ImageFile table, but at least in my case it is empty. The only thing to figure out is how Rekordbox decides where to store the image (somewhere in /shared/Artwork/<three letters>/<UUID>. I'll do some testing to see if it works to just place the new image in the same directory.

I did some testing, and it seems to work to replace the three artwork images. However, i still don't know how Rekordbox decides which parent folder to use for the artworks, specifically the name before the UUID directory. As stated above, it is a three character directory, for example:
 b16, 859, 4d1, ...

It seems the name is chosen randomly (not a part of the UUID or something else). Will have to look into that more.
If you only want to update the artwork of tracks that already had a different artwork, you can use the code snipped below. I will add the feature as soon as it is possible to add new artworks.
Code
import os
from PIL import Image
import numpy as np
from skimage.transform import resize
from pyrekordbox import Rekordbox6Database

def pad_image(image: np.ndarray) -> np.ndarray:
    shape = image.shape
    size = max(shape)
    dw, dh = size - shape[1], size - shape[0]
    padw, padh = dw // 2, dh // 2
    padding = [(padh, dh - padh), (padw, dw - padw)]
    if len(shape) == 3:
        padding.append((0, 0))

    image_padded = np.pad(image, padding, mode="constant", constant_values=0)
    assert image_padded.shape[0] == image_padded.shape[1], "Image is not square"
    return image_padded

def rescale_image(data, size, anti_aliasing=False):
    if size[0] != size[1]:
        raise ValueError("Image size must be square")
    rescaled = resize(data, size, anti_aliasing=anti_aliasing, preserve_range=True)
    return rescaled.astype(data.dtype)

def save_artworks(db, content, src_file, anti_aliasing=True):
    # Read image data
    with open(src_file, "rb") as fp:
        image = np.array(Image.open(fp))

    # Pad image to square
    if image.shape[0] != image.shape[1]:
        image = pad_image(image)

    # Create medium image (240x240)
    image_m = rescale_image(image, (240, 240), anti_aliasing)
    # Create small image (80x80)
    image_s = rescale_image(image, (80, 80), anti_aliasing)

    # Save images to content folder
    img_relpath = os.path.dirname(os.path.normpath(content.ImagePath))
    img_relpath = img_relpath.lstrip(os.path.sep)
    img_root = os.path.join(db.share_directory, img_relpath)

    path = os.path.join(img_root, "artwork.jpg")
    with open(path, "wb") as fp:
        Image.fromarray(image).save(fp)

    path_m = os.path.join(img_root, "artwork_m.jpg")
    with open(path_m, "wb") as fp:
        Image.fromarray(image_m).save(fp)

    path_s = os.path.join(img_root, "artwork_s.jpg")
    with open(path_s, "wb") as fp:
        Image.fromarray(image_s).save(fp)

if __name__ == "__main__":
    db = Rekordbox6Database()
    src = <File path to new artwork>
    cid = <ContentID>

    content = db.get_content(ID=cid)
    save_artworks(db, content, src, anti_aliasing=True)

hey, thanks for your work!
i have a question -
is it possible to edit the metadata of songs on Rekordbox programmatically?
ideally i mean adding a new column that i could then see while mixing, or maybe also just modifying a tag/comment?
of course this is possible manually but i'm asking programatically.

Not sure what you mean by  manually/programmatically.
For now, you can only change values of songs that don't link to other tables (for example Artist, Album, ...). Check the documentation of the DjmdContent table which fields store data directly:
https://pyrekordbox.readthedocs.io/en/stable/formats/db6.html#djmdcontent
If you want to change the title of a track, for example, you can do so by assigning a new value to the attribute:
from pyrekordbox import Rekordbox6Database

db = Rekordbox6Database()
content = db.get_content(ID=<id of song>)
content.Title = "New title"
db.commit()  # Save changes

Make sure to back up your Rekordbox library before making changes!

Hey @jcardonne, does the script work for you?

Hey @dylanljones, late answer I know haha
I don't know why I can't get the share_directory from the DB
AttributeError: 'Rekordbox6Database' object has no attribute 'share_directory'

Hey @jcardonne, are you using the newest version? The attribute name changed recently.
share_directory should work:
https://github.com/dylanljones/pyrekordbox/blob/eb0bf179f987361f0a54514884a9576cc98dfd51/pyrekordbox/db6/database.py#L262-L266

I updated my packages and that worked, thanks!


Better ODT support

Sorry for such a long pull request, I work on this on my free time so I just have to convolute all it at once.
First of all I've made various enhancements on the basecode to make it more readable, nothing crazy just some more docblocks and space between logic units.
Now the real stuff:
Firstly: the biggest and most important change made is the improvement done over convertOdtToHtml(). After an extensive refactoring of this method it is now capable of translating paragraphs, headings, tables, links and lists from an .odt file to an HTML string. It could also be possible to include images as base64 data, but not sure if that should be made since it could mean a very important performance downgrade. That is yet to be tested.
Also and sadly, after an extensive review of the OASIS standard I've realized it's not possible to parse bold and italic spans since these are dynamically set by the text editor and don't have any common, consistent naming.
Secondly; there is also now a better support for raw HTML files making use of tags stripping, and only taking in those HTML tags that have a Markdown equivalent. Why Markdown? This syntax defines and shapes the text in documents, since this parser aims to extract the text from documents, I consider sharing characteristics with Markdown in terms of tags support to be an understandable decission.
Future of this package:
Lastly, while working on this updates, I've come to some doubts about the parser itself. I myself have added some trimming in order to output what I consider to be a "better" result, however this trimming may come unsolicited for a parser as this package is.
This leads to the fact that the output this parser returns for different inputs is very inconsistent (for instance, the result from parsing .rtf to .docx) or it is highly transformed, as I said, artificially adding or removing certain elements that could fall out of the scope of a parser.
Should it remain this way? Or should the outcome be standarized?
I'm looking forward to your opinion about this changes and my questions about the parser.
Thanks in advance and have a nice day :)

Thanks @facuonline for your input on this.
I agree that we should definitely standardise the output, however, I think that there should also be an option for the raw parse result - what are your thoughts on this?
I'm at work at the moment, so I will have a look at the PR more in depth later on and merge it in.
Luke

There are now two problems to solve:
Problem 1: Standarization
What is raw and what should the standard output be like? Different formats offer very different raw data, .rtf is itself raw data, while .odt or .doc have layers to their data, which is xml that appears zipped firstly and then it comes with different xml subsets.
Raw solution 1: raw is the most basic string that can be obtained out of a file. Without considerations about the format they come in, i.e no tags stripping, no trimming, no manipulation of the data.
Raw solution 2: raw is the most basic meaningful string that can be obtained out of a file. Taking in consideration meta values, stripping tags and/or trimming garbage data, but no formatting.
Not a fan of solution 2 because there are so many different variables when it comes to extracting meaningful text out of a file format that it's basically better to just skip to the standarized output parsing...
I'm convinced that the best solution to standarize the parsed output is to make it an HTML valid string, Markup syntax compatible.
HTML is the basic language of the web, I think it doesn't require much explanation why the output should be HTML formatted. However it's important to make it Markup compatible, taking only those pieces of HTML that can be represented in Markup. Why? Because doing so ensures that 1. The text is meaningful. Our output only represents what's intended to be in the document and ignores everything else that is not real text data. 2. Reduce the workload of the development. Markup is very strict yet universal, adjusting to it's parameters ensures a one-size fits all solution that will safely enough distinguish between meaningful data and make it valid for every web service.
Problem 2: API
How to define the usage to obtain raw data and formatted data?
So far this package offers a non object API but just a collection of static methods. This is neither right or wrong, works fine and performs well. However there could be some advantages we could benefit from by taking a more OO approach as proposed in solution 1:
Solution 1: Convert this class in an object to transform the current usage API into something like following:
$parser = new \LukeMadhanga\PHPDocumentParser;

$parser->fromFile('path/to/file', 'mimetype');
$parser->toHTML(); // HTML formatted, markup compatible text
$parser->raw(); // raw data

$parser->fromString('some raw html string maybe', 'html');
$parser->toHTML(); // HTML, markup compatible text
$parser->raw(); // same string

This API presents itself format agnostic and hence easier to remember and understand. It also means that we can easily obtain formatted or raw results by passing the data once, against the current format driven API. Solution 1 parses once, formats on demand, solution 2 both parses and formats on demand.
Solution2: Maintain static methods.
// 3rd param `true` tells the method to return HTML
$html = \LukeMadhanga\PHPDocumentParser::parseFile('path/to/file', 'mimetype', true);
$raw = \LukeMadhanga\PHPDocumentParser::parseFile('path/to/file', 'mimetype', false);

PS: I yet can't take a decission about base64 encoding images found in OASIS (.odt, .odf) raws to be able to serve them in the case of HTML formatted output.

Hi @facuonline . I've had a chance now to look at the PR in close-detail and I'm happy with the work that you have done here - Thanks.
I'm a fan of Solution 1, although personally, I would expect
$parser = \LukeMadhanga\PHPDocumentParser::fromFile('/path/to/file.ext');
$parser->toHtml();

or
$parser = new \LukeMadhanga\PHPDocumentParser(path/string)
$parser->toHtml();

I am still perfectly happy with Solution 1 as it is, but just being pedantic, I would rename from to set
$parser = new \LukeMadhanga\PHPDocumentParser;

$parser->setFile('path/to/file', 'mimetype');
$parser->toHTML(); // HTML formatted, markup compatible text
$parser->raw(); // raw data

$parser->setString('some raw html string maybe', 'html');
$parser->toHTML(); // HTML, markup compatible text
$parser->raw(); // same string

What are your thoughts on this?


translate form builder data into japanese

WIP:

[x] home
[ ] getStarted
[ ] api
[ ] advanced
[x] builder
[ ] faq
[x] nav
[ ] generic

awesome! thanks


Feat auth: allow additional secrets as env variables

What kind of change does this PR introduce?
Feature
What is the current behavior?
It is currently not possible to pass values of other secrets to the auth service. This means adding external identity providers will cause you to add the secrets in plain text.
What is the new behavior?
By using envFrom we can reference a secret that contains all the env variables to configure for example an external identity provider.
Additional context
None

@arpagon When can we merge this?


Proposal: Merge bootstrap into ping monitoring node?

@andrew-cc-chen @henryxu1997 @hivanco @michaelweiyuzhao
Just want to get you guys' opinions and see if anyone has reasons to oppose this change? It would decrease the # of repos and merge the functionality.

Since the ping monitoring is currently strategically placed at points where traffic must go through (e.g. default gateway / router), it serves as an ideal location to collect metrics from.
It's also a logical assumption that peers could look for bootstraps by attempting to connect to their default gateway.

I think we could integrate the option of reading a key from disk into NewNode() just for flexibility's sake. In that case, the only difference between the bootstrap and any other node is the master bootstrap list we keep.


Tags not written out correctly in unify.String()

I have the following rule:
import "pe"
import "cuckoo"

global rule payload : tag1 tag2
{
meta:
	description = "Some payload"
strings:
	$mz = "MZ"
	$r1 = /some regex/ nocase wide
//	$r2 = /20[10][1-9](0[1-9]|1[0-5])([0-2][1-9]|3[01])/
	$s1 = "import1"
	$s2 = "import2"
	$s3 = "user-agent"
	$s4 = "accept"
condition:
	$mz at (0) and
    (pe.characteristics & pe.DLL)
	3 of ($r*, $s*) //and
//	for any i in (1..#r2) : (#r2 > 1)
}

When the rule is parsed and then converted back into a yara rule string, the following happens:
import "pe"
import "cuckoo"

global rule payload : tag1tag2 {
...

Note that tag1 and tag2 have been (effectively) concatenated.
Additionally, I have used the API to add some tags of my own:
    newTags := []string{"MalwareName", "MalwareType", "tagX"}
    p2.Rules[0].Tags = append(p2.Rules[0].Tags, newTags...)

Turning this parser into a string yields the following:
import "pe"
import "cuckoo"

global rule payload : tag1 tag2 MalwareName MalwareTypetagX {
...

Note that MalwareType and tagX have effectively been concatenated, in this case.

Fixed in d0ff531e737c586a0561832ffc903cca1c7e7f68


Remove feature group

removes feature group per schema PR #672

these commits should be squashed

hopefully I did this right

looks good to me
+1

ok - will resolve the nit and squash

squashed and updated


Set up travis to deploy to PyPi

Right now we need to manually upload ga4gh source distributions through the PyPi web interface to release a version.  It would be much better if we had Travis do this for us.  Here's a HOWTO: http://docs.travis-ci.com/user/deployment/pypi/

This seems like overkill and unnecessary work to me; once you have your PyPI credentials set up, you can use
python setup.py sdist upload

and it's done.

That's actually not the recommended way to do it since it goes over plaintext
https://python-packaging-user-guide.readthedocs.org/en/latest/distributing.html#uploading-your-project-to-pypi

Didn't realise that, thanks. Still, twine looks pretty straightforward.

Closing this due to inactivity. Twine is working fine for now I think.


Request: convert ios template project from obj-c to swift

As Swift is the future of development with any Apple product, I'd like to know if the ios template that is generated from flutter create could be a swift template rather than an obj-c template.
The code readability of swift in substantially higher that obj-c and seeing as how one of flutters goals is to make development across platforms simple, readability is an easy way to help with simplicity.

I like this idea. I personally find myself able to read and understand Swift, but not ObjC.

I like it to; Swift is a lot easier to read.
But for plugins / native interop purposes we likely will need to support both. AFAIK that should be possible, as iOS supports projects with both using "mix and match":
https://developer.apple.com/library/content/documentation/Swift/Conceptual/BuildingCocoaApps/MixandMatch.html

@mit-mit if we change the output of flutter create to use Swift, how would that impact plugins?
(agreed that our plugins should support user code written in both objc and swift)

We think it will be fine, but we need to do more work to confirm it.

Changing this to supporting Swift as an option, in addition to OjbC. This could be something like flutter create -swift myapp

I suppose it's planned, but an option for swift plugin template like flutter create --plugin -swift myplugin would be handy.

@rxlabz wrote one btw: https://github.com/rxlabz/FlutterSwiftPlugin-template

Closed per #10259


python310Packages.mockupdb: switch to pytestCheckHook

Description of changes
Fix build (https://hydra.nixos.org/build/197448243)

disable failing tests

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
22.11 Release Notes (or backporting 22.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 199600 run on x86_64-linux 1

2 packages built:

python310Packages.mockupdb
python39Packages.mockupdb


SAP HANA XS Advanced, Creating a Node.js Module

I have following section for nodejs module in my mta.yaml but it doesn't work.
App Build is successful but can't run xsjs file. Running on  xsjs port directly  index.xsjs it says unauthorized. Running on web module port it says file not found.

name: core_xsjs
type: nodejs
path: core_xsjs
provides:

name: core_xsjs_api
properties:
url: '${default-url}'
name: ''
requires:
name: UAA-service
name: hdi_db
name: db

Hi, Teejay,
The yaml looks fine but what you are describing could be either the folder structure or the routing. The call to the index.xsjs saying unauthorized is expected as your end point is now the web module. So you should be calling the same URl as the index.html but with index.xsjs, provided that the routing is configured properly.
Please create a question in the Q&A (answers.sap.com) including screenshots of the routing configuration, the folder structure, the web module URL and how the same URL with the proper folder structure fails. You would not believe how much screenshots help up help you.
Best,
Lucia.