Legacy chaincode support with invoke

I have a Legacy chaincode where I need to pass Arguments to Init function, To support legacy chaincodes there is
peer chaincode invoke cli command  with --isInit   parameter, see it here - https://hyperledger-fabric.readthedocs.io/en/release-2.2/commands/peerchaincode.html?highlight=invoke#peer-chaincode-invoke.

I've reviewed the details and there is a reason why the invokes were not added. This is because it would break the idea that the Ansible Modules should be idempotent; there's no way to know if a transaction has been submitted already. other modules can be safely run repeatedly without issue - not the same with an invoke.
Having said that, this init-required is a special case, so I'll look at if it can be added to the committed_chaincode task - so that when the chaincode is committed, the init transaction is sent at the same time.
The restriction would be that if the init transaction fails and it needed resubmitting then that would need to be done by other means.
Here's the candidate PR:
https://github.com/IBM-Blockchain/ansible-collection/pull/592


feat(dispatch): apply a new design to the dispatch component

Apply an updated design to the dispatch component, which is based more closely on material design.
Additionally, factor much of the code in dispatch.component.ts into functions.
What new features does this PR implement?
Applies a new design to the dispatch component.
What bugs does this PR fix?
This PR does fix one bug I discovered while testing: In the current implementation the maxFileSizeValidator fails to correctly apply an error when a file that is too large is selected. It is checking control.value.size, which is undefined, instead of control.value.files[0].size. Additionally, changeProject uploads the file regardless of whether this error is applied in the current implementation. Both of these errors are fixed with the implementation in this commit.
Does this PR introduce any additional changes?
I did include large style changes in this commit. Apologies, I know that makes for a harder review. I can attempt to break them out into separate commits later this week if you'd prefer. Adding comments and breaking code into functions is a part of my process to ensure I understand what I'm reading, and in this case I've left those changes in since they should improve general readability and testability of this component as well.

Many of the large functions within dispatch.component.ts have been broken into smaller functions.
Many large expressions within dispatch.component.ts have been broken down into multiple smaller expressions.
Class variables that are redundant or can be local variables have been removed from dispatch.component.ts.
I've added comments in a few places I felt they could help.
I've updated linebreaks and indentation style to match Google's style guide. I hope this isn't an unwelcome change. I saw a couple format prs go by that made me think the more aggressive line breaking was being removed going forwards. I can reset this if desired.

How have you tested this PR?
I've run existing dispatch tests, but have not written new ones for the dispatch component. A lot of the dispatch component is a little too UI focused for effective unit testing, but in a future commit it would be great to break the logic for loading data and updating model into a separate controller class that could be more easily tested. I could update this PR with another commit to implement new tests later this week if we want to get those ready before merging.
For now I've manually exercised that the controls update as intended, that errors display when and only when inputs are incorrect, that controls populate as they should when opening the route with query params, that controls enable and disable options as model, algorithm, and policy selections are made, that controls populate as they should when a sed doc spec for a selected project loads, and that form submission completes successfully.
Here is a video displaying what the new UI looks like:
https://user-images.githubusercontent.com/1418389/155122999-cf90b41f-bcc3-42a0-949d-692ad4d8fb61.mov

We discussed applying this design to the other forms within dispatch, and of course to the new edit simulation form. I haven't had time to finish these changes yet but hope to later this week. I can either update this PR with additional commits or create new PR's for these changes as preferred.

I did include large style changes in this commit. . Apologies, I know that makes for a harder review. I can attempt to break them out into separate commits later this week if you'd prefer.

No worries, this looks great! I think the code is much more readable this way

Many of the large functions within dispatch.component.ts have been broken into smaller functions.
Many large expressions within dispatch.component.ts have been broken down into multiple smaller expressions.
Class variables that are redundant or can be local variables have been removed from dispatch.component.ts.
I've added comments in a few places I felt they could help.

Thank you so much for this. This is definitely an area we have not focused on as much as I would like since we have been rapidly prototyping. This is really helpful

I've updated linebreaks and indentation style to match Google's style guide. I hope this isn't an unwelcome change. I saw a couple format prs go by that made me think the more aggressive line breaking was being removed going forwards. I can reset this if desired.

So we use prettier for formatting, so this might actually end up getting auto formatted out when committing. Generally, running 'nx format' will format all the files to the settings in the .prettier.rc. If you have any preferences to change the prettier settings, feel free to make a PR. Prettier ensures that each dev does not need to think about formatting much
I really like the new clean look of the UI. Do you think the user experience is better without the icons? That is the only thing that caught my eye that I thought was good to have in the older design. Or was it making the UI too cluttered/busy? I don't know much about the actual principles of UI other than personal opinions

The new style looks nice! We're not working on the UI very much, so I think its fine to group changes to all of the forms (at least the existing ones) into 1 PR so they are all deployed with the same style together. We can merge those changes in when you're ready.
Improving the UI will be a big improvement for users. After the content of the database of modeling projects, a UI for modifying simulation projects and re-running them is the main requests we've gotten. As Bilal alluded to, we've focused our attention on building the abstractions, building backend services to run simulations, and curating content (simulation methods, models, data visualizations). We haven't had as much bandwidth as we'd like for the UI.

Thanks!
I'll take a look at the prettier settings. You're right running format does re-format a lot of my changes. Does the team currently have strong preferences about formatting? I tend to lean towards one-lining expressions whenever they can fit in the line limit, as long as the information content isn't too out of hand. A bit of flexibility can be nice with formatting I find. Making simple areas dense and complex areas more sparse can help a reader understand what is boilerplate they can skim and what is meaty code to read closely, but I understand consistency is valuable as well.
As for the icons, I did find the previous UI a little too dense, and that removing the icons was an improvement to that. However I'm no designer, I'm just operating from intuition and personal preference, so take it with a grain of salt! One bit of reasoning I'll add here is that a lot of these icons either aren't instantly recognizable, aren't closely related to the function of the form section, or are redundant.
For example the algorithms, models, and policy fields are all right next to each other and all have the same paper icon. So this is using a fair amount of space, is redundant, and it's not an icon that I personally would immediately recognize or that would help me understand the purpose of the field. The simulation tool section has a gear icon, which is related to a tool, but it's not an icon I recognize or that would help me understand this field either more quickly or more correctly than by just reading the description alone. In cases like this my preference is to lean towards a more minimal design, where call to actions and text content are front and center, and icons that don't have an immediate value add are trimmed.
But I could certainly create a version with the icons re-added so they could be compared if you'd like, or simply add them back in!

I don't have strong preferences about format. Bilal set up the automatic TypeScript formatting. The Python formatting follows PEP8. Line length is one of the style rules I've overrode for Python because I find the default too short.
I agree that the icons are closely associated with the domain content. We've largely stuck with built in icons from FontAwesome. Ideally, we'd create a more informative icon set.

I'll take a look at the prettier settings. You're right running format does re-format a lot of my changes. Does the team currently have strong preferences about formatting? I tend to lean towards one-lining expressions whenever they can fit in the line limit, as long as the information content isn't too out of hand. A bit of flexibility can be nice with the formatting I find. Making simple areas dense and complex areas more sparse can help a reader understand what is boilerplate they can skim and what is meaty code to read closely, but I understand consistency is valuable as well.

No preference really as long as it's consistent.  I think? the settings for the prettier are just the prettier defaults, which are somewhat widely used, but if you have any preferences that can be added to the prettier settings file, just run nx format to apply to all the files.
Regarding that using density would separating out boilerplate blocks with extra line breaks and comments work?

Just pinging here to let you know I haven't forgotten about cleaning this PR up and getting it ready to merge! My team got caught up in a re-org at Facebook. The dust hasn't settled yet and my time/energy has been a casualty. I should be able to carve out a slice of time early this week. If this is blocking any tasks that you'd like to start asap let me know and I'll make this a priority!

Thanks for the update. This isn't blocking anything. We're still trying to wrap up integrating more models (content for the database).

I've added a few more commits to this PR tonight, which cover my work so far towards applying the new design to the Create Project form. I'm afraid to say I also went a little beyond scope here again and made an additional UI tweak. I noticed there was a lot of logic in create-project.component.ts around updating and clearing field values when other field values are modified by the user such that they are no longer compatible.
This is unideal because it can be disorienting to a user to see things changing in their form that they don't expect, the logic is complicated and there were broken edge cases, and it allows for the potential for a user to create a project with different values than they thought they entered. And worse, when the introspection request completes it auto-fills several forms without warning, potentially overwriting values the user may have entered without them knowing.
To solve this, I thought this form might be better represented as several sequential steps that flow from one to the next. If one field depends on another, then it comes in a later step, so that by the time the user sees any given field and has a chance to edit it there is no longer any possibility of it becoming incompatible with its dependencies and needing to be cleared or reset. Additionally, a spinner is inserted into the flow while introspection takes place, so fields are prefilled before the user sees them.
The end result is a decrease in the amount of syncing logic that has to be maintained in code, and IMO a better user experience, but of course that is subjective. This can all be put back into a single page form with a little work if thats the design you'd prefer!
Other than this, the design changes are similar to the last round: increasing margins and padding, increasing font size, removing header backgrounds and some borders and icons, generally giving the form more space.
Here is a video of me doing a quick test run through the form:
https://user-images.githubusercontent.com/1418389/159645280-6f2d3f5e-663f-4d8b-a0f9-f57e587d6cf8.mov
Similarly to last round, I've also done some code cleanup as I've gone through the relevant files. This amounts to a lot of breaking things into separate functions, creating a few shared lips for decreasing boilerplate, and breaking out a lot of the logic for translating the data model from combine classes to dispatch classes and back into a utility class of static functions.
This will hopefully improve readability and definitely improves testability. I plan on doing more manual testing with this commit in the near future, as well as documenting the new class/functions, and writing tests. The logic for paging the forms could also be broken into a shared form paging component lib if it needs to be used elsewhere. So this isn't quite ready to land yet and if you pull for local testing you may find broken edge cases, but I figured I'd get some feedback on the design changes before doing any more polish.
Does the multi-step form design seem like a good fit? Any other changes you'd like to see added or reverted?

Additional important note: I added a commit here to update the prettier settings. The change sets the new line guide to 120 characters per line, and makes it so brackets stay on the same line when prettier inserts newlines. I found these settings to provide a more comfortable experience, but let me know if you disagree and I'll set them back.

@jonrkarr Thoughts on the multi-step form approach vs maintaining the single page form? I'll have some time for tests and polish this weekend, but want to make sure that approach is ok before continuing.

Thanks for these suggestions! Breaking the steps down is nice. I agree this is probably much easier to understand. The videos are really helpful.
(a) Could you change the order of the steps

(No change): Choose model format / model
(No change): Chose modeling framework / type of simulation / algorithm
Review/modify namespaces
Review/modify model changes
Review/modify which variables should be recorded
Review/modify algorithm parameters

(b) Could you display the algorithm parameters and compatible simulation tools in the same pane? Because this is related, I think this is helpful to display together.
(c) The very last buttons ("Simulate" and "Download") are much larger than the next/back buttons. Should these be stylistically congruent?
We should probably merge this soon to ensure there's no painful merging (not that we intend to make any changes related to this soon).

@jmdetloff please let me know when you'd like this merged.

In case its not obvious, the CodeFactor and SonarCloud failures can be ignored. We're not following those strictly.

The multistep form is much better! I had been hoping to make these workflows multistep forms as well, definitely agree its a better user experience.
The sonarcloud checks did pick up two potential bugs that should be fixed though. Let me know if they are false positives and we ca then ignore them

Woops, looks like those two bugs are both referring to a typo that slipped in, I'll get rid of that and resubmit. I also have a couple commits with tests incoming. After those arrive this should be good to land!
Can definitely move around the order of the forms and combine algorithm parameters and simulation tools. 👍
As for the final buttons, I went back and forth a bit on the styling. On one hand they look a little odd with the extra space they create in the layout, on the other hand they are the main call to action for the form so emphasizing them made sense to me. I also tried them center and left aligned, and stretching to span the width of the pane. I'll try another version with them smaller to match the next/back buttons and post a screenshot.

Another quick question: The run flow could be factored into a multi-step form as well if that's the UI you'd prefer. Should I make that change also? If I recall correctly the fields aren't quite as interdependent on that one, but there are some selectors whose options update or clear in response to other fields being edited, so multi-step could have value there as well.

Another quick question: The run flow could be factored into a multi-step form as well if that's the UI you'd prefer. Should I make that change also? If I recall correctly the fields aren't quite as interdependent on that one, but there are some selectors whose options update or clear in response to other fields being edited, so multi-step could have value there as well.

Up to you.
When a file/url is added there is some introspection that disables the simulation tools that are not compatible with that model. This could cause edge cases like the one you noticed earlier when the model is changed, so if you think it would be better to make it modular go for it.
On the other hand, as someone who often uses the form to test the backend,  having to do multiple clicks to step through might also be annoying.
I'll defer to you on this, the changes so far have been great, so whatever you think works best.
If it isn't too much trouble, #4390 could be included as well. When I am submitting multiple runs to test the backend typing out names gets annoying.  Making the name optional entirely would be possible, but this is incorporated into the UI/API/database models, so would need changes throughout.
I thought a better method might be to have the form generate a random name if the user does not provide one, kind of how docker creates an "adjective_scientist" name for containers. Unsurprisingly, someone has made an npm package to do this. https://www.npmjs.com/package/randimals

Thank's for your patience with this PR! I've been team transitioning over to PyTorch at Meta, so ramp up study has eaten a lot of my time the past few weeks.
I've pushed a new commit to this PR this morning in which I make some changes/additions:

The changes @jonrkarr requested have been made: The steps are reordered, simulation tools and algorithm parameters exist in the same form step, and the style of the simulate and download buttons have been changed to match that of the next button.
The logic related to paging through the form steps has been factored out into a new component, MultiStepFormComponent. This will allow for easy re-use if needed in the future, clearer code in the create-project component, and isolated tests of the form functionality.
A good deal of static logic related to creating a CombineArchive from form data, determining which project options are compatible with each other, and introspecting a SedDocument have been broken out into functions in the modules archive-creation, compatibility, and introspection respectively, within the simulation-project-utils-service project. This logic is all state free, so moving it into functions will make for easier reading and testing, and allow the components that use them to focus more on presentation logic.

There's a lot of code moving around here, but the cumulative effect of factoring out the form paging logic into MultiStepFormComponent, creating individual sub-components for each step of the form, and moving static logic into new modules is that the create-project.component.ts file has shrunk down from an intimidating ~1900 lines of code to a much more manageable ~180 lines of code. The newly created components and modules are of a similar scale.
I've written some tests, and stubbed out some more, but this PR has ultimately touched quite a bit of code. So, there are many more tests that could and should be written. I'll keep adding them as long as this PR is open, but IMO this should be solid for landing in its current state. I've run it through the paces manually, and will attach a video. I've been testing primarily by using the model at http://bigg.ucsd.edu/static/models/e_coli_core.xml, playing around with the different form settings, and then running the simulation or downloading. If there are other important flows I should make sure to exercise let me know and I can run through them and attach videos as well.
What is still not included in this PR, but can be added in follow up PR's:

The run flow has not been changed to a multi-step form. @bilalshaikh42's comment about this adding friction to a flow that is run through frequently is a good point. In this case the fullscreen form might strike the better balance between clarity and efficiency. This can always be changed down the line if needed.
#4390 I'll dig into this next, and either add it to this PR, or spin up a new PR if this one has already landed.
Complete test coverage - I've written some tests for the new MultiStepFormComponent, and a handful more for the create project form steps, but there are plenty more tests to write to achieve full coverage. The good news is that testing should be much easier going forwards with the more modular code.

https://user-images.githubusercontent.com/1418389/167296702-d98ad6d3-6eea-405c-8efb-7a87a9ad2e12.mov

:tada: This PR is included in version 9.14.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:

What is still not included in this PR, but can be added in follow up PR's:

The run flow has not been changed to a multi-step form. @bilalshaikh42's comment about this adding friction to a flow that is run through frequently is a good point. In this case the fullscreen form might strike the better balance between clarity and efficiency. This can always be changed down the line if needed.
Allow simulations submission without specifying a name  #4390 I'll dig into this next, and either add it to this PR, or spin up a new PR if this one has already landed.
Complete test coverage - I've written some tests for the new MultiStepFormComponent, and a handful more for the create project form steps, but there are plenty more tests to write to achieve full coverage. The good news is that testing should be much easier going forwards with the more modular code.

@jmdetloff
I have merged this in to get some conflicts with #4451 resolved, so feel free to open additional prs for these! Thanks again.

The improved form looks great!
The most helpful contribution would be the UI we discussed for editing simulation projects.
The ability to submit projects without a name isn't a priority. Plus, we could easily address this by automatically generating a name such as {simulation-tool-id} {simulation-tool-version} run {time-stamp}.
Complete test coverage would be great, but this is already better than we have for nearly all of our UI components. We have extensive testing for the underlying simulation logic, but limited testing above that.


Restructure - Go LIVE

Pushing the new structure.

Docs Build status updates of commit 8989688:
:warning: Validation status: warnings

File
Status
Preview URL
Details

msteams-platform/bots/how-to/conversations/send-proactive-messages.md
:warning:Warning

Details

msteams-platform/whats-new.md
:warning:Warning

Details

.gitignore
:white_check_mark:Succeeded

.openpublishing.redirection.json
:white_check_mark:Succeeded

msteams-platform/.vs/ProjectSettings.json
:white_check_mark:Succeeded

msteams-platform/.vs/VSWorkspaceState.json
:white_check_mark:Succeeded

msteams-platform/.vs/msteams-platform/v16/.suo
:white_check_mark:Succeeded

msteams-platform/.vs/slnx.sqlite
:white_check_mark:Succeeded

msteams-platform/404.md
:white_check_mark:Succeeded

msteams-platform/TOC.yml
:white_check_mark:Succeeded

msteams-platform/assets/images/authentication/auth-bot-app-service-in-group.png
:white_check_mark:Succeeded

msteams-platform/assets/images/authentication/auth-bot-app-service.png
:white_check_mark:Succeeded

msteams-platform/assets/images/authentication/auth-bot-channels-registration-group.PNG
:white_check_mark:Succeeded

msteams-platform/assets/images/authentication/auth-bot-channels-registration.PNG
:white_check_mark:Succeeded

msteams-platform/assets/images/authentication/auth-bot-connect-to-teams-channel.png
:white_check_mark:Succeeded

msteams-platform/assets/images/authentication/auth-bot-connection-test-accept.PNG
:white_check_mark:Succeeded

msteams-platform/assets/images/authentication/auth-bot-connection-test-token.PNG
:white_check_mark:Succeeded

msteams-platform/assets/images/authentication/auth-bot-deployed-logout.PNG
:white_check_mark:Succeeded

msteams-platform/assets/images/authentication/auth-bot-identity-connection-adv1.PNG
:white_check_mark:Succeeded

msteams-platform/assets/images/authentication/auth-bot-login-deployed-token.PNG
:white_check_mark:Succeeded

msteams-platform/assets/images/authentication/auth-bot-login-deployed.PNG
:white_check_mark:Succeeded

msteams-platform/assets/images/authentication/auth-bot-login-emulator-token.png
:white_check_mark:Succeeded

msteams-platform/assets/images/authentication/auth-bot-login-emulator.PNG
:white_check_mark:Succeeded

msteams-platform/assets/images/authentication/auth-bot-ngrok-start.PNG
:white_check_mark:Succeeded

msteams-platform/assets/images/authentication/auth-bot-teams-ngrok-testing.png
:white_check_mark:Succeeded

msteams-platform/assets/images/authentication/auth-bot-teams-upload.png
:white_check_mark:Succeeded

msteams-platform/assets/images/bots/bot-file-upload-permission-card.png
:white_check_mark:Succeeded

msteams-platform/assets/images/desktop-client.png
:white_check_mark:Succeeded

msteams-platform/assets/images/expert-user.png
:white_check_mark:Succeeded

msteams-platform/assets/images/link-unfurling.PNG
:white_check_mark:Succeeded

msteams-platform/assets/images/messaging-extension-invoke-locations.png
:white_check_mark:Succeeded

msteams-platform/assets/images/microsoft-teams-admin-center-screenshot.PNG
:white_check_mark:Succeeded

msteams-platform/assets/images/search-extension.png
:white_check_mark:Succeeded

msteams-platform/assets/images/tab-images/configurationPage.PNG
:white_check_mark:Succeeded

msteams-platform/assets/images/tab-images/homePage.PNG
:white_check_mark:Succeeded

msteams-platform/assets/images/tab-images/personalTab.PNG
:white_check_mark:Succeeded

msteams-platform/assets/images/tab-images/storeApp.png
:white_check_mark:Succeeded

msteams-platform/assets/images/tab-images/teamsTabScreenshot.PNG
:white_check_mark:Succeeded

msteams-platform/assets/images/tab-images/workloads.PNG
:white_check_mark:Succeeded

msteams-platform/assets/images/task-module.png
:white_check_mark:Succeeded

msteams-platform/assets/yeoman-demo.gif
:white_check_mark:Succeeded

msteams-platform/assets/yeoman-images/teams-connector-1.png
:white_check_mark:Succeeded

msteams-platform/assets/yeoman-images/teams-connector-10.png
:white_check_mark:Succeeded

msteams-platform/assets/yeoman-images/teams-connector-11.png
:white_check_mark:Succeeded

msteams-platform/assets/yeoman-images/teams-connector-2.png
:white_check_mark:Succeeded

msteams-platform/assets/yeoman-images/teams-connector-3.png
:white_check_mark:Succeeded

msteams-platform/assets/yeoman-images/teams-connector-4.png
:white_check_mark:Succeeded

msteams-platform/assets/yeoman-images/teams-connector-5.png
:white_check_mark:Succeeded

msteams-platform/assets/yeoman-images/teams-connector-6.png
:white_check_mark:Succeeded

msteams-platform/assets/yeoman-images/teams-connector-7.png
:white_check_mark:Succeeded

This comment lists only the first 50 files in the pull request.

msteams-platform/bots/how-to/conversations/send-proactive-messages.md

Line 55: [Warning] Invalid link: 'fetching the team roster'. The file bots/how-to/get-teams-context.md doesn't contain a bookmark named 'fetching-the-team-roster'.
Line 57: [Warning] Invalid link: '@mentioned in a channel conversation'. The file bots/how-to/conversations/channel-and-group-conversations.md doesn't contain a bookmark named '-mentions'.
Line 58: [Warning] Invalid link: 'receive the conversationUpdate'. The file bots/how-to/conversations/subscribe-to-conversation-events.md doesn't contain a bookmark named 'team-member-or-bot-addition'.

msteams-platform/whats-new.md

Line 16: [Warning] Invalid link: 'Initiate actions with messaging extensions'. The file concepts/messaging-extensions/create-extensions.md doesn't contain a bookmark named 'request-to-install-your-conversational-bot'.
Line 17: [Warning] Invalid link: 'Get context for your tab'. The file concepts/tabs/tabs-context.md doesn't contain a bookmark named 'retrieving-context-in-private-channels'.
Line 19: [Warning] Invalid link: 'Respond with bot message from task module'. The file concepts/messaging-extensions/create-extensions.md doesn't contain a bookmark named 'respond-with-an-adaptive-card-message-sent-from-a-bot'.
Line 22: [Warning] Invalid link: 'Message Actions'. The file concepts/messaging-extensions/create-extensions.md doesn't contain a bookmark named 'action-type-message-extensions'.
Line 23: [Warning] Invalid link: 'Link unfurling'. The file concepts/messaging-extensions/search-extensions.md doesn't contain a bookmark named 'receive-requests-from-links-inserted-into-the-compose-message-box'.
Line 27: [Warning] Invalid link: 'Deep linking to a chat'. The file concepts/deep-links.md doesn't contain a bookmark named 'deep-linking-to-a-chat'.
Line 29: [Warning] Invalid file link:(~/scenarios/limited-tenant.md).
Line 32: [Warning] Invalid link: 'Deep linking to a chat'. The file concepts/deep-links.md doesn't contain a bookmark named 'deep-linking-to-a-chat'.
Line 37: [Warning] Invalid file link:(~/scenarios/lob-scenarios-landing-page.md).
Line 38: [Warning] Invalid link: 'Configuration page height'. The file resources/design/framework/tabs.md doesn't contain a bookmark named 'configuration-page-height'.
Line 39: [Warning] Invalid link: 'Adaptive card'. The file concepts/Cards/cards-reference.md doesn't contain a bookmark named 'adaptive-card'.
Line 39: [Warning] Invalid link: 'Adaptive card actions in Teams'. The file concepts/Cards/cards-actions.md doesn't contain a bookmark named 'adaptive-cards-actions'.
Line 45: [Warning] Invalid link: 'Updating your app'. The file publishing/apps-publish.md doesn't contain a bookmark named 'updating-your-app'.
Line 46: [Warning] Invalid link: 'The tab configuration page is significantly taller'. The file resources/design/framework/tabs.md doesn't contain a bookmark named 'configuration-page-height'.
Line 47: [Warning] Invalid file link:(~/troubleshoot/faq.md#teams-apps-and-authentication-with-guest-users-and-microsoft-teams-free).
Line 49: [Warning] Invalid link: 'Featured code samples'. The file samples/code-samples.md doesn't contain a bookmark named 'featured'.
Line 50: [Warning] Invalid link: 'Adaptive card'. The file concepts/Cards/cards-reference.md doesn't contain a bookmark named 'adaptive-card'.
Line 50: [Warning] Invalid link: 'Adaptive card actions in Teams'. The file concepts/Cards/cards-actions.md doesn't contain a bookmark named 'adaptive-cards-actions'.
Line 34: [Suggestion] Link 'https://docs.microsoft.com/en-us/adaptive-cards/' contains locale code 'en-us'. For localizability, remove 'en-us' from links to most Microsoft sites. NOTE: This Suggestion will become a Warning on 12/20/2019.

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

Docs Build status updates of commit 8a14786:
:warning: Validation status: warnings

File
Status
Preview URL
Details

msteams-platform/bots/how-to/conversations/send-proactive-messages.md
:warning:Warning

Details

msteams-platform/whats-new.md
:warning:Warning

Details

.gitignore
:white_check_mark:Succeeded

.openpublishing.redirection.json
:white_check_mark:Succeeded

msteams-platform/.vs/ProjectSettings.json
:white_check_mark:Succeeded

msteams-platform/.vs/VSWorkspaceState.json
:white_check_mark:Succeeded

msteams-platform/.vs/msteams-platform/v16/.suo
:white_check_mark:Succeeded

msteams-platform/.vs/slnx.sqlite
:white_check_mark:Succeeded

msteams-platform/404.md
:white_check_mark:Succeeded

msteams-platform/TOC.yml
:white_check_mark:Succeeded

msteams-platform/assets/images/authentication/auth-bot-app-service-in-group.png
:white_check_mark:Succeeded

msteams-platform/assets/images/authentication/auth-bot-app-service.png
:white_check_mark:Succeeded

msteams-platform/assets/images/authentication/auth-bot-channels-registration-group.PNG
:white_check_mark:Succeeded

msteams-platform/assets/images/authentication/auth-bot-channels-registration.PNG
:white_check_mark:Succeeded

msteams-platform/assets/images/authentication/auth-bot-connect-to-teams-channel.png
:white_check_mark:Succeeded

msteams-platform/assets/images/authentication/auth-bot-connection-test-accept.PNG
:white_check_mark:Succeeded

msteams-platform/assets/images/authentication/auth-bot-connection-test-token.PNG
:white_check_mark:Succeeded

msteams-platform/assets/images/authentication/auth-bot-deployed-logout.PNG
:white_check_mark:Succeeded

msteams-platform/assets/images/authentication/auth-bot-identity-connection-adv1.PNG
:white_check_mark:Succeeded

msteams-platform/assets/images/authentication/auth-bot-login-deployed-token.PNG
:white_check_mark:Succeeded

msteams-platform/assets/images/authentication/auth-bot-login-deployed.PNG
:white_check_mark:Succeeded

msteams-platform/assets/images/authentication/auth-bot-login-emulator-token.png
:white_check_mark:Succeeded

msteams-platform/assets/images/authentication/auth-bot-login-emulator.PNG
:white_check_mark:Succeeded

msteams-platform/assets/images/authentication/auth-bot-ngrok-start.PNG
:white_check_mark:Succeeded

msteams-platform/assets/images/authentication/auth-bot-teams-ngrok-testing.png
:white_check_mark:Succeeded

msteams-platform/assets/images/authentication/auth-bot-teams-upload.png
:white_check_mark:Succeeded

msteams-platform/assets/images/bots/bot-file-upload-permission-card.png
:white_check_mark:Succeeded

msteams-platform/assets/images/desktop-client.png
:white_check_mark:Succeeded

msteams-platform/assets/images/expert-user.png
:white_check_mark:Succeeded

msteams-platform/assets/images/link-unfurling.PNG
:white_check_mark:Succeeded

msteams-platform/assets/images/messaging-extension-invoke-locations.png
:white_check_mark:Succeeded

msteams-platform/assets/images/microsoft-teams-admin-center-screenshot.PNG
:white_check_mark:Succeeded

msteams-platform/assets/images/search-extension.png
:white_check_mark:Succeeded

msteams-platform/assets/images/tab-images/configurationPage.PNG
:white_check_mark:Succeeded

msteams-platform/assets/images/tab-images/homePage.PNG
:white_check_mark:Succeeded

msteams-platform/assets/images/tab-images/personalTab.PNG
:white_check_mark:Succeeded

msteams-platform/assets/images/tab-images/storeApp.png
:white_check_mark:Succeeded

msteams-platform/assets/images/tab-images/teamsTabScreenshot.PNG
:white_check_mark:Succeeded

msteams-platform/assets/images/tab-images/workloads.PNG
:white_check_mark:Succeeded

msteams-platform/assets/images/task-module.png
:white_check_mark:Succeeded

msteams-platform/assets/yeoman-demo.gif
:white_check_mark:Succeeded

msteams-platform/assets/yeoman-images/teams-connector-1.png
:white_check_mark:Succeeded

msteams-platform/assets/yeoman-images/teams-connector-10.png
:white_check_mark:Succeeded

msteams-platform/assets/yeoman-images/teams-connector-11.png
:white_check_mark:Succeeded

msteams-platform/assets/yeoman-images/teams-connector-2.png
:white_check_mark:Succeeded

msteams-platform/assets/yeoman-images/teams-connector-3.png
:white_check_mark:Succeeded

msteams-platform/assets/yeoman-images/teams-connector-4.png
:white_check_mark:Succeeded

msteams-platform/assets/yeoman-images/teams-connector-5.png
:white_check_mark:Succeeded

msteams-platform/assets/yeoman-images/teams-connector-6.png
:white_check_mark:Succeeded

msteams-platform/assets/yeoman-images/teams-connector-7.png
:white_check_mark:Succeeded

This comment lists only the first 50 files in the pull request.

msteams-platform/bots/how-to/conversations/send-proactive-messages.md

Line 55: [Warning] Invalid link: 'fetching the team roster'. The file bots/how-to/get-teams-context.md doesn't contain a bookmark named 'fetching-the-team-roster'.
Line 57: [Warning] Invalid link: '@mentioned in a channel conversation'. The file bots/how-to/conversations/channel-and-group-conversations.md doesn't contain a bookmark named '-mentions'.
Line 58: [Warning] Invalid link: 'receive the conversationUpdate'. The file bots/how-to/conversations/subscribe-to-conversation-events.md doesn't contain a bookmark named 'team-member-or-bot-addition'.

msteams-platform/whats-new.md

Line 16: [Warning] Invalid link: 'Initiate actions with messaging extensions'. The file concepts/messaging-extensions/create-extensions.md doesn't contain a bookmark named 'request-to-install-your-conversational-bot'.
Line 17: [Warning] Invalid link: 'Get context for your tab'. The file concepts/tabs/tabs-context.md doesn't contain a bookmark named 'retrieving-context-in-private-channels'.
Line 19: [Warning] Invalid link: 'Respond with bot message from task module'. The file concepts/messaging-extensions/create-extensions.md doesn't contain a bookmark named 'respond-with-an-adaptive-card-message-sent-from-a-bot'.
Line 22: [Warning] Invalid link: 'Message Actions'. The file concepts/messaging-extensions/create-extensions.md doesn't contain a bookmark named 'action-type-message-extensions'.
Line 23: [Warning] Invalid link: 'Link unfurling'. The file concepts/messaging-extensions/search-extensions.md doesn't contain a bookmark named 'receive-requests-from-links-inserted-into-the-compose-message-box'.
Line 27: [Warning] Invalid link: 'Deep linking to a chat'. The file concepts/deep-links.md doesn't contain a bookmark named 'deep-linking-to-a-chat'.
Line 29: [Warning] Invalid file link:(~/scenarios/limited-tenant.md).
Line 32: [Warning] Invalid link: 'Deep linking to a chat'. The file concepts/deep-links.md doesn't contain a bookmark named 'deep-linking-to-a-chat'.
Line 37: [Warning] Invalid file link:(~/scenarios/lob-scenarios-landing-page.md).
Line 38: [Warning] Invalid link: 'Configuration page height'. The file resources/design/framework/tabs.md doesn't contain a bookmark named 'configuration-page-height'.
Line 39: [Warning] Invalid link: 'Adaptive card'. The file concepts/Cards/cards-reference.md doesn't contain a bookmark named 'adaptive-card'.
Line 39: [Warning] Invalid link: 'Adaptive card actions in Teams'. The file concepts/Cards/cards-actions.md doesn't contain a bookmark named 'adaptive-cards-actions'.
Line 45: [Warning] Invalid link: 'Updating your app'. The file publishing/apps-publish.md doesn't contain a bookmark named 'updating-your-app'.
Line 46: [Warning] Invalid link: 'The tab configuration page is significantly taller'. The file resources/design/framework/tabs.md doesn't contain a bookmark named 'configuration-page-height'.
Line 47: [Warning] Invalid file link:(~/troubleshoot/faq.md#teams-apps-and-authentication-with-guest-users-and-microsoft-teams-free).
Line 49: [Warning] Invalid link: 'Featured code samples'. The file samples/code-samples.md doesn't contain a bookmark named 'featured'.
Line 50: [Warning] Invalid link: 'Adaptive card'. The file concepts/Cards/cards-reference.md doesn't contain a bookmark named 'adaptive-card'.
Line 50: [Warning] Invalid link: 'Adaptive card actions in Teams'. The file concepts/Cards/cards-actions.md doesn't contain a bookmark named 'adaptive-cards-actions'.
Line 34: [Suggestion] Link 'https://docs.microsoft.com/en-us/adaptive-cards/' contains locale code 'en-us'. For localizability, remove 'en-us' from links to most Microsoft sites. NOTE: This Suggestion will become a Warning on 12/20/2019.

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Draft: Improve line rendering

This PR is an intent to improve line rendering by

Trying to have an extra (offset, (dash1, dash2)) sypported type instead of hardcoded
Trying to fix line width between backend
Trying to have consistent linestyle between backend
Trying to have consistent linestyle between curve/shape

An example was added to simplify the checks

I have tried to to do as much as i can, but i can't finalise that

[ ] (1) In opengl the period looks hardcoded, maybe there is a missing thing in the gl program
[ ] (2) In opengl Curve linestyle have twice the period of Shape linestyle
[ ] The hardcoded offset +0.33 is probably not at the right location
[ ] Was not tested with screen with different pixel densities

Changelog:

This PR have to be cherry picked in small commits

closing this PR since its proposition has been reused and reworked in other PRs.


Derive Reflect for Exposure

Objective

Don't crash when loading a scene with a camera

Solution

Derive Reflect for Exposure

Closes https://github.com/bevyengine/bevy/issues/11905

Merging as trivial, thank you!


Unable to exclude from coverage

I cloned the example from here: https://github.com/bahmutov/testing-react
I added nyc to the package.json:
{
    ...
    "scripts": {
        ...
        "start-dev": "react-scripts -r @cypress/instrument-cra start"
        ...
    },
    ...
    "nyc": {
        "exclude": "src/constants/actionTypes.js"
    }
}

However I'm still seeing this file in window.__coverage__:
/.../testing-react/src/constants/actionTypes.js: {path: "/.../testing-react/src/constants/actionTypes.js", statementMap: {…}, fnMap: {…}, branchMap: {…}, s: {…}, …}

Am I missing something?

Hi @wdm954
You can still have extra covered files in the window.__coverage__ file, but they will be filtered when the report is generated


Cannot override packages

I have added a package file private/amos/packages.el like this,
;; -*- no-byte-compile: t; -*-
;;; private/amos/packages.el
(package! centered-window-mode :recipe (:fetcher github :repo "amosbird/centered-window-mode"))

However it still uses the original package.
Doing make update fails without verbose error messages.
Looking for outdated packages...
1 packages will be updated:

+ centered-window-mode   20170528.25    -> 20171005.1538

Proceed? (y or n) y
Updating centered-window-mode
  FAILED
Finished!
Reloading...
Recompiling
Recompiled 0/234 file(s) (0 ignored)
Compiling /home/doom/.emacs.d/core/core.el...

Ah, the package was already installed by another module (installed with package.el). You recipe is trying to install it with quelpa and fails because it can't tell where the existing package was installed from.
I'll look into a fix for this over the weekend. For the time being you can fix this by manually deleting ~/.emacs.d/.local/packages/elpa/centered-window-mode*, then running make install again.

A solution for this is now in develop. It needs a little more testing, but should make its way into master within the next couple of days.

This is now in master.

Hi, I still cannot replace elpa package with quelpa.
 ❯ make install                                                                              (4s 401ms)
1 packages will be installed:

+ dired-k (ELPA -> QUELPA)

y
Installing dired-k
  ALREADY INSTALLED
Proceed? (y or n)


Beautiful authorization window

Input labels are reduced to their normal size.
Grouped to avoid long strips, as well as to place them in the center of the window even after resizing.
Now authorization window has an icon.

норм


Sync data to Elastic Search via ABC would hit error when delete or edit data in PostgreSQL

abc version
Version:        0.6.6
Variant:         !oss
Go version:   go1.9
OS:               windows
Arch:            amd64
Hi, I've been successfully using ABC to sync data to ES when insert a record, but the error shows when delete or edit data.
←[31mERRO←[0m[0015] elastic: Error 400 (Bad Request): Validation Failed: 1: id is missing;[type=action_request_validation_exception]  ←[31mversion←[0m=5 ←[31mwriter←[0m=elasticsearch
And btw, does ABC support sync DB views with Elastic Search?
Checks

[x] I have mentioned my abc version in the issue, if applicable.
[x] This issue is not a duplicate.
[x] This issue has a description.

Hello Amber, in order to tail the changes in postgres db we need to explicitly provide _id for the doc that gets indexed to elasticsearch, otherwise it will be autogenerated and the next time we want to update or delete that doc we would not have its _id to do so and we would subsequently encounter the error you did above. One solution would be to use the rename transform function that abc provides. For example, for a postgres relation we would be indexing individual rows as separate docs in elasticsearch, and it would be preferrable to use the PRIMARY KEY for the relation as the doc _id. I believe, the transform file, assuming transform_file.js, would look something like this:
t.Source("source", source, "/.*/")
  .Transform(rename({ field_map: { <primary_key_field>: "_id" } }))
  .Save("sink", sink, "/.*/");

and using the same transform file, the abc command would look something like:
abc import --src_type=postgres --src_uri="postgresql://<user>:<password>@localhost:5432/<db>" --transform_file="transform_file.js"<EMAIL_ADDRESS>If you need more help regarding transforms, check out this link.


Implement Percentage support for gap styles

Summary:
Changelog [Internal]:

Added percentage value for flex layout gap
Wired up to pass proper available width and height to implement this feature

Differential Revision: D56002340

We have some tests for this in Taffy's test suite (search for gap_percentage here: https://github.com/DioxusLabs/taffy/tree/main/test_fixtures/flex). In particular you may want to add tests for the case where the container's size isn't a fixed size, which can require re-resolving the gap size once the container size is computed.

We have some tests for this in Taffy's test suite (search for gap_percentage here: https://github.com/DioxusLabs/taffy/tree/main/test_fixtures/flex). In particular you may want to add tests for the case where the container's size isn't a fixed size, which can require re-resolving the gap size once the container size is computed.

Thank you so much for the feedback on this. I am looking at how to improve our test cases as well as visualization aspect of it.


make HELM_HOME portable

allow cacheFile to be relative for local index
uses a symlink with a relative path on unix
final (part4) piece of https://github.com/kubernetes/helm/pull/3327

depends on part 2: #3596

part 2 obsoleted by https://github.com/kubernetes/helm/pull/3613
this now depends on part 3 for the "path/filepath" import

Closing this PR due to inactivity. Helm 3 now follows the XDG base directory standard so the issue this PR is trying to address for Helm 2 is no longer relevant in Helm 3. As per the announcement on the mailing list, Helm 2.15.0 is the last release where we will be merging new features for Helm 2. Thanks!


Add in support for casting binary to string

This is needed to be able to accelerate zordering in delta lake open source.

build

build


Fixes WB-17620.
Made release-core.yml actually work. We use the macOS-14 runner to build the ARM64 wheel including the AppleStats binary. We don't build AppleStats for x86_64 because delocate fails (see comment in release-core.yml): this means that Apple GPU metrics are missing when running on x86_64.
I updated the job to use actions/setup-go and actions/setup-py to speed things up on macOS and Windows, but this is not possible on Linux (see comment).
I also replaced the "WrapBdistWheel" hack in setup.py by a more concise hack and added a comment for any poor soul condemned to deal with this mess.
Things that are bad and out of our control:

Architecture names are not standardized (aarch64 == arm64, x86_64 == amd64), different tools and systems use different names for the same stuff
setuptools seems to have no natural way to declare that a wheel is platform-specific and not pure; we have to hack around it
cibuildwheel builds some wheels using x86_64 Python via Rosetta on ARM64 macOS, causing platform.machine() to not give us the target architecture
cibuildwheel works differently for Linux than for Windows/macOS, forcing the configurations to be different (on macOS/Windows we can use GitHub actions for setting up Go)
The Swift compiler produces a binary that references nonexistent libraries (/usr/lib/swift/libswiftCore.dylib and so on), but the binary still works (?!?!). But when compiling an x86_64 binary, it references those libraries via the "@rpath/" mechanism rather than "/usr/lib/..." like on ARM64. This breaks the "delocate" step in cibuildwheel: on ARM64 it simply ignores /usr/lib/... references (that don't exist), but on x86_64 it tries to find them and complains that it can't.

Testing
See successful run: https://github.com/wandb/wandb/actions/runs/8118682668/job/22193382223
Go to the action summary and download the artifacts at the bottom. The artifacts include all built wheels; pip install the appropriate one for your machine and run some wandb code.
Further, you can use the file command to see the architecture a binary is built for. Unzip any .whl file (they're just .zip files), and run file wandb-core and file AppleStats to see their architectures. In particular, the arm64 macOS wheels correctly contain arm64 binaries, and the x86_64 macOS wheels contain x86_64 binaries.

i think we can do a follow up, just add some simple tests to see that we built everything properly: https://cibuildwheel.readthedocs.io/en/stable/options/#testing


problem when running "./vcpkg install tesseract:x64-windows"

There was an issue running "./vcpkg install tesseract:x64-windows"
This should be included in my report:
Package: opengl:x64-windows
Vcpkg version: 2019.09.12-nohash
and this is the complete output of my "./vcpkg install tesseract:x64-windows" command:
The following packages will be built and installed:

giflib[core]:x64-windows
leptonica[core]:x64-windows
libjpeg-turbo[core]:x64-windows
liblzma[core]:x64-windows
libpng[core]:x64-windows
libwebp[core]:x64-windows
opengl[core]:x64-windows
tesseract[core]:x64-windows
tiff[core]:x64-windows
zlib[core]:x64-windows
Additional packages (*) will be modified to complete this operation.
Starting package 1/10: zlib:x64-windows
Building package zlib[core]:x64-windows...
A suitable version of git was not found (required v2.21.0). Downloading portable git v2.21.0...
Extracting git...
-- Downloading http://www.zlib.net/zlib-1.2.11.tar.gz...
-- Extracting source D:/INSTALL/Development/vcpkg-master/downloads/zlib1211.tar.gz
-- Applying patch cmake_dont_build_more_than_needed.patch
-- Using source at D:/INSTALL/Development/vcpkg-master/buildtrees/zlib/src/1.2.11-f690224aeb
-- Configuring x64-windows
-- Building x64-windows-dbg
-- Building x64-windows-rel
-- Installing: D:/INSTALL/Development/vcpkg-master/packages/zlib_x64-windows/share/zlib/copyright
-- Performing post-build validation
-- Performing post-build validation done
Building package zlib[core]:x64-windows... done
Installing package zlib[core]:x64-windows...
Installing package zlib[core]:x64-windows... done
Elapsed time for package zlib:x64-windows: 33.44 s
Starting package 2/10: libpng:x64-windows
Building package libpng[core]:x64-windows...
-- Downloading https://github.com/glennrp/libpng/archive/v1.6.37.tar.gz...
-- Extracting source D:/INSTALL/Development/vcpkg-master/downloads/glennrp-libpng-v1.6.37.tar.gz
-- Applying patch use-abort-on-all-platforms.patch
-- Applying patch fix-libm-unix.patch
-- Using source at D:/INSTALL/Development/vcpkg-master/buildtrees/libpng/src/v1.6.37-6445e5863e
-- Configuring x64-windows
-- Building x64-windows-dbg
-- Building x64-windows-rel
-- Performing post-build validation
-- Performing post-build validation done
Building package libpng[core]:x64-windows... done
Installing package libpng[core]:x64-windows...
Installing package libpng[core]:x64-windows... done
Elapsed time for package libpng:x64-windows: 17.66 s
Starting package 3/10: libjpeg-turbo:x64-windows
Building package libjpeg-turbo[core]:x64-windows...
-- Downloading https://github.com/libjpeg-turbo/libjpeg-turbo/archive/2.0.2.tar.gz...
-- Extracting source D:/INSTALL/Development/vcpkg-master/downloads/libjpeg-turbo-libjpeg-turbo-2.0.2.tar.gz
-- Applying patch add-options-for-exes-docs-headers.patch
-- Applying patch workaround_cmake_system_processor.patch
-- Using source at D:/INSTALL/Development/vcpkg-master/buildtrees/libjpeg-turbo/src/2.0.2-3c759cce19
-- Downloading http://www.nasm.us/pub/nasm/releasebuilds/2.14.02/win32/nasm-2.14.02-win32.zip...
-- Configuring x64-windows
-- Building x64-windows-dbg
-- Building x64-windows-rel
-- Performing post-build validation
-- Performing post-build validation done
Building package libjpeg-turbo[core]:x64-windows... done
Installing package libjpeg-turbo[core]:x64-windows...
Installing package libjpeg-turbo[core]:x64-windows... done
Elapsed time for package libjpeg-turbo:x64-windows: 32.06 s
Starting package 4/10: liblzma:x64-windows
Building package liblzma[core]:x64-windows...
-- Downloading https://github.com/xz-mirror/xz/archive/v5.2.4.tar.gz...
-- Extracting source D:/INSTALL/Development/vcpkg-master/downloads/xz-mirror-xz-v5.2.4.tar.gz
-- Applying patch enable-uwp-builds.patch
-- Using source at D:/INSTALL/Development/vcpkg-master/buildtrees/liblzma/src/v5.2.4-66bfe33102
-- Configuring x64-windows
-- Building x64-windows-dbg
-- Building x64-windows-rel
-- Installing: D:/INSTALL/Development/vcpkg-master/packages/liblzma_x64-windows/share/liblzma/copyright
-- Performing post-build validation
-- Performing post-build validation done
Building package liblzma[core]:x64-windows... done
Installing package liblzma[core]:x64-windows...
Installing package liblzma[core]:x64-windows... done
Elapsed time for package liblzma:x64-windows: 22.08 s
Starting package 5/10: tiff:x64-windows
Building package tiff[core]:x64-windows...
-- Downloading http://download.osgeo.org/libtiff/tiff-4.0.10.tar.gz...
-- Extracting source D:/INSTALL/Development/vcpkg-master/downloads/tiff-4.0.10.tar.gz
-- Applying patch fix-stddef.patch
-- Applying patch cmakelists.patch
-- Using source at D:/INSTALL/Development/vcpkg-master/buildtrees/tiff/src/4.0.10-229dd2f530
-- Configuring x64-windows
-- Building x64-windows-dbg
-- Building x64-windows-rel
-- Installing: D:/INSTALL/Development/vcpkg-master/packages/tiff_x64-windows/share/tiff/copyright
-- Performing post-build validation
-- Performing post-build validation done
Building package tiff[core]:x64-windows... done
Installing package tiff[core]:x64-windows...
Installing package tiff[core]:x64-windows... done
Elapsed time for package tiff:x64-windows: 1.006 min
Starting package 6/10: giflib:x64-windows
Building package giflib[core]:x64-windows...
-- Note: giflib only supports static library linkage. Building static library.
-- Downloading https://nchc.dl.sourceforge.net/project/giflib/giflib-5.1.4.tar.bz2...
-- Extracting source D:/INSTALL/Development/vcpkg-master/downloads/giflib-5.1.4.tar.bz2
-- Applying patch msvc-guard-unistd-h.patch
-- Applying patch fix-compile-error.patch
-- Using source at D:/INSTALL/Development/vcpkg-master/buildtrees/giflib/src/5.1.4-8e625bd21e
-- Configuring x64-windows
-- Building x64-windows-dbg
-- Building x64-windows-rel
-- Installing: D:/INSTALL/Development/vcpkg-master/packages/giflib_x64-windows/share/giflib/copyright
-- Performing post-build validation
-- Performing post-build validation done
Building package giflib[core]:x64-windows... done
Installing package giflib[core]:x64-windows...
Installing package giflib[core]:x64-windows... done
Elapsed time for package giflib:x64-windows: 13.13 s
Starting package 7/10: opengl:x64-windows
Building package opengl[core]:x64-windows...
CMake Error at ports/opengl/portfile.cmake:20 (message):
Cannot find Windows 10.0.17763.0 SDK.  File does not exist: C:\Program
Files (x86)\Windows Kits\10\Lib\10.0.17763.0\um\x64\OpenGL32.Lib
Call Stack (most recent call first):
scripts/ports.cmake:94 (include)

Error: Building package opengl:x64-windows failed with: BUILD_FAILED
Please ensure you're using the latest portfiles with .\vcpkg update, then
submit an issue at https://github.com/Microsoft/vcpkg/issues including:
Package: opengl:x64-windows
Vcpkg version: 2019.09.12-nohash
Additionally, attach any relevant sections from the log files above.

Hi @pbvelikov, the error info shows:
Cannot find Windows 10.0.17763.0 SDK. File does not exist: C:\Program
Files (x86)\Windows Kits\10\Lib\10.0.17763.0\um\x64\OpenGL32.Lib

Please install Windows 10 SDK using Visual Studio Installer first.
Thanks.

Hi,
I have this SDK installed a long time ago, but on disk D:, so the file in question is available and located here: D:\Windows Kits\10\Lib\10.0.17763.0\um\x64\OpenGL32.Lib
Why is ./vcpkg looking for it on C:?

Duplicated to  https://github.com/microsoft/vcpkg/issues/8288.


Cannot GET /

Hi.
I wanted to try este out, did the clone, npm install, gulp.
Got this:
`

gulp
[15:18:03] Requiring external module babel-core/register
[15:18:04] Using gulpfile D:\Dev\postil7\gulpfile.babel.js
[15:18:04] Starting 'env'...
[15:18:04] Finished 'env' after 104 μs
[15:18:04] Starting 'server'...
[15:18:04] Starting 'server-hot'...
[bg] Starting node ./webpack/server
[15:18:04] Finished 'server-hot' after 6.47 ms
[15:18:04] Starting 'server-nodemon'...
[nodemon] 1.9.1
[nodemon] to restart at any time, enter rs
[nodemon] watching: .
[nodemon] starting node src\server
Hot server started at port 8080
Api is available at /api/v1
App is available at /
Server started at port 8000
Hash: 4127774f0eca7e90e40b
Version: webpack 1.13.0
Time: 10782ms
Asset     Size  Chunks             Chunk Names
af9dab4d8a927671c4d54a5cb741d1d5.ico  1.15 kB          [emitted]
app.js  7.16 MB       0  [emitted]  app
webpack built 4127774f0eca7e90e40b in 10782ms
`

Then when opening http://localhost:8080/ I get:
Cannot GET /
Suggestions?

Your app is listening on port 8000.


Allow rendering charts from embedded file system (embed.FS)

How to categorize this PR?

/area dev-productivity
/kind enhancement
What this PR does / why we need it:
It is now possible to render Helm charts from embedded file systems (embed.FS) which allows to better structure the code. As a consequence, I have moved the charts/istio to pkg/operation/botanist/component/istio/charts/istio.
Which issue(s) this PR fixes:
Part of #2754
Follow-up of #5946
Special notes for your reviewer:
/cc @acumino
Release note:

It is now possible to render charts from embedded file systems (`embed.FS`). The `Render` method of the `chartrenderer.Interface` in favour of `RenderEmbeddedFS`. The `Apply`/`Delete` methods of the `kubernetes.ChartApplier` interfaces are deprecated and in favor of `{Apply,Delete}FromEmbeddedFS`. They will be removed in a future version. You should consider adapting your code to the newly introduced methods.

/assign acumino

/approve


How to use JSONB columns with SupabaseModel

Consider a table on supabase called SomeEntity with a column MyJsonbColumn types as JSONB.
We have this class:
namespace Name.Space.Models
{
    [Table("SomeEntity")]
    public class SomeEntity: SupabaseModel
    {
        [PrimaryKey("_id")]
        public int Id { get; set; }
        [Column("stringcolumn")]
        public string StringColumn { get; set; }
        [Column("status")]
        public JsonDocument Status { get; set; }  // this is not good, which type must be used?
    }
}

How can this be done?

Could you try:
[Column("status")]
 public Dictionary<string, object> Status { get; set; }

And see if that works?

Works like a charm!
Example of how two different JSONB columns look like:
[
    ...
    "data": null,  // null JSONB column
    "status": {  // boolean entries
      "active": true,
      "ignore": false
    },
    ...
]

Thanks

@fenandosr for the record, you ought to be able to create a C# class to hydrate into and use that as the type if the JSONB data is consistent.

[Column("status")]
public JsonData Status { get; set; }  // this is not good, which type must be used?

//....

public partial class JsonData
    {
        [JsonProperty("data")]
        public object Data { get; set; }

        [JsonProperty("status")]
        public Status Status { get; set; }
    }

    public partial class Status
    {
        [JsonProperty("active")]
        public bool Active { get; set; }

        [JsonProperty("ignore")]
        public bool Ignore { get; set; }
    }

Ok, I will test this and post if anything something is useful.
We appreciate your support.

In C# 9+ you can also use a record nicely:
using Postgrest.Attributes;
using Postgrest.Models;

[Table("games")]
public class Game : BaseModel
{
    [PrimaryKey]
    public string? id { get; set; }

    [Column]
    public Dictionary<string, GameState>? state { get; set; }
}

public record GameState(string color, /* etc */);


Integrate List Report into ABAP Fiori Launchpad: Execute transport task (Optional)

Tutorials: https://developers.sap.com/tutorials/abap-environment-abap-flp.html
In step 6, Execute transport task (Optional) it's mentioned to execute the transport task. Please rename "execute" to "release" the transport task.
Finally, please add another section after releasing transport task, to also release the transport request. Only if also the transport request is released, the space/page objects would be exported and could be imported e.g. in production system.

Hi @peterpersiel,
thank you for your feedback. I created follwing PR: #20764 to solve this issue.
Best regards,
Merve


Post vacation cleanup

Just noticed a couple things to clean up while I was reviewing the PRs I missed.

+1 to changes in user_functions, -1 to commitlog

Rebased the stress commit out. Once travis is happy I'll merge.


Conditional filenames with illegal characters

Bug description
Conditional file naming does not work if tag contains illegal characters in Windows e.g. : ? "
Steps to reproduce
File naming is set as %artist:spaces%/%websitename% %date:format=yyyy.MM.dd% <"re:zero_kara_hajimeru_isekai_seikatsu"?re-zero:%copyright%> %character% %artist% %id%.%ext%
Expected behavior
Image should be saved as "re-zero" as copyright tag but is saved as "re_zero_kara_hajimeru_isekai_seikatsu" instead. Using "re_zero_kara_hajimeru_isekai_seikatsu" in conditional file naming gives the same result.
Also tried on tags containing other illegal characters and got similar results.

An unrelated question but is there a way to add ratings into default file naming to determine folder name? Basically I would like to save all "questionable" and "explicit" images in a separate folder, but also set up conditionals to rename certain tags e.g. "kaguya-sama_wa_kokurasetai_~tensai-tachi_no_renai_zunousen~" to "kaguya-sama".
Edit// Never mind, had the general idea but couldn't get it to work earlier but figured it out. In case anyone's interested: %artist:spaces%/rating:explicit|rating:questionable?NSFW:%nothing%/%websitename% %date:format=yyyy.MM.dd% %copyright% %character% %artist% %id%.%ext%
On that note maybe good to have more examples in the filename documentation for people to refer to or just copy-paste.
System information

OS: Windows 10
Grabber version: 7.5.1

Indeed, that was caused by the : causing it to be treated as a meta-tag search (like rating:safe).
I just pushed a fix.


add JIRA issue tracker link

Cordova CI Build has one or more failures.
Commit     - Link
Dashboard - Link

Builder Name
Console Output
Test Report
Device Logs

Windows 8.1 Store
Link
Link
Link

Windows 10  Store
Link
Link
Link

Windows 8.1 Phone
Link
Link
Link

iOS
Link
Link
Link

Android Mac
Link
Link
Link

No more JIRA, Github issues it is 👍
Thanks for your PR anyway of course.


Unable to install

Unable to Install VSCODE server on YUNOHOST
Context

Hardware: * LINODE NANO*
YunoHost version:  <IP_ADDRESS> (stable).
I have access to my server: Through SSH | through the webadmin
Are you in a special context or did you perform some particular tweaking on your YunoHost instance?: no

Steps to reproduce
Clicking general Install button on Yunohost
Logs
https://paste.yunohost.org/raw/emoxukofoc

This app require a dedicated domain name.
As written in the log :
This URL is either unavailable, or conflicts with the already installed app(s):
 * maindomain.tld/phpsysinfo → PhpSysInfo (phpsysinfo)
 * maindomain.tld/mantis → Mantis (mantis)
 * maindomain.tld/monica → Monica (monica)
 * maindomain.tld/ihatemoney → I Hate Money (ihatemoney)
 * maindomain.tld/jupyterlab → JupyterLab (jupyterlab)
 * maindomain.tld/diagram → Diagrams.net (diagramsnet)
 * maindomain.tld/mindmaps → Mindmaps (mindmaps)
 * maindomain.tld/nodered/admin → Node-RED (nodered)
 * maindomain.tld/nodered → Node-RED (endpoints) (nodered)
 * maindomain.tld/nodered/ui → Node-RED (ui) (nodered)
 * maindomain.tld/nullboard → Nullboard (nullboard)
 * maindomain.tld/mymind → My Mind (my-mind)
 * maindomain.tld/kanboard → Kanboard (kanboard)
 * maindomain.tld/kanboard/jsonrpc.php → Kanboard (api) (kanboard)
 * maindomain.tld/nextcloud → Nextcloud (nextcloud)

You'll need to create a subdomain like code.maindomain.tld then install the app on this domain.


Send XACML policy to Authzforce through console

Hi!
We are using IdM, PEP and Authzforce for the FIWARE Devguide APP. What we are doing is to provide a set of users, organizations, applications, roles and permissions by default (to be used with Docker).
Right now we did good until here, but now we are a bit stuck at some point. We have the whole environment running but, the only thing that we still were not able to do is to send XACML policy to Authzforce from the beginning.
By default Authzforce permits all, but we would like to parse all the generated roles and permissions in our application without having to navigate through Horizon interface. Is there any way to trigger that? (we would like to avoid the step of going to IdM - Application - Manage Roles - Save)
We've tried ourselves re-writing the function handle from /horizon/openstack_dashboard/dashboards/idm/myApplications/workflows.py, but in the end we've messed with the environments and paths so we couldn't apply it in an easy way.
Any help would be appreciated! :)

Hi, you have to use the AuthZForce API directly. Here you will find the documentation:
http://catalogue.fiware.org/enablers/authorization-pdp-authzforce
If you have doubts regarding the use of it please contact the GE owner (contact info included also there)
BR


Context.Done is not honored during chan sends in publishBlock

Honor context.Done to avoid endlessly locking the operation.
This might also create deadlocks.
Select {
Case <-ctx.Done:
Case HeaderCh<-value
}
See BB audit issue:
Finding 022 - Context.Done is not honored during chan sends in
publishBlock

Handled by https://github.com/rollkit/rollkit/pull/1434. Shutdown synchronization ensures that channels won't be closed before the call to publishBlock ends and AggregationLoop does a clean exit.


enhancement optimize delete statement can not do partition prune when column is not lower case

Proposed changes
optimize delete statement can not do partition prune when column is not lower case
for example, this sql will send finishRealtimePush rpc for all partitions
delete from tbl where Dt = '2024-01-01'

run buildall


Cleanup, making TPD calculations easy, commenting, split to different functions

Cleanup, indentation added configurable cycles.
Now it behaves according to the README

Hi,
I have updated the code to display also the TPD calculation for the settings you use. Do some changes also for the stepper motor, led, IDLE etc
Thanks !!! The 3D print was amazing


Remove wildcard warning as they aren't allowed in top level item groups

Description of the new feature / enhancement
Wildcards are not allowed in toplevel ItemGroups in vcxproj; they must
be generated by targets.
Terminal has possible base solution for us
https://github.com/microsoft/terminal/pull/6406
Scenario when this would be used?
removing warnings while compiling
Supporting information
No response

/dup #23459


Set the correct stream header state for h2c upgrades

Motivation:
It is possible to create streams in the half-closed state where the
stream state doesn't reflect that the request headers have been sent by
the client or the server hasn't received the request headers. This
state isn't possible in the H2 spec as a half closed stream must have
either received a full request or have received the headers from a
pushed stream. In the current implementation, this can cause the stream
created as part of an h2c upgrade request to be in this invalid state
and result in the omission of RST frames as the client doesn't believe
it has sent the request to begin with.
Modification:
The DefaultHttp2Connection.activate method checks the state and
modifies the status of the request headers as appropriate.
Result:
Fixes #7847.

This seemed easy enough to change, so I just went ahead and did it. If #7847 makes incorrect assumptions fft close this.

CC @zpencer

So I'd agree this is closer, but it seems incomplete. In the GET case, won't the client have already half-closed? That doesn't seem handled here. But when doing a streaming POST the client won't have half-closed, and then it's not clear how the stream state is updated.
Could you expand on your motivation some? Did this break your application, or did confuse some part of the rest of the HTTP/2 stack? I'm sort of assuming for now that it just broke your application.

@ejona86, this change only concerns the h2c upgrade, so my following comments should be taken in that context.
For upgrade requests, Stream 1 is implicitly "half-closed" from the client toward the server, so for all requests, not just GET requests, it's half closed. The current upgrade mechanisms (HttpServerUpgradeHandler) don't facilitate streaming an upgrade request so this doesn't attempt to address the complexity associated with that, though even in that case it would have still received the headers.
The state I'm trying to rectify is the HTTP Request/Response Exchange. For the upgrade request, the client-to-server half of the stream is already complete, so the headers have been sent by the client and received by the server when we call the respective upgrade methods. This is mostly just bookkeeping, but it is important for the onStreamError method which calls to the resetStream. Without this change, a problem with the upgrade request can cause a client to omit a RST frame here since it thinks it hasn't sent the request at all.
Why this seems weird is that the stream state and request/response state are encoded in two orthogonal ways (State state and byte metaState, respectively) even though there is a bit of coupling between them from the protocols perspective. For example, a half-closed stream will always have completed the request headers.

I misunderstood the context of this bug, and forgot the stream was already half-closed. That made me mix things together with #7657. Sorry.

Without this change, a problem with the upgrade request can cause a client to omit a RST frame here since it thinks it hasn't sent the request at all.

Ah, excellent. That sort of thing should be mentioned in "motivation".

Why this seems weird is that the stream state and request/response state are encoded in two orthogonal ways (State state and byte metaState, respectively) even though there is a bit of coupling between them from the protocols perspective. For example, a half-closed stream will always have completed the request headers.

This is good "motivation" as well. And it points out that I don't think this fix is in the correct spot. It seems this is a bug within DefaultHttp2Connection.java. The createStream(_, /*halfClosed =*/ true) should not produce a stream in such a broken state. Either that method or DefaultStream's constructor seem more fitting places for the fix.

@ejona86, I'll see about making the halfClosed = true case produce a correct state. Thanks for the feedback.

@ejona86, ptal. I'm not sure it's right: there are a lot of ways to do a state transition with a stream, so let me know if you see a problem.

@ejona86 can you check again please?


[WIP] Adding collection of docker and openshift versions

This pr counts on having ops-metrics-client, which will send metric to Hawkular and Zabbix.

@zgalor have a look.

depends on https://github.com/openshift/openshift-tools/pull/1621

@zgalor closing pr, take it over :)


How to retrieve position of handlandmarks in Android example

Hi,
In the Android Example the landmarks are rendered onto the camera output, how can I access / retrieve the information of those landmarks? I can't seem to find any example of how to actually get the data information from the detected result. I would like to get information about how big is the bounding box of the hand, and position of the landmarks etc.
A hint in the right direction is highly appreciated. Haven't worked with TF before.
Thanks in advance.

Is there an update on this? I would also like to be able to retrieve the information of the landmarks.

Waiting for an update on this. The locations of the different parts of the hand as well as the bounding box are very useful. Please help.

Duplicate of #64

@Hemanth715 @GGamer217 @Graphicscore Fixed in v0.6.6 Pls check it out and let us know


chore: enable dependabot for maven dependencies of openapi-maven-plugin

Keeping dependencies updated is usually a good practice, at least on the maven plugin itself.

Thank you. Truth to be told, I'm thinking of removing the sample angular projet and put it in a dedicated repository. It was my initial goal, but the plugin has evolved a lot since. :)


mkdir overwrites existing directories/files.

Contrary to file-based implementations of mkdir, vfsStream overwrites existing directories and files instead of triggering a warning and returning false.
Example:
<?php
require('vfsStream/vfsStream.php');
vfsStream::setup('root');
$dir = vfsStream::url('root/dir');
assert(mkdir($dir));
assert(!mkdir($dir)); // This assertion should succeed.

Thanks for the report, this bug has been fixed.

vfsStream 0.11.1 containing the bugfix has just been released.


no-return-await

Check to see that I'm not returning awaits:
https://eslint.org/docs/rules/no-return-await

Closed via ec627f70290df7d00f43b389fb78b95e5e813654


Fix broken link

Fix broken link sending to Azure App Service on Linux FAQ

@fatosmorina : Thanks for your contribution to the Azure documentation! The author, @SyntaxC4, has been notified to review your proposed change.

#sign-off


[MINOR] Modify return type description

Change Logs
[MINOR] Modify return type description
Impact
[MINOR] Modify return type description
Risk level (write none, low medium or high below)
none

@danny0405 Hey, can you help me take a look at this PR?


failed to broadcast on aws ec2

While trying to publish live streams on was gets connection error,
similar Linux version on the local machine works fine.
Error message

You need to set inbound rules.

Go to security groups on AWS console
Select the security group attached to your server
Add these inbound rules
Port: 8000 Source: <IP_ADDRESS>/0(everywhere)
Port: 8443 Source: <IP_ADDRESS>/0
Port: 1935 Source: <IP_ADDRESS>/0


DMP doesn't broadcast DataParallel ShardingType embedding table from the process with rank 0 to all other processes

DMP should broadcast DataParallel ShardingType embedding table param from the process with rank 0 to all other processes in the group to make sure that all model replicas start from the exact same state.
However, it has been observed that DMP currently designates all sharded_parameter_names as part of the params_and_buffers_to_ignore list for Distributed Data Parallel (DDP) operations. This behavior leads to a situation where DMP omits the necessary synchronization of the DataParallel ShardingType embedding table parameters during the initialization phase. As a consequence, model replicas may start from different states, which could result in inconsistent training outcomes and potentially compromise model convergence.
class DefaultDataParallelWrapper(DataParallelWrapper):
    ...

    def wrap(
        self,
        dmp: "DistributedModelParallel",
        env: ShardingEnv,
        device: torch.device,
    ) -> None:
        if isinstance(dmp._dmp_wrapped_module, DistributedDataParallel) or isinstance(
            dmp._dmp_wrapped_module, FullyShardedDataParallel
        ):
            return
        sharded_parameter_names = set(
            DistributedModelParallel._sharded_parameter_names(dmp._dmp_wrapped_module)
        )
        self._ddp_wrap(dmp, env, device, sharded_parameter_names)

    ...

class DistributedModelParallel(nn.Module, FusedOptimizerModule):
    @staticmethod
    def _sharded_parameter_names(module: nn.Module, prefix: str = "") -> Iterator[str]:
        module = get_unwrapped_module(module)
        if isinstance(module, ShardedModule):
            yield from module.sharded_parameter_names(prefix)
        else:
            for name, child in module.named_children():
                yield from DistributedModelParallel._sharded_parameter_names(
                    child, append_prefix(prefix, name)
                )

@tiankongdeguiji
DDP tensors are replicated in the state_dict that should be loaded on each rank.
So no comm_ops are necessary to broadcast them.

@IvanKobzarev
In TorchRec DMP, all parameters of ShardedModule (Including ShardingType==DataParallel) are added to the params_and_buffers_to_ignore list for DDP. This configuration prevents DDP from broadcasting these parameters. However, for proper functionality, it is crucial to ensure that parameters of the ShardedModule with ShardingType equal to DataParallel are indeed broadcasted.
// torchrec/distributed/model_parallel.py
class DefaultDataParallelWrapper(DataParallelWrapper):
    ...

    def _ddp_wrap(
        self,
        dmp: "DistributedModelParallel",
        env: ShardingEnv,
        device: torch.device,
        ddp_ignore_param_names: Set[str],
    ) -> None:
        pg = env.process_group
        if pg is None:
            raise RuntimeError("Can only init DDP for ProcessGroup-based ShardingEnv")
        all_parameter_names = set(dict(dmp.named_parameters()).keys())
        if len(all_parameter_names - ddp_ignore_param_names) == 0:
            return
        DistributedDataParallel._set_params_and_buffers_to_ignore_for_model(
            module=dmp._dmp_wrapped_module,
            params_and_buffers_to_ignore=ddp_ignore_param_names,
        )
        # initialize DDP
        dmp._dmp_wrapped_module = cast(
            nn.Module,
            DistributedDataParallel(
                module=dmp._dmp_wrapped_module.to(device),
                device_ids=None if device.type == "cpu" else [device],
                process_group=pg,
                gradient_as_bucket_view=True,
                broadcast_buffers=False,
                static_graph=self._static_graph,
                find_unused_parameters=self._find_unused_parameters,
                bucket_cap_mb=self._bucket_cap_mb,
            ),
        )
        if self._allreduce_comm_precision == "fp16":
            dmp._dmp_wrapped_module.register_comm_hook(
                None, ddp_default_hooks.fp16_compress_hook
            )
        elif self._allreduce_comm_precision == "bf16":
            dmp._dmp_wrapped_module.register_comm_hook(
                None, ddp_default_hooks.bf16_compress_hook
            )

    def wrap(
        self,
        dmp: "DistributedModelParallel",
        env: ShardingEnv,
        device: torch.device,
    ) -> None:
        if isinstance(dmp._dmp_wrapped_module, DistributedDataParallel) or isinstance(
            dmp._dmp_wrapped_module, FullyShardedDataParallel
        ):
            return
        sharded_parameter_names = set(
            DistributedModelParallel._sharded_parameter_names(dmp._dmp_wrapped_module)
        )
        self._ddp_wrap(dmp, env, device, sharded_parameter_names)

Hi, @henrylhtsang @IvanKobzarev @joshuadeng

@tiankongdeguiji
I suspect this isn't really a problem. I tested it with the NCCL model_parallel test_sharding_dp by printing the state_dict out, and found them to be the same.
I suspect it is working due to this hack
https://github.com/pytorch/torchrec/blob/main/torchrec/distributed/embeddingbag.py?fbclid=IwAR0LKttb3ZOvOhIh1lAfnbO7hUR16YrPp9kPfZSM3WW4pHvv800z0G3a718#L499-L500

@henrylhtsang yes, that is where DDP modules are set up (using actual DDP) to make these data_parallel tables call all_reduce to get the correct gradients.
Why do you call this part a hack?
@tiankongdeguiji , is your concern that all_reduce won't be called during training? or is your concern that restoring from checkpoint will be incorrect?

@henrylhtsang
In the test_sharding_dp function, the state_dict of global_model is copied to local_model, and the global_model is initialized using torch.manual_seed(0). This ensures  that all model replicas start from identical initial parameters.
https://github.com/pytorch/torchrec/blob/main/torchrec/distributed/test_utils/test_sharding.py#L376
However, it is important to note that without performing this state_dict copying at the beginning of training, as is typically done, the DataParallel ShardingType embedding table parameters in different replicas would start from differing states.

@tiankongdeguiji can you try to inspect the state dict of local model right after local_model = DistributedModelParallel(, ie before the copy_state_dict?
When I ran it, it showed those parameters are the same.

@henrylhtsang
Can you run torchrun --master_addr=localhost --master_port=54926 --nnodes=1 --nproc-per-node=2 --node_rank=0 debug_dp_shard.py,  and use the enviroment torchrec==0.6.0+cu121, torch==2.2.0+cu121, fbgemm-gpu==0.6.0+cu121?
debug_dp_shard.py
import os
from typing import Dict, cast

import torch
import torch.distributed as dist
import torchrec
from torch import nn
from torchrec import EmbeddingBagCollection
from torchrec.distributed.embeddingbag import EmbeddingBagCollectionSharder
from torchrec.distributed.model_parallel import (
    DistributedModelParallel,
)
from torchrec.distributed.planner import EmbeddingShardingPlanner, Topology
from torchrec.distributed.planner.types import ParameterConstraints
from torchrec.distributed.types import ModuleSharder, ShardingType
from torchrec.optim import optimizers
from torchrec.optim.apply_optimizer_in_backward import apply_optimizer_in_backward
from torchrec.optim.keyed import CombinedOptimizer, KeyedOptimizerWrapper
from torchrec.optim.optimizers import in_backward_optimizer_filter
from torchrec.sparse.jagged_tensor import KeyedJaggedTensor

large_table_cnt = 2
small_table_cnt = 2
large_tables = [
    torchrec.EmbeddingBagConfig(
        name="large_table_" + str(i),
        embedding_dim=64,
        num_embeddings=4096,
        feature_names=["large_table_feature_" + str(i)],
        pooling=torchrec.PoolingType.SUM,
    )
    for i in range(large_table_cnt)
]
small_tables = [
    torchrec.EmbeddingBagConfig(
        name="small_table_" + str(i),
        embedding_dim=64,
        num_embeddings=1024,
        feature_names=["small_table_feature_" + str(i)],
        pooling=torchrec.PoolingType.SUM,
    )
    for i in range(small_table_cnt)
]

def gen_constraints(
    sharding_type: ShardingType = ShardingType.DATA_PARALLEL,
) -> Dict[str, ParameterConstraints]:
    large_table_constraints = {
        "large_table_" + str(i): ParameterConstraints(
            sharding_types=[sharding_type.value],
        )
        for i in range(large_table_cnt)
    }
    small_table_constraints = {
        "small_table_" + str(i): ParameterConstraints(
            sharding_types=[sharding_type.value],
        )
        for i in range(small_table_cnt)
    }
    constraints = {**large_table_constraints, **small_table_constraints}
    return constraints

class DebugModel(nn.Module):
    def __init__(self):
        super().__init__()
        self.ebc = EmbeddingBagCollection(tables=large_tables + small_tables, device="meta")
        self.linear = nn.Linear(64 * (small_table_cnt + large_table_cnt), 1)

    def forward(self, kjt: KeyedJaggedTensor):
        emb = self.ebc(kjt)
        return torch.mean(self.linear(emb.values()))

rank = int(os.environ["RANK"])
if torch.cuda.is_available():
    device = torch.device(f"cuda:{rank}")
    backend = "nccl"
    torch.cuda.set_device(device)
else:
    device = torch.device("cpu")
    backend = "gloo"
dist.init_process_group(backend=backend)
world_size = dist.get_world_size()
print("world_size:", world_size)

model = DebugModel()
apply_optimizer_in_backward(optimizers.Adagrad, model.ebc.parameters(), {"lr": 0.001})

topology = Topology(world_size=world_size, compute_device=device.type)
constraints = gen_constraints(ShardingType.DATA_PARALLEL)
planner = EmbeddingShardingPlanner(
    topology=topology,
    constraints=constraints,
)
sharders = [cast(ModuleSharder[torch.nn.Module], EmbeddingBagCollectionSharder())]
plan = planner.collective_plan(model, sharders, dist.GroupMember.WORLD)

sharded_model = DistributedModelParallel(
    model,
    plan=plan,
    sharders=sharders,
    device=device,
)
dense_optimizer = KeyedOptimizerWrapper(
    dict(in_backward_optimizer_filter(sharded_model.named_parameters())),
    lambda params: torch.optim.Adam(params, lr=0.001),
)
optimizer = CombinedOptimizer([sharded_model.fused_optimizer, dense_optimizer])
print(f"rank:{rank},sharding plan: {plan}")

batch_size = 64
kjt = KeyedJaggedTensor(
    keys=["large_table_feature_" + str(i) for i in range(large_table_cnt)]
    + ["small_table_feature_" + str(i) for i in range(small_table_cnt)],
    values=torch.cat([
        torch.randint(0, 4096, (batch_size * 2,))
        , torch.randint(0, 1023, (batch_size * 2,))]
    ),
    lengths=torch.ones(batch_size * (small_table_cnt + large_table_cnt), dtype=torch.int32),
).to(device=device)
losses = sharded_model.forward(kjt)
torch.sum(losses, dim=0).backward()
optimizer.step()

dist.barrier()
for k, v in sharded_model.named_parameters():
    if 'ebc' in k:
        t_list = [torch.zeros_like(v) for _ in range(world_size)]
        dist.all_gather(t_list, v)
        if rank == 0:
            print(k, t_list[0].equal(t_list[1]))

It will print the following log which indicates ebc parameters are not the same.
ebc.embedding_bags.large_table_0.weight False
ebc.embedding_bags.large_table_1.weight False
ebc.embedding_bags.small_table_0.weight False
ebc.embedding_bags.small_table_1.weight False

@tiankongdeguiji Okay I think you are 100% right. Sorry I didn't understand your point on the torch seed part.
I looked into it. A few points:

The problem isn't with training. It seems like they are different at the initialization.
The "hack" is working perfectly. If you print self._lookups[index].state_dict() right after that point, they are the same.
What I think the problem is is self._initialize_torch_state(), which initialize the tables separately. I tested putting torch.manual_seed(0) before the self._initialize_torch_state() line, and your code are printing 4 Trues. Please test this and report back to see if this works.

Followups:

Fix the problem.
Remove torch.manual_seed in tests

@henrylhtsang Yes, it works. However, I think it's better to broadcast these parameters on rank 0 to other ranks, like DDP.

@tiankongdeguiji fyi I raised the issue to the team already. Probably will wait a bit.
On the other hand, any suggestions on how to fix this in a nice way? Maybe remove the names from _sharded_parameter_names?

@henrylhtsang We are unable to remove the names from _sharded_parameter_names, as the dist.Reducer within DDP is incapable of managing the parameters associated with the DataParallel ShardingType embedding table. At present, I invoke dist._broadcast_coalesced for these parameters following DMP.

@tiankongdeguiji fyi landed the fix https://github.com/pytorch/torchrec/commit/cc482f8a5f80fd8975de82ad22b65cda3348d872
Basically every time we call reset_parameters, we will also broadcast the re-initialized DP tables from rank 0 to all other ranks
Though be sure to not forget to apply optimizer to DP tables

Thanks

@tiankongdeguiji fyi landed the fix cc482f8
Basically every time we call reset_parameters, we will also broadcast the re-initialized DP tables from rank 0 to all other ranks
Though be sure to not forget to apply optimizer to DP tables


upstart + npm run-script (npm ERR! cb() never called!)

logs contains this
npm ERR! cb() never called!
npm ERR!

each time i try to use upstart to start my service with npm
#!upstart
description "service-name"
author      "kilian ciuffolo"

start on startup
stop on shutdown

respawn
respawn limit 5 60

script
    export PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games"
    npm run-script /root/server start-service >> /var/log/service/hub.log 2>&1
end script
```

node v0.4.12
npm 1.0.103

Was an npm-debug.log file created in the cwd?

nope :(

I'm running some tests between v1.0.101 and v1.0.103 with safe-perms true/false.

ok this is the result of the test:

npm v1.0.101: success with unsafe-perms, fail without unsafe perms
npm v1.0.102: fail with unsafe-perms, fail without unsafe perms
npm v1.0.103: fail with unsafe-perms, fail without unsafe perms

Very interesting.  Is upstart running the script as root?  Can you please share the output of this?
sudo npm config ls

Try witn 1.0.104.

it works! ty @isaacs


razor components should allow custom attributes

currently, when assigning html attributes that are not explicitly defined on razor components, system will throw exception. e.g. we can not assign an id to EditForm. custom attributes should be passed down to component's root element and preserved  to apply customization.

Thanks for contacting us, @kennywangjin.
@danroth27 don't we have a separate issue with a similar ask? I tried to find it, but with no luck.

I think this is it: https://github.com/aspnet/AspNetCore/issues/5071

Closing as a dup.


Catch case where constructorArguments are not set

I am hoping such a minor fix does not need a full test.  I will write one up if needed.
When attempting to automap incompatible classes, this line generates a null pointer exception instead of the much more helpful mapping exception that is intended.
ParameterTypes are never set in SimpleConstructorResolverFactory when targetParameters and parameterNames (line 128) do not match up.
As a result, this exception, I assume is intended, for this situation is never thrown.

Thanks !


Restore MyEtherWallet and have a shortcut access to the desktop

Hi,
Can someone help me restore MyEther Wallet?
Also, how do I create a Desktop icon to access it.?
Thanks,
Tony

Can someone help me restore MyEther Wallet?

Can you please explain what you mean by this?

Also, how do I create a Desktop icon to access it.?

http://www.laptopmag.com/articles/how-to-create-desktop-shortcuts-for-web-pages-using-chrome

Hi, and thank you for the response. I'm trying to get a visual wallet icon of My Ether Wallet that similar to any other offline wallet like Jaxx, Exodus, hat show up as icons on my desktop. For some reason, I thought I could do this. Or, is etherscan.io the only way to view the account?

Ah. I'm guessing you can manually change the icon yourself but since we're a website it's probably going to be the Chrome icon by default. Here's the logo.

Etherscan is quickest way to view transaction history and current balance. You can also do so on the  https://www.myetherwallet.com/#view-wallet-info page or download the chrome extension here and you can add your wallet and have quick access to your balances in chrome.


Docker Compose containerization

Containerization of CTFd into gunicorn, Postgres, and data containers for easy backups and upgrades. Can be quickly tested by running:
docker-compose build
docker-compose up

Data persistence across application source code updates is also tested and working.
Dockerfile has been modified to:

Install postgresql-dev and psycopg2
Perform a COPY of the current directory instead of cloning the repo again

Awesome! I like the PR and really needed to move to a proper database. A few things though (don't hate me):

Can you swap postgresql with mysql?
And use an image from an official docker repo over a public one? Perhaps this one?

or MariaDB

No problem! Both requested changes are in the new commit.

Hot

Actually hot


Graphics objects can't be interactive

Graphics objects can't be interactive.
In the version of 2.1.3, Pixi v2 Specific New Features contain "Graphics objects can now be interactive." , this is amazing, and it last stable in 2.2.0, 2.2.1, 2.2.2. But after that, from 2.3.0 till 2.5.0 this feature is missing.
Please fix it. Thank you!
testgraphics.txt

Your testgraphics.txt file is empty for me. Please can you upload it as a gist, or just paste it into a comment here.

test

function create() {
this._graphicsHat = game.add.graphics(0, 0);
var createCanvas = drawHat(this._graphicsHat, 0, 0);
this._canvasHat = game.add.sprite(0, 0);
this._canvasHat.addChild( createCanvas );
// click:
this._canvasHat.inputEnabled = true;
this._canvasHat.input.useHandCursor = true;
this._canvasHat.events.onInputDown.add(clickHat, this);
}
function drawHat(ctx, xoff, yoff) {
ctx.beginFill(0xff0000);
// ctx.beginPath() en Pixie.js
ctx.lineStyle(4, 0x00ffff, 1);
ctx.moveTo(279 + xoff, 301 + yoff);
ctx.bezierCurveTo(263 + xoff, 296 + yoff, 262 + xoff, 269 + yoff, 273 + xoff, 253 + yoff);
ctx.bezierCurveTo(276 + xoff, 224 + yoff, 302 + xoff, 159 + yoff, 310 + xoff, 147 + yoff);
ctx.bezierCurveTo(346 + xoff, 93 + yoff, 405 + xoff, 75 + yoff, 420 + xoff, 74 + yoff);
// a lot of more ctx.
return ctx;
}
function clickHat() {
console.log('click...');
}

Hi photonstorm, thank you for reply, I had test from every version from 2.0.6 to 2.5.0, only 2.1.3, 2.2.0, 2.2.1, 2.2.2 can enabled "inputEnabled", "useHandCursor", "events" on graphics, other versions still remain default cursor when mouse over, and no click event can be added on it.

Thanks for submitting this issue. We have fixed this, and the fix has been pushed to the dev branch.


Installation of h2o4gpu on Kaggle using reticulate package

Environment (for bugs)

OS platform, distribution and version (e.g. Linux Ubuntu 16.04): Kaggle
Installed from (source or binary): -
Version: -
Python version (optional): 3.7.11 (default, Jul 27 2021, 14:32:16)  [GCC 7.5.0]
conda version: conda 4.10.3'
GPU model (optional): -
CPU model: -
RAM available:  13GB CPU, 15.9 GB GPU

Kaggle config in an R notebook:
python:         /miniconda/envs/r-reticulate/bin/python
libpython:      /miniconda/envs/r-reticulate/lib/libpython3.7m.so
pythonhome:     /miniconda/envs/r-reticulate:/miniconda/envs/r-reticulate
version:        3.7.11 (default, Jul 27 2021, 14:32:16)  [GCC 7.5.0]
numpy:          /miniconda/envs/r-reticulate/lib/python3.7/site-packages/numpy
numpy_version:  1.18.5
python versions found:
/miniconda/envs/r-reticulate/bin/python
/usr/bin/python3
/usr/bin/python
Code used
devtools::install_github("h2oai/h2o4gpu", subdir = "src/interface_r")
Description
Trying to install the h2o4gpu in a Kaggle R notebook results in a failure. Installation process and error message are pasted below.
Package was used and installed on the platform without problems at least until 28th September, 2021. Now it throws out an error connected to the LightGBM package.
Downloading GitHub repo h2oai/h2o4gpu@HEAD
'/usr/bin/git' clone --depth 1 --no-hardlinks --recurse-submodules https://github.com/h2oai/cub /tmp/Rtmp0osEej/remotesf6fa0ec80/h2oai-h2o4gpu-aaf7795/src/interface_r/../../cub
'/usr/bin/git' clone --depth 1 --no-hardlinks --recurse-submodules https://github.com/h2oai/xgboost /tmp/Rtmp0osEej/remotesf6fa0ec80/h2oai-h2o4gpu-aaf7795/src/interface_r/../../xgboost
'/usr/bin/git' clone --depth 1 --no-hardlinks --recurse-submodules https://github.com/h2oai/py3nvml /tmp/Rtmp0osEej/remotesf6fa0ec80/h2oai-h2o4gpu-aaf7795/src/interface_r/../../py3nvml
'/usr/bin/git' clone --depth 1 --no-hardlinks --recurse-submodules https://github.com/h2oai/scikit-learn.git /tmp/Rtmp0osEej/remotesf6fa0ec80/h2oai-h2o4gpu-aaf7795/src/interface_r/../../scikit-learn
'/usr/bin/git' clone --depth 1 --no-hardlinks --recurse-submodules https://github.com/google/googletest.git /tmp/Rtmp0osEej/remotesf6fa0ec80/h2oai-h2o4gpu-aaf7795/src/interface_r/../../tests/googletest
'/usr/bin/git' clone --depth 1 --no-hardlinks --recurse-submodules https://github.com/h2oai/LightGBM.git /tmp/Rtmp0osEej/remotesf6fa0ec80/h2oai-h2o4gpu-aaf7795/src/interface_r/../../LightGBM
Warning message in system(full, intern = TRUE, ignore.stderr = quiet):
“running command ''/usr/bin/git' clone --depth 1 --no-hardlinks --recurse-submodules https://github.com/h2oai/LightGBM.git /tmp/Rtmp0osEej/remotesf6fa0ec80/h2oai-h2o4gpu-aaf7795/src/interface_r/../../LightGBM' had status 1”
Error: Failed to install 'h2o4gpu' from GitHub:
Command failed (1)
Traceback:

devtools::install_github("h2oai/h2o4gpu", subdir = "src/interface_r")
pkgbuild::with_build_tools({
.     ellipsis::check_dots_used(action = getOption("devtools.ellipsis_action",
.         rlang::warn))
.     {
.         remotes <- lapply(repo, github_remote, ref = ref, subdir = subdir,
.             auth_token = auth_token, host = host)
.         install_remotes(remotes, auth_token = auth_token, host = host,
.             dependencies = dependencies, upgrade = upgrade, force = force,
.             quiet = quiet, build = build, build_opts = build_opts,
.             build_manual = build_manual, build_vignettes = build_vignettes,
.             repos = repos, type = type, ...)
.     }
. }, required = FALSE)
install_remotes(remotes, auth_token = auth_token, host = host,
.     dependencies = dependencies, upgrade = upgrade, force = force,
.     quiet = quiet, build = build, build_opts = build_opts, build_manual = build_manual,
.     build_vignettes = build_vignettes, repos = repos, type = type,
.     ...)
tryCatch(res[[i]] <- install_remote(remotes[[i]], ...), error = function(e) {
.     stop(remote_install_error(remotes[[i]], e))
. })
tryCatchList(expr, classes, parentenv, handlers)
tryCatchOne(expr, names, parentenv, handlers[[1L]])
value[3L]

As of October 4th the issue resolved itself. Not sure whether it was connected to the LightGBM submodule update or a Kaggle problem.


Persist static properties

Hey there! We use recompose in a React Native app that also uses react-navigation. We enhanced one of our page components with composed withProps() and connect() and the returned component of withProps() lacked the navigationOptions that we expected to have (because it's a static property and this is a new component) what's confused us though is that connect() supported that use case and persisted the static properties. I wonder what is the approach for this use case and why does the behaviour differ from react-redux

Have you tried hoistStatics?