Handle GroupDiscoveryFailedError in proxy.go (#5596)

Description of the change:
Similar to the fix in helm, this fix allows GroupDiscoveryFailedError to not error out the process of managing apiserver resource types.
Motivation for the change:
The proposed fix in #5596 requires that the cluster's state be fixed (upgrading / cleaning up the metrics server apiserver resource). While this is correct, it's not a fix that can be made by the author of an operator as it requires the user of the operator to fix their cluster. Fixing this on the client side (in the operator itself) will save headaches and support tickets in the future.
Checklist
If the pull request includes user-facing changes, extra documentation is required:

[x] Add a new changelog fragment in changelog/fragments (see changelog/fragments/00-template.yaml)
[x] Add or update relevant sections of the docs website in website/content/en/docs

Hi maintainers! I think this fix should address the linked issue, but I'm still working on getting a valid repro case that will allow me to fully validate. I'll also plan to look into a unit test to exercise the corner case before moving to full review.

I've verified that make test-e2e-ansible passes cleanly. I'm still trying to learn my way around the test framework, but it looks like the most appropriate place to add tests would be in cluster_tests.go. It's not clear to me quite how we would simulate this kind of apiserver failure in those tests, so I'm open to suggestions!

@gabe-l-hart Thanks for taking the time to make this contribution!
I looked through the changes and it seems relatively straightforward. I would definitely like to find a way to add a test case for this PR before merging though to ensure the change is working as expected. cluster_test.go definitely looks to be the right place to add a test case.
Regarding the implementation of the test case itself, this comment on the Helm issue you linked makes me think as part of the test case we would need to install an api-service onto the test cluster and then intentionally orphan it somehow (I'm honestly not sure how to do this - not very familiar with aggregated apiservers).

Thanks for the pointer @everettraven! I'll look into adding a test and see how far I get. I also don't know yet how to simulate an orphaned apiserver (we just saw this come up on a customer cluster, but I have no idea how it got in that state!) but I'm sure it should be doable somehow, if not in a real cluster at least in some kind of a mock.

I also don't know yet how to simulate an orphaned apiserver (we just saw this come up on a customer cluster, but I have no idea how it got in that state!) but I'm sure it should be doable somehow, if not in a real cluster at least in some kind of a mock.

@gabe-l-hart I was doing some pretty rudimentary digging around and I think a good starting point for understanding how the apiservers work could be: https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/apiserver-aggregation/
I also saw this: https://kubernetes.io/docs/reference/kubernetes-api/cluster-resources/api-service-v1/ - it looks to go over the APIService resource and I wonder if we could somehow use that to fake out the cluster we are using for testing that there is an apiserver running for a specific group/version without there actually being a backend implemented (maybe this would cause the error to happen?).
Just some more thoughts from doing some brief digging around about aggregated apiservers - I'm looking forward to seeing what you come up with (and feel free to ping me on here if you just want to bounce ideas around :))


[Bug] Map isn't loading

When I disable internet, the map isn't loading obviously. Before a text was shown that there is no internet connection.
I guess after you changed it from the string to coordinates it messed it up. Idk

This is low priority imho. I can still see the map if I click on it. I just reported it as it still a bug I'd say

Where did you download it?

I've just tested it and I can confirm this is the case for me as well. I'm on a Pixel 7 with GrapheneOS. I'm running the latest version available from GitHub releases.

It has been fixed already in a previous commit, check latest build from Github Actions


value:Cloneable can be assigned to any

Description:
$title. Shouldn't be allowed since value:Cloneable contains error.
Steps to reproduce:
import ballerina/lang.value;

public function main() {
    value:Cloneable x = error("x");
    any y = x; // Should fail since `value:Cloneable` includes `error` and `x` can hold an error.
}

Affected Versions:
slp9-SNAPSHOT

This may break some existing codes. Below is allowed before.
eg:
public function main() {
    readonly x = "sd";
    any y = x; 				// incompatible types: expected 'any', found 'readonly'
}

This also requires updating several AssertEqual helper methods argument in test suite from any|error to any|readonly|error.
ToString method should also be changed to accept any|readonly

This may break some existing codes. Below is allowed before.
eg:
public function main() {
    readonly x = "sd";
    any y = x; 				// incompatible types: expected 'any', found 'readonly'
}

This also requires updating several AssertEqual helper methods argument in test suite from any|error to any|readonly|error.
ToString method should also be changed to accept any|readonly


fix: old 3scale releases returned 404 Not found on backend endpoints

heads up @whitingjr

@eguzki thanks for the heads up.


update readme.md

the documentation needs to be updated with new features (example: page number ) in readme for newest features you can refer to example.py

Heyy, I've decided to take up this issue.
One small problem, I'm running into the following error :
searchengine = SearchEngine("search engine", PAGE_NUMBER)
TypeError: SearchEngine.init() takes 2 positional arguments but 3 were given
I don't get how there are three arguments here. Could you help me out ?

assign me this task @codewithnick and i'll try to understand the code and start working immediately. I might also help with other issues in you repo.


🌐 Add Chinese translation for Tutorial - Metadata and Docs URLs

🌐 Add Chinese translation for Tutorial - Metadata and Docs URLs #1228

Great, thank you @blt232018 ! :rocket:
And thanks for the reviews @waynerv and @Laineyzhang55 ! :cake: :coffee:


Is this project still being maintained?

Activity from the maintainers seems very low and there are hundreds of PRs and issues.
Is this project still being maintained, or can someone suggest an alternative?

I think this project is the best bet for a angular native implementation of a select2 field.
There have to be many developers which are using this directive and they know the way around the hard edges (including me).
Another alternative could be Angular Material with the nice chips: https://material.angularjs.org/HEAD/#/demo/material.components.chips

Thanks. I agree - this is still a good project, I'm just concerned that it's not really being maintained and therefor it will quite soon become not such a good project :(

There have to be many developers which are using this directive and they know the way around the hard edges (including me).

This is what concerns me :) I'm also making some modifications and have submitted a few PRs over the past few months, but there are now around 150 open PRs, and 470 or so open issues. It would be great if the maintainers no longer have the time to support it if others were able to...
The longer PRs are left unprocessed, the more difficult it is to merge them as there will be conflicts...

I'm also a bit concerned. I use it throughout my project and I don't have a better choice for now.
Also see this discussion: https://github.com/angular-ui/ui-select/issues/1096

Are either of you interested  in taking this project over?

@ProLoser , I'm interested also in development this project. Ready to help

@sars one way to help could be to try checking duplicated issues and posible PRs

Are either of you interested in taking this project over?

I would be happy to help with the project. I think since this is such a popular project that it needs a few people to help maintain it as it needs to be regularly maintained. Currently, we go for months without any activity from the maintainers, and hence we end up with a totally unmanageable number of PRs and Issues... I don't mean to criticise, people have a lot of demands on time these days, but the project can't thrive if nothing changes for months on end...
I'm not currently hugely familiar with the source, but would be willing to help maintain it. I would be happy to initially spend a few days going over the issues found in #1096. From there I would suggest it might be easier to close "all" old issues (say, older than 4 months) - or at least request that it is still valid. Likewise for PRs - many already have conflicts and need to be updated against the last codebase. This could allow the project to be brought back into a manageable situation where a few people should be able to keep on top of things every day or three (and I would be happy to support this too)...
Just my 2p :) Maybe the above suggestion is a little drastic, but this is a nice project, and I think it needs something to bring it up to date...

@cdjackson That's fine.

@cdjackson agree also

Just an update on what I'm doing. I've removed a lot of watches which i don't think are necessary, and are probably bogging down the digest cycle. I've completely removed tagging and refresh from the core and have added lifecycle handlers (currently onKeypress, onBeforeRemove, onBeforeSelect and onBeforeDropdown) which should allow tagging and refresh to be implemented externally - or various other forms of the control.
The reason I picked on tagging as a starting point is it is spread throughout the control and it made the code a bit messy (IMHO!). Possibly the onKeypress() callback doesn't allow as rich a tagging implementation (I've not spent too much time testing this), but it is ok, and these callbacks ought to allow better implementations than my quick test tagging implementation. I've also used this to create a combo-box control, which allows you to select from the list, or type in whatever you like, and this ought to be possible with these callbacks.
I've also updated the build to produce separate implementations for the different templates (the templates account for ~10k), and separated out the sort directive... It's still work in progress, but thought I'd mention what I was doing... If it's outside of what is required for this repo, then at least I'll be in a position to make my combo-box control :)
A rough estimate of size reduction (comparing to minified versions)-:
Original ~35k
Remove tagging ~31k
Remove refresh ~30k
Remove templates ~25k with just a single theme
The sort directive is about 1k as well... I've also fixed a few bugs along the way...
This is clearly a significant (and breaking!) change to the current repo (well, if you are using tagging or refresh anyway!), so I'm not sure if it's of interest or not. If it is, I'm happy to tidy, sort out the tests and provide it for review - otherwise I'll just stick with this in my projects :)

You should open a pull request so we can start providing feedback (although I expect it to be nearly unrecognizable lol). Alternatively, perhaps we should give you push rights and let you work on a branch on this repo so that we can keep this a collaborative effort and other people might be poised to support your endeavors.
One thing I'd like to see is that we still (to some degree) implement some of the most critical prior features, but doing so in this composeable approach. For instance, demoing how tagging or custom entries would be done with this new API will help other devs further rip out core code.

What is refresh btw? And what sorts of watchers have been removed? If these are things that can be converted to JIT (Just In Time) evaluations, that's always better. It's not like you need to keep the settings up to date while you are NOT interacting with it, and it's not like it's a reasonable expectation for the widget to update while you ARE in the middle of interacting (or if this is the case, allow people to do this themselves).

although I expect it to be nearly unrecognizable

Maybe not unrecognizable, but certainly a diff will tell you (I think) that every line has changed as I've reformatted, added more comments, and fixed a few bits of code...

perhaps we should give you push rights and let you work on a branch

That's also fine - I'm not sure that you can limit access to a branch, but I can play nicely :)

One thing I'd like to see is that we still (to some degree) implement some of the most critical prior features

I agree - I have re-implemented tagging (in a basic sense!) for exactly this reason (and I intend to use it!). Currently, it probably doesn't support a bunch of features, but using the onKeypress, it should be possible to do most/all of what the current binding does. For example, tagging works fine (from my basic  tests) but I only support enter to end tagging, not a list of tokens, but this should be easy to implement.
Additionally, I have the combo style, which uses the single select to add custom entries - maybe the control already did this, but I couldn't see how and the docs weren't forthcoming...
Both of these are working in a demo form (bootstrap only) that's included in my test branch (newdemo.html/js).

What is refresh btw?

It is/was an async loading callback. I didn't look at the implementation in detail, but it called a user specified method when the control was opened, and then called it again every user defined seconds.  I've not tried to implement this in my demo, but it shouldn't be hard - I've added a callback for the dropdown opening and closing, so the user can start/stop a timer to do whatever async stuff they like!

And what sorts of watchers have been removed

Many of the watches/observes on configuration options - it's possible I might have gone too far, but things like searchEnabled, sortable, all the tagging options (tokens, label etc) were watched but these have been removed now anyway. Relevant options like the searchEnabled could be added back into a method that re-evaluates on dropdown if someone really thinks they need to be changed dynamically - this would be simple... I'm sure someone will tell me there's a use case, but I was in a slightly ruthless mood :)
The way things are at the moment is I've not added extra API methods for manipulating the control within the callbacks. I'm split between adding specific public API methods and potentially reducing flexibility, or just allowing interaction directly with the controller. I'm happy to take 'angular-ui' standard practice/guidance if there is any on how you'd prefer to see this...
Currently my mods are in https://github.com/cdjackson/ui-select/tree/refactor_branch

@cdjackson will need some place to provide feedback on your branch.
Few notes, I was poking around your addons, you are not making of this controller methods. The first and foremost priority is to expose EVERYTHING as a controller method. This allows you to inject the directive-controller and override the methods (making onWhatever callbacks a redundant convenience tool).
I don't remember what the lookup MAP is for, but I don't really know if we should bother with it anymore. If we use the mapped keys explicitly in our code, that's fine, but if we're trying to expose some sort of convenience to users, it should be removed. Let devs sort out keycode aliasing themselves.
 Instead, simplify and expose the collection of items we use to render the dropdown. This will allow users to hack this list at any time they want with whatever they want. This feature definitely does not belong in the core.
Again, please RELIGIOUSLY pore over this new directive I assembled (and the README examples) for guidelines on what I'd like to see: https://github.com/angular-ui/ui-mention/blob/master/src/mention.es6.js
I agree with removing all these silly watchers. We can simply set boolean flags on the uiSelect controller that are checked whenever the control is interacted with.

The first and foremost priority is to expose EVERYTHING as a controller method. This allows you to inject the directive-controller and override the methods (making onWhatever callbacks a redundant convenience tool).

Yep - that's what I thought you might say, and that's fine. Would you keep the callbacks for application level calls?

I don't remember what the lookup MAP is for, but I don't really know if we should bother with it anymore.

It's used to detect key changes - I've already deleted most of it and just kept the special keys which are used (that removed a couple of kb).

Yep - I deleted it :) The same functionality should be possible using onDropdown (etc) callbacks.

@cdjackson so right now the issue with stuff like onBeforeSelect (which I would just call beforeSelect) and onAfterSelect and other such callbacks is that they are still us explicitly exposing API, which although is a step in the right direction, is not exactly what I had in mind for this iteration.
Instead, just do onSelect which contains OUR code. Essentially you're not exposing optional callbacks just so the user can hook into them, which means we are trying to plan for and accommodate for the user. Instead, we should be exposing what we are and need to do anyways in a way that can be overridden.
For instance:
// public api: $select.onBeforeSelect()
// public api: $select.onAfterSelect()

Which is called from private api
// private api:
onSelect(choice) {
  // do prep
  if ($select.onBefore) $select.onBefore()
  // do swap
  if ($select.onAfter) $select.onAfter()
  // do cleanup
}

The issue with this approach ^ is we are still hiding code from being modified, and what we publicly exposed might eventually not be enough if the user so chooses, or we have to keep exposing more and more parameters to these callbacks, etc.
This is not designed with refactoring-friendly mentality.
Instead, watch how we can lump all of this together into a simpler api that gives devs more control:
$select.onSelect(choice) {
  // do prep
  // do swap
  // do cleanup
}

// The dev can then override this mandatory method which actually contains logic (and not just stubs) to yield the same results as above:
var onSelect = $select.onSelect;
$select.onSelect = function(choice) {
  // do custom prep (beforeSelect)
  onSelect.call($select, choice);
  // do custom cleanup (afterSelect)
}

We have now given devs the ability to add before, after or just completely replace the select behavior completely if they so choose. There is no 'optional' apis that we tried to 'guess' at, nor do we have to come up with stateful parameters to provide to these callbacks or worry about them messing up the state in any way that wasn't already expected.
So again, write EVERY SINGLE PIECE OF LOGIC that we use as a publicly replaceable/accessible method. This gives people the ability to monkey patch and extend behavior.
In addition, try to shrink down methods as much as possible into implementation-agnostic methods.
This means onSelect() should essentially do nothing more than call a series of more concise methods like so:
$select.onSelect(choice) {
  this.setChoice(choice);
  this.render();
  this.close();
}

By breaking this up into smaller, yet still publicly exposed methods, we allow devs to be less wary about completely modifying or replacing behavior. They don't have to copy-paste a 40-line function just to tweak 1 line.

@ProLoser :+1: This is pretty.

This is just object oriented methodologies.

While I agree on most of the proposed directions, this specific point:

So again, write EVERY SINGLE PIECE OF LOGIC that we use as a publicly replaceable/accessible method. This gives people the ability to monkey patch and extend behavior.

I cannot agree. I believe it's designer responsibility to carefully decide what can be replaced.  Otherwise a) no method could refer to outer scope and b) update that component on a project would not be an easy task.

a) having a directive refer to outer scope is extremely, like ridiculously, bad design. You should use isolate scope or $parse instead to treat things internally.
b) I don't know why updating would be hard.
All I'm saying is instead of doing:
function x() {...}
you do
ctrl.x = function() {...}
That's it.
This means that all the logic that is intended to be used internally is externally exposed and properly documented regarding it's API. As long as the API is properly adhered to, then anyone who wishes to can extend, supplement or replace that logic with their own without us having to 'plan' for it. This is the basis of composibility, modularity and extendability. The biggest negative is probably around letting people directly access internal data/state such as the items or the visible state, however if we keep our methods small and concise and provide proper methods (setters) to handle even the mildest complexity (ctrl.choose = (item) => { ... }) then it shouldn't matter.

a) having a directive refer to outer scope is extremely, like ridiculously, bad design. You should use isolate scope or $parse instead to treat things internally.

Agreed. But I refer to lexical scopes, that is, closures.
Regarding point (b), it is that any bugs or features made into new releases that happen to alter public methods, each user is in charge of updating his/her code accordingly. If an user is encouraged to replace any method, changing any logic there, all updates are guarantees of breaking changes. It doesn't matter how much code coverage there's in component's or user code, it'll fail on integration.
Technical support would also be challeging. Versioning will be in a gray area, because a given version may function some way in one environment, yet not work in another. Instead of promote liberty to users, it could make the component less attractive. I believe instead a compromise should be taken, one that does not create breaking changes.

This appears to be a pretty old conversation, but I'd like to throw my hat in the ring here.
Another thing that would immensely improve the extendability of the design is to move all the DOM interaction out of uiSelectCtrl into link functions on the directives -- in addition to the improved API you have already been discussing.
Talk is cheap so here's my offer - I am willing to go and do that kind of refactor. But I don't want to step on any toes if there's already a big effort underway per the above conversation. @cdjackson & @ProLoser - I'd appreciate your take on this.

@meyerds I haven't followed up on this recently (sadly) but I've been considering taking my own stab at rewriting this plugin because my company really needs it.
However, if I were to do it myself there's a good chance I'd be starting over from scratch because I'm fairly unhappy with the state of the code as it is, and it'd be extremely dumb. At most it'd have single select and multi-select and a powerful yet minimal API, but that's about it (similar to ui-mention which I recently created).
Poking around @cdjackson's fork it looks strong but there's just too much code for me to thorughly review myself and I'm not really seeing anyone else helping vet the code.
What exactly were you thinking of doing for your attempt at building this out?

@ProLoser I'm in a similar situation with my company - I need to be able to extend the behavior. Specifically intercept keystrokes in the search input and conditionally let them propagate through. No way to do that currently because the keyup and keydown handlers are registered in the uiSelectCtrl instead of in a link function.
Anyway. The little bit that I've poked around the current codebase, I'd agree that starting over from scratch would be the ideal approach. Lots of issues there. Unfortunately, I don't think my client would pay for that scope of change. I could probably get them to buy off on fixing the current codebase just for my issue specifically.
The issue I've got isn't high priority to the client though, so if someone like you or @cdjackson are serious about starting over from scratch I'd be happy to contribute in whatever way makes sense. Even if on personal time. Seems like a worthwhile endeavor.

@meyerds if I were to start over, you'd essentially have 2 elements, a dropdown and an input. And I would not make these part of the template. You'd have to insert the dropdown yourself, but it would provide you with a scope reference to the controller so you'd just do <a ng-click="$select.choose(item)"> etc. This would give you full control to do almost everything, except for multiple-select.
Then I'd want to implement multiple select which is trickier because the input widget is actually several DOM elements (it's a list of small anchors which can be highlighted, have cancel buttons, and the typeable area has to keep resizing to fill the remaining space).  This would be isolated to a secondary directive that adds 'multiple' behavior to JUST the input part. I don't know if there are any other features I'd really care about, because if you have access to the native <input> you can toggle things like being enabled/disabled. Hook into click events, put your own conditions around the choose() function and can probably easily mess with things like the bindings if we do: $element.on('click', function(){ ctrl.click() }) which would allow you to override the ctrl.click property with your own variant.

@ProLoser @meyerds I'd be happy to do fill-work bringing a new project following @ProLoser's simple OO approach up to parity via examples / extensions. If I had time I'd be in for the lib-side work, but I just can't commit to that in a reasonable timeframe right now. I'm following this issue; if you move the discussion, please ping me.

I'm thinking perhaps we should start a new issue (or repurpose this one?) as a new breakdown of what the core features are, a loose api, and a 'nice-to-have' feature list and approximations of how these could be accomplished by mish-mashing our api together. I think the biggest flaw with my overly-simplistic approach is I tend to not opt for dealing with positioning or managing of the dropdown whatsoever.

Seems to me that managing the dropdown (includes positioning) is a core responsibility in the sense that default behavior is necessary - can't demo without it - but superficially I think we could handle that in the same way (extend/override).

@jnelson but think about bootstrap, which actually requires you to place the dropdown HTML onto the page and the relevant container. The only thing it really does is toggle visibility of the dropdown, but that's about it.
If you're looking for a do-everything-in-one-html-tag sort of directive we could wrap it but I'd rather just ask other people to wrap it in a proprietary directive so they can configure all the convoluted variations they wish into it.

I happily lump "toggling visibility" in with "managing the dropdown" and concede that it could be nearly all of it. :+1:
The way https://github.com/angular-ui/bootstrap/blob/master/src/dropdown/dropdown.js accomplishes its goal isn't crazy. It's not what we're discussing, but maybe it's an interesting reference point. This repo has all its select2 baggage, but my guess is that a large number of users aren't attached to that original intent.
I'm still a bit stuck on the wise idea to provide "functionality via documentation" of a simple, flexible API. That seems technically smart and might also be a nice contribution target. You could call it a state machine but I'd rather see a user-manipulable promise pipeline (or something better). Series of events, things can happen.
And then there's "build the core in pure JS, then wrap that project in an Angular adapter". (I haven't checked to see whether this exists yet.)

Closing this issue - currently this project is being maintained by @user378230 and @aaronroberson , and I'm just going through and helping out with basic issue triaging/PR code reviews.
This library could use a lot of love though, so any high quality help is welcome, whether it is directly with helping create high quality features/bug fixes or creating up to date Plunkers and verifying or debunking issues.

Thank you so much for your help @wesleycho!


[PL-46234] Update SMP external cloud-based MongoDB tutorial for TLS support for 0.13.0

Update SMP external db tutorials for TLS support for 0.13.0
For reviewers, a preview is available:
External cloud-based MongoDB
What Type of PR is This?

[ ] Issue
[x] Feature
[ ] Maintenance/Chore

If tied to an Issue, list the Issue(s) here:

Issue(s)

House Keeping
Some items to keep track of. Screen shots of changes are optional but would help the maintainers review quicker.

[x] Tested Locally
[ ] Optional Screen Shoot.

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app. Current Draft URL is: https://65b12f7da64cb61cbb6e29f8--harness-developer.netlify.app

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app. Current Draft URL is: https://65b938be2cc4ca1d52ac05c8--harness-developer.netlify.app

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app. Current Draft URL is: https://65b94296088606201c82edaa--harness-developer.netlify.app

@kapilgarg1996 approved via Slack


Can't resync media image after installation

I followed all the steps including setting up the base_url for the media url. At first when I run the bin/magento s3:storage:export all assets were uploaded in the s3. Then I enabled the module. After that I uploaded a new image again (programatically generating new image). But the image is still being saved in pub/media folder. Now I tried running s3:storage:export again but I get the error You are already using S3 as your media file storage backend!. Now I can't sync my assets now. I thought the script will handle all the syncing to the s3. So whenever I upload an image into the pub/media the module will handle the exporting of it to the S3.
I guess I was wrong with that. Can you tell me how can I resync the content of my server and the s3? and how do I auto upload it into the s3?

to export again images, try to disable the storage, export the images and then enable again

I  have also faced the same issue , Everything configured perfectly as said by "@ramzdam , When i   upload the five product images in to magento2 upload not happening  in to aws s3 bucket, Instead of all four images  showing the static image content , Only one image was uploaded to s3 bucket .
the same i was uploaded to another product it was not uploaded none of the images to s3 , instead of shows static image content from magento2.
As well as product image keep buffering buffering in store front end.


WebSocket API on Development

WebSocket API initial commit

You are welcome to contribute but this PR is far from ready to merge.


WIP : Market Oracle V2

Any thoughts on MarketOracle.sol? ignore the rest for now.

Any thoughts on MarketOracle.sol ignore the rest for now. @nithinkrishna @brandoniles

@nithinkrishna @brandoniles  Made a round of changes based on your comments.

Ready for full review. I will also be doing 1 more review round.
@brandoniles @nithinkrishna  .

All done.


Add lego-bricks extension

Description
Look up LEGO sets, colors, bricks etc.
Screencast

Checklist

[X] I read the extension guidelines
[X] I read the documentation about publishing
[X] I ran npm run build and tested this distribution build in Raycast
[X] I checked that files in the assets folder are used by the extension itself
[X] I checked that assets used by the README are placed outside of the metadata folder

@pernielsentikaer  Thanks for the comments. Suggested changes have been made, please review :)


Feedback form enhacement

@anuragverma108 I can make the feedback form look more appealing.
Please assign it to me with the gssoc and hactoberfest tags.
PFA
Thank you!

ok .


Closes #2549 Allow container datasource to be used directly or proxied

Closes #2549
By proxying the datasource with a class loaded by the app classloader, it resolves some classloading issues, but now by default it won't proxy the datasource. Tomcat currently requires the proxy option to be turned on.
This may be tricking Hikari into thinking it is delegating to an existing Hikari datasource but it is really a non-Hikari datasource. Might be better to just add a small datasource proxy class rather than using Hikari in a way it probably doesn't intend for people to use it. Since I am creating the datasource with a HikariConfig object, the HikariDataSource sets a fastPathPool instance variable and when getConnection is called it just delegates to that (the non-hikari container datasource).
I didn't document this yet, but it adds one new configuration property that is false by default. You can wait to merge until later.
When I actually tried to set properties on the Hikari pool, making it a pool of a pool, I got an error like this:
Caused by: org.postgresql.util.PSQLException: Cannot change transaction read-only property in the middle of a transaction.
	at org.postgresql.jdbc2.AbstractJdbc2Connection.setReadOnly(AbstractJdbc2Connection.java:749) ~[postgresql-9.4-1203-jdbc42.jar:9.4]
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_111]
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_111]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_111]
	at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_111]
	at org.apache.tomcat.jdbc.pool.ProxyConnection.invoke(ProxyConnection.java:126) ~[tomcat-jdbc.jar:?]
	at org.apache.tomcat.jdbc.pool.JdbcInterceptor.invoke(JdbcInterceptor.java:108) ~[tomcat-jdbc.jar:?]
	at org.apache.tomcat.jdbc.pool.DisposableConnectionFacade.invoke(DisposableConnectionFacade.java:81) ~[tomcat-jdbc.jar:?]
	at com.sun.proxy.$Proxy110.setReadOnly(Unknown Source) ~[?:?]
	at com.zaxxer.hikari.pool.PoolBase.setupConnection(PoolBase.java:400) ~[HikariCP-2.6.1.jar:?]
	at com.zaxxer.hikari.pool.PoolBase.newConnection(PoolBase.java:361) ~[HikariCP-2.6.1.jar:?]
	at com.zaxxer.hikari.pool.PoolBase.newPoolEntry(PoolBase.java:199) ~[HikariCP-2.6.1.jar:?]
	at com.zaxxer.hikari.pool.HikariPool.createPoolEntry(HikariPool.java:444) ~[HikariCP-2.6.1.jar:?]
	at com.zaxxer.hikari.pool.HikariPool.access$200(HikariPool.java:71) ~[HikariCP-2.6.1.jar:?]
	at com.zaxxer.hikari.pool.HikariPool$PoolEntryCreator.call(HikariPool.java:631) ~[HikariCP-2.6.1.jar:?]
	at com.zaxxer.hikari.pool.HikariPool$PoolEntryCreator.call(HikariPool.java:617) ~[HikariCP-2.6.1.jar:?]
	at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_111]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[?:1.8.0_111]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[?:1.8.0_111]
	... 1 more

Ignore what I said about Hikari just delegating to the container datasource, I misread the code, it is still creating some sort of pool around the underlying datasource, I think. Hikari is using javassist to proxy the connections. If we did make a non-pooling datasource instead of using hikari, we could probably use their ProxyFactory class, but that can wait until someone observes a problem with the Hikari proxied datasource that they can't get around by using the non-proxied version.

Hold off on merging this for a bit. It probably fixes the reported issue and is no worse than what was there previously but I still have concerns about hikari pooling a pool. I just tried a fix that seems to work and doesn't do that.

Sure. Keep the PR updated please, as always...and no rush, we have plenty of time left until the final GA.

I think the DataSourceProxy I added should avoid problems of having a pool wrapping a pool. If you have  a hikari pool that doesn't test connections then they could go bad and hikari won't know. You could also have situations where container pool is testing connections in background and closing them without hikari knowing or closing them for being idle without hikari knowing. Or maybe since hikari would be holding on to connections for a long time the underlying pool might think they were leaked and close them. The potential problems really boggle the mind. If you have to configure things like validation queries then it sort of defeats part of the purpose of using a single container datasource.
One other note, if the Beans.java class were a spring ApplicationContextAware bean it would be easy to check for a datasource spring bean using  the dataSourceName and fall back to a JNDI lookup if the bean wasn't found. That would allow someone to define a datasource in Spring and share it across different cas modules without having to know the CAS bean names for each datasource. That can wait until someone needs it.

Thanks, as always. I'll try to take a pass tomorrow.

One other note, if the Beans.java class were a spring ApplicationContextAware bean it would be easy to check for a datasource spring bean using the dataSourceName and fall back to a JNDI lookup if the bean wasn't found.

There is a workaround for this, where you should be able to use ApplicationContextProvider.getApplicationContext() to get by. Might come in handy.

Reviewed. All good.


HDDS-7319. To support s3g usernames with Fair Call Queue from hadoop-commons

What changes were proposed in this pull request?
Currently, when using the FairCallQueue, there is no S3G impersonation, the recorded user for every call is the S3G
special user which makes the FairCallQueue ineffective since it's only reading one user.
OM - S3G communication has been optimized to use a single connection and therefore every user is masked under the special S3G user.
The UGI object on the Schedulable class always points to the S3G user. To fix the issue at hand, we would like to

expose on the Schedulable the existing CallerContext field from the ipc/Server
set the CallerContext on Ozone client side before submitting the request
implement an Ozone side IdentityProvider which will access the CallerContext and use it to makeIdentity()

Every time an Ozone user would like to use the FairCallQueue, that user should also include the OzoneIdentityProvider
on the config file like so
ipc.9862.identity-provider.impl=org.apache.hadoop.ozone.om.helpers.OzoneIdentityProvider

POC doc: https://docs.google.com/document/d/18FWaaA78eyzWEd3ECJKIB4ip0sZBik1sQgQhOj0ZT0I/edit#
What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-7319
How was this patch tested?
New unit tests were added for OzoneIdentityProvider. Furthermore, a robot acceptance test was added for using the FairCallQueue and testing the caller from the metrics.
This is a gist with manual testing for the FairCallQueue with 2 S3G users while OzoneIdentityProvider is utilizing the newly exposed CallerContext field: https://gist.github.com/xBis7/b8247986e718417a4b48320eab6efeda

Thanks @xBis7 for the changes to the s3g fair call queue to properly identity the rpc user for the requests.
Can we not reuse the static OzoneManager.setS3Auth and OzoneManager.getS3Auth for the  OzoneIdentityProvider.makeIdentity getS3Auth?  Is it necessary to redefine the thread local in the OzoneManagerProtocolServerSideTranslatorPB.java?
Also, it looks like an integration test in the CI stopped the green workflow.  Check to see if unrelated.

@neils-dev Thanks for looking into this PR.
In OzoneManagerProtocolServerSideTranslatorPB during processRequest we assign the S3Authentication value to static OzoneManager.setS3Auth here, making it available to OM for all the necessary actions but right before exiting processRequest we reset it's value to null here. By the time we try to access it in OzoneIdentityProvider it's value is always null.
On the other hand, the ThreadLocal field S3_AUTH_THREAD_LOCAL in OzoneManagerProtocolServerSideTranslatorPB is only used for OzoneIdentityProvider so we don't need to reset it's value explicitly.

Also, it looks like an integration test in the CI stopped the green workflow. Check to see if unrelated.

It's unrelated. I have 3 green builds in a row in my fork's CI.

@jojochuang Thanks for the review.

We need to publish the config change somewhere in the user doc.

Apart from the site, do you think there is a suitable doc in the project to add it?

@neils-dev @kerneltime I updated the patch based on Neil's comments and the discussions on the latest community call. I removed the thread local variable in OzoneManagerProtocolServerSideTranslatorPB and after S3 authentication during processing a request, we get the S3 user and set it as the login user in UserGroupInformation. Then in OzoneIdentityProvider we get the user directly from the UserGroupInformation and not from the Schedulable object.

I'm converting this PR into a draft as I'm looking into verifying what @neils-dev pointed out about the thread context. If he is right, I'm going to look for a different implementation.

UPDATE: @neils-dev was right. The Ozone S3G uses a permanent connection to the OM which masks the initial user under the special S3G user. With the above approach, we update the ipc/Server.Handler which sets the metrics, but the ipc/Server.Reader still holds the S3G user UGI which is used in DecayRpcScheduler.makeIdentity(). Therefore, the metrics might display the correct user but there is still no impersonation underneath it.
The approach on this PR, will need some additional changes in order to work. I'll leave it as a draft while I'm working on a fix.

UPDATE:
https://github.com/apache/hadoop/pull/5540 (exposed the CallerContext field on the Schedulable) got merged into hadoop trunk and got backported to branch-3.3.
The current PR remains a draft as it depends on the latest hadoop version or at least a release of the latest 3.3 version. By setting in ozone-main pom.xml file, the hadoop-version to 3.3.9-SNAPSHOT we can test this patch locally.

UPDATE:
I added an acceptance test to verify that the FairCallQueue is working as expected. This PR should be ready to be reviewed and merged. It remains a draft as it depends on a newer hadoop version than the one from the latest release (3.3.5).

A few days ago, hadoop-commons version 3.3.6 was released which included https://github.com/apache/hadoop/pull/5540 making the current PR mergeable. I bumped Ozone hadoop-commons version to 3.3.6 and moved the PR out of draft.

Still need HADOOP-18769 to create a hadoop3.3.6 docker image, and HDDS-8911 for Ozone to use Hadoop 3.3.6 runtime

Still need HADOOP-18769 to create a hadoop3.3.6 docker image, and HDDS-8911 for Ozone to use Hadoop 3.3.6 runtime

@jojochuang Thanks for the info. I was able to build ozone with 3.3.6 and didn't have any dependency issues and thought it was fine to just bump the version. I will wait for these two tasks to complete.

Open up for precommit check.

There are two approvals so I'm merging this one.

@jojochuang Thanks for merging this.


Tour the SAP Community

Tutorial URL: https://developers.sap.com/tutorials/community-start.html
Please specify the step you are referring to
Hi Team,
Pls help...
Step 10 in this tutorial has a bug... despite selecting the correct answers & following all the instructions it wont go ahead, here's attaching a screenshot for your refernce.
Regards,
Taher +918850518284

This is not a technical issue, since the answers provided are different than what is defined for this question. I'll let @lenastodal  discuss the actual qustion

Hi @tglobalops - "Standalone videos are not recommended" shouldn't be selected as it's not correct.
Then the step should work and be validated.
Thanks and best,
Lena


Add dependency and features for time crate

I think there might be an issue in this crates Cargo.toml, I needed to add this line to get things working:
time = { version = "0.3.20", features = ["wasm-bindgen"] }

Since this is a tracing-web crate, I guess this feature should always be enabled, right? Or at least I think this should be mentioned in the README and example.

You are right, that feature needs to be enabled for .with_timer(UtcTime::rfc_3339()) to work. Easy to miss due to feature unification. I will add a note to the docs.


Paste the info from the blog post into the project docs

The blog post gives a lot good context and info we can include in the project itself
https://docs.google.com/document/d/1oH45HrxL2znmEETOY28UjwsLGyhDAOSoOQrR7-4isuo/edit#heading=h.6382g3xf7ld6

Is it enough to just link to the blogpost from README @tzach ?


Server not automatically updating Mods on all Servers

I am experiencing a weird issue.
We are running a root server hosting 4 Ark Servers connected within a cluster.
For about 2 weeks now, arkmanager is not automatically updating all 4 Servers, only the first or only the first and second server.
All log entrys below the timestamp "2019-07-12 01:29:53" are done manually with "arkmanager checkmodupdate @all"
Cronjob to update Server and Mods:
0 * * * * /usr/local/bin/arkmanager --cronjob update @all  --saveworld --warn --update-mods --args  -- >/dev/null 2>&1

Arkmanager Version:
Version: 1.6
Channel: master
Commit: 5d839a7
Release Tag: v1.6.47
Blob SHA: 17673f8d40e090d1b57e371a274b0c3781b0995d

That looks almost as if you used & instead of && when chaining checkmodupdate / update / restart commands together.  & will background the previous command, while && will only execute the next command if the previous command succeeds.

Hey, thanks for the fast reply :)
What do you mean by "&" and "&&"?

I'm getting something similar.  Mod updates on first map, but the rest fail to notice there's an update with the mod.  When I try to log in I get a mod version mismatch.  checkmodupdate returns All mods are up to date.  My cron is:
*/15 * * * * /usr/local/bin/arkmanager update --update-mods --safe --warn --backup @all > /var/log/arktools/cron.log
Dunno if it matters, but they all share a staging dir.

Ah right, I had also set the staging to true with a directory for it, disabled it like a week ago, since i thought aswell, that it is causing this problem.

Just so we are clear on how far this reaches if it is a problem in the tools. I am having a similar issue and I have been updating my servers manually the whole time. It started about 3 weeks ago i'd say?
I log in and enter these commands:
arkmanager stop @all
arkmanager update --update-mods @all
after that it goes through and updates the first instance both ark and the mods
then it seems to skip updating the mods on every other instance but still updates ark
As others have said when I check to see if a instance has outdated mods via
arkmanager checkmodupdate @
it just returns that all mods are up to date even if they arent. the update screen never said it was downloading updates for it and never did because upon starting all instances only the first one has the correct mods.
The only workaround I have found for this is to update ark itself if it needs it then run arkmanager installmods @all
this forces it to grab all of the mods regardless of whether it needs to or not
Version: 1.6
Channel: master
Commit: f69ba15
Release Tag: v1.6.53
Blob SHA: 6d19e8bff03048aec6541c7301512df6f4d30500
All of my instances are separate directories that are clustered.

@TheMourningWolf If the error persists, please create a new issue.


When chart name is "y", helm package creates "true-.tgz"

My assumption is that "y" is a valid truthy value, and somewhere in the helm package codepath converts that to the string "true".
Example:
$ helm create y
Creating y
$ helm package y
Successfully packaged chart and saved it to: /tmp/true-0.1.0.tgz

Output of helm version:
version.BuildInfo{Version:"v3.5.4", GitCommit:"1b5edb69df3d3a08df77c9902dc17af864ff05d1", GitTreeState:"clean", GoVersion:"go1.15.11"}

/assign
Looks like a yaml Unmarshal problem, I will look into it

Before:
[akaris@linux tmp]$ helm package y
Successfully packaged chart and saved it to: /tmp/true-0.1.0.tgz
[akaris@linux tmp]$ helm create no
Creating no
[akaris@linux tmp]$ helm package no
Successfully packaged chart and saved it to: /tmp/false-0.1.0.tgz
[akaris@linux tmp]$ helm create TRUE
Creating TRUE
[akaris@linux tmp]$ helm package TRUE
Successfully packaged chart and saved it to: /tmp/true-0.1.0.tgz

[akaris@linux tmp]$ helm create y
Creating y
[akaris@linux tmp]$ helm package y
Successfully packaged chart and saved it to: /tmp/y-0.1.0.tgz
[akaris@linux tmp]$ helm create no
Creating no
[akaris@linux tmp]$ helm package no
Successfully packaged chart and saved it to: /tmp/no-0.1.0.tgz
[akaris@linux tmp]$ helm create TRUE
Creating TRUE
[akaris@linux tmp]$ helm package TRUE
Successfully packaged chart and saved it to: /tmp/TRUE-0.1.0.tgz


feat(mainnet.ts): add ceth and cwbtc

:tada: This PR is included in version 1.9.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Fix typos and improve wording in the documentation.

 :)

Don't forget to also update http://cppcms.com/files/nowide/html/


DataLossError: Unable to parse tensor from stored proto when loading saved large tf dataset.

Please make sure that this is a bug. As per our
GitHub Policy,
we only address code/doc bugs, performance issues, feature requests and
build/installation issues on GitHub. tag:bug_template
System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow):
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Win 10 x64
TensorFlow installed from (source or binary): pip
TensorFlow version (use command below): 2.4.0rc3
Python version: 3.8
CUDA/cuDNN version: 11, 8.0.2
GPU model and memory: RTX 3080 10GB

Code:
https://drive.google.com/file/d/1oIjXa-gH-g8SB4jGisu8nPxMd5TJgoDa/view?usp=sharing
Full error log:
---------------------------------------------------------------------------
DataLossError                             Traceback (most recent call last)
~\anaconda3\envs\hk2\lib\site-packages\tensorflow\python\eager\context.py in execution_mode(mode)
   2112       ctx.executor = executor_new
-> 2113       yield
   2114     finally:

~\anaconda3\envs\hk2\lib\site-packages\tensorflow\python\data\ops\iterator_ops.py in _next_internal(self)
    729     with context.execution_mode(context.SYNC):
--> 730       ret = gen_dataset_ops.iterator_get_next(
    731           self._iterator_resource,

~\anaconda3\envs\hk2\lib\site-packages\tensorflow\python\ops\gen_dataset_ops.py in iterator_get_next(iterator, output_types, output_shapes, name)
   2577     except _core._NotOkStatusException as e:
-> 2578       _ops.raise_from_not_ok_status(e, name)
   2579     except _core._FallbackException:

~\anaconda3\envs\hk2\lib\site-packages\tensorflow\python\framework\ops.py in raise_from_not_ok_status(e, name)
   6861   # pylint: disable=protected-access
-> 6862   six.raise_from(core._status_to_exception(e.code, message), None)
   6863   # pylint: enable=protected-access

~\anaconda3\envs\hk2\lib\site-packages\six.py in raise_from(value, from_value)

DataLossError: Unable to parse tensor from stored proto. [Op:IteratorGetNext]

During handling of the above exception, another exception occurred:

DataLossError                             Traceback (most recent call last)
<ipython-input-6-a2ce188c35b6> in <module>
----> 1 tds1_1 = next(iter(tds1_loaded.take(1)))

~\anaconda3\envs\hk2\lib\site-packages\tensorflow\python\data\ops\iterator_ops.py in __next__(self)
    745   def __next__(self):
    746     try:
--> 747       return self._next_internal()
    748     except errors.OutOfRangeError:
    749       raise StopIteration

~\anaconda3\envs\hk2\lib\site-packages\tensorflow\python\data\ops\iterator_ops.py in _next_internal(self)
    737         return self._element_spec._from_compatible_tensor_list(ret)  # pylint: disable=protected-access
    738       except AttributeError:
--> 739         return structure.from_compatible_tensor_list(self._element_spec, ret)
    740
    741   @property

~\anaconda3\envs\hk2\lib\contextlib.py in __exit__(self, type, value, traceback)
    129                 value = type()
    130             try:
--> 131                 self.gen.throw(type, value, traceback)
    132             except StopIteration as exc:
    133                 # Suppress StopIteration *unless* it's the same exception that

~\anaconda3\envs\hk2\lib\site-packages\tensorflow\python\eager\context.py in execution_mode(mode)
   2114     finally:
   2115       ctx.executor = executor_old
-> 2116       executor_new.wait()
   2117
   2118

~\anaconda3\envs\hk2\lib\site-packages\tensorflow\python\eager\executor.py in wait(self)
     67   def wait(self):
     68     """Waits for ops dispatched in this executor to finish."""
---> 69     pywrap_tfe.TFE_ExecutorWaitForAllPendingNodes(self._handle)
     70
     71   def clear_error(self):

DataLossError: Unable to parse tensor from stored proto.

@Crispy13
Unable to acess the code shared, please share colab gist with error reported and access to view it.

@Saduf2019
I'm sorry. I uploaded it to colab.
https://colab.research.google.com/drive/1LhN2VjBKkxev0T3hIbgvJq0nrkGJRY0A?usp=sharing

@ymodak
I ran the code shared  and colab crashes, please find the gist here.

I see that the colab crashes due OOM  and works if we reduce tensor size.

@Crispy13  Please let us know the update on this.Thanks!


add unit test for colored output

https://github.com/killercup/output-rs/issues/5

Never mind, I'll merge this as is and split up the tests later on.
Thanks again!


quick note on docstring parser stuff from slack

This is what aegis said on Slack about trying to parse docstrings; just passing it along in case relevant:
"We've talked about it briefly but there's definitely no standard for how talon docstrings should look yet"
"knausj doesn't really have magic docstrings that need parsing yet, and probably shouldn't do that until we come up with a standard for it"

I put the docstring parser in by request, so that we could try out that particular format :)


PR labeler version

Update the PR version number for new permissions.
PR Checklist

[ ] Give a detailed description of the PR above.
[ ] Document changes in the CHANGES.rst file. See existing changelog for examples.
[ ] Add tests, if applicable, to ensure code coverage never decreases.
[ ] Make sure the docs are up to date, if applicable, particularly the docstrings and RST files in docs folder.
[ ] Ensure linear history by rebasing, when requested by the maintainer.

Codecov Report

Merging #64 (f035764) into main (093a531) will not change coverage.
The diff coverage is n/a.

:exclamation: Current head f035764 differs from pull request most recent head 024345b. Consider uploading reports for the commit 024345b to get more accurate results

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
@@           Coverage Diff           @@
##             main      #64   +/-   ##
=======================================
  Coverage   80.62%   80.62%
=======================================
  Files          13       13
  Lines         449      449
=======================================
  Hits          362      362
  Misses         87       87


Fixes wrong context attribute usage and use node native execution to prevent cron scheduler issue

Hi,
with this PR I fixed the usage of the gitName context variable instead of gitUser, which isn't defined, and introduced a different way to do the commit: only if is necessary in order to avoid an execution fail if I use it with a cron scheduler

@ilteoood thanks for figuring out the issue!
It's been merged and released as v0.0.3-prerelease.


[Backend Config ~ Reference ~ DO NOT MERGE] move H5DataIO from add_* functions and methods to dataset configuration

The goal of this PR is to:

Remove H5DataIO from add_* functions and methods. These functions and methods should now only be used to assemble the data and metadata, but should be backend-agnostic.
Add configure_datasets method to BaseDataInterface

this PR is blocked by https://github.com/hdmf-dev/hdmf/pull/868

This can at last be closed as all relevant aspects used for reference have been merged


Query 7 failed due to using left semi join and be referenced in where clause.

Query 7 failed and the exception is as follows:
FAILED: SemanticException [Error 10004]: Line 26:26 Invalid table alias or column reference 'q07_cat_avg_price'
Left semi join is used in query 7 and is referenced in where clause. It seems that hive doesn't support this usage.
"The restrictions of using LEFT SEMI JOIN is that the right-hand-side table should only be referenced in the join condition (ON-clause), but not in WHERE- or SELECT-clauses etc."
https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Joins#LanguageManualJoins-JoinSyntax

Hi,
thanks for reporting.Seams we where to eager with trying to be more in line with the original syntax of the corresonding TPC-DS query. I reverted q7 back to a normal join.
best regards
Michael


FAKECHROOT_DIR_MAP

This isn't a udocker issue, but more of question on the use of FAKECHROOT_DIR_MAP for mapping directories into a container when using F* modes.
I'm wanting to use fakechroot directly and want to map directories into the chroot in the same fashion as udocker.
I have googled for info and only two results come up; both for this repository.
I've been through the sources of udocker, jorge-lip/libfakechroot-glibc-udocker and dex4er/fakechroot and cannot figure out how it works.
I've tried setting the FAKECHROOT_DIR_MAP environment variable and then running fakechroot ... on my Ubuntu 18.04 setup, but it doesn't work.
Running the same using udocker with F1 mode works. I put print statements in the udocker.py file to see the command line and environment so I could mimic it, albeit with the fakechroot command, but it doesn't work. I maintain the heroku-buildpack-r which would greatly benefit from this.
Any pointers and help would be greatly appreciated.

Nevermind, I've found it in the jorge-lip/libfakechroot-glibc-udocker repo under the udocker-1 branch.


Serialisation benchmarks?

Sorry, me again.
Would be amazing to add serialisation to the benchmarks for the libraries which support it, as well as loading.

What kind of api are we expecting, other than dump/dumps?

I was thinking just dumps. Please excuse me being dumb, if it's already available somewhere and I missed it?

No, you are not at all. The speed tests are currently just for load/loads.
Just not sure what data to be used for the tests. Maybe we just use the dicts loaded from the toml files that are currently used for speed testing for load/loads?

Yes, that's what I would do.

Sounds good to me.

That should guarantee that all types are tested. You could also do a round trip test to confirm libraries process data correctly in and out.

It's easy for the round trip from dict -> string -> dict, since dicts are relatively easier to compare in python, but tougher from string -> dict -> string due to the arbitrariness of strings as there could be comments and extra spaces.
We could probably do:
(toml files)
   string      ->       dict      ->    string     ->      dict
                         |                                  |
                         +----------- comparison -----------+

Yes, that's what I was thinking.
I guess you could compare toml string after the first loop when it shouldn't change anymore.

I guess you could compare toml strings after the first loop when it shouldn't change anymore.

I think this is more like a reproducibility test. The same dict should produce/be dumped into the same string.

True.

Dumping benchmarking is added.
As for the reproducibility test, let's see if we have a better reason to add it since the tests for None-value behavior were already included. Other inconsistencies that I can think of for now include float number precision, date time formatting, etc. But they are not essential for a toml parser for general use.
We can open another issue if we find those problems matter in more scenarios.


EVEREST-1070 replacing the restore icon

replaced the icon for the "replace to this DB" menu item

can be moved to upstream


Give users randomized stats

The stats would be various values from rps. Some that make sense, some that don't. Some would be specific to certain people. Random seed would be based on unique user id.
Thoughts @TheNickmaster21 @zjbullock @Adams880 @IAmBitfrost ?

Definitely would be interested to work on this!


Section on choosing the elements to be shown

I just wanted to show tracks and waypoints in my project and i took me a while to understand how to set this option. So I thought to include a little section to easily show how it is done.
Best regards, and thx for creating this project 👍

Thx! That was actually very helpfull!


Decompose addmm with Gemm | feat(torchlib)

Decompose addmm with Gemm by creating a special variant for FLOAT and conditionally check for the ranks if the input tensors. The if branch is expected to be folded away by constant folding passes.
I have not found other instances where Gemm is used in the torch.onnx exporter.
Fixes https://github.com/microsoft/onnxscript/issues/1089
cc @baijumeswani

aten::mm is also exported as Gemm by torchscript exporter. Although I'm not sure if it is better to export as Gemm or Matmul in this case.

aten::mm is also exported as Gemm by torchscript exporter. Although I'm not sure if it is better to export as Gemm or Matmul in this case.

Yeah Gemm seems to be more specialized for that case. Backends should just do the right thing for matmul imo


Use MongoDB's native TTL indexes for entity expiration

Current coverage is 85.76% (diff: 100%)

Merging #82 into master will increase coverage by 0.01%

@@             master        #82   diff @@
==========================================
  Files            58         58
  Lines          3739       3744     +5
  Methods           0          0
  Messages          0          0
  Branches        345        345
==========================================
+ Hits           3206       3211     +5
+ Misses          448        446     -2
- Partials         85         87     +2

Powered by Codecov. Last update ac0f0c7...a3f9973


Investigate Network Performance of Downloads

It just seems a lot slower than it should be. Perhaps this is the async? Perhaps the CSP module?

@sherl0cks combined download size is ~675MB. Depends on CDN performance

@sabre1041 Something is wrong ansible side. I'm getting order of magnitude longer downloads in the playbooks than I via the browser, and both are going through the CDN


hdf5_00 checksum calculation

Motivation and Context
Why is this change required? What problem does it solve?:
new hdf5 checksum calculation so we don't rely on fletcher32 which will not catch user edited dataset changes as mistakes upon read
Description
Describe your changes in detail:
using xxhash library and edited hdf5_00 backend format spec in incompatible method with previous repos.
Types of changes
What types of changes does your code introduce? Put an x in all the boxes that apply:

[ ] Documentation update
[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[x] Breaking change (fix or feature that would cause existing functionality to change)

Is this PR ready for review, or a work in progress?

[ ] Ready for review
[x] Work in progress

How Has This Been Tested?
Put an x in the boxes that apply:

[x] Current tests cover modifications made
[ ] New tests have been added to the test suite
[x] Modifications were made to existing tests to support these changes
[ ] Tests may be needed, but they are not included when the PR was proposed
[ ] I don't know. Help!

Checklist:

[x] My code follows the code style of this project.
[x] My change requires a change to the documentation.
[x] I have updated the documentation accordingly.
[x] I have read the CONTRIBUTING document.
[x] I have signed (or will sign when prompted) the tensorwork CLA.
[x] I have added tests to cover my changes.
[x] All new and existing tests passed.

There may be a major bug in the hdf5_00 backend because some of the checksums are failing on certain machines. I'll have to dig into this further. WIP. Do NOT merge.


Implement search based on database views for directory listing page

Jira Description
Basic searching capability for columns on the grid. It should hit the database for the query. It should either update or make use of vw_LeadershipProfileList view.
Jira Metadata
Key: LP-36
Epic: n/a
Created: 2021-01-29T14:36:55.000-0600
Reporter: Yogiraj Aradhye

Jira Comment
A new view was created to accommodate an advanced search feature where the client indicated a broader ability to search records outside the profile scope. vw_StaffSearch was implemented to accommodate the UI queries.
Jira Metadata
Created: 2021-03-24T01:00:28.850-0500
Commenter: Mark Harvey


9.0.8 latest poketracker error

Get this in console evertime someone does /lastlegend
https://pastebin.com/yNYtpdPc

Pixelmon bug


How do i actually switch keyboard layouts?

I love this WM probably the most of all and would love to actually use it, so far only two issues are actually problematic: right click tray menus in waybar are all over the place and behind the bar and i did not find a way to do this in config and fcitx5, which i used in Sway, does not work at all, even IN the app testing line
Is there a way to switch any other way?

both problems are documented in the wiki.
ibus: https://github.com/hyprwm/Hyprland/wiki/Uncommon-tips-&-tricks
waybar: https://github.com/hyprwm/Hyprland/wiki/FAQ


BZ1078112 - Move upload and refresh buttons into toolbar

Depends on https://github.com/uberfire/uberfire-extensions/pull/211

+1 to merge (test clearly passes.... and I wait to be educated)

Merged into 6.4.x


Experimental Details

Thanks for sharing your great work!
I have a question about your experimental details on CMU Panoptic dataset: which sequences do you train and test on?

train:
160224_haggling1
160226_mafia2
160224_mafia1
160224_mafia2
160224_ultimatum1
160224_ultimatum2
valid:
160422_mafia2
test sampled from：
160226_haggling1
160422_haggling1
160226_mafia1
160226_ultimatum1
160422_ultimatum1
160906_pizza1

Thanks for quick your response!
I download the dataset. However, some 3D annotations from activities are not provided, such as the activity of mafia, 160224_ultimatum1 and 160224_ultimatum2. How do you solve this problem?

This is because some annotations of Panoptic only support coco15, you can try to remove the suffix "coco19" of the download path.

Thanks!

train: 160224_haggling1 160226_mafia2 160224_mafia1 160224_mafia2 160224_ultimatum1 160224_ultimatum2
valid: 160422_mafia2
test sampled from： 160226_haggling1 160422_haggling1 160226_mafia1 160226_ultimatum1 160422_ultimatum1 160906_pizza1

Dear authors, do the training samples also come from the cameras 16 and 30?


SizedBox Default Size Based on Parent Widget

Use case
Currently, the SizedBox widget does not have any default size behavior when used within parent widgets like Column or Row. This can lead to inconsistencies in layout and make it more difficult to write reusable code.
Proposal
To improve the usability of the SizedBox widget, we propose adding a default size behavior based on the parent widget. Specifically, if the SizedBox is a child of a Column widget, it should have a default height of 16.0 pixels. Similarly, if the SizedBox is a child of a Row widget, it should have a default width of 16.0 pixels.
Benefits:

Reduced boilerplate code: Developers will not need to manually set the size of SizedBox widgets when used within Column or Row widgets, which can simplify their code and reduce the likelihood of errors.

Improved consistency: The default size behavior will ensure that SizedBox widgets have a consistent appearance when used within different parent widgets. This will make it easier for developers to write reusable code that looks and behaves consistently across different layouts.

Enhanced accessibility: The default size behavior will make it easier for assistive technologies to identify and interact with SizedBox widgets. This is important for ensuring that Flutter applications are accessible to users with disabilities.

The default size behavior could be implemented by adding a new constructor to the SizedBox widget that takes the parent widget as a parameter. This constructor would then set the appropriate default size based on the parent widget's type.
Conclusion:
We believe that adding a default size behavior to the SizedBox widget would be a valuable addition to the Flutter framework. This would improve the usability of the widget, reduce boilerplate code, and enhance the overall consistency and accessibility of Flutter applications.

Container does that so I don't think there is any need for this. If you wish to create a reusable SizedBox widget, you can do so and publish it on pub.dev.

@soltanzadehErfan
Check above comment / solution and see if it helps to answer your case or not.


make internal broadcast and unbroadcast both primitives

At @dougalm's suggestion, I took a stab at making our internal broadcast and unbroadcast functions into primitives. They seem to form a nice pair!
This might prevent graph expansion and be a bit faster, though I haven't actually run asv on this change yet.
Any thoughts on this first pass, @j-towns?

Yeah looks like a good idea. I'd been meaning to switch to using numpy's broadcast_to inside our broadcast for a while.

Also I feel like we should get the 'optional vspace checking' setup as a matter of priority so that we can rigorously test that these functions are outputting the correct thing.

Btw I think this change was inspired by your improvements to the dot VJPs.
Re: vspaces, I generally agree, though I'm thinking that if vspaces are primarily for testing, we should use them extensively in our testing code but not incur the costs at runtime for every grad eval.

Yeah cool I'm totally agreed that's the right approach re: vspaces
On Wed, 13 Sep 2017 at 16:48, Matthew Johnson<EMAIL_ADDRESS>wrote:

Btw I think this change was inspired by your improvements to the dot VJPs.
Re: vspaces, I generally agree, though I'm thinking that if vspaces are
primarily for testing, we should use them extensively in our testing code
but not incur the costs at runtime for every grad eval.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/HIPS/autograd/pull/292#issuecomment-329211253, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AOjguxl1x1F_UjUW09Y9fAc3olejst2pks5sh_lqgaJpZM4PU72r
.

I've effectively incorporated these changes into #312.


Image flicker when using fadeInTransition with Picture

Hey, firstly, thanks so much for making these image tools for Astro, so helpful :)
Hopefully I'm not missing something in the docs, but I am experiencing images flickering on load when using the fadeInTransition prop with the <Picture/> component. When I visit a page, an image fades in as expected, but when I then make a subsequent visit to the page (or refresh my browser), I briefly see the image for a short moment before it disappears and then fades in. I'm seeing this both locally and when deployed.
I can't diagnose exactly why this is happening, but I'm guessing it's something to do with the image being cached by the browser on the first request and then on subsequent visits to a page, that cached image is briefly being shown before it's hidden and faded in. Is it possible to prevent this from happening as it somewhat undermines the intent to always have a graceful fading in of images?
Cheers – happy to help dig in and try to diagnose this if I can be of use and I've attached a screen cap to show what I'm seeing and the <Picture /> code I'm using is below:
<Picture
    src={src}
    alt={alt}
    placeholder="dominantColor"
    format={["webp"]}
    quality={90}
    fadeInTransition={{
      delay: "0.0s",
      duration: "0.4s",
      timingFunction: "ease-in",
    }}
 />

https://user-images.githubusercontent.com/265403/195322171-69d5dbf1-11ac-4f15-ba37-cad923dc7e0e.mov

Did you ever find a workaround for this? I am also getting a flicker on page load

@LushawnDev No, I ended up giving up on this in favour of the image integration Astro are actively developing — https://docs.astro.build/en/guides/integrations-guide/image/. Think I'll enhance that with some custom JS of my own to do the fading etc.

Yeah I was actually thinking of doing the exact same, but using the new
Astro Assets feature which is going to be replacing their component.
The only thing I really wanted to keep from this integration is that it
generates all the different image sizes automatically, I need to find
another way to do that as it's so convenient.
On Mon, 22 May 2023, 22:47 Guy Moorhouse, @.***> wrote:

@LushawnDev https://github.com/LushawnDev No, I ended up giving up on
this in favour of the image integration Astro are actively developing —
https://docs.astro.build/en/guides/integrations-guide/image/. Think I'll
enhance that with some custom JS of my own to do the fading etc.
—
Reply to this email directly, view it on GitHub
https://github.com/RafidMuhymin/astro-imagetools/issues/130#issuecomment-1558066342,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/APCDDRXVKFNVJXGLIE62DFDXHPNGZANCNFSM6AAAAAARDFA3DA
.
You are receiving this because you were mentioned.Message ID:
@.***>


Automate /action or petition

Military/veterans sign the petition, but the data remains in the rails app and not automatically into the newsletter for the SITREPs.

We use mailchimp for our SITREPs. About 111 veterans are signed up. Need to automate data input and touch it once, not having to take the data from active-record and then manually input it into mailchimp. Was looking at this: https://github.com/mailchimp/email-blueprints.
Also, need to automate Tweets to U.S. Senators and U.S. Representatives based on veterans zipcode. Manually inputting this as well. Live tweet: https://twitter.com/operation_code/status/590256827067015168.

Ideally, we could use Change.org API and Twilio API allowing both civilians and military/veterans to sign the petition to expand New GI Bill to include code schools, locate their local representative, and contact them free of charge using Twilio.

I've done this before - building a nightly job which synced all data in a Rails app with Mailchimp, making heavy use of this awesome Mailchimp API wrapper https://github.com/amro/gibbon.  Gibbon makes it very easy to access the Mailchimp API from within a Rails application using Ruby.
Would be happy to integrate this as a first step, though eventually I will need the Mailchimp credentials to make this work.

Hi @nellshamrell- just sent you an invite. Thanks!

Got it.  TY!

@nellshamrell -- so this relates a bit to #44 . Total requirements:

automate signup between rails app and MailChimp (no need to manually input from database copy/paste to MailChimp)
automate signup between rails app and Slack (Software Mentor Protégé Program) Operation Code community
Integrate Change.org & Twilio API to automate "traditional letter to Congressman" based on service-member or veterans zip code (from their response in the rails app)
Auto tweet based on service-member or veterans zip code response vs. manually having to input every time (Change.org & Twilio API should solve this). Example tweet: https://twitter.com/operation_code/status/591029490798714880.
Thanks @nellshamrell!

I think I'll take on the automate signup between the rails app and MailChimp first, submit a pull request, get that deployed in, then move onto the next requirement (though if anyone else feels like working on the other requirements while I'm working on the Mailchimp piece please feel free!)

How I imagine the Mailchimp aspect of this working:

How I imagine the Mailchimp part of this working:

@nellshamrell -- perfect!

Mailchimp is currently transitioning from v2.0 of their API to v3.0.  Documentation on what methods we should use to subscribe veterans through the API is not clear, pinged Mailchimp over twitter for clarification on which API we should use.

Thanks for the heads up :+1:

Thanks Nell for the heads up.
On Sun, May 10, 2015 at 8:52 PM, Fernando Paredes<EMAIL_ADDRESS>wrote:

Thanks for the heads up [image: :+1:]
--
Reply to this email directly or view it on GitHub
https://github.com/OperationCode/operationcode/issues/22#issuecomment-100756174
.

--
David
http://davidmolina.github.io/ | t. @davidcmolina
https://twitter.com/davidcmolina

Updating:
We can indeed use the Mailchimp v3.0 (got a response back from the Mailchimp API team.  Although Gibbon, a great wrapper for use when calling the Gibbon API from Ruby code, currently does not support v3.0, the maintainer is working on updating it now (I'm going to be helping him test it when it is ready).
I would recommend waiting until Gibbon is updated.  We could theoretically call the Mailchimp API directly...but that would be messy and difficult to maintain.  The maintainer of Gibbon has been very responsive both in maintenance of Gibbon and responding to any questions or issues with it.

@nellshamrell I'm all for waiting. No need to do more work than necessary

Awesome. Let's hold off.
On Tue, May 12, 2015 at 7:50 PM, Fernando Paredes<EMAIL_ADDRESS>wrote:

@nellshamrell https://github.com/nellshamrell I'm all for waiting. No
need to do more work than necessary
--
Reply to this email directly or view it on GitHub
https://github.com/OperationCode/operationcode/issues/22#issuecomment-101488364
.

--
David
http://davidmolina.github.io/ | t. @davidcmolina
https://twitter.com/davidcmolina

Be great to tie in our Twilio.org account and send vets a text (https://twitter.com/robertlenne/status/607901970733658112).

What would you want the texts to say? I can implement twilio pretty easily.

Maybe it's three part: (1) print letter to the veteran's or civilian
supporters member of Congress using the Change.org API, (2) a call to that
office or text?, and (3) an automatic tweet to that veteran's or civilian
supporters Member of Congress and x2 U.S. Senators. Example tweet:
https://twitter.com/operation_code/status/589545850126368769
On Tue, Jun 16, 2015 at 12:53 PM, Glenn Espinosa<EMAIL_ADDRESS>wrote:

What would you want the texts to say? I can implement twilio pretty
easily.
—
Reply to this email directly or view it on GitHub
https://github.com/OperationCode/operationcode/issues/22#issuecomment-112549979
.

--
David
t. @davidcmolina https://twitter.com/davidcmolina
http://davidmolina.github.io/


Can't build 0.12 - Error

Hi, I have a problem building the last version with 17.5. I tried ./rebar3 upgrade, but I still can't solve the problem. Thx in advance for any help you are able to provide :-)
===> Upgrading cuttlefish
===> Failed to fetch and copy dep: {git,
"git://github.com/tsloughter/cuttlefish.git",
{ref,
"f5f6fdc1b0dd8ebe909b81f0b38f5715d4e869e6"}}

Hi Simone, thanks for reporting!
./rebar3 upgrade will try to upgrade all deps, which is not what you'll want to do, because lock the dependencies per release. (see rebar.lock).
But the reason why it failed to just fetch cuttlefish must be somewhere else.
@dergraf any more thoughts?

Difficult to say from here.. But 'upgrade' is not the needed here.
Cheers

To me it looks like you're either having network problems or the network you're on is restricted. I get the very same error if I try to build VerneMQ on a network that doesn't allow the port (9418 I think) for the git protocol through.
Can you try on another network?

a similiar issue when running #make rel on my macbook
─$ make rel                                                                                1 ↵
/Users/jgomes/Repositories/Github/vernemq/rebar3 release --overlay_vars vars.config
===> Verifying dependencies...
===> Fetching cuttlefish ({git,
"git://github.com/tsloughter/cuttlefish.git",
{ref,
"f5f6fdc1b0dd8ebe909b81f0b38f5715d4e869e6"}})
===> Failed to fetch and copy dep: {git,
"git://github.com/tsloughter/cuttlefish.git",
{ref,
"f5f6fdc1b0dd8ebe909b81f0b38f5715d4e869e6"}}
make: *** [rel] Error 1

@ejaigom can you verify if you can telnet to github.com at port 9418:
If your network allows the git protocol/port 9418 it should look like this
$ telnet github.com 9418
Trying <IP_ADDRESS>...
Connected to github.com.
Escape character is '^]'.

If not it looks like this:
$ telnet github.com 9418
Trying <IP_ADDRESS>...
telnet: Unable to connect to remote host: Connection refused

In which case you should try to again from a network with less restrictions.
@dergraf do you see an issue if we change our dependencies to use https instead of git? The likelihood of https being blocked on a network is probably much lower than git being blocked...

@ejaigom @Reddone I've now seen the same error on a network without restrictions and tracked it down to a problem with the cuttlefish dependency/git repo. We've upgraded the cuttlefish dependency which removes the problem for us, so please try and fetch vernemq from master and see if you still have the problem. This fix will of course be included in the next official release.

@larshesel yeap, It works!

yeah, it works. Thank you very much!!!

Thanks for confirming the fix!


[HUDI-6588] Fix duplicate fileId on TM partial-failover and recovery

Change Logs
https://github.com/apache/hudi/issues/8892
Risk level (write none, low medium or high below)
Documentation Update
Contributor's checklist

[x] Read through contributor's guide
[x] Change Logs and Impact were stated clearly
[ ] Adequate tests were added if applicable
[ ] CI passed

@hudi-bot run azure

Thanks for the contribution, can you explain what the purpose of the PR @big-doudou ?

Thanks for the contribution, can you explain what the purpose of the PR @big-doudou ?

https://github.com/apache/hudi/issues/8892#issuecomment-1632159235

@danny0405   Can you help review this pr, this error can be reproduced 100%, please check for details https://github.com/apache/hudi/issues/8892#issuecomment-1632159235

Flink sink hudi uses bucket index. If the amount of data between checkpoints is relatively large, part of the data will be flushed to hdfs first, and a file ID will be generated at this time. If the TM restarts abnormally before the checkpoint is completed, this code will judge flink job partial-failover and recovery, and Bootstrap() will not be executed. Therefore, the previously generated instant is reused, and the old log file will not be cleaned up, resulting in duplicate file ids in the same bucket.

I think this piece of code at that time was to solve the problem of instant!=null blocking writing during fault recovery, but it ignored the special case of repeating the file id under the bucket index

I think this piece of code at that time was to solve the problem of instant!=null blocking writing during fault recovery, but it ignored the special case of repeating the file id under the bucket index

I think this piece of code at that time was to solve the problem of instant!=null blocking writing during fault recovery, but it ignored the special case of repeating the file id under the bucket index

Maybe you are right, but in StreamWriteOperatorCoordinator#subtaskFailed, we would anyway cleans the write metadata of the failed task, maybe it is because the event does not got received by the coordinator ?
The #subtaskFailed is invoked right before a new attempt got scheduled.

I think this piece of code at that time was to solve the problem of instant!=null blocking writing during fault recovery, but it ignored the special case of repeating the file id under the bucket index

Maybe you are right, but in StreamWriteOperatorCoordinator#subtaskFailed, we would anyway clean the write metadata of the failed task, maybe it is because the event does not got received by the coordinator ?
The #subtaskFailed is invoked right before a new attempt got scheduled.
How to clean the write metadata in StreamWriteOperatorCoordinator#subtaskFailed
Is it the behavior of OperatorCoordinator after receiving BootstrapEvent?

I think this piece of code at that time was to solve the problem of instant!=null blocking writing during fault recovery, but it ignored the special case of repeating the file id under the bucket index

Maybe you are right, but in StreamWriteOperatorCoordinator#subtaskFailed, we would anyway clean the write metadata of the failed task, maybe it is because the event does not got received by the coordinator ?
The #subtaskFailed is invoked right before a new attempt got scheduled.

How to clean the write metadata in StreamWriteOperatorCoordinator#subtaskFailed
Is it the behavior of OperatorCoordinator after receiving BootstrapEvent?

Each failed attempt of a subtask would trigger invocation of StreamWriteOperatorCoordinator#subtaskFailed, the original write metadata would got cleaned,

Each failed attempt of a subtask would trigger invocation of StreamWriteOperatorCoordinator#subtaskFailed, the original write metadata would got cleaned,

The StreamWriteOperatorCoordinator#subtaskFailed just set eventBuffer=null. How does this affect metadata cleaning?

How does this affect metadata cleaning?

It removes the preceeding partial metadata if there is any.

failover causes the bootstrap event not to be sent

Why we must send the bootstrap event in this case, the bootstrap event itself is a empty event without any metadata.

failover causes the bootstrap event not to be sent

Why we must send the bootstrap event in this case, the bootstrap event itself is a empty event without any metadata.

instantA is not commit, The instantA.log file has not been cleaned up
send the bootstrap event to clean up these files and metadata

send the bootstrap event to clean up these files and metadata

The log files are actually cleaned by the cleaner/rollback procedure, senind the bootstrap event does not work as you expected. Before these log files are removed, they are actually invisible to the reader/view(or there should be a bug).

StreamWriteOperatorCoordinator#handleBootstrapEvent() -> initInstant() -> startInstant() -> this.writeClient.startCommit() -> tableServiceClient.rollbackFailedWrites() -> rollbackFailedWrites() -> rollback()
so TM need send bootstrap event to rollback old instant.Otherwise, two files with the same bucket id will be generated under the current partition

StreamWriteOperatorCoordinator#handleBootstrapEvent() -> initInstant() -> startInstant() -> this.writeClient.startCommit() -> tableServiceClient.rollbackFailedWrites() -> rollbackFailedWrites() -> rollback() so TM need send bootstrap event to rollback old instant.Otherwise, two files with the same bucket id will be generated under the current partition

Only global failover would trigger the rollback, partial failover would not.

Maybe you are right, if the partial failover will not be rolled back, how should these log files be cleaned up. Duplicate bucket id files cause tasks to fail to start all the time.

how should these log files be cleaned up. Duplicate bucket id files cause tasks to fail to start all the time

The log expected to be cleaned when the instant is committed (we have a marker machanism to ensure the retried files got cleaned), then issue here is why these partitial files are visible to the BucketStreamWriter, that's the direction we should dig into.

how should these log files be cleaned up. Duplicate bucket id files cause tasks to fail to start all the time

The log expected to be cleaned when the instant is committed (we have a marker machanism to ensure the retried files got cleaned), then issue here is why these partitial files are visible to the BucketStreamWriter, that's the direction we should dig into.
See BaseHoodieWriteClient#commit (line 279), the finalizeWrite would clean up the retried files based on markers.
In BucketStreamWriteFunction:line 160, you can debug whether these intermediate logs are visible to the function view getLatestFileSlices, if you have a existing table to test this, would be helpful.

Thank you for such a detailed answer.  let me test it.


Workflows

Expanding your app to allow users to select multiple workflows, bookmark them, define their order, and establish workflow dependencies is a powerful feature. Here's a plan on how to implement this:
Enhanced Workflow Management

Selecting Multiple Workflows:

Add a feature that allows users to select multiple workflows from a list. You can use checkboxes next to each workflow for selection.

Bookmarking Workflows:

Provide an option for users to bookmark selected workflows for quick access later.

Defining Workflow Order:

Allow users to define the order in which the selected workflows will be executed. This order can be defined by dragging and dropping the selected workflows into a sequence.

Workflow Dependencies:

Enable users to establish dependencies between workflows. This means that the output of one workflow can serve as the input for another.
Implement a visual interface that allows users to connect workflows, indicating their dependencies. For example, you can use arrows or lines to show the flow of data between workflows.

Executing Multiple Workflows:

Implement a button or action that triggers the execution of the selected workflows in the defined order.
Display the results of each workflow, and if a workflow depends on the output of a previous one, ensure that the data flows correctly.

Viewing and Managing Bookmarks:

Create a dedicated section where users can view and manage their bookmarked workflows. This section should include options to execute, edit, or remove bookmarked workflows.

Saving Workflow Configurations:

Allow users to save their workflow configurations for future use. This can be in the form of presets or saved workflow sequences.

User Guidance:

Include tooltips or a help section to guide users on how to use these new features effectively.

By implementing these enhancements, your app will provide users with a more advanced and flexible workflow management system, empowering them to combine, order, and execute workflows as needed for their creative processes. Make sure to maintain a clean and intuitive user interface to ensure a smooth user experience.

Certainly, you can use query parameters to include data flow descriptions within your web application. Query parameters are commonly used to pass information between web pages and applications. To incorporate data flow descriptions into query parameters, you can follow these general steps:

Define a Query Parameter Structure: Determine the structure of query parameters that will represent your data flow descriptions. Consider using key-value pairs or other formats that suit your needs.

Encode Data Flow Information: Encode the relevant data flow information into the query parameters. This could include workflow details, data sources, transformation steps, or any other relevant data.

Generate URLs: Generate URLs that include the encoded query parameters. These URLs will be used to trigger specific actions or workflows within your application.

Parse Query Parameters: In your web application, implement logic to parse the query parameters from incoming URLs. Extract the encoded data flow descriptions from these parameters.

Execute Actions: Based on the extracted data flow descriptions, execute the corresponding actions or workflows within your application.

Handle Errors: Implement error handling to deal with cases where query parameters are missing or malformed.

Here's a simplified example of how you might structure query parameters to include data flow descriptions:
https://example.com/app?workflow=workflow1&source=data_source1&transform=transform_step1,transform_step2&output=output1

In this example:

workflow specifies the workflow to be executed.
source indicates the data source to be used.
transform lists the transformation steps to be applied.
output defines the output destination.

You can customize this structure to match the specific requirements of your application and the level of detail you need for data flow descriptions.
Remember to ensure that your web application properly encodes and decodes query parameters to handle special characters and maintain data integrity.


API Build new ManifestCache based on PSR-16 SimpleCache

Rebased on top of https://github.com/silverstripe/silverstripe-framework/pull/6693
Fixes https://github.com/silverstripe/silverstripe-framework/issues/6647
Merge with:

[ ] https://github.com/silverstripe/silverstripe-config/pull/8 (requires tag before framework CI will pull in changes)
[ ] https://github.com/silverstripe/silverstripe-admin/pull/2
[ ] https://github.com/silverstripe/silverstripe-cms/pull/1762

The main reason for the size of this change is that this change includes additional consolidation of getModules() duplicated between ClassManifest and ModuleManifest. I've remove this from ClassManifest, and added full tests in ModuleManifestTest.
I've updated various other classes, such as i18nTextCollector, to use the new Module manifest, and remove instances of the legacy module registration system.
I've noticed significant performance increases after switching to the optimised ManifestCacheFactory. No official benchmarks as of yet, but it is observably faster. :D

OK - needs rebasing

Ignore that review - it was for 95a48aa only

Rebased.

Rebased again.

test failures related?

No, config module was moved from PSR-6 to PSR-16 cache, so we can't cleanly run CI until everything is merged.

ah - ok

Merged the config patch and tagged as 1.0.0-alpha7, re-running tests now.

@tractorcow is there a patch to behat-extension too?

Fatal error: Call to undefined method SilverStripe\Core\Manifest\ClassManifest::getModules() in
/vendor/silverstripe/behat-extension/src/SilverStripe/BehatExtension/Console/Processor/LocatorProcessor.php
on line 64

I'm merging this, as the merge of config broken master installs.

@tractorcow please push a behat PR asap

PR for behat at https://github.com/silverstripe/silverstripe-behat-extension/pull/149


[BUG] - Error: UPGRADE FAILED: YAML parse error on fleet/templates/deployment.yaml: error converting YAML to JSON: yaml: line 27: did not find expected key

Rancher Server Setup

Rancher version: rancher/rancher:v2.7.0
Installation option (Docker install/Helm Chart):

If Helm Chart, Kubernetes Cluster and version (RKE1, RKE2, k3s, EKS, etc): RKE1

Proxy/Cert Details: Nginx as a Reverse proxy and wildcard

Information about the Cluster

Kubernetes version: 1.24.8
Cluster Type (Local/Downstream): Imported

User Information

What is the role of the user logged in? Admin

Describe the bug
After we've upgraded our previous rancher version (2.6.9) to the newest version (2.70) with the following command:
helm upgrade rancher rancher-stable/rancher --namespace cattle-system --set hostname=rancher-ouze.calcard.com.br --version 2.7.0 --set ingress.tls.source=secret --set privateCA=true
We're experiencing a lot of this "helm operations" with the following error message.
helm upgrade --force-adopt=true --history-max=5 --install=true --namespace=cattle-fleet-system --reset-values=true --timeout=5m0s --values=/home/shell/helm/values-fleet-101.0.0-up0.5.0.yaml --version=101.0.0+up0.5.0 --wait=true fleet /home/shell/helm/fleet-101.0.0-up0.5.0.tgz
Error: UPGRADE FAILED: YAML parse error on fleet/templates/deployment.yaml: error converting YAML to JSON: yaml: line 27: did not find expected key

To Reproduce
Execute the command:
helm upgrade rancher rancher-stable/rancher --namespace cattle-system --set hostname=rancher-ouze.calcard.com.br --version 2.7.0 --set ingress.tls.source=secret --set privateCA=true
Result
helm upgrade --force-adopt=true --history-max=5 --install=true --namespace=cattle-fleet-system --reset-values=true --timeout=5m0s --values=/home/shell/helm/values-fleet-101.0.0-up0.5.0.yaml --version=101.0.0+up0.5.0 --wait=true fleet /home/shell/helm/fleet-101.0.0-up0.5.0.tgz
Error: UPGRADE FAILED: YAML parse error on fleet/templates/deployment.yaml: error converting YAML to JSON: yaml: line 27: did not find expected key
**Expected Result**

Screenshots

Logs:
`kubectl logs -f rancher-5d94d9b74f-4kx5q -n cattle-system
``2023/01/10 17:47:04 [ERROR] Failed to install system chart fleet: failed to install , pod cattle-system/helm-operation-xpg2g exited 123`

Any news?

We are facing the same after updating Rancher from 2.6.9 to 2.6.10. Every some minute, there is a broken Hem operation with this log in the Helm container:
helm  upgrade --force-adopt=true --history-max=5 --install=true  --namespace=cattle-fleet-system --reset-values=true --timeout=5m0s  --values=/home/shell/helm/values-fleet-100.2.0-up0.5.1.yaml  --version=100.2.0+up0.5.1 --wait=true fleet  /home/shell/helm/fleet-100.2.0-up0.5.1.tgz

any news, we want to fix rancher with security release 2.6.10.

There is an error in the deployment.yml of the fleet upgrade. See also Issue #1166
For me, this workaround avoided the execution of the wrong yml code :
The fleet upgrade works and the error vanishes if the feature "continuous-delivery (Gitops components in fleet)" is activated in the global Rancher settings. This triggers the change of the value to true and prevents the error.

After upgrading to Rancher 2.7.1, I still have the same issue.
`

--
1/31/2023 22:32:12 | SUCCESS: helm upgrade --history-max=5 --install=true --namespace=cattle-fleet-system --timeout=10m0s --values=/home/shell/helm/values-fleet-crd-101.0.1-up0.5.1.yaml --version=101.0.1+up0.5.1 --wait=true fleet-crd /home/shell/helm/fleet-crd-101.0.1-up0.5.1.tgz
1/31/2023 22:32:12 | ---------------------------------------------------------------------
1/31/2023 22:32:12 | helm upgrade --history-max=5 --install=true --namespace=cattle-fleet-system --timeout=10m0s --values=/home/shell/helm/values-fleet-101.0.1-up0.5.1.yaml --version=101.0.1+up0.5.1 --wait=true fleet /home/shell/helm/fleet-101.0.1-up0.5.1.tgz
1/31/2023 22:32:12 | Error: UPGRADE FAILED: YAML parse error on fleet/templates/deployment.yaml: error converting YAML to JSON: yaml: line 27: did not find expected key

`

I've downgraded the fleet from cattle-fleet-system to fleet:0.3.900+up0.3.9 as a Workaround and still have the same issue (I lot of helm failed operations).

I removed the fleet helm as a terrible workaround.

I really need to activate this feature to get rid off the error? As a adminstrator for various clusters managed by rancher I don't want to disable all not must needed features for my users

@sass1997 , This was the only workaround I find to solve the problem.

@sass1997 , This was the only workaround I find to solve the problem.

Was more a question to the general scenario. Many thanks for sharing the workaround :)