Custom improvements/changes to SC2, maybe of interest for v3?

I work on a larger game and we added some improvements mainly to increase speed, Main changes are:

Reduce the number of clone() calls on passage change. For maxHistory = 1 vanilla SC makes 3 clone() calls, we got it down to 1. No idea if this is possible for higher maxHistory, though I am pretty sure at least one clone() could be removed.
Disabled save compression for auto saves
Increase the loading speed of the saves menu by storing each slot separately and store an extra object holding the saves description so only that needs to be loaded to display the saves menu.

Not sure if any of this interests you or how far along you are with v3 and it's only really important on games with a large state.
In any case, link to our repo if interested: https://gitgud.io/Arkerthan/sugarcube-2
There's also a list of all our changes in the README.

Cloning

Reduce the number of clone() calls on passage change. For maxHistory = 1 vanilla SC makes 3 clone() calls, we got it down to 1. No idea if this is possible for higher maxHistory, though I am pretty sure at least one clone() could be removed.

Removing the instance from historyDeltaEncode() should probably be okay with the codebase as it currently stands.
Removing the instance from momentCreate() means that references are preserved through navigation, which is a problem—in general, at least.  If any of code, even accidentally, begins to depend on that behavior, then it will break on session/save load as references cannot currently be preserved when roundtripping through serialization.  You may be able to make that tradeoff for your project, but it's not something that I can, or should, force on everyone.

Save compression

Disabled save compression for auto saves

Synchronous browser storage will likely always be a very limited commodity—certainly Web Storage is likely to be.  Trading space usage for computation time may be the right tradeoff for your project, but I think I'll stick with space efficiency for now.
As point of consideration.  You may be better served by dropping the delta coding step altogether, rather than compression on the autosave.  No idea on the how that would play out for your project, but that may be a better tradeoff.
NOTE: I'm unsure how you're providing access to your project, but as an FYI.  If you're giving players the files for local play, then you should know that different browsers handle local Web Storage differently.  E.g., Firefox gives each path its own storage instance, while Chrome—probably all Blink-based browsers—gives you a single storage instance for all local play.
I described this in more detail once at tfgames.site, relevant parts quoted below:

Web Storage instances are segmented by origin and some browsers differ in how the file:// schema demarcates origins.
Chrome does work as you've described, one origin for local files—most, if not all, browsers based on Chromium probably do.
Not sure about Safari, but it's likely in the Chromium camp.
Firefox is probably the odd duck out.  It treats the file:// schema similar to how it treats the http:// schema, specifically it grants separate origins based on the full directory path—i.e., file:///C:/games/alfa/game.html and file:///C:/games/bravo/game.html have separate origins, thus separate Web Storage instances.

The gist here is that Chrome users, if playing locally, are more likely to need the space savings—even if it's only on the one autosave, since your sharing with anything else they're storing locally.  Your players may never run into trouble because of it, but I'd keep an eye out just in case.

Save manifest

Increase the loading speed of the saves menu by storing each slot separately and store an extra object holding the saves description so only that needs to be loaded to display the saves menu.

Having separate manifest and data objects for saves isn't a bad idea.  The current system is, mostly, a holdover from v1, when developers' aspirations were more modest.

SugarCube v3

Not sure if any of this interests you or how far along you are with v3 and it's only really important on games with a large state.

Per v3:

How game state is recorded is changing significantly, so cloning won't be a concern in the future.
While plans are to use IndexedDB as the primary storage provider, compression is likely to remain—most people aren't playing on potatoes, so the computational hit shouldn't be an issue; especially in light of the state changes.
Saves are also changing, somewhat because of state changes, but I do like the idea of having a separate manifest so I'll probably end up doing something to that effect.

Cloning

Reduce the number of clone() calls on passage change. For maxHistory = 1 vanilla SC makes 3 clone() calls, we got it down to 1. No idea if this is possible for higher maxHistory, though I am pretty sure at least one clone() could be removed.

Removing the instance from historyDeltaEncode() should probably be okay with the codebase as it currently stands.
Removing the instance from momentCreate() means that references are preserved through navigation, which is a problem—in general, at least.  If any of code, even accidentally, begins to depend on that behavior, then it will break on session/save load as references cannot currently be preserved when roundtripping through serialization.  You may be able to make that tradeoff for your project, but it's not something that I can, or should, force on everyone.

Save compression

Disabled save compression for auto saves

Synchronous browser storage will likely always be a very limited commodity—certainly Web Storage is likely to be.  Trading space usage for computation time may be the right tradeoff for your project, but I think I'll stick with space efficiency for now.
As point of consideration.  You may be better served by dropping the delta coding step altogether, rather than compression on the autosave.  No idea on the how that would play out for your project, but that may be a better tradeoff.
NOTE: I'm unsure how you're providing access to your project, but as an FYI.  If you're giving players the files for local play, then you should know that different browsers handle local Web Storage differently.  E.g., Firefox gives each path its own storage instance, while Chrome—probably all Blink-based browsers—gives you a single storage instance for all local play.
I described this in more detail once at tfgames.site, relevant parts quoted below:

Web Storage instances are segmented by origin and some browsers differ in how the file:// schema demarcates origins.
Chrome does work as you've described, one origin for local files—most, if not all, browsers based on Chromium probably do.
Not sure about Safari, but it's likely in the Chromium camp.
Firefox is probably the odd duck out.  It treats the file:// schema similar to how it treats the http:// schema, specifically it grants separate origins based on the full directory path—i.e., file:///C:/games/alfa/game.html and file:///C:/games/bravo/game.html have separate origins, thus separate Web Storage instances.

The gist here is that Chrome users, if playing locally, are more likely to need the space savings—even if it's only on the one autosave, since your sharing with anything else they're storing locally.  Your players may never run into trouble because of it, but I'd keep an eye out just in case.

Save manifest

Increase the loading speed of the saves menu by storing each slot separately and store an extra object holding the saves description so only that needs to be loaded to display the saves menu.

Having separate manifest and data objects for saves isn't a bad idea.  The current system is, mostly, a holdover from v1, when developers' aspirations were more modest.

SugarCube v3

Not sure if any of this interests you or how far along you are with v3 and it's only really important on games with a large state.

Per v3:

How game state is recorded is changing significantly, so cloning won't be a concern in the future.
While plans are to use IndexedDB as the primary storage provider, compression is likely to remain—most people aren't playing on potatoes, so the computational hit shouldn't be an issue; especially in light of the state changes.
Saves are also changing, somewhat because of state changes, but I do like the idea of having a separate manifest so I'll probably end up doing something to that effect.

Posting this here since I don't really want to create accounts at every site who feels the need to reinvent the damn login wheel.
Here's a tip or two for other changes in your custom version.
→ Ignore expired array
If your project doesn't use anything that uses the expired moment array—i.e., the visited family of functions or State.hasPlayed()—then I'd simply comment out/remove that whole if statement in stateMarshal().  There's no point in adding an .expired property at all if it's always going to be an empty array.
		// I.e., comment out or remove this whole `if` statement block.
		if (_expired.length > 0) {
			stateObj.expired = [..._expired];
		}

I'd also find the following, in the historyCreate() function:
		if (Config.history.maxStates > 0) {
			while (historySize() > Config.history.maxStates) {
				_expired.push(_history.shift().title);
			}
		}

And make the following change:
		if (Config.history.maxStates > 0) {
			while (historySize() > Config.history.maxStates) {
				_history.shift();
			}
		}

I.e., do shift expired moments off the history array, but do not append them to the expired moments array.  Basically, there's no need to push moment passage names onto the expired array at all if you're only going to discard them later.
It would be cleaner to remove all expired moment related code, but the above will get you the benefits without having to muck around with anything else.

→ Use render() instead of a local wikifier, making it possible to render DOM...

Use render() instead of a local wikifier, making it possible to render DOM directly in special passages

I assume that I'm looking at the current <Passage>.render() method from the fc branch.  If not, then this may be off.
You've replaced a call to the Wikifier with a call to <Passage>.render(), which itself calls the Wikifier.  The balance of the changes being something like the following.
Original code:

Trims the processed text.
Renders directly to the page element.

Changed code:

Does not trim the processed text.
Creates a document fragment and renders to that, only later appending its contents to the page element.
Checks an if conditional—I'm assuming that it will always fail in your case, so you aren't also updating the excerpt.

To illustrate what I mean, you went from this:
				new Wikifier(el, Story.get(ids[i]).processText().trim());

To essentially this:
				const frag = document.createDocumentFragment();
				new Wikifier(frag, Story.get(ids[i]).processText());

				if (Config.passages.descriptions == null) {
					this._excerpt = Passage.getExcerptFromNode(frag);
				}

				el.append(frag);

That seems like a net loss to me, so I'm unsure how that's a good change for any project, let alone yours.  The only possibly non-negative change there is the dropping of the trim, and there are probably better ways to get that if that's what you need—though I'd probably suggest checking why that's a necessity and fixing that.

Posting this here since I don't really want to create accounts at every site who feels the need to reinvent the damn login wheel.
Here's a tip or two for other changes in your custom version.
→ Ignore expired array
If your project doesn't use anything that uses the expired moment array—i.e., the visited family of functions or State.hasPlayed()—then I'd simply comment out/remove that whole if statement in stateMarshal().  There's no point in adding an .expired property at all if it's always going to be an empty array.
		// I.e., comment out or remove this whole `if` statement block.
		if (_expired.length > 0) {
			stateObj.expired = [..._expired];
		}

I'd also find the following, in the historyCreate() function:
		if (Config.history.maxStates > 0) {
			while (historySize() > Config.history.maxStates) {
				_expired.push(_history.shift().title);
			}
		}

And make the following change:
		if (Config.history.maxStates > 0) {
			while (historySize() > Config.history.maxStates) {
				_history.shift();
			}
		}

I.e., do shift expired moments off the history array, but do not append them to the expired moments array.  Basically, there's no need to push moment passage names onto the expired array at all if you're only going to discard them later.
It would be cleaner to remove all expired moment related code, but the above will get you the benefits without having to muck around with anything else.

→ Use render() instead of a local wikifier, making it possible to render DOM...

Use render() instead of a local wikifier, making it possible to render DOM directly in special passages

I assume that I'm looking at the current <Passage>.render() method from the fc branch.  If not, then this may be off.
You've replaced a call to the Wikifier with a call to <Passage>.render(), which itself calls the Wikifier.  The balance of the changes being something like the following.
Original code:

Trims the processed text.
Renders directly to the page element.

Changed code:

Does not trim the processed text.
Creates a document fragment and renders to that, only later appending its contents to the page element.
Checks an if conditional—I'm assuming that it will always fail in your case, so you aren't also updating the excerpt.

To illustrate what I mean, you went from this:
				new Wikifier(el, Story.get(ids[i]).processText().trim());

To essentially this:
				const frag = document.createDocumentFragment();
				new Wikifier(frag, Story.get(ids[i]).processText());

				if (Config.passages.descriptions == null) {
					this._excerpt = Passage.getExcerptFromNode(frag);
				}

				el.append(frag);

That seems like a net loss to me, so I'm unsure how that's a good change for any project, let alone yours.  The only possibly non-negative change there is the dropping of the trim, and there are probably better ways to get that if that's what you need—though I'd probably suggest checking why that's a necessity and fixing that.

Big thanks for answering in such detail and going through our changes.

Browser handle local storage differently.

Really interesting, thanks.
The only way to play our game is with local play. While we support all modern browsers, we actually recommend Firefox because it is possible to increase local storage limits. 500kB+ (compressed!) saves are relatively common. At that point we decided it's more important to make auto saves happen at a decently fast speed over space savings. Firefox behaving differently with local storage didn't factor in that decision though, might make sense to reevaluate this.

There's no point in adding an .expired property at all if it's always going to be an empty array.

Thanks for the tip, wasn't around when it was done, just copied it over to a proper repository instead of a big patch file, so I didn't actually know what it did.

I assume that I'm looking at the current <Passage>.render() method from the fc branch. If not, then this may be off.

We extend the Passage class and overwrite render() so it simply executes a DOM generating function and returns that. This means we are not creating any Wikifier. So yes, only makes sense for our project.

Big thanks for answering in such detail and going through our changes.

Browser handle local storage differently.

Really interesting, thanks.
The only way to play our game is with local play. While we support all modern browsers, we actually recommend Firefox because it is possible to increase local storage limits. 500kB+ (compressed!) saves are relatively common. At that point we decided it's more important to make auto saves happen at a decently fast speed over space savings. Firefox behaving differently with local storage didn't factor in that decision though, might make sense to reevaluate this.

There's no point in adding an .expired property at all if it's always going to be an empty array.

Thanks for the tip, wasn't around when it was done, just copied it over to a proper repository instead of a big patch file, so I didn't actually know what it did.

I assume that I'm looking at the current <Passage>.render() method from the fc branch. If not, then this may be off.

We extend the Passage class and overwrite render() so it simply executes a DOM generating function and returns that. This means we are not creating any Wikifier. So yes, only makes sense for our project.


fix4

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.Preetam Dwivedi seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.

[!WARNING]
This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

Current dependencies on/for this PR:

#277  👈
#275 : 1 other dependent PR (#276 )
master

This stack of pull requests is managed by Graphite.


Could not create class file from file

Hello,
I am working on some tests and when i change fitnesse fixtures, i oftenly get error:

org.givwenzen.reflections.ReflectionsException: could not create class file from file

From this, i cant know where is the error. How or where can i find more detailed information on what is actually the problem?

Can you please give more information on fitnesse versions and project setup and how are you running your tests. When does this error started appearing?

Nevermind, I solved the error

O, thats, great! Do you mind sharing your solution so others can benefit :)


Display that binding is disabled

On macOS if you take a screenshot you disable binding until next cmd press. One way to ease the problem, and to generally improve the UX of binding, would be to display that binding is currently disabled.
Because we want to support cmd+shift for moving along straight lines and not binding it’s not easy to prevent the screenshot shortcuts (cmd+shift+4 etc) to break the UI.

you disable binding until next cmd press

Can you elaborate what you mean by that? Binding disabling happens only while Cmd is being pressed.

When you press CMD+SHIFT+4, the key down is registered, but the key up isn't. So binding is disabled "permanently".

I see. My first suggestion was to reset on blur but now I see you're already doing that. What's the mechanism then? OSX fires keydown when you press the cmd, but finishing the combo makes it so that keyup isn't fired (without any blur event and similar)?
Specifically the binding case can be fixed by moving temp-disabling from appState to pointerDownState. But not sure how you'd want to solve this in a general way (for other APIs).

I am not sure we can move it to pointerDownState, then you’d have to keep pressing CMD on every drag, which might be inconvenient.

What do you mean. Currently you do have to press Cmd on every drag. In the future we may implement a mode toggle, which will live in appState for sure, but then there won't be any issue with Cmd down/up events.

You can press cmd, drag, drag, then release. If we moved the state to pointedDownState, you’d have to press cmd, drag, release and press, drag, release. Right?

Nope, pointerdown event contains modifier keys information, too.

Ah, that sounds good then! Less state to keep in appState is great.


Serverless command "sam" not found. Did you mean "start"?

I've followed the instructions but get the following error:
▶ serverless sam export --output ./sam-template-generated.yml

  Serverless Error ---------------------------------------

  Serverless command "sam" not found. Did you mean "start"? Run "serverless help" for a list of all available commands.

  Get Support --------------------------------------------
     Docs:          docs.serverless.com
     Bugs:          github.com/serverless/serverless/issues
     Issues:        forum.serverless.com

  Your Environment Information ---------------------------
     Operating System:          darwin
     Node Version:              12.10.0
     Framework Version:         1.54.0
     Plugin Version:            3.1.2
     SDK Version:               2.1.2
     Components Core Version:   1.1.1
     Components CLI Version:    1.2.3

Apologies, I neglected to save my serverless.yml


fix: disable urllib3 warnings related to connection pool

I was spammed with those warnings when adding assets to a kili project:
ARNING:urllib3.connectionpool:Connection pool is full, discarding connection: storage.googleapis.com. Connection pool size: 10
WARNING:urllib3.connectionpool:Connection pool is full, discarding connection: storage.googleapis.com. Connection pool size: 10
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 47/47 [00:06<00:00,  7.64it/s]
  0%|                                                                                | 0/11 [00:01<?, ?it/s]WARNING:urllib3.connectionpool:Connection pool is full, discarding connection: storage.googleapis.com. Connection pool size: 10
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 11/11 [00:03<00:00,  3.09it/s]
  0%|                                                                                | 0/14 [00:01<?, ?it/s]WARNING:urllib3.connectionpool:Connection pool is full, discarding connection: storage.googleapis.com. Connection pool size: 10
WARNING:urllib3.connectionpool:Connection pool is full, discarding connection: storage.googleapis.com. Connection pool size: 10
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 14/14 [00:03<00:00,  4.09it/s]
  0%|                                                                                | 0/20 [00:01<?, ?it/s]WARNING:urllib3.connectionpool:Connection pool is full, discarding connection: storage.googleapis.com. Connection pool size: 10
WARNING:urllib3.connectionpool:Connection pool is full, discarding connection: storage.googleapis.com. Connection pool size: 10
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 20/20 [00:02<00:00,  6.86it/s]
  0%|                                                                                | 0/15 [00:01<?, ?it/s]WARNING:urllib3.connectionpool:Connection pool is full, discarding connection: storage.googleapis.com. Connection pool size: 10

yeah indeed I had them too since yesterday.
Did you understand what was happening exactly ?

yeah indeed I had them too since yesterday. Did you understand what was happening exactly ?

Apparently good news is  that no data is lost
I think it's due  to the fact that we now use a common requests.Session for all http requests done in our repo (before we would open a session for each request)
So in a requests.Session is assigned an requests HTTP adapter, which has a connection pool of size 10 by default
urllib3 throws this error because the connection that just finished will be discarded after the request is completed (because the pool is full, as mentioned). This means that this particular connection is not going to be re-used in the future.
Because a urllib3 PoolManager reuses connections, it will limit how many connections are retained per host to avoid accumulating too many unused sockets.
https://stackoverflow.com/questions/53765366/urllib3-connectionpool-connection-pool-is-full-discarding-connection


Fix a bug where the get_ordered_languages would mutate in place the list of configured languages.

Coverage decreased (-0.1%) to 85.294% when pulling f43a89eb14d36d1483ff57ee95658b29854a778b on fix-get_ordered_languages into a4da06bfc2837a4a935c894f7ba086e978739d8b on master.


Unnecessary focus call

https://github.com/cryptounicorns/desktop/blob/b54d3896e2fd87cfc1bfbd67d83d31136d11b4b5/src/app/main.cljs#L39-L43
https://github.com/electron/electron/blob/master/docs/api/browser-window.md#winshow

win.show()
Shows and gives focus to the window.

Valid point, sir


Safe and Strict option not available for Univocity

Description : Safe and Strict option not available for Univocity

Safe and Strict properties are implemented in Input Univocity component. It is not applicable in case of output since spark does not expose api to apply schema while writing files. It only allows to select fields from the schema of dataframe to be written.
The code is checked-in in engine_intergrator branch.

Closing this as the fix was applied.


[Bug]  Closing buffer and preview twice.

If closes the current buffer(:bd or :bw) keeps open outline as single buffer (full console).
I think, should be closed too outline, or if is another buffer, Keeps open, but for other buffer.

Preview show 2 windows, 1 with content and the second empty.

If closes the current buffer(:bd or :bw) keeps open outline as single buffer (full console).
I think, should be closed too outline, or if is another buffer, Keeps open, but for other buffer.

This issue still seem to be relevant atm, but as a workaround you can use plugins like vim-bbye, which will retain your window layout by opening another buffer when you delete this one.

Preview show 2 windows, 1 with content and the second empty.

The second preview buffer is for additional information, if available. It should now "No info available" in the second buffer in newer versions.


Version number should be added to logs and possibly error messages

When diagnosing an issue, it is sometimes difficult to verify whether an issue should have been fixed or not if the diagnostic output doesn't indicate the version. It would be much easier if all diagnostic logs had an version included, like this:
"diagnostic error line here .... (version 3.12.43)"
Steps to implement:

[ ] Move version detection earlier in NBCLI
[ ] Update NBCLIErrorHandler to add details when called

This looks very easy. Will pass the version here


Admin - Product Management

HI Roshini ,
Please can you work on Seller Terminal Product management  .We will send you url and login id in chat .
Looks like login is not working.
When Click "Product Management"

Show entries must have a small drop down symbol to select the relevant numbers
Search must scrutinize the data by just typing the content, not after enter.
Titles needed are:
SR.NO, PRODUCT ID, MERZIDO SKU, SELLER SKU, SELLER ID, PRODUCT NAME, RETAIL PRICE, SALE PRICE, QUANTITY, STATUS, ACTION
S.NO, Etc.,....... all the titles must have beside (half) downward & upward arrow to align the data's ascending & descending based on clicking
Beside Show Entries, must have ALL, LIVE, PENDING, REJECTED, DELETED options to select the exact details, In deleted must have an option to restore.

ADD: Mandatory fields
BRAND*
SELLER SKU ID*
WARRANTY TYPE*
WARRANTY PERIOD*
DELIVERY OPTION*
PRODUCT NAME*
HIGHLIGHTS*
PRODUCT DESCRIPTION*
VIDEO URL (NOT MANDATORY)
COLOR FAMILY*
MERZIDO SKU* (SYSTEM GENERATED - NON EDITABLE)
BARCODE*
QUANTITY*
RETAIL PRICE*
SALE PRICE* (MUST BE LESSER THAN RETAIL PRICE, IF EQUAL OR HIGHER, SYSTEM SHOULD THROUGH ERROR)
START DATE OF PROMOTION* (REFER TO THE SALE PRICE) Must open calendar with default today's date
END DATE OF PROMOTION* (REFER TO THE SALE PRICE)Must open calendar with default today's date
FREE ITEMS
WHAT'S IN THE BOX*
PACKAGE WEIGHT (KG)
PACKAGE DIMENSIONS (CM)
(LENGTH X WIDTH X HEIGHT)
PRODUCT WEIGHT (KG)
PRODUCT DIMENSIONS (CM)
(LENGTH X WIDTH X HEIGHT)
IMAGE (MAX 6 IMAGES IN MIN 600 X 600 PX)
All must be in one page with scroll down / up option.
follow the layout and small description written below each fields

In this model now we can add product with attributes and values


Remove configurable search-index

search-admin currently allows users to choose between the mainstream and government search indexes for saving recommended links. Based on comments on https://github.com/alphagov/search-admin/pull/48 and a conversation with Tara Stockford, we’ve decided to send all recommended links to the mainstream index by default. This commit removes the option for the user to choose the index.
The migration from the previous commit has been edited (rather than creating a new migration to reverse the changes) since that commit has not yet been deployed.
Trello: https://trello.com/c/WY4JGxKG/39-manage-external-recommended-links-in-search-admin
Follow-up to: https://github.com/alphagov/search-admin/pull/48
Before:

After:

LGTM 👍
Note - this might break on integration because the migration has been edited, but it should sort itself when the database is rebuilt tomorrow. We're not planning to deploy search admin until the related PRs are merged.


 I've noticed this sometimes as well in other repos / projects, sometimes downloading & compiling a dep just takes a while (so suspiciously long that I kill & restart the process a few times...). Does this continue hanging for more than a minute?

apparently it's downloading issue, today my friend in Indonesia have the similar problem XD
that lz4 repo just ~50MB testcase but very slow to be downloaded


Error trying to parse UInt32 values bigger than Integer.MaxValue

TypeUtils.toSqlType(String) returns Types.INTEGER for "UInt32" which leads to ClickHouseResultSet.getObject() trying to parse it through getInt(columnIndex) and results in failure for values bigger than Integer.MaxValue.

Thanks for report.
This is going to be as it is done in mysql driver for example, unsigned types are taken one step up. So that
UInt32 -> Long
UInt64 -> BigInteger

Merged to master. Will be in the next release version.


Replace deprecated cri options for Falco 0.39.0

/area documentation
What would you like to be added:
--cri and --disable-cri-async will be deprecated in 0.39.0 and will be removed in 0.40.0 as it is possible to see here. Any reference to these options should be removed in favor of  the corresponding -o container_engines.cri.<option> alternative.
Why is this needed:
This is needed in order to align the documentation with the new upcoming falco version.

@ekoops  I want to work on this documentation issue as a part of hackoctober 2024

@ekoops I want to work on this documentation issue as a part of hackoctober 2024

Nice! Please feel free to move ahead!

Hi @vidhvath28 . Did you decide to work on it?


Fix broken timed screen locking

The was actually no error. I just removed the opacity and added support for dark mode.


Support for managing env vars via kustomize

Feature Request
Is your feature request related to a problem?
If an application accepts configuration via a number of environment variables, these variables may have hard-coded defaults in the spec/env section of app-deploy.yaml that are suitable for (say) a development environment.
Kustomize can be used to patch values in the deployment to tailor it for different environments (eg. test or production).  Kustomize has built-in knowledge of the structure of standard Kubernetes types (Deployment, Service etc) and the env section of a vanilla Deployment is recognized as a map which is keyed by the name field.  It can therefore be patched either to add new entries or replace the values for existing ones.
However, Kustomize has no knowledge for custom resource types.  As a result, trying to apply a Strategic Merge Patch to the env list of an AppsodyApplication CR will result in the entire list being replaced.  Similarly, an SMP cannot be used to update an existing value in place.
A workaround is to use JSON patches (using kustomize's patchesJson6902 directive rather than patchesStrategicMerge), although these are also unable to refer to named elements of the list, so are limited to append/prepend, or replacing an entry by numeric index.
Another option is for all the 'dynamic' values in the env list (ie. values that may change at deploy time) to be references to ConfigMaps or Secrets,  and for kustomize to define the contents of those resources instead.  This would avoid needing to patch the env list at all and is the route I'm going to explore next.  But I imagine it could cause surprise if patching does not behave the same as with a vanilla Deployment.
Describe the solution you'd like
To be able to apply patches to map types in the AppsodyApplication CR the same way that I can for a vanilla Kubernetes Deployment resource.
Having read around a bit, I found discussion on this topic in https://github.com/kubernetes-sigs/kustomize/issues/1510 - from which I understand that it is not yet possible to educate kustomize on the structure of a custom resource, although there is an issue open for that support here: https://github.com/kubernetes-sigs/kustomize/issues/2339

thanks for the feature request @djones6
Do you have any outlook on when that kustomize support will be coming in?
My fear is that we have a few other places that a person could also want to "kustomize", such as volumes, etc.   So if we had a special extra section for env vars we would likely need to address the other spots too - and all that would be not needed anymore once the native CR kustomize support is available  =)
For GitOps, is it reasonable to assume that for now each kustomize patch would contain the complete set of env vars, even if that means duplicating some key/values across environments?
Another strategy - create a ConfigMap for your "static" env vars, and use envFrom to point to that.  This leaves you free to override the env array via kustomize patches.


Bringing Umple's APIs to the metamode and runtime

Umple generates a hand full of useful APIs for attributes and association. Currently, those methods are generated by UmpleTL. It means they are not available in the runtime model of Umple systems. This limits us regarding many analysis we can do at the runtime and causes to have duplication in UmpleTL's template files. Furthermore,  This will be a great base for the future extensions such as using Language Server Protocol.
One of the concerns is how to deal with the executable body of these methods. I think we can create those APIs (their signature at the modeling level) and then tag them as auto-generated. Then code generated will know how to deal with that.
This idea can be extended in an advance form to state machines.
More info about those APIs can be found here

Another concern might be the size of the model at the runtime. I think this can be dealt later with some optimization. It might not be even an issue with the current advance in technology.

Sounds interesting, but I think explaining what you mean by example would be important. Give an example trivial umple model with two classes, an association and an attribute, and describe the effect of what you would like to see

This is in the direction relationship with issue #462

Consider the following example,
class A{
  Integer x;
  0..1 -- * B;
}
class B{
}

Umple generates API for the attribute x and also the association 0..1 -- * B; The list of APIs for different elements are listed here
These APIs are generated by the different code generators (Java, PHP, etc). It means if a modeling element wants to check a method called void getX() in the class A, the list of methods for the class A will not show it. In these cases, the developer of that element starts to hack the code. The person implements another function which reacts like the one exists in the code generators and produce the list of auto-generated methods (sometimes, it's just a manual list). This brings lots of boiling code. Furthermore, it's hard to update to date all those manual codes with their own hard-coded names. This becomes much worse for cases that we do lots of post checking to make sure the model is valid.
For example, to check if there is no conflict between used-defined methods and auto-generated methods.
By having those methods at the metamodel level, we guarantee that the regular methods such as getAllMethods will always show the right number of methods. Then, a normal iteration on this list can satisfy many post-checkings.
Another example,
class A{
  Integer x;
}
trait T{
  Integer getX();
  void setX();
  void pMethod(){
  //***
  }
}

This example points out that the client of T must have two methods called getX and setX. When the algorithm tries to check the list of methods of the class A, there are not such methods. The algorithm needs to go through a complicated process to figure out if there are some auto-generated methods or not.
I believe this a great step for reducing dependency on code generators and have more concrete metamodel. If Umple wants one day in the future to have its own binary code, so there won't be any issue regarding the APIs.
It also brings many benefits for having a smart compiler at the IDE level (a future roadmap). For example, if users press ctrl+space, they will see those auto-generated methods as methods they can call for a specific class

@vahdat-ab I have started trying to orient myself with some of the compiler code, and I want to make sure I am on the right track. I found the method analyzeAttribute in UmpleInternalParser_CodeClass.ump. Is this the correct point to identify attributes to add API methods for?

@katcavers yes. this is the place Analyzer analyzes the abstract syntax tree

@vahdat-ab
I found I was getting a whole bunch of errors when I build because duplicate methods are being generated. This makes sense, as my change adds a get method for any attribute to the model, and the generator generates the API methods as well as adds any methods existing in the model; so two get methods with the same name are being added for each attribute.
I remember saying that we wanted the generation of the API methods to remain in the generators. So, I have added an additional type, fAutoAPI, to the Source enum of the Method class and set the get method's source to this. Then, I check in the Java generator class_MethodDeclaration.ump and if the source of the method is fAutoAPI I skip generating this method. Is this the right approach?

To me, it looks ok.  I wonder if you detected for what reason fAuto is used? I thin it's there for this purpose.
The way I was thinking is this:
At the code generation level, the generator detects an auto-generated method, and then it generates the method along with the necessary implementation.

let's go in the way you describe. If we want to do any modification later it will be at the code generation level. You just make sure you do all required checkings at then modeling level.

Sounds good.I think eventually implementing the auto-generated methods to be generated by the generators makes sense. This would just be a huge change to the code generation side, so it is easier to do this step first.

I am having an issue with my changes and the Ruby generator. Many of the test cases are failing because empty lines are being inserted. I believe it may be due to the fact that we skip over autogenerated API methods, but somehow a new line is still inserted. For example, I have attached two files; the expected vs actual Ruby code.
The only difference is there are two extra empty lines at the end of the actual output.
actual.txt
expected.txt

@vahdat-ab I also have two more questions.
First, in our initial discussion for this project, we talked about constructors. Would you be able to expand again on whether I should be creating/modifying constructors when an attribute is encountered in the model?
Second, you also mentioned I should make a method that could remove these autogenerated methods from the model. Would this be a method that removes all the API methods, or a method that could take for example an attribute as a parameter, and delete the API methods for only that attribute?

@ahmedvc

but somehow a new line is still inserted. For example, I have attached two files; the expected vs actual Ruby code.

I wonder if you can comment on this? How can we remove those empty lines generated because of passing some conditions?

First, in our initial discussion for this project, we talked about constructors. Would you be able to expand again on whether I should be creating/modifying constructors when an attribute is encountered in the model?

yes. This is a critical issue. You should create a constructor and feed its parameters.

Would this be a method that removes all the API methods, or a method that could take for example an attribute as a parameter, and delete the API methods for only that attribute?

The method should have a parameter such as an attribute or association and then remove related auto-generated APIs. You can also implement a method that removes all of them. This is going to be easy because there is a property that specifies which methods are auto-generated.

I think I actually just figured out why it's happening. Before, there were no methods in the model for this specific case. But now there are, as I am adding get and set API methods at the model level. In the generator, a check is done on the class being processed is done; if the class has methods, it then iterates through all the methods and at the end appends to the string builder two new lines. For this case, the if statement used to prove false, but now it is true because of the API methods I add at the model level. So even though no code is generated for these methods, those two new lines are added.

Thanks Vahdat, I will continue working on these.

@katcavers great.

@vahdat-ab For the methods to remove autogenerated methods, I believe it makes sense to add these methods to the UmpleClassifier class in Umple.ump. Do you agree, or is there some other place you had in mind? Thanks!

why do yo want to have it in UmpleClassifier?
To me it makes sense to have because we can have attributes in UmpleClass and UmpleTrait. Do you have other justifications?

Ah, actually, yes your way does make more sense. I think I was thinking to have it at the higher level of the hierarchy, but UmpleClass is the only one to have attributes, so adding it there is the better way to go.

Hi Vahdat! I have submitted now to my PR the functionality for constructors. I am now going to be moving on to Associations. There are a lot more autogenerated API methods  for associations, so I am thinking I should start first with simple get and set again?

OK. Thanks. do it as you wish :-)

For associations we have get, set and add (among many). I wouldn't add set unless I also added add, otherwise it will be a bit off-balance; set is used for '1' ended associations and add for '*'.

Thanks for pointing that out Tim! I will follow your advice.

@vahdat-ab I wanted to give you an update. I am testing out and trying to figure out how I am going to handle associations. I have realized that I will have to handle them a bit differently than attributes. For attributes, I was able to add the autogenerated methods right away, because the class they were associated with had already been added to the model. However, for associations both ends of the association may not have been added to the model yet, and so I am not able to add the autogenerated methods from analyzeAssociation(). What I am thinking I will do is add them when postTokenClassAnalysis() is called. This way, I will be able to access all the classes I need. Another option is to add some at the analyzeAssociationLevel, and then create a list similar to unlinkedAssociationVariables() where I keep track of the ends for which I need to generate methods.

Putting the code in postTokenClassAnalysis() is the right place. This makes it more module and predictable.  Even, you could do the same for attributes :-).
NOTE: The way you implemented attributes is faster than putting it in postTokenClassAnalysis(). However, it is not modular.

Yes, as I mentioned in the meeting last Friday, the API generated for associations is considerating, and the logic for determine what will be generated is non-trivial (there are I think about 14 cominations of multiplicity, but there are also other factors that change the API). This took a whole PhD (Andrew Forward) and has been evolved since then to add features and fix bugs.
I think that doing this manually will be a challenge. The logic for the set of methods to be generated is already encapsulated in the generator; you would either need to reverse engineer this and hence create a module that somehow duplicates it, although the module will not be as complex as the generators since you do not care about the contents of the bodies.
In the long run it will be necessary to constantly verify that whatever is generated matches the signatures found in the model. Otherwise changes may result in mismatches creeping in. We can take advantage of that by right now starting wtth that verifier: Instrument the Java generator to emit a message (only in a debugging mode) when it generates a method that does not have a corresponding method in the model, and vice-versa. i.e. it checks methods off  lists are they are generated and then spits out the difference.
Next you start adding the methods to the model and watch the above list of messages about mismatches getting smaller and smaller. You won't be able to be perfect in the week or so you have left, but you can get part way, and leave it for another student to finish.

@TimLethbridge Thanks for the clarification Tim. I will start doing as you have suggested. Do you know of anywhere in the Umple project where something similar, in terms of debugging trace statements, has been done that I could take a look at?

@vahdat-ab After looking more into the Trace capabilities (both the code and user manual), I do not think I will be able to use it. While many of the ideas are shared between Trace elements and what I need, I think that I would be using the Umple Trace in a way it was not necessarily intended, and would have to make changes to it that would not be in line with its purpose. I am going to go forward with creating my own implementation of debug statements to verify autogenerated methods.

Ok. That's fine.
On Sun, Apr 2, 2017, 5:45 PM katcavers<EMAIL_ADDRESS>wrote:

@vahdat-ab https://github.com/vahdat-ab After looking more into the
Trace capabilities (both the code and user manual), I do not think I will
be able to use it. While many of the ideas are shared between Trace
elements and what I need, I think that I would be using the Umple Trace in
a way it was not necessarily intended, and would have to make changes to it
that would not be in line with its purpose. I am going to go forward with
creating my own implementation of debug statements to verify autogenerated
methods.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/umple/umple/issues/962#issuecomment-291017514, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ADg6LFjClzPvof4G0JhbV76Y4Ufrgr6aks5rsBb_gaJpZM4LqIz3
.

--
Vahdat Abdelzad.
Ph.D. Candidate in Software Engineering, University of Ottawa,
School of Electrical Engineering and Computer Science, Room 4027,
800 King Edward Avenue, Ottawa, ON, Canada, K1N 6N5,
Tel:+1<PHONE_NUMBER> ext. 6428  Cellphone: +1<PHONE_NUMBER>
e-mail<EMAIL_ADDRESS><EMAIL_ADDRESS>&
<EMAIL_ADDRESS><EMAIL_ADDRESS>https://uottawa.academia.edu/VahdatAbdelzad
https://uottawa.academia.edu/VahdatAbdelzad

@vahdat-ab Hi Vahdat, I have finished putting in my debug statements and am now setting up tests for the autogenerated associations. However, I noticed there are a large number of files in the UmpleTL folder of UmpleToJava which do not seem to be used. For example, association_SetOneToOptionalN.ump. I did a search for their code in the project in Eclipse (including generated code), and nothing came up. I want to make sure that I am not missing anything as I will be relying somewhat on these debug statements. Are these just old files that were once used, and just haven't been deleted? Or are they being used somewhere other than the generator?

@katcavers I'm not aware of them.
@TimLethbridge might know about them.

Hmmm. I don't know. You say 'a large number'. If it was a few I would think they might have been left behind. But I would be rather surprised if it is a large number. One strategy would be to delete them temporarily and do a full build and see what happens! remember that these files are compiled separately; then the generated code is transferred from their directory to the general umple generated code directory.

@TimLethbridge @vahdat-ab After your comments, I realized last night that I had missed files used in JavaSpecGenerator - I went and added the debug statements to the files there, and the number of unused files is a small one now. Oops! And thanks! :)

@TimLethbridge @vahdat-ab Documentation has been added: https://github.com/umple/umple/wiki/Adding-Autogenerated-Methods-at-the-Model-Level
I am in the middle of exams, so I will keep adding to it as I have time and think of what else I should write.

NOTE: Read above wiki page. All work for associations has been done in branch issue962.

I will attempt to replace the previous testing method that was implemented with a second one, both are explained on the wiki page.


(MAINT) Update Gemfile and spec test value generation

This commit adds a SHA reference to the MOF Gem that is retrieved from
git.  The MOF repoisitory has no release tags or branches so a commit
SHA is used.
This commit also adds spec test values for the real32 and real64
datatypes.  These datatypes are floating point numbers represented as
strings.  These were missing and caused spec unexpected test failures.

The sync.yaml also needs hit, although this is likely just temporary. I'm :+1:


Addressed safety issues up to 2024-08-16

No review needed.

coverage: 36.575%. remained the same
when pulling 50b93738d0a20c7fb36d8dc3e88936c2d9f7a7fc on andy/safety-2024-08-16
into c1a510527f183eb4fb5ebdb1f8c3fa84b35dcea1 on master.


Focus shifting to notification when I get new notf.

For example I`m writing some text some where when I got a notification my cursor or focus disappear.

Thank you for reporting!


Browser?

This lib would rock my world to have in the browser world :) Is it possible you think?

browserify?


URL without apidocs?

Where should https://bl-lookup-sri.renci.org/ point?   If you append "apidocs", you go to the swagger ui page, but without the apidocs, I think it should route you there also.  Note that this extends across nodenorm, edgenorm and aragorn services.
@patrickkwang do you agree?

Yes, if we're standing up a service that is only an API, the root should go to the Swagger UI.


AWS - Failing while configuring vault (StatefulSet timeout)

Which version of kubefirst are you using?
kubefirst-cli golang utility version: v2.2.15
Which cloud provider?
AWS
Which installation type?
CLI
Which distributed Git provider?
GitHub
What is the issue?
Error
Error: the StatefulSet was not created within the timeout period

If a detailed error message was available, please make the necessary corrections before retrying.
You can re-run the last command to try the operation again.

Follow up to: https://github.com/kubefirst/kubefirst/issues/1773#issuecomment-1696441165

Status

Gitops repo is created; workflow is successful

Metaphor repo is created; workflow is blocked on publish

Everything up until Configuring Vault completed; this step still showing 0% after 10 minutes

Logs
Whole log file is too long to paste here (and also not sure whether it's leaking sensitive info) but here's the end of the log before it errors out:
2023-08-30T20:19 INF cmd/aws/create.go:1187 > Setting argocd username and password credentials
2023-08-30T20:19 INF cmd/aws/create.go:1198 > argocd username and password credentials set successfully
2023-08-30T20:19 INF cmd/aws/create.go:1200 > Getting an argocd auth token
2023-08-30T20:19 INF pkg/argocd/auth.go:75 > using argocd url http://localhost:8080
2023-08-30T20:19 INF cmd/aws/create.go:1207 > argocd admin auth token set
2023-08-30T20:19 INF cmd/aws/create.go:1238 > applying the registry application to argocd
2023-08-30T20:19 INF cmd/aws/create.go:1257 > waiting for vault pods to be ready
2023-08-30T20:19 INF pkg/k8s/exec.go:281 > waiting for vault StatefulSet to be created using label app.kubernetes.io/instance=vault
2023-08-30T20:29 ERR pkg/k8s/exec.go:303 > the StatefulSet was not created within the timeout period
2023-08-30T20:29 ERR cmd/aws/create.go:1267 > Error finding Vault StatefulSet: the StatefulSet was not created within the timeout period

Code of Conduct

[X] I agree to follow this project's Code of Conduct

Relevant comment from Clay for someone else's issue

Confirmation that I was able to get AWS running before with the same stack, but haven't been able to replicate since

Noticing Principal IAM user error when I check the console manually. This could be related. However, it's odd to me that it passed both EKS deployment and Argo configuration only to fail at Vault config

Noticed that external DNS was throwing an error

Failed to load logs: container "external-dns" in pod "external-dns-7ff5444dd5-95qv8" is waiting to start: CreateContainerConfigError

Reason: BadRequest (400)

After restarting, found a new bug
Error: secret "aws-auth" not found

When I check Secrets, I see there's an aws-secret. Wondering whether:
a. This is a rename issue (ie. it's looking for aws-auth but should be looking for aws-secret
b. There's a second secret name aws-auth which isn't being generated

Looking for aws-auth:

The only place I have been able to find reference to aws-auth is in an archived repo:

https://github.com/kubefirst/gitops-gh-template/blob/364ca3b6950b549c67eb470c19088bdfa43245ed/registry/base/aws-auth.yaml#L4
https://github.com/kubefirst/gitops-gh-template/blob/364ca3b6950b549c67eb470c19088bdfa43245ed/terraform/base/eks/variables.tf#L2

I was able to find aws-secret (which is created and available on the cluster) in the ci-template repo here:

https://github.com/kubefirst/ci-template/blob/9125cbafabb658f3b5df4e9fe8ee4a42586cf73e/terraform/secret/main.tf#L17

Found a small bug on https://docs.kubefirst.io/aws/explore/vault

Command listed:

~/.k1/tools/kubectl -n vault get secrets/vault-unseal-secret --template='{{index .data "root-token"}}' | base64 -d

Actual path to/tools/:

Options:

Install tools in root directory like the docs mention (easier, especially since multiple clusters would use common binary)
Update docs to mention project name (eg. ~/.k1/<MY_CLUSTER>/tools/kubectl)

@alechp all this was great work. The ci-template I believe is no longer used. We are using the gitops-template now. I believe this is fixed in #1789. Feel free to re-open if not.

Deploy finished!! Thank you @Claywd. Definitely fixed the aws-auth issue above


2021 Annual Report tweets: Monthly Community Meetings

Provide the actual tweet content between the commented colons

Community involvement in Kubernetes is about trust and transparency. In 2021 we rebooted our Monthly Community Meetings to include discussions. Full report at  https://www.cncf.io/reports/kubernetes-annual-report-2021/

/validate


[FLINK-21240][jdbc] Fix JDBC row converter doesn't support external LocalDateTime type

What is the purpose of the change

This pull request fix the JDBC row converter didn't consider external LocalDateTime type

Brief change log

Add external LocalDateTime type support in AbstractJdbcRowConverter

Verifying this change

Add unit test

Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): (no)
The public API, i.e., is any changed class annotated with @Public(Evolving): (no)
The serializers: (no)
The runtime per-record code paths (performance sensitive): ( no)
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (no)
The S3 file system connector: (no)

Documentation

Does this pull request introduce a new feature? (no)
If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)

@flinkbot run azure


CMake build error corner case

@dagar FYI
How to reproduce:

Try to build PX4 without having the arm compiler installed
Install arm compiler
Try to build PX4 (without doing a 'make distclean')

If no 'make distclean' after the compiler installation is executed you get the following error.
Console output:
make px4fmu-v5_default upload
-- Build Type: MinSizeRel
-- PX4 VERSION: v1.7.3-296-g26f9e56
-- CONFIG: nuttx_px4fmu-v5_default
-- Build Type: MinSizeRel
-- The ASM compiler identification is GNU
-- Found assembler: /home/roman/gcc-arm-none-eabi-5_4-2016q2/bin/arm-none-eabi-gcc
-- Found PythonInterp: /usr/bin/python (found version "2.7.12")
-- Found PY_jinja2: /usr/local/lib/python2.7/dist-packages/jinja2
-- C compiler: arm-none-eabi-gcc (GNU Tools for ARM Embedded Processors) 5.4.1 20160609 (release) [ARM/embedded-5-branch revision 237715]
-- C++ compiler: arm-none-eabi-g++ (GNU Tools for ARM Embedded Processors) 5.4.1 20160609 (release) [ARM/embedded-5-branch revision 237715]
-- Building and including px4io-v2
-- Using C++03
-- Release build type: MinSizeRel
-- Adding UAVCAN STM32 platform driver
-- NuttX: px4fmu-v5 nsh cortex-m7
-- ROMFS: px4fmu_common
-- Configuring done
-- Generating done
-- Build files have been written to: /home/roman/drones/Firmware/build/px4fmu-v5_default
[5/906] Building px4io-v2
-- Build Type: MinSizeRel
-- PX4 VERSION: v1.7.3-296-g26f9e56
-- CONFIG: nuttx_px4io-v2_default
-- Build Type: MinSizeRel
-- The ASM compiler identification is GNU
-- Found assembler: /home/roman/gcc-arm-none-eabi-5_4-2016q2/bin/arm-none-eabi-gcc
-- Found PythonInterp: /usr/bin/python (found version "2.7.12")
-- Found PY_jinja2: /usr/local/lib/python2.7/dist-packages/jinja2
-- C compiler: arm-none-eabi-gcc (GNU Tools for ARM Embedded Processors) 5.4.1 20160609 (release) [ARM/embedded-5-branch revision 237715]
-- C++ compiler: arm-none-eabi-g++ (GNU Tools for ARM Embedded Processors) 5.4.1 20160609 (release) [ARM/embedded-5-branch revision 237715]
-- NuttX: px4io-v2 nsh cortex-m3
-- Configuring done
-- Generating done
-- Build files have been written to: /home/roman/drones/Firmware/build/px4io-v2_default
[157/157] Linking CXX executable nuttx_px4io-v2_default.elf
[11/906] Running dsdl compiler
FAILED: cd /home/roman/drones/Firmware/src/modules/uavcan/libuavcan/libuavcan && /usr/bin/python /home/roman/drones/Firmware/src/modules/uavcan/libuavcan/libuavcan/dsdl_compiler/libuavcan_dsdlc test/dsdl_test/root_ns_a test/dsdl_test/root_ns_b /home/roman/drones/Firmware/src/modules/uavcan/libuavcan/libuavcan/../dsdl/uavcan -Oinclude/dsdlc_generated && /usr/bin/cmake -E touch /home/roman/drones/Firmware/build/px4fmu-v5_default/libuavcan_dsdlc_run.stamp
Traceback (most recent call last):
  File "/home/roman/drones/Firmware/src/modules/uavcan/libuavcan/libuavcan/dsdl_compiler/libuavcan_dsdlc", line 59, in <module>
    from libuavcan_dsdl_compiler import run as dsdlc_run
  File "/home/roman/drones/Firmware/src/modules/uavcan/libuavcan/libuavcan/dsdl_compiler/libuavcan_dsdl_compiler/__init__.py", line 17, in <module>
    from uavcan import dsdl
ImportError: No module named uavcan
[11/906] Generating parameters.xml
ninja: build stopped: subcommand failed.
Makefile:153: recipe for target 'px4fmu-v5_default' failed
make: *** [px4fmu-v5_default] Error 1

It looks like we're missing a few dependencies at the UAVCAN level.


Restart Containers on Cert Renewal

SCENARIO:
Let's say you have a micro-service that doesn't only serve HTTP traffic, but also other kind of traffic (e.g. FTP).
Now let's say that we do not only want to encrypt the web traffic, but also the ftp traffic (using FTPS). This can be done easily by integrating the keys generated by the letsencrypt-companion into the FTP container and use them for FTPS.
The problem now arises on Cert renewal. While the letsencrypt-companion restart the nginx reverse-proxy container (meaning that the web traffic uses the new certificates), the FTP container still uses the old ones and also needs to be restarted.
SUGGESTION:
Add an extra environmental variable (e.g. LE_RESTART), that if set true, signals the letsencrypt-companion to restart the respective container.
To avoid overheads, it would also make sense to additionally check the LETSENCRYPT_HOST and only restart if the respective cert of the container is renewed.

I get the idea and see how it could be useful but that's going pretty far from the original project scope.

Hmm, I understand that (even going by the name) the companion is thought to interact only with the nginx container and (if seperate) the docker-gen container.
But the functionality requested is perfect for docker-letsencrypt-nginx-proxy-companion and can be implemented in a few lines. (only adding additional calls to "docker-kill" function).
If I make a PR, will it be merged?
I mean I can always fork the repo and do what I want (which is probably what you are going to suggest to me), but I'm unsure how long I will be able to keep it up to date with this one.

Due to some personal issues I have not had the time yet, but I will fork and make a PR in the next 2-3 weeks.

So, PR is live and the fork can be found here.
Due to the unsure nature of the merging of this PR, I will try to keep the fork updated with this repo for as long as possible.

@bplein
If you want a working le-companion that restarts containers NOW, head over to Greek64/docker-letsencrypt-companion, download/pull the master branch, and build the le-companion from the downloaded dockerfile.
This will build the latest le-companion release with the requested feature.
As said above, I will try to keep that repo updated until this PR gets merged.
I'm not entirely sure if what you suggest - having the same LETSENCRYPT_HOST on different containers with different VIRTUAL_PORTS - will work.
What I can say though is, that the le-companion only uses the LETSENCRYPT_* environmental variables. The VIRTUAL_* (and by that the key VIRTUAL_PORT) environmental variables are used directly by the nginx.
It now comes down to if le-companion can have the same LETSENCRYPT_HOST for multiple containers, where it uses the same certs, but also checks for renewal for all containers (which would be necessary to restart all containers when a renewal happens). My bash-fu is not at a high level, but I guess the "groupBy" and "trim" commands in letsencrypt_service_data.tmpl would prevent this.
I would suggest doing something like in Mailu, where the nginx of the front-end (WebGUI) is also responsible for the encryption and forwarding of all relevant ports (IMAP, POP3, etc). This would allow you to only have to "link" the mailcow front container with le-companion.

THanks.
I prefer to have my own independent proxy in front of mailcow and all other containers. It's a more general approach and allows me to customize the proxy and not be concerned that maybe the application (mailu, mailcow, nextcloud, etc.) is going to make a change to the proxy that might break things.


Add pgBouncer to funnel connections

Addresses #236.

Closing this effort.  See https://github.com/rustprooflabs/pgosm-flex/issues/236#issuecomment-1100654026

Rebased

Current version appears to be working when running manually.  Fails make though.  I went back and forth trying to find an obvious fix, didn't find anything.  Current error from make:
# process DC file, pretending it's a region instead of subregion
docker exec -it \
	-e POSTGRES_PASSWORD=mysecretpassword \
	-e POSTGRES_USER=postgres \
	-u 1000:1000 \
	pgosm python3 docker/pgosm_flex.py  \
	--layerset=minimal \
	--ram=2 \
	--region=region-dc \
	--skip-qgis-style --skip-nested # Make this test run faster
2023-10-18 20:57:05,951:INFO:pgosm-flex:pgosm_flex:PgOSM Flex starting...
Setting up pgBouncer configuration files
Running pgbouncer as postgres user
Traceback (most recent call last):
  File "/app/docker/pgosm_flex.py", line 625, in <module>
    run_pgosm_flex()
  File "/usr/local/lib/python3.9/dist-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.9/dist-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.9/dist-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.9/dist-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/app/docker/pgosm_flex.py", line 103, in run_pgosm_flex
    pgbouncer.run()
  File "/app/docker/pgbouncer.py", line 58, in run
    subprocess.run(
  File "/usr/lib/python3.9/subprocess.py", line 505, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/usr/lib/python3.9/subprocess.py", line 951, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/lib/python3.9/subprocess.py", line 1823, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
PermissionError: [Errno 1] Operation not permitted
make: *** [Makefile:174: docker-exec-region] Error 1

It appears to be working nicely.  Haven't tested with external connections yet, or with append mode.  It does appear to be working as expected on internal Postgres so hopefully it just works from here...
docker stop pgosm && docker build -t rustprooflabs/pgosm-flex .
docker run --name pgosm -d --rm \
    -v ~/pgosm-data:/app/output \
    -v /etc/localtime:/etc/localtime:ro \
    -e POSTGRES_PASSWORD=$POSTGRES_PASSWORD \
    -p 5433:6432 \ # Port for pgBouncer instead of base Postgres
    -d rustprooflabs/pgosm-flex

time docker exec -it \
    pgosm python3 docker/pgosm_flex.py \
        --ram=8 \
        --region=north-america/us \
        --subregion=district-of-columbia \
        --pgbouncer-pool-size=10

New output to logs at beginning. Running docker exec multiple times respects the --pgbouncer-pool-size setting.  If it's enabled it'll always restart pgbouncer to ensure config is proper.  If it's disabled (-1) it will attempt to stop the pgbouncer service.
2023-10-22 12:08:40,321:INFO:pgosm-flex:pgosm_flex:PgOSM Flex starting...
2023-10-22 12:08:40,322:WARNING:pgosm-flex:pgbouncer:Saving password in plain text within the container for pgBouncer.
2023-10-22 12:08:40,322:INFO:pgosm-flex:pgbouncer:Setting up pgBouncer configuration files with pool size 10
2023-10-22 12:08:40,325:INFO:pgosm-flex:pgbouncer:pgBouncer started
2023-10-22 12:08:40,325:INFO:pgosm-flex:db:Checking for Postgres service to be available

Crud, this actually wasn't working as I had expected.  After making the osm2pgsql connections go through pgBouncer I'm getting failures.
2023-10-23 17:01:15,341:INFO:pgosm-flex:helpers:2023-10-23 17:01:15  Storing properties to table '"public"."osm2pgsql_properties"'.
2023-10-23 17:01:15,353:INFO:pgosm-flex:helpers:2023-10-23 17:01:15  ERROR: SQL command failed: EXECUTE set_property(attributes,false)
2023-10-23 17:01:15,354:INFO:pgosm-flex:helpers:2023-10-23 17:01:15  ERROR: Database error: FATAL:  prepared statement did not exist
2023-10-23 17:01:15,354:INFO:pgosm-flex:helpers:server closed the connection unexpectedly
2023-10-23 17:01:15,354:INFO:pgosm-flex:helpers:This probably means the server terminated abnormally
2023-10-23 17:01:15,354:INFO:pgosm-flex:helpers:before or while processing the request.
2023-10-23 17:01:15,354:INFO:pgosm-flex:helpers:(7)
2023-10-23 17:01:16,361:ERROR:pgosm-flex:pgosm_flex:Failed to run osm2pgsql. Return code: 1
Failed to run osm2pgsql. Return code: 1 - Check the log output for details

Looking deeper and reading closer I found on this page:

"Note: This tracking and rewriting of prepared statement commands does not work for SQL-level prepared statement commands such as PREPARE, EXECUTE, DEALLOCATE, DEALLOCATE ALL and DISCARD ALL. "

Searching in osm2pgsql I found this in properties.cpp
    db_connection.exec(
        "PREPARE set_property(text, text) AS"
        " INSERT INTO {} (property, value) VALUES ($1, $2)"
        " ON CONFLICT (property) DO UPDATE SET value = EXCLUDED.value",
        table_name());
    db_connection.exec_prepared("set_property", inserted.first,
                                inserted.second);

I'm unsure how to proceed on this ATM.  I'm going to have to do some deeper digging and testing to figure out how this should/could work.

Maybe refactoring prepared queries to use PQprepare is what's needed.   With the way PQexecPrepared is used it seems PQprepare might be an option too.

https://github.com/osm2pgsql-dev/osm2pgsql/discussions/2118

Rebased, setup osm2pgsql repo URL to be dynamic for planned testing.

Recent work in osm2pgsql to reduce idle connections has removed my reasoning to work further on this. See #376 for details.  The initial import w/ default layerset dropped from 43 connections down to 10.  Replication updates dropped from 207 connections down to 17.


Automate config-related renders

Make it easier to add a config parameter

This could be fixed in general by making WTForms pack and unpack into their respective documents automatically


⚠ Jenkins Plugins Artifactory has degraded performance

In a0baac4, Jenkins Plugins Artifactory (https://get.jenkins.io/plugins/artifactory/) experienced degraded performance:

HTTP code: 200
Response time: 1355 ms

Resolved: Jenkins Plugins Artifactory performance has improved in f80efb5 after 1 hour, 2 minutes.


Create closeOnBlur option

Currently the menu always closes on blur: https://github.com/riophae/vue-treeselect/blob/d4943eb8287a5253516501f29326074b2038c434/src/components/Input.vue#L99
For mobile this can be a problem: 1. User taps treeselect, 2. keyboard opens, taking up half the display, 3. user scrolls to see bottom of list but the keyboard obscures it so they keep scrolling until the top of the treeselect leaves the viewport, causing a blur.
Idea: Add a closeOnBlur option, default true, and the line of code linked above would only execute if this this is true.
@riophae two questions:

Would this have any adverse side-effects I'm not aware of? I.e. are there other scenarios that a 'blur' event may occur and we don't want to turn off the closing behavior in those scenarios?

If no adverse side-effects, would you approve a pull request with this option? Any changes from my proposal you'd like to see?

Note for future reference, if we go forward with this, it could possibly solve https://github.com/riophae/vue-treeselect/issues/284

has something been done for that ? I'm having that issue on mobile and it's annoying to not be able to close/hide the keyboard.

@mstratman what's scheduling bro


HDF5 Cloud Storage

Is there interest in storing data in the cloud?  E.g. using AWS S3.  With HDF5 this is problematic since h5py requires a posix-based filesystem.  I maintain the h5pyd project (https://github.com/HDFGroup/h5pyd) which gets around this by providing a h5py compatible api to a sharded data store (similar to zarr).  I think it should be possible to have h5ad support either h5py or h5pyd, but first wanted to gauge interest in this approach.
Thanks!

Hey! We definitely have interest in cloud based storage, but so far have been largely eyeing Zarr and possibly arrow for this. This looks interesting. My initial thoughts compared to the other formats:
Strengths:

hdf5 is our most common backend, most common way files are distributed
Good cross language support compared to other cloud friendly formats

Weaknesses:

Requires HDF5 service running on the server side, not sure how hard/ common this will be

Needs investigation:

Drop in support?
How does performance look?

Could be of interest to @ryan-williams, @ilan-gold, @joshua-gould

@ivirshup I haven't had time to dig into this on the AnnData side yet, but a good starting point here for remote data (wihtout switching to a new hdf5 reader, which is what what this issue seems to be about) would be to literally just try passing in a URL to a zarr-backed AnnData store to the AnnDataconstructor since (I think) zarr supports URLs natively.  For example, this line makes me think that this should just work out of the box:
https://github.com/theislab/anndata/blob/0ec97410702f61b29acef80b6377d1232699fa94/anndata/_io/zarr.py#L252
So for example, if you have an AnnData store you could do
adata.write_zarr('path/to/local/my_store.zarr')

Then in a shell (since I don't know offhand in python)
gsutil cp -r path/to/local/my_store.zarr gs://my_bucket/

And then
import anndata as ad
import aiohttp, requests, zarr, fsspec
adata_cloud = ad.read_zarr(my_google_url)

This doesn't seem to work for my examples (I get an empty AnnData store from https://storage.googleapis.com/vitessce-demo-data/anndata-test/pbmc3k_processed.zarr although there is no error) but this would be the idea if it worked.  Maybe I missed something though.  I'll do some more digging.  Despite this apparently not working with remote zarr stores (I really think
I am doing something wrong, or there is only a small change we need), we have read them in via a custom loaders in the browser via zarr.js.  For example https://portal.hubmapconsortium.org/browse/dataset/ea4cfecb8495b36694d9a951510dc3c6 uses remote AnnData stores written to zarr for visualization.
Just wanted to chime in.  The TL;DR is basically that zarr supports remote data in theory and my inability to get it working here probably has more to do with my inexperience with zarr than anything AnnData specific.    I'll try to look into this more.

It seems like it's more or less drop in.  I was a bit confused with the dispatching logic at first, but my naive attempt seems to work: https://github.com/HDFGroup/anndata/commit/1a4833fed8d9cdeec1d297e224fbeffc030dc304, at least for something like:
import numpy as np
import scipy as sp
import scipy.sparse
from anndata import AnnData, read_h5ad
A = np.random.rand(400, 30)
A[A<0.7]=0
A = sp.sparse.csr_matrix(A)
ad = AnnData(A)
ad.write('hdf5://home/john/anndata/sparse_dataset.h5ad', force_dense=True)

I use the "hdf5://" prefix on the filename to indicate that this is meant to be written to the server rather than a local HDF5 file.  After this runs I can do: $ hsls -r /home/john/anndata/sparse_dense_dataset.h5ad (hsls is the server equivalent to h5ls) and get:
/ Group
/X Dataset {400, 30}
/obs Group
/obs/_index Dataset {400}
/var Group
/var/_index Dataset {30}

I haven't tried running any of the benchmarks, but likely it would significantly slower than writing to local HDF5 files since there's extra latency in making off-box requests to the server.  Benefit is that you can target AWS S3, Azure Blob, or other object storage systems.  Since the server (HSDS) mediates access to the storage system, users don't need to have credentials to a cloud provider, just a username/password with the service.  Also, if the client is running outside the cloud provider, there's less data movement since only actual read and write selections need to be transferred (rather than entire files).
You do need to have the service running - it can be setup on Docker or Kubernetes.  Fairly easy to install and scale up or done based on usage requirements.
Let me know if this seems interesting to anyone.  If so I'd be happy to flesh out the h5pyd integration.

Sorry for the long response time here!
I would really like to see this functionality integrated upstream. Is the goal here for hsds to become something like another driver for hdf5?
Getting AnnData to be more cloud friendly is a current priority – so it's under active development. Until we have nailed down our use cases and usage patterns I think I would prefer to keep a smaller number of backends supported in this library, just so there's less to migrate and support while this evolves. Especially if the end goal for this particular backend is to be available through an API we already support.

No problem!  Thanks for getting back on this.
The goal of HSDS is to support the use of HDF in a cloud-native context.  This means having a REST-based API, ability to run in distributed systems (e.g. Kubernetes), dynamically scale, and to work well with object storage backends (e.g. S3).  Since it's based on the HDF5 data model, it's relatively simple for h5pyd to support most of h5py's api.
So it's something less than an entirely new backend for anndata - see the PR.  We pull in both the h5py and h5pyd packages and then tweak the dispatch logic based on if we are dealing with HDF5 files or HSDS server.
BTW, I've been thinking about sparse data support in HSDS recently and judging from the example above, that's important for AnnData.  It would be interesting to think about sparse-specific methods in h5pyd/HSDS.  (Of course that would make the backend logic more complicated, but I don't think we'll see sparse data methods in h5py anytime soon)

Hi, I was wondering if there is any update on this issue. I have a big file on S3 and I would love to be able to read it directly, as for example in h5py using ros3, see here
This is read-only access, but I believe it would help in many use cases.

@djarecka found the

remfile library that provides a file-like object for reading a remote file over HTTP, optimized for use with h5py

@Koncopd said in https://github.com/scverse/anndata/pull/1322#issuecomment-1905727154 that s3fs works well.
I think we should

Check what API we want. Maybe we just accept URIs and allow configuring handlers for certain URI schemes?
If we commit to directly supporting something, we should benchmark remfile and s3fs. The latter is more general, but why not support the more efficient option?

@djarecka

Hi, I was wondering if there is any update on this issue. I have a big file on S3 and I would love to be able to read it directly, as for example in h5py using ros3, see here

Right now, you can do:
import anndata as ad
import h5py
import remfile

ADATA_URI = "https://allen-brain-cell-atlas.s3.us-west-2.amazonaws.com/expression_matrices/WMB-10Xv2/20230630/WMB-10Xv2-TH-log2.h5ad"

file_h5 = h5py.File(remfile.File(ADATA_URI), "r")

# Read the whole file
adata = ad.experimental.read_elem(file_h5)

# Read the file like "backed"
def read_w_sparse_dataset(group: "h5py.Group | zarr.Group") -> ad.AnnData:
    return ad.AnnData(
        X=ad.experimental.sparse_dataset(group["X"]),
        **{
            k: ad.experimental.read_elem(group[k]) if k in group else {}
            for k in ["layers", "obs", "var", "obsm", "varm", "uns", "obsp", "varp"]
        }
    )

adata = read_w_sparse_dataset(file_h5)
adata.X
# CSRDataset: backend hdf5, shape (131212, 32285), data_dtype float32

@flying-sheep

Maybe we just accept URIs and allow configuring handlers for certain URI schemes?

See also

https://github.com/scverse/anndata/issues/657

Was just looking into the remfile solution and it is very cool.  It could be made to work with read_elem_as_dask so going to open a PR to begin the investigation of this.


Getting started with xtensor and arrow

I need to significantly improve how I use python and c++ together. I am currently using pybind11 and return numpy arrays to be ingested into pandas dataframes. The data exchange is a big performance bottleneck and the code is difficult to maintain. I believe that both xtensor and arrow will help me, but I am confused about which to use first and how xtensor and arrow work together--both now and in the future. (I realize this is not a typical bug type of issue, but I am not sure where is the right place to ask.) THANKS!

This is definitely the right place to ask! We also have a gitter channel.
Regarding your issue, I think you should start with xtensor-python. It is based on pybind11 and allows you to operate on numpy arrays in place.
@wolfv has started working on Arrow bindings for xtensor, but this is an early stage project; we plan to be fully interoperable with Arrow in a near future, though.

So for my current efforts where I need to improve the data exchange performance between python and C++, I can/should use xtensor, xtensor-python (and likely xtensor-io) for a relatively easy transition from what I am doing now? I believe this to be the case--the xtensor documentation and examples seem to be good and the cookie cutter looks to make using xtensor straightforward. In doing so, it is reasonable to assume that there will be xtensor related work to provide arrow bindings so (in the near future) I will benefit from the various packages that either have or are working on arrow interoperability?

So for my current efforts where I need to improve the data exchange performance between python and C++, I can/should use xtensor, xtensor-python (and likely xtensor-io) for a relatively easy transition from what I am doing now?

I think so,  xtensor  provides a similar API to the one of Numpy and  xtensor-python  makes you able to operate on Numpy arrays in-place, so no copy is involved. Thus if you are used to Numpy, you'll feel at home with xtensor!

it is reasonable to assume that there will be xtensor related work to provide arrow bindings so (in the near future) I will benefit from the various packages that either have or are working on arrow interoperability?

Yes we plan to continue the development of Arrow bindings and push them to the level of the other bindings (xtensor-python, xtensor-julia and xtensor-R)

Sounds good! Thank you for your comments. I look forward to moving to xtensor and especially the shiny new arrow bindings when they are available.


Incorrect URL for OpenAI compatibility (missing /v1)

System Info
N/A
Information

[ ] Docker
[ ] The CLI directly

Tasks

[ ] An officially supported command
[ ] My own modifications

Reproduction
By setting the OPENAI_BASE_URL environment variable to: "http://<server ip>:<port>/v1" and using this API with the openai client, results in 404. See here for how OpenAI expects the base_url to be used: https://github.com/openai/openai-python/blob/5cfb125acce0e8304d12bdd39b405071021db658/src/openai/_client.py#L110
        if base_url is None:
            base_url = os.environ.get("OPENAI_BASE_URL")
        if base_url is None:
            base_url = f"https://api.openai.com/v1"

So, the /embeddings API endpoint should be changed to /v1/embeddings (or at least added as an alias for any unfortunate clients who have implemented workarounds for the incorrect path).
test code:
import openai

#client = openai.Client(base_url="http://localhost:80") # works with text-embeddings-inference
client = openai.Client(base_url="http://localhost:80/v1") # 404, correct usage
emb = client.embeddings.create(input=['this is a 404'], model="text-embeddings-latest")
print(emb)

Expected behavior
not 404

#112


Use Express Server for API Calls

For all api calls we should set up an express server that actually fetches this data. This requires us to separate the client side served code out from a background node server. Things to do for this:

[ ] Create server directory in src
[ ] Add index.js to server directory
[ ] Refactor github helper for server side
[ ] Refactor githubHelper on client side to query express server
[ ] Figure out dev environment configuration
[ ] Figure out express server url on client side
[ ] Figure out production environment configuration

@cubadomingo let me know what you want and I'll take the rest

Might want to dynamically set the port express runs on based upon an open port check. Currently it sets it based upon an env var PORT and/or defaults to 5000


Latest release (6.0.25) broken on CentOS in Docker

There seems be a bug/feature/strangeness in Chef regarding the service_resource_providers selection which breaks the latest release on CentOS 6.6 running in Docker. I just tested my cookbook (which worked yesterday with the 6.0.24 release) with kitchen-docker_cli which throws this:
================================================================================
    Error executing action `start` on resource 'mysql_service[atlassian]'
    ================================================================================

    Mixlib::ShellOut::ShellCommandFailed
    ------------------------------------
    service[atlassian :start mysql-atlassian] (/tmp/kitchen/cache/cookbooks/mysql/libraries/provider_mysql_service_upstart.rb line 41) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
    ---- Begin output of /sbin/start mysql-atlassian ----
    STDOUT:
    STDERR: start: Unable to connect to Upstart: Failed to connect to socket /com/ubuntu/upstart: Connection refused
    ---- End output of /sbin/start mysql-atlassian ----
    Ran /sbin/start mysql-atlassian returned 1

    Resource Declaration:
    ---------------------
    # In /tmp/kitchen/cache/cookbooks/atlassian-database/recipes/mysql.rb

     62: mysql_service settings['instance'] do
     63:   action :start
     64: end

Of course using upstart on CentOS does not work. The chef-shell reveals the problem (latest release 6.0.25 checks if :upstart is included):
chef > Chef::Platform::ServiceHelpers.service_resource_providers
 => [:upstart, :redhat]

I am also gonna forward this to the chef bugtracker. Maybe they will fix it, but for sure a fix in this cookbook would be a lot easier and faster. Maybe just check if it includes :upstart and not :redhat (ugly but it would fix the problem)
EDIT: I just learned that CentOS 6.6 comes with upstart installed, so this does not seem to be a Chef bug.

Hello!
This was fixed up in 12.4.1. Also the cookbook =)


[usbdev] USB TODOs

Description

[ ] When usbdev is stubbed, register interface is intended to survive, so presumably the alert(s) should too. The only alert is that relating to T-L traffic/register access
[x] PR #17673 - merged in
[ ] Rest of issues tracked in https://github.com/lowRISC/opentitan/issues/17892

https://github.com/lowRISC/opentitan/pull/17673


Change type signature of quantized and its default value for python < 3.11 compatibility

Fixes issue #530

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.Sam Passaglia seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.


Documentation Update: Document support for pyproject.toml file

On the Python language documentation, pyproject.toml is not mentioned for configuring pyright including venv.
Since pyproject.toml is supported, a quick note seems needed.

Fixed in https://github.com/zed-industries/docs/commit/097f268e979b7687272ed2c43bb307082ba265f7.


Fix documentation for order-dependent lists

Description
Fixes #1523
Completed Tasks

[X] I have read the Contributing Guide.
[X] The pull request is complete (implemented / written).
[X] Git commits have been cleaned up (squash WIP / revert commits).
[ ] I wrote tests and ran bundle exec rake locally (if code is attached to PR).

Thanks for the contribution!


Optimize Taskhandler delete queries

1. Why is this change necessary?
The two scheduled tasks can quickly fail due to innodb_lock_wait_timeout being 30-50s by default. For example on many systems tested deleting 1000 cart rows took min 3 to max 5 seconds.
2. What does this change do, exactly?
Adding a do-while loop to execute the query with a limit of 1000 and repeating till executeStatement returns 0
3. Describe each step to reproduce the issue or behaviour.
Add 20.000 carts that are older then 120 days and try to run the scheduled task
4. Please link to the relevant issues (if any).
https://github.com/shopware/platform/issues/2532
5. Checklist

[] I have written tests and verified that they fail without my change
[x] I have created a changelog file with all necessary information about my changes
[] I have written or adjusted the documentation according to my changes
[] This change has comments for package types, values, functions, and non-obvious lines of code
[x] I have read the contribution requirements and fulfil them.

Hey @Isengo1989, thanks for the PR. It is now merged : )


show vehicle route+ID+heading when clicking on its marker.

like clicking on the marker opens a small pop-up beside it. look through the deck-gl or mapbox-gl sites for how to do this.

closing due to merge #23


Update RELEASE_NOTES.md

Description

Checklist

[ ] If I added any command snippets, the steps they belong to follow the prompt conventions (see example).
[ ] If I added a new directory, I also updated .github/CODEOWNERS with the corresponding team in Cray-HPE.
[ ] My commits or Pull-Request Title contain my JIRA information, or I do not have a JIRA.

I'm just here to witness the historical event.


libusb_get_parent

Required Info

Camera Model
R200

Firmware Version

Operating System & Version
Ubuntu 16.04 LTS

Kernel Version (Linux Only)
Linux version 4.4.0-98-generic (buildd@lcy01-03) (gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.4) )

I fellow the instructions of installation;
when I executed realsense-viewer,
it return below:
realsense-viewer: symbol lookup error: /usr/local/lib/librealsense2.so.2: undefined symbol: libusb_get_parent

I have reinstalled the libusb-1.0-0-dev for several times, but still useless.
What should I do?

Hi @ChampionMas, please try to do what they offering at this thread.
Generally, the ldd command on Linux is a tool that prints the dependencies of a binary file. You can try to use it on the realsense-viewer tool ($ ldd realsense-viewer) to figure out where it expecting to find the libusb file.

Hi @ChampionMas
Please use below legacy librealsense1 build instead of librealsense2 which is for D400 series and SR300.
https://github.com/IntelRealSense/librealsense/tree/legacy

Great Respects for @icarpis and @freemanlo ! Thank you !
I tried legacy and fellow the instructions of installation and it did work.
In addition, I found that when compile, we had better add -lusb-1.0 for libusb-1.0.18 or later version. Just like: (for mono source file)
g++ source.cpp -o target -lusb-1.0 -lrealsense `pkg-config --cflags --libs opencv`

or it will show errors:
/usr/local/lib/librealsense.so: undefined reference to `libusb_strerror'
/usr/local/lib/librealsense.so: undefined reference to `libusb_get_port_numbers'
/usr/local/lib/librealsense.so: undefined reference to `libusb_get_parent'
/usr/local/lib/librealsense.so: undefined reference to `libusb_error_name'
collect2: error: ld returned 1 exit status

Hope that helps others !

For me, this error appeared, after initially things working fine. It could be as a result of me installing MatrixVision mvAcquire SDK. Just wanted to point out that in my case this was not due to incorrect version of librealsense as things were working fine.


Fix typo on "autoremapping".

Update warning strings to be consistent with the rest of the code base, which uses "Command or Control".
For example, see line 497 of the same file.

Thanks!


Uninitialized Constant Formulaic (NameError)

Hi, I am trying to integrate Formulaic into my RSpec test suite. But whenever I include the config.include Formulaic::Dsl, type: :feature line in the spec_helper.rb, I am getting the following error.
I am running on Rails <IP_ADDRESS> and the Formulaic version in 0.3.0. I tried the same with another app running on 4.2.5. It's been the same. It would be really great if someone can point what I am doing wrong.
Thank You.
Here is the trace.
/home/p150/Documents/code/booker/spec/spec_helper.rb:18:in `<top (required)>': uninitialized constant Formulaic (NameError)
	from /home/p150/.rbenv/versions/2.3.0/lib/ruby/site_ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in `require'
	from /home/p150/.rbenv/versions/2.3.0/lib/ruby/site_ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in `require'
	from /home/p150/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.4.3/lib/rspec/core/configuration.rb:1295:in `block in requires='
	from /home/p150/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.4.3/lib/rspec/core/configuration.rb:1295:in `each'
	from /home/p150/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.4.3/lib/rspec/core/configuration.rb:1295:in `requires='
	from /home/p150/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.4.3/lib/rspec/core/configuration_options.rb:109:in `block in process_options_into'
	from /home/p150/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.4.3/lib/rspec/core/configuration_options.rb:108:in `each'
	from /home/p150/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.4.3/lib/rspec/core/configuration_options.rb:108:in `process_options_into'
	from /home/p150/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.4.3/lib/rspec/core/configuration_options.rb:21:in `configure'
	from /home/p150/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.4.3/lib/rspec/core/runner.rb:105:in `setup'
	from /home/p150/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.4.3/lib/rspec/core/runner.rb:92:in `run'
	from /home/p150/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.4.3/lib/rspec/core/runner.rb:78:in `run'
	from /home/p150/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.4.3/lib/rspec/core/runner.rb:45:in `invoke'
	from /home/p150/.rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/rspec-core-3.4.3/exe/rspec:4:in `<top (required)>'
	from /home/p150/.rbenv/versions/2.3.0/bin/rspec:22:in `load'
	from /home/p150/.rbenv/versions/2.3.0/bin/rspec:22:in `<main>'

Did you require 'formulaic'?

Yes, I tried that as well.

Can you share your spec_helper


best or last fix

Hey, I tried running your action for my chess.com profile and encountered an error, I don't play bullet at all, so I don't have a "best" rating and thus the action fails. See the image below.

I have created this fix to allow users to choose their best of recent ratings and it defaults to recent ratings.

oh I didn't know that, first time writing an action. Thanks :)
fixed.


[extra7141] Fix AccessDenied issue when ssm:GetDocument

Context
Missing permission was exposing AccessDenied error message.
Also missing check was generating false positive.
Description
Add check to validate access denied when get document from SSM.
Add missing action permission to allow ssm:GetDocument.
Policy actions sorted to keep them in alphabetical order.
Issue #1016
License
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

$ ./prowler -p default -f sa-east-1 -c extra7141
                          _
  _ __  _ __ _____      _| | ___ _ __
 | '_ \| '__/ _ \ \ /\ / / |/ _ \ '__|
 | |_) | | | (_) \ V  V /| |  __/ |
 | .__/|_|  \___/ \_/\_/ |_|\___|_|v2.7.0-24January2022
 |_| the handy cloud security tool

 Date: Wed Feb  2 22:05:13 -03 2022

 Color code for results:
 -  INFO (Information)
 -  PASS (Recommended value)
 -  WARNING (Ignored by whitelist)
 -  FAIL (Fix required)

 This report is being generated using credentials below:

 AWS-CLI Profile: [default] AWS API Region: [sa-east-1] AWS Filter Region: [sa-east-1]
 AWS Account: [xxxx5262] UserId: [AROAxxxx:xxxxxx]
 Caller Identity ARN: [arn:aws:sts::xxxx5262:assumed-role/xxxxx/xxxxxx]

7.141 [extra7141] Find secrets in SSM Documents - ssm [Critical]
       INFO! sa-east-1: Access Denied trying to get document

$ ./prowler -p default -f sa-east-1 -c extra7141
                          _
  _ __  _ __ _____      _| | ___ _ __
 | '_ \| '__/ _ \ \ /\ / / |/ _ \ '__|
 | |_) | | | (_) \ V  V /| |  __/ |
 | .__/|_|  \___/ \_/\_/ |_|\___|_|v2.7.0-24January2022
 |_| the handy cloud security tool

 Date: Wed Feb  2 22:05:46 -03 2022

 Color code for results:
 -  INFO (Information)
 -  PASS (Recommended value)
 -  WARNING (Ignored by whitelist)
 -  FAIL (Fix required)

 This report is being generated using credentials below:

 AWS-CLI Profile: [default] AWS API Region: [sa-east-1] AWS Filter Region: [sa-east-1]
 AWS Account: [xxxx5262] UserId: [AROAxxxx:xxxxx]
 Caller Identity ARN: [arn:aws:sts::xxxx5262:assumed-role/xxxx/xxxxx]

7.141 [extra7141] Find secrets in SSM Documents - ssm [Critical]
       PASS! sa-east-1: No secrets found in SSM Document test-prowler-leo

I couldn't make it FAIL. How it check for secret?
My test document is the one below
{
  "schemaVersion": "2.2",
  "description": "Command Document Example JSON Template",
  "parameters": {
    "Message": {
      "type": "String",
      "description": "Example",
      "default": "Hello World"
    },
    "dbpass": {
      "type": "String",
      "description": "Example",
      "default": "temp123"
    }
  },
  "mainSteps": [
    {
      "action": "aws:runPowerShellScript",
      "name": "example",
      "inputs": {
        "runCommand": [
          "Write-Output {{Message}}"
        ]
      }
    }
  ]
}

$PROFILE_OPT shouldn't be enclosed within "${}" because it contains spaces and we need to extract both values.


License updates for dependabot/github_actions/actions/github-script-6

This PR was auto generated by the licensed-ci GitHub Action.
It contains updates to cached github/licensed dependency metadata to be merged into dependabot/github_actions/actions/github-script-6: (branch, PR).
The licensed-ci action will add comments to this PR with the status of license checks.  Please make any changes needed to fix any status failures on this branch, then merge license updates into your branch.
If updates are unexpected, please check the github/licensed changelog for recent updates.

The licensed-ci GitHub Action updated cached license metadata based on changes to dependabot/github_actions/actions/github-script-6: (branch, PR).
licensed status result: SUCCEEDED
All status checks succeeded, please merge when ready.
Checking cached dependency records for licensed
.............................
29 dependencies checked, 0 errors found.

/cc @dependabot[bot]


[Question] What are recommended parameters for transforming existing 1920x1080 footage?

Hi! I'm trying to transform some existing footage that has resolution 1920x1080 and I'm wondering what are the correct parameters to set during inference. I.e., I know there's width/height, crop top/left, and target width/target height.
My intuition is to

set target_width/target_height to  672 x 384 (most closely matches 1920x1080)
set og_width/og_height to 1920x1080
Resize the footage to 672x384 before feeding it into the model

Does this sound correct?

For usage with ControlNet @ inference? Yes that is correct!