Control protocol header format

How do we separate the individual parts of the header?

We could use separate frames, but that creates some overhead
We could use specified lengths
We could use a separation character
a mix: Separate recipient and sender with a char and have a fixed length for the messageID. The rest is the conversation ID

I propose the following scheme with a separation character (i.e. forbidden character):
"Recipient name";"Sender name";"ConversationID";"MessageID".
The message ID might be some binary data (#16). So we can look for the first three semi colons and the rest is the message ID independent of the ASCII representation.
Consequences:

ConversationID/reply-reference cannot contain a semi colon and therefore cannot be any binary number.

Alternative:
If we have a messageID with a fixed byte length, we can do the following header: "Recipient";"Sender";MessageIDConversationID
The code would look for the first two semi colons for the recipient and sender. Afterwards it takes a certain number of bytes for the message ID and the rest is the ConversationID, which can be flexible in length and can be have any bytes value.

I think the most convenient way would be to use separate frames. Yes, it does create a bit of overhead, but I do not think this is critical for us - speed is important, but readability and versatility are too. With frames, we can use zmq's multipart framework, and directly get a well separated list of items, where we can have the first 1 to n frames as header, and then the n+1st frame (which is also the last) as the payload.
Like this, we do not have problems with forbidden characters, or need anything extra in the implementation code which separates them, we can simply use whatever methods are present (in python we have framelist.pop(0)) in the programming language in use for an implementation. So essentially, we always have some overhead, I think, either directly inside of zmq, or in our implementation around it, just that if we do it within zmq we do not have to concern ourselves too much with it.

For the names, the dot "." is already reserved due to a namespaces.
Ok, so we keep it simple: Frame separation is already needed, so why not use that even more instead of inventing new methods.
In that case (separated by frames), we can choose any order. I'd say: Recipient, Sender, MessageID (because necessary), ConversationID (because optional).

As I understood it so far, Avro can take should a lot of that burden (but maybe I skim-read too quickly).
How about: we put in the first frame(s) only enough info so that zmq can do its thing (topic for pub/sub filtering, whatever else), and the rest gets put into Avro messages, where this gets (de)composed into fields, and validated, etc., and we don't need to mess around with separation characters or separate frames?
Does that work?
The nice thing about Avro is that we get schemas for our messages, so the structure is "in the code", schemas get exchanged during handshake, so participating Components know what to expect, the schemas are used to validate and process the messages, and we don't have to keep track of those details manually (with hand-wrought formats).

I am not perfectly sure about Avro (I did not look at it as detailed as I looked at zmq), but what we are looking at here will not be included, I think, since it is mostly about routing. For routing, we need separate frames, because when sending a multipart message from the ROUTER socket, the first frame needs to be the recipients ID (I think), and nothing else, so we cannot stuff everything inside of it. Also, the conversationID is to track the ongoing conversation, and be sure that the answer we got is now the one we actually asked right here, and not the answer to a previous question to an instrument which reacted slower, especially if we look at asynchronously working Directors, where we need to be able to tie replies to the requests, on one and the same socket.
I don't think it Avro is supposed to do routing, is it?

but what we are looking at here will not be included, I think, since it is mostly about routing.

Ah, then I got confused, because you guys were mentioning "Recipient, Sender, MessageID, ConversationID ", I thought for the zmq routing part indeed you only need the recipient.
All the other parts I'd have considered part of the message, e.g. the conversationID to associate it with the correct conversation -- this would/couldhappen with an unpacked message anyway, so might as well serialise everything together.
The only thing I was not sure about is that we would duplicate the "recipient" information, once in the first zmq frame, then in the message header (packed in the zmq payload in the avro message).

I don't think it Avro is supposed to do routing, is it?

No, afaik not. But zmq only needs the recipient for the routing, right? all the rest (convo tracking, message id) is stuff that LECO components put on top, right, as zmq is not doing that, either?

Oh, you intend, that we serialize the receiver, sender, message, and conversation IDs via avro?
I intended to do this header (routing information) manually (this issue) and add avro on top (another issue).

Yeah, indeed. If we are already using a specified/schematised message format, why do the header stuff manually separately if we don't need it for the transport layer?

The only requirement, from zmq, is, that the topic of the data protocol is in the first frame. Everything else is up to us.

or the recipient of a control message, IIUC Benjamin's reply above.

The only thing I was not sure about is that we would duplicate the "recipient" information, once in the first zmq frame, then in the message header (packed in the zmq payload in the avro message).

ZMQ does not need the recipient information in the first header. ZMQ does just send messages over tcp connections. Our Coordinators need somehow the "recipient" information to match that (human readable) name to its own (arbitrary, not selectable) zmq socket addresses.
Basically, the data protocol does not need any "zmq" header. We could do everything in avro (or whatever).
I wanted to have a three step communication: zmq manages the connections. Then we have routing information (human readable) to route messages where we want them. Finally some serialization helps to transport data. The last two are arbitrary distinctions in my head.

For routing, we need separate frames, because when sending a multipart message from the ROUTER socket, the first frame needs to be the recipients ID (I think)

@bilderbuchi that first frame is the address the zmq socket assigns to the other side. We map our own names to that (arbitrarily chosen address).

Yeah, indeed, I did. If we are already using a specified/schematised message format, why do the header stuff manually separately if we don't need it for the transport layer?

I understood that Routing, that we set up, as "Transport Layer", therefore my requests in the PR.
It is good to have the routing information external to the payload, such that the Coordinators do not have to deserialize the whole payload, just to know, to whom to deliver the message.

It is good to have the routing information external to the payload, such that the Coordinators do not have to deserialize the whole payload, just to know, to whom to deliver the message.

Thanks, I'll have to think.
We ahve to find out if a Coordinator could pull out routing info from an Avro message without unpacking the whole thing -- unpacking a multi-MB oscilloscope trace or image just to get the recipient humand-readable name would indeed be quite wasteful. Maybe the header is distinctly deserialisable.
At the same time, we need to make sure that the Avro message retains enough context (I had considered the routing info just that) so that it remains "meaningful"/interpretable once its separated from the transport layer and goes through our system.

I try to explain how a zmq ROUTER socket works:

the socket binds to some host and port (as known from tcp).
Other sockets (from different programs, for example Components) connect to this ROUTER socket, let's call these other sockets "peer". (I think, that each connecting peer gets assigned an address, some random byte number, at this point)
Whenever some peer sends a message, for example "some example message", the ROUTER socket returns (at readout) the peer's address (i.e. "iojy")  and the message, that is for example "iojy", "some example message" (as two frames).
If you want to send a response to that peer (and not any other one), you send again two frames: "iojy", "my response" to the ROUTER port. Now the ROUTER knows to which peer it has to send the message and it sends (over the wire) only "my response" as the underlying tcp connection is 1on1.
These addresses exist ONLY in that single ROUTER socket and are there, to distinguish the different connections (as the ROUTER is 1 to many).
If a peer destroys its socket and connects with a new socket, it will get a new address!

What our Coordinators have to do, is to know, the peer better than by its (arbitrary) address. Therefore we introduced (for our own code) the recipient name. Now a Coordinator can look up the address that corresponds to that recipient's name and send it a message.
For ease of usage, we kept the recipient's name and the sender's name and message id (basic message information) outside of the payload.
That way, we can route messages without knowing the content. Similar to a letter: Outside you have recipient and sender (in case of an error or rejection) and a date (poststamp).
This issue is exactly about that envelope.

Thanks, that's informative!
I have read the Avro spec, and there does not seem to be anything in it regarding selectively unpacking a message. There was a section on message frames, but that doesn't seem applicable for us.
Therefore, I agree now that the routing information should not live exclusively in the Avro message, and has to be included in the zmq message somehow.
When planning the message format (this will not be restricted to the header, I think), please keep in mind the further flow of the message -- how do we keep the routing info near/associated with the Avro message - do we wrap both in some container? duplicate the info in the Avro message? Do both parts get deserialised into some data structure upon arrival (that is then passed on to the business logic? Something else?
Also, by gut feeling I would prefer if the header fields are separated by something more resilient than a magic character (zmq frames sounds fine), even if we pay a little overhead cost. If that should turn out to be a bottleneck later, we can always change that then.

(I think, that each connecting peer gets assigned an address, some random byte number, at this point)

This does not necessarily need to be true. If the peer first sets an identity to itself, this identity is what the ROUTER socket will see. If the ROUTERs are allowed to choose the identities of peer Components by themselves (random bytes), how would you know that identity (even more so if you do a second hop through a second Coordinator) when writing your Director which tries to request data from a specific Actor? The random peer ID works well if you have a ROUTER connected to a REQ socket, because then the ROUTER takes the first frame from the REQ as their identity, do something with the payload, and answer to it using the identity which it generated (and which contents do not matter for this purpose). In the zmqguide it is sometimes written that the ROUTER cannot start sending messages, because it does not know where to send it in advance. This is only true in certain messaging patterns, especially with all these service-oriented servers which do some jobs for clients, who do not care who actually provides them with some data (imagine a weather information service, as used as example in the zmqguide). We have a fundamentally different problem, in that we want data from very precisely defined Components, and yes, I think the user has to hand out those identities at some point. After all, they need to know whom to talk to, the temperature of which controller they now need/want to change, or the motor setting of a very specific mirror. This needs to be transparent when writing sequences, and we need some names for it.
What is important for the ROUTER-DEALER connection here, is that the DEALER sets their identity BEFORE connecting to the ROUTER, so that the ROUTER receives this set identity, and does not set its own random ID for this particular connection.
For this reason, we really need the Recipients ID and the Senders ID in the header, preferable in the header frames, so we can use them without deserializing whatever payload comes with the message, as discussed above. For me, this is almost as @bmoneke put it:

zmq manages the connections. Then we have routing information (human readable) to route messages where we want them. Finally some serialization helps to transport data. The last two are arbitrary distinctions in my head.

It is just that the human-readable names can be used and are important for the zmq connection management, because this is intimately tied to how zmq routes manages (in our current way of thinking with ROUTER sockets).

Oh, thanks @bklebel for reminding of that possibility to set your own identity beforehand, which I had forgotten (I did not need it).
Here is the corresponding text from the guide https://zguide.zeromq.org/docs/chapter3/#Identities-and-Addresses
With that, it could be necessary to overhaul the routing concept. I have to think about it.

I think the routing concept is just fine, it's only that for sending something between Coordinators, the corresponding sending socket needs to get an additional frame prepended to the rest for routing to the second Coordinator, who in turn needs to scrap that first frame (if this happens to be a ROUTER socket which will make it visible), as well as the first part of the recipientID which contains its own identity (zmq would try to send something to "Coord2.Comp1" which does not exist, because the only zmq identities which might exist are "Coord2" and "Comp1", connected to each other respectively).
So the principle of the routing stays the same, but the frames and implementation might get a bit hairy.

(I did not need it).

 How do you know where you're sending messages? I am intrigued :D - or did you chose a fundamentally different approach to send command/control messages to Actors so far?

Until recently, I did not send to Actors, just to ha handful of Observers and I used a new port number for each one...
Only in my test implementation, sparked by our discussion, did I implement a routing mechanism. I put the receiver name in the first data frame (together with sender name and message ID). Then the Coordinator would create a dictionary of sender names (from data) and their addresses (very first ROUTER frame).
If we use, as we should, routing via frames, we have to change the routing (and framing) a bit. I open a new issue.

We agreed upon using separate frames for the header.
In #34 we found out, that we need more than one frame (for better routing) for the recipient and that it should be the first (few) frames.

Message format is: Namespace|Recipient||More frames to be determined (Namespace frame is optional)
Where do we place the additional information (sender, messageID etc.?)
The sender is added by the Coordinator (in #34), therefore it is fixed to two frames. So we have some options:

Namespace|Recipient||Senderframes|| Content (Separation indicates begin of more data, so number of senderframes is flexible)
Namespace|Recipient||2 Senderframes| More content (Senderframes are the first two content frames)
Namespace|Recipient||Content||Senderframes (Senderframes are the frames after the last empty frame)

As we agreed upon only two Coordinators per message (and even multi-hop would not increase the number of address frames), we could do the following (based onproposal 2), regarding the full header:
A. Namespace|Recipient||2 Senderframes| MessageID|ConversationID|Content
B. Namespace|Recipient||2 Senderframes| Content (including MessageID and ConversationID, determined by message layer)
I prefer option 2, but I'm unsure whether suboption A or B, I tend slightly to B

Current state is:

Recipient full Id|Sender full Id|More frames, because the first two frames are necessary for the routing.

Now the question is, what else should enter the header, what the content.
I think it is good, if we have at least a third header frame, which indicates the type of content, such that the recipient knows how to interpret it.

The third frame could be one byte for the content type (for example binary, avrò, json..., maybe also the serialization version) and the message ID. (my preference)
Alternatively, we could require, that the third frame (if present) is always serialized in our serialization scheme (current contender is avro). Then this third frame can indicate whether bytes follow thereafter (for example) and it could contain message ID and reply reference.

If the third frame contains "metadata", we would remain more flexible.
E.g. if we place the content type in there, in case we need special rules for sender/recipient of SIGN_IN messages, we could determine that the message is a SIGN_IN without deserializing the payload frame.
Also, we would be free to put more stuff in there like the message id etc.
Maybe include a version identifier, so we can deal with it cleanly if the format of the third frame evolves.

For what else we could/should include in the header, maybe this can serve as a "reading inspiration". (Not judging technically if that is applicable/relevant for us).

I think it would be a good idea to have one or more additional header frames for metadata. The CoAP protocol @bilderbuchi linked to is quite interesting I think, especially regarding the header. From an implementation point of view, it might be good to put the header frames in the order in which they are needed when reading them. Initially, having the routing frames at the front was motivated by using them together with the zmq identity itself, but this is not a thing for us anymore. Therefore the order could be (strongly taking from CoAP):

version
type (i.e. Sign_IN/GET/whatever)
recipient full ID
sender full ID
message ID and conversation ID (could also be separated into multiple frames, with the conversation ID coming first)
payload

The version of the protocol in use is a good point from CoAP which @bilderbuchi linked to, although this opens a fully new rabbit-hole of "how to deal with messages from a different version than my current one", as seen from a particular Components' view. Still, that might be better than "wth do you want to tell me with that? ". Also, if we put the version all the way to the front, we are free to change anything and everything behind that from version to version (even though it might not be advisable in itself), and still get a working "this version is no longer supported" response, without errors, since the rest of the message does not need to be looked at.
In general I would like to avoid serializing this too much, I simply do not know how we would benefit from it, if you have a good reason I am open for it.

Coap is a message protocol over a tcp connection, so no routing is necessary.
We, however, have to include routing information.
The Coordinators have to use the routing information to get the messege, wherever it should go.
Therefore, I'd put the routing information in front of the type.
Maybe even in front of the version.
It the version is first, the formatting of the routing information could change potentially (depending on the version.
If the routing is fixed, the version could be combined with different things (message ID...) into one frame.

I think version first is a safer choice, in case we want to make changes to the routing info in the future (e.g. allow more levels to address Paramaters directly, like Node1.CA.voltage, or for deeper hierarchies).
Question: Is it even necessary (with the current status) anymore to have separate frames for the sender/recipient, or could we just have one header frame (with a number of bytes containing all the necessary info)?
Or is that inconvenient from a technical/performance point of view? IIRC, we stepped away from mutating the sender/recipient info along the message flow, so the header would be unchanged for the message lifetime (which seems convenient), correct?
I would not serialise the header info at all, just encode info into an array of bytes (easily decoded).

By separating it into different frames, we can avoid additional forbidden symbols, which we would otherwise need to separate the information. Also, at least in python, you can do a very straightforward implementation for the CCoordinator:
msg = zmq.recv_multipart()
version = msg.pop(0)
if version is not my_version:
    ...send an error
type = msg.pop(0)
if type is SIGN_IN:
    do something myself
else:
    recipient = msg.pop(0)
    ...look up the recipient in the zmq.identity table
    zmq.send_multipart([zmq ID for recipient] + [version, type, recipient] + msg)

Here also my reason to put the type before the routing becomes maybe more apparent, in that we can pick out the SIGN_IN easily. In the example above, the messageID and conversationID are in the rest of the msg, this could be a separate frame, but it is not relevant for the Control Coordinator

Just an implementation detail: I would not pop parts of the msg, as the Coordinator normally gives the message as is to the recipient (or other Coordinator).
We do not need to pick the SIGN_IN first: The Coordinator looks, whether a message is for itself (Recipient==self) and then it handles the message, be it SIGN_IN or something else.
In most cases, the Coordinator looks in vain at the type.
In fact, it has too look twice at the type, if the message is destined at the Coordinator:

Look at type: It is not sign_in
look at recipient: Oh, it is for me.
Look at type again: What should I do?

With a fixed Recipient name (Coordinator or None or whatever we decide), the Coordinator does not need to look at any other field than the Recipient.
We allowed to leave out Recipien namespace, therefore "Coordinator" is an allowed recipient ID.
We could also allow to leave the Recipient empty for the SIGN_IN message. So:
msg = zmq.recv_multipart()
if msg["version"] != my_version:
    send_error()
if msg["recipient"]:
    do_routing()  # might include to myself and then doing sign_in()
elif msg["type"] == SIGN_IN:
   sign_in()
else:
   send_error()

Note: I used dictionary keys as placeholders for the actual position.

I would not serialise the header info at all, just encode info into an array of bytes (easily decoded).

👍

IIRC, we stepped away from mutating the sender/recipient info along the message flow, so the header would be unchanged for the message lifetime (which seems convenient), correct?

You're correct.

By separating it into different frames, we can avoid additional forbidden symbols, which we would otherwise need to separate the information.

👍 For that reason, I'm for four header frames: version, recipient, sender, "content header" (type etc.).

We allowed to leave out Recipien namespace, therefore "Coordinator" is an allowed recipient ID.

Only for the signin, right? In that case I guess we might as well leave it out, completely.

No, we allowed the Recipient to be only the Component ID, if it is in the same Node.
The Sender ID has always to be complete.

Ok, thanks. We could keep in mind that we could revisit that if it makes our logic flow less complex.

For that reason, I'm for four header frames: version, recipient, sender, "content header" (type etc.).

That looks good to me too, I agree with the reasoning about looking at the type twice, and very often unnecessarily looking at the type.

We reached a conclusion, I close this issue

Should the issue not be closed when the conclusion/relevant information turns up in main? I.e. when a PR that records the issue's results is merged?

I think so too - it is a bit cumbersome to have those issues still open even though we reached a conclusion, but as long as it is not done with a merged PR, I would also leave the corresponding issues open. I am going to reopen this now.

it is a bit cumbersome to have those issues still open even though we reached a conclusion

That was my reasoning for those many issues with a single PR, as an exception to the default rule to close with PRs.
So, no exception 😁

Well, you can summarize the conclusion/takeaway in a "last" comment. Then people looking at the issue know immediately what the result of a lenghty discussion was.
When it has been added in a PR, you can link to that, too, in a comment.
If you put "closes #blabla" in the description/first post of a Pr, that issue will automatically be closed when the PR is merged.

Good, now I wanted to write a summarizing comment, in which I say that we agreed on everything, and find that I am not sure we did ^^
Here is the summary so far
Summary
We agreed on the following Control protocol header format:

version
recipient
sender
content header

where "content header" includes
not agreed on
the content header:
it could include the message type (e.g. SIGN_IN), as well as the conversation ID and message ID, separated by....a ";"? Or do we want to break that into multiple frames too?

For the transport layer, it does not matter (so for my PR it is done).
The format of the header (four frames) is done.
The content of these frames is not yet fully agreed upon (version and content header).
The content header enters the message layer, I'd say.
The version formatting is also open, but can be discussed separately.

Yes, let's discuss the content of these frames separately.


Snapshot: fix hostpath not describing snapshot properly

Here's a small fix for the DescribeSnapshot method in hospath. Now it returns conditions based on the ones in provided VolumeSnapshotData. Given how hostpath works this should be sufficient I think.

/lgtm

/lgtm

/assign

/lgtm
/approve


LinearRelative, Mixture, and Prior

Resolves #66 and #75.
WIP on Mixture

I'm going to merge all these and then we can confirm everything works against main (fix issues in new issues), and when all good we can tag.


hyperledger-fabric: init at 1.3.0

Motivation for this change
closes https://github.com/NixOS/nixpkgs/issues/52542
cc: @rsoeldner
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[X] NixOS
[X] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[X] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Assured whether relevant documentation is up to date
[X] Fits CONTRIBUTING.md.

Looks fine and builds locally, I can see the help menus. Though I won't be any help in testing the function ATM.

@worldofpeace I'm using this nix derivation already and (at least until now) it works perfectly.

@worldofpeace I'm using this nix derivation already and (at least until now) it works perfectly.

Sounds good, I'll merge this :+1:


Largest Contentful Paint image was not lazily loaded - Typo?

"Largest Contentful Paint image was not lazily loaded"
Isn't actually "Largest Contentful Paint image was lazily loaded"? It shouldn't be lazy-loaded right?

Sorry https://github.com/GoogleChrome/lighthouse/issues/13048


[CP Staging] fix: empty list does not turn to empty state screen

Details

The root cause of this issue was that we assumed that the backend clears all unneeded keys in the SearchResults data snapshot. Now, we are checking if the data contains at least one transactions key.
Fixed Issues

$ https://github.com/Expensify/App/issues/44803
PROPOSAL: N/A
Tests

Precondition: User has only unpaid expenses.

Go to Search > Shared.
Select all the expenses.
Click on the dropdown > Delete.
Delete the expenses.
The empty state screen will show up after deleting all the expenses.

Offline tests

N/A
QA Steps

[ ] Do the Tests.

PR Author Checklist

[x] I linked the correct issue in the ### Fixed Issues section above
[x] I wrote clear testing steps that cover the changes made in this PR

[x] I added steps for local testing in the Tests section
[x] I added steps for the expected offline behavior in the Offline steps section
[x] I added steps for Staging and/or Production testing in the QA steps section
[x] I added steps to cover failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[x] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)
[x] I tested this PR with a High Traffic account against the staging or production API to ensure there are no regressions (e.g. long loading states that impact usability).

[x] I included screenshots or videos for tests on all platforms
[x] I ran the tests on all platforms & verified they passed on:

[x] Android: Native
[x] Android: mWeb Chrome
[x] iOS: Native
[x] iOS: mWeb Safari
[x] MacOS: Chrome / Safari
[x] MacOS: Desktop

[x] I verified there are no console errors (if there's a console error not related to the PR, report it or open an issue for it to be fixed)
[x] I followed proper code patterns (see Reviewing the code)

[x] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick)
[x] I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
[x] I verified that comments were added to code that is not self explanatory
[x] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[x] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method

[x] If any non-english text was added/modified, I verified the translation was requested/reviewed in #expensify-open-source and it was approved by an internal Expensify engineer. Link to Slack message:

[x] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[x] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is either coming verbatim from figma or has been approved by marketing (in order to get marketing approval, ask the Bug Zero team member to add the Waiting for copy label to the issue)
[x] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[x] I verified the JSDocs style guidelines (in STYLE.md) were followed

[x] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[x] I followed the guidelines as stated in the Review Guidelines
[x] I tested other components that can be impacted by my changes (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar are working as expected)
[x] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[x] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[x] I verified that if a function's arguments changed that all usages have also been updated correctly
[x] If any new file was added I verified that:

[x] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[x] If a new CSS style is added I verified that:

[x] A similar style doesn't already exist
[x] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(theme.componentBG))

[x] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[x] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[x] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[x] If the PR modifies a component or page that can be accessed by a direct deeplink, I verified that the code functions as expected when the deeplink is used - from a logged in and logged out account.
[x] If the PR modifies the UI (e.g. new buttons, new UI components, changing the padding/spacing/sizing, moving components, etc) or modifies the form input styles:

[x] I verified that all the inputs inside a form are aligned with each other.
[x] I added Design label and/or tagged @Expensify/design so the design team can review the changes.

[x] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[x] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.

Screenshots/Videos
This functionality is only supported on wide screen.

MacOS: Chrome / Safari

https://github.com/Expensify/App/assets/104823336/b144f318-2cc3-4f83-ac05-f1f83c59f91d

MacOS: Desktop

https://github.com/Expensify/App/assets/104823336/b5c036ee-267e-44e3-9227-867b513549a5

:hand: This PR was not deployed to staging yet because QA is ongoing. It will be automatically deployed to staging after the next production release.

🚀 Cherry-picked to staging by https://github.com/tgolen in version: 9.0.4-5 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

@Expensify/applauseleads please QA this PR and check it off on the deploy checklist if it passes.

🚀 Deployed to staging by https://github.com/luacmartins in version: 9.0.4-6 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

🚀 Deployed to production by https://github.com/Julesssss in version: 9.0.5-13 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

🚀 Deployed to production by https://github.com/thienlnam in version: 9.0.6-8 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅


Unicode/string parsing error

I'm trying to parse structured metadata from this url.  I first executed this code on the example URL https://www.optimizesmart.com/how-to-use-open-graph-protocol/:
import extruct
import requests
from w3lib.html import get_base_url

def extract_metadata(url):
    r = requests.get(url)
    base_url = get_base_url(r.text, r.url)
    data = extruct.extract(r.text, base_url=base_url)
    return(data)

url = 'https://www.optimizesmart.com/how-to-use-open-graph-protocol/'
data = extract_metadata(url)
print(data)

And works just fine. However, this block of code:
url = 'https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/G4TBLF'
data = extract_metadata(url)
print(data)

returns this error
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-13-f0db0dd65eaf> in <module>()
      1 url = 'https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/G4TBLF'
----> 2 data = extract_metadata(url)
      3 print(data)

<ipython-input-3-25c85aeebf1a> in extract_metadata(url)
      2     r = requests.get(url)
      3     base_url = get_base_url(r.text, r.url)
----> 4     data = extruct.extract(r.text, base_url=base_url)
      5     return(data)

/usr/local/lib/python3.5/dist-packages/extruct/_extruct.py in extract(htmlstring, base_url, encoding, syntaxes, errors, uniform, return_html_node, schema_context, **kwargs)
     50         raise ValueError('Invalid error command, valid values are either "log"'
     51                          ', "ignore" or "strict"')
---> 52     tree = parse_xmldom_html(htmlstring, encoding=encoding)
     53     processors = []
     54     if 'microdata' in syntaxes:

/usr/local/lib/python3.5/dist-packages/extruct/utils.py in parse_xmldom_html(html, encoding)
     14     """ Parse HTML using XmlDomHTMLParser, return a tree """
     15     parser = XmlDomHTMLParser(encoding=encoding)
---> 16     return lxml.html.fromstring(html, parser=parser)

/usr/local/lib/python3.5/dist-packages/lxml/html/__init__.py in fromstring(html, base_url, parser, **kw)
    874     else:
    875         is_full_html = _looks_like_full_html_unicode(html)
--> 876     doc = document_fromstring(html, parser=parser, base_url=base_url, **kw)
    877     if is_full_html:
    878         return doc

/usr/local/lib/python3.5/dist-packages/lxml/html/__init__.py in document_fromstring(html, parser, ensure_head_body, **kw)
    760     if parser is None:
    761         parser = html_parser
--> 762     value = etree.fromstring(html, parser, **kw)
    763     if value is None:
    764         raise etree.ParserError(

src/lxml/etree.pyx in lxml.etree.fromstring()

src/lxml/parser.pxi in lxml.etree._parseMemoryDocument()

ValueError: Unicode strings with encoding declaration are not supported. Please use bytes input or XML fragments without declaration.

Any idea what is going on here?  Seems like an lxml.etree parsing error?  Can I somehow modify r.text to fix this error?  Any help is appreciated...

You cannot parse a unicode string that contains an encoding declaration, see here. In Harvard's html the encoding is specified in the first line. You can just encode the text before passing it to extruct: data = extruct.extract(r.text.encode('utf8'), base_url=base_url)
Strings encoding are very confusing, this article helped me understanding the basis :)

Thank you @Kebniss, worked perfectly!  Your help (and your addition to my must-read list) is much appreciated!

I think it's possible to make extruct work on such cases, and it should be a responsibility of the library.

Not about Unicode, but I got an issue when parsing from json-ld structure has hex string in this url
The root cause is because of the description which is having hex string, and it is fixed by removing \x according to this article
I think this case should be handled as well. Does anyone have any idea?


Port FSTLevelDB to C++

... along with its LRU delegate.

Note that in addition to the diffbase on #3754, this PR does not yet build because the existing LevelDB components haven't yet been cut over to using this yet (coming in #3756).
Consequently the actual merge target for this PR is a persistence-master branch that I'll merge once all dependent PRs have also been reviewed.

Changes since your last review are effectively cosmetic but are nontrivial, so PTAL despite your approval:


[GHSA-rq7c-8f3g-q36h] A Denial of Service vulnerability exists in qs up to 6.8...

Updates

Affected products
CVSS
Description
Severity
Summary

Closes https://github.com/ljharb/qs/issues/436.

Hey @ljharb. Thanks for the contribution, but I can't actually accept this for technical reasons. This is in our unreviewed pool for which the path out is to add package info. If it helps, the lack of package info means that this advisory is invisible to anything using out data.

@darakian thanks, the only thing i care about is that this irresponsibly reported invalid CVE doesn't create noise for my users :-) feel free to update or replace this PR if there's a better way to ensure that.

You shouldn't see anything from this. If you do see any alerts please ping me and I'll get that corrected for you asap. I'll go ahead and close this if you're cool with that.


Σ => DefiniteIntegral

I think this makes more sense and allows the user to write:
d = Interval([-1.,1.])
Σ = DefiniteIntegral(d)

One option down the road is to override type manipulation, e.g.:
+{OT<:Operator}(A::Operator,::Type{OT})=A+OT()

Then we could
typealias Σ DefiniteIntegral
and then
I+Σ
would resolve.  This would avoid having to define Σ everytime


test: cleanup semaphore after usage

On systems that use System V semaphores, it is necessary to destroy
semaphores to free up system resources when the process finishes
execution.

CI: https://ci.nodejs.org/view/libuv/job/libuv-test-commit/486/

Unrelated flake on the win2012 buildbot, otherwise green.  Landed in bf144cee4, thanks John.


Specify document type without needing to save it

Lately, as a software developer, I've been working with multiple, interconnected (yet unrelated) systems to provide additional functionalities for each one.
As a result, during my tests, I need to parse and understand outputs with wildly varied formats.
VSCode has been of great help to understand these outputs - colored syntax and the ability to format text in a file is a big time saver when doing this.
Thing is, sometimes the output I get does not always come as a file, per se. Sometimes I need to deal with data copied & pasted from other tools (such as Visual Studio, when debugging). So, for example, when I need to quickly look at an XML output of a process, most of the time I need to copy the output into VSCode, save it to a new file with the correct extension (usually at the Desktop folder), and only then VSCode recognizes it as an XML file that I can format and visualize correctly.
It would be of great help if I had a way to quickly specify the type / format of the text I'm currently pasting into VSCode without resorting to save a blob of text that I won't need to see again after 5 minutes, as a file with a specific extension, taking unnecessary space into my hard drive.
For example, being able to rename (and specify an extension to) a newly opened (but not yet saved) document, by clicking on its name in the upper tab bar, does the trick for me.

You can change the language mode by clicking "Plain text" in the status bar or running the "Change Language Mode" command. Does that help?

Oh. Yes, that actually helped.
My apologies. Wasn't aware of that feature.
Thanks!


COM port hang in WSL

I have as system debugging module called a Black Magic Probe (info is here: https://github.com/blacksphere/blackmagic) This system runs a GDB server on the embedded processor and presents itself to the host computer as two serial ports over USB. One is the GDB server, and one is a local UART that you can connect to the system under test.
Typically in gdb you would connect to the target using the command
target extended-remote /dev/ttyACM0
That is how it would work on say Ubuntu. On WSL I use target extended-remote /dev/ttySx where x is replaced with the COM port number for the gdb server port.
After freshly plugging in the BMP, this gdb command hangs, hard. The only way to recover is to unplug the BMP so that the serial device errors out.
It also fails if I am using the Windows native version of the arm embedded tools from ARM.  With a freshly plugged in device.
However, if I run Putty and open the serial device, it show the $OK that indicates the GDB server is waiting. And now if I go back to WSL and try it, it works fine. And will continue to work until the next time I unplug and re-plug in the USB device.
You can read the ACM implementation in the BMP code repo above. It is pretty vanilla. I'm trying to figure out what is different after Putty has talked with it. The only difference in the stty -a -F /dev/ttySx output is that before Putty the speed is 0 and after Putty the speed is 9600. I tried manually setting the speed using stty on a freshly plugged in device and it still didn't work. Needed to open it up with Putty first.
Any thoughts on how I might go about debugging this further?

I've noticed a similar problem when I was working with an Arduino. I could only upload to the device if I had uploaded from the IDE first. I thought it was a bug with Arduino but The symptoms were the exact same.

I'm running into this same issue. The putty workaround is much appreciated but it would be great to see this get sorted out.

Any updates on this one? I've been opening putty to "initialize" the port every time I connect the USB device, for at least a year now.
The BMP COM port works without extra configuration in Windows/Ubuntu/Mac but not WSL, so it seems like this is an issue specific to WSL?

For anyone following along with this issue; I took the advice from @Brian-Perkins above and wrote a small program to check the device state with the TIOCMGET ioctl.
Here's the code:
#include <termios.h>
#include <fcntl.h>
#include <sys/ioctl.h>
#include <stdio.h>
#include <unistd.h>

#define DEVICE "/dev/ttyS4"

#define BIT_SET(a, b) ((a & b) == b)

int main(void)
{
    int fd, serial;

    fd = open(DEVICE, O_RDONLY);
    ioctl(fd, TIOCMGET, &serial);
    printf("%s TIOCMGET 0x%02x\n", DEVICE, serial);

    printf("TIOCM_LE \t%i\n", BIT_SET(serial, TIOCM_LE));
    printf("TIOCM_DTR \t%i\n", BIT_SET(serial, TIOCM_DTR));
    printf("TIOCM_RTS \t%i\n", BIT_SET(serial, TIOCM_RTS));
    printf("TIOCM_ST \t%i\n", BIT_SET(serial, TIOCM_ST));
    printf("TIOCM_SR \t%i\n", BIT_SET(serial, TIOCM_SR));
    printf("TIOCM_CTS \t%i\n", BIT_SET(serial, TIOCM_CTS));
    printf("TIOCM_CAR \t%i\n", BIT_SET(serial, TIOCM_CAR));
    printf("TIOCM_CD \t%i\n", BIT_SET(serial, TIOCM_CD));
    printf("TIOCM_RNG \t%i\n", BIT_SET(serial, TIOCM_RNG));
    printf("TIOCM_RI \t%i\n", BIT_SET(serial, TIOCM_RI));
    printf("TIOCM_DSR \t%i\n", BIT_SET(serial, TIOCM_DSR));

    close(fd);
}

Withe the device freshly plugged in, within WSL I get:
$ gcc serial.c -o serial && ./serial
/dev/ttyS4 TIOCMGET 0x04
TIOCM_LE        0
TIOCM_DTR       0
TIOCM_RTS       1
TIOCM_ST        0
TIOCM_SR        0
TIOCM_CTS       0
TIOCM_CAR       0
TIOCM_CD        0
TIOCM_RNG       0
TIOCM_RI        0
TIOCM_DSR       0

and after running Putty to "activate" the port for WSL I get
$ gcc serial.c -o serial && ./serial
/dev/ttyS4 TIOCMGET 0x06
TIOCM_LE        0
TIOCM_DTR       1
TIOCM_RTS       1
TIOCM_ST        0
TIOCM_SR        0
TIOCM_CTS       0
TIOCM_CAR       0
TIOCM_CD        0
TIOCM_RNG       0
TIOCM_RI        0
TIOCM_DSR       0

I then altered the code above to set the "correct"(?) state using TIOCMSET before closing the device in the original code above:
    serial = 0x06;
    ioctl(fd, TIOCMSET, &serial);

After TIOCM_DTR was set to 1 I was able to connect to a freshly connected BMP debugger without having to open the device in Putty.
From here I'm able to have my flash.sh script call the small program above before starting arm-none-eabi-gdb, which solves my problem of having to "initialize" the port with Putty each time I plug it in. This solution is good enough for me, though I'm sure many others have run into this issue and given up.
I'd appreciate any input on if this is something which should be fixed in WSL or on the devices side? To me it still appears to be WSL bug since I don't see this behavior on MacOS, Windows, Ubuntu (native), although it's possible all of those OSes could be masking bad device behavior.


Rewrite code to make xo happy without any rules

The long-term goal is to comply with all of xo's standards, so that we can remove these rules. I already made the code comply to many rules, but these are left.

[ ] react/prop-types
[ ] react/jsx-no-bind
[ ] new-cap
[ ] complexity
[x] no-warning-comments (https://github.com/zeit/hyper/commit/630f0f4504532ed20c0a706ded08d7d225be5ec4)
[x] react/jsx-filename-extension
[x] react/no-danger (https://github.com/zeit/hyperterm/pull/756)
[x] react/no-string-refs (https://github.com/zeit/hyperterm/pull/766)
[x] react/jsx-key (https://github.com/zeit/hyperterm/pull/767)
[x] quote-props
[x] import/no-extraneous-dependencies
[x] no-nested-ternary

I can work on this!

The changes for this issue are massive in some cases. I'm planning to send several PRs for each rule fulfilled.

@davegomez Cool, thanks! 😘

react/prop-types: Since we don't have any prop types from before, wouldn't it be better to add flow? It can be added incrementally as well, and we don't have to use it for anything else than props.
react/no-danger: https://github.com/zeit/hyperterm/pull/723#discussion_r80324406 (the only place we use dangerouslySetInnerHTML it's the only way we can do it, and as the link mentions I think the name covers it more than enough by itself)

Context of why those rules should be enabled: https://github.com/zeit/hyperterm/pull/723#discussion_r79976176

react/no-danger: #723 (comment) (the only place we use dangerouslySetInnerHTML it's the only way we can do it, and as the link mentions I think the name covers it more than enough by itself)

Then I would recommend just disabling it inline where it's used with a ESLint directive:
// eslint-disable-line react/no-danger

So are we gonna add the Proptypes or don't?
And I agree with @sindresorhus that we should use the inline comment to disable the react/no-danger rule.

@leo @sindresorhus How do we want to handle quote-props?
It seems like the only place we are using them is for CSS pseudo-classes (e.g., :hover), and it seems rather silly to have to quote all the properties off of that 😅
Should we disable with inline comments?

I don't suppose we could switch the xo rule to as-needed?

@maxdeviant Done: https://github.com/sindresorhus/eslint-config-xo/commit/abe5ed61425b143f2ee70e0734f0c26ff7f77a55 I've been meaning to change it. It sounded good initially, but became annoying after awhile. Will be part of the next XO release being released soon (in a few days to a week).

@sindresorhus Okay, sounds good :)
I'll double check, but once we update XO then we should able able to re-enable the rule and cross quote-props off the list :tada:

@leo As per @sindresorhus' comment, no-nested-ternary no longer applies, so we can remove from this list once we update our version of XO.

@maxdeviant Got it!

@leo My bad, forgot to mention that part :sweat_smile:

We don't use xo in Hyper anymore... 😊


Enable Sendability for AsyncStream and AsyncThrowingStream

This makes AsyncStream and AsyncThrowingStream both conditionally Sendable when their Element is Sendable and removes the runtime assertion associated with that behavior.

@swift-ci please smoke test

@swift-ci Please Test Source Compatibility

@swift-ci please test linux

@swift-ci please smoke test


Flutter wrapper to bundle flutter within a project

It would be ideal if all devs and CI would use the exact same version of flutter to build and develop an app. Right now it's hard to bundle flutter within a repository. Git submodule would be one solution but it is currently not working https://github.com/flutter/flutter/issues/3770.
Gradle had the same problem and solved this with the gradle wrapper, automatically downloading the gradle binaries to a fixed version which is part of the projects repository. Instead of calling gradle everybody is used to call ./gradlew when executing gradle tasks.
There should be a ./flutterw which does exactly this, downloading the correct flutter version based on a version file which can be part of the repository.
working flutterw implementation
Here is my current ./flutterw implementation which works by putting it top level into a flutter project:
#!/bin/bash

# Use this flutter wrapper to bundle flutter within your project to make sure everybody builds with the same version
# put `flutterw` into the root of your flutter project and execute it. It will download all dependencies automatically when calling it.
# Use `./flutterw upgrade` to upgrade the flutter version, using the channel defined in flutter/wrapper/flutter-wrapper.properties

FLUTTERW_DIR="$(pwd -P)/flutter/wrapper"
FLUTTER_DIR="$(pwd -P)/.flutter"
WRAPPER_PROPERTIES="$FLUTTERW_DIR/flutter-wrapper.properties"
FLUTTER_REPO_URL="https://github.com/flutter/flutter.git"

DEFAULT_CHANNEL="alpha"

# Check if wrapper config file exists otherwise create with default values
if [ ! -d $FLUTTERW_DIR ]; then
    mkdir -p $FLUTTERW_DIR
fi

# Create wrapper properties file if it doesn't exit
if [ ! -f $WRAPPER_PROPERTIES ]; then
    # Use alpha here as recommended, will be replaced with explicit sha1 once cloned
    echo "flutter_channel=$DEFAULT_CHANNEL" > $WRAPPER_PROPERTIES
fi

# read all properties from config file
while IFS='=' read -r key value; do
    key=$(echo $key | tr '.' '_')
    eval "${key}='${value}'"
done < "$WRAPPER_PROPERTIES"

# Clone Flutter when not already cloned in project dir, the outer git repo will ignore this sub repo
if [ ! -d $FLUTTER_DIR ]; then
    echo "Downloading Flutter with wrapper"
    git clone $FLUTTER_REPO_URL $FLUTTER_DIR
fi

upgradeWrapperConfigFile () {
    FLUTTER_SHA1="$(git -C $FLUTTER_DIR rev-parse HEAD)"
    if [ ! -z "$FLUTTER_SHA1" ]; then
        # Write config file with updated version
        echo "flutter_channel=$flutter_channel" > $WRAPPER_PROPERTIES
        echo "flutter_version=$FLUTTER_SHA1" >> $WRAPPER_PROPERTIES
    else
        echo "Error: could not get version from flutter repo"
        exit 1
    fi
}

FLUTTER_SHA1="$(git -C $FLUTTER_DIR rev-parse HEAD)"

# When wrapper file version was changed, update flutter accordingly before executing
if [ "$flutter_version" != "$FLUTTER_SHA1" ]; then
    # use version when available, otherwise the channel on initial checkout
    if [ -z $flutter_version ]; then REV="$flutter_channel"; else REV="$flutter_version"; fi

    echo "Change Flutter version to $REV"

    git -C $FLUTTER_DIR checkout $REV -q
    upgradeWrapperConfigFile

    # when changing version also update binaries
    echo "Upgrading engine"
    $FLUTTER_DIR/bin/flutter precache
    $FLUTTER_DIR/bin/flutter packages upgrade
fi

# Flutter wrapper operates with detached HEAD. Upgrade won't work, flutter has to be on a branch to pull
# Checkout channel before upgrading
if [[ $@ == *"upgrade"* ]]; then
    echo "Switch to channel '$flutter_channel' before upgrading"
    git -C $FLUTTER_DIR checkout $flutter_channel -q
fi

set -e
"$FLUTTER_DIR/bin/flutter" "$@"

# Automatically update the flutter version in wrapper after upgrade; or at initial run without existing version
if [[ $@ == *"upgrade"* ]]; then
    upgradeWrapperConfigFile
fi

Benefits:

No more "works on my machine". Everybody uses the same flutter version (exact commit)
App version and flutter version are locked together. You'll never build an old app version with a never flutter version which could fail due to breaking changes or misbehave because of implementation detail changes.
Flutter doesn't have to be explicitly installed on CI, neither on any development machine. Basically all the Flutter installation & setup will be obsolete. It's only required to create a new project which could be moved into the flutter IntelliJ plugin.
It would simplify the logic to get the path of the flutter sdk by pointing to a fixed location within the repository ('../../.flutter')
I could replace the logic with

project.ext.setProperty('flutter.sdk', file('../../.flutter').absolutePath)
apply from: "${rootProject.projectDir}/../.flutter/packages/flutter_tools/gradle/flutter.gradle"

It's easy to change the channel or specific commit of flutter. flutter upgrade is great but using a specific commit requires git checkout.

Missing

flutterw.bat for windows users
The IntelliJ plugin should automatically use the local flutter sdk (./flutterw) if available instead of the global one. This is already kind of working because flutter remembers which sdk was last used and it gets picked up by IntelliJ.

The way we've seen teams work around #3770 for now is to make their own fork of flutter/flutter and all pull from that fork.

The main purpose of flutterw is that a flutter version - a file, containing the channel and exact commit hash - is part of the project and added to CVS. By using ./flutterw instead of flutter it is guaranteed that the app builds with the flutter version it was made for.
A fork is not a workaround to bind a specific flutter version to a project. It would fail when someone updates the fork. It would require everyone to update their dev machines and the CI environment to guarantee stable builds for everyone.
Working on different projects would require me to update all projects at once or me to manually change the version everytime I switch the project (multiple times a day).
Also, building a hotfix release for a previous version could cause new problems because the flutter version has also changed.
I don't think I have to add more arguments why the flutter version should be bundled with a project. In the end flutter is a dependency and dependencies shouldn't use dynamic versions.

The solution we've been suggesting more recently is to use git submodules to import flutter as part of the git repo for your app.

@passsy Would you please release the flutterw.bat Version? thank you!

@Hixie it feels like this is a great candidate to either 1) Handle in a 1st party way out of the box built in or 2) Officially endorse passsy's solution; what do you think?

Yeah, we should document the recommended solution. See https://github.com/flutter/website/issues/1845 which is tracking that suggestion. PRs welcome. :-)


Support for mybatis parent 35

Coverage decreased (-0.004%) to 87.322% when pulling 7caac3a3816f67240a6ecb907ab0f17f05a67589 on hazendaz:master into 94f474bf5c9400024e7530a9d38410aad1e9dc4b on mybatis:master.


Current cask of vagrant is incompatible with current cask of Virtualbox - Vagrant Cask needs to be updated to 1.8.5

Current Cask of of vagrant is incompatibel with current cask of VirtualBox
Well, we updated the packages here at the company and and now our Vagrant machines dont run.. Do you think the update of the Vagrant cask can be done quite fast? Otherwise, is there a possibility to downgreade and pin some versions, like Virtualbox 5.0.* until the issue is fixed?
Vagrant 1.8.4
VirtualBox Version 5.1.0 r108711
Thats the message from the vagrant installed with brew cask
The provider 'virtualbox' that was requested to back the machine
'default' is reporting that it isn't usable on this system. The
reason is shown below:

Vagrant has detected that you have a version of VirtualBox installed
that is not supported by this version of Vagrant. Please install one of
the supported versions listed below to use Vagrant:

4.0, 4.1, 4.2, 4.3, 5.0

A Vagrant update may also be available that adds support for the version
you specified. Please check www.vagrantup.com/downloads.html to download
the latest version.

Output of `brew doctor`
ALL OK

Output of `brew cask doctor`
ALL OK

Right at the moment, someone updated the cask and now everything works fine again.


FINERACT-1120-TFA-authentication-issue-fix

Description
Describe the changes made and why they were made. Ignore if these details are present on the associated Jira ticket
Checklist
Please make sure these boxes are checked before submitting your pull request - thanks!

[ ] Commit message starts with the issue number from https://issues.apache.org/jira/projects/FINERACT/. Ex: FINERACT-646 Pockets API.

[ ] Coding conventions at https://cwiki.apache.org/confluence/display/FINERACT/Coding+Conventions have been followed.

[ ] API documentation at fineract-provider/src/main/resources/static/api-docs/apiLive.htm has been updated with details of any API changes.

[ ] Integration tests have been created/updated for verifying the changes made.

[ ] All Integrations tests are passing with the new commits.

[ ] Submission is not a "code dump".  (Large changes can be made "in repository" via a branch.  Ask on the list.)

Our guidelines for code reviews is at https://cwiki.apache.org/confluence/display/FINERACT/Code+Review+Guide

The change itself looks OK to me - but I have no idea if it is enough to fix TFA, or if there are other issues that need to be resolved. Do we have any way of testing this or are there any instructions on how it should work?

This appears to be the back-port of #1297 to 1.4.0 for FINERACT-1136 .. but 1.4.0 has meanwhile sailed without this. @fynmanoj @vidakovic do you want to just close this PR, or do something else with it? -- PS: If were were to still merge this PR to the 1.4.0 now, it would be very confusing... see FINERACT-1154.

@avikganguly01 do you know more about this?


Create RSS Feed for PDS News

OpenPlanetary Slack has a new #planetary-news-feeds channel which could easily link into this RSS Feed. Other feeds could also be easily linked.

Closing as wontfix


Add TileDB IVF PQ

What
Adds IVF PQ. There is still more we can do to tune num_subspaces=dimensions/2, but the goal is to get something in quickly so I can add it to the benchmark runner script.
Testing

The performance of IVF_PQ seems low, have you investigated this?

Yep, agree, it's not very good. I believe it may be because we need to tune num_subspaces better - right now we create dimensions / 2, which is quite a lot. Though with that many I would have expected to have higher recall. For now my plan is:

Fix the known bugs in the index - https://github.com/TileDB-Inc/TileDB-Vector-Search/blob/main/apis/python/test/test_ingestion.py#L815 in particular is a case where I expect 100% recall but we get 98% recall. If this is indeed a bug, fixing it may help these benchmarks as well.
Once that has been tested, test out tuning num_subspaces.


🌟Feature request: support for svelte4

Is your feature request related to a problem? Please describe.
add support for svelte4

I'm curious about the effort that would entail. Will probably wait for the official release (since now it's in preview) to see if they provide any instructions 😄

It's released! The one thing I noticed that is relevant to this project is this: https://svelte.dev/docs/v4-migration-guide#sveltecomponenttyped-is-deprecated


MAINT: refactor sqlite array type handling

[CHANGED] make sure BytesIO instances are closed when
finished.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

14 of 14 (100.0%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.006%) to 91.719%

Totals

Change from base Build<PHONE_NUMBER>:
0.006%

Covered Lines:
29650

Relevant Lines:
32327

💛  - Coveralls

codacy issue relates to Markdown formatting that's used by scriv for producing changelog. It's safely ignored.

The pull request reminds me I should detail the solution in the issue (#1776) linked to by the warning.


:seedling: Bump github/codeql-action from 2.3.3 to 2.3.4" - Revert

Reverts ossf/scorecard#3064
https://github.com/ossf/scorecard/issues/3063

Resolved by #3072


Escape characters are highlighted in character arrays

Describe the bug
When initializing a raw string variable, escape characters (backslash + char) get highlighted by default. This occurs for character arrays (using single quotes), but not for strings (using double quotes).
For example, when defining a Windows path:
temp_path = '\\somedrive\path\to\right\folder\';
In this case, the escape characters \\, \r, \t, and \f characters would be highlighted.
If you do the same but with double quotations, the error does not occur. This is the expected behavior.

Hi @goldrik , thanks for reporting this!
For clarity, can you please attach an image showing this behavior?

Yup, here's the same string, initialized as a character array, then as a string.

I don't think it is a bug that the escape characters are being highlighted in character vectors - they are explicitly being tokenized as constant.character.escape.matlab. It is definitely incorrect, however, that they are being treated differently between char vectors and string arrays.
I am moving this issue over to the MATLAB-Language-grammar repository, because this is an issue with the grammar.


radio does not work as mentioned in the docs

the radio button example given in the README.md does not work reliably and gives this error
Expression has changed after it was checked. Previous value: 'undefined'. Current value:
to make it work reliably , you need to have the name attribute in the group and radio buttons. The name attribute should match.
If this is by design , the documentation is not correct

Similar issue with the alpha.  @rahulbats suggestion removed the errors.


Summation symbol

Currently unable to write summation symbol with Drasil.

Can you give a little more detail and/or an example of where you need it? Is it just a matter of adding Sigma to the language or expressing a sum in an equation/expression?

This issue also applies to my example.  In my case I require it (as well as product notation symbol) in the context of equations/expressions.  See attached image from my SRS:

I require it as well in a similar context (equations in a table), for example:

Summation is implemented, however, I have not solidified the design choices for displaying bounds. That may take a little while.


core dumped in normal code execution or kernel dead in jupyter notebook after calling crop function of raw object..

trying to crop the raw object based on some indices. This is killing the kernel in Jupyter notebook. Running in python 2.7 also generates core dump. What is the reason of this?

how did you install python / jupyter?
show us what
mne.show_info()
tells you

python is already installed on ubuntu. Jupyter was installed via pip.
Since program is terminated, after crop(), no information is obtained.
on running mne.show_info(), i got attribute error: module object has no attribute 'show_info'

sorry i meant
mne.sys_info()
did you do:
sudo pip install jupyter
?

Platform:      Linux-4.4.0-98-generic-x86_64-with-Ubuntu-16.04-xenial
Python:        2.7.12 (default, Nov 19 2016, 06:48:10)  [GCC 5.4.0 20160609]
Executable:    /home/gnl/TF/bin/python
CPU:           x86_64: 16 cores
Memory:        31.3 GB
mne:           0.15.2
numpy:         1.13.1 {lapack=openblas, blas=openblas}
scipy:         0.19.1
matplotlib:    2.0.2
sklearn:       0.19.0
nibabel:       2.2.1
mayavi:        Not found
pycuda:        Not found
skcuda:        Not found
pandas:        Not found

yes i installed jupyter as you have mentioned.

also getting same problem of kernel dead while running get_data() method of raw object.

well that's really dangerous to do
sudo pip install
I would recommend you install ubuntu version of jupyter with apt-get
or use anaconda but do not mix the 2.

pip install was done by first creating virtualenv so I think this might not be the problem.

you have an install problem for sure. it's not an mne problem.
try using anaconda

One more point I want to mention is that when I am trying to do crop directly on raw object it is not showing any error. But when different raw objects are appended in list and then crop and get_data method on elements of list are giving kernel dead error.

can you share a code snippet to replicate?

working case:
raw_eeg = mne.io.read_raw_eeglab(path_to_set_file, preload='True')
raw_eeg.crop()
raw_eeg.get_data()

non working case:
raw_eeg_obj = []
for index in range(13):
    raw_eeg = mne.io.read_raw_eeglab(path_to_set_file[index], preload='True')     # need .set files
    raw_eeg_obj.append(raw_eeg)
for index in range(13):
    raw_eeg_obj[index].crop()            # this is not working
    raw_eeg_obj[index].get_data()      # this is also not working

this is not a self-contained example using sample dataset. to try quickly I
should just need to "copy and paste".
thx

Providing full code to replicate the issue.
However you need to download the dataset, extract it and change the initial path in the code where all extracted folders are placed.
You need to download the files MM05.tar.bz2 and so on from the website http://www.cs.toronto.edu/~complingweb/data/karaOne/karaOne.html
import scipy.io as sio
import mne
import os

full_data = []
trial_indices = []

initial_path = '/home/gnl/Abhi/imagined_UoT/kara_one_classify/dataraw/'
dir1 = os.listdir(initial_path)
second_path = '/spoclab/users/szhao/EEG/data/'
for file_ in dir1:
    path = initial_path+file_+second_path
    dir2 = os.listdir(path)
    if dir2[0] != 'P02':
        # collect desired files for data, indices and labels
        files = os.listdir(path+dir2[0])
        set_file = [s for s in files if '.set' in s][0]
        index_file = [s for s in files if 'epoch_inds' in s][0]
        dir3 = os.listdir(path+dir2[0]+'/kinect_data')
        label_file = [s for s in dir3 if 'p.txt' in s][0]
        # now load the data of each subject
        raw_eeg = mne.io.read_raw_eeglab(path+dir2[0]+'/'+set_file, preload='True')
        full_data.append(raw_eeg)
        indices = sio.loadmat(path+dir2[0]+'/'+index_file)
        trial_indices.append(indices)

trial_data_full = []

def data_by_trial():
    for subject in range(13):
        for trial in range(trial_indices[subject]['thinking_inds'].shape[1]):
            # get indices of the trial
            indices = trial_indices[subject]['thinking_inds'][0][trial][0]/1000.0
            # crop data sequence
            tempData = full_data[subject].copy().crop(start=indices[0],stop=indices[1])
            # append to trial_data_full
            trial_data_full.append(tempData)

data_by_trial()

you don't replicate with eg the sample dataset?

I am unable to replicate on mne.datasets.eegbci dataset.
Can you please check on the karaone dataset provided on the link above?
I am continuously facing the issue on karaone dataset.

can you share with me privately a dropbox folder with the script and the
data next to it.
it will make it trivial for me (and others) to replicate

Sorry, I do not have Dropbox account.
I believe that by following steps mentioned in the above post, one can easily download the dataset, copy-paste the code and replicate the issue.
Thanks

what file should I use? each is 2GB....

Yes I know, each file is pretty heavy. You have to download 13 files starting from MM05.tar.bz2 to MM21.tar.bz2.

ok that won't happen. Please make an effort to make a minimal reproducible
example.

If the files are so big maybe the problem is just memory-related. Does crop work for preload=False (it should)? BTW @abhiramsingh0 it should be preload=True not preload='True', although it shouldn't make a difference in the code (but preload='False' will behave as if it was True so better use booleans).

@mmagnuski , thanks for pointing out the preload point. Also, I am loading the .set file and associated .fdt file is in the same folder. Size of each .fdt is about 600MB and I am loading 13 of these different files. You can see mne.sys_info() output that I have pasted above. Now how do I detect if problem is related to memory or mne as there seems to be enough memory for loading these files.

Try concatenating just two of them. Then 3, then 4, then ... etc. Watch your memory consumption as it goes. See if crash coincides with running out of memory.
MNE provides preload=temp_fname to load data into a memmapped file to get around memory limitations if you can't use preload=False.

@abhiramsingh0 is this issue solved?

I am closing this. @abhiramsingh0 reopen if the problem is still there.


New package: AtomGraphs v0.1.0

Registering package: AtomGraphs
Repository: https://github.com/Chemellia/AtomGraphs.jl
Created by: @rkurchin
Version: v0.1.0
Commit: ae18cde0401d9c5c46881a313eb8ce23b4116498
Reviewed by: @rkurchin
Reference: https://github.com/Chemellia/AtomGraphs.jl/commit/ae18cde0401d9c5c46881a313eb8ce23b4116498#commitcomment-63038746
Description: Graph-building for AtomicGraphNets

👋 General maintainers! Two things here:

ItemGraphs hasn't had any commits in almost two years...
I would argue "Atom" and "Item" are distinct enough concepts that there's a low chance of confusion.

AtomGraphs is a very natural name for this package (which provides the AtomGraph type and some functions for constructing instances), but if these arguments don't convince you, I'll try to think of another name...


RuntimeError: ; Unable to find the type for stream "image".

OS Platform and Distribution
Linux Ubuntu 18.04.6 LTS (Bionic Beaver)
Compiler version
Programming Language and version
Python 3.8.12
Installed using virtualenv? pip? Conda?(if python)
MediaPipe version
0.10.7
Bazel version
XCode and Tulsi versions (if iOS)
Android SDK and NDK versions (if android)
Android AAR (if android)
None
OpenCV version (if running on desktop)
<IP_ADDRESS>
Describe the problem
I am trying to build some applications using mediapipe on a Nvidia Jetson machine (4 CPU cores ARMv8 Processor, NVIDIA Tegra X1). I had installed a mediapipe version through pip, but it was not using the GPUs available on the machine.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
To make use of the available GPUs, I followed the documentation available here:

https://jetson-docs.com/libraries/mediapipe/overview
https://jetson-docs.com/libraries/mediapipe/l4t32.7.1/py3.8.0

However, when I am trying to run the sample program, it throws the following error:
Complete Logs
Traceback (most recent call last):
  File "ar-filters/app.py", line 12, in <module>
    face_mesh_videos = mp_face_mesh.FaceMesh(static_image_mode=False, max_num_faces=1, min_detection_confidence=0.5, min_tracking_confidence=0.3)
  File "/home/epl/.local/lib/python3.8/site-packages/mediapipe/python/solutions/face_mesh.py", line 95, in __init__
    super().__init__(
  File "/home/epl/.local/lib/python3.8/site-packages/mediapipe/python/solution_base.py", line 276, in __init__
    canonical_graph_config_proto = self._initialize_graph_interface(
  File "/home/epl/.local/lib/python3.8/site-packages/mediapipe/python/solution_base.py", line 428, in _initialize_graph_interface
    self._input_stream_type_info = {
  File "/home/epl/.local/lib/python3.8/site-packages/mediapipe/python/solution_base.py", line 429, in <dictcomp>
    get_name(tag_index_name): get_stream_packet_type(tag_index_name)
  File "/home/epl/.local/lib/python3.8/site-packages/mediapipe/python/solution_base.py", line 426, in get_stream_packet_type
    validated_graph.registered_stream_type_name(stream_name))
RuntimeError: ; Unable to find the type for stream "image".  It may be set to AnyType or something else that isn't determinable, or the type may be defined but not registered.

I see similar issues reported earlier as well:

https://github.com/google-ai-edge/mediapipe/issues/2767
https://github.com/google-ai-edge/mediapipe/issues/1651

But, could not conclusively determine the solution for this.
Happy to provide additional details.
Any assistance will be appreciated. Thanks in advance!

Hi @Kalit31,
Based on the information you have shared, there appear to be two issues:

Unsupported Python Version: It seems you are using an unsupported Python version. The supported versions, as mentioned in the documentation, are 3.9 to 3.12.
Legacy Face Mesh: It looks like you might still be using the legacy face mesh, which has been upgraded to the Face Landmarker Task API. Please note that support for the legacy solution has ended(see here), and you can find more details in the documentation.

Please let me know if you need further clarification or assistance!
Thank you!!

Hi @kuaashish,
Thanks for the quick response!
Yes, I was using the legacy face mesh. The below docs seem to be outdated then?

https://jetson-docs.com/libraries/mediapipe/overview
https://jetson-docs.com/libraries/mediapipe/l4t32.7.1/py3.8.0

Let me try with 'Face Landmarker Task API'. Will update you.

Thanks for the detailed response!
Is there any way I can use legacy versions of the face mesh and make use of the GPU support available in the Jetson machine?
The legacy face mesh was working perfectly fine, just that it was not making use of the available GPUs. Considering the limited (or unavailable) support for the new Task API on Jetson, can you suggest some approach to make use of the legacy version?

Hi @Kalit31,
There is a community-based implementation available at this GitHub repository, which supports GPUs and is based on legacy solutions. While it may work with the GPU, for support with the new Task API on Jetson, we recommend submitting a feature request here, and request you to close this issue, Future implementations may be considered based on community demand and our discussion.
Thank you!!


Gradle generated incorrect build/scripts/app and build/scripts/app.bat

The generated "app" script sets CLASSPATH=$APP_HOME/lib/app.jar:...
There is no /lib folder so it throws ClassNotFoundException but there a /libs folder was generated. Adding an "s" fixes the script. Same typo is in the distribution/app.tar and app.zip.
Expected Behavior

Generated build/scripts need to have correct CLASSPATH with libs rather than lib folder.
Current Behavior

Context

Run generated script to run the executable jar.
Steps to Reproduce

Run gradlew init and build and note the typo in the build/scripts/app file: lib versus libs
Your Environment

Build scan URL:

Gradle 7.5
Build time:   2022-07-14 12:48:15 UTC
Revision:     c7db7b958189ad2b0c1472b6fe663e6d654a5103
Kotlin:       1.6.21
Groovy:       3.0.10
Ant:          Apache Ant(TM) version 1.10.11 compiled on July 10 2021
JVM:          11.0.11 (AdoptOpenJDK 11.0.11+9)
OS:           Mac OS X 10.15.7 x86_64

I don't believe this is the case.
Here's the directory structure for the application project generated by gradle init with 7.5:

app/build/install/
└── app
├── bin
│   ├── app
│   └── app.bat
└── lib
├── app.jar
├── checker-qual-3.12.0.jar
├── error_prone_annotations-2.7.1.jar
├── failureaccess-1.0.1.jar
├── guava-31.0.1-jre.jar
├── j2objc-annotations-1.3.jar
├── jsr305-3.0.2.jar
└── listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar

lib is the correct path.

@big-guy
Can I ask you a question about how do you generate above file three output/

@CoXier it's just tree

@CoXier it's just tree

Thanks a lot.


implement deployment of new mifos web-app instead of the community app.

Right now, mojafos deploys a Mifos X with the old community app. We need to make it so that it deploys the latest stable release of the web app.

@tdaly61 I think this is actually an opportunity for us to actually get rid of helm completely. We could generate k8s deployment manifests from docker-compose files.
What do you think?


Electron 0.36.0 fullscreen shows titlebar on OS X Yosemite

The latest version of Electron (0.36.0) still has an issue on Yosemite with fullscreen.
Basically when in fullscreen mode the window shows a textured title at the top:

This happens regardless if the window frame option is set to true or false (the only difference being you don't see the traffic lights on hover when set to false). However setting the transparent option to true removes the titlebar completely and solves it.
So in short it needs to hide the titlebar when transparency is set to false and in fullscreen and then ONLY show the title when hovering and the frame is set to true.

Is this the same issue with #1057? I think the problem is no one else can reproduce this.
Also can you try setting the type attribute of BrowserWindow to textured?

I get similar on Linux if resizable is set to false (as it is in #1057 – which makes sense to me).
Wonder if it's the same root issue...
(and my apologies for the noise if it isn't :P)

@zcbenz Tried it with type set to textured, but still get the issue. This is the code to reproduce:
app.on('ready', function() {
  mainWindow = new BrowserWindow({
    'width': 800,
    'height': 600,
    'autoHideMenuBar': true,
    'useContentSize': true,
    'titleBarStyle': 'hidden-inset',
    'frame': false,
    'fullscreen': true,
    'type': 'textured'
  });
  mainWindow.loadURL('file://' + __dirname + '/index.html');
  mainWindow.focus();
});

Okay just found out some more information!
Basically if you set the titleBarStyle (even though the frame is set to false) it will show the titlebar all the time like the screenshots above. This is a bug as I think once frame is set to false, anything related to the frame such as traffic light buttons should be ignored.
However there is still a slight issue (even when removing the titleBarStyle) as when you hover to the top to access the menu bar you see this:
This happens when frame is set to true and false. It should only appear if the frame is set to true when hovering. Setting the frame to be transparent however solves this as well, but these are all workarounds and not expected behaviour IMO.

Didn't know you were using the titleBarStyle option, this is actually a duplicate of #2810.


Partial evaluation phase

Since CPS and closure conversion is now done, we might as well have this. :shrug:

Needs #108 first.


Support .NET Core / VS2017 RTM Tooling

Microsoft has released VS2017 with its associated msbuild-based dotnet tool implementation.
Fixie 2.0.0-alpha-0001 was a worthwhile experiment for the now-deprecated project.json / xproj tooling, but maintaining that branch is no longer our goal. All of the ideas from that branch that are relevant to the VS2017 tooling have already been included in the develop branch, our main line of development.
This is a parent issue for the effort to work with the VS2017 tooling.  If an issue is marked Up for Grabs, please claim it by joining the discussion on that issue.  If an issue is marked In Progress, it has been claimed, but please join in the discussion to help out!

[x] Remove <DebugType>full</DebugType> from Fixie.Tests.csproj (No longer applicable)
[x] Integrate with dotnet test #167
[x] Build packages without a nupsec file #168 (No longer applicable)
[x] Implement dotnet fixie to replace the console runners
[x] Target netstandard

How is this progressing? Any timeline, yet?

Prerelease version 2.0.0-alpha-0002 is ready for early adopters to experiment with.
See https://github.com/fixie/fixie.integration for a working example of a solution which depends on the new tooling.
VS2017 Update 3 and above are officially supported, and only for the modern compact csproj format.  It would be useful to get information on whether there are issues on VS2015 and the older verbose csproj format in case it's easy to improve that scenario, but we may have to use 2017 as a cutoff for 2.x, since so much of the surrounding ecosystem has changed.
Known issues:

TestDriven.NET integration only works for test assemblies targeting the full .NET Framework. .NET Core support depends on TestDriven.NET itself adopting .NET Core.
Test Explorer will have no source code line number information available for projects targeting .NET Core. You will not be able to double-click an item in Test Explorer to navigate to the code for that test, and you will not be able to get right-click context menu items on test methods to run those tests. You'll have to view and run tests only through Test Explorer.

To install the Fixie library, add the following package to your test assembly:
dotnet add package Fixie --version 2.0.0-alpha-0002
To install the dotnet fixie command, add the following package to your test assembly:
dotnet add package Fixie.Console --version 2.0.0-alpha-0002
From within a test project folder,
dotnet fixie --configuration $configuration --no-build

Closing this issue. We'll track further work on .NET Core support in separate issues.

What issue is tracking .NET Core support? I couldn't find it? I saw there are alpha of 2.0 in Nuget, but I can't find the issue tracking .NET Standard support?

This closed ticket covered the work that got us to the current alpha.
#166 is the main up-for-grabs obstacle that needs to be addressed for a final 2.x release.
I have a handful of nice-to-have things in motion such as phasing out the last remaining usages of AppDomain, but there isn't an Issue tracking that.

Awesome! I sent a PR to try and fix #166. Let's hope that after that you can release a beta? I want to use it! :)


Fix the position of the imu in the waist

It fixes #260

Here is the resulting PR:

https://github.com/icub-tech-iit/ergocub-software/pull/268

Obviously the V1/SN000 is missing, the interesting thing is that:

#266

created a regression, maybe this:

https://github.com/icub-tech-iit/creo2urdf/issues/109

is mandatory to make it work

After merging:
-https://github.com/icub-tech-iit/creo2urdf/issues/109
The changes in the urdf are fine:

https://github.com/icub-tech-iit/ergocub-software/pull/270

Tested here:

https://github.com/icub-tech-iit/ergocub-software/issues/260#issuecomment-2402412497


improve portability

Move all environment variables from sensors to wf-templates.yaml that define environment and update readme.

Done.


Review launched effect in UI for dispatcher

ArtworkColorBackground uses LaunchedEffect and does what could be CPU intensive work.  We should consider moving this and other examples to another dispatcher like Dispatchers.Default.
No point doing this work on the main thread.

Closing as stale


Creating pipes for GWakeup: Too many open files Trace/breakpoint trap (core dumped)

Describe the bug
When i try to open multiple streams i run into too many open files error
To Reproduce
Please describe the steps to reproduce the behavior:

Run 30 + streams simultaneously on TensorRT runtime with rtsp as input
The GPU usage is around ~35% during period

Expected behavior
A clear and concise description of what you expected to happen.
Screenshots
If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):

Ubuntu 22
Tesla T4

Hi @udhay24 ,
Does it work if you increase the default limit?
You can do something like:
ulimit -n 2048

By default, it is in most cases 1024

Hi @miguelaeh ,
i tried increasing the ulimit and it fixed the issue.
Currently I am using a 15Gb ram and 4 VCPU spec, as i approach the 30+ streams. the RAM reaches the limit and it crashes the application.  Do you have a suggested configuration or options i can use to make it efficient
Thanks

Yeah, that fixed the RAM overflow issue
Thanks a lot


xtensa/esp32s3: Enable booting from MCUboot bootloader

Summary
This PR intends to add support for booting an ESP32-S3 NuttX application firmware image from MCUboot.
Impact
Only for boards based on ESP32-S3 chip.
Testing

pip install imgtool, this is MCUboot tool for signing the firmware images.
./tools/configure.sh -E esp32s3-devkit:mcuboot_nsh
make bootloader
make download ESPTOOL_PORT=/dev/ttyUSB0 ESPTOOL_BINDIR=.

Hi @almir-okato please check about the CI fail for xtensa


empty string should be a valid dropdown selection

When I create a drop down with the empty string as an option I get an error when I attempt to select that option.  I think I've also seen the error at the time the dropdown is created also:
import ipywidgets
dd = ipywidgets.Dropdown(description="",
            options=["", "solid", "dotted", "dashed"], value="solid")
dd

Thanks for reporting this. It seems to work well now in master:


can you put the render result after the code in the example?

if there have a render result after example in readme.md would be better.

Hi @cuivip
Thank you for your comment. I plan to create a more in depth documentation similar to the vue-i18n docs in the upcoming week and will included several examples.


UVI Request

--- UVI JSON ---
{
  "vendor_name": "Linux",
  "product_name": "Kernel",
  "product_version": "versions from v5.6 to before v5.10.46",
  "vulnerability_type": "unspecified",
  "affected_component": "unspecified",
  "attack_vector": "unspecified",
  "impact": "unspecified",
  "credit": "",
  "references": [
    "https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=71921690f9745fef60a2bad425f30adf8cdc9da0",
    "https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=fb3a948143688e14e2cfd2a2812877923d0e5e92"
  ],
  "extended_references": [
    {
      "type": "commit",
      "value": "71921690f9745fef60a2bad425f30adf8cdc9da0",
      "note": "introduced"
    },
    {
      "type": "commit",
      "value": "fb3a948143688e14e2cfd2a2812877923d0e5e92",
      "note": "fixed"
    }
  ],
  "reporter": "joshbressers",
  "reporter_id": 1692786,
  "notes": "",
  "description": "ethtool: strset: fix message length calculation\n\nThis is an automated ID intended to aid in discovery of potential security vulnerabilities. The actual impact and attack plausibility have not yet been proven.\nThis ID is fixed in Linux Kernel version v5.10.46 by commit fb3a948143688e14e2cfd2a2812877923d0e5e92, it was introduced in version v5.6 by commit 71921690f9745fef60a2bad425f30adf8cdc9da0. For more details please see the references link."
}
--- UVI JSON ---

This issue has been assigned UVI-2021-1000984