Keys iterator of the store::UnorderedMap has a huge performance drop after removing elements

Problem

Create store::UnorderedMap
Insert 1000 elements.
Remove 900 elements.
Now, iteration through 100 keys takes O(1000) time instead of O(100) expected.

Trying to take just a constant number of keys gives an inconsistent performance with increasing gas consumption 3TGas -> 10 TGas -> 20 TGas ->....., while removing bunches of elements from the map.
Cause
FreeList is now used for the keys in the store::UnorderedMap, there is an iterator that goes through the "holes" in the vector of keys which never shrinks when elements are removed.
Example
Here, iterating through 100 keys after removals gives 0.5 TGas consumption instead of 0.04 TGas if 100 elements were just inserted.
    pub fn test(&mut self) {
        let mut map = UnorderedMap::new(b'a');
        for i in 0..1000 {
            map.insert(i, i);
        }

        // Keep only 100 elements.
        for i in 0..900 {
            map.remove(&i);
        }

        let gas = env::used_gas();
        let _ = map.keys().collect::<Vec<_>>();
        let gas = (env::used_gas() - gas).0 as f32 / Gas::ONE_TERA.0 as f32;

        env::log_str(&format!("used gas: {:?}", gas));  // 0.5 TGas
    }

0.5 TGas - 100 keys, after removals.
0.04 TGas - 100 keys, no removals, just inserts.
x10 performance drop.
It's worse when there are thousands of elements.

Another problem is that skip is not cheap (FreeList::Iterator::skip) because it's not lazy, it reads every value while moving to the nth position to figure out whether a slot is occupied or not. And it's another issue. E.g. even in a case with no removals, skipping over couple of thousand element may not be possible because gas will finish earlier.

I found an enhancement ticket outlining the same issue: https://github.com/near/near-sdk-rs/issues/990.
defrag method was introduced to fight the fragmentation of the FreeList via removing the "holes" in it: https://github.com/near/near-sdk-rs/pull/1023/files#diff-793a8b595e5ccce11a932ce7a41e0ea3a313790edc40bc4ff0ad5943a112a673R230
However, the problem lies in the fact that running through the FreeList slots is not cheap itself (see another ticket I created https://github.com/near/near-sdk-rs/issues/1135), and iterating through the UnorderedMap/FreeList is limited to 2-3000 first elements: skip/defrag/next/everything is limited to 300 TGas. Defragmentation can defragment just the head of the data structure.
UnorderedMap is unreachable higher than ~2000 elements via its Iterator.

@fadeevab From what I can tell, the old near_sdk::collections::UnorderedMap does not suffer from this issue since it is based on Vector instead.
@austinabell Could you help to shed some light on the motivation for using FreeList over Vector for new store collections? Did you have any solution for dealing with iteration/slicing efficiently?

It's a tradeoff, yes sth like skip iteration will be worse, but for removals for example, the old implementation does a lot of reads and writes to have to update all new indices for any element removed. (Note you can minimize the amount of reads/writes of this old implementation slightly, but all comes at the cost of breaking changes, which I wouldn't recommend)
There's no perfect solution, and decisions were made to optimize the most commonly used patterns. Possible those are incorrect or prohibitive in real world cases.
Also unclear why skip iteration is an important pattern for an unordered map, the ordering isn't stable. You just want to load values out of the maps in chunks or something?

@frol Indeed, collections::UnorderedMap's iterator works as expected, I've already verified it.
@austinabell

Also unclear why skip iteration is an important pattern for an unordered map, the ordering isn't stable. You just want to load values out of the maps in chunks or something?

One scenario is storing and iterating many data entries in chunks. Say, having 10k entries it's possible to iterate them by 1-2k per 1 call with collections::UnorderedMap and it's impossible with store::UnorderedMap which can never skip a few thousand entries because it's not lazy.

Another scenario: getting a random element in UnorderedMap - it's impossible with a big map, because nth(5000) element is not reachable because nth is not lazy and consumes too much gas jumping to the nth element.

Anyways, UnorderedMap provides an iterator that should meet Iterator API expectations. I can use LookupMap when I don't need iteration.

Also unclear why skip iteration is an important pattern for an unordered map, the ordering isn't stable. You just want to load values out of the maps in chunks or something?

Well, the order is stable if one makes several requests at the same block height and the most common scenario is to fetch the data in pages, and it seems that the current implementation degrades the performance significantly turning it into a storage-inefficient LookupMap.

@fadeevab What would be the ideal solution? I don't have the capacity to dive into it, do you?

The challenges I see:

Performance issue itself
Migration plan for the existing contracts (or maybe give up and mark these collections as #[deprecated] with a clear message that iteration over these collections can be slow)

The challenges I see:

Performance issue itself
Migration plan for the existing contracts (or maybe give up and mark these collections as #[deprecated] with a clear message that iteration over these collections can be slow)

This seems like not too bad of a solution:

Add a defrag method for migrations.
Add a remove_defrag method for everything going forward. Or be opinionated and pack this functionality into remove.

What do you think @frol ?

@ruseinov The issue is that self.unordered_map.iter().skip(100000).take(3) attempts to query at least 100003 keys even when FreeList is defragmented since there is no efficient .nth method implemented and so it just calls next() 100000 times for no good reason.
Quick ugly solution - implement .nth and if FreeList is defragmented, make it efficient, otherwise fallback to slow version. But I fear that there might be situation when defrag won't be able to complete within the gas limit and then what? And still, it sounds like a major footgun and you need to know how to heal it...

@ruseinov The issue is that self.unordered_map.iter().skip(100000).take(3) attempts to query at least 100003 keys even when FreeList is defragmented since there is no efficient .nth method implemented and so it just calls next() 100000 times for no good reason.
Quick ugly solution - implement .nth and if FreeList is defragmented, make it efficient, otherwise fallback to slow version. But I fear that there might be situation when defrag won't be able to complete within the gas limit and then what? And still, it sounds like a major footgun and you need to know how to heal it...

Yeah, I hear you. We could implement an alternative solution to defrag, but that'd require a migration that spans several blocks, which is a pain for several reasons. Though that in combination with the new logic would eventually lead to better performance for old contracts and good performance on fresh ones out of the box.
We could also make sure that the collection is using the old logic unless it's been defragmented, then a migration will be a soft requirement.
Another thing I thought about is that we could allow for partial defrag to limit gas used, but I need to see the current layout and the code to figure out if it's even possible. It does not seem like the migration mechanism supports a multi-block migration, though I'm not very familiar with it yet. Any solutions to keep calling defrag till it's finished, reserving some GAS on each block?
What are the alternatives.

When you are talking about deprecating collections - which ones do you mean? store::UnorderedMap and FreeList? In which case we'll need to replace them with something new and those names would be forever lost. But that can be fixed by versioning or changing namespace, e.g. store::v2::UnorderedMap.
.. not sure what else could be done here.

Seems like the whole point of the redesigned store::UnorderedMap (with FreeList) was optimizing the removal operation making removal super fast.
@frol I imagine someone may need such a map for small collections: quick removals, periodic defrags in a single block, etc. From my perspective, the ideal solution would be renaming the store::UnorderedMap into DefragMap/SmallMap/etc. just for repurposing (with tons of disclaimers). (@ruseinov it basically goes with what you propose as an alternative).
On a side note, I found a nice improvement to the collections::UnordredMap which could be added: a pop operation - despite it sounds weird but if you need to clean the map randomly, it's faster by removing from that tail, ~O(1) :) (because it doesn't need rearranging the key indexes map).

On a side note, I found a nice improvement to the collections::UnordredMap which could be added: a pop operation - despite it sounds weird but if you need to clean the map randomly, it's faster by removing from that tail, ~O(1) :) (because it doesn't need rearranging the key indexes map).

I'm not sure what the purpose of this would be, if you'd want to clean the map randomly - might as well clear it, right? Because it means that data it stores does not matter, unless I'm missing the point here.

Seems like the whole point of the redesigned store::UnorderedMap (with FreeList) was optimizing the removal operation making removal super fast.

We might need to revisit this to see exactly how much faster it ends up being and what purpose it serves. I reckon if we're talking small collections - there are other collections that could be used. If the collection is big and it becomes fragmented due to removes - then what's the point of this optimisation?
Let's imagine some scenarios:

There is a big collection that has it's elements removed on occasion and gets fragmented. Sounds like a waste. Could be periodically defragged, but that's extra magic that should be hidden somewhere.
There is a big collection that pretty much never gets stuff removed from it. Why not disable the method and have another one that marks records as expired or something similar.
There is a small collection, we don't need to use a scalable UnorderedMap for this scenario at all.

Correct me if I'm wrong about the above.
I'd like to see the use-cases that prove we need such optimisations in the first place. A trade-off that leaves the collection infinitely growing does not sound like a sustainable one.

@frol I imagine someone may need such a map for small collections: quick removals, periodic defrags in a single block, etc. From my perspective, the ideal solution would be renaming the store::UnorderedMap into DefragMap/SmallMap/etc. just for repurposing (with tons of disclaimers). (@ruseinov it basically goes with what you propose as an alternative).

Could be done, but then that means that for older contracts they'd need to re-integrate and rename + still do a migration of some sort. Plus if we want to use the same name/namespace for the new collection - some of that stuff could get lost in translation and things won't get migrated at all, potentially causing the data to be either corrupt or fragmented.
There's another truth, all the contracts that currently use UnorderedMap could be upgraded to a new one that would start defragmenting itself upon removal, so they would stop growing unnecessarily. But we still need to provide the migration option.

I'm not sure what the purpose of this would be, if you'd want to clean the map randomly - might as well clear it, right? Because it means that data it stores does not matter, unless I'm missing the point here.

@ruseinov  You miss the point that it can't clear more than ~2-3k elements per block. The same problem as with defrag which can't defrag the entire collection with 10k elements. clean will desperately fail exceeding the gas limit.

@ruseinov You miss the point that it can't clear more than ~2-3k elements per block. The same problem as with defrag which can't defrag the entire collection with 10k elements. clean will desperately fail exceeding the gas limit.

I have mentioned about defrag that is can be limited to max consumed gas and do partial defrag.
Is it not possible to just drop the whole data structure without actually having to delete each element? In theory that should be cheaper, depending on how those are stored.

@ruseinov Then we miss the following real use case: I want to select 1k random elements, process them and drop. clear can be optimized the way you mentioned in any case.
Let's not discuss popproposal too much here, because it doesn't solve this particular issue/ticket :) I rather have to create another feature requests.

I've been thinking about this more, I guess we should go with defrag the element on remove approach. Here's why:

If a collection is rarely removed from, then why should we care much about performance?
If a collection is removed from often - then it makes even less sense, because it will consume more and more unjustified storage.

I still have to investigate performance implications of that though. And there's still a question of defrag migrations, I'd like to hear from somebody who has migration experience to see if there is a way to do a multi-block/partial migration trick.

@ruseinov The fastest "defrag on remove" approach is "swap remove" which is already implemented in collections::UnorderedMap for removals.
I believe that either someone really needed often fast removals (though you think it makes no sense) and we have to keep the current structure under another name (as I mentioned here https://github.com/near/near-sdk-rs/issues/1134#issuecomment-1930873993), or it was unfortunate optimisation attempt which led to more optimizations like defrag which led to this ticket and it means the map should be deprecated (as @frol mentioned here https://github.com/near/near-sdk-rs/issues/1134#issuecomment-1929649535).

@ruseinov The fastest "defrag on remove" approach is "swap remove" which is already implemented in collections::UnorderedMap for removals.
I believe that either someone really needed often fast removals (though you think it makes no sense) and we have to keep the current structure under another name (as I mentioned here #1134 (comment)), or it was unfortunate optimisation attempt which led to more optimizations like defrag which led to this ticket and it means the map should be deprecated (as @frol mentioned here #1134 (comment)).

Indeed, I didn't realize that defrag has already been implemented. Looking at all of this now I'd say it's probably easier to deprecate this altogether (store::{UnorderedSet, UnorderedMap}) or indeed rename it. At the very least these collections have to have a different name, otherwise it's confusing, especially considering implementation and performance differences.

If we opt for a rename - it's probably useful to mention trade-offs directly in module documentation. It is mentioned in the comment to remove method, but that seems easy to overlook.

I'll raise a draft PR with a rename an additional comments.

@ruseinov The fastest "defrag on remove" approach is "swap remove" which is already implemented in collections::UnorderedMap for removals.
I believe that either someone really needed often fast removals (though you think it makes no sense) and we have to keep the current structure under another name (as I mentioned here #1134 (comment)), or it was unfortunate optimisation attempt which led to more optimizations like defrag which led to this ticket and it means the map should be deprecated (as @frol mentioned here #1134 (comment)).

Indeed, I didn't realize that defrag has already been implemented. Looking at all of this now I'd say it's probably easier to deprecate this altogether (store::{UnorderedSet, UnorderedMap}) or indeed rename it. At the very least these collections have to have a different name, otherwise it's confusing, especially considering implementation and performance differences.
If we opt for a rename - it's probably useful to mention trade-offs directly in module documentation. It is mentioned in the comment to remove method, but that seems easy to overlook.

Correction: this only needs to take care of store::UnorderedMap. UnorderedSet had a wrong example with a reference to UnorderedMap that is now fixed.

@ruseinov @fadeevab I feel that store::Unordered* collections do not match the use-case they were designed for:
https://github.com/near/near-sdk-rs/blob/e78e06adabb532f6ca29aab58cf22beba48be01f/near-sdk/src/store/unordered_map/mod.rs#L19-L21
If you cannot iterate after a certain size, it is pointless to use the collection, and LookupMap is more storage efficient if you don't need iteration.
So I see it in the following way:

Old contracts that use store::Unordered* collections are unlucky and their devs will have to plan their migration manually, if necessary
New contracts will benefit from having a correct store::Unordered* collections

We should provide Unordered* collections that provide a reliable iteration feature. We can avoid breaking changes by implementing versioning for the collection header, so a custom borsh deserializer will try to deserialize the new version first and fallback to the old version (and that will define if FreeList or the new implementation should be used). YET, I don't think the effort is worth it. Yet, I don't see a chance to reclaim store::Unordered* name, so even if we rename it in 5.0 release, I won't be comfortable introducing a new implementation with the same name store::UnorderedMap since it is too easy to miss the release or use some intermediate crate (such as near-contract-standards) that could use store::UnorderedMap and you would suddenly hit the state inconsistency after SDK upgrade.
That said, I suggest we just mark the store::Unordered* collections with #[deprecated] and clear message what is the issue there, and implement properly working Unordered collections with new names, e.g. store_v2::UnorderedMap or store::UnorderedMapV2 (other suggestions for naming are welcome!)

That said, I suggest we just mark the store::Unordered* collections with #[deprecated] and clear message what is the issue there, and implement properly working Unordered collections with new names, e.g. store_v2::UnorderedMap or store::UnorderedMapV2 (other suggestions for naming are welcome!)

Right, I would probably go for versioning the module name. It's really a pity that we can't type alias our way through it.

@frol

That said, I suggest we just mark the store::Unordered* collections with #[deprecated] and clear message what is the issue there, and implement properly working Unordered collections with new names, e.g. store_v2::UnorderedMap or store::UnorderedMapV2 (other suggestions for naming are welcome!)

I agree, and it's a pity that it happened.
By the way, UnorderedSet has also been affected. And TreeMap seems not affected, although it's experimental (FreeList seems used there as a "LookupMap with len()").
@ruseinov

I would probably go for versioning the module name.

Sad to lose the good package name, I would consider renaming the types, e.g. into IterableMap and IterableSet, I'm just not in favor of "v2" pattern in the symbol names. Overall, I'm 50/50, whatever.

@frol

That said, I suggest we just mark the store::Unordered* collections with #[deprecated] and clear message what is the issue there, and implement properly working Unordered collections with new names, e.g. store_v2::UnorderedMap or store::UnorderedMapV2 (other suggestions for naming are welcome!)

I agree, and it's a pity that it happened.
By the way, UnorderedSet has also been affected. And TreeMap seems not affected, although it's experimental (FreeList seems used there as a "LookupMap with len()").
@ruseinov

I would probably go for versioning the module name.

Sad to lose the good package name, I would consider renaming the types, e.g. into IterableMap and IterableSet, I'm just not in favor of "v2" pattern in the symbol names. Overall, I'm 50/50, whatever.

It's a matter of taste, I'd prefer a concise name for the struct and a version in module path (e.g. store::v2::UnorderedMap) than an ambiguous name.

@ruseinov store::v2:: looks fine.

@fadeevab There are definitely pros and cons to various naming:

versioned modules (store_v2:: or store::v2::) has a downside that after you import it, it is harder to spot if your code uses old or v2 version of the structs, also, it is not clear which module is actually newer store or store::v2 (as v2 might have been an old version and store is the re-export of the most recent version)
versioned structs (UnorderedMapV2) is just ugly, and while it is more explicit, it is still not clear if it is an old one v2 or it is actually the latest one
new name (IterableMap) adds even more naming to the plate in addition to collections::UnorderedMap, store::UnorderedMap which might be confusing, yet, I actually like the explicitness of the naming. Technically, UnorderedMap works according to its name, but looking at it now, I think that might have actually contributed to the fact that it is not really good for iterations

I like IterableMap the best out of all the above options.

I like IterableMap the best out of all the above options

@frol Cool, I'm keen to vote for this naming because it was my idea and because I made a room for myself to change my mind 😄

I'm just not in favor of "v2" pattern in the symbol names. Overall, I'm 50/50, whatever.

There is an alternative, we could move the old UnorderedMap into a v1 path and make it a breaking change.
The issue I see with IterableMap is that it tells us pretty much nothing. Most of the map implementations are iterable, even the UnorderedMap that has performance issues. It's kinda generic.

I'm assuming that we want to take this further and implement an actual map that will be optimised for iteration. Should a separate issue be created for that?
It would be nice to flesh out how exactly this should work though. FreeList does not seem like a good option since we've just deprecated both collections backed by it.

The issue I see with IterableMap is that it tells us pretty much nothing.

@ruseinov well, it tells us what we literally overlooked last time: iterableness. And it's a map.

The issue I see with IterableMap is that it tells us pretty much nothing.

@ruseinov well, it tells us what we literally overlooked last time: iterableness. And it's a map.

Nope, all that it tells us is that the map is iterable, as in could be iterated upon.
Frangible* tells us that a collection gets fragmented, that's informative. Why? Because all the others aren't.
When you call one collection an IterableMap and the other one is called UnorderedMap, and both can be iterated upon - it's not really informative.

We already have a collections::UnorderedMap and it would be nice to understand what would be the difference between that and a potential new collection, if any.

The whole idea of store over collections was the attempt to avoid a footgun:
struct {
  accounts: UnorderedMap<AccountId, UnorderedMap<TokenId, Balance>>
}

fn update(&mut self) {
    let account_tokens = self.accounts.get("frol.near").unwrap();
    account_tokens.insert("tkn", 10);
    // WARNING: Unless you do the following line, you get your state in a broken state since the key-value in account_tokens is updated, but the length is not updated
    self.accounts.insert("frol.near", account_tokens);
}

store API resolves this issue by implementing commit on drop.

When you call one collection an IterableMap and the other one is called UnorderedMap, and both can be iterated upon - it's not really informative.

I would deprecate UnorderedMaps after we extensively test new collections.
I would compare IterableMap not with UnorderedMap, but with LookupMap, which is not iterable, so when you contrast LookupMap with IterableMap it makes sense, at least to me.

The whole idea of store over collections was the attempt to avoid the footgun:

I'm going to throw some questions here as I'm trying to understand the current situation.
Any reason those collections (collections) have not been deprecated or modified? I fail to understand what's the point of having both store and collections, there might be something I don't know though.
How should the users know to avoid this footgun if both collections store their data on a trie and don't explicitly mention these differences?
I'm happy to take a stab at IterableMap, once we get rid of Unordered* and indeed compare it to LookupMap it will start making sense.

Any reason those collections (collections) have not been deprecated or modified?

It is just a lack of testing for store types. The goal was to deprecate old collections once store is tested. It would be awesome if you can take a lead here!

Also, methods of store:: types are compatible with std HashMap methods, while collections:: maps are not, such compatibility should not be lost...

@ruseinov yes, looks good...

@ruseinov Looks good to me

Yeah, I encountered similar problem 1.5 year ago when I started to use the "new" near_sdk::store in place of the "old" near_sdk::collections. Moreover, the store version tries to cache the data, and it's way more complicated.
I use and recommend near_sdk::collections everywhere.

I really thing we should take an advantage of the fact that we are using a tree for store and enable prefix scan rather adding limitations and making our life harder. I proposed this in: https://github.com/near/NEPs/issues/515
In fact every DB is using a kind of a tree, so even if we will do commitment with a different structure, the DB will allow us to do prefix scan. TBH, all good DBs provide prefix scan feature.

I really thing we should take an advantage of the fact that we are using a tree for store and enable prefix scan rather adding limitations and making our life harder. I proposed this in: near/NEPs#515
In fact every DB is using a kind of a tree, so even if we will do commitment with a different structure, the DB will allow us to do prefix scan. TBH, all good DBs provide prefix scan feature.

It does not seem relevant to the current task, however a separate issue can be created.
@fadeevab @frol I've been looking into "migrating" collections::UnorderedMap to allow for flush functionality. It seems reasonable to use store::Vector for this. Or rather migrating store::UnorderedMap to a new key storage.

@fadeevab Please, review and play with #1164


Unvendor napari/utils/colormaps/vendored ?

🧰 Task
It seem to have been added back in #89 ! From a mix of skimage/matplotlib ?
It looks like we now depends on skimage, so maybe we don't need to vendor some of the code in colormap ?

Needs #5181 to be merged first I think.


Issue Booting SpotX On Fresh Install

🧐 Have you read the FAQ and viewed similar issues?

[X] I have read the FAQ
[X] I have checked older issues, open and closed
[x] I did a clean install of SpotX as described in the FAQ

Сountry of your account
USA
What is your Spotify Plan?
Free
Do you have Spicetify installed?
No
ℹ Computer information
- Spotify: <IP_ADDRESS>1
- Windows: Windows-10
- PowerShell: 5.1.19041.4522

- Processor: Core i5-12600K
- Motherboard: MSI-B760 Pro DDR4
- Graphics: RTX-4070

📝 Description
When Doing A Fresh Install Of Spot X It Brings Me To The Standard Login Screen, But After Just Brings me To A Black Screen.
Same Issue As (https://github.com/SpotX-Official/SpotX/issues/367) But I Couldent Get It To Work At All
I Tried Running The Powershell Cmd And .Bat File For Old Theme But Both Have This Issue
Stock Spotify Boots Fine After Login Its Just With SpotX Applied. Sorry To Be A Bother, Thanks For Any Help <3
📸 Screenshots

Thanks for reporting this, it looks like there are some problems with the latest patcher update

Hey Thanks For The Quick Reply :D
Sorry If me Asking This Is Annoying But If The Issue Is With A Recent Update Are There Legacy Patches That Could Be Used Temporarally?

try again

Oml That Fixed It :D
Thanks So Much Love Your Work <3


No screen after restart

Love the project thank you very much - I've built a couple of these for workshop and home and work very well.. Some streams can crash the system at times but hey - its all good.
Did have a problem on certain ESP Dev boards - After a .restart() (called after a crash or from the config menu) the TFT screen would only display the startup and not the main details screen. Traced the problem to the timers not restating after an esp.restart - This is likely an issue with the particular clone boards im using dollartek ones are fine.
After a software restart the timers wouldnt restart clean so the refresh screen was never called (or checks for encoder etc.)
Resolved by adding:
timerDetachInterrupt(timer);
timerEnd(timer);
before the .restart in code and resolved issue - May help someone else.
Also a small change i made was within the streamtitle update - A few stations (like capital Radio) send a streamtitle update 2 or 3 times a second so caused the display to flicker on the title and track. Added a simple compare to previous stored to resolve this. so no update if the same.
Has anyone had any success with the BBC M3u8 streams?
Many thanks again
John

Thanks, I will correct it in the next release.


Azure tests

Coverage decreased (-0.4%) to 75.005% when pulling c1747ef572bc5218d466fc6cfdc3d156516f236f on yitam:azureTests into d238411a805f130c45dc4100c39a9f3a42b11515 on Microsoft:dev.

Coverage remained the same at 75.365% when pulling c1747ef572bc5218d466fc6cfdc3d156516f236f on yitam:azureTests into d238411a805f130c45dc4100c39a9f3a42b11515 on Microsoft:dev.

Coverage remained the same at 75.365% when pulling 465a72352f81e75fa71aeda5b1655a3274ee995e on yitam:azureTests into d238411a805f130c45dc4100c39a9f3a42b11515 on Microsoft:dev.

Coverage remained the same at 75.365% when pulling e93a40561a0bb35a96d01a1d48e92ce17d5ea1a1 on yitam:azureTests into d238411a805f130c45dc4100c39a9f3a42b11515 on Microsoft:dev.

Coverage decreased (-23.09%) to 52.271% when pulling dcb232e7b9f7c5abe6dce67601082d1e97bf053a on yitam:azureTests into d238411a805f130c45dc4100c39a9f3a42b11515 on Microsoft:dev.

Coverage decreased (-23.09%) to 52.271% when pulling 00aa8ad7c80a8ca1e8235929c0d5e1b608a56372 on yitam:azureTests into d238411a805f130c45dc4100c39a9f3a42b11515 on Microsoft:dev.

Coverage remained the same at 75.365% when pulling f03b62e6b2ef2931e0997bf257b1cc1c56755cdd on yitam:azureTests into d238411a805f130c45dc4100c39a9f3a42b11515 on Microsoft:dev.


Add 'conan' rosdep key to be resolved using pip

This enables using conan (pypi package) package manager to resolve dependencies (e.g. using conan-cmake) which may not be available via the usual rosdep sources, while respecting the usual workflow to build packages from source.
I reckon this may not be desired for packages to be released as binaries using the buildfarm, but it would be very helpful for packages which are never intended to go through the binary release process.

Apologies for missing that one. I believe this now conforms to the rules.
Isn't it a bit confusing to prepend the key with python3- even though the same dependency is available for Python2 as well? This should even resolve to the Python2 version in systems where ROS_PYTHON_VERSION is set to 2, shouldn't it?

Thanks all!


[BUG] API Error

*** roPresence v1.0.4 Launched: Fri Dec 13 2019 17:40:35 GMT+1100 (Australian Eastern Daylight Time) ***

roPresence Express kill server online.

Non-terminal slave process launched.
RPC: Attempting to login through IPC.
RPC: Logged in as kyse<PHONE_NUMBER>99801088).
roPresence API Error - roPresence ran into an error and had to stop. This error is mainly due to a remote API problem.
Please restart the presence.
roPresence: Stopping. Exiting cmd in 10 seconds.

Fixed.


Some merge cells occurs TypeError

the test Numbers file merge_cells.numbers.zip, base on tests/data/custom-format-stress-template.numbers.
Example code:
from numbers_parser import Document
doc = Document("merge_cells.numbers") # see the attachment merge_cells.numbers.zip
sheets = doc.sheets()
for s in sheets:
    print(s.name)
tables = sheets[0].tables()
for t in tables:
    print(t.name)
    rows = t.rows(values_only=True)

Result:
Sheet 1
Table 1
Traceback (most recent call last):
  File "parser.py", line 15, in <module>
    rows = t.rows(values_only=True)
  File "/Users/donly/.pyenv/versions/3.6.15/lib/python3.6/site-packages/numbers_parser/document.py", line 69, in rows
    for col_num in range(self.num_cols)
  File "/Users/donly/.pyenv/versions/3.6.15/lib/python3.6/site-packages/numbers_parser/document.py", line 69, in <listcomp>
    for col_num in range(self.num_cols)
  File "/Users/donly/.pyenv/versions/3.6.15/lib/python3.6/site-packages/numbers_parser/document.py", line 109, in cell
    return Cell.factory(self._model, self._table_id, row_num, col_num)
  File "/Users/donly/.pyenv/versions/3.6.15/lib/python3.6/site-packages/numbers_parser/cell.py", line 44, in factory
    cell = BulletedTextCell(row_num, col_num, cell_value.bullets)
  File "/Users/donly/.pyenv/versions/3.6.15/lib/python3.6/site-packages/numbers_parser/cell.py", line 141, in __init__
    super().__init__(row_num, col_num, value["text"])
TypeError: 'NoneType' object is not subscriptable

Testing Environment:

Python 3.6.15
macOS 13.4.1

I see no errors in version 4.0.0. The method API for sheets and tables was deprecated in 3.0 and has now been removed in 4.0. This is the new API:
from numbers_parser import Document

doc = Document("tests/data/issue-59.numbers")
sheets = doc.sheets
tables = sheets[0].tables
assert tables[0].cell("J2").value == "Saturday, 15 May 2021"

You're right, @masaccio.
This problem comes from numbers-parser 2.3.13 which I installed using python3 -m pip install numbers-parser.
How to install the latest version 4.0.0?

pip won't upgrade major versions automatically so you need to explicitly tell it to upgrade using python3 -m pip install --upgrade numbers-parser.


Test Protocol client

This test client is used to validate the communication channel and protocol between this client and language server. The client can send request and notification to server based on the language protocol and expect response for the request. Server can also send specific notification to client. Any custom protocol in addition to the language server protocol can be built on to od existing channel.
Mocha can used to run tests for this protocol client.

Hi @iyyappam, I'm your friendly neighborhood Microsoft Pull Request Bot (You can call me MSBOT). Thanks for your contribution!

It looks like you're a Microsoft contributor (Iyyappa Murugandi). If you're full-time, we DON'T require a Contribution License Agreement. If you are a vendor, please DO sign the electronic Contribution License Agreement. It will take 2 minutes and there's no faxing! https://cla.microsoft.com.

TTYL, MSBOT;

Hi @iyyappam, I'm your friendly neighborhood Microsoft Pull Request Bot (You can call me MSBOT). Thanks for your contribution!

It looks like you're a Microsoft contributor (Iyyappa Murugandi). If you're full-time, we DON'T require a Contribution License Agreement. If you are a vendor, please DO sign the electronic Contribution License Agreement. It will take 2 minutes and there's no faxing! https://cla.microsoft.com.

TTYL, MSBOT;

Hi
Will this pull request be merged at some point?  Could really use this.
Max

I will close this PR. Most of it got implemented by having the new npm protocol module which only depends on node and can be integrated into test runners like mocha.


x/pkgsite: remove retracted versions from search results

We don't insert a retracted version into search results when we process it, but we also don't remove it if the retraction happens later.

I've lost track of the open-source status of pkg.go.dev - is this something that I can help with, or is the source Google-internal?

@akshayjshah https://github.com/golang/pkgsite

Now live.


[Snyk] Upgrade @sanity/data-aspects from 2.33.2 to 2.35.3

This PR was automatically created by Snyk using the credentials of a real user.Snyk has created this PR to upgrade @sanity/data-aspects from 2.33.2 to 2.35.3.
:information_source: Keep your dependencies up-to-date. This makes it easier to fix existing vulnerabilities and to more quickly identify and fix newly disclosed vulnerabilities when they affect your project.

The recommended version is 21 versions ahead of your current version.
The recommended version was released a year ago, on 2022-12-21.

The recommended version fixes:

Severity
Issue
PriorityScore (*)
Exploit Maturity

Undesired Behavior SNYK-JS-STYLEDCOMPONENTS-3149924
78/1000  Why? Confidentiality impact: None, Integrity impact: Low, Availability impact: Low, Scope: Unchanged, Exploit Maturity: No data, User Interaction (UI): None, Privileges Required (PR): None, Attack Complexity: High, Attack Vector: Network, EPSS: 0.01055, Social Trends: No, Days since published: 357, Reachable: No, Transitive dependency: Yes, Is Malicious: No, Business Criticality: High, Provider Urgency: Medium, Package Popularity Score: 99, Impact: 4.19, Likelihood: 1.86, Score Version: V5
No Known Exploit

Incomplete List of Disallowed Inputs SNYK-JS-BABELTRAVERSE-5962462
78/1000  Why? Confidentiality impact: None, Integrity impact: Low, Availability impact: Low, Scope: Unchanged, Exploit Maturity: No data, User Interaction (UI): None, Privileges Required (PR): None, Attack Complexity: High, Attack Vector: Network, EPSS: 0.01055, Social Trends: No, Days since published: 357, Reachable: No, Transitive dependency: Yes, Is Malicious: No, Business Criticality: High, Provider Urgency: Medium, Package Popularity Score: 99, Impact: 4.19, Likelihood: 1.86, Score Version: V5
No Known Exploit

Improper Verification of Cryptographic Signature SNYK-JS-BROWSERIFYSIGN-6037026
78/1000  Why? Confidentiality impact: None, Integrity impact: Low, Availability impact: Low, Scope: Unchanged, Exploit Maturity: No data, User Interaction (UI): None, Privileges Required (PR): None, Attack Complexity: High, Attack Vector: Network, EPSS: 0.01055, Social Trends: No, Days since published: 357, Reachable: No, Transitive dependency: Yes, Is Malicious: No, Business Criticality: High, Provider Urgency: Medium, Package Popularity Score: 99, Impact: 4.19, Likelihood: 1.86, Score Version: V5
No Known Exploit

(*) Note that the real score may have changed since the PR was raised.

Release notes

Package name: @sanity/data-aspects

2.35.3 - 2022-12-21

2.34.3-cdr-preview.20 - 2022-10-20

2.34.3-cdr-preview.19 - 2022-10-20

2.34.3-cdr-preview.17 - 2022-10-20

2.34.3-cdr-preview.14 - 2022-10-19

2.34.3-cdr-preview.7 - 2022-10-14

2.34.3-cdr-ga-4.23 - 2022-10-20

2.34.3-cdr-ga-2.19 - 2022-10-20

2.34.3-cdr-ga.20 - 2022-10-20

2.34.2-empty-template-cli.0 - 2022-10-13

2.34.1-canary.0 - 2022-10-12

2.33.4-shopify.8 - 2022-09-22

2.33.4-performance-opts.16 - 2022-09-22

2.33.4-performance-opts.15 - 2022-09-22

2.33.4-performance-opts.13 - 2022-09-21

2.33.3-performance-opts.14 - 2022-09-19

2.33.3-performance-opts.10 - 2022-09-16

2.33.3-performance-opts.9 - 2022-09-16

2.33.3-performance-opts.5 - 2022-09-15

2.33.3-performance-opts.4 - 2022-09-14

2.33.3-edge.8 - 2022-09-20

2.33.2 - 2022-09-14

    from @sanity/data-aspects GitHub release notes

Commit messages

Package name: @sanity/data-aspects

d5ef5ca v2.35.3
e78489f chore: upgrade release notes template for 2.x
a7e00a0 fix(cli): specify 2.x version ranges for dependencies (#3993)
0dba94b chore: deprecate v2 modules (#3997)
a464646 chore: simplify v2 tag publish command
d477e34 chore: deleted plugins that now live outside the monorepo (#3899)
f544e11 fix(form-builder): remove distinctUntilChanged function from handleQueryChange in referenceInput (#3901)
11988da fix(form-builder): improve inputmode detection for number input and fix tests
cf1c64b chore: update v2 publish command to tag release with v2
4d48d8b ci: update branch name
9bf408d v2.35.2
d746aae fix(base): skip cross dataset validation on mutations (#3856)
25de30f v2.35.1
00fb5d6 fix(react-hooks): issue with `useEditState` low priority
3dbbd9e v2.35.0
6b0afbd refactor(form-builder): simplify cross dataset reference config (#3670)
5d1e161 feat(core): include (non-materialized) cross-dataset references in GraphQL schemas
9c6dd6b fix(schema): include `_dataset` field for cross-dataset references
3a2fc45 chore(default-layout): re-enable search sorting (#3765)
3ab221a fix(desk-tool): fix issue with history pane not showing the earlier version (#3759)
797315a fix: persist split panes on reload
6b32c5e v2.34.2
c387259 feat: add empty schema (#3756)
2de70e2 v2.34.1

Compare

Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open upgrade PRs.
For more information:
🧐 View latest project report
🛠 Adjust upgrade PR settings
🔕 Ignore this dependency or unsubscribe from future upgrade PRs

Closing this as this is related to studio v2


Fix to #3499 - Exception with in-list where clause across joined table

Problem was that during navigation expansion we would add the created join clauses into the query model that the navigation was used. However, sometimes the navigation references a query source from the outside query, and the joins should be added there instead.
Fix is to keep track of hierarchy of query models and testing whether the given query model is the right one to add the join to (i.e. it contains a declaration of the query source that the join references)
Also added some minor fixes - we were not performing navigation expansions for result operators and we were not translating EF.Property() inside Contains result operator properly.

@anpete

:shipit:


⚠ Bofrost.at has degraded performance

In 5b8ef68, Bofrost.at (https://www.bofrost.at) experienced degraded performance:

HTTP code: 200
Response time: 1127 ms

Resolved: Bofrost.at performance has improved in c77185e after 7 minutes.


Merge develop in release 1.3.1 branched off main (#462)

Add cookie consent to error pages

Add cookie consent to error pages

Add IDs to all forms that didn't have one

This has been requested by the metrics team. Whereas I've ensured all
IDs are prefixed with analytics- I have not changed any existing IDs
since these might be included in existing reports etc.

Add custom error view to handle cookie consents

Fix: TypeError: 'NoneType' object does not support item assignment (#425)

Don't bother to apply cookie middleware to non-frontend URLs and handle cases where context_data is None

Add tests for new behaviour

Placate linters

DF-357: Upgrade Wagtail to 3.0.1 (#430)

Upgrade Wagtail and run poetry update

Update project to account for module renaming

Use FieldPanel where more specific panels are not needed

Set use_json_field=True for all StreamFields

Rename BASE_URL setting to WAGTAILADMIN_BASE_URL

Upgrade plugins to Wagtail 3.0 compatible versions

Remove redundant/unused body field from InsightsIndexPage (#431)

Add cookie consent to error pages

Add custom error view to handle cookie consents

Load the necessary (and only necessary) template tags for error templates

Fix broken urls

Allow static files to be served by Django when DEBUG=False

Added URL paths for error page to the public_urls when debug is true for development

Rearrange the feature insights location in the admin and templates

Add analytics tracking for interactions with search result link

Fixing unit tests and formatting.

Fixing flake8 and isort formatting

Fixing duplicated insights

Remove un used cached_property from the insight model

Df 340 refine results by heading typeface is roboto should be supria sans condensed (#441)

H3 font change across site

Moving the H3 element to outside of the form, see DF-315

Moving the logic block for  element

Breadcrumbs feature added  (#439)

Add initial breadcrumb to site

Introduce .breadcrumb-content

Relocate breadcrumb section, making parent the final and change font

Adjustment added padding.

Change font to "Open Sans"

Fix unit tests

Unit test will pass after merging with Develop

Introducing nav tag, aria-lable and aria-current

switched "ul.breadcrumb li a"  text-decoration to underline and "ul.breadcrumb li a:hover" to none.

Remove card group record summary body (#433)

Remove card group record summary body

Remove failing test

Feature/df 255 add zero results message (#445)

html elements for no results on search

moving no results to a separate html block / additional sass added

seperate block of html for no results

no search results for catalogue and website tabs

bullet point styling chenge

Html and Python logic for show/hide elements based on number of results

search filters functionality

breadcrumb display and functionality (#448)

Initial html/css build of new label

aria described by element added, as span is now a block level object we need to add a role element in as well

Added a date to know when to show new label

Un 266 (#450)

Add initial breadcrumb to site

Introduce .breadcrumb-content

Relocate breadcrumb section, making parent the final and change font

Adjustment added padding.

Change font to "Open Sans"

Fix unit tests

Unit test will pass after merging with Develop

Introducing nav tag, aria-lable and aria-current

switched "ul.breadcrumb li a"  text-decoration to underline and "ul.breadcrumb li a:hover" to none.

Removing breadcrumb from searches, changing font size and making first element home

reduce font size to 0.8em

Fix merge issue

removing aria-current="page" from home page li

fixing merge issue

Added data-label to satisfy UN-272 as well

Added data-label in appropriate places

Added logic to HTML to show/hide new label

Fixing isort formatting

Un 248 (#442)

Filter out duplicated pages from latest items

typo error

using comprehension list to filter the more stories

Limit to 3 max on latest query

Fixing formats

Fixing formats with black

Removing featuredcollections from the InsightsIndexPage model

Fixing flake errors

styles and logic for stories published less than 21 days

Put the "days" value back to 21 days

Minor style change on breadcrumbs. (#453)

Co-authored-by: JamesWChan<EMAIL_ADDRESS>Co-authored-by: Gwyn Jones<EMAIL_ADDRESS>Co-authored-by: mo zaki<EMAIL_ADDRESS>Co-authored-by: Andy Babic<EMAIL_ADDRESS>Co-authored-by: andrewC<EMAIL_ADDRESS>Co-authored-by: JohnHeeryTNA<EMAIL_ADDRESS>Co-authored-by: JohnHeeryTNA<EMAIL_ADDRESS>Related ticket(s):

<- Paste ticket URL here ->

About these changes
<- Add human-readable description here ->
How to check these changes
<- Where possible, provide guidance to help your reviewer ->
Dear reviewer, I promise I have:

[ ] Checked things thoroughly myself before handing over to you.
[ ] Included the ticket number in the PR title to help us keep track of changes
[ ] Ensured that my PR does not include any irrelevant commits.
[ ] Waited for all CI jobs to pass before requesting a review.
[ ] Added/updated tests and documentation where relevant.

All good!


Improved iPad Support

This includes a couple of fixes to make CodeMirror work better on an iPad.

Mobile Safari handles z-index within a specific parent element and positioning type - with the previous version the selection would hide the code and the gutter background would hide the line numbers.  This does mean that the gutter background won't cover horizontally scrolled code, but using a fixed gutter runs into other issues on iOS anyway.

I have replaced onMouseDown with a set of touch event handlers when running on iOS.

Mobile safari does not generate mouse events until it knows if it is a click or double click - i.e. it waits 300ms then fires all the events at once, so anything timing related won't work. The new implementation moves the cursor immediately and decides later if it also needs to select the word/line.
To avoid conflicts with scrolling, drag and selection are triggered with either a 100ms pause before movement or movement on the second touch.

This should be superseded by the recently landed mobile work.


Add additional arrowheads to the interface

There are more types of arrowheads implemented than what's available on the interface. In order to add them to the interface there should be a more.../less... link should allow showing/hiding additional arrowheads in the arrowhead popout.

Add the show/hide logic to the arrowhead popout
Add the additionally implemented arrowheads to the conditionally hidden section
Move the T arrow end to the conditionally hidden section
Add "crow's feet" arrowhead and put it in the hidden section

packages/excalidraw/element/types.ts contains the type Arrowhead which has the names for the currently hidden arrowheads.

I would like to work on it. To make sure I understand you correctly, by "crow's feet" arrowhead you mean something like this

@j-krolak yes, exactly on the "crow's feet". Please link a PR (even if it's draft) so everybody can follow your work.

Hey @mtolmacs , I've made a PR https://github.com/excalidraw/excalidraw/pull/8616  which resolves this issue.


Enable enter key continuation for GFM task list

This adds an optional 3rd capturing group for task lists and passes it to the enter completion context.
Assumed we don't care if GFM is enabled or not.
closes #1

Is it intentional that this copies the 'done' status from the previous item, rather than creating new items as [ ]?

Looks good, thanks!


installer.sh: update EPEL installer, readability

Fixes #1664
EPEL install reworked for RHEL/CentOS, cleanup of code.

/packit retest-failed

@UntriexTv in order to pass the DCO check you need to use the -s flag for git commit. Can you amend this commit with a '-s' so it adds your signature to the commit?

Is this okay @mpeters ? Or should I also squash the commits

@UntriexTv it's fine to use multiple commits instead of squashing (although I prefer to squash myself just for my personal work flow) as long as all of the commits are signed. So you can either squash or amend+sign all of the commits. Your choice.

/packit retest-failed

Please add this comment once more, the failing test should be fixed now.

/packit retest-failed

@UntriexTv can you add a signoff line to either all commits or squash them together?
@sergio-correia had you a chance to test this?

Sorry for late reply, it should work now.


v3.0.0

deprecated skipWhileRefresh
renamed skipWhileRefresh flag to pauseInstanceWhileRefreshing
changed default behaviour of pauseInstanceWhileRefreshing

Thanks. When can we expect this version ?

Closing in favour of #111


rpc: Add WebUI statistics procedure

Summary
Send all data used for the WebUI statistics table in one command.
This will have to be expanded in the future, but number of online characters, accounts and characters in general is not available at this time afaik.

@miles200 I believe I would want to add those bits but I'm still not sure how.


Add ssh_user to cifmw_bootstap_local_instances

Add the ssh_user option to local instances. If we deploy an actual CRC node locally we need to set ssh user core for this instance.
The ssh_user is set used for ansible_ssh_user in the inventory and defaults to cloud-user.

lgtm


[stable/anchore-engine] add openshift deployment option

Signed-off-by: Brady Todhunter<EMAIL_ADDRESS>
Is this a new chart

NOTE: We're experiencing a high volume of PRs to this repo and reviews will be delayed. Please host your own chart repository and submit your repository to the Helm Hub instead of this repo to make them discoverable to the community. Here is how to submit new chart repositories to the Helm Hub.

What this PR does / why we need it:
Which issue this PR fixes
(optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged)

fixes #

Special notes for your reviewer:
Checklist
[Place an '[x]' (no spaces) in all applicable fields. Please remove unrelated fields.]

[X] DCO signed
[X] Chart Version bumped
[x] Variables are documented in the README.md
[X] Title of the PR starts with chart name (e.g. [stable/chart])

/assign zhill

/lgtm


Applying -AsText to multiple columns?

Could I get some guidance on how to force more than one date column to be imported as text?
This works great! But I also need to import HireDate as text...
Import-Excel -Path C:\test.xlsx -AsText "BirthDate"

This works for me:
Import-Excel -path C:\temp\AsText.xlsx -AsText 'HireDate', 'BirthDate'

before -AsText

after -AsText


Proposal: Add byte[] to image converter

Rationale
byte[] to image converter is one of the most used ones in xaml and it would be nice to not have to implement it in almost every new project like your every kind of the visibility converter

Something for the https://github.com/CommunityToolkit/WindowsCommunityToolkit ???

Something for the https://github.com/CommunityToolkit/WindowsCommunityToolkit ???

yep


Allow a role to be assumed before checking Docker images in AWS ECR

This is to resolve #6152
Very much a work-in-progress. I've tested this code with:

An ECR repository that gives read permissions to the user via a resource policy
A user with an identity policy to read the ECR repository
(both of the existing use-cases)

And a new use case:

A user with permission to assume a role which has an identity policy to read the ECR repository

I still have a few challenges:

Writing the tests to verify this behaviour
Documenting how to use this feature

Mostly "which repo has the user-facing docs that need updating?"

Understanding how the Credentials Proxy gets used from the Docker package updater

And if this change affects it

Any advice or guidance on these would be greatly appreciated!
I spotted a bug with bin/dry-run when credentials get set: they're not converted to Dependabot::Credentials. A follow-up to #8967?
I also saw that regctl assumes it's already logged in, which isn't always true.

Unfortunately, I don't think this is going to work, in production the VM that runs dependabot does not have access to credential data. We run a short-lived job specific proxy that intercepts requests and attaches any credentials that are configured for requests that match the configured registry. So while this might work in dry-runs, in production it would fail.
Unfortunately said proxy is not publicly available today

Thanks for the explanation, I was worried about that. I see that the  Dependabot CLI can set up a local environment with the credential proxy, but I guess that it's aimed at internal team members?

Thanks for the explanation, I was worried about that. I see that the Dependabot CLI can set up a local environment with the credential proxy, but I guess that it's aimed at internal team members?

Yeah the proxy container can be pulled publicly and contains an executable version of the proxy, but the source code is not available today. We've talked about opening this up as well to allow contributions such as yours but it's not something we have concrete plans for right now, as there's a bit of red tape involved and so many competing priorities. I wish I had a more satisfying answer to be honest.

No worries, thanks for confirming that. I'll drop this PR: the actual AWS SDK code change is pretty trivial, so no big deal to recreate later when/if the proxy gets opened up.


remove hub-crds references

Signed-off-by: Ray Harris<EMAIL_ADDRESS>

/hold

Midstream PR:
https://gitlab.cee.redhat.com/acm-cpaas-midstream/multiclusterhub-operator/-/merge_requests/97

/lgtm @ray-harris remove the hold once the downstream has been merged

@JakobGray The midstream PR is failing. Talked with Brian Smith and the upstream has to be merged first.

/unhold


Field aliases don't work with replace

Initial Checks

[X] I confirm that I'm using Pydantic V2

Description
Generalized replace doesn't take into account field aliases.
When a field has an alias, generated __replace__ (and therefore copy.replace in Python 3.13) only recognizes the original field name and not the alias.
I expect parity between constructor (accepts the alias), attribute access (accepts the alias) and replace (only accepts the original name).
See also #11009.
Example Code
from copy import replace
from pydantic import BaseModel, Field

class One(BaseModel):
    foo: str = Field(..., alias="bar")

one = One(bar="a")
print(one)
one1 = replace(one, foo="b")
print(one1)  # replaced
one2 = replace(one, bar="c")
print(one2)  # not replaced

print(one.foo)  # AttributeError as expected

Python, Pydantic & OS Version
pydantic version: 2.10.2
        pydantic-core version: 2.27.1
          pydantic-core build: profile=release pgo=false
                 install path: /home/alex/.cache/pypoetry/virtualenvs/mybox-YUnsYrZx-py3.13/lib/python3.13/site-packages/pydantic
               python version: 3.13.0 (main, Oct  8 2024, 00:00:00) [GCC 14.2.1 20240912 (Red Hat 14.2.1-3)]
                     platform: Linux-6.11.8-300.fc41.x86_64-x86_64-with-glibc2.40
             related packages: mypy-1.13.0 typing_extensions-4.12.2
                       commit: unknown

Internally, BaseModel.__replace__ forwards the call to model_copy. Pydantic does not perform any validation with the provided data, and the alias logic is part of the validation process. As such, you should provide the original field name.
The behavior is debatable (maybe in particular regarding copy.replace), but I believe most of the model_copy/copy.replace calls are performed using data you know about.
Besides, invalid identifiers can be used in aliases:
class Model(BaseModel):
    a: int = Field(alias='invalid-id')

If we were to use aliases, you'd have to call replace this way: copy.replace(model_inst, **{'invalid-id': 1}).


Create a new pull request by comparing changes across

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.03%) to 95.665%

Totals

Change from base Build<PHONE_NUMBER>:
-0.03%

Covered Lines:
2025

Relevant Lines:
2077

💛  - Coveralls


tls1_set_groups_list: freeing *pext before overwriting

calling SSL_CTX_set1_groups_list() twice on one SSL_CTX* caused a memory leak visible in valgrind:
4 bytes in 1 blocks are definitely lost in loss record 1 of 1
at 0x4841888: malloc (vg_replace_malloc.c:381)
by 0x4B1EE96: CRYPTO_memdup (in libcrypto.so.3)
by 0x48993A0: tls1_set_groups_list (in libssl.so.3)
by 0x487AA7E: ssl3_ctx_ctrl (in libssl.so.3)
by 0x1091EA: main (mem_leak.c:10)
LEAK SUMMARY:
definitely lost: 4 bytes in 1 blocks
Freeing *pext to fix it.
CLA: trivial

It would be nice to have a test for this. Probably just something simple in test/sslapitest.c which creates an SSL_CTX/SSL object and calls SSL_CTX_set1_groups_list twice on it. Our CIs with enable-asan on them would detect any resulting mem leaks.

ok, I added a simple test case to est/sslapitest.c

https://www.openssl.org/policies/cla.html

Unfortunately this goes beyond what we can accept with "CLA: trivial". Please can you submit a CLA?

Actually my work was just to provide a one line fix, this is a trivial change.  The testcase was your addition and is your intellectual property.  Thus please merge it without me sending my private data and signature.

Actually my work was just to provide a one line fix, this is a trivial change.  The testcase was your addition and is your intellectual property.  Thus please merge it without me sending my private data and signature.

I only provided the outline of an idea. You provided the implementation. Unfortunately I still consider the changes to be beyond what we can accept with "CLA: trivial".
Aside from that this PR now looks good to me.

Actually my work was just to provide a one line fix, this is a trivial change.  The testcase was your addition and is your intellectual property.  Thus please merge it without me sending my private data and signature.

I only provided the outline of an idea. You provided the implementation. Unfortunately I still consider the changes to be beyond what we can accept with "CLA: trivial".

So we agree, that we disagree on that point. I consider my duty as open source contributor on this subject as done, it's upon you to use or reject my patch.

This PR is in a state where it requires action by @openssl/otc but the last update was 30 days ago

As we cannot accept the test, could you please revert your patch to the original one? We can accept that one. We'll have to write the test ourselves from scratch.

I've added two commits that restores things to the original commit.  git rebase -i --autosquash will put it together well.

Test deferred to another PR, which will need to be written

Considering my involvement, I can't see myself approving

thanks for all your effort, is there anything left for me to do? e.g. do I need to set approval: ready to merge?

No need.  The bot that usually does this is just a bit fidgety about comments showing up after the approvals (or if it is after the approved: done label was set, I don't quite recall).

Merged
3.0:
fcf3a9f7c6a10acb2d92f03aec5e45df7dd712d5 tls1_set_groups_list: freeing *pext before overwriting
3.1:
2607ea3060d8fc825d24753233e817eba6c4dab4 tls1_set_groups_list: freeing *pext before overwriting
master:
62ea5ffa7c8882ba90b26ab1deb0d977dcb5165c tls1_set_groups_list: freeing *pext before overwriting


client.js/index.js relationship, impossible to access store

containers/App.js seems the connection point to the reducer, but it is only include in index.js
client.js seems the entry point during dev, which points directly to components/app.js
I can't access the store as I'm using directly component/app.js (which is stateless)
I tried to modify this way but both components expect a single root and I'm not sure what is the proper way
  <AppContainer>
    <Provider store={store}>
      <App />
    </Provider>
  </AppContainer>,

Also, it would be nice to add a note about how to use previous generator version, I have to teach React from time to time and solving a problem like this in a class is quite complicated (whatever it is a bug or not, I got stuck for more than an hour now and I'm giving up).

Hey, this is related to this issue => https://github.com/stylesuxx/generator-react-webpack-redux/issues/56 which I fixed in develop, closing down here.


Quickstart docs are outdated

Bazel version is outdated.
my 2 cents will be to concentrate the documentation in one place because the readme and the quickstark will keep drifting apart

Hi @chinchaun, thanks for the recommendation! We are planning on automating the process of keeping README and Quickstart in-sync. Also happy to have anyone work on that.

Fixed by:

#1127


WIP: feat/types

#36

I realized copying the tags field wouldn't work: no way to delete types. I will switch to a different type input field and commit.

Gonna pass on writing integration tests for now in the interest of time. As we're moving to a repo with submodules, I'll be implementing the integration tests over there with time.


Fix piwik missing slash problem

If there is no trailing slash in the Piwik URL, it won't load piwk.js or piwik.php. I added a slash in the view to fix it.

What happens if the user supplies a / at the end of their URL?

Maybe rtrim($url, '/')?

Sure, good idea!

Surely this is still wrong?

It should be done earlier on when the config is set in the first place. We need to validate it correctly.

Why do you say this is still wrong?
I'm not fond of modifying user's input at registering the config: it might be disturbing to see something different from what you typed earlier. But your advice will prevail, I don't really care.

@willdurand modifying the user input here is important because if we were to use this value anywhere else, we'd need to keep sanitising it. Since this is a setting, we should do that before it goes into the database.

Ok.

Merged and fixed.


Add Show Comments button

@shirhatti Please review

@javiercn @Rick-Anderson @rachelappel @Erikre Please review

:shipit:

I propose you do one of these two things:

Replace the Show comments with Show/Hide toggle functionality
Hide/Disable the Show comments button after you press it.

Right you can repeatedly press the show comments, and there is no feedback to indicate nothing happens after the first time you press it

@shirhatti I opted for #2.

#2 is the right choice.

:shipit:

cd2b93ccd18fe57cbdf7f71e61349fdc4a26ce3f


chore: release v0.8.1

fix: mongodb connexion (#297)
feat: use grpc to cancel tasks and sessions (#299)
ci: fix npm installation (#320)
chore: changed path to /old-admin/ (#469)
chore: update version for old GUI (#470)
chore: release 0.8.1

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.1 out of 2 committers have signed the CLA.:white_check_mark: esoubiran-aneo:x: Faust1-2was-AneoYou have signed the CLA already but the status is still pending? Let us recheck it.


bug-fix: fix an issue that VirtualService hosts with * will be rejected

What type of PR is this?

/kind bug
What this PR does / why we need it:
Which issue(s) this PR fixes:

Fixes #154
Does this PR introduce a user-facing change?:

fix an issue that VirtualService hosts with `*` will be rejected

/lgtm
/approve
Thanks for the PR!

@LiorLieberman could you review & approve the PR? Thanks!


Allow auth between the Ollama server/client

What are you trying to do?
I'm trying to secure the connection between ollama client and server.
HTTPS works fine:

Basic auth does not work with ollama client:

How should we solve this?
Allow to set basic auth in OLLAMA_HOST or other variables for the client.
What is the impact of not solving this?
The only way to secure the connection between ollama client and ollama server is with IP filtering. Basic auth would be a bit better.
Note that for any other client (such as python requests), this is not an issue since those packages can manage basic auth (or other auth mechanisms for that matter).
Anything else?
Allowing the server to use TLS would be nice too. Traffic is still cleartext between the reverse proxy and ollama server.
Displaying a better error message with http status code would be nice.

I came here to request this feature. Right now the code in /envconfig/config.go will interpret the ":" separating the username from the password as the separator for the tcp port number.

This is a duplicate of issue #1053 and should be closed.


Question: Do Tx Queries get instrumented with SQLComments?

Sorry if this is a dumb question, but if I use SQL transactions, will I still get those transactional queries instrumented with SQL Comments:
    result, err := tx.ExecContext(ctx, "INSERT INTO album_order (album_id, cust_id, quantity, date) VALUES (?, ?, ?, ?)",
        albumID, custID, quantity, time.Now())

Hello, yes it works also for transactions.


Popover on latest iOS fills whole screen, covering everything with background color

Design as follows:
Very basic UIView, containing text field and button, being called as popover. Modeled in Storyboard with latest XCode
Behavior: iOS 6.x: works perfect, popup show in size modeled, calling view is visible.
With upcoming version of iOS the popover content area is the same (and actually shows the content).
Nevertheless the screen is filled up with the background color I set for UIView in the nss file, covering and hiding the view the popup is being called from.
Only solution I came up with up to now  to get a properly sized popover is by setting a global exclude for UIView class,
[NUISettings setGlobalExclusions:@[@"UIView"]];
which is not a good solution.
Showing up on iPad Mini on lastest beta
Cross check with iPad4 on iOS6.1
Works fine on app without NUI.

Any update on this one? This is a definitely a blocker

Due to issues like this, I would recommend generic classes not be used (i.e. UIView, UIButton, UITextField, etc.). Instead specify a custom class in the .nss file (i.e. MyView, MyButton, MyTextField) and then assign views to use the respective style classes via assigning the object's 'nuiClass' property. This approach allows for deliberate control over what to style which I believe is the best practice to adhere to.

Makes sense. Thank you

@timbodeit perhaps we should consider adding a "Best Practices" section to the README (or better yet, the Wiki) discouraging this usage of styling? I don't really want to deprecate this functionality as it can come in handy for fast prototyping/demo purposes.


Implement compressed training and test on AutoPET

[x] Implement saving as npz
[ ] Implement loading as npz
[ ] Test training time for 3D
[ ] Test training time for 2D
[ ] Write in-code warnings and such
[ ] Add docs in GitHub

@asbjrnmunk if you have any suggestions in general for this lmk here


Handle inexhaustive match patterns

Gleam v0.33 release flagged an inexhaustive match:
warning: Inexhaustive patterns
   ┌─ /gap/src/gap/myers.gleam:80:3
   │
80 │ ╭   case diag, limit, paths {
81 │ │     0, 0, [path] -> #(path, [])
82 │ │     diag, _limit, [path, ..] as paths if diag == neg_limit -> #(
83 │ │       move_down(path),
   · │
95 │ │     }
96 │ │   }
   │ ╰───^

The missing patterns are [] and [_]. I'm not sure what the algorithm is supposed to do here, otherwise I would send a PR.

Closed by #6


ally.disengage re-enables previously disabled elements

You have 2 inputs, one has a disabled attribute by default.
You call ally.maintain.disabled().
Both inputs are now disabled.
You call ally.disengage().
Both inputs are now enabled.
The problem is that the input that was disabled by default needs to remain disabled after ally.disengage.
https://jsbin.com/fotejizolo/1/edit?html,css,js,output

@rodneyrehm we may have an easier time fixing this now with the inertElementCache. I don't think we need to add the element to the inertElementCache if it was already inert, because then when disengage is called we can just disengage elements that Ally made inert.
Thoughts?

@rodneyrehm I tested this in maintain/disabled.js renderInert method and it seems fix the issue, not sure if there's something else that would need to be done to cover other inert cases:
if (!element.disabled && !element.inert) {
  this._inertElementCache.push(element);
  makeElementInert(element);
}

Where's element.inert coming from? You'd probably want to switch the condition to use ally.is.disabled instead, to make sure that the fix does not only apply to form controls, but all disabled elements.

I'm not sure that element.inert is a thing in any browsers yet. The most basic need is that we need to protect against is a form element that is :disabled before maintain.disabled was engaged. I don't think ally needs to deal with the element if it was already disabled. Does that sound right?

An element is "disabled" if it has disabled property set to true (form controls), or if it has the attribute data-ally-disabled (everything else).
An element that is disabled at the time of engaging ally.maintain.disabled should not be disabled again, and should not be enabled upon disengaging ally.maintain.disabled.
The MutationObserver should identify elements that became enabled in case they were not disabled because of 2 (i.e. listen to the attributes disabled and data-ally-disabled in addition to tabindex)


Update deps

Codecov Report

Merging #25 (9fe45ec) into main (b341fa5) will not change coverage.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##             main      #25   +/-   ##
=======================================
  Coverage   94.61%   94.61%
=======================================
  Files           4        4
  Lines         762      762
=======================================
  Hits          721      721
  Misses         41       41

Impacted Files
Coverage Δ

lib/authorize.js
100.00% <100.00%> (ø)

lib/helpers.js
89.59% <100.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


FAQ page

got the FAQ page done

new questions can be added by adding new FAQuestions to the page :)

 FAQ Page - Web App


Assign parameter names to values when calling a function to avoid setting of unneeded optional parameters

Suggestion
Parameter assignation when calling a function, to make code more readable and to omit optional parameters that would otherwise not need be defined in the function call
🔍 Search Terms
assign parameters calling function assignation optional omit
List of keywords you searched for before creating this issue. Write them down here so that others can find this suggestion more easily and help provide feedback.
✅ Viability Checklist
My suggestion meets these guidelines:

[X] This wouldn't be a breaking change in existing TypeScript/JavaScript code
[X] This wouldn't change the runtime behavior of existing JavaScript code
[X] This could be implemented without emitting different JS based on the types of the expressions
[X] This isn't a runtime feature (e.g. library functionality, non-ECMAScript syntax with JavaScript output, new syntax sugar for JS, etc.)
[X] This feature would agree with the rest of TypeScript's Design Goals.

⭐ Suggestion
When calling a function it is currently not possible to assign parameters to their defined param name in the calling function, this has as a result that when using optional parameters and you wish to use any optional parameter after the first optional parameter, that you would need to set a value to all optional parameter before the one you are setting.
This simply results in setting some optional parameters value equal to the values that were already set as either their default value or null in the function definition...
This is not very readable and causes clutter and is prone to error when multiple optional parameters are present on a function
📃 Motivating Example

Imagine following situation, we have a function with optional params
So either declared like this:
function exampleFunc(x: string, y?: string, z?: string): void{
//Logic here does not matter for the feature
}

Or declared like this:
function exampleFunc(x: string, y: string = null, z: string = null): void{
//Logic here does not matter for the feature
}

Now if I wish to call this function and I only want to set the x and z parameters I am obliged to set the y param to it's default value as shown below
exampleFunc("someValue", null, "someValue");
It would make more sense if I could call the function as stated below and omit the optional param I don't want to change the default value for...
exampleFunc(x: "someValue", z: "someValue");
This would omit the need to set the y parameter to null since it already is null (or any other value that was set in the function definition) by default...
Stackblitz example: https://stackblitz.com/edit/typescript-ckfrdm
I feel this would greatly improve readability and omit code clutter
💻 Use Cases

I would like to be able to assign parameter names and set values to these names, in order to omit the setting of unneeded optional parameters.
Current shortcomings of this feature: => This feature not existing introduces the following shortcomings int TS code

code clutter,
less readable,
prone to error

I don't use any workaround as there are none, at this time you are obliged to set all optional params until you have reached the one you wish to use....

Duplicate of https://github.com/microsoft/TypeScript/issues/467

This is one of the features that needs to be supported by JavaScript first.

I don't use any workaround as there are none

I mean, you can pass an object and destructure it, which is how this is usually done in vanilla JS too:
interface FooOptions {
    foo: string;
    bar: string;
    baz?: number;
}

function foo({ foo, bar, baz }: FooOptions) {
    console.log(foo, bar, baz);
}

// prints "fooey, bard, undefined"
foo({ foo: "fooey", bar: "bard" });

I don't use any workaround as there are none

I mean, you can pass an object and destructure it, which is how this is usually done in vanilla JS too:
interface FooOptions {
    foo: string;
    bar: string;
    baz?: number;
}

function foo({ foo, bar, baz }: FooOptions) {
    console.log(foo, bar, baz);
}

// prints "fooey, bard, undefined"
foo({ foo: "fooey", bar: "bard" });

@fatcerberus yeah I added that to the stackblitz and the issue just now! thx for the info though.
Still feel it would be nice if we could just name the params like in C# for instance, oh well it'll do just fine with passing  the object and destructuring it indeed :)

For what it's worth, you don't need the cast as INamedParameters.  You do need to include your default values in the interface, though: Playground
It'd be nice if you could write the types inline as function foo({ foo: string, bar?: number }) {}, but that doesn't work because it conflicts with the syntax for renaming destructured properties. 😢

You checked the box for

This isn't a runtime feature (e.g. library functionality, non-ECMAScript syntax with JavaScript output, new syntax sugar for JS, etc.)

but this is definitely a runtime feature.


Grass & dirt sprites need to be replaced / resprited

Description
To be more specific, the offending sprites in question are Resources\Textures\Tiles\Planet\dirt.rsi and Resources\Textures\Tiles\Planet\Grass.

Hilariously, these sprites are literally just straight up taken from Trials of Mana.
Outside of the obvious, these definitely do not fit the art style anyways (they're from an entirely different game, duh) and are pretty hueshifted & low in detail.
We have a few sprites in the repo to replace these already, namely grass.png, grassdark.png, and grasslight.png in the Resources\Textures\Tiles folder.. They are pretty noisy and likely won't look nearly as good, but should do the job alright.
 They are pretty important to making the grass biome look good.

Ideally the grasses also need variants


Bug in dealing with dir name?

Dr. Pearson,
I got the same error as in the issue 14. For sequence fasta file at /path/to/mySeq.fasta, fasta36 produced *** ERROR [comp_lib9.c:642] cannot open library /path/to/mySeq.fastax. The offending dir name is "1341I124_1348I1_1349I1" (where "I" is the upper case of "i"). I changed the path name and it then worked.
Fasta package version is 36.3.8h.

If your fasta file is at "/path/to/mySeq.fasta", then I am puzzled why the error message said "cannot open library /path/to/mySeq.fastax", since your sequence is in .fasta, not .fastax.  I suspect that something was miss-typed.

Thank you very much for quick response.
All the paths and fasta files are programmatically generated:
my_path=/path/to/fasta
[[ -d $my_path ]] || mkdir -p $my_path
then write fasta to $my_path/mySeq.fasta
if [[ -s $my_path/mySeq.fasta ]]; then
# run fasta program
fi
I also manually checked the file path.
I am not sure if the dir path depth (the path is like /root/sub/sub-sub/.../mySeq.fasta) or overall path length causes the problem. After moving mySeq.fasta one level up, fasta program ran successfully.

Thank you very much for quick response.
All the paths and fasta files are programmatically generated:
my_path=/path/to/fasta
[[ -d $my_path ]] || mkdir -p $my_path
then write fasta to $my_path/mySeq.fasta
if [[ -s $my_path/mySeq.fasta ]]; then
# run fasta program
fi
I also manually checked the file path.
I am not sure if the dir path depth (the path is like /root/sub/sub-sub/.../mySeq.fasta) or overall path length causes the problem. After moving mySeq.fasta one level up, fasta program ran successfully.

Thank you very much for quick response.
All the paths and fasta files are programmatically generated:
my_path=/path/to/fasta
[[ -d $my_path ]] || mkdir -p $my_path
then write fasta to $my_path/mySeq.fasta
if [[ -s $my_path/mySeq.fasta ]]; then
# run fasta program
fi
I also manually checked the file path.
I am not sure if the dir path depth (the path is like /root/sub/sub-sub/.../mySeq.fasta) or overall path length causes the problem. After moving mySeq.fasta one level up, fasta program ran successfully.

The problem you are seeing is occurring because the MAX_FN (max filename) defined in defs.h is:
#define MAX_FN 120
The simplest solution is to increase that length.  I suggest:
#define MAX_FN MAX_STR    -- which will change it to 512.
A better solution would be to allow the filename to be an arbitrary length, but that would require quite a few other changes.

Thanks for your advice!

The latest version of the fasta package (v36.3.8i) increases the maximum query file name length to 512, and provides an error message if it is exceeded.


#241 test_update_endpoint_state

Codecov Report

Merging #291 into master will increase coverage by 4.86%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master     #291      +/-   ##
==========================================
+ Coverage   67.25%   72.12%   +4.86%
==========================================
  Files          22       22
  Lines        1072     1087      +15
  Branches       91       91
==========================================
+ Hits          721      784      +63
+ Misses        335      286      -49
- Partials       16       17       +1

Impacted Files
Coverage Δ

poseidon/baseClasses/Rabbit_Base.py
100% <ø> (ø)
:arrow_up:

...bstraction/test_NorthBoundControllerAbstraction.py
100% <100%> (ø)
:arrow_up:

...llerAbstraction/NorthBoundControllerAbstraction.py
62.74% <0%> (+31.37%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 224347e...3611492. Read the comment docs.


Before the HOL : fix unclear steps

region to eastus/westus
Azure DevOps: service principal (manual)
Azure DevOps: "Clone" button x2
PRIVATEKEYNAME

LGTM!


[Trivial] Fix typo

Found typos at:

network_graph.h
acti_func.cpp.h
bn_latyer.h
common_properties.h
concat_layer.cpp

:memo: TAOS-CI Version: 1.5.20200925. Thank you for submitting PR #2231. Please a submit 1commit/1PR (one commit per one PR) policy to get comments quickly from reviewers. Your PR must pass all verificiation processes of cibot before starting a review process from reviewers. If you are new member to join this project, please read manuals in documentation folder and wiki page. In order to monitor a progress status of your PR in more detail, visit http://ci.nnstreamer.ai/.

Nice catch! LGTM.


Fix Services and Requests count in dashboard

Requests count would now be derived from 3 Request promise arrays for Pending, Approved and Denied.
Each promise array would comprise of 2 promises for the two types of Request -
ServiceReconfigureRequest and ServiceTemplateProvisionRequest
This change is mainly done due to the API design limitation on how it handles the logical OR operation.
https://bugzilla.redhat.com/show_bug.cgi?id=1333942

@himdel Please review


Odd sizing when using smaller index types in new varlen objects

In my application I'm dealing with essentially small arrays containing u32 (or at least something similar). There are many (thousands) of them, and each will need some metadata. As such as I was hoping to define it using varlen as:
#[define_varlen]
pub(crate) struct A {
	header: u16,
	// other_header: u8,
	#[varlen]
	data: Array<u32, u8>,
}

However, this seems to result in header (and other_header) in a 32 bit block, then the length of in a separate 32-bit block. I would have assumed that only the first 32-bit block would be required, and that the following test case would succeed.
let x: VBox<A> = VBox::new(a::Init {
	header: 5u16,
	// other_header: 2u8,
	data: Array::try_clone_from_slice(&[1u32]).unwrap(),
});
assert_eq!(
	x.calculate_layout().size,
	size_of::<u32>() + // headers + (maybe alignment) + size
		x.refs().data.len() * size_of::<u32>() // array elements
);

As an alternative, I though I could use the macros to define the type instead to see if that would work:
#[define_varlen]
pub(crate) struct A {
	header: u16,
	// other_header: u8,
	#[controls_layout]
	len: u8,
	#[varlen_array]
	data: [u32; *len],
}

However, this fails because of the type of len:
error[E0308]: mismatched types
  --> src/a.rs:70:14
   |
63 | #[define_varlen]
   | ---------------- expected `usize` because of return type
...
70 |     data: [u32; *len],
   |                 ^^^^ expected `usize`, found `u8`

Any help with a solution would be appreciated

Explaining this scenario:
#[define_varlen]
pub(crate) struct A {
	header: u16,
	// other_header: u8,
	#[varlen]
	data: Array<u32, u8>,
}

In this case Array<u32, u8> has its own alignment (4 bytes) and size (((len+1)*4) bytes), both of which are independent of the struct that the Aray<u32, u8> is embedded into. This is necessary so that you can pass around an &Array<u32, u8> and it means the same thing regardless of whether it came from struct A or struct B. Unfortunately this wastes some space on padding, similar to how struct boundaries in Rust can also cause some wasted space due to padding.
You're going in the right direction with your second approach. You just need to fix the type error: [u32; *len] needs the length to be of type usize, but in your case it's of type u8. The length can be an arbitrary const-evaluatable expression, so the the solution is simple: just add the cast. The following should work, I believe:
#[define_varlen]
pub(crate) struct A {
	header: u16,
	// other_header: u8,
	#[controls_layout]
	len: u8,
	#[varlen_array]
	data: [u32; *len as u8],
}

Thank you for the explanation. That does indeed solve my problem. I just have to fix the cast to go to usize in your fragment:
	data: [u32; *len as usize],

I had somehow assumed that the #[define_varlen] and #[varlen] would (or at least be able to) split the length from the data, but your explanation about being able to pass (references) around makes a lot of sense.
Maybe you could add some warnings when creating Array types where the Len type is smaller than the element type. It that constructing these combination would never make sense from a memory saving perspective.
Maybe the use-case where the length is packed with other data would also be a good (additional) example for the documentation.
Thanks for the help and for the great crate. It is exactly what I needed!


getOptimizerName convert function to property

Convert function to property
public abstract fun getOptimizerName(): String
=>
public abstract val optimizerName: String
in Optimimizer.kt

@quickstep24 agree, with this change to be more Kotlin-idiomatic. If you want, you could create a PR for this and same cases

I would like to do this, but I have a question: Should this be done while keeping and deprecating the existing getXyz methods? In my own projects I only start to use the proper deprecation cycle at 1.0.0 and beyond, but there are many different opinions about that ^^

I would like to do this, but I have a question: Should this be done while keeping and deprecating the existing getXyz methods? In my own projects I only start to use the proper deprecation cycle at 1.0.0 and beyond, but there are many different opinions about that ^^

And the next question: Structural search revealed things like:
https://github.com/JetBrains/KotlinDL/blob/b1c89e5cc1ce0c873b06e183d15d28c1f819a41c/api/src/main/kotlin/org/jetbrains/kotlinx/dl/api/core/layer/Layer.kt#L82-L84
Judged by the name alone, this is also a getter which could be a val property instead, but the implementations do some considerable work there. Should I leave them as they are, or convert them too? Personally, I tend to shy away from method names that look like getters/setters, because getters/setters/properties often look like very cheap operations and this might confuse people. @zaleslaw What is your take on this?

And the next question: Structural search revealed things like:
https://github.com/JetBrains/KotlinDL/blob/b1c89e5cc1ce0c873b06e183d15d28c1f819a41c/api/src/main/kotlin/org/jetbrains/kotlinx/dl/api/core/layer/Layer.kt#L82-L84
Judged by the name alone, this is also a getter which could be a val property instead, but the implementations do some considerable work there. Should I leave them as they are, or convert them too? Personally, I tend to shy away from method names that look like getters/setters, because getters/setters/properties often look like very cheap operations and this might confuse people. @zaleslaw What is your take on this?

@LostMekka no worry about deprecation, feel free to remove methods and convert to properties. But I'm not sure about getWeigths - the idea was to have an abstract method that should be overridden in each layer and sometimes it has a very large body. Maybe you have any idea about renaming this method?

@LostMekka no worry about deprecation, feel free to remove methods and convert to properties. But I'm not sure about getWeigths - the idea was to have an abstract method that should be overridden in each layer and sometimes it has a very large body. Maybe you have any idea about renaming this method?

I'm not sure there either. How about fetchWeights, readWeights, or maybe extractWeights? I don't feel very competent in this particular domain, so I don't know which word would fit best. 😅

I'm not sure there either. How about fetchWeights, readWeights, or maybe extractWeights? I don't feel very competent in this particular domain, so I don't know which word would fit best. 😅


drivers: entropy: mcux_trng: fix error after reset

There are situations after certain types of resets of the
board, that the underlying TRNG IP reports an error when
making a request for TRNG_GetRandomData(). The function
will clear the error bit but doesn't try to retrieve the
entropy data so this change will retry once on the error.
The observation is that the error only occurs once and
since this function is likely only called a couple of
times to provide a seed to RNG functions there is little
impact.
Fixes #37619
Signed-off-by: David Leach<EMAIL_ADDRESS>

Turns out the fix is also in the SDK 2.10 driver which is an active PR


[dart2wasm] Try/catch in an async function doesn't emit catch_all for JS exceptions

If the catch guard in a try/catch block is general enough to cover JS exceptions, the lowering to wasm needs to include a catch_all block, as that's the only way to catch JS exceptions (they have no exception tag). This works in synchronous functions. For example, in the following code:
import 'dart:js_interop';

@JS('window.navigator.userAgent')
external JSString get userAgent;

void main() {
    String? userAgentString = null;
    try {
        userAgentString = userAgent.toDart;
    } catch (e) {
    }
    print('userAgent: $userAgentString');
}

The try/catch block lowers to:
try $label0
  call $userAgent
  call $JSStringToString|get#toDart
  local.set $var0
  br $label0
catch $tag0
  local.set $var2
  local.set $var1
  block
    local.get $var1
    local.set $var3
    local.get $var2
    local.set $var4
    local.get $var3
    drop
    local.get $var4
    drop
    br $label0
  end
  rethrow $label0
catch_all
  call $StackTrace.current
  local.set $var2
  call $new _JavaScriptError._
  local.set $var1
  block
    local.get $var1
    local.set $var5
    local.get $var2
    local.set $var6
    local.get $var5
    drop
    local.get $var6
    drop
    br $label0
  end
  rethrow $label0
end $label0

However, if this is in an async context instead, the catch_all block is not actually emitted. For example, in the same code, but in with an async main function:
import 'dart:js_interop';

@JS('window.navigator.userAgent')
external JSString get userAgent;

Future<void> main() async {
    String? userAgentString = null;
    try {
        userAgentString = userAgent.toDart;
    } catch (e) {
    }
    print('userAgent: $userAgentString');
}

The try/catch block lowers to:
try
  local.get $var6
  call $userAgent
  call $JSStringToString|get#toDart
  struct.set $<context file:///Users/jacksongardner/Desktop/scratch/assert_text.dart:6:18> $field0
  br $label2
catch $tag0
  local.set $var7
  local.set $var8
  local.get $var0
  local.get $var8
  struct.set $_AsyncSuspendState $field6
  local.get $var0
  local.get $var7
  struct.set $_AsyncSuspendState $field7
  br $label1
end

Notice the missing catch_all block. As a result, JS exceptions escape this try/catch and cause failures.
The specific customer who this is affecting is the flutter firebase packages, which have this try/catch here: https://github.com/firebase/flutterfire/blob/213608cf7d889aeb60084ef5fb74a1e74178d7ba/packages/firebase_core/firebase_core_web/lib/src/firebase_core_web.dart#L254 which normally allows them to continue and properly initialize the firebase app.

I think probably the issue is that the current mechanisms for async try/catch of JS exceptions actually do work if the exception is thrown asynchronously, because it looks like in the async context we do catch JS exceptions and probably convert them to a dart exception: https://github.com/dart-lang/sdk/blob/8af6f94b61292c25a957ee1814501d0e82b39f73/pkg/dart2wasm/lib/async.dart#L793
So the try/catch lowering assumes that it has already been converted to a dart exception with the $tag0 tag. However, this doesn't actually work when the exception is thrown synchronously, because the try/catch lowering is responsible for directly catching the thrown exception (rather than something transformed by the async error handling context)

Related? https://github.com/dart-lang/sdk/issues/55466

We need the try_catch blocks we have in the sync code generator in try-finally and try-catch compilers in async code generator as well.
Test file:
import 'dart:js_interop';

@JS()
external void eval(String code);

@JS()
external void throwFromJS();

void defineFunctions() {
  eval(r'''
    globalThis.throwFromJS = function() {
      throw 'exception from JS';
    };
  ''');
}

Future<void> catchInAsync() async {
  try {
    throwFromJS();
  } catch (e) {
    print("Error caugh in async function");
  } finally {
    print("Finally block in async function");
  }
}

void catchInSync() {
  try {
    throwFromJS();
  } catch (e) {
    print("Error caugh in sync function");
  } finally {
    print("Finally block in sync function");
  }
}

Future<void> main() async {
  await catchInAsync();
  catchInSync();
}

catchInSync works as expected, but catchInAsync crashes as the JS exception isn't caught.

I think probably the issue is that the current mechanisms for async try/catch of JS exceptions actually do work if the exception is thrown asynchronously

This is right, if the JS exception is caught in an await it's handled. Example:
import 'dart:js_interop';

@JS()
external void eval(String code);

@JS()
external void throwFromJS();

void defineFunctions() {
  eval(r'''
    globalThis.throwFromJS = function() {
      throw 'exception from JS';
    };
  ''');
}

Future<void> catchInAsync() async {
  try {
    throwFromJS();
  } catch (e) {
    print("Error caugh in async function");
  } finally {
    print("Finally block in async function");
  }
}

Future<void> main() async {
  try {
    await catchInAsync();
  } catch (e) {
    print("Exception caught in main");
  }
}

Here main catches the JS exception, but catchInAsync doesn't (because of this bug).
The fix should just be a matter of adding catch_all blocks in try-finally and try-catch blocks.
We can actually avoid catch_all blocks when a call is known to not make JS calls because the only reason why we have catch_all is to catch JS exceptions. I think we probably doesn't have this information today. I'll file an issue for this for now.

One other possibility that has occurred to me is to actually wrap every JS interop entry point with a try/catch in JavaScript that converts any JS exception to a dart exception right at the JS/Wasm boundary. That would theoretically allow us to eliminate the catch_all blocks from all the dart2wasm code completely, since it can assume that any exceptions flowing through the wasm code will already be converted to a dart exception. This also would allow us to get more detailed/complete information about the JS exceptions, since we can actually introspect info about the JS exception (such as stack trace, exception type, message, etc) in a JS catch block that is not accessible with a catch_all instruction in wasm.
On the other hand, exceptions flowing through an export that comes from a @Native annotation might disobey this assumption. Also the performance overhead of wrapping all JS interop calls with a try/catch block is unclear. JIT may be fairly smart at eliminating the overhead for calls that can't actually throw.

Fix in progress in https://dart-review.googlesource.com/c/sdk/+/363000.

This also would allow us to get more detailed/complete information about the JS exceptions, since we can actually introspect info about the JS exception (such as stack trace, exception type, message, etc)

I wonder if we can get this information by using catch_all_ref instead of catch_all, and implementing JS functions to get the stack trace (and other info) from passed exception references.

That is also a possibility, but catch_all_ref is fairly recent and browser support is limited. Whereas catching the JS exception in JS would be supported by all browsers today.


Fixed link to samples

It looks like the link to spec which should be used as example of API usage is broken.
I've fixed on.

Thank you :)


initializer.rb missing in 0.7.5?

Looking at
https://github.com/NoamB/sorcery/tree/master/lib/sorcery/railties
There doesn't seem to be an initializer.rb file, whereas in 0.7.0 there is.
Not sure if this is auto generated or not but rake sorcery:bootstrap fails in 0.7.5 without it.

Found the 0.7.5 process is different to the railscasts tutorial.  Working with the new commands.


Implement the Why block

Description
Implement the Why block as designed in #114. Note that this task doesn't account for the scroll animation.
Acceptance Criteria

[ ] Implement the why block for desktop
[ ] Implement the why block for tablet
[ ] Implement the why block for phone
[ ] DIDs and Verifiable Credential should have a link to https://w3c-ccg.github.io/did-spec/ and https://www.w3.org/TR/verifiable-claims-data-model/ respectively

To be defined

[ ] Have a link for DIDs and Verifiable Credentials spec pages, needs design

@andreforsousa can you review the "To be defined" and provide us with a answers for these points?


Domain name Donation/Sponsorship

I offer to provide the domain https://algorithms.codes for use by the organization and I will bear the costs for its registration (including the SSL certs, if needed).
The individual programming language sites can be hosted:

https://c.algorithms.codes
https://cpp.algorithms.codes
https://python.algorithms.codes
https://go.algorithms.codes
https://java.algorithms.codes
etc

That's really great man @kvedala

It looks like the new algorithms website is at https://the-algorithms.com. Who's sponsoring it? What's the status of this issue?

At the bottom of the page, it says that Varcel is sponsoring it.  Are they paying money to TheAlgorithms for use of our content and logo? @anupkumarpanwar

it says that Vercel is sponsoring it

https://github.com/TheAlgorithms/TheAlgorithms.github.io/pull/25#issuecomment-787775124

So Vercel is sponsoring hosting? Is it sponsoring the domain name as well?