Kernel crashes on grids generation for large countries at zoom_level = 18

📓  Context
Below I wrote down a series of assumptions and tests I did to see if it would fix the problem leading me to concluding that morecantile seems to be the culprit in the grids generation issues we've been facing.
Reiterating that the initial goal was to generate bingtile zoom level 18 grids for Indonesia to match the resolution of the other data marts.
🧪 Assumptions and tests
1. Indonesia is very large for the usual grid generation workflow.

Kernel keeps shutting down for all admin levels.
Tried different administrative levels. Tested from adm0 to adm3 and all failed.
Memory reaches 99% and then shuts down. Experiencing memory errors.

2. Since even at the lowest level the kernel dies, perhaps the geometries of the regions are too complicated adding to the complexity of generating grids.

Simplify geometries at 20m.
Tried this for adm2 and adm3 --- both still failed.
Simplify even further to reduce geometry complexities.

Generate bounding boxes all over the adm2 regions.
Generate intermediate bounding boxes (bingtile = 16) all over the adm2 bounding boxes for the whole indonesia. (THIS WORKED 🟢 )
Generate the high res bingtiles over the intermediate grids for each adm2 area. (THIS FAILED 🔴)

3. The jump in sizes from a small adm2 area might be causing out of memory issues since there are areas that could be too large to process.

Chunk the intermediate grids for a constant size of geometries to be inputted to the grid generation function.
Tried 150k, 100k, 60k, 40k, 15k, and 5k chunk sizes, but all failed.
“Silver lining” is that this was the longest run. It always ran for 40-50 mins. no matter the chunk size threshold.

4. Overhead / some memory retention is happening within geowrangler code because simplification and chunking at varied sizes does not solve the problem.

Added psutil functions to track memory at each loop during the chunking. Tested this for chunk_size of 15,000 intermediate grids at each iteration.
Chunk 4

Memory usage after grid generation chunk 4: 9473.66 MB
Memory usage after saving file chunk 4: 9576.93 MB
Memory usage after cleanup chunk 4: 8964.80 MB

Chunk 6

Memory usage after grid generation chunk 6: 10835.73 MB
Memory usage after saving file chunk 6: 10966.70 MB
Memory usage after cleanup chunk 6: 10473.70 MB

5. Kyle’s fix for the PH unfiltered grids for generate_grid function works for the whole PH at adm0.

Failed. Tried this for the whole intermediate grids and for adm0.
Reason for using the whole dataframe as in input to grid generation is to see where the function will stop in the pipeline
FINDINGS: Kernel crashes at Get all tiles for polygons section (related to next)

7.  Tile retrieval is causing an additional processing overhead due to list comprehensions

First test: Convert list comprehension to a generator object that yields each value instead of storing to a list and see if it goes past the function. Reason is the list might be causing the memory retention issues.

FALSE. Already a generator object.

No more optimizations to do on this function. At this point it seems to be related to morecantile as Butch has mentioned.

8. Inspect Morecantile

Open item, have not done a deep-dived yet on the package itself. Will do other optimizations at a lower level instead (currently ongoing).

@tm-abby-moreno FastBingTileGenerator is now merged to master! 🔥 See sample usage here.
Thanks so much for your preliminary investigation that uncovered the bottlenecks. It cascaded into the development of the faster grinding approach 😄


OpenSSL include issues

I recently updated my system, and have a new Node 17.1.0
I had already done some of the porting to the new SSL version in 17, so I have OPENSSL_MIN_API defined, previously =30000 was good, but now, I get an error
/home/d3x0r/.cache/node-gyp/17.1.0/include/node/openssl/macros.h:139:4: error: #error "The requested API level higher than the configured API compatibility level"

It looks like the headers should be configured with 30000, but the symbol for OPENSSL_CONFIGURED_API seems to be 0 (both in node-gyp and cmake-js).
this is the addon I'm trying to build https://github.com/d3x0r/sack.vfs
Is there some openssl.config.h I'm supposed to be including now that isn't otherwise just naturally included?
gyp info it worked if it ends with ok
gyp info using<EMAIL_ADDRESS>gyp info using<EMAIL_ADDRESS>| linux | x64
gyp info spawn make
gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]
make: Entering directory '/home/d3x0r/work/javascript/sack.vfs/build'
  CXX(target) Release/obj.target/sack_vfs/src/vfs_module.o
In file included from /home/d3x0r/.cache/node-gyp/17.1.0/include/node/openssl/././archs/linux-x86_64/asm/include/openssl/safestack.h:19,
                 from /home/d3x0r/.cache/node-gyp/17.1.0/include/node/openssl/./safestack_asm.h:11,
                 from /home/d3x0r/.cache/node-gyp/17.1.0/include/node/openssl/safestack.h:4,
                 from ../src/global.h:52,
                 from ../src/vfs_module.cc:6:

/home/d3x0r/.cache/node-gyp/17.1.0/include/node/openssl/macros.h:139:4: error: #error "The requested API level higher than the configured API compatibility level"  (for API > 10000 )

/home/d3x0r/.cache/node-gyp/17.1.0/include/node/openssl/macros.h:155:4: error: #error "OPENSSL_API_COMPAT expresses an impossible API compatibility level"
  155 | #  error "OPENSSL_API_COMPAT expresses an impossible API compatibility level"
      |    ^~~~~  (for API=0)

cc1plus: note: unrecognized command-line option ‘-Wno-unknown-warning’ may have been intended to silence earlier diagnostics
make: *** [sack_vfs.target.mk:215: Release/obj.target/sack_vfs/src/vfs_module.o] Error 1
make: Leaving directory '/home/d3x0r/work/javascript/sack.vfs/build'
gyp ERR! build error
gyp ERR! stack Error: `make` failed with exit code: 2
gyp ERR! stack     at ChildProcess.onExit (/home/d3x0r/.npm/lib/node_modules/node-gyp/lib/build.js:194:23)
gyp ERR! stack     at ChildProcess.emit (node:events:390:28)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (node:internal/child_process:290:12)
gyp ERR! System Linux 5.15.6-arch2-1
gyp ERR! command "/usr/bin/node" "/home/d3x0r/.npm/bin/node-gyp" "build" "-D" "--version"
gyp ERR! cwd /home/d3x0r/work/javascript/sack.vfs
gyp ERR! node -v v17.1.0
gyp ERR! node-gyp -v v8.4.1
gyp ERR! not ok

This system is Arch too ; is there maybe a incompatibilty between what they build and what you build for released openssl/* headers from node?
Once I worked around the above errors so it would build then the symbol SSL_get_peer_certificate is getting redefined to SSL_get1_peer_certificate, and this symbol does not exist, but there doesn't seem to be any other way to not get that symbol redefined with get1 in the headers distributed to compile against.

It doesn't really appear to be node-gyp at fault...

@d3x0r What was the issue & how did you fix it?

https://github.com/nodejs/node/issues/40575  this was a similar issue that was in node...
https://github.com/nodejs/node/issues/40575#issuecomment-1030524881   this is the summary of the workaround.


Question: Does this work on server-side only?

Same as title.

Same as title.

No, it adds new blocks and is needed on both client and server

Ah I see. Thank you.
On Sun, May 7, 2023 at 6:01 PM dhyces @.***> wrote:

Same as title.
No, it adds new blocks and is needed on both client and server
—
Reply to this email directly, view it on GitHub
https://github.com/dhyces/waxable-coral/issues/1#issuecomment-1537549957,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ANWBAIJYT2R5UYYREMMXUSTXFALT7ANCNFSM6AAAAAAXZDMUTU
.
You are receiving this because you authored the thread.Message ID:
@.***>


Added Icons8

It's free for open source and free for smaller/personal/learning projects. There's a paid tier for enterprise.
The terms are similar to Noun Project (which you accepted).
Of course, it up to you; if not, just say no and I'll understand :)

Thanks!

I created a repo with os icons from icons8, you should remember to add a note like
OS icons provided by icons8.


Cannot support multi-dimension arrays: GameStudio crash

Release Type: GitHub
Version: v3.1
Platform(s): Windows 10 x64
Describe the bug
Trying to use a script with a 3 dimension array as a public member crashes GameStudio. Might even crash on an array of size 2.
To Reproduce
Steps to reproduce the behavior:

Make a script with a public (custom class?) variable with 2+ dimensions
Try and attach that script to an Entity in GameStudio

Log and callstacks
Exception: ArgumentException: Cannot support dimension [3] for type [GentrieveX.Scripts.WorldGen.ROOM_CELL_TYPE[,,]]. Only supporting dimension of 1
at Xenko.Core.Reflection.ArrayDescriptor..ctor(ITypeDescriptorFactory factory, Type type, Boolean emitDefaultValues, IMemberNamingConvention namingConvention)
at Xenko.Core.Reflection.TypeDescriptorFactory.Create(Type type)
at Xenko.Core.Reflection.TypeDescriptorFactory.Find(Type type)
at Xenko.Core.Reflection.ObjectDescriptor.b__45_4(FieldInfo fieldInfo)
at System.Linq.Enumerable.WhereSelectArrayIterator2.MoveNext()    at System.Linq.Enumerable.WhereEnumerableIterator1.MoveNext()
at System.Collections.Generic.List1.InsertRange(Int32 index, IEnumerable1 collection)
at Xenko.Core.Reflection.ObjectDescriptor.PrepareMembers()
at Xenko.Core.Reflection.ObjectDescriptor.Initialize(IComparer1 keyComparer)    at Xenko.Core.Reflection.TypeDescriptorFactory.Find(Type type)    at Xenko.Core.Reflection.ObjectDescriptor.<PrepareMembers>b__45_4(FieldInfo fieldInfo)    at System.Linq.Enumerable.WhereSelectArrayIterator2.MoveNext()
at System.Linq.Enumerable.WhereEnumerableIterator1.MoveNext()    at System.Collections.Generic.List1.InsertRange(Int32 index, IEnumerable1 collection)    at Xenko.Core.Reflection.ObjectDescriptor.PrepareMembers()    at Xenko.Core.Reflection.ObjectDescriptor.Initialize(IComparer1 keyComparer)
at Xenko.Core.Reflection.TypeDescriptorFactory.Find(Type type)
at Xenko.Core.Reflection.ObjectDescriptor.b__45_4(FieldInfo fieldInfo)
at System.Linq.Enumerable.WhereSelectArrayIterator2.MoveNext()    at System.Linq.Enumerable.WhereEnumerableIterator1.MoveNext()
at System.Collections.Generic.List1.InsertRange(Int32 index, IEnumerable1 collection)
at Xenko.Core.Reflection.ObjectDescriptor.PrepareMembers()
at Xenko.Core.Reflection.ObjectDescriptor.Initialize(IComparer`1 keyComparer)
at Xenko.Core.Reflection.TypeDescriptorFactory.Find(Type type)
at Xenko.Core.Reflection.DataVisitorBase.VisitCollection(IEnumerable collection, CollectionDescriptor descriptor)
at Xenko.Core.Assets.CollectionIdGenerator.VisitCollection(IEnumerable collection, CollectionDescriptor descriptor)
at Xenko.Core.Assets.CollectionIdGenerator.VisitObject(Object obj, ObjectDescriptor descriptor, Boolean visitMembers)
at Xenko.Core.Reflection.DataVisitorBase.Visit(Object obj, ITypeDescriptor descriptor)
at Xenko.Core.Assets.CollectionIdGenerator.VisitObjectMember(Object container, ObjectDescriptor containerDescriptor, IMemberDescriptor member, Object value)
at Xenko.Core.Reflection.DataVisitorBase.VisitObject(Object obj, ObjectDescriptor descriptor, Boolean visitMembers)
at Xenko.Core.Assets.CollectionIdGenerator.VisitObject(Object obj, ObjectDescriptor descriptor, Boolean visitMembers)
at Xenko.Core.Reflection.DataVisitorBase.Visit(Object obj, ITypeDescriptor descriptor)
at Xenko.Core.Reflection.DataVisitorBase.VisitCollection(IEnumerable collection, CollectionDescriptor descriptor)
at Xenko.Core.Assets.CollectionIdGenerator.VisitCollection(IEnumerable collection, CollectionDescriptor descriptor)
at Xenko.Core.Assets.CollectionIdGenerator.VisitObject(Object obj, ObjectDescriptor descriptor, Boolean visitMembers)
at Xenko.Core.Reflection.DataVisitorBase.Visit(Object obj, ITypeDescriptor descriptor)
at Xenko.Core.Assets.CollectionIdGenerator.VisitObjectMember(Object container, ObjectDescriptor containerDescriptor, IMemberDescriptor mem

If you try and add it without clicking & dragging, it just doesn't list it as part of the entity. It should still show the script attached, but skip showing the problematic variable in the inspector.

Multi-dimension array are not supported (and it would be hard to support them).

I don't care for them to be supported, I just don't want GameStudio to crash if it comes across one. Just skipping over that value in the inspector would work for me.
The callstack shows the specific type of array that caused the problem for me: [,,]

Crash suppressed & tested handled well in my fork: https://github.com/phr00t/xenko/commit/09493ae29d59036e3909698eafed88ed11aaa561

I don't think putting a try/catch around it and returning null is the best option. It is possible to check if it is a multi-dimension array by using type.GetArrayRank() != 1. There are also quite a few spots that would cause a NullReferenceException if ITypeDescriptorFactory.Find returned null. Not sure why you didn't get that problem though.


WIP: update to Forge 1.18.1

special support for Patchouli removed, therefore mixin removed
Forge version updated to 1.18.1-39.0.66
Java updated to JDK 17, CI updated accordingly
build.gradle etc. updated according to the new Forge MDK, this includes but is not limited to:

ForgeGradle updated to 5.1.+
Gradle itself (alongside Gradle wrapper) updated to 7.3

Mod version updated to 3.0.0

IMO update of Minecraft version should be a semver-breaking change, so I am updating major version here. I can revert it back if desired.

Things to be fixed

[ ] download URL should be updated
[ ] nullpinter mentioned that the download process calls for a redesign

我已经更新好了：https://github.com/CFPAOrg/I18nUpdateMod2/tree/1.18
但是目前 1.18 的翻译仓库还没做校对，有些是 1.16 的转过来的，存在比较多的问题


Add .govuk-hidden { display: none }

We need to be able to hide and show elements by toggling the class name.
.govuk-visually-hidden isn't always right as we may not want the thing to be perceivable to screen readers. And we'd rather not have to add aria-hidden="true" as well.
[hidden] { display: none; } is nice but not supported very well. Even in browsers that you can patch, you require the same amount of code. And in any case, IE8, for example, doesn't work because while the JS can change the attribute, the page won't update the CSS to reflect the change.

i think this should be an override, and as such .govuk-!-display-none
any thoughts?

Why do you think this, considering we have govuk-visually-hidden already?
On Fri, 9 Nov 2018, 09:32 Jani Kraner<EMAIL_ADDRESS>wrote:

i think this should be an override, and as such .govuk-!-display-none
any thoughts?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/alphagov/govuk-frontend/issues/1060#issuecomment-437302816,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AACRKy8MlCc_qgo2HyrmE_UGkmsXbWE9ks5utUvBgaJpZM4YU_5R
.

to me it feels more of an override than a utility, but it can honestly be either

We have a need for this too. I'm tempted to use the hidden attribute though, given that IE8 isn't that big a concern for us.

We have a use case for hiding an element on a page from the screen readers as well.
Could you give an estimate on when this issue might be actioned?

Will be released in this milestone soon: https://github.com/alphagov/govuk-frontend/milestone/27 thanks everyone :)


Weight-of-authority parentheticals

Weight-of-authority parentheticals (e.g., "per curiam," "en banc") are not rendered.

Fixed in commit 13a2c1f.


Add Developer Workflow Content

Replaced the placeholder text with content for the developer workflow page.

OK, I'll fix the typos tonight.
As for why a rebase, instead of a merge... Well, I based this document on a similar one that I use at Johnson Controls. So, the original idea to rebase instead of merge came from there. I think the motivation was to keep the history as clean, and linear as possible.  That's less of an issue for our smaller project, but unless you feel strongly about it, I'd like to keep it.

No, I don't feel strongly about it. Sounds like it's just a difference of
"Centralized Workflow" vs "Gitflow", which is what I'm more used to.
https://www.atlassian.com/git/tutorials/comparing-workflows
Centralized with rebasing is probably better here.
On Wed, May 31, 2017 at 12:28 PM, David Snyder<EMAIL_ADDRESS>wrote:

OK, I'll fix the typos tonight.
As for why a rebase, instead of a merge... Well, I based this document on
a similar one that I use at Johnson Controls. So, the original idea to
rebase instead of merge came from there. I think the motivation was to keep
the history as clean, and linear as possible.  That's less of an issue for our smaller project, but unless
you feel strongly about it, I'd like to keep it.
—
You are receiving this because your review was requested.
Reply to this email directly, view it on GitHub
https://github.com/KenoshaLUG/KenoshaLUG.github.io/pull/1#issuecomment-305259011,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AEDO4iF6J93a5NoWvNUa5YP8R7hfDRlOks5r_aM7gaJpZM4Nq66D
.

--
-Jameson
Omnipresent Garage Banjo
http://jamesonrobert.com


Add infinite scrolling  for tts

Describe your suggested feature
I am tired of manually changing chapter and then turn on tts again and again , can you please add feature so the tts will not stop and reading chapters continuously
Other details
No response
Acknowledgements

[X] I have searched the existing issues and this is a new ticket, NOT a duplicate or related to another open or closed issue.
[X] I have written a short but informative title.
[X] If this is an issue with a source, I should be opening an issue in the sources repository.
[X] I have updated the app to version 2.0.0.
[X] I will fill out all of the requested information in this form.

Duplicate of #759


Opening file from TOC much slower that files tree

Current behavior

Clicking to open a file from the TOC is slow
Selecting that same file from the files tree is fast

Expected behavior

Both should be fast

Fixed in next release


OsString cannot be serialized as a JSON map key

Playground example
As discussed in https://stackoverflow.com/q/51276896/113632, JSON only supports string keys, which OsString (technically) is not. Unlike a custom struct, however, it's not straightforward (or possible?) for a user to define a custom Serialize implementation for OsString or Map<OsString, ...> types to work around this issue.
One approach I could imagine is serializing OsString objects as Base64 (or another textual encoding), though I don't know whether it would be safe/acceptable to change its serialized form. Alternatively perhaps Serde could represent maps with non-string keys as an array of pairs in JSON.
serde-rs/json#465 is related but essentially the opposite problem, since PathBuf is serialized as a string.

I think I would prefer to keep the current Serialize impl on OsString, but if you need it you can serialize the OsString map keys in a different way like this:
use serde::Serialize;
use serde::ser::{SerializeMap, Serializer};
use std::collections::HashMap;
use std::ffi::OsString;

fn main() {
    let mut map = HashMap::new();
    map.insert(OsString::from("key"), true);

    let ser = serde_json::value::Serializer;
    let serialized = serialize_with_lossy_string_keys(&map, ser).unwrap();
    println!("{}", serialized);

    #[derive(Serialize)]
    struct Struct {
        #[serde(serialize_with = "serialize_with_lossy_string_keys")]
        map: HashMap<OsString, bool>,
    }

    let serialized = serde_json::to_string(&Struct { map }).unwrap();
    println!("{}", serialized);
}

fn serialize_with_lossy_string_keys<'a, M, T, S>(value: &'a M, serializer: S) -> Result<S::Ok, S::Error>
where
    &'a M: IntoIterator<Item = (&'a OsString, &'a T)>,
    T: 'a + Serialize,
    S: Serializer,
{
    let mut map = serializer.serialize_map(None)?;
    for (k, v) in value {
        map.serialize_entry(&k.to_string_lossy(), v)?;
    }
    map.end()
}


Updates create and repeat test run execution to match backend changes

Issue: Remove TestRunConfig #14
Backend PR counterpart: [Fix] remove TestRunConfig usage from TH Backend (#581) #6

Updates create test run to swap test_run_config_id with selected_tests

Updates repeat test run execution to use new repeat end point

Removes test run config API calls

@gvargas-csa can you review as well please ?

@gvargas-csa can you review as well please ?

Done


Signals are not handler properly

My term is completely broken after finishing application with SIGINT/SIGTERM (cursor and input characters are invisible, x axis positioning is broken too), I guess these signals should be handled to terminate resources properly.
I'm using Arch Linux and termite terminal emulator.

Yup, tui programs almost always need to install signal handlers.
Run stty before and after to find the diff.
Quick fix is reset.


Proxy converts HTTP POST to GET when the HUD is enabled

Describe the bug
If ZAP is used to proxy requests from another application to a web server, and the ZAP HUD is enabled, then when the application issues an HTTP POST request, it is forwarded to the web server as a GET request. Note that this does not affect HTTPS requests.
Steps to reproduce the behavior
Save the attachment as "post.html".
Open it in a web browser (I used Firefox 109.0.1).
Set the browser's proxy to the ZAP proxy at localhost:8080.
Use the ZAP toolbar button to enable the HUD.
Press the "post" button on the form.
The ZAP HTML history windows shows a new HTTP GET request to http://httpbin.org/post.
The browser displays "405 Method Not Allowed".
Expected behavior
ZAP should show a POST request to "http://httpbin.org/post" and the web browser should show the header and contents of the POST request. This behavior is observed when the ZAP HUD is disabled.
Software versions
Version: 2.12.0
Screenshots
No response
Errors from the zap.log file
No response
Additional context
No response
Would you like to help fix this issue?

[ ] Yes
post.html.txt

Moved to the HUD repo, most likely caused by the HTTPS upgrade done by the HUD.


Backup and restore database between RDS clusters

The first operation we will perform will be from the current "production" RDS cluster to the the scratch cluster.
I believe this can and maybe should be done via the console or aws command line since these operations are one-offs.
Docs here: https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_CommonTasks.BackupRestore.html

After some research and testing it would appear that the RDS concept of a cluster endpoint is what we need to get a few benefits:

A persistent db host that lives with the stack (scratch, etc). The cluster endpoint is its own resource, with its own lifecycle, and critically its own db host dns entry.
Db clusters can be swapped by binding cluster identifier to the ClusterEndpoint object in pulumi code. The current cluster can be stored in config.
Snapshotting, cloning and restoring can be done using all the Aurora tools built for that.
A stack can still be stood up backed by a brand new db cluster. This is desirable for testing out new flower releases for example.
But critically, existing stacks can have arbitrary (but still tracked in stack state) db clusters behind them.

The critical thing to know is that RDS restoring or cloning any RDS cluster creates a new cluster. In other words, you can't restore from one cluster into another, or from a point in time backup into an existing cluster. It's always a new cluster that you get. So the task is having a way to seamlessly switch between those items. Custom endpoints are the way to do that.
So the workflow for restoring from a backup is:

Create a snapshot
Restore the snapshot (creates new cluster)
Change the stack config value to bind to that new cluster's identifier

Because the db host is persistent, there's no need restart pods to get the new env variable. One caveat: we should use the same db creds between clusters.

It looks like this was a dup of #108.


Graphman copy indexing optimizations

To reduce the size of a DB shard without downtime, graphman copying deployments out is a good option. But it needs a few optimizations to be usable en masse, including on index creation. This track two of those optimizations:

Create the indexes based on the index creation DDL of the source, not based on the default.
Create the BTree indexes after the copy is done, and only the GiST afterwards.

@incrypto32 will mentor @itsjerryokolo on this task.

We can also apply this for grafts, and copy over custom statistics in addition to the indexes.

Here's my take on this:

In general, we should defer creating BTree attribute indexes on all subgraphs until the subgraph is synced. What we really want is to kick off creating those indexes so that they are ready when the subgraph is synced, but not much before then, but as a first step it's ok to create them when the subgraph becomes synced.
All other indexes (not BTree or not an attribute index) should be created when the subgraph is first deployed as creating them later can be very very slow with bad consequences for the rest of the system.
For copies and grafts, we do not want to create the indexes that we create by default for new subgraphs. Instead, we should look at the source of the graft/copy, and recreate all the indexes that actually exist on the source


Fix x0 and possible memory leak

In https://github.com/libprima/prima/issues/148 the ability of the user to specify x0 from the C interface was removed. This PR fixes that. I noted this at the time in a comment in https://github.com/libprima/prima/commit/488abdabbf86931dcf83ad71d0c5b30ba5644825 but I suppose you missed it.
In the issue you objected to setting result->x to x0, but there is no other reasonable way to provide x0 to the Fortran code. In the issue you mentioned that result->x was set to a reasonable value even though no solver was selected, but the correct solution to that would be to have prima_check_problem return a non-zero exit code when it doesn't select a solver, as I mentioned in the issue.
Separately, I discuss this in my comment on the commit linked above as well, the modifications from that issue led to a possible memory leak. If prima_init_result returned 0 but prima_check_problem did not, we would return to the user with a non-zero exit code and also with allocated memory that the user would be expected to free. Of course it's not appropriate to both return a non-zero exit code and expect the user to clean up after us. The easiest way to solve this problem is to just check the problem before initializing the result.

Thank you @nbelakovski . I see the problem. I was confused by the original comment.
I will merge it for the moment, but still have related changes to make.
Thanks.


Reenable Jenkins comment triggers

In response to Jenkins going down (https://github.com/apache/beam/issues/21824), we disabled most of our comment triggers.
Since then, we believe that we've gotten a handle on what is causing the issue and have mitigated it by disabling many bot comments from the ghprb plugin (see https://lists.apache.org/thread/x6mx3o1tkf1299ssw7l63y37lfpw1zmp for context).
This reenables all comment triggers in a single easily revertable PR to reenable this workflow.
Fixes #22168

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Choose reviewer(s) and mention them in a comment (R: @username).
[x] Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
[ ] Update CHANGES.md with noteworthy changes.
[x] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.
To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md
GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

LGTM. Thanks for looking into this!


[Bug] Black screen after surviving fall + strange soft-lock after diving/DBNO-ing

(Black screen) The screen stays black after falling and not dying [Godmode/Buddha] (i was specifically falling with the DBNO dive, don't know if that affects the results in any way.
(Soft-lock) stuck in place after diving to a slightly raised surface in front of me and attempting to roll instead of continuing to slide
I am using the reanimated beatrun animations, not the default ones. I don't have any addons that would affect beatrun.
https://github.com/JonnyBro/beatrun/assets/143871014/11c6370a-6875-4e6e-8f2e-4c62e3e9ac03

I don't have any addons that would affect beatrun.

Are you sure? I saw on the video that you have addons beside beatrun

yeah i have addons besides beatrun, but 99% of them are all weapons, viewbob, model/texture replacers, vmanip stuff, etc, etc.
the reason im not worried about viewbob is this addon which adds a layered system to viewbob/calcview in gmod, making them not replace eachother, but stack on instead
the ONE addon i think even has a possibility of causing this is this one which allows the player to walk and jump off of props correctly, instead of all glitchy-like. ill try to disable it and do the glitch again, and ill get back to you on the results (along with disabling all my mods besides beatrun if it doesnt fix it)

ok so the soft-lock this stopped happening??? i didnt change my addons and went on and did it with the same addons as a test and it just didnt happen anymore???
The black screen after not dying to a fall still happens of course. this is a problem with beatrun, i remember it happening back when i had EA to the original beatrun (i even tried fixing it myself in the code just by getting rid of the black screen effect in the files, but i just dont know where its located)
im pretty sure the black screen after not dying to a fall is just because it doesnt check if you died from falling, it just checks that you WOULD have died, and that you hit the ground, so if you find the code hopefully its not too hard to fix

what a nice reshade

dude black screen after dying in godmode thats just part of the addon when you die from a height you can't survive that effect triggers just equip runner hands and press r.

dude black screen after dying that's just part of the addon when you die from a height you can't survive that effect triggers just equip runner hands and press r.


Azure servers spinning forever if Create Firewall Dialog is opened and closed

Azure Data Studio Version: master

Steps to Reproduce:

Open a SQL Server with Firewall Rule in the Azure Extension.
The firewall rule dialog opens.
Press Cancel.

Expected - Nothing happens
Actual - The server node is spinning forever.

Backlog'd for now

This is now fixed.


Consider using XMLHttpRequest.timeout

A synchronous request can "hang" some browsers if the connection is timing out. You should be able to set a timeout so that the request won't hang (although I admit I haven't tried it in beforeunload/unload events!).
You mostly need your fallback for IE and Safari:
IE: timeout property is supported since at least IE8 - see https://msdn.microsoft.com/en-us/library/cc304105(v=vs.85).aspx
Safari: timeout property is supported since Safari 6.1 - see Note 2 and Show all for http://caniuse.com/#search=XMLHttpRequest advanced features
Cheers

@MorrisJohns what are your thoughts on how the timeout should be set?

Comments about using synchronous XHR say that it delays the browser from loading the next page.  I wouldn't want to go much over that (since if you are hitting the limit, you are most likely just affecting navigation within your own site)

1 second seems too long in most cases, I suggest use a short number like 200ms by default.

Happy to merge in a PR if someone wants to take care of the timeout

Hi, curious if a PR for this issue is still desired/welcome. I'm thinking a default of something like 200ms as @hax suggested (or maybe 250?), which can be overridden by setting a number on the object itself, something like:
navigator.sendBeacon.timeout = 500 // let it wait longer
or, perhaps we don't like to add a public property since this is a polyfill. Maybe instead we support a global variable on the navigator object, something like this or similar:
navigator._sendBeaconTimeout = 500

setting xhr.timeout throws error Timeouts cannot be set for synchronous requests made from a document. so what I did is implement setTimeout and call xhr.abort if request is not yet complete.
sendBeacon(url, data, {timeout: 250}); // ms

Will this actually work?  If the request is synchronous, how would we ever reach the block of code that aborts the XHR?

I too was looking for a timeout in the fallback solution here for Safari.  Because the synchronous XHR effectively blocks the event loop, xhr.abort() is invoked after the response is received regardless of the setTimeout.
https://jsfiddle.net/e7ovu5to/4/
Anyone solve this?

@schmod @artz ah good point. Looks like it might not be possible then

Closing this issue since it looks like it might not be possible to add a timeout to a synchronous call.


Issues importing singles sections

When trying to run an import containing new sections I run into the following error:
Error: Class 'Twig_Loader_String'  in app/services/TemplatesService.php:169
Stack trace:
#0 /app/services/TemplatesService.php(363): Craft\TemplatesService->getTwig('Twig_Loader_Str...', Array)
#1 /app/services/EntriesService.php(157): Craft\TemplatesService->renderObjectTemplate('{section.name|r...', Object(Craft\EntryModel))
#2 /app/services/SectionsService.php(663): Craft\EntriesService->saveEntry(Object(Craft\EntryModel))
#3 /vendor/itmundi/schematic/src/Services/Sections.php(269): Craft\SectionsService->saveSection(Object(Craft\SectionModel))
I've checked the trace, and it appears the issue stems from Craft having a hardcoded setting for titleFormat = false when creating the single entry type. It doesn't matter what is in the actual schematic export, when saving the first entry, craft will always generate a title. I've fixed it by setting this property to true (SectionsService line 576) but this isn't a maintainable solution of course.
Any ideas?

@FreekVR Format of single title seems more like a craft issue to me. We don't want to disturb craft default functionality too much.
Error might also be related to autoloading of craft vendor packages like mentioned in #102

See #102, fixed. Thanks :)


Documentation

Hi,
Thanks for this package. Can you please provide link to any documentation or manual?
Thanks again!
Vivek

Ya, that's will be good, even if it wiil just simple example how to use it)

+1 👍
Working on it as you can see from: aesthetics: /doc.
I will fix and update this thread after deploying this on RTH.io.


Typo "Jupyter notebook"→"Jupyter Notebook"

https://docs.microsoft.com/en-us/azure/machine-learning/how-to-set-up-vs-code-remote

@hyoshioka0128 : Thanks for your contribution! The author(s) have been notified to review your proposed change.

Note that the product name is Jupyter Notebook. Referring to multiple instances of the product would seem to be "Jupyter Notebook files" or similar. Although the organization refers both to "a Jupyter Notebook," "Jupyter notebooks," and "Jupyter Notebooks" on its website.

#sign-off


Invoice Design Can Be Enhanced

Bagisto Version: Master
Steps To reproduce--
1. Log in as a customer.
2. Click on orders inside the profile section.
3. View any order.
4. Click on the invoice tab.
5. Click on the print button
6. Open the downloaded pdf
Expected result: The Invoice logo should be in the center of the page if the user is downloading the invoice
Actual result: The Invoice logo is not in the center of the page if the user is downloading the invoice. Please find the screenshot Here

No need for this issue, the existing design looks fine.

Considering this as an enhancement as is not breaking anything.

@jitendra-webkul
Now, this issue is fixed. so I am closing this issue.
And I have attached a screenshot Here, for future reference.


DistributedDataParallel Not Working

Hi, I've been stuck on this for days, so I decided to make an issue.
When I run DistributedDataParallel with the PyTorch launch module, I see that one machine will start training without waiting for the other one to start; this is different than if I run it without the launch module. WIthout the launch module, I am also letting one process have access to multiple GPUs, rather than just one.
I'm following your implementation of DistributedDataParallel in your Medium article: https://medium.com/huggingface/training-larger-batches-practical-tips-on-1-gpu-multi-gpu-distributed-setups-ec88c3e51255

Hi @moinnadeem,
What is the hardware you are using and what is the exact command you are using to run DistributedDataParallel with the PyTorch launch module?


breadcrumb authoring changes

Please always provide the GitHub issue(s) your PR is for, as well as test URLs where your change can be observed (before and after):
Fix # No-JIRA
Test URLs:

Before: https://main--world-bank--aemsites.hlx.live/
After: https://worldbank-breadcrumb-authoring-changes--world-bank--aemsites.hlx.live/

@akki761 is this change needed? if yes, resolve conflict and merge PR


Periodically run IAP endpoint ready test against auto deployments

Follow on to #42
We should setup a periodic test that runs the IAP endpoint is ready test against the auto deployments.

Needed to grant workload identity the serviceAccountTokenCreator
gcloud iam service-accounts add-iam-policy-binding --project=kubeflow-ci  --role roles/iam.serviceAccountTokenCreator   --member "serviceAccount:kubeflow-ci.svc.id.goog[kf-ci/kf-ci]"<EMAIL_ADDRESS>

The endpoit_ready_test.py is still failing when using workload identity with error
google.auth.exceptions.TransportError: Error calling the IAM signBytes API: b'{\n  "error": {\n    "code": 403,\n    "message": "Permission iam.serviceAccounts.signBlob is required to perform this operation on service account projects/-/serviceAccounts/kubeflow-testing@kubeflow-ci.iam.gserviceaccount.com.",\n    "status": "PERMISSION_DENIED"\n  }\n}\n'

Latest test still failed
https://prow.k8s.io/view/gcs/kubernetes-jenkins/logs/kubeflow-gcp-blueprints-master-periodic/1277776211300323329
https://kf-ci-v1.endpoints.kubeflow-ci.cloud.goog/tekton/#/namespaces/kf-ci/pipelineruns/gcp-kf-ready-6vlzk
The task executed command:
args:
  - '-m'
  - kubeflow.testing.get_kf_testing_cluster
  - '--base=$(inputs.params.testing-cluster-pattern)'
  - '--location=$(inputs.params.testing-cluster-location)'
  - get-credentials

It looks like its not running the new version
https://github.com/kubeflow/testing/blob/master/acm-repos/kf-ci-v1/namespaces/kf-ci/tekton.dev_v1alpha1_task_iap-ready.yaml#L40

Config management isn't syncing master.
   git:
      policyDir: /acm-repos/kf-ci-v1
      proxy: {}
      secretType: none
      syncBranch: gcp_blueprints
      syncRepo: https://github.com/jlewi/testing.git

Nomos sync is still failing
nomos --contexts=kf-ci-v1 status
Connecting to clusters...
Current   Context    Status           Last Synced Token   Sync Branch
-------   -------    ------           -----------------   -----------
          kf-ci-v1   ERROR            ccefc51a            master

Config Management Errors:
kf-ci-v1   KNV2010: unable to update resource: could not patch: Internal error occurred: failed calling webhook "pilot.validation.istio.io": Post https://istio-galley.istio-system.svc:443/admitpilot?timeout=30s: no endpoints available for service "istio-galley"

source: namespaces/auto-deploy/networking.istio.io_v1alpha3_virtualservice_auto-deploy-server.yaml
namespace: auto-deploy
metadata.name: auto-deploy-server
group: networking.istio.io
version: v1alpha3
kind: VirtualService

For more information, see https://cloud.google.com/anthos-config-management/docs/reference/errors#knv2010

Looks like its: kubeflow/testing#708

It should be sync'd now
nomos --contexts=kf-ci-v1 status
Connecting to clusters...

INFO|2020-07-01T01:31:51|/usr/local/lib/python3.8/dist-packages/oauth2client/transport.py|157| Attempting refresh to obtain initial access_token
Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 192, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/srcCache/kubeflow/testing/py/kubeflow/testing/get_kf_testing_cluster.py", line 429, in <module>
    fire.Fire(CredentialHelper)
  File "/usr/local/lib/python3.8/dist-packages/fire/core.py", line 138, in Fire
    component_trace = _Fire(component, args, parsed_flag_args, context, name)
  File "/usr/local/lib/python3.8/dist-packages/fire/core.py", line 463, in _Fire
    component, remaining_args = _CallAndUpdateTrace(
  File "/usr/local/lib/python3.8/dist-packages/fire/core.py", line 672, in _CallAndUpdateTrace
    component = fn(*varargs, **kwargs)
  File "/srcCache/kubeflow/testing/py/kubeflow/testing/get_kf_testing_cluster.py", line 384, in get_credentials
    raise ValueError(message)
ValueError: No clusters found matching: project: kubeflow-ci-deployment, location: us-central1-c, pattern: kf-master-(?!n\d\d)

It used docker image: gcr.io/kubeflow-ci/test-worker-py3@sha256:804c6cc8a73face69d79c60bcd85b93fa04218db9ee31127fb529b41fcab43ac
The kf ready test which worked used it correctly as well
gcr.io/kubeflow-ci/test-worker-py3@sha256:804c6cc8a73face69d79c60bcd85b93fa04218db9ee31127fb529b41fcab43ac
nomos --contexts=kf-ci-v1 status
Connecting to clusters...
Current   Context    Status           Last Synced Token   Sync Branch
-------   -------    ------           -----------------   -----------
          kf-ci-v1   SYNCED           3fbde6d3            master

That is the write image sha
https://github.com/kubeflow/testing/blob/master/images/latest_image.json

kf-ready is using the old style format
https://github.com/kubeflow/testing/blob/master/acm-repos/kf-ci-v1/namespaces/kf-ci/tekton.dev_v1alpha1_task_kf-ready.yaml#L37

The flag name is wrong it should be pattern not location.

kubeflow/testing#712 seems to fix the get-credentials issue. Now the test is timing out with.
INFO|02:09:16|/workspace/kfctl-repo/py/kubeflow/kfctl/testing/util/gcp_util.py|93| https://kf-vbp-0701-ab4.endpoints.kubeflow-ci-deployment.cloud.goog: Endpoint not ready, exception caught 'str' object has no attribute 'text', request number: 3
INFO|02:09:26|/workspace/kfctl-repo/py/kubeflow/kfctl/testing/util/gcp_util.py|81| Trying url: https://kf-vbp-0701-ab4.endpoints.kubeflow-ci-deployment.cloud.goog

I suspect this is the offending line:
https://github.com/kubeflow/kfctl/blob/baf59c2692f45847bbd042c78a751a761b2b7eaa/py/kubeflow/kfctl/testing/util/gcp_util.py#L85

Test is passing
https://k8s-testgrid.appspot.com/sig-big-data#kubeflow-gcp-blueprints-master-periodic&group-by-hierarchy-pattern=[\w-]%2B
https://prow.k8s.io/view/gcs/kubernetes-jenkins/logs/kubeflow-gcp-blueprints-master-periodic/1278420908561666048


Broken side bar on mobile

I see that when we download the vuestic from github and build it, in the mobile version, the side bar wont work. no modifications done in the code.
there is some overlap in mobile version. if i make the visibility hidden for the va sidebar then the visible links are clickable.

Fixed long ago.


deprecated<EMAIL_ADDRESS>

npm WARN deprecated<EMAIL_ADDRESS>graceful-fs version 3 and before will fail on newer node releases. Please update to<EMAIL_ADDRESS>as soon as possible.

Redux doesn't depend on graceful-fs, so I don't think this is a Redux issue.

@JinxiuLee Are you using Flow? flow-bin may need updated dependencies.

I cloned repository yesterday and when I was installing the npm packages redux required(npm install), I got the warning in my terminal. So that's why I published it here as an issue.

@JinxiuLee Maybe mocha https://github.com/mochajs/mocha/issues/2121#issuecomment-187367807

@JinxiuLee When you install something npm still checks all deps.

@kouhin Yes, the Mocha team are on it: https://github.com/mochajs/mocha/pull/2129/files.
Also, Redux repo includes an old version of gitbook-cli: 0.3.6. Latest version is 2.1.2.


[REEF-245]  Move Network test from REEF.Tests to REEF.Network.Tests proj

This PR is add a new test project for Network unit tests and move Network unit test cases to the new project
JIRA: REEF-245. (https://issues.apache.org/jira/browse/REEF-245)
This closes #
Author: Julia Wang  Email<EMAIL_ADDRESS>

I'll test this.


Showdown escaping HTML Chars but not constantly

In regards to #218 where the outcome was that showdown didn't escape any HTML-Special-Chars I now have the Problem that sometimes HTML-Special Chars are converted and sometimes they aren't.
Here is some Code to reproduce this:
const showdown = require("showdown")
const converter = new showdown.Converter({ simpleLineBreaks: true });

converter.makeHtml("&")
// Output is: '
&
'

converter.makeHtml("*&*");
// Output is: '
&
'

As far as I know see, HTML-Special-Chars are replaced when in plain Text but not, when they are formatted somehow.
Is there any way to turn this fully of or on for all text? I've searched through the config-options but didn't find any setting for this.

Don't know why this has been closed "by me", so reopening now


pre-commit: add hook to trim trailing whitespace

https://github.com/pre-commit/pre-commit-hooks#trailing-whitespace
Description
This PR adds another hook or test to our pre-commit framework.
This hook automatically trims trailing whitespace when run locally.
Follow up PRs can target more file types if needed.
Extra unneeded whitespace is extra data.  Save the planet :)

Types of changes

[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] New feature (non-breaking change which adds functionality)
[ ] Bug fix (non-breaking change which fixes an issue)
[ ] Enhancement (improves an existing feature and functionality)
[X] Cleanup (Code refactoring and cleanup, that may add test cases)
[ ] build/CI

Feature/Enhancement Scale or Bug Severity
Feature/Enhancement Scale

[ ] Major
[ ] Minor

Bug Severity

[ ] BLOCKER
[ ] Critical
[ ] Major
[ ] Minor
[ ] Trivial

Screenshots (if appropriate):
How Has This Been Tested?
pre-commit run --all-files
or
pre-commit run trailing-whitespace --all-files

How did you try to break this feature and the system with this change?

Hey @DaanHoogland  when the hook runs on GitHub with actions it is just a check that will either pass or fail.
But when pre-commit is setup locally and run this hook will also automatically trim trailing whitespace.
So yes it is a corrective edit.  So all these whitespace clean ups were done automatically by the hook.

@blueorangutan package

@jbampton a [SL] Jenkins job has been kicked to build packages. It will be bundled with  KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

Packaging result [SF]: ✔ el7 ✔ el8 ✔ el9 ✔ debian ✔ suse15. SL-JID 9015

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 31.16%. Comparing base (cadbb56) to head (9177024).
 Consider uploading reports for the commit 8b3d91f to get more accurate results

Additional details and impacted files
@@              Coverage Diff              @@
##               main    #8205       +/-   ##
=============================================
+ Coverage     15.13%   31.16%   +16.02%
- Complexity    13469    34041    +20572
=============================================
  Files          4863     5404      +541
  Lines        326031   380305    +54274
  Branches      45838    55506     +9668
=============================================
+ Hits          49349   118524    +69175
+ Misses       270066   246077    -23989
- Partials       6616    15704     +9088

Flag
Coverage Δ

simulator-marvin-tests
24.93% <ø> (+9.80%)
:arrow_up:

uitests
4.34% <ø> (?)

unit-tests
16.88% <ø> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

@blueorangutan package

@blueorangutan package

@jbampton a [SL] Jenkins job has been kicked to build packages. It will be bundled with  KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

Packaging result [SF]: ✔ el7 ✔ el8 ✔ el9 ✔ debian ✔ suse15. SL-JID 9599

@blueorangutan test

@DaanHoogland a [SL] Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

[SF] Trillian Build Failed (tid-10214)

@blueorangutan test

@DaanHoogland a [SL] Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

[SF] Trillian test result (tid-10245)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 49836 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr8205-t10245-kvm-centos7.zip
Smoke tests completed. 130 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test
Result
Time (s)
Test File

test_01_events_resource
Error
414.61
test_events_resource.py


General Code Improvement 1

This pull request is focused on resolving occurrences of Sonar rule
squid:S1488 Local Variables should not be declared and then immediately returned or thrown
squid:UselessParenthesesCheck Useless parentheses around expressions should be removed to prevent any misunderstanding
squid:S1155 Collection.isEmpty() should be used to test for emptiness
squid:HiddenFieldCheck Local variables should not shadow class fields
You can find more information about the issue here:
https://dev.eclipse.org/sonar/rules/show/squid:UselessParenthesesCheck
https://dev.eclipse.org/sonar/rules/show/squid:S1155
https://dev.eclipse.org/sonar/rules/show/squid:HiddenFieldCheck
https://dev.eclipse.org/sonar/rules/show/squid:S1488
Please let me know if you have any questions.
Zeeshan Asghar

I don't like unnecessary suffixes in names, like "Local" in "securityContextRepositoryLocal". Please change the name to something else.

@peholmst PR updated.


[Question] Does it work with REA v2?

Hi @satya164,
I am currently migrating from react-navigation to react-native-navigation and just noticed that it does not support top bars by default, so I searched a little bit and found out this library.
Does it work with the new react-native-reanimated v2?
Thank you for your amazing work 😎

I was not able to make it work with reanimated 2 with the simple example from the doc :'(
The app crash instantly with this error:
libc++abi.dylib: terminating with uncaught exception of type NSException
*** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '-[REACondNode beginContext:prevCallID:]: unrecognized selector sent to instance 0x60000041afd0'

@nelsonprsousa Do you have any idea why this library move away from reanimated?

@nelsonprsousa Do you have any idea why this library move away from reanimated?

Yes, check https://twitter.com/satya164/status/1369344748620836867 and https://twitter.com/satya164/status/1369344244608098307 tweets from @satya164 explaining why.
That being said, we upgraded to 3.0.0 but reverted because the tab animation is lagging for whatever reason with 3.0.0.


Recreate Rejected VPC Endpoints

This commit handles the case when a VPC Endpoint is put into the rejected state by deleting the VPC Endpoint. This has the net effect of the controller recreating the VPC Endpoint.
Test plan in stage:

Create a stage HCP cluster
Reject it's VPC Endpoint
Observe the updated code delete the VPC Endpoint, recreate it, and update Route53

It takes quite some time for it to get to a steady state (~30 minutes), so there's room for improvement as we're polling AWS every 15minutes by default if we don't run into errors, but this helps the uninstall case where we don't even need to wait for the VPC Endpoint to get into an available state.
Fixes OSD-17219

/lgtm


Copy manifest and bundle from S3

Using a python script to download the manifest file and bundle from S3 to jenkins

[x] Provide S3 read write methods as a utility for other modules
[x] Provide s3 read/write role permission support for Jenkins and local dev

This is done with #367


download users location data

as a user I want to be able to download the raw data of the location that i have loaded to healthsites.io so that I can use it elsewhere

we are using shapefiles for all world or per country


Custom size

I want add custom size for every smile by attribute (like bbcode)
[emojiname-size]
And this should look like image with width=size and  height=size

Sample.
[fun-30]
Must replace with:

Is that possible?

+1

How it suppose to look like in the web?
Is the user has a scrollbar?

+1


Scroll to hide keyboard

Firstly, thanks for a great package!
I just downloaded and updated to latest version, now the keyboard is "stuck" and not hiding when scrolling tableview. Everything works fine though except for the visual scroll to hide keyboard. Any suggestions what it might be?

Thanks for quick support! Ill look it up! cheers


Android parcelable support

implemented support for android.os.parcelable

Automated message from Dropbox CLA bot
@konovalov-aleks, it looks like you've already signed the Dropbox CLA. Thanks!

Hmm we're back at the problem of stuffing the YAML files with additional content, in this case it is even mandatory for projects where the feature isn't actually used. Considering the Parcelable interface requires writeToParcel and CREATOR do we need the YAML entries?

I've made some suggestions in line comments.
As a high-level comment I wonder if this command-line argument approach is the best, or if it would be better to let Parcelable be specified on individual records using a deriving clause (similar to "deriving ord").  That would provide conditionality in a way which is clear in the code where parcelability is desired, as well as making it optional and thus not forcing it to be implemented on external types in yaml if it won't be used.
Maybe for maximal flexibility you'd need both a deriving clause to say that parcelable is desirable on Android, and a cmd-line arg indicating that the Java code is for Android.  That would only be necessary if the same IDL needs to support Android Java and non-Android Java generation in separate runs.

Hi @artwyman, thanks for your feedback!
I've implemented your suggestions. Please review my code

This feature would be great!
@konovalov-aleks any news on this?
Thanks

@konovalov-aleks This feature is great!
Since this PR is inactive for some time, I pick the changes here and make the changes requested by @artwyman.
Hope this is ok to you.
If you want, you can pick my PR and continue the work here :)
Thanks

Replaced by #308


Packaging: Past versions of packages are deleted from APT repositories.

We automate Crate installation via the use of Debian packages. I would like to be able to pin the version number and be able to provision systems using the same package version and then upgrade when we are ready.
Currently it seems that only the most recent version is being kept in e.g. https://cdn.crate.io/downloads/deb/stable/dists/xenial/main/binary-amd64/
Would it be possible to keep old releases available so that one could for example apt-get install crate=2.1.7
Thanks!

hi @jeffnappi, we are already looking into that as another user reported something similar. see the discussion here: https://askubuntu.com/questions/977160/install-a-specific-version-version-not-listed-using-madison we'll keep you posted.

Hey @jeffnappi - This is because we were using the reprepo tool to manage our debian and ubuntu repositories, which unfortunately has the limitation of only allowing one version per package and distribution. We will be moving our release infrastructure to use something that allows us to keep old releases available. :slightly_smiling_face:

In addition to this, more recently we're missing some of the newer builds for older distros - in my case Debian Xenial.
https://cdn.crate.io/downloads/deb/stable/pool/main/c/crate/

@JAC2703 I would open a separate issue for that! It looks like there's only builds for artful now.

@JAC2703 @jeffnappi Thanks for bringing this to our attention! Yeah, it looks like only artful builds have been uploaded. We'll look into this & let you know when the other distributions are up. :slightly_smiling_face:

Sorry for late response. Ticket raised under #7175

It's been a long time since this got raised, but we finally made some changes to our packaging pipeline and as part of that we now preserve older versions in the packages index.
There is a new repository: https://cdn.crate.io/downloads/debian/stable/ - we didn't redeploy old packages. For now only 5.5.1 is available.


Feature: Better error messages

When a cast function raises unexpectedly it can be tricky to see what field actually caused it. This can be helped by unit testing the cast fns (so you are sure they handle the cases you expect) and then possibly IO.inspecting the value as it comes in. But it would be nicer to be able to see which field caused the error just straight off the bat.
The obvious thing to do would be to wrap the cast__fns in a try catch but I don't like that at all...

This applies to casting fns that are atoms and casting fns that are MFA tuples

Improved in https://github.com/Adzz/data_schema/pull/40
We could keep evolving the error message that is returned, eg calling the trace we give a schema-trace
suggestion it could be something like:
CSchema: field(:c)
  BSchema: has_one(:b)
    ASchema: has_one(:a)


Allow ctrl-r ctrl-a ctrl-e.

Add 3 functions for vim.
ctrl-r for backward search.
ctrl-a to the beginning of line
ctrl-e to the end of line.

Keep the original functions.


Add 'vivid' and 'midair'

This PR is not in valid yaml syntax, it's missing a trailing colon.

@snoyberg thanks; fixed now.


Allow empty path elements (MLDB-1635)

Self explanatory... this allows for paths to contain empty elements, just like JSON (which allows empty keys).

+1


"Empty response" when delete is used on Google Accounts

Describe the bug
Running into issue with message "Empty response" when delete is used on Google (both free gmail and g-suite account).
Used config
$options = [
                    'options' => [
                        'soft_fail' => false,
                        'debug' => true,
                    ],
                ];
$connectionConfig = [
                    'host'          => -,
                    'port'          => 993,
                    'encryption'    => 'ssl',
                    'validate_cert' => true,
                    'fetch' => \Webklex\PHPIMAP\IMAP::FT_PEEK,
                    'sequence' => \Webklex\PHPIMAP\IMAP::ST_UID,
                    'fetch_flags' => true,
                    'soft_fail' => false,
                    'username'      => emailAddress,
                    'password'      => AppPasswordFromGmail,
                    'protocol'      => 'imap',
                    'timeout'       => 180,
                    'dispositions' => ['attachment', 'inline'],
                    'decoder' => [
                        'message' => 'utf-8', // mimeheader
                        'attachment' => 'utf-8' // mimeheader
                    ]
                ];

StackTrace
[2024-02-05 05:17:37] production.WARNING: array (
  'Email Attempted' =><EMAIL_ADDRESS>  'Error Message' => 'Command failed to process:
Causes:
        - Empty response
Commands send:
        TAG20 UID FETCH 2886 (FLAGS)\\r\\n
Responses received:
        TAG20 OK Success\\r\\n
Error occurred',
  'Error StackTrace' => '#0 /masked-home-location/application/vendor/webklex/php-imap/src/Connection/Protocols/Response.php(318): Webklex\\PHPIMAP\\Exceptions\\ResponseException::make()
#1 /masked-home-location/application/vendor/webklex/php-imap/src/Connection/Protocols/Response.php(308): Webklex\\PHPIMAP\\Connection\\Protocols\\Response->validate()
#2 /masked-home-location/application/vendor/webklex/php-imap/src/Message.php(585): Webklex\\PHPIMAP\\Connection\\Protocols\\Response->validatedData()
#3 /masked-home-location/application/vendor/webklex/php-imap/src/Message.php(1254): Webklex\\PHPIMAP\\Message->parseFlags()
#4 /masked-home-location/application/vendor/webklex/php-imap/src/Message.php(1194): Webklex\\PHPIMAP\\Message->setFlag()
#5 /masked-home-location/application/app/Application call: Webklex\\PHPIMAP\\Message->delete()

[2024-02-05 05:18:20] production.WARNING: array (
  'Email Attempted' => 'Google Workplace account',
  'Error Message' => 'Command failed to process:
Causes:
        - Empty response
Commands send:
        TAG23 UID FETCH 2632 (FLAGS)\\r\\n
Responses received:
        TAG23 OK Success\\r\\n
Error occurred',
  'Error StackTrace' => '#0 /masked-home-location/application/vendor/webklex/php-imap/src/Connection/Protocols/Response.php(318): Webklex\\PHPIMAP\\Exceptions\\ResponseException::make()
#1 /masked-home-location/application/vendor/webklex/php-imap/src/Connection/Protocols/Response.php(308): Webklex\\PHPIMAP\\Connection\\Protocols\\Response->validate()
#2 /masked-home-location/application/vendor/webklex/php-imap/src/Message.php(585): Webklex\\PHPIMAP\\Connection\\Protocols\\Response->validatedData()
#3 /masked-home-location/application/vendor/webklex/php-imap/src/Message.php(1254): Webklex\\PHPIMAP\\Message->parseFlags()
#4 /masked-home-location/application/vendor/webklex/php-imap/src/Message.php(1194): Webklex\\PHPIMAP\\Message->setFlag()
#5 /masked-home-location/application/app/Application call: Webklex\\PHPIMAP\\Message->delete()

Expected behavior
Using delete on email should not be throwing error and just delete the message.
Seems IMAP is returning correctly (with message OK Success) but library is still throwing error with Empty Response
Desktop / Server (please complete the following information):

OS: Ubuntu 22
PHP: 8.1
Version : 5.5.0
Provider : Gmail

I'm seeing the same thing, using PHP-IMAP for the first time.
I don't understand the protocol enough to debug why this is happening, but can see a clear difference in the Response when parseFlags() is called when initially querying messages, compared to from within the setFlag() called inside delete():
On load:

delete() -> setFlag() -> parseFlags():


Add Privacy Manifest (Enforce in App Store on May 1st)

Hello -
I'm wondering what plans there are to add a Privacy Manifest?
The App Store will be enforcing this more strictly starting on May 1st.
Thanks!
https://developer.apple.com/support/third-party-SDK-requirements/
https://developer.apple.com/documentation/bundleresources/privacy_manifest_files/describing_use_of_required_reason_api

Are you sending your app in prod with DebugSwift? hehehe (Normally is only to Debug), I don't know if are needed to debug the manifest :(

@MaatheusGois Yes, I only use it to debug the app in non-production environment. However, when I add DebugSwift into my package.swift file, it will be included in the production app despite we are not using it. As far as I know, there is currently no way for me to import swift package conditionally based on the build configuration: https://github.com/apple/swift-package-manager/pull/2428#issuecomment-614903476

@yungkienpoh-prestolabs you can try this, because is only UserDefaults and Files that DebugSwift uses :)

@MaatheusGois I have a look at the code, do I need to declare File Timestamp? I don't see the library trying to get this information in the code?

I think it's because of FileManager, but I'm not sure... If you can, you can use this tool to find: https://github.com/stelabouras/privacy-manifest


UI- RKE Templates: Updating Cluster to a new RKE Template Revision does not take into effect

What kind of request is this (question/bug/enhancement/feature request):
Bug - from a user
Steps to reproduce (least amount of steps as possible):

Create a new RKE Template with Revision R1- Set some settings say
"Node Port Range: 30000 - 32767" and "Docker Root Directory: /var/lib/docker123"

Clone a new Revision R2 and set these values to
"Node Port Range: 30000 - 32769" and "Docker Root Directory: /var/lib/docker678"

Add a Cluster C using the RKE Template Revision R1. Wait for the cluster to become active. Verify that the cluster is tied to the correct Revision R1.

Edit the Cluster and switch to the Revision R2. Click Save.

Cluster does not get updated with new settings, the RKETemplate Revision tied to the Cluster still remains the same - R1

It is seen that the new clustertemplaterevision Id is not sent to the backend in cluster update request.

Note that if there are any settings in the new revision that are set as "overrides" or questions, upon updating the cluster, these settings get updated because the new values are being sent to the backend in the "answers" field of the update cluster API call.
Result:
Cluster is not updated to new revision.
Other details that may be helpful:
Environment information

Rancher version (rancher/rancher/rancher/server image tag or shown bottom left in the UI): v2.3.0
Installation option (single install/HA): single

This is very similar to the issue that was fixed earlier - https://github.com/rancher/rancher/issues/22920

Tested with v2.3.1-rc1. The above test case was tested and worked successfully. Also, the below test cases were tested :

Create a RKE template T1
Create revision R1 with K8s version fixed (v1.14.7-rancher1-1)
Create revision R2 as a new revision from default with K8s version fixed (v1.15.4-rancher1-2)
and node port range and docker directory set
Create cluster with R1.
Edit cluster and change revision to R2
Cluster goes to updating state and updates the cluster K8s version to v1.15.4 as expected
Create a RKE template T1
Create revision R1 with K8s version set as override (v1.14.7-rancher1-1)
Create revision R2 as a new revision from default with K8s version set as override (v1.15.4-rancher1-2) and node port range and docker directory set
Create cluster with R1.
Edit cluster and change revision to R2
Cluster goes to updating state and updates the cluster K8s version to v1.15.4 as expected
Create a RKE template T1
Create revision R1 with K8s version set as override (v1.14.7-rancher1-1)
Create revision R2 as a new revision from default with K8s version fixed (v1.15.4-rancher1-2)
and node port range and docker directory set
Create cluster with R1.
Edit cluster and change revision to R2
Cluster goes to updating state and updates the cluster K8s version to v1.15.4 as expected
Create a RKE template T1
Create revision R1 with K8s version fixed (v1.14.7-rancher1-1)
Create revision R2 as a new revision from default with K8s version set as override (v1.15.4-rancher1-2)  and node port range and docker directory set
Create cluster with R1.
Edit cluster and change revision to R2
Cluster goes to updating state and updates the cluster K8s version to v1.15.4 as expected

The same tests were repeated with creating R2 as a clone of revision R1


Attachment component

Adds multiple static meshes, skeletal meshes, and actors to the character mesh.

Sorry, but I am rejecting this pull request.
Attachment management is not what ALS is responsible for and it is mostly individual for each project. For this reason I perviously moved the attachment logic directly into the example character blueprint so that you can easily get rid of that logic and use your own if needed.


Roll Flutter Engine from 9c3f66d822e4 to 0e5df26e590f (1 revision)

https://github.com/flutter/engine/compare/9c3f66d822e4...0e5df26e590f
2024-08-30<EMAIL_ADDRESS>Roll Dart SDK from 139867e57ba9 to 88d311c9a531 (1 revision) (flutter/engine#54886)
If this roll has caused a breakage, revert this CL and stop the roller
using the controls here:
https://autoroll.skia.org/r/flutter-engine-flutter-autoroll
Please CC matanl@google.com,rmistry<EMAIL_ADDRESS>on the revert to ensure that a human
is aware of the problem.
To file a bug in Flutter: https://github.com/flutter/flutter/issues/new/choose
To report a problem with the AutoRoller itself, please file a bug:
https://issues.skia.org/issues/new?component=1389291&template=1850622
Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md

Commit queue failed; closing this roll.


getting the ASpIRE chain models

How can I make offline ASpIRE chain models to work using pykaldi. When we download the model from the Kaldi repo, it says we need to run the scripts to make it as a model. When I try to make the model, it gives error command tree-info not found. In a forum, I saw it is because of compilation error with Kaldi. Do we need to install Kaldi from the source to create the model?

Hi, you can take a look at https://github.com/pykaldi/pykaldi/tree/master/examples/setups/aspire which downloads the models you need.


fix: add makeValidEnumIdentifier for enumsAsConst

Description
Properly escape enum identifiers when enumsAsConst is used
Related #9149
Type of change

[x] Bug fix (non-breaking change which fixes an issue)

How Has This Been Tested?
Tested change locally using a basic enum like
enum SomeType {
  _401K
  Other
}

Test Environment:

OS: macOS

Checklist:

[X] I have followed the CONTRIBUTING doc and the style guidelines of this project
[X] I have performed a self-review of my own code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[x] My changes generate no new warnings
[x] I have added tests that prove my fix is effective or that my feature works
[x] New and existing unit tests pass locally with my changes
[ ] Any dependent changes have been merged and published in downstream modules

Is there anything I'm missing in the PR before it can be reviewed?


[Foreman::CLI] use proper check to see if file exists

swapped exist?/exists? methods with file?, so foreman can ignore
directories. I came across this issue when having '.env' directory
with a bunch of .yml files inside. foreman start tried to load
default '.env' configuration and failed because it was a directory.

Thanks!


Sometimes unreasonable memory consumption

I have several bee instances running, using a build based on https://github.com/ethersphere/bee/commit/62c576e74389ddf88c6c16488ab992dc4b9aa38a (specifically, with https://github.com/ethersphere/bee/pull/1282 already included).
While memory consumption seems to have considerably improved compared to v0.5.0 (where all bee processes seem to slowly crawl to higher and higher memory allocation all the time), I currently have one process which spiked to >16G resident memory (more than double of that virtual memory) in less than 12 hours and another one which went to ~1,4G RES / 27G VIRT.
The other bee processes remain in a range of few 100 MB RES.
    PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
  45263 bee       30  10   38.9g  18.1g   9808 S 106.6  28.9 375:27.05 bee114
  39512 bee       30  10   27.9g   1.4g   8844 S  14.3   2.3 402:36.97 bee101
  43449 bee       30  10 4711728 335536   9140 S  10.6   0.5 201:39.09 bee206
  39519 bee       30  10 1011664 310980   9896 S  34.2   0.5 167:23.21 bee102
    415 root      19  -1<PHONE_NUMBER>20 302068 S   8.3   0.5  49:18.66 systemd-journal
  43479 bee       30  10 1972428 214256   9484 S   7.0   0.3 176:33.06 bee209
  43378 bee       30  10 1698172 203304   9976 S  15.0   0.3 197:03.33 bee105
...

(no, I'm not running >100 bees on that machine, it's just a numbering scheme - 1xx use disk1, 2xx use disk2)
I checked open sockets and files, but nothing abnormal there when comparing "misbehaving" to "behaving" bees.
Configuration wise they are equal, only different port numbers.
One difference I notice - which may or may not be relevant: bee114 (the excessive one) has the smallest DB size at the moment:
du -smx /data1/bees/*
...
16880   /data1/bees/111
10909   /data1/bees/112
15054   /data1/bees/113
9813    /data1/bees/114
...

The difference is not huge (they all range btw. ~10G and ~20G at the moment), but still...
Since the process is still running (for now - I fear at some point it will exhaust all memory), I could provide further insights if somebody can give me a hint what to look for.
clef is not used here.
PS: that's how overall memory allocation changed over time:

I guess the upshot around 9AM was bee114 getting too excited about something.

~12 hours later it looks like this:
    PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
  45263 bee       30  10   38.9g   7.2g   8648 S  15.9  11.5 613:52.01 bee114
  39512 bee       30  10   27.9g   1.2g   8632 S   6.3   1.8 481:49.65 bee101
  43419 bee       30  10 7725484 451520   9564 S  14.3   0.7 349:52.20 bee109
    415 root      19  -1<PHONE_NUMBER>64 311960 S   5.3   0.5  96:01.32 systemd-journal
  43449 bee       30  10 4712240 305440   6248 S   8.3   0.5 311:13.86 bee206
 119494 bee       30  10 3277160 271972   8752 S  15.0   0.4 169:22.06 bee216

So, VIRT has remained the same, but RES has come done considerably for bee114.

This I find interesting:

I don't really know, but would guess that the drop occured where the graph shows the drop for "committed memory". I didn't start or stop anything at that time, looks to me like the system reacting on mounting memory pressure - see swap activity going up around the same time.

I just observed such an "incident" in realtime - a spike from <1 GB RES to >2GB RES in many small increments in less than a minute.
It was while the log got a lot of entries like this:
Mar 01 13:54:09 nody bee103[1110537]: time="2021-03-01T13:54:09+01:00" level=error msg="pushsync: new stream for peer 74733b46fcd92b4f9766bace70f223f9ed201393333f02f27a2fae9c06c018b0: send headers: read message: context deadline exceeded"
Mar 01 13:54:09 nody bee103[1110537]: time="2021-03-01T13:54:09+01:00" level=error msg="pushsync: new stream for peer 74733b46fcd92b4f9766bace70f223f9ed201393333f02f27a2fae9c06c018b0: send headers: read message: context deadline exceeded"
Mar 01 13:54:10 nody bee103[1110537]: time="2021-03-01T13:54:10+01:00" level=error msg="pushsync: new stream for peer 74733b46fcd92b4f9766bace70f223f9ed201393333f02f27a2fae9c06c018b0: send headers: read message: context deadline exceeded"

so maybe there's a hint there.

On Discord:

One only throws evel=debug msg="could not handle protocol pushsync/1.0.0: stream pushsync: peer 395c0aa9be3d8dff863dbd3120e338f37257138b3489e104d50609374f40a6dc: error: send receipt to peer 395c0aa9be3d8dff863dbd3120e338f37257138b3489e104d50609374f40a6dc: stream reset"
and this fills RAM up to OutOfMemory exception

Hi @d10r , we got several improvements, currently in master that would address this. We will release shortly. We would greatly appreciate if you could have a look and validate if the situation improves. Closing this issue for the time being as this is a very general issue and not directly actionable.

thx for the update.
I updated to 0.5.4-08c92e9 and restarted the nodes about 18h ago.
The issue at hand doesn't seem to have gone away:

If you have any idea how I could give more useful feedback, let me know.
I'm not familiar with Go, but could probably manage to dig a bit deeper anyway, e.g. by attaching a profiler, if pointed to the right docs and given specific enough instructions about what to look for.
I'm also reachable on the Swarm Mattermost for clarifications.

Here's the output of http://localhost:1635/debug/pprof/allocs?debug=1 of this instance:
     PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
3396077 bee       30  10   18.6g   1.3g  12972 R  32.3   2.0 818:34.71 bee115

bee_115_allocs.txt

Is this running on a magnetic disk?  Or is the disk I/O otherwise saturated, possibly by too many nodes running?  I've seen similar (without knowing how to look up the allocation details) on nodes running on machines whose disk writing is falling behind the network's ability to deliver data, especially with pullsync and lots of peers.

the additional profiling data requested in Mattermost:
pprof.bee.goroutine.001.pb.gz
pprof.bee.alloc_objects.alloc_space.inuse_objects.inuse_space.001.pb.gz
@ldeffenb it's running on NVMe disks. However it may well be possible that it's saturated.
As suggested, I'm running multiple nodes with the default storage size and there's > 1TB storage available.
I thought since it's fast storage and overall a performant machine, that should be possible.
If not, what's the recommended way to provide that amount of storage to Swarm? Increase the capacity per node instead?

This is the only published guidance on storage space: https://docs.ethswarm.org/docs/advanced/large-node-operators
It's all new software, so we're experimenting.  If you have a way to check the underlying host disk accesses (I use "sudo atop" along with munin performance monitoring), I suspect you'll find that the disks are 100% busy.  That causes I/O backlogs into the bee nodes which will cause increased memory consumption as it tries to write (and read) more data than the disks can handle.
The same thing can happen with even a single node if the data is on magnetic media.
Free disk space (> 1TB available) is not a measure of disk subsystem performance, but you probably already know that.
And there's an old maxim that software will grow to and beyond the limits of any hardware you can throw at it.  "fast storage" and "performant machine" can still be overrun with too much load.  And right now, bee nodes are not light resource users.  Hopefully it will improve, but the approach seems to be "functionality first", "performance later".

the additional profiling data requested in Mattermost:
pprof.bee.goroutine.001.pb.gz
pprof.bee.alloc_objects.alloc_space.inuse_objects.inuse_space.001.pb.gz
@ldeffenb it's running on NVMe disks. However it may well be possible that it's saturated.
As suggested, I'm running multiple nodes with the default storage size and there's > 1TB storage available.
I thought since it's fast storage and overall a performant machine with high bandwidth, that should be possible.
If not, what's the recommended way to provide that amount of storage to Swarm? Increase the capacity per node instead?

@d10r sorry for taking so long to reply. We've looked into the heap dumps you've provided but have not found anything particularly wrong there. The problem is that the go runtime just doesn't release the allocated memory back to the OS. For users that have a lot of RAM this issue is particularly visible, since the OS doesn't need to reclaim the memory anyway and can just keep on allocating. A nice test would be to have a different process that actually allocates a lot of active memory to run alongside the bee nodes in order to verify that the just stop allocating new memory.
In any case, this should be improved with go 1.16 if I'm not mistaken. We haven't upgraded yet, but hopefully before the next release is out.


Onboarding for WIAD TEST

Application submitted 2021-09-13T09:46:20.000Z by Grace TEST

[ ] Location email access
[ ] Set up event page

if new location

[ ] Set up new location page
[ ] Set up new location email address

Closing WIAD


Complete the removal of the cluster config

Description
This PR completes the removal of the cluster config, removing the remaining elements concerning the discovery component. Additionally, it refactors the discovery component to replace the usage of the CRD client.
Fixes #(issue)
How Has This Been Tested?
Please describe the tests that you ran to verify your changes. Please also note any relevant details for your test configuration.

[ ] Unit testing (existing)
[ ] E2E testing (existing)

/test

/test

/build

/build

/test

/test

/test

/test

/test

/test

/test

/test

/test

/test

/merge


[vim] rewrite search/replace queries that end in $

resolves long-standing issue https://github.com/codemirror/CodeMirror/issues/2806
changes the search and replace strings coming into vim into a form that the codemirror searchcursor code is able to handle.
%s/$/foo/   ->  %s/\n/foo\n/
I did not intend to change package-lock, but wasn't able to find a way to remove it from this pull request.  I'm hoping that will be a relatively easy thing for you to work around when merging this in.

Thanks, merged as f93a5281. It might be worthwhile to spend some time learning git concepts (rebase, in particular) in order to be able to submit clean pull requests (having a single patch and no extra stuff makes it easier for me to merge).


[Python][CI] test-conda-python-3.11-spark-master fails, requires Java 17

Describe the bug, including details regarding any error messages, version, and platform.
test-conda-python-3.11-spark-master fails with:
Downloaded from gcs-maven-central-mirror: https://maven-central.storage-download.googleapis.com/maven2/com/github/os72/protoc-jar-maven-plugin/3.11.4/protoc-jar-maven-plugin-3.11.4.jar (10 MB at 11 MB/s)
Using `mvn` from path: /spark/build/apache-maven-3.9.4/bin/mvn
Error:  Failed to execute goal org.apache.maven.plugins:maven-enforcer-plugin:3.3.0:enforce (enforce-versions) on project spark-parent_2.13:
Error:  Rule 1: org.apache.maven.enforcer.rules.version.RequireJavaVersion failed with message:
Error:  Detected JDK version 1.8.0-382 (JAVA_HOME=/opt/conda/envs/arrow/jre) is not in the allowed range [17,).
Error:  -> [Help 1]
Error:
Error:  To see the full stack trace of the errors, re-run Maven with the -e switch.
Error:  Re-run Maven using the -X switch to enable full debug logging.
Error:
Error:  For more information about the errors and possible solutions, please read the following articles:
Error:  [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
1
Error: `docker-compose --file /home/runner/work/crossbow/crossbow/arrow/docker-compose.yml run --rm -e SETUPTOOLS_SCM_PRETEND_VERSION=14.0.0.dev256 conda-python-spark` exited with a non-zero exit code 1, see the process log above.

https://github.com/ursacomputing/crossbow/actions/runs/6424410884/job/17445002795
Component(s)
Python

I suppose if Spark has dropped Java 8, there's no reason we can't drop Java 8 for Arrow 14 or Arrow 15?

+1! I just opened https://github.com/apache/arrow/issues/38051 today.
Arrow v15 is the current target release. I figured it would be better to have more discussion on the mailing list before dropping support.


Flashing window, Focus loop

Hello I used your class in my project, however I have a problem and I can not solve it. I made a program to open the last windows that were open in the last session of the user. But the program loops in the Focus methods, but the downloaded example opens the windows direct and does not loop, I did the test opening empty windows and it worked, this loop is generated only when I put my controls in, I'm not getting to understand, Has anyone had this problem?

I solved
In method "FocusedValueChanged" I changed this line
mdiChild.Dispatcher.BeginInvoke(new Func<IInputElement, IInputElement>(Keyboard.Focus), System.Windows.Threading.DispatcherPriority.ApplicationIdle, mdiChild.Content);
for this
mdiChild.Dispatcher.BeginInvoke(new Func<IInputElement, IInputElement>(Keyboard.Focus), System.Windows.Threading.DispatcherPriority.ApplicationIdle, mdiChild);


ssh as root should not display menu

If I SSH in as root I get greeted with the same menu as on console, ideally only display that on pty's and serial.
Talking about this menu:
 0) Logout                             7) Ping host
 1) Assign Interfaces                  8) Shell
 2) Set interface(s) IP address        9) pfTop
 3) Reset the root password           10) Filter Logs
 4) Reset to factory defaults         11) Restart web interface
 5) Reboot system                     12) Upgrade from console
 6) Halt system                       13) Restore a configuration

Well this is something that should be discussed in detail, because it is not as easy as one might think. Maybe I'm overly concerned, but the state of the installation's security could be severely improved here.
(1) the menu is a shell, so whenever you are root you get your shell. I fixed this behaviour. The previous "run a shell script if X otherwise run the menu" is not a good practise although it may seem convenient.
(2) root access should be very very limited. I am inclined to kick "permit root login", which would solve the issue because you can then add a wheel user and use a proper sudo instead. create an additional user and you are without said menu.
(3) add a different user that is invoked in the console or for the shell. essentially you can pick your shell in the user manager or at least you should be able to. I don't know where that leaves root...
(4) I have come to appreciate the menu even on ssh for development and testing and I know that a lot of novice users like it too, because it's convenient. It is also more "secure" in the sense that you cannot do potentially disastrous things by default, meaning getting to the actual shell.
(5) we have zapped the spurious admin user which was essentially root and want to go further with privilege separation. That is somewhat related to (3), but plays another important role in not visibly gaining root access. Maybe we must scramble the root user and install su/sudo to work with wheel in the appropriate manner--I actually like sudo better, because then we don't run into the "change the root shell" for different applications argument.
What do you think? All feedback will help... :)

I like the 2/3/5 combo,

Force don't permit root login over ssh
default admin should not be root
make users in the admin group have sudo
none of the 'admin' users should be special (you can delete all but one - the current logged in one)
allow root on physical or serial console only, with a 'menu' and an option to add a new admin use (in case of a lock out) all
allow non-root users to choice a shell (including menu)

That should cover most bases and be a lot more secure?

Also sudo isn't as useful because I'd have to implement password-less sudo anyways.
Ansible is using SSH keys for authentication.

With the next release the following works:

In the user manager, add a different users for your shell access, add "admins" group, save. Then, in the user's config, add privileges for "User - System - Shell account access". su(1) will work then, but still launch the root menu. It's not going away that easily.
sudo(8) will be part of the next release by default, if you change its config to support `wheel' users to run commands you'll have effectively circumvented the root shell.


Anti double clientids

This rerolls the clientid if it already exists. Also adds a puts to announce new clients

Good idea!  I believe that storing is as an in-memory constant like that will break down as soon as this is running in more than one process.  For example, if we had multiple web servers running at once, or even if we just run puma with multiple workers (bundle exec puma -w 2).  Each process will have its own separate CLIENT_IDS constant.
Instead, I'd suggest storing that list in Redis, like we're doing with the global list of messages.  Maybe use sadd and sismember to store + check in a redis Set?

OK, I'll close this then and look into redis remembering stuff


Multiple modifiers in constructor shorthand are not highlighted

Multiple modifiers, like private readonly, public readonly, etc, are not correctly highlighted. Only the first of the modifiers is highlighted; the other words are highlighted as identifiers:

class Renamer {
  constructor (private readonly srcName: string, private readonly target: string, readonly copy: boolean = false) {}
}

It looks like it's supposed to be parsed, but the order is wrong? readonly private is a syntax error.


Clusters with IPv6 for cluster/service CIDR propagate the endpoints correctly, but libreswan can't make the connections

What happened:
Installed submariner with '--load-balancer' option onto 2 kubernetes cluster running on AWS using kubespray, but i don't see a connection on gateway, i can see load balancer up and running, it accepts connection from ports 4500 and 4490
What you expected to happen:
Gateway to have connection on it
How to reproduce it (as minimally and precisely as possible):
deploy kubernetes cluster using kubespray onto AWS, and subctl join using load-balancer option
Anything else we need to know?:
Environment:

Diagnose information (use subctl diagnose all):

Cluster "cluster1"
 ✓ Checking Submariner support for the Kubernetes version
 ✓ Kubernetes version "v1.20.7" is supported

 ✓ Checking Submariner support for the CNI network plugin
 ✓ The detected CNI network plugin ("calico") is supported
 ✓ Calico CNI detected, checking if the Submariner IPPool pre-requisites are configured.

 ✗ Checking gateway connections
 ✗ There are no active connections on gateway "xxxxxxxxx"

 ✓ Checking Submariner pods
 ✓ All Submariner pods are up and running

 ✓ Non-Globalnet deployment detected - checking if cluster CIDRs overlap
 ✓ Clusters do not have overlapping CIDRs

 ✓ Checking Submariner support for the kube-proxy mode
 ✓ The kube-proxy mode is supported

subctl show connections
Cluster "cluster1"
 ✗ Showing Connections
 ✗ No connections found

Gather information (use subctl gather): Attaching the gather logs for cluster1 and cluster2
Cloud provider or hardware configuration: AWS
Install tools: subctl
Others:

cluster1.zip
cluster2.zip

From the operator:
I0905 22:52:44.641752       1 network.go:56] controller_submariner "msg"="Discovered K8s network details"  "clusterCIDRs"=["<IP_ADDRESS>/18,fd85:ee78:d8a6:8607::2:0000/112"] "plugin"="calico" "serviceCIDRs"=["<IP_ADDRESS>/18,fd85:ee78:d8a6:8607::2000/116"]
I0905 22:52:44.641772       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="<IP_ADDRESS>/18,fd85:ee78:d8a6:8607::2:0000/112" "type"="Cluster"
I0905 22:52:44.641789       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="<IP_ADDRESS>/18,fd85:ee78:d8a6:8607::2000/116" "type"="Service"

whack error: "submariner-cable-aws-dev03-10-129-10-49-0-1" IPv4 address may not contain `:' "fd85:ee78:d8a6:8607::3000/116"
W0905 22:56:50.560508       1 libreswan.go:277] error exit status 33 whacking with args: [--psk --encrypt --forceencaps --name submariner-cable-aws-dev03-10-129-10-49-0-1 --id @<IP_ADDRESS>-0-1 --host <IP_ADDRESS> --client <IP_ADDRESS>/18 --to --id @<IP_ADDRESS>-1-0 --host <IP_ADDRESS> --client fd85:ee78:d8a6:8607::3000/116 --ikeport 4500]
whack error: "submariner-cable-aws-dev03-10-129-10-49-0-1" IPv4 address may not contain `:' "fd85:ee78:d8a6:8607::3000/116"

On my dual-stack cluster , I encounter  similar  ipv6 issue ,
does  submariner support ipv6 ?
on gateway pod

I0915 09:17:24.143445       1 libreswan.go:370] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster1-172-110-185-45-0-1 --id <IP_ADDRESS> --host <IP_ADDRESS> --client <IP_ADDRESS>/16 --ikeport 4500 --to --id <IP_ADDRESS> --host <IP_ADDRESS> --client fc02::/108 --ikeport 4500]
whack error: "submariner-cable-cluster1-172-110-185-45-0-1" IPv4 address may not contain `:' "fc02::/108

on router agent pod

E0915 09:49:37.265455       1 endpoint_created.go:40] Error handling created endpoint: LocalEndpointCreated failed: "MTU handler" returned error: error adding entry "fc06::/108" to set "SUBMARINER-LOCALCIDRS": exit status 1 (ipset v7.6: Syntax error: '108' is out of range 0-32

On my dual-stack cluster , I encounter similar ipv6 issue ,
does submariner support ipv6 ?

Currently, Submariner does not support IPv6. It is on the roadmap, but not on the priority list of items.

bump

This is still relevant, but its on roadmap.

This is now fixed as part of https://github.com/submariner-io/submariner/issues/1739


Dark Theme

Going with the trend of modern applications nowadays, would it be possible to implement a dark theme to Polaris as a toggle?
I've hacked together a basic CSS override that works once a user is signed in (might need tweaking etc. for accessibility).
.paneHeader, .paneContent, player, router route auth {
    background-color: rgb(27, 38, 44);
}

.paneContent * {
    color: rgb(255, 255, 255);
}

.paneHeader * {
    color: rgb(255, 255, 255);
}

breadcrumbs ul li, .paneContent button, .paneHeader button {
    background-color: rgb(15, 76, 117);
    color: rgb(255, 255, 255);
}

.paneHeader select, .paneHeader input, .paneContent select, .paneContent input {
    color: rgb(0, 0, 0);
}

button {
    border: 0 !important;
}

playlist tr:nth-child(2n) {
    background-color: rgb(35, 46, 52);
}

player * {
    color: rgb(255, 255, 255) !important;
}

router route settings div, router route browser div, router route playlists div, playlist, playlist div, router route player {
    border-color: rgb(35, 46, 52) !important;
}

menu .selected {
    background-color: rgb(15, 76, 117);
}

I think this is a good idea, I can't promise when I'll get to it but this is definitely something I want to implement.

Turns out I did have a bit of time to start working on this!
What do you think of this prototype?

I think the white on blue contrast is a bit difficult to read on the thinner text in the breadcrumbs, but otherwise is just what I was looking for in a dark theme.
Keeping the highlight consistent across both is ideal, so it may be worth increasing the font weight in the breadcrumbs if possible (600 seems OK) or alternately in the dark theme the highlight could be dimmed down to increase the contrast between the text & bg.
The second option also makes the UI WCAG compliant which is always a bonus :).
breadcrumbs ul li {
    font-weight: 600;
}

breadcrumbs ul li {
    background-color: #095d78;
}

menu .selected {
    background-color: #095d78;
}

playlist tr.nowPlaying td, playlist tr.nowPlaying td * {
    background-color: #095d78;
}

playlist tr.nowPlaying td.song .trackArtist {
    color: #DDD;
}

Thanks! I agree about the font weight, I think that would also benefit the light theme so I'll change it.
I think I'll make several variants of the accent color, including lower contrast one like your second image.

I merged the theming work to master, this was fun to iterate on. I ended up shipping 4 theme bases (Light/Dark/Blue/Brown) and a color picker for the accent color.
The settings are saved for each user in the Polaris DB, and adding new theme bases is very easy (https://github.com/agersant/polaris-web/commit/9e9dd56b0ac505230c3894161b7eded4063a72d3).
Here is a screenshot from each base:

If you are setup to run Polaris from latest, I would love to hear some feedback before this makes it into a release.


Add custom scripts to MV2 Webextension

We need to add UserALE.js filters to the Webextension so that we don't log EVERYTHING.
Example Custom Script:
window.userale.filter(function (log) {
    var type_array = ['mouseup', 'mouseover', 'mousedown', 'keydown', 'dblclick', 'blur', 'focus', 'input', 'wheel'];
    var logType_array = ['interval'];
    return !type_array.includes(log.type) && !logType_array.includes(log.logType);
});

Note objects like "window" may not exist.
Investigate whether these should be added directly to Background or Content pages, or attached as other scripts.
https://developer.chrome.com/docs/extensions/mv2/getstarted/

add ref to 'custom.js' in rollup.config.js @ https://github.com/UMD-ARLIS/incubator-flagon-useralejs/blob/534ff9ff3e6586d1cf1d5b9718945d46479a3af3/rollup.config.js#L67
in order to appropriately transpile custom script

see also: https://github.com/UMD-ARLIS/incubator-flagon-useralejs/issues/12#issuecomment-1244277428

try using https://github.com/UMD-ARLIS/incubator-flagon-useralejs/blob/534ff9ff3e6586d1cf1d5b9718945d46479a3af3/example/webpackUserAleExample/index.js#L49
as example for scripting useralejs API call from 'custom.js'

The custom filter should be working now! You were right, the issue was rooting from the fact that it wasn't transpiling correctly! I just pushed MV2 up, and will work on MV3 now.


Function insert is missing from the Wrapping behaviour

I couldn't find any reason for this to be the case in the documentation or the issue tracker, so I'm guessing it got missed? If there is a reason that insert is missing for the Wrapping behaviour, it would be good to add it to the docs.

I apologize for the delayed response. I missed the notification and am only just now getting back to you. The missing inesrt  is intentional. The reason for this is that it's undecided which side of the element should be popped when the deque is full.
Think of a deque [a, b] whose capacity is 2:
   a      b
^     ^      ^
0     1      2

insert an element to index 2 is the same as to index 0, therefore popping a or b could both be reasonable.


in FASTFortranVisitor, #visitFASTFortranTLabelledStatement: does not exist but is called

it is called in many statement visit methods
Note: in FASFortranExporterVisitor is has been replaced by #visitFASTFortranAbstractStatement:

It's no longer the case


Show-hint overflow in 5.16

Commit df3a0ae0fb7876a4ef58c324878e6eea03b0d66f introduced max-width: 19em; to the .CodeMirror-hints class, and removed it from the .CodeMirror-hint class.
If any user of the dialog has previously styled the dialog with some width value, it now has to fight the max-width declaration, too. In my app this caused a lot of font overlapping:

I sort of get why this change was made, but would it be simpler to just have a normal width declaration?

A plain width property would, depending on the value we give it, either always make the dialog huge, or make it needlessly small for short hints. I don't like that solution very much either. Doesn't just setting max-width: none in your CSS provide a proper solution?

It does, but it also feels like a bit of a hack. Is no max-width at all a sane default for CodeMirror? If not, why not a much higher one?


No video in headset (LINUX STEAM DECK)

Description
When i connect to my Quest 2, my headset can give all its controller and headset data, but can't display anything inside the headset.

Hardware
My steam deck, An Oculus quest 2 256gb launch model.
CPU:
Steam deck built in Amd CPU
GPU:
Integrated graphics, (Very powerfull, but lacks VRAM)
GPU Driver Version:
Latest
Audio:
None (I use bluetooth headphones)
Installation
ALVR Version:
Latest
ALVR Settings File:
Default
SteamVR Version:
Latest, previous builds
Install Type:

[ ] Portable (zip)

OS Name and Version (winver on Windows or grep PRETTY_NAME /etc/os-release on most Linux distributions):
Steam OS

Did you  add ~/.local/share/Steam/steamapps/common/SteamVR/bin/vrmonitor.sh %command% to the commandline options of SteamVR (SteamVR -> Manage/Right Click -> Properties -> General -> Launch Options) ?

Also please keep in mind

Will Steam Deck work with PC VR headsets?
While technically a PC VR headset can be connected, the Steam Deck is not optimized for PC VR experiences.

Quote from Steamdeck.com