Remove gratuitous JSON.stringify from parseHandles utility

Description
We added the encode and decode functions onto IFluidSerializer a few years back, and updated the utility makeHandlesSerializable to use encode, but left the inverse utility parseHandles using the old (less efficient) way of doing serializer.parse(JSON.stringify(value)).
This PR switches to using serializer.encode which does an object walk itself, and avoids translating through string.
Reviewer Guidance
I'm assuming there is adequate test coverage of this code path.  I also haven't run any benchmarks - @DLehenbauer any risk that somehow this is slower than the roundtrip through string?
Merge-Tree fuzz test failures
Merge-Tree fuzz tests seeds 29 and 47 are failing with assert 0x04f - "minSeq of collab window > target minSeq!".  This is surprising but they both failed for two runs in a row, so something is going on here.  I'm not able to look into it for now so converting to a draft PR.

@connorskees - Funny - this doesn't fail those fuzz tests anymore...
@DLehenbauer, @Abe27342 - Any thoughts on whether this is a safe change to make?

@connorskees - Funny - this doesn't fail those fuzz tests anymore...
@DLehenbauer, @Abe27342 - Any thoughts on whether this is a safe change to make?

I'd want to understand the source of failure first. Do you remember exactly which tests were failing? i.e. were they the ones in sequence? or some of the test farms in merge-tree?
We've recently changed the scenarios fuzz tests exercise to support the stashed op flow, so ones built on top of the DDS fuzz harness won't be exercising the same scenarios they did 1 month ago, but the issue could still be prevalent (especially if it was only 2 seeds)

https://github.com/microsoft/FluidFramework/pull/19242 may have had an effect in making those tests no longer fail

@Abe27342 can you link the commit(s) that changed the fuzz harness?  I'd like to try the combo of Connor's PR + old fuzz harness and see which tweaked it.

@vladsud - This is one gratuitous string roundtrip that I was talking about this morning.

@markfields #20508 and various follow-ups to use random.handle() in different packages (these are made by Jillian, check on github), #20358, #19802, #19701, etc.... It'll be hard to narrow down

Is there some write up that tells us where we were, where we are today, and what else we can do to make things better?
My main concern - when I did profile matrix 2 months back (at large scale), almost all the time in profiler was handle parsing.
Should it be better now? I'll re-meassure, but having some English narrative somewhere with status would be very useful (including assessment of possible next steps and SWAGs)

Is there some write up that tells us where we were, where we are today, and what else we can do to make things better? My main concern - when I did profile matrix 2 months back (at large scale), almost all the time in profiler was handle parsing. Should it be better now? I'll re-meassure, but having some English narrative somewhere with status would be very useful (including assessment of possible next steps and SWAGs)

I'll write down e2e op submit/processing perf benchmarking / optimization for next semester planning.

Closing this in favor of #21102. At this time, not planning to dig into the fuzz test failures.  But open to feedback that we should prioritize that investigation.  It's hard for me to imagine where we would have a correctness dependency on the LHS here - we've been using the serializer.encode in production for a long time which does the same object walk.  And @jzaffiro added serialier.decode calls to the DDS fuzz tests last month, and we aren't seeing problems since then.


Add isDragged to Draggable

Description
Just like with Hoverable's isHovered, for Draggable a isDragged would be very useful.
Checklist
Before you create this PR confirm that it meets all requirements listed below by checking the relevant checkboxes ([x]). This will ensure a smooth and quick review process.

[x] I read the Contributor Guide and followed the process outlined there for submitting PRs.
[x] My PR includes unit or integration tests for all changed/updated/fixed behaviors and are passing (See Contributor Guide).
[x] My PR does not decrease the code coverage, or I have a very special case and explained on the PR description why this PR decreases the coverage.
[x] I updated/added relevant documentation (doc comments with ///) and updated/added examples in doc/examples.
[x] I have formatted my code with ./scripts/format.sh and the Flame analyzer (./scripts/analyze.sh) does not report any problems.
[x] I read and followed the Flame Style Guide.
[x] I have added a description of the change under [next] in CHANGELOG.md.
[x] I removed the Draft status, by clicking on the Ready for review button in this PR.
[x] I am willing to follow-up on review comments in a timely manner.

Breaking Change
Does your PR require Flame users to manually update their apps to accommodate your change?

[ ] Yes, this is a breaking change (please indicate a breaking change in CHANGELOG.md).
[x] No, this is not a breaking change.

LGTM, would be nice to have a test asserting the new field too

Good idea, done!


Ica overlay tmax documentation

Reference issue
Fixes #8987.
What does this implement/fix?
Fixed the documentation to say tmax = None defaults to 3s

thx @sfc-neuro


Merge UAT to Master 18 Sep 2018

Coverage decreased (-0.002%) to 78.671% when pulling 94d5d545f107c5da026724cf69b63380344be712 on uat into 4d55f478e21a50c7c0b6871be5093bee545a0a5f on master.


chore(curriculum): add quotes to strings in Number Sorter

Checklist:

[x] I have read and followed the contribution guidelines.
[x] I have read and followed the how to open a pull request guide.
[x] My pull request targets the main branch of freeCodeCamp.
[x] I have tested these changes either locally on my machine, or GitPod.

Related to #51036

We don't necessarily want to put every string in quotation marks.
So to my understanding we want things like this to be quoted:

Assign the string "hello" to the input value of the input element.
Assign the class value "a-class-with-an-index-{i}" to the variable of let index


Bracketed-paste mode

[x] I have searched the issues of this repo and believe that this is not a duplicate

Hi! This is a feature-reqest to support bracketed-paste mode in Hyper.
The basics:

Upon receiving DECSET 2004 (ESC+[?2004h), Hyper should enter “bracketed-paste mode”.
During this, any ⌘V or click-and-drag should …

send ESC+[200~,
send the pasted text,
then send ESC+[201~.

Upon receiving DECRST 2004 (ESC+[?2004l), Hyper should then exit this mode, if it was previously in it.

This is supported by xterm, urxvt, gnome-terminal, iTerm 2, Terminal.app, so on and so forth.

As an extra reference, right now Hyper is vulnerable to the attack on this proof of concept:
http://thejh.net/misc/website-terminal-copy-paste
The same code with iTerm 2, zsh and oh-my-zsh does not suffer from this vulnerability on my setup.


Fixed typo on Data Compression doc

Changed the data compression option for adding archival compression. It is correct in the examples, but in the description before the examples it stated COLUMNSTORE instead of COLUMNSTORE_ARCHIVE.

@jpomfret : Thanks for your contribution! The author, @MikeRayMSFT, has been notified to review your proposed change.

@jpomfret thank you for your contribution. #sign-off


Adds APPROTECT security check and unlock on nRF52

As discussed in #775
With these changes, auto_unlock works with nRF52 targets:
(venv) $ pyocd commander --target nrf52840
0000228:WARNING:target_nRF52:NRF52840 APPROTECT enabled: will try to unlock via mass erase
Connected to NRF52840 [Halted]: I3FSNZOV
>>>

and the commander has reasonable UX around the function:
(venv) $ pyocd commander --target nrf52840 -N
>>> initdp
>>> status
Security:       Locked
>>> unlock
>>> status
Security:       Unlocked
>>> reinit
>>> status
Security:       Unlocked
Core 0 status:  Halted
>>>

Tested locally against an nRF52 DK and and nRF52840 DK using CMSIS DAP.

Brief reminder that this doesn't work with ST-Link v2 firmware > 2J29S7 . IIRC, that's not a deal breaker because the combination isn't common... All the testing was with LPC-Link2.

/morph test

Documentation content is preliminary. Wouldn't mind comments on the structure and scope. Seems like it'll be a primer on commander, but I'm open to refactoring.
Will flesh out content and squash the commits before merge.

Thanks, I really appreciate the docs. Even preliminary/draft docs are far better than none.

/morph test

I'm pretty happy with the scope of the documentation now.
I pared it down to a brief discussion on security features instead of a primer on pyOCD commander. I'll reread it again tomorrow for spelling grammar, but it's probably quite near final form.

Great, thanks. Just let me know when you're happy and I'll merge.

Great, thanks. Just let me know when you're happy and I'll merge.

I'm happy with it now. Thanks.

Really great document, thanks 😄 . There's only the one comment about auto unlocking on commander versus the other commands that should be clarified.


[SPARK-45257][CORE][FOLLOWUP] Correct the from version in migration guide

What changes were proposed in this pull request?

Correct the from version in migration guide
Why are the changes needed?

Address comments  https://github.com/apache/spark/commit/8d599972872225e336467700715b1d4771624efe#r128053622
Does this PR introduce any user-facing change?

No
How was this patch tested?

Review
Was this patch authored or co-authored using generative AI tooling?

No.

thanks, merged to master


flutter.io/setup/ should say to make a permanent change to your path

We suggest several times running commands that update the PATH environment variable, but we never actually tell the developer to make those changes permanent. We just give the command that will make it work that one time. We should either give a command that actually has a permanent effect, or, more likely, just say that doing so is recommended so that you don't have to keep resetting the PATH.

This has regressed. The /setup/ page no longer makes a permanent change to your path anymore.

@LarkAscending @lukechurch did we do this? do we plan on doing this?

This issue was actually a feature request for Flutter, so I didn't close it.
But yes, https://github.com/flutter/website/pull/413 applies a documentation patch.

Great, thanks!
If the tool needs to do something more, let's open an issue for that.

Could be wrong but I believe the fix is an installer...and we don't have an issue for that.


Could SilkierQuartz tartget net standard 2.1 ?

Hi guys,
I just try it out with a cluster situation and this repo perform pretty awesome, and i was really serious planning to bring this to production.
But could be it possible to support target framework net standard 2.1 ?
The reason behind this is i was come accross some of Quartz management UI repo and they was not quite actively after a while.
And i plan to upgrade my project for quite a long time.
So to ensure to break nothing, target net standard 2.1 will be pretty nice, or otherwise, moving from net5 to net6 and net6 to net8 would be painful or may be the worst case is to drop support to this one.
Thanks to bring us this awesome tool anyway :D

That's a good idea!，Can you submit a PR?

https://github.com/maikebing/SilkierQuartz/pull/66
Here you go bro, along with result after it work, but take the consider as i was noted in detailed there.
Happy weekend 🎉 🎉

Thank you for your work. But conflicts with #65
Because
Net Standard 2.1：
public JsonResult(object value, JsonSerializerSettings serializerSettings);
Net 5.0 :
public JsonResult(object value, object serializerSettings);

This issue cannot be solved. and I'm closing it now.


(Windows specific) Some files won't extract because of case sensitivity issues

When trying to extract jacket.ifs from pop'n music there are 3 files which can't be extracted on Windows due to the lack of case sensitive filenames. To give an example, jacket.ifs contains both ha_floccinau.ifs (the real jacket) and ha_Floccinau.ifs (a beta jacket). When extracted, there will only be a single ha_floccinau.ifs, which is actually ha_Floccinau.ifs because it overwrote ha_floccinau.ifs.
Extracting the same jacket.ifs works fine on *nix and gives both ha_floccinau.ifs and ha_Floccinau.ifs as expected.

This is literally impossible to fix cleanly, and has been a problem with zip/tar extraction on windows since time immemorial. It is a problem with NTFS itself.

Impossible to fix cleanly, sure, especially when it comes to repacking, but it would be nice to have warnings when this occurs or a flag to at least allow for renaming of collisions. As it is, ifstools silently drops data without any warnings which is not good.

Fair enough. I'll think of a way to accomodate.

Can I get a copy of the ifs for testing?

@mon I've prepared a small sample IFS file that shares the same properties.
https://www.mediafire.com/file/fvhqgyvtdzki2kt/test.ifs/file
> ifstools test.ifs
test.ifs
Extracting...
b.txt
B.txt
a.txt
A.txt
100%|██████████████████████████████████████████████████████████████████████████████████| 4/4 [00:00<00:00, 2046.75it/s]

> dir test_ifs
05/02/2020  09:35 PM    <DIR>          .
05/02/2020  09:35 PM    <DIR>          ..
05/02/2020  09:32 PM                 5 a.txt
05/02/2020  09:32 PM                 5 b.txt
               2 File(s)             10 bytes
               2 Dir(s)  81,690,640,384 bytes free

Should be fixed in 57e909572e4f250143b4f96fcbe4db4e3f3a6a43 via the new --rename-dupes flag, please test it and if it's working fine I'll make a new release, build exes etc.
Also open to any last-minute changes to the help text, argument name, or rename suffix.

Tested using the test.ifs and problem jacket.ifs and they both 1) give a warning when extracting without --rename-dupes, and 2) properly attach (1) numbering to the duplicate filenames.
LGTM.
Thanks @mon!


Update the Choreo Connect diagram

Purpose

To adhere to the latest Choreo diagram standards.

Corrects https://github.com/wso2-enterprise/choreo/issues/5110

Corrects wso2-enterprise/choreo#5110

Is this the correct issue to link here?


[WIP] address parsing

Here's an initial look to get an idea of what my address parsing changes would do. Strategy is to parse through the class+protocol pair, then leave the remainder for the NA layer. MPI and CCI already ignore the host name, so there was not much to do there. BMI requires reconstructing the "protocol://hostname:port", but that's a fairly simple piece of code.
Note that addr_lookup in the top-level NA layer just peeks at the class name, and passes the remainder down to the plugin. The BMI and CCI plugins then pass them directly to the libs, while MPI parses the port string starting from ';', effectively ignoring the "protocol://" portion.
TODO:

[x] Modify tests to init clients in anon mode ("class+protocol"). Currently passes in the server string to initialize.
[x] Modify tests to always get the server address through file. Right now it does sometimes but not others. Will be simpler to just have one code path, then we can test things like having anon-mode servers. This actually isn't the case, I misread the code.
[x] Get rid of hard-coded tcp default protocol in tests (or just special case MPI to "dynamic") - use the NA_*_TESTING_PROTOCOL variables for this instead.
[x] Get rid of default protocol check in test code, enforce given protocol
[x] Turn MPI error checks (bad protocol, existence of hostname during initialization) into errors once tests are changed. They are currently warnings.
[x] Rebase on Phil's cci_create_endpoint_at branch once it gets merged, and incorporate call into code.
[x] Special-case address creation in test code. BMI, CCI+tcp(+verbs?) - hostname:port. CCI+sm - pid:tag (need to check format). MPI: nothing. Right now, hardcodes host string to hostname:port unless using MPI.
[x] Similarly, modify BMI to allow anon-mode servers? This might require a BMI change, if so, then just error out when no hoststring is provided at the NA layer. Not worth the trouble.

Might have forgot one or two...

I ljust ooked at it quickly so I may have missed one or two things but yes that looks good to me at least, thanks! Regarding the default protocol in the tests, I had also added these variables in CMakeLists.txt: NA_BMI_TESTING_PROTOCOL, NA_CCI_TESTING_PROTOCOL, etc
where you can pass a list of protocols that need to be tested per plugin, for example you can pass "tcp;sm" for CCI. Maybe that would be enough to just use that by default? and then we can just get rid of the default tcp protocol.

Sounds good to me!
One question is how to handle MPI static vs dynamic mode. There's some trickery involved in the test files / cmake to accommodate the different modes. Might inadvertently break something during these changes...

Yeah... well I guess that will be the occasion to clean it up :)

So I'll let this PR sit for a while while the create_endpoint_at stuff finalizes, then I'll modify the CCI NA to use it. The BMI anon server hostname stuff is probably not hard to accommodate in the BMI code, but let's just ignore it for now. After that, should be ready to go!

Oh, and as a result, we're now testing CCI+SM by default as well. More coverage!

Nice thanks! Yes I guess we can put tcp/sm by default, should be supported everywhere and user can still remove it anyway.

So I tried this PR out with CCI 2.0 and it failed :(. Specifically, it hung on the mercury_rpc_cci_sm test.  Works on mercury master though, which is weird since I didn't change anything in the CCI NA... Will look into this probably tomorrow.

Hmm ok, yes that seems weird...

On Jun 29, 2016, at 10:05 AM, John Jenkins<EMAIL_ADDRESS>wrote:
So I tried this PR out with CCI 2.0 and it failed :(. Specifically, it hung on the mercury_rpc_cci_sm test. Works on mercury master though, which is weird since I didn't change anything in the CCI NA... Will look into this probably tomorrow.

That is odd. Right now, master and 2.0 are identical.

Sorry, I meant mercury master, not cci (tests pass fine with cci 2.0 there).

Looked into this a bit more - the mercury test succeeded on mercury's git master b/c it wasn't testing sm by default (this PR does). So the issue seems to be with our use of CCI. I'll make a mercury issue for this.

I've fixed up the test address generation code and tests are back to green locally. Before a pull happens, want to summarize where we are and propose a simplification.
Initialization
When not listening, any plugin can pass in an address string of the form [<class>+]<protocol>[://]". I.e. no hostname/port expected. (the []` means optional here).

BMI ignores passed hostnames AFAICT.
CCI will try to create its endpoint with whatever you give it, so non-listeners can request a specific port, for example.
MPI expects no hostname and errors out otherwise.

In listen mode, the expected hostname is plugin dependent.

BMI expects <host>:<port>
CCI falls back to cci_create_endpoint when a host string isn't provided, using an OS assigned port for TCP,Verbs and the process PID for SM.
CCI+TCP,Verbs expect <port> if an addr is provided.
CCI+SM expect <id1>:<id2> if an addr is provided.

We can simplify the BMI,CCI split by changing the BMI NA to just expect a port and appending "localhost" under the covers, as there's no other valid option. Thoughts?
Lookup
Nothing has changed here. Class still optional, and "full" host strings are expected.

Does id (below) mean process id? I hope you can clarify it with concrete example for CCI+SM case (and others).

•CCI+SM expect  :  if an addr is provided.

Thanks, that looks good to me. Yes I would be fine with BMI just having localhost...

AFAICT, the first id being a PID is just a convention used as a default in CCI. See README.ctp.sm in CCI for more info.

Okay, I'll change BMI over and push the changes. Once all are OK with them, I'll rebase my branch to a single commit and you can pull.

So I have a better understanding of the test suite now :). Should be ready to go. Would someone like to give it a try locally?
Also, where should these formatting options be documented? Doesn't seem like it fits cleanly in the doxygen setup...

Ok thanks! let me give it a try. I think it should be documented on the website, or alternatively on the wiki for now?

Over there? http://mercury-hpc.github.io/documentation/#network-abstraction-layer

CMake Error at Testing/na/CMakeLists.txt:38 (if):
if given arguments:
 "bmi" "STREQUAL" "mpi" "AND" "STREQUAL" "static"

An empty protocol is being passed in. Not sure why that would be the case...

Yes that's my fault :) some leftovers. I fixed it now, everything looks good!

Excellent, I'll squash my commits and push.

OK wait, can you also fix the unused variables before you do that?

/home/jsoumagne/workspace/mercury/source/Testing/na/na_test.c: In function ‘na_test_gen_config’:
/home/jsoumagne/workspace/mercury/source/Testing/na/na_test.c:201:10: warning: unused variable ‘na_hostname’ [-Wunused-variable]
char na_hostname[NA_TEST_MAX_ADDR_NAME];
^
/home/jsoumagne/workspace/mercury/source/Testing/na/na_test.c: In function ‘NA_Test_server_init’:
/home/jsoumagne/workspace/mercury/source/Testing/na/na_test.c:493:17: warning: variable ‘nret’ set but not used [-Wunused-but-set-variable]
na_return_t nret;
/home/jsoumagne/workspace/mercury/source/src/na/na_cci.c: In function ‘na_cci_initialize’:
/home/jsoumagne/workspace/mercury/source/src/na/na_cci.c:445:11: warning: unused variable ‘string_port’ [-Wunused-variable]
char *string_port = NULL;
^
/home/jsoumagne/workspace/mercury/source/src/na/na_cci.c:437:15: warning: unused parameter ‘listen’ [-Wunused-parameter]
na_bool_t listen)

Huh, I didn't see those warnings. I'll clean those up.

Oops, didn't have warnings set...

OK thanks! no problem ;)

Okay, changes are in one happy commit. Let's see what travis thinks...

It is going to complain because I haven't updated cci there... let me fix that

I'll do that on this branch, as it will fail on the current master.

No the current master uses the new cci now

Just give me 2 seconds because I also need to clear the cache for travis

Okay, let me know when I should re-rebase.

No wonder it's taking a long time for travis to run - you're building gcc from scratch every time :-P. Is that really necessary?

:) haha no it's not building from scratch every time, it builds it once and then uses the cache. Yeah unfortunately that's the only solution I found to be able to use tsan etc because ubunta packages are just buggy

That's good to know, though I was under the impression travis builds proceeded under a "clean" environment. Is there an option to hold onto a workspace similar to e.g. jenkins?
I take it the PPA route wasn't helpful?

I pushed now but it may be better to wait until the first build is done so that your pull request can use the build cache. Yes the environment is clean but you can cache dependencies so that you don't have to build them every time. In that case I specify a directory that I want to cache.
Yeah the other problem with travis is that repositories need to be white listed (which is not something I control) and unfortunately none of them have the right packages :'(

Ah ok that's a nice feature.
I'll probably do the rebase + test tomorrow morning, I'll be taking off soon.

Yeah it usually takes about 45 minutes if we build gcc as well ;)

Well sorry, I forgot that the tarball has configure files already but not the repo... so have to rebuild again

OK it passed this time so you can rebase now when you want

I merged #72 into master so just as a heads up you'll get a conflict when you rebase, you may want to squash first and then rebase... although the conflict should be easy to solve, it's only the cci_create_endpoint() stuff...

Thanks, I'll patch things up and give it a go.

The changes I needed to make are purely cosmetic. I've run into a problem though: mercury_bulk_cci_tcp with NA_CCI_USE_POLL hangs. I'll investigate further.

Yes this is normal, we are missing a fix in CCI for the tcp plugin so it will hang for now if you have NA_CCI_USE_POLL enabled, see comment in #72

Ah, thanks for the clarification, should've checked there first. I'll retry the tests with NA_CCI_USE_POLL and without cci+tcp and if that succeeds I think we're good to go.

Hooray, tests are green! It's been quite a journey to just handle addresses a little differently :).

Merged ;)

Looks like there are some errors with tsan
=================================================================
==22274==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 22 byte(s) in 1 object(s) allocated from:
    #0 0x7fb26ac0197a in malloc (/lib64/libasan.so.2+0x9897a)
    #1 0x40b538 in NA_Test_server_init /home/jsoumagne/workspace/mercury/source/Testing/na/na_test.c:442
    #2 0x405e86 in HG_Test_server_init /home/jsoumagne/workspace/mercury/source/Testing/mercury_test.c:323
    #3 0x403247 in main /home/jsoumagne/workspace/mercury/source/Testing/test_server.c:57
    #4 0x7fb26905657f in __libc_start_main (/lib64/libc.so.6+0x2057f)

Direct leak of 16 byte(s) in 1 object(s) allocated from:
    #0 0x7fb26ac0197a in malloc (/lib64/libasan.so.2+0x9897a)
    #1 0x7fb26a2e0d5d in na_bmi_addr_self /home/jsoumagne/workspace/mercury/source/src/na/na_bmi.c:866
    #2 0x7fb26a2d61f0 in NA_Addr_self /home/jsoumagne/workspace/mercury/source/src/na/na.c:629
    #3 0x40b4b6 in NA_Test_server_init /home/jsoumagne/workspace/mercury/source/Testing/na/na_test.c:436
    #4 0x405e86 in HG_Test_server_init /home/jsoumagne/workspace/mercury/source/Testing/mercury_test.c:323
    #5 0x403247 in main /home/jsoumagne/workspace/mercury/source/Testing/test_server.c:57
    #6 0x7fb26905657f in __libc_start_main (/lib64/libc.so.6+0x2057f)

SUMMARY: AddressSanitizer: 38 byte(s) leaked in 2 allocation(s).

Same with CCI

https://cdash.hdfgroup.org/viewTest.php?onlyfailed&buildid=20630

And failures on OSX now, not sure what's going on... https://cdash.hdfgroup.org/index.php?project=MERCURY

Leaks shouldn't be too hard to fix. I'll look at that now. Not sure what's up with the OSX stuff.

Closing that now since discussion was moved to #114


PODAAC Solr search server down

The server @ http://podaac.jpl.nasa.gov/dmasSolr/solr/ is not up yet.
I have posted a thread in the forum @ https://podaac.jpl.nasa.gov/forum/viewtopic.php?f=6&t=367
We need to await the reply in the thread and if there is none then I think we need to find an alternative.

Thanks Omkar nice
On Wednesday, June 15, 2016, Omkar<EMAIL_ADDRESS>wrote:

The server @ http://podaac.jpl.nasa.gov/dmasSolr/solr/ is not up yet.
I have posted a thread in the forum @
https://podaac.jpl.nasa.gov/forum/viewtopic.php?f=6&t=367
We need to await the reply in the thread and if there is none then I
think we need to find an alternative.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/lewismc/podaacpy/issues/25, or mute the thread
https://github.com/notifications/unsubscribe/ABHJlz9UXGax8hxQfof7RdBxsnU13Xopks5qL7zpgaJpZM4I2IZu
.

--
Lewis

So as it turns out the server is not down at all... you are just using the wrong server!
The following lines need to be changed ASAP
https://github.com/lewismc/podaacpy/blob/master/podaac/podaac.py#L455
https://github.com/lewismc/podaacpy/blob/master/podaac/podaac.py#L485
Please update this PR ASAP thanks @Omkar20895

Yeah sure.

@lewismc This issue has been solved and I am closed it. Thanks.

Thanks
On Thursday, June 16, 2016, Omkar<EMAIL_ADDRESS>wrote:

Closed #25 https://github.com/lewismc/podaacpy/issues/25.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/lewismc/podaacpy/issues/25#event-694564744, or mute
the thread
https://github.com/notifications/unsubscribe/ABHJl3R5PL4N9iZc1nxdmac-sDXqCjBbks5qMUOAgaJpZM4I2IZu
.

--
Lewis


Items: Add struct initializations methods

Fixes #64
These methods are useful to mirror the default values that are set on objects in the backend. Also, all map and slices are initialized with zero length to allow calling append.

The reviewer is on PTO. Since the tests are passing I will merge this without a review.


Stock Span Problem

Question : The stock span problem is a financial problem where we have a series of n daily price quotes for a stock and we need to calculate span of stock’s price for all n days.
The span Si of the stock’s price on a given day, i is defined as the maximum number of consecutive days just before the given day, for which the price of the stock on the current day is less than or equal to its price on the given day.

Can you assign me this issue
Language - C++
Time complexity-O(n)


Strange huge diff atol value when optimizing a model

System Info
optimum: 1.7.3
python: 3.10.10
OS: Linux Ubuntu 22.10
ORT-GPU: 1.14.1
Pytorch: 2.0.0
transformers: 4.27.4

Who can help?
@michaelbenayoun @fxmarty
Information

[ ] The official example scripts
[ ] My own modified scripts

Tasks

[ ] An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
[ ] My own task or dataset (give details below)

Reproduction
Here the command line I use:
optimum-cli export onnx -m cross-encoder/mmarco-mMiniLMv2-L12-H384-v1 --optimize O3 test/mmarco-mMiniLMv2-L12-H384-v1

Logs I get:
Framework not specified. Using pt to export to ONNX.
Automatic task detection to sequence-classification.
Using framework PyTorch: 2.0.0
Overriding 1 configuration item(s)
        - use_cache -> False
================ Diagnostic Run torch.onnx.export version 2.0.0 ================
verbose: False, log level: Level.ERROR
======================= 0 NONE 0 NOTE 0 WARNING 0 ERROR ========================

/home/jplu/miniconda3/envs/cluster-inference/lib/python3.10/site-packages/optimum/onnxruntime/configuration.py:735: FutureWarning: disable_embed_layer_norm will be deprecated soon, use disable_embed_layer_norm_fusion instead, disable_embed_layer_norm_fusion is set to True.
  warnings.warn(
Optimizing model...
symbolic shape inference disabled or failed.
Configuration saved in test/mmarco-mMiniLMv2-L12-H384-v1/ort_config.json
Optimized model saved at: test/mmarco-mMiniLMv2-L12-H384-v1 (external data format: False; saved all tensor to one file: True)
Validating ONNX model test/mmarco-mMiniLMv2-L12-H384-v1/model.onnx...
        -[✓] ONNX model output names match reference model (logits)
        - Validating ONNX Model output "logits":
                -[✓] (2, 1) matches (2, 1)
                -[x] values not close enough, max diff: 4.666891098022461 (atol: 0.0001)
The ONNX export succeeded with the warning: The maximum absolute difference between the output of the reference model and the ONNX exported model is not within the set tolerance 0.0001:
- logits: max diff = 4.666891098022461.
 The exported model was saved at: test/mmarco-mMiniLMv2-L12-H384-v1

I get an atol for 4.6 which is really huge. Any idea why? Same diff warning happens with 02 , O4 and with --device cuda  --fp16.
Expected behavior
Having a reasonable atol diff between the original model and the exported ONNX one.

I do not have any issue on my side until O3. Something in the optimization seems to mess up with the accuracy.

Humm this is pretty strange. I did run it 10 times in a row and I always got a diff > 4 for the 10 runs. Anything I can check on my side to see if a setting or something is wrongly set?

I do not observe those big differences. Thta's weird. Can you reinstlal optimum?

It could be an accuracy issue in an ORT op as well. If you'd like to investigate, it can be worth to compile ORT from source with the ORT_DEBUG_NODE_IO_DUMP_OUTPUT_DATA=1 environment variable.

@michaelbenayoun I did reinstall optimum to 1.8.3dev0 and still the same issue with also 5 other models. Now it is even worse, I get atol diff of around 7.
@fxmarty I don't have the proper tools installed to build ORT on the machine I'm currently doing these tests and I'm not admin so it can take several days to have ORT compiled, as IT won't take my demand as a big priority. Any other thing I can do to help debugging this?

My CUDA version is 11.8, this might be the reason? Here I see that the CUDA version for 1.14 is 11.6.

I have tested on three different machines with brand new fresh python environment via conda, they all have a different configuration with different kind of GPU, and I still have the same issue on these three machines.

@jplu I would advise to open a reproducible issue on ORT repo with minimal dependency on optimum.

@fxmarty Ok thanks, I will try to figure this out, how to properly reproduce without optimum.
Also, any chance you can share what is your env on which you are running this without issue, please? Such as ORT version (compiled or not?), optimum, Python and CUDA version? Maybe I can try the same.

@fxmarty I think there is an issue in the way optimum compute the diff. Here the small experiment I have done:

Create an ONNX export without optimization:

optimum-cli export onnx -m cross-encoder/mmarco-mMiniLMv2-L12-H384-v1 no_optim/

Here validation is ok:
Validating ONNX model test/mmarco-mMiniLMv2-L12-H384-v1/model.onnx...
        -[✓] ONNX model output names match reference model (logits)
        - Validating ONNX Model output "logits":
                -[✓] (2, 1) matches (2, 1)
                -[✓] all values close (atol: 0.0001)

Create an ONNX export with the O2 optimization:

optimum-cli export onnx -m cross-encoder/mmarco-mMiniLMv2-L12-H384-v1 --optimize O2 o2_optim/

Here the validation fails:
Validating ONNX model test/mmarco-mMiniLMv2-L12-H384-v1_o2/model.onnx...
        -[✓] ONNX model output names match reference model (logits)
        - Validating ONNX Model output "logits":
                -[✓] (2, 1) matches (2, 1)
                -[x] values not close enough, max diff: 6.75302267074585 (atol: 0.0001)

doing the diff manually:

import onnxruntime as ort
import numpy as np
from transformers import AutoModelForSequenceClassification, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("cross-encoder/mmarco-mMiniLMv2-L12-H384-v1")
model_pt = AutoModelForSequenceClassification.from_pretrained("cross-encoder/mmarco-mMiniLMv2-L12-H384-v1")
ort_sess_no_optim = ort.InferenceSession('no_optim/model.onnx', providers=['CPUExecutionProvider'])
ort_sess_o2 = ort.InferenceSession('o2_optim/model.onnx', providers=['CPUExecutionProvider'])
inputs_ort = tokenizer("bla", return_tensors='np')
inputs_pt = tokenizer("bla", return_tensors='pt')
outputs_no_optim = ort_sess_no_optim.run(None, dict(inputs_ort))
outputs_o2 = ort_sess_o2.run(None, dict(inputs_ort))
outputs_pt = model_pt(**inputs_pt).logits.detach().numpy()
print(f"Are ORT no optim and PT close enough? : {np.allclose(outputs_no_optim, outputs_pt, atol=1e-4)}")
print(f"Are ORT O2 and PT close enough? : {np.allclose(outputs_o2, outputs_pt, atol=1e-4)}")
print(f"Are ORT no optim and ORT O2 close enough? : {np.allclose(outputs_no_optim, outputs_o2, atol=1e-4)}")

And here the output I have:
Are ORT no optim and PT close enough? : True
Are ORT O2 and PT close enough? : True
Are ORT no optim and ORT O2 close enough? : True

They are all close, while when I do the convert with the O2 optim with the command line I get something very far to close.


The new layer selection dialog should be closeable with Ctrl+W just as the other windows.

I admire the problem with the non-triggering actions. One might hack this for the upcoming release:
    def keyPressEvent(self, event):
        super().keyPressEvent(event)
        if event.key() == QtCore.Qt.Key_W and event.modifiers() == QtCore.Qt.ControlModifier:
            self.reject()

The dialogue can be closed with "ESC", btw, which also rejects the dialogue. It is IMO not obvious that this is a dialogue, because it is not modal. Maybe it should be derived from MainWindow instead...?

This here is probably better:
--- a/mslib/msui/multilayers.py
+++ b/mslib/msui/multilayers.py
@@ -75,6 +75,9 @@ class Multilayers(QtWidgets.QDialog, ui.Ui_MultilayersDialog):
         self.listLayers.setColumnHidden(2, True)
         self.listLayers.header().setSectionResizeMode(0, QtWidgets.QHeaderView.Stretch)

+        self.close_shortcut = QtWidgets.QShortcut(QtGui.QKeySequence("Ctrl+W"), self)
+        self.close_shortcut.activated.connect(self.reject)
+
     def remove_filter_triggered(self):
         self.leMultiFilter.setText("")
         if self.filter_favourite:

Would still be interesting to know why the "standard way" does not work.

Thanks for looking into this yourself.
The reason shortcuts aren't working seems to me that actionCloseWindow is never actually added to the window, and I am not sure why. Maybe I am using Designer wrong?
So basically this line is missing in ui_wms_multilayers.ui:
<addaction name="actionCloseWindow"/>
Adding it manually is making it work again.

Mmh. I had no luck doing that myself in Creator. I remember now having had the same problem and - apparently? - coming to the same solution. Adter addding the element to the XML, it appears in creator, but I couldn't quickly figure out a way to do that in the GUI. I think, I could drag the action to the list of objects on the upper right, but am unsure about that. Or using some other dialog? mmh. Strange.
Thanks!

You can drag and drop Actions into menus through the Designer, e.g. in ui_mainwindow.ui. I think they are not meant to be used standalone without a menu.


Regression: ZADD scores are using exponents

The current version in master uses fmt.Sprint for float64 args, causing strings such as "1.590540802326684e+18":
https://github.com/go-redis/redis/blob/3d03b1fa6a1ed0775d4b032861da0637b45437da/command.go#L103
This is a regression caused by https://github.com/go-redis/redis/commit/0944d0167b5886c195a151a5553b9fbdff0815d7 where formatFloat got removed.
Expected:
client.ZAdd(ctx, "key", &redis.Z{1590540802326684000.0,"test"}) should result in Redis command ZADD key<PHONE_NUMBER>326684000 test.
Suggested fix: add a case float64: to func appendArg doing the same strconv.FormatFloat as before: strconv.FormatFloat(f, 'f', -1, 64)

The current version in master uses https://github.com/go-redis/redis/blob/master/internal/proto/writer.go#L86-L89 which should correctly append floats. Do you have code that demonstrates the problem?

Hmm, it's fairly easy to repro:
package main

import (
	"context"
	"github.com/go-redis/redis/v8"
)

func main() {
	client := redis.NewClient(&redis.Options{})
	client.ZAdd(context.TODO(), "test", &redis.Z{Score: 1e23, Member: "x"})
}

Result:
$ redis-cli zrange test 0 0 withscores
1) "x"
2) "9.9999999999999992e+22"


[core,fastpath] fix too verbose log

for fastpath_recv_update_synchronize only skip the available bytes as older servers tend to send short packets. This avoids (too) verbose logging.

@mfleisz expand the code above, there is already one.


The variable is not visible after the match statement.

This issue has been moved from a ticket on Developer Community.

On the Python 3.11 64-bit interpreter in Microsoft Visual Studio Enterprise 2019 Version 16.11.21. The variable is not visible after the match statement in the development environment. Throws an error in the context "unexpected token 'number'".

Original Comments
Feedback Bot on 12/11/2022, 06:46 PM:
(private comment, text removed)

Original Solutions
(no solutions)

Hi, could you provide a code sample for us to repro the issue? Also could you please try on VS 2022 and see if the issue still persists? Thanks!

Here is a sample code. Haven't tried the 2022 version. But I'll let you know as soon as I try it.

Updated to version 2022. There is no such error anymore.


Add german TIN validator

This PR will implements the Validor for German Tax Identification Number (TIN) mentioned in #47

For the source of the algorithm should i link the original repo that i found? Other documentation is mostly in German, so i do not know if that's of any help?

For the source of the algorithm should i link the original repo that i found? Other documentation is mostly in German, so i do not know if that's of any help?

It's fine if the documentation is in another language. Any (correct) source will do :pray:
If the documentation is enough, then we don't need to link to any other implementations imo

There is an official document by ELSTER (that is the german institute to make your tax declarations online). They have a downloadable pdf where the details are explained in Chapter 2.2.
https://download.elster.de/download/schnittstellen/Pruefung_der_Steuer_und_Steueridentifikatsnummer.pdf
Should I use the download URL  to this PDF or this a problem since download urls tend to disappear sooner or later?

There is an official document by ELSTER (that is the german institute to make your tax declarations online). They have a downloadable pdf where the details are explained in Chapter 2.2.
https://download.elster.de/download/schnittstellen/Pruefung_der_Steuer_und_Steueridentifikatsnummer.pdf
Should I use the download URL to this PDF or this a problem since download urls tend to disappear sooner or later?

It's fine to link to the PDF
That's a risk we're taking with all links - but at least one can derive some information from the URL itself and try to find another source :+1:

It's fine to link to the PDF
That's a risk we're taking with all links - but at least one can derive some information from the URL itself and try to find another source :+1:

Allright, i will link it.
I just saw that they have some example values as well for the TIN in the documentation, maybe i should use those in our tests as well?
There is a disclaimer below saying something like:
"The ID was taken from the concept of the Federal Information Technology Center and serves
solely for the purpose of illustrating the implementation. Under no circumstances may the ID be used for test or real transfers!"
I do not want to get you guys in trouble after all. The documentation said that for testing purposes an ID with a leading 0 should be used. I could disable the leading 0 check if the APP_ENV is set to TEST if you like. Alternatively i could at least use my TIN for the German Test if you like.

I do not want to get you guys in trouble after all. The documentation said that for testing purposes an ID with a leading 0 should be used. I could disable the leading 0 check if the APP_ENV is set to TEST if you like. Alternatively i could at least use my TIN for the German Test if you like.

We'll trust the algorithm and we'll trust you :smile:
In that case you can go ahead and use test IDs which they allow - no need for edge cases

So i can leave everything like it is? :D Sorry for the confusion

So i can leave everything like it is? :D Sorry for the confusion

Use invalid IDs that the German authorities allow - whatever is legal

So i will use testing IDs with a leading 0. That would then cause the test to fail because valid TIN's must not start with a 0. If you like i can at least use my own TIN to have one valid ID in the test.

So i will use testing IDs with a leading 0. That would then cause the test to fail because valid TIN's must not start with a 0. If you like i can at least use my own TIN to have one valid ID in the test.

Ahhhh I finally understood what you meant...
Well I'm not a fan of environment variables and workarounds for testing in packages.
I also find it weird that we're not allowed to use IDs when these don't actually encode any information (especially if you used a generator for them)
I say we keep it as it is. If someone asks us to remove them, we will :man_shrugging:

Ok perfect :+1: So this is all good now? :D

All good from my end!
@JoaoFSCruz como take a looksie

@all-contributors please add @bofalke for documentation


[ui] footer icon's

Feature / Issue
We need the icons designed by Ampillo to be included in the react native base icon set.
Acceptance criteria

[ ] Icon's should be ready to get consumed

Depends on

#47

I have the source images he sent. They are in three sizes (small, medium, large) and need to be converted to TTF. I'll assign this to me (looks to figure out how)


⚠ Adacta-fintech.com has degraded performance

In 6fb1c42, Adacta-fintech.com (https://www.adacta-fintech.com) experienced degraded performance:

HTTP code: 200
Response time: 911 ms

Resolved: Adacta-fintech.com performance has improved in 10c7c13 after 11 minutes.


Contact links for JMP

JMP page links below for Twitter and Facebook. Should I use<EMAIL_ADDRESS>or should we set up<EMAIL_ADDRESS>

Facebook: https://www.facebook.com/pages/Jewish-Moments-Project/1728422597383553?ref=hl
Twitter: https://twitter.com/JMomentsProject

@iCenterScott The email address wasn't a suggestion, it was only a placeholder. Let me know what you want it to be.

<EMAIL_ADDRESS>is good

merged


gRPC improvement: unary-unary interface to local collector

Is your feature request related to a problem?
Refer to issue https://github.com/PowerLoom/snapshotter-lite-local-collector/issues/20 on local collector
Describe the solution you'd like
Refer above.
Describe alternatives you've considered
NA
Additional context
NA

This has already been included in the releases since v1.6


Cargo time machine (generate lock files based on old registry state)

Some time ago I wanted to check how much faster my library has got in various Rust versions. So I cloned the repo and checked out an older git commit and used rustup to get an older rustc and tested it both with the older rustc and the newer one... it downloaded various dependencies and tried to build it (with the older rustc) but then it failed because apparently the crates on crates.io required newer Rust versions than the one I was benchmarking my library with. So I figured out a trick: I've told cargo to not use crates.io as a registry source but my own private clone, and I made that clone point to a commit from back when the compiler got released. This worked really well!
Now to my feature request. I'd like to have this automated, via a flag in cargo: if you invoke cargo generate-lockfile --registry-time 2017-01-01, cargo would check out a commit from that day from the registry and use that commit for lockfile generation.
I think it is justified to call this feature "time machine" because it emulates the time from back then.
Everyone who has missed the presence of a Cargo.lock can feel this I think :).

This sounds like it'd be really awesome as some kind of sub-command/cargo wrapper!
So you might run cargo time-machine bench --registry-time 2017-01-01 -p my-crate and it'll run benchmarks using the version of rustc and your crate closest to that date. You could use cargo time-machine install --registry-time 2017-01-01 -p my-crate for installation, and so on.

@Michael-F-Bryan as we've got other commands to influence cargo resolution like #4100 , I think the best place for integration is Cargo itself.

Is the publication date of a package stored somewhere? With that, it would be possible to filter package versions from ret just before the sort_unstable_by:
https://github.com/rust-lang/cargo/blob/7ba6e497be1b818a6ea4628f46a533a9c7c9f871/src/cargo/core/resolver/dep_cache.rs#L166-L187

Alternatively, to use a specific git hash one could modify this git fetch:
https://github.com/rust-lang/cargo/blob/058baec9d9bba1d3417881ae1d5efc27c84d956b/src/cargo/sources/registry/remote.rs#L220-L223

@fpoli the publication date is stored in git history, as in you have to find the commit that introduces the crate. Which is quite involved algorithmically and in time overhead as well. I recommend going via git commit hashes which is what I originally envisioned as well.

Could crate publication dates be detected once and added to the index data directly (with future additions adding the data automatically)? It should just be a blame for each entry, but that's mostly a guess not knowing the way the index is stored off-hand.

@mathstuf that wouldn't detect yanked/unyanked crates. It seems that you can yank and un-yank crates arbitrarily often.

Hmm. It seems that the index could be fetched from an arbitrary refspec.
See that https://github.com/mathstuf/rust-keyutils/blob/master%40{2020-01-01}/.cirrus.yml is returning valid contents. So, at least for github-hosted index files, this kind of URL abuse is possible. Not so sure about other index hosting locations though.

Hmmm nice it works from Github's API as well:
curl -i 'https://api.github.com/repos/est31/cargo-udeps/commits/master@\{2020-01-01\}'

It could be special cased for github with a fall back to a full clone of the index repo when it doesn't detect github or hits an API limit or other HTTP error.

I've implemented this:
https://lib.rs/crates/lts

We talked about this recently somewhere.  I wonder if it was in person at RustConf which means no notes.
I would expect this to be a part of an interface for cargo generate-lockfile.  It doesn't need to exist everywhere.
It can't fully reproduce a lockfile from a past state because the lockfile only resolves maximally for the subsection of the dependency tree that changed.  However, still being able to generate it for a given time can be useful.
When we had only the git registry, it would be easy to think we could use the git history until we take squashing into account.
Now we also have the sparse registry, so we'd need a design that can take that into account, including

What time resolution can work (or if we use some kind of counter, how to turn that into real-world measurement)
At what stage in the publish process do we capture that timestamp

Timestamps, instead of counters, seem really useful for dealing with the human side to this. I don't think we need fine resolution on this, if two packages are published in close succession, oh well.
I bet the server could even backfill the timestamps.
I don't think the specific stage will matter all that much either.
The big question though is yanks.  We don't have a history of when things have been yanked and unyanked.


[New Project]Want to add a maze solver in cpp-projects

Describe the project you want to add with tech stack
I will create a CPP program using dfs(Depth First Search) algorithm which will find the shortest path from a start point to an exit point
Expected behavior
It will solve a maze by finding the shortest path
Screenshots (optional)
Additional context (optional )

Cpp codes are not accepted under GSsoc23


 The other is lost.
This is best explained with a video:
https://imgur.com/a/8nvxyJB

These versions were in use:

I've updated to 1.21.9 and it doesn't reproduce anymore, indeed.


Add a docker file to support php 7.1.4

Add a docker file to support php 7.1.4.
Repository switch has been needed for php modules.

@RobLoach  I'm happly using your Docker images.
Now I need support for php 7.1.4, I'm trying to make it work, but not being an expert on DevOps stuff, I would like if you can to get some help to achieve the goal.

Hi @Pherserk ,
The current base-image uses php:7-lapine pointing to the following versions : 7.1.5-alpine, 7.1-alpine, 7-alpine, alpine (7.1/alpine/Dockerfile).
If the tag alpine isn't working as intended it is because the images available on Docker hub were not rebuilt in a while (last build 8 months ago).
Your current PR is not necessary to fix the problem but remains a good workaround for people wanting a more up-to-date version.
ping @RobLoach : you should allow Docker hub to dynamically rebuild the images on upstream update (base image).

Thanks for the ping! This looks good.


1.17

Do you plan to update this mod for 1.17?

https://github.com/ChloeDawn/Plated/releases/tag/2.1.0 for 1.17 is now available (CurseForge release pending)

Awesome, thanks!


Add the ability to pin certain files in the open file list

Add the ability to pin files to the top of the open file list. Allows developer to easily go back to the files most important to them to accomplish their tasks.

#12622


Fix typos

Pull Request Test Coverage Report for Build 63

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 75.909%

Totals

Change from base Build 61:
0.0%

Covered Lines:
167

Relevant Lines:
220

💛  - Coveralls

Thanks!


Excluding system tests from being installed.

@tseaver I noticed this while trying to from system_tests import populate_datastore and the import failed due to reasons that were "resolved" in #1349.
Then I peaked and realized that the error was coming from
/usr/local/lib/python2.7/dist-packages/system_tests/populate_datastore.py
/usr/local/lib/python2.7/dist-packages/gcloud/datastore/client.py

Notice that in our repo:
>>> from setuptools import find_packages
>>> find_packages()
['gcloud',
 'system_tests',
 'gcloud.storage',
 'gcloud.pubsub',
 'gcloud.dns',
 'gcloud.bigtable',
 'gcloud.bigquery',
 'gcloud.resource_manager',
 'gcloud.search',
 'gcloud.datastore',
 'gcloud.streaming',
 'gcloud.storage.demo',
 'gcloud.bigtable.happybase',
 'gcloud.bigtable._generated',
 'gcloud.datastore.demo',
 'gcloud.datastore._generated']

(This is due to the fact that system_tests/__init__.py was added in #954 / #273)

Note that excluding in MANIFEST.in  will exclude them from the sdist, which can make it hard for downstreams to test (if that matters).
On what host were those /usr/local/lib/python2.7/dist-lib/ paths found?

Ubuntu 14.04

@tseaver PTAL. I did a hacky thing my modifying sys.path in __main__.

We still don't want to ship /usr/local/lib/python2.7/dist-packages/system_tests/ to people

You mean we don't want those files to be there after they run sudo pip install gcloud?  I'm thinking of downstream packagers, who might want to be able to run all the tests after creating a .deb/.rpm from the sdist.

You mean we don't want those files to be there after they run sudo pip install gcloud

Yep. Is there a middle ground?

@tseaver Bump

@tseaver Bump

@tseaver PTAL

I'm still against stripping the system tests from the sdist.  Can we move the importable support code into a subpackage of gcloud, and just keep system_tests off the PYTHONPATH?

Why do we want to ship our system tests?

So that downstream packagers (e.g., .deb/.rpm) can run them to verify they have working code?

Our system tests? These require a ton of set-up, including having to go to the Cloud Console and download a key.

I've gotten feedback from .deb/.rpm maintainers in the past that they wanted to be able to verify the working state of the code they package by running tests afterward.  Maybe they wouldn't run the system tests.
OTOH, as the APIs change over time, asking users to run them to help debug configuration issues vs. software might be reasonable.

It's not like git clone is difficult for the .deb/.rpm maintainers, or for users that are able to offer useful help

@tseaver WDYT of my comment above?

LGTM, on the basis that gcloud isn't the type of library we would expect to see as an OS-level dependency. Nor do I think it's a good idea to pursue that path, as it would hold our releases and support subject to OS release cycles (which are generally very slow).

Filed #1451 for discussion of the possibility to ship the system tests as gcloud.system_tests. Merging for now.


[bug] tooltip placement issues

Issue
The placement of the tooltip doesn't seem to be aware of the window size and positioning. In the example screenshot below, the tooltip is flowing off the screen despite it having plenty of space to align itself above.
I would recommend looking at something like PopperJS for better positioning system.

Thanks for the recommendation Austin. Looks like Popper.js has been renamed and reimplemented as https://github.com/floating-ui/floating-ui . We're somewhat judicious when it comes to dependencies, but this looks like a good library to me. I appreciate that it's minimal and lightweight.

May I work on it? Can someone help me as I am new to open source contribution? @danmarshall

Hi @pree-T , you are absolutely welcome to contribute :)
We use the pull request (PR) process to review incoming code contributions. Please send a PR and I can review your code and give guidance in the specific areas.
This is a monorepo, and is a little more complicated than a traditional repo. Hopefully this doesn't dissuade you, but since you mentioned you're new to OSS, I wanted to let you know.

Hi, @danmarshall  Is this problem currently fixed only by modifying the Sanddance-Explorer package?

PR #493 allows for a custom tooltip provider.


Create Main

Description:
Describe your changes.
Related issue:
Add link to the related issue.
Check list:

[ ] Mark if documentation changes are required.
[ ] Mark if tests were added or updated to cover the changes.

Yes

I'm going to close the pull request because it looks like a spam.


feat: alizer impl in go - analyze

Still needs to understand something about golang bc i'm having some trouble with the ci workflow but the code should be ready to be reviewed in the meantime

@kadel can you leave a review here? If it's good for you i'm going to merge it otherwise i'm ready to work on any change required. Let me know. Thanks!!


Terraform files

Add Terraform files for deploying the updated manual ingest Lambdas.

If data_providers is being used to set tags, you need to capitalize the Project tag values

They are... but the tags are set up directly in the Lambda.


Not opening documents of size more than 100mb.

I have raised this issue here : https://github.com/cetra3/onlyoffice-alfresco/issues/29
As per suggestion, I have tried by increasing the value for "maxDownloadBytes" in /etc/onlyoffice/documentserver/default.json file but it didn't helped.
Please guide if I can try any other thing.
Thanks.

@LinneyS @StasStryukov91 I've moved this issue to CommunityServer

@monicakumari sorry, some mixup. Your issue should be stayed in DocumentServer repo and should be fixed by configuration of FileConverter:converter:inputLimits:zip:uncompressed, if not - feel free to comment
@StasStryukov91 sorry, you should create new issue at https://github.com/ONLYOFFICE/CommunityServer


Issues with uploading the same file(s) multiple times

This is a great plugin and everything is working flawlessly for me except one major pain point. It seems that if the files that get selected match that of the last selection, the callback will not get fired. This is happening in every browser I tested in (all versions of IE from 8-11, latest Chrome, latest Firefox, latest Safari)
Steps to replicate:

Add to HTML: (enabling multiple selection will make no difference)
ng-file-select="upload($files)"

Add to controller:
$scope.upload = function($files) {
  console.log('upload!');
}

Select any file(s) via uploader

See upload! in console

Select exact same file(s) via uploader

Nothing happens

As long as the selection isn't identical to that of the previous selection, the callback will get fired, but if it matches exactly it will silently fail with no possible way of catching.
Has anyone else run into the same issue? The only workaround I've found other than reloading the page is to re-render the HTML.

Hi danialfarid
onclick="this.value=null" is not working with safari. Please let me know your suggestion on this.
Thanks.

You won't need this anymore. Does the demo page work?

In windows safari it not working


Implementing MFA suppression via remembered devices, getting 'Invalid device key given.' exception

The Question

I am currently trying to implement MFA suppression via remembered devices in Cognito. I am trying to implement this workflow as documented here (https://aws.amazon.com/premiumsupport/knowledge-center/cognito-user-pool-remembered-devices/) but seem to have ran into an exception when calling ConfirmDeviceAsync().
Amazon.CognitoIdentityProvider.Model.InvalidParameterException: 'Invalid device key given.'
var respondToAuthChallengeRequest = new RespondToAuthChallengeRequest
{
    ChallengeName = ChallengeNameType.SMS_MFA,
    ChallengeResponses = new Dictionary<string, string>
    {
        {
            "SMS_MFA_CODE", mfaCodeSubmission.Code
        },
        {
	    "USERNAME", mfaCodeSubmission.Username
        }
    },
    ClientId = _amazonCognitoConfiguration.Value.ClientId,
    Session = mfaCodeSubmission.SessionId
};
var respondToAuthChallengeResponse = await _amazonCognitoIdentityProvider.RespondToAuthChallengeAsync(respondToAuthChallengeRequest, cancellationToken);

user.Device = new CognitoDevice(new DeviceType { DeviceKey = respondToAuthChallengeResponse.AuthenticationResult.NewDeviceMetadata.DeviceKey }, user);
var deviceSecretVerifierConfigType = user.GenerateDeviceVerifier(respondToAuthChallengeResponse.AuthenticationResult.NewDeviceMetadata.DeviceGroupKey, mfaCodeSubmission.Code, user.Username);
var confirm = new ConfirmDeviceRequest
{
    AccessToken = respondToAuthChallengeResponse.AuthenticationResult.AccessToken,
    DeviceKey = respondToAuthChallengeResponse.AuthenticationResult.NewDeviceMetadata.DeviceKey,
    DeviceSecretVerifierConfig = new DeviceSecretVerifierConfigType
    {
        PasswordVerifier = deviceSecretVerifierConfigType.PasswordVerifier,
        Salt = deviceSecretVerifierConfigType.Salt
    }
};
var response = await _amazonCognitoIdentityProvider.ConfirmDeviceAsync(confirm, cancellationToken);

It is also important to mention that ConfirmDeviceAsync() seems to work if we are doing SRP Auth WITHOUT SMS_MFA challenge. It doesn't seem like I am missing anything, any help is appreciated.
Environment

Build Version: 3.0.0
OS Info: Windows 10
Build Environment: Visual Studio
Targeted .NET Platform:  .NET Core 3.1

We also use aws-sdk-net-extensions-cognito 2.2.2 to call GenerateDeviceVerifier to create the PasswordVerifier and Salt.

This is a :question: general question

@ashishdhingra looks like that was it. Instead of passing in the email address, it actually needed the Cognito Id, which was a GUID associated to the User. I am able to call ConfirmDeviceAsync() from the SMS_MFA challange now.

@yenhanshih Glad that it worked out for you. Closing this issue for now.


More control for RefuelerPylon options.

Description
RefuelerPylon uses up wood in Kilm when not needed and cooks boar meat automatically.
Is the feature related to a problem?
RefuelerPylon cooks meat I need for other things and uses wood meant for fires in the kilm.
Solutions
There is a option to make the RefuelerPylon stop cooking "ManageCookingStations" but there need to be one for the Kilm as well.
Your Ideal Solution
Options within the RefuelerPylon UI/UX to enable/disable certain items/tasks

Options within the RefuelerPylon UI/UX to enable/disable certain items/tasks

I also think that a UI implementation would be very nice. I don't want to do that at this stage though just because I am not interested in creating such a mechanism myself at this point in time. However I will describe how it could be done:
Cycler
Every time user presses a key while looking at the pylon, it cycles through an option. Then the user can press another key to toggle it on/off. For example: Cook meat? True or Make coal? True
This is relatively crude but right now I can't think of a better way. It's very doom-engine-like.
Technical Approach
Decide on exact settings
You need to decide on each and every possible option. For example, are we being as specific as choosing exactly which types of meat to cook, or as broad as meats in general?
Storing Pylon-specific Settings
Users are probably gonna want to have different pylons for cooking/managing different things. A one-size-fits-all via the server-side config file won't cut it. At the same time, you said you want a UI/UX solution. This means client-side config file options are also not ideal.
Solution: ZDO entries.

ZDO

Valheim uses things called ZDOs to store stuff across sessions. For example, when I create a minion I store its creator as a ZDO so that it can be read at any time. It's just a key/value. You can see that here.
With that in mind, when the user is configuring a refueler via the game by pressing E, it is probably writing these options as ZDOs on the pylon like zNetView.GetZDO().SetBool("CookMeat", true)
Maybe the user who made the pylon also gets recorded in the ZDO so that only they are able to mess with the pylon's settings

Display

You probably need to create a subclass of Container that has different HoverText stuff and listens for these keypresses and displays stuff from the stored ZDOs.

Config Files

Storing the key used for cycling and the key used for toggling as client-side key entries would be necessary.
Maybe arrow keys left/right to cycle through the options and then up/down to toggle the true/false or whatever


Fix build.

Closes #1.
Getting a segfault on BWindows::Bounds(), near WindowTab::GetPreferredSize(), albeit with different parameter types for GCC2 vs x86 (or x86_64).

I wound up re-creating the repository first to restore the commit history from CVS, and then manually merged this PR after that.


Search/filtered history

Say I want to check how much I've spent on cucumbers historically... how can I see I've spent 2 buck on a cucumber last week

cancelled


Fix User identity stuck in RoadRunner environment

Q
A

Is bugfix?
✔

New feature?
❌

Breaks BC?
❌

Tests pass?
-

Fixed issues
#162

@rustamwin have you tested it locally?

I haven't tested yet

Wait, why user object is shared between two separate requests?

@rob006 whole container is shared. That's how RoadRunner gains its performance. We can deal with it in three ways:

Avoid having state in services.
Clean up state individually for services that need it.
Have a special DI container mode for services that will be reset on each request.

This change won't fix the problem for sure.

@samdark Can we detect which services depends (directly on indirectly) on request? Basically all of them should be reset.

@rob006 I don't think we can do that in runtime.


[BUG] Pre-upgrade pod should event the reason for any failures.

Describe the bug
When the pre-upgrade job/pod runs and finds a problem, it exits with a failure, but only the fact of its failure is recorded in events, not why:
38m         Warning   BackOff                  pod/longhorn-pre-upgrade-4bvzf                                            Back-off restarting failed container longhorn-pre-upgrade in pod longhorn-pre-upgrade-4bvzf_longhorn-system(34b56717-49be-451e-97c4-6428554bf5f4)
38m         Warning   BackoffLimitExceeded     job/longhorn-pre-upgrade                                                  Job has reached the specified backoff limit
38m         Normal    SuccessfulDelete         job/longhorn-pre-upgrade                                                  Deleted pod: longhorn-pre-upgrade-4bvzf

To Reproduce
Helm install 1.4.3.
Make a volume.
Helm install 1.5.4.  (volume is still running 1.4.3 engine)
Helm install 1.6.2.  (install fails, pre-upgrade hook failed)
Helm reports
helm upgrade longhorn longhorn/longhorn --namespace longhorn-system --version 1.6.2

Error: UPGRADE FAILED: pre-upgrade hooks failed: 1 error occurred:
	* job longhorn-pre-upgrade failed: BackoffLimitExceeded

Reason for the failure is a mystery.
Expected behavior
There is an event about the precise nature of the pre-upgrade check that failed.
Support bundle for troubleshooting
A support bundle is not useful, because the job and pod are immediately deleted.  See also https://github.com/longhorn/longhorn/issues/9448 for a suggestion on keeping the pod around for some period of time after it dies, so that logs can be grabbed.  Also a valuable thing to have, but there is still a race to get them.  There should also be an event that would last longer.
Environment

Longhorn version: 1.6.2
Impacted volume (PV): N/A.  All of them.
Installation method (e.g. Rancher Catalog App/Helm/Kubectl):  Helm
Kubernetes distro (e.g. RKE/K3s/EKS/OpenShift) and version:  N/A

Number of control plane nodes in the cluster:
Number of worker nodes in the cluster:

Node config

OS type and version:
Kernel version:
CPU per node:
Memory per node:
Disk type (e.g. SSD/NVMe/HDD):
Network bandwidth between the nodes (Gbps):

Underlying Infrastructure (e.g. on AWS/GCE, EKS/GKE, VMWare/KVM, Baremetal):
Number of Longhorn volumes in the cluster:

Additional context

Workaround and Mitigation
Guesswork, based on the knowledge of what the pre-upgrade pod actually does.

Results from testing the pre-upgrade job.  Event:
2m33s               Normal    Created                 Pod/longhorn-pre-upgrade-v5tqq    Created container longhorn-pre-upgrade
2m33s               Normal    Pulled                  Pod/longhorn-pre-upgrade-v5tqq    Container image "jmunson4suse/longhorn-manager:9569-test8" already present on machine
2m33s               Warning   FailedUpgradePreCheck   /longhorn-pre-upgrade             failed to upgrade since upgrading from v1.6.2 to v1.8.0-dev for minor version is not supported

Logs:
time="2024-10-14T22:05:14Z" level=info msg="Running pre-upgrade..." func=main.main.PreUpgradeCmd.func5 file="pre_upgrade.go:44"
W1014 22:05:14.854603       1 client_config.go:659] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
time="2024-10-14T22:05:14Z" level=info msg="Checking if the upgrade path from v1.6.2 to v1.8.0-dev is supported" func=util.checkLHUpgradePath file="util.go:238"
time="2024-10-14T22:05:14Z" level=info msg="Done with Run() ... err is failed to upgrade since upgrading from v1.6.2 to v1.8.0-dev for minor version is not supported" func=app.preUpgrade file="pre_upgrade.go:80"
time="2024-10-14T22:05:14Z" level=info msg="Sleeping for log collection ..." func=app.preUpgrade file="pre_upgrade.go:84"
time="2024-10-14T22:05:14Z" level=info msg="Event(v1.ObjectReference{Kind:\"\", Namespace:\"longhorn-system\", Name:\"longhorn-pre-upgrade\", UID:\"\", APIVersion:\"\", ResourceVersion:\"\", FieldPath:\"\"}): type: 'Warning' reason: 'FailedUpgradePreCheck' failed to upgrade since upgrading from v1.6.2 to v1.8.0-dev for minor version is not supported" func="record.(*eventBroadcasterImpl).StartLogging.func1" file="event.go:377"

If applied again, no harm done:
$ k apply -f preupgrade-job.yaml
job.batch/longhorn-pre-upgrade unchanged
$ kl get jobs
NAME                   COMPLETIONS   DURATION   AGE
longhorn-pre-upgrade   0/1           5m6s       5m6s

@james-munson Please update the upgrade and important note in https://longhorn.io/docs/1.8.0/. Thank you.

@james-munson I tried to test this issue by following https://github.com/longhorn/longhorn/issues/9569#issue-2565079099 to install Longhorn v1.4.3 -> create a volume -> upgrade to Longhorn v1.5.4 -> upgrade to Longhorn v1.6.x-head. Although v1.6.x-head includes the relevant commits from this ticket https://github.com/longhorn/longhorn-manager/pull/3211, I'm still unable to get the reason why the upgrade failed after the pre-upgrade pod failed:
$ helm install longhorn longhorn/longhorn --namespace longhorn-system --create-namespace --version 1.4.3
WARNING: Kubernetes configuration file is group-readable. This is insecure. Location: /home/yang/Code/automation/terraform2/aws/ubuntu/k3s.yaml
WARNING: Kubernetes configuration file is world-readable. This is insecure. Location: /home/yang/Code/automation/terraform2/aws/ubuntu/k3s.yaml
kubectl get pods -n longhorn-system -w
NAME: longhorn
LAST DEPLOYED: Mon Oct 21 14:15:57 2024
NAMESPACE: longhorn-system
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
Longhorn is now installed on the cluster!

Please wait a few minutes for other Longhorn components such as CSI deployments, Engine Images, and Instance Managers to be initialized.

Visit our documentation at https://longhorn.io/docs/

# create volume

$ helm upgrade longhorn longhorn/longhorn --namespace longhorn-system --version 1.5.4
WARNING: Kubernetes configuration file is group-readable. This is insecure. Location: /home/yang/Code/automation/terraform2/aws/ubuntu/k3s.yaml
WARNING: Kubernetes configuration file is world-readable. This is insecure. Location: /home/yang/Code/automation/terraform2/aws/ubuntu/k3s.yaml
kubectl get pods -n longhorn-system -w
Release "longhorn" has been upgraded. Happy Helming!
NAME: longhorn
LAST DEPLOYED: Mon Oct 21 14:19:19 2024
NAMESPACE: longhorn-system
STATUS: deployed
REVISION: 2
TEST SUITE: None
NOTES:
Longhorn is now installed on the cluster!

Please wait a few minutes for other Longhorn components such as CSI deployments, Engine Images, and Instance Managers to be initialized.

Visit our documentation at https://longhorn.io/docs/

$ helm upgrade longhorn ./longhorn/chart/ -n longhorn-system # upgrade to v1.6.x-head
WARNING: Kubernetes configuration file is group-readable. This is insecure. Location: /home/yang/Code/automation/terraform2/aws/ubuntu/k3s.yaml
WARNING: Kubernetes configuration file is world-readable. This is insecure. Location: /home/yang/Code/automation/terraform2/aws/ubuntu/k3s.yaml
Error: UPGRADE FAILED: pre-upgrade hooks failed: 1 error occurred:
	* job failed: BackoffLimitExceeded

$ kubectl get events -n longhorn-system -w
<PHONE_NUMBER>977-1206-driver-longhorn-io became leader
0s          Normal    SuccessfulCreate         job/longhorn-pre-upgrade                                                  Created pod: longhorn-pre-upgrade-w7xvj
0s          Normal    Scheduled                pod/longhorn-pre-upgrade-w7xvj                                            Successfully assigned longhorn-system/longhorn-pre-upgrade-w7xvj to ip-10-0-1-206
0s          Normal    Pulled                   pod/longhorn-pre-upgrade-w7xvj                                            Container image "longhornio/longhorn-manager:v1.6.x-head" already present on machine
0s          Normal    Created                  pod/longhorn-pre-upgrade-w7xvj                                            Created container longhorn-pre-upgrade
0s          Normal    Started                  pod/longhorn-pre-upgrade-w7xvj                                            Started container longhorn-pre-upgrade
0s          Normal    Pulled                   pod/longhorn-pre-upgrade-w7xvj                                            Container image "longhornio/longhorn-manager:v1.6.x-head" already present on machine
0s          Normal    Created                  pod/longhorn-pre-upgrade-w7xvj                                            Created container longhorn-pre-upgrade
0s          Normal    Started                  pod/longhorn-pre-upgrade-w7xvj                                            Started container longhorn-pre-upgrade
0s          Warning   BackOff                  pod/longhorn-pre-upgrade-w7xvj                                            Back-off restarting failed container longhorn-pre-upgrade in pod longhorn-pre-upgrade-w7xvj_longhorn-system(331fe695-ea93-4383-ac04-9443d0833fc0)
0s          Warning   BackOff                  pod/longhorn-pre-upgrade-w7xvj                                            Back-off restarting failed container longhorn-pre-upgrade in pod longhorn-pre-upgrade-w7xvj_longhorn-system(331fe695-ea93-4383-ac04-9443d0833fc0)
0s          Normal    SuccessfulDelete         job/longhorn-pre-upgrade                                                  Deleted pod: longhorn-pre-upgrade-w7xvj
0s          Warning   BackoffLimitExceeded     job/longhorn-pre-upgrade                                                  Job has reached the specified backoff limit

Did I miss anything? Or could you help provide more details and instructions on how to properly test this issue? Thank you!

That looks like the sequence I would expect.  I will try to repro it myself.

I was hoping that the eventBroadcaster.Shutdown() call would force a flush of the queued event before the panic, and it appeared to do so, in some testing.  But there is still a race.  Here is an example, using a 1.7.3 build with the change:
37s                 Normal    Killing                 Pod/longhorn-pre-upgrade-fnp7x   Stopping container longhorn-pre-upgrade
16s                 Normal    SuccessfulCreate        Job/longhorn-pre-upgrade         Created pod: longhorn-pre-upgrade-cxc7s
15s                 Normal    Scheduled               Pod/longhorn-pre-upgrade-cxc7s   Successfully assigned longhorn-system/longhorn-pre-upgrade-cxc7s to jbm-trial-pool2-254cd2ae-vsblf
13s (x2 over 15s)   Normal    Started                 Pod/longhorn-pre-upgrade-cxc7s   Started container longhorn-pre-upgrade
13s (x2 over 15s)   Normal    Created                 Pod/longhorn-pre-upgrade-cxc7s   Created container longhorn-pre-upgrade
13s (x2 over 15s)   Normal    Pulled                  Pod/longhorn-pre-upgrade-cxc7s   Container image "longhornio/longhorn-manager:v1.7.3-dev-20241020" already present on machine
12s                 Normal    SuccessfulDelete        Job/longhorn-pre-upgrade         Deleted pod: longhorn-pre-upgrade-cxc7s
12s                 Warning   BackoffLimitExceeded    Job/longhorn-pre-upgrade         Job has reached the specified backoff limit
11s (x2 over 12s)   Warning   BackOff                 Pod/longhorn-pre-upgrade-cxc7s   Back-off restarting failed container longhorn-pre-upgrade in pod longhorn-pre-upgrade-cxc7s_longhorn-system(5b58983a-d5bb-4bbc-90ff-6df91da65f73)

And here it is with a modified version that does a sleep before panicking:
95s                 Normal    SuccessfulCreate        Job/longhorn-pre-upgrade         Created pod: longhorn-pre-upgrade-fnp7x
94s                 Normal    Scheduled               Pod/longhorn-pre-upgrade-fnp7x   Successfully assigned longhorn-system/longhorn-pre-upgrade-fnp7x to jbm-trial-pool2-254cd2ae-vsblf
93s                 Normal    Pulling                 Pod/longhorn-pre-upgrade-fnp7x   Pulling image "jmunson4suse/longhorn-manager:9569-17301"
86s                 Normal    Pulled                  Pod/longhorn-pre-upgrade-fnp7x   Successfully pulled image "jmunson4suse/longhorn-manager:9569-17301" in 7.006984287s (7.007005255s including waiting)
86s                 Normal    Created                 Pod/longhorn-pre-upgrade-fnp7x   Created container longhorn-pre-upgrade
86s                 Normal    Started                 Pod/longhorn-pre-upgrade-fnp7x   Started container longhorn-pre-upgrade
85s                 Warning   FailedUpgradePreCheck   /longhorn-pre-upgrade            failed to upgrade since upgrading from v1.5.4 to v1.7.2 for minor version is not supported

(I then killed the pre-upgrade job/pod manually.)

@yangchiu, Hopefully, with that change (merged yesterday), the event will be visible.

Verification passed on master-head (longhorn-manager 235b1ae) by the following test steps. The reason why the upgrade failed can be found in the event: failed to upgrade since upgrading from v1.4.3 to v1.8.0-dev for minor version is not supported:
# 1. Install Longhorn v1.4.3 by helm:
$ helm install longhorn longhorn/longhorn --namespace longhorn-system --create-namespace --version 1.4.3

# 2. Create a volume

# 3. Upgrade Longhorn to v1.6.x-head by helm:
$ helm upgrade longhorn ./longhorn/chart/ -n longhorn-system
WARNING: Kubernetes configuration file is group-readable. This is insecure. Location: /home/yang/Code/automation/terraform2/aws/ubuntu/k3s.yaml
WARNING: Kubernetes configuration file is world-readable. This is insecure. Location: /home/yang/Code/automation/terraform2/aws/ubuntu/k3s.yaml
Error: UPGRADE FAILED: pre-upgrade hooks failed: 1 error occurred:
	* job failed: BackoffLimitExceeded

$ kubectl get events -n longhorn-system -w
19s         Normal    SuccessfulCreate         job/longhorn-pre-upgrade                                  Created pod: longhorn-pre-upgrade-8v5c2
18s         Normal    Scheduled                pod/longhorn-pre-upgrade-8v5c2                            Successfully assigned longhorn-system/longhorn-pre-upgrade-8v5c2 to ip-10-0-1-233
18s         Normal    Pulling                  pod/longhorn-pre-upgrade-8v5c2                            Pulling image "longhornio/longhorn-manager:master-head"
14s         Normal    Pulled                   pod/longhorn-pre-upgrade-8v5c2                            Successfully pulled image "longhornio/longhorn-manager:master-head" in 3.611412654s (3.61143456s including waiting)
14s         Warning   FailedUpgradePreCheck    /longhorn-pre-upgrade                                     failed to upgrade since upgrading from v1.4.3 to v1.8.0-dev for minor version is not supported
7s          Normal    Pulled                   pod/longhorn-pre-upgrade-8v5c2                            Container image "longhornio/longhorn-manager:master-head" already present on machine
7s          Normal    Created                  pod/longhorn-pre-upgrade-8v5c2                            Created container longhorn-pre-upgrade
7s          Normal    Started                  pod/longhorn-pre-upgrade-8v5c2                            Started container longhorn-pre-upgrade
7s          Warning   FailedUpgradePreCheck    /longhorn-pre-upgrade                                     failed to upgrade since upgrading from v1.4.3 to v1.8.0-dev for minor version is not supported
6s          Normal    SuccessfulDelete         job/longhorn-pre-upgrade                                  Deleted pod: longhorn-pre-upgrade-8v5c2
6s          Warning   BackoffLimitExceeded     job/longhorn-pre-upgrade                                  Job has reached the specified backoff limit
5s          Normal    Killing                  pod/longhorn-pre-upgrade-8v5c2                            Stopping container longhorn-pre-upgrade


youtube-dl only downloading some videos of YouTube channel. How to download them all?

Checklist

[x] I'm asking a question
[x] I've looked through the README and FAQ for similar questions
[x] I've searched the bugtracker for similar questions including closed ones

Question

I am trying to download all the videos on a channel that has 59 videos on it but youtube-dl is only detecting and downloading 11 videos from the channel. I searched through the open and closed issues but could not find one that matched my exact issue. What should I do differently? This is the command that I am using:

$ youtube-dl -v --cookies "cookies.txt" -i -f best[height<=360] "https://www.youtube.com/channel/UC_AkEUxYb7TvE0PPHQxQIog"

When you go to your youtube channel, you can go to the tab "videos".
Here usually, you have a link "play all". Click on it.

You'll be launched to the playlist of every video of that channel.
You have two ways to get the link of the playlist alone.

You can click on the title of the playlist and get redirected to the playlist

From here, you'll have the link of the playlist

You can take from the link of the page you are in, only the ID of the playlist

After that, you just have to download the whole playlist.
Hope it helps.

#16212


LaLiga not working

Curl
curl -X 'GET'
'https://transfermarkt-api.vercel.app/competitions/search/LaLiga?page_number=1'
-H 'accept: application/json'
Request URL
https://transfermarkt-api.vercel.app/competitions/search/LaLiga?page_number=1
Server response
Code	Details
500
Undocumented
Error: response status is 500
Response body
Download
{
"error": {
"code": "500",
"message": "A server error has occurred"
}
}

The same for Champions League

Hello @ITibirna!
Thanks for reporting this issue. It looks like it is a bug in cases where the search returns only one page. I will open a PR to fix it


Make executeWorkbench work after VSCode restarts

vscode.openFolder can be used to restart the VSCode under different workspace root. In that case the current socket is closed, and then the new connection appears after the VSCode is up again.
This PR clears pending message callbacks when the connection is closed and re-assigns the _promisedSocket to a new socket when new connection appears.
Fixes https://github.com/webdriverio-community/wdio-vscode-service/issues/62
Had to hard-code vscode version to 1.85.2 when running tests locally, since newer chromedriver versions live in a new location. Seems like there's a PR already to fix it https://github.com/webdriverio-community/wdio-vscode-service/pull/105

Released as part of v6.0.0!


Star tracks quickly cause upper pane to run out of vertical real estate.

I'd like to selectively star tracks, but the inability to scroll or shrink the view with the upper pane quickly results in very little real-estate to see and compare other tracks below, and also limits the ability to add more than the pane can display.
Could both top and bottom view have scroll panes?  Also let me pick where the window split occurs?  Otherwise, I'm going to have to only use this profiler on a vertically oriented external monitor.   I'm on a 16" MBP and that has a lot of vertical resolution (1920px), but it's not enough for even one track with a flamegraph like we get using Atrace.

Folding this into https://github.com/google/perfetto/issues/429


(fleet) persist proxy settings during setup

This PR persists the proxy setup passed at install in the agent/installer configuration.

/merge


Need to add data source for new species

Api: https://api.atlasapprox.org/v1/data_sources

d_melanogaster
h_miamia
s_mansoni

And nematostella I guess, and lamnia
On Wed, Sep 27, 2023, at 16:11, YingX97 wrote:

Api: https://api.atlasapprox.org/v1/data_sources

d_melanogaster
h_miamia
s_mansoni
—
Reply to this email directly, view it on GitHub
https://github.com/fabilab/cell_atlas_approximations_API/issues/9, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAJFEAG6IN2Z5IU6LHMXIJ3X4O7SRANCNFSM6AAAAAA5IZND6Y.
You are receiving this because you are subscribed to this
thread.Message ID:
@.***>

added in 1022763


[Component] InputText Component for Storybook docs

TODO:

[x] Move the InputText component from coop-ds-storybook to this repo, under the react-ui package
[x] Remove the pcss imports from jsx
[x] Remove the pcss file from component directory
[x] Have component styles installed at react-ui package root level as a package dependency
[x] Import component style .pcss at a global level in .storybook/preview.js
[x] Have stories attached to the component that cover all states of the component
[x] Add tests for the component using jest and testing-library/react
[x] Remove any static html examples from component directory
[x] Avoid any state management work for the time being, if it can be avoided.

Resolved in #400


sasquatch: init

Fixes #66270 .
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
[x] Fits CONTRIBUTING.md.

Regarding your comment about the -lgcc_s flag, I do not know.
I had a look around:

The squashfs patch does not seem to touch anything related to pthread_cancel;
The used Makefile (from squashfs, used in sasquatch's build script), does not have any kind of mention of the gcc_s lib.

So, I would not be inclined adding it.

Regarding the CVEs, I have can found two related to sasquatch: CVE-2015-4645 and CVE-2015-4646.
As per this comment on devttys0/sasquatch#5, fixes are taken care of upstream (in squashfs).

Thanks. Maybe @ruuda, @danbst, or @charles-dyfis-net could take a quick look as people familiar with squashfsTools.

Looks good to me. I think it is good to keep the sources separate and not base this package on squashfstools. Otherwise, when there is a squashfstools update and the patches no longer apply, we either need to wait for upstream Sasquatch to update the patches before we can upgrade squashfstools, or we need to disable the sasquatch package. Now they can coexist.

@Pamplemousse at this point the only thing missing is a license then. Ready for merge once that situation is resolved. Thanks! :+1:

Thanks @Pamplemousse, and welcome to the team :tada:


Small UI tweaks (control shortcut on video overlay, select dropdown re-styling)

This PR re-styles the recently added select form.

Also adds a shortcut for peripheral control on the video overlay, inspired by the need of it during my daily use.

lgtm


ASes with policy

50999 ASes out of 95911 have at least one policy recorded.
IPython record.
In [1]: import pandas as pd

In [2]: df = pd.read_csv("as_neighbors_vs_rules.csv")

In [3]: df
Out[3]:
       aut_num  neighbor  import  export
0       202125         3       2       2
1       266498        29       4       3
2       132756        -1       0       0
3        41937        13       8       8
4       396741         1       0       0
...        ...       ...     ...     ...
95906   327736        -1       0       0
95907   201276        -1       2       2
95908    32284         2       0       0
95909   399684         3       0       0
95910   136615         1       0       0

[95911 rows x 4 columns]

In [4]: df.count()
Out[4]:
aut_num     95911
neighbor    95911
import      95911
export      95911
dtype: int64

In [6]: df[(df['import'] > 0) | (df['export'] > 0)]
Out[6]:
       aut_num  neighbor  import  export
0       202125         3       2       2
1       266498        29       4       3
3        41937        13       8       8
6       211753         1       1       1
7         1126       134     148     149
...        ...       ...     ...     ...
95901   204472        -1       0       3
95902   207560         2       3       3
95904   201318         2       3       3
95905    17759         1       3       3
95907   201276        -1       2       2

[50999 rows x 4 columns]

In [7]: df[(df['import'] > 0) | (df['export'] > 0)].count()
Out[7]:
aut_num     50999
neighbor    50999
import      50999
export      50999
dtype: int64

Following data from #19.

Very interesting. I was expecting this number to be much lower. Can you plot a CDF? (If you export a file with the number of RPSL rules per AS, one AS per line, I can send you a notebook back. Although these days ChatGPT might just generate the matplotlib code for you.)

Regenerated the same stats as in #19 using latest parse results. They are marginally different.

IPython history.
In [2]: import pandas as pd

In [3]: df = pd.read_csv("as_neighbors_vs_rules.csv")

In [5]: df['ports'] = df['import'] + df['export']

In [6]: df
Out[6]:
       aut_num  neighbor  import  export  ports
0       137136         1       0       0      0
1        27322        -1       0       0      0
2       206870         3       2       2      4
3        60824         2       2       2      4
4       139868         2       0       0      0
...        ...       ...     ...     ...    ...
95906    41439         2       0       0      0
95907    61414        11       0       0      0
95908    28637        29       0       0      0
95909   205649        -1       3       3      6
95910   328130        -1       0       0      0

[95911 rows x 5 columns]

In [8]: import matplotlib.pyplot as plt
   ...:
   ...: # Assuming you have already loaded your DataFrame df
   ...:
   ...: # Sort the "ports" column in ascending order
   ...: sorted_ports = df['ports'].sort_values()
   ...:
   ...: # Calculate the cumulative distribution function (CDF)
   ...: cdf = sorted_ports.reset_index(drop=True).cumsum() / sorted_ports.sum()
   ...:
   ...: # Create a plot of the CDF
   ...: plt.figure(figsize=(10, 6))
   ...: plt.plot(sorted_ports, cdf, marker='x', linestyle='none')
   ...: plt.xlabel('Ports')
   ...: plt.ylabel('CDF')
   ...: plt.title('CDF of Ports Column')
   ...: plt.grid()
   ...: plt.show()

CDF plot.

That is exactly it. Thanks. Somewhat interesting again! When plotting graphs, I usually try to put in some effort to make the graph stand out alone. I usually avoid titles, but always add meaningful labels on each axis. In here, I think we should have:
Y = "Cumulative Fraction o ASes"
X = "Number of Import and Export Rules"
Given the graph is concentrated close to zero, plotting the X-axis from 1 to 10000 using a log-scale might be better to give us a  better idea of the data on the left.

Wrote this up right now, will polish and put it up on my website later... but should be useful to give you a head start:
https://docs.google.com/document/d/1Q5ZrvAOA0fHn883DQaPovIfpz4JB-orKFA5ZSfuWLng/edit?usp=sharing

The above plot was just for verifying the content.
I have set up scripts in the paper repo so we can hopefully automate this in the future.
Wide version
CDF-AS-rules.pdf

Squared version
CDF-AS-rules-squared.pdf

The CDF does look exponential.
PNG preview.

+1; I think the logscale helps in identifying breakpoints (e.g., 45% of ASes have 100 or less rules and almost 30% have more than 1000 rules). Really? 1000 rules? What are these people doing? :)

All the ASes with more than 1000 rules (96 in total). (whois them to see their rules)
AS4455, AS12552, AS15925, AS10429, AS21385, AS59613, AS3303, AS29119, AS61374, AS8422, AS3257, AS3216, AS9121, AS3327, AS50952, AS31133, AS29076, AS8359, AS6667, AS20485, AS6730, AS5607, AS8897, AS7717, AS24940, AS8881, AS8447, AS8469, AS12731, AS8426, AS6881, AS30740, AS62047, AS48200, AS9031, AS30781, AS9002, AS43252, AS56665, AS4589, AS12350, AS12874, AS62499, AS16150, AS8657, AS44654, AS48793, AS13285, AS9049, AS6661, AS20912, AS5430, AS43531, AS56890, AS8220, AS49666, AS63034, AS3326, AS31449, AS15435, AS29527, AS10102, AS1299, AS3356, AS49544, AS42579, AS13237, AS12859, AS12389, AS41692, AS6805, AS13101, AS8545, AS6695, AS61955, AS1273, AS48850, AS12880, AS2603, AS6762, AS8455, AS8492, AS5413, AS3209, AS20562, AS42476, AS9033, AS41327, AS47228, AS8928, AS16298, AS43760, AS13037, AS5511, AS25152, AS2119

[ ] Look into the Y-axis error in the CDF.

I went and changed to a proper way to generate CDF (using matplotlib 3.8). This looks more like it.
Preview and PDF files.

CDF-AS-rules-squared.pdf
CDF-AS-rules.pdf

Ha, this makes more sense! (It seems like you were weighting ASes by their number of rules, which we could also show. We would just need to re-label the Y axis to "Cumulative Fraction of Communities" and the Y axis to "Number of Import and Export Rules by Controlling AS".)
93% of ASes with 10 or less rules.
Now... I prefer when CDFs cover the whole [0, 1] range on the Y axis. One option to make this even better is to plot a CCDF (Complementary CDF). It's the same data you have, but you make do:
ccdf_points = [(x, 1-y) for x, y in cdf_points]
set_ylim((1e-6, 1)) # adjust accordingly, just can't do down to 0]
set_logscale("y")

Potential TODO: add dots of significant ASes on the graph.

[ ] Exclude the impact from AS Relational DB as discovered in https://github.com/SichangHe/internet_route_verification/issues/64#issuecomment-1762962607.

Updated Import/export rules stats in the text.
Running at internet_route_verification_meta/scripts.
In [5]: from scripts.csv_files import as_neighbors_vs_rules
In [6]: FILE = as_neighbors_vs_rules
In [7]:     df_raw = pd.read_csv(FILE.path)
   ...:     # Remove ASes not in IRR.
   ...:     df = df_raw.drop(df_raw[df_raw["import"] == -1].index)
   ...:     df["rules"] = df["import"] + df["export"]

In [8]: df[df['rules'] == 0]
Out[8]:
       aut_num  provider  peer  customer  import  export  rules
1        58421         2     0         0       0       0      0
2        64070        -1    -1        -1       0       0      0
3       150451        -1    -1        -1       0       0      0
12      398994         1     1         0       0       0      0
19      398037         3     0         0       0       0      0
...        ...       ...   ...       ...     ...     ...    ...
95836    45163        -1    -1        -1       0       0      0
95837    63633        -1    -1        -1       0       0      0
95838   396609        -1    -1        -1       0       0      0
95839   141451        -1    -1        -1       0       0      0
95840   150084         1     0         0       0       0      0

[27840 rows x 7 columns]

In [9]: df[df['rules'] == 0].__len__() / len(df)
Out[9]: 0.3537439168498494

In [10]: df[df['rules'] > 10].__len__() / len(df)
Out[10]: 0.0879404327772201

In [11]: df[df['rules'] >= 10].__len__() / len(df)
Out[11]: 0.1086263198688708

In [12]: df[df['rules'] >= 1000].__len__() / len(df)
Out[12]: 0.0012833382040888933

In [13]: df[df['rules'] > 1000].__len__() / len(df)
Out[13]: 0.001257925566384163

In [14]: df[df['rules'] >= 1000]
Out[14]:
       aut_num  provider  peer  customer  import  export  rules
1466     48850        -1    -1        -1     520     520   1040
1480     13037         3   103         5     724     726   1450
2066     30740         3    89         3     783     783   1566
4143     13285         6   191         5     589     587   1176
4210      8426         7   971        17    3073    3073   6146
...        ...       ...   ...       ...     ...     ...    ...
89544    10102        -1    -1        -1    1024      75   1099
91014     3356         0    71      6459    5706    5344  11050
91950     9049         4   165       410     734     734   1468
92030     1299         0    48      2307    5724       4   5728
92509     9033        -1    -1        -1    1770    1770   3540

[101 rows x 7 columns]

Update after handling PeerAS: as_neighbors_vs_rules5.csv.gz, little difference.