Run Parsl in Headless mode in MareNostrum5

Describe the bug
All tasks are assigned to the same node.
To Reproduce

Install via conda Parsl 2024.9.9 with Python 3.12.5 on MareNostrum 5.
Login to MareNostrum 5 and request a 2 node allocation with salloc.
Run  the NERSC's script for Headless execution on node 0 without the max_workers=1 option.
Wait for execution.
Check output with all tasks reporting on the master node.

Expected behavior
That some tasks are assigned to the other node.
Environment

OS: Red Hat Enterprise Linux 9.2 (Plow)
Python version: 3.12.5
Parsl version: 2024.9.9

Distributed Environment

Where are you running the Parsl script from?
From the master node.
Where do you need the workers to run ?
At the master node and the rest of nodes in the allocation.

Stdout
[<AppFuture at 0x7f34570ce3c0 state=pending>, <AppFuture at 0x7f34583c2210 state=pending>, <AppFuture at 0x7f345723daf0 state=pending>, <AppFuture at 0x7f34570ce420 state=pending>]
Got result:  uname_result(system='Linux', node='gs06r3b58', release='5.14.0-284.30.1.el9_2.x86_64', version='#1 SMP PREEMPT_DYNAMIC Fri Aug 25 09:13:12 EDT 2023', machine='x86_64')
Got result:  uname_result(system='Linux', node='gs06r3b58', release='5.14.0-284.30.1.el9_2.x86_64', version='#1 SMP PREEMPT_DYNAMIC Fri Aug 25 09:13:12 EDT 2023', machine='x86_64')
Got result:  uname_result(system='Linux', node='gs06r3b58', release='5.14.0-284.30.1.el9_2.x86_64', version='#1 SMP PREEMPT_DYNAMIC Fri Aug 25 09:13:12 EDT 2023', machine='x86_64')
Got result:  uname_result(system='Linux', node='gs06r3b58', release='5.14.0-284.30.1.el9_2.x86_64', version='#1 SMP PREEMPT_DYNAMIC Fri Aug 25 09:13:12 EDT 2023', machine='x86_64')

If you have lots of workers, and a small number of short running tasks, there's no guarantee that tasks will run on both nodes. That is usually the problem (with expectations about the submitted workload, not problem with parsl) when I see this behaviour in tests.
Make sure the tasks are long running (several seconds) and that you submitted enough to saturate every worker throughout the system (eg task count = cores per node * number of nodes * arbitrary factor 3)

Oh, I see! Thank you, @benclifford.


(#355) Add common Emacs keybindings to Edit_field

Adds the most common Emacs keybindings to the console.
static helper methods are named according to the GNU Emacs
terminology:
https://www.gnu.org/software/emacs/manual/html_node/emacs/Moving-Point.html

@bepzi looks good to me! Thank you for the contribution! :+1:


8.3.x Used terser breaks bundle when using option mangle.keep_fnames

🐞 Bug report
Command (mark with an x)

- [ ] new
- [x ] build
- [ ] serve
- [ ] test
- [ ] e2e
- [ ] generate
- [ ] add
- [ ] update
- [ ] lint
- [ ] xi18n
- [ ] run
- [ ] config
- [ ] help
- [ ] version
- [ ] doc

Is this a regression?
Yes. Was working in 7.3.9.
Description
After upgrading @angular/cli from version 7.3.9 to 8.3.15, it became impossible to use terser option mangle.keep_fnames when overriding webpack configuration.
This seems to be due to a known tercer problem that was fixed in version<EMAIL_ADDRESS>https://github.com/terser/terser/blob/master/CHANGELOG.md#v439
Versions of related packages are already updated in master branch.
https://github.com/angular/angular-cli/pull/15871 and https://github.com/angular/angular-cli/pull/15920
Is it possible to apply these changes also to branch 8.3.x?
🌍 Your Environment

Angular CLI: 8.3.15
Node: 10.15.3
OS: win32 x64
Angular: 8.2.12
... animations, common, compiler, compiler-cli, core, forms
... language-service, platform-browser, platform-browser-dynamic
... router

Package                           Version
-----------------------------------------------------------
@angular-devkit/architect         0.803.16
@angular-devkit/build-angular     0.803.16
@angular-devkit/build-optimizer   0.803.16
@angular-devkit/build-webpack     0.803.16
@angular-devkit/core              8.3.16
@angular-devkit/schematics        8.3.15
@angular/cdk                      8.2.3
@angular/cli                      8.3.15
@angular/material                 8.2.3
@ngtools/webpack                  8.3.16
@schematics/angular               8.3.15
@schematics/update                0.803.15
rxjs                              6.5.3
typescript                        3.5.3
webpack                           4.39.2

Closed via https://github.com/angular/angular-cli/commit/882954b5275f0bf40634ed160704f718a5d41ebc

@alan-agius4,
<EMAIL_ADDRESS>also has a<EMAIL_ADDRESS>dependency.
So users who override terser-webpack-plugin may also have issue if they do not add correct version of plugin to their package.json.
Some projects are not able to add new dependencies to their package.json.
Could you please update also terser-webpack-plugin to 2.2.1 as part of this issue?
It has also already updated in master https://github.com/angular/angular-cli/pull/15920
This will also avoid multiple versions of the same library.

The terser webpack plugin has a dependency for terser of “^4.1.2” which is a range that includes the latest version of 4.x.
https://github.com/webpack-contrib/terser-webpack-plugin/blob/aa12914c34f3eac7280052dad3de465a163630d7/package.json#L46
As a result no changes are necessary to the CLI for terser 4.3.9 or newer to be used.

@clydin you're right. Version 4.8.9 will be used. I think it will cover almost all common cases


Fixed #31750 -- Made models.Field equality compare models for inherited fields.

https://code.djangoproject.com/ticket/31750
One question I've thought of but not answered is what behavior is best to expect with concrete inheritance, or what the behavior in that circumstance is currently and would be after this change.
Arguably, the field would then really be on the base class, but I'm not sure what I might expect to be the case as a consumer of the field. For example, would the field de-duplicate in a set? My intuition is that they really should also be different in that case, unless they all report to have the same model.

I'm not sure why pull-requests-javascript failed. I don't see any helpful output, it looks like it's just timing out waiting for something. Strikes me as likely to be unrelated.

@ryanhiebert it seems that I cannot pushed edits. Can you allow edits from maintainers?

You probably cannot because it's a corporate fork. I sent PR #13401 with edits.


Add option to prioritize hidden items list

Closes https://github.com/runelite/runelite/issues/3942

Updated to support  prioritize highlight specific > block specific > highlight wildcards > block wildcards


Nginx refactor for #401

This PR does two things

let's someone drop their own vhost config into the container at /etc/nginx/conf.d/default.conf.tpl. Also, nginx is run without a config file path which is slightly clearer imo.
upgrades the container from buster to bullseye

This is for #401

Looks like a very reasonable set of changes - I'll test it out later!


Provide packages for Ubuntu and Arch

Our team uses your product but we have not found any packages for Ubuntu or Arch. Please provide them. Thanks

How to install Mkcert on Ubuntu/Pop OS:-
1)From your home folder, Need to wget the latest Release & specify where to put it ->
sudo wget https://github.com/FiloSottile/mkcert/releases/download/v1.4.3/mkcert-v1.4.3-linux-amd64 -O /usr/local/bin/mkcert
2)Then change permissions:
sudo chmod +x /usr/local/bin/mkcert
3)Now cd into the folder if need be and run:
mkcert -install
4)Need to install local certificates for localhost:
mkcert localhost
It will generate two .pem files. You can use them.
On a React Project using VSCode you can do the following, in your VSCode Json file, have something like this under scripts:
  "dev": "BROWSER=Browser location HTTPS=true SSL_CRT_FILE=/folder/mkcert/localhost.pem SSL_KEY_FILE=/folder/mkcert/localhost-key.pem react-scripts start"
Then run npm run dev

Thanks for your response.
Is there a better way to install and update it?
We would love to have something like
apt-get install mkcert

so we can build a docker that always install and updates via a package manager without thinking about how to install it manually

np, it's not an official app so you have to manually get the latest binaries and install them into a folder. It's probably simpler to install the binaries than to spin up a docker container.

For anyone coming here, mkcert is packaged for Arch.


Dataloader throws error during iter()

Hello, I'm trying to get the repo to work with PyTorch 0.4.
While most of the changes are rather trivial, the sampler this repo uses, return both index and aspect ratio (correct me if it is something else, but it is a tuple and the batch sampler assume integer), there isn't any straightforward way to fix it with the new dataloader structure introduced in pytorch/pytorch#1867.
What would you think is the better way to make it compatible without breaking anything?
Thank you

I am a little confused. Doesn't same batch sampler exists in PyTorch 0.3.1 ?

I see. The explicit int cast is the evil. A straight forward way to overcome this problem is providing a BatchSampler that do not cast the items from Sampler.


popover does not position itself correctly

      <button uib-popover="text wont fit on one line" popover-placement="top" type="button" class="btn btn-default">Inside</button>

This worked correctly in a older version of ui-bootstrap.
(Tested in IE, Edge, Chrome)
https://plnkr.co/edit/SDBsPANUYPydQxaL5UqU?p=preview

Closing this as this project is no longer being maintained, per the README and the ISSUE_TEMPLATE.


Qt: onroad HUD

[x] qt onroad HUD
[x] remove nvg related functions after confirming that the ui is the same as before.

Would be nice to finish off the NVG -> Qt move


Script to download audio from youtube videos

config.py file with settings and constants
references to all codes and libraries used
readme.md file that explains how to use the script
Issue: AudioDownloader #1

Description
Please include a summary of the change and which issue is fixed. List any dependencies that are required for this change.
Fixed #1
Replace issue_no with the issue number which is fixed in this PR
Type of change
Please delete options that are not relevant.

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update
[ ] Documentation Update

Checklist:

[ ] My code follows the style guidelines(Clean Code) of this project
[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[x] I have created a helpful and easy to understand README.md
[ ] I have made corresponding changes to the documentation
[x] My changes generate no new warnings
[ ] I have added screenshots(if any) that prove my fix is effective or that my feature works

All done!

@all-contributors please add gudeliauskaspam for code, doc


figure out a better way to track the agreement dates

From @mengwong on September 28, 2015 13:36
(TODAY vs the actual signed date in the spreadsheet.)
when a spreadsheet is actually signed, we should detect that using the appropriate signature-engine adapter, and record the new date into the spreadsheet.
Copied from original issue: legalese-io/legalese-google-app#5

From @mengwong on September 28, 2015 13:39
there are two parts to this -- every agreement should have a unique variable name for its signing date.
when an agreement is created, we should record that date as a provisional, right in the spreadsheet.
when we notice that the e-signature system has recorded a signature complete state for the agreement, we promote the provisional to confirmed.
i think the way to do this is we create a _provisional extension to each whatever_agreement_date.

From @mengwong on September 28, 2015 13:39
also think about amendments and restatements.


Notifications 1 hour before booking

Describe the feature
It would be nice to notify users about their upcoming bookings.
Suggested solution
Send a message to user 1 hour before their booked time. Also display a button to cancel booking (if a user suddenly realizes that they cannot come to the room)
Additional context
No response

Solved by #61


Cant download the models stuck at 249m

https://huggingface.co/alibaba-pai/pai-easyphoto/tree/main

We used to have a backup on Hugging Face, but we haven't maintained it much recently. You can try to see if it can still help you.
https://www.modelscope.cn/ ，
If it's a dependency model from Modelscope, it's indeed a bit challenging for us to resolve. You can visit Modelscope's website to find the models we use and check if they offer a convenient way for you to download them.

https://huggingface.co/alibaba-pai/pai-easyphoto/tree/main

I downloaded all these models but it still trying download another model propbably from modelscope
But which models and where im going to put it ? 249 Mb ?

you can pdb to trace which model you are now downloading， and we will soon update it to huggingface，sorry too many details should be maintain better

update model on Our models can be found here: https://huggingface.co/alibaba-pai/pai-easyphoto/tree/main

@Someonetoldme584  please try find models on our updated Huggingface

I have the similar issue when photo inference:

get stuck at a model downloading with 855M. Can you show how to use pdb to trace? Thanks


I'm getting this error when running build_and_place_block_array

ValueError                                Traceback (most recent call last)
 in
----> 1 placed_spotmask = build_and_place_block_array(props_array, spot_mask, params, return_type='region')
~/Code/serology-COVID19/array_analyzer/extract/image_parser.py in build_and_place_block_array(props_array_, spot_mask_, params_, return_type)
602     target = np.zeros(spot_mask_.shape)
603     target[int(x_min-temp_origin[0]):int(x_min+template.shape[0]-temp_origin[0]),
--> 604            int(y_min-temp_origin[1]):int(y_min+template.shape[1]-temp_origin[1])] = template
605
606     if return_type == 'product':
ValueError: could not broadcast input array from shape (1248,1672) into shape (714,820)

Getting the same error on master. @bryantChhun - would you fix this. After that, I'll enable PR reviews for master to avoid accidental damage to master.

This is related to pixel size.  Ivan's PR #19 fixes this by giving you convenient block to hard code one or the other.

Because this is directly related to Issue #31 I am closing this.


Content: Student: BCSC: Create your profile

As I student I want clear instructions when logging in with my BC Services Card for the first time and creating my profile.
Acceptance Criteria

[ ] Common to Both Remove text: "Use your most up to date personal information. We'll use the same information to help you apply for financial aid."
[ ] Common to Both Revise text: "Profile Information" to "Profile information:
[ ] BCSC only Revise text: Remove "here" from sentence and add "." after "up-to-date."
[ ] Common to Both Revise info box header: "How to update your information" to "Information"
[ ] BCSC only Revise info box text:  "Your information above is taken from...." to "Your information above is taken from your BC Services Card account. If the information is incorrect, please update your information with BC Services Card account.

[ ] Common to Both Revise text: "Your consent" to "Declaration"

Additional Information:

[ ] Related to 3456 account declaration updates

Updated ticket to clarify between the common and login-specific items for BCeID/BCSC users

@bidyashish Thank you, definitely a miss on my part. Updated the AC.


Stop removing nil messages in stream responses

I was the original author of this change some time ago (#605), and I now think it is a mistake. If redis decides to return nil values then I don't think aioredis should eat them.
In my case I am receiving nil values when performing an XCLAIM. Why does this happen? I'm not sure*, but I know that I definitely want to be able to ACK these nil messages in order to stop my consumer continually reclaiming them.
Removing of nil values by aioredis means the response I get looks like, "you could not claim this message," not the more accurate, "you claimed this message but there is no body".

Perhaps they have been trimmed from the end of the stream but redis still sees them as claimed, so just returns nil rather than the message body.

Note that for now I am monkey patching this in Lightbus.

What do these changes do?
This change stops the removal of nil values returned from the stream commands.
Are there changes in behavior for the user?
As we don't really know why Redis returns nil values... maybe? See my speculation above.
Related issue number
My original issue of #605 which added this issue in the first place.
Checklist

[x] I think the code is well written
[ ] Unit tests for the changes exist
[x] Documentation reflects the changes (there was no documentation for the original change in #605)
[x] If you provide code modification, please add yourself to CONTRIBUTORS.txt

The format is <Name> <Surname>.
Please keep alphabetical order, the file is sorted by names.

[x] Add a new news fragment into the CHANGES/ folder

name it <issue_id>.<type> (e.g. 588.bugfix)
if you don't have an issue_id change it to the pr id after creating the PR
ensure type is one of the following:

.feature: Signifying a new feature.
.bugfix: Signifying a bug fix.
.doc: Signifying a documentation improvement.
.removal: Signifying a deprecation or removal of public API.
.misc: A ticket has been closed, but it is not of interest to users.

Make sure to use full sentences with correct case and punctuation, for example:
Fix issue with non-ascii contents in doctest text files.

I have now fixed the tests I caused to fail. There are some other failures, but they seem to be unrelated to my change.

@adamcharnock -
Thanks for your submission and apologies for the radio silence! Maintenance for this project has resumed.
Would you be willing to rebase your changes on the latest master and resolve your merge conflicts?
We can ignore the CI failures for now, there is a separate effort to get those fixed by migrating to GitHub Actions.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

No worries @seandstewart! Rebase done

@adamcharnock Awesome! Thank you. Just need you to sign the CLA and then we're good to merge.

@seandstewart – No problem, CLA signed :-)


problem to install pebbles (poutablueprint) on cPouta

Hi,
I encountered a problem when trying to install the Pebbles server on cPouta. I followed the steps according to the documentation http://<IP_ADDRESS>/pbdoc/how_to_install_on_cpouta.html#part-2-install-software, but the command
  $ ./pebbles/scripts/install_pb.bash
was aborted with the error message
TASK: [docker_host | Add cloud-user to docker group and create ssh key] *******
fatal: [localhost] => error importing module in /usr/lib/python2.7/site-packages/ansible/modules/core/system/user.py, expecting format like 'from ansible.module_utils.basic import *'

FATAL: all hosts have already failed -- aborting

I've checked the file /usr/lib/python2.7/site-packages/ansible/modules/core/system/user.py and the format is exactly the same as stated in the error message.
Any suggestions is more than welcomed.
Thanks very much!
/Nanjiang
The complete output of the command is attached
logfile_install_pebbles.txt

Hi,
I think your issue may be because you don't have ansible installed outside of virtual environments on your machine and the local command may expect to find it installed. Just a guess, I've run into it myself.
On a side note we are moving away from a bash-script and into a more pure Ansible deployment of pebbles https://github.com/CSC-IT-Center-for-Science/pebbles-deploy/ . It's still a work in progress so feedback on the documentation etc. is welcome.

Hi Jyrsa,
Thanks for you reply. But I do have ansible installed outside of virtualenv at /usr/lib/python2.7/site-packages/.
Best,
Nanjiang

And just to make sure,  you are using a Centos 7 image?

Yes. /Nanjiang

I suggest you try running the same Ansible command with one or more -v options to get more output. Also the version of Ansible might be to blame. Are you using 1.9.x or 2.x?

I found a problem with the version of ansible. I installed ansible 1.9.2 at /usr/lib/python2.7/site-packages/, but after running the script install_pb.bash, ansible was reinstalled with the 2.2.1
I tried to run pebbles/scripts/install_pb.bash by first activating the virtualenv and install dependencies with
pip install -r pebbles/requirements.txt
By doing so, I did make some progress. However, it finally aborted again at this step
TASK: [single_server_with_docker | Pull images] *******************************
changed: [localhost] => (item=redis:2.8)
changed: [localhost] => (item=postgres:9.3)

TASK: [single_server_with_docker | Bring up container for db] *****************
failed: [localhost] => {"changes": ["{\"status\":\"Trying to pull repository docker.io/library/postgres ... \"}\r\n", "{\"status\":\"Pulling from docker.io/library/postgres\",\"id\":\"9.3\"}\r\n", "{\"status\":\"Digest: sha256:5f81f286b90da10da7f014cd4947ba73c211aed55d45489ae960c4426e78ec03\"}\r\n"], "failed": true, "status": "Digest: sha256:5f81f286b90da10da7f014cd4947ba73c211aed55d45489ae960c4426e78ec03"}
msg: Unrecognized status from pull.

FATAL: all hosts have already failed -- aborting

It seems pulling the docker image from docker.io/library/postgres failed. However, when I run the command
docker pull docker.io/library/postgres
in the bash terminal, it worked well.
Any ideas?
Best,
Nanjiang

The install_pb.bash has not been designed to be run from inside a virtual environment.
Did you remember to run yum update? I tried the install script and it seems to work for me.
Are you maybe running out of disk space?

Oh, and I had forgotten to link to the documentation, it's now also hosted on GitHub http://csc-it-center-for-science.github.io/pebbles/ .

I've managed to install Pebbles on cPouta. The problem is with the version of ansible. Every time when I run the command
./pebbles/scripts/install_pb.bash
a folder pouta-blueprints at the same level of the folder of pebbles was created. The version of ansible specified in the script pouta-blueprints/scripts/install_pb.bash is 2.2.1 while the version of ansible specified in the script pebbles/scripts/install_pb.bash is <IP_ADDRESS>.
I solved the problem by modifying the version of ansible in the script pouta-blueprints/scripts/install_pb.bash to <IP_ADDRESS> and then run the command
./pouta-blueprints/scripts/install_pb.bash
I guess the problem is with the initializing script which creates the pouta-blueprints. Maybe a bug?`
Best,
Nanjiang

Can you please tell me the HEAD of the checkout you have under pebbles/ ? It sounds like an error I thought I fixed around early January.

Hi,
I think it is this one

95c2dd5 - (4 days ago) Update documentation -  (origin/gh-pages)

And I used this command to checkout the repo
$ git clone --branch v3.0.4 https://github.com/CSC-IT-Center-for-Science/pebbles.git
Best,
Nanjiang

On 20 Mar 2017, at 13:29, Jyrsa<EMAIL_ADDRESS>wrote:
Can you please tell me the HEAD of the checkout you have under pebbles/ ? It sounds like an error I thought I fixed around early January.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/CSC-IT-Center-for-Science/pebbles/issues/587#issuecomment-287745426, or mute the thread https://github.com/notifications/unsubscribe-auth/AFkD-rdm0xVFxO-vK0JC18m-FDaYIYL3ks5rnnEpgaJpZM4MfI3u.

Ahh, clearly an omission in the documentation. Instead of tag "v3.0.4" you should checkout the master branch instead. Just omitting the --branch option should fix your problem.

I see. I will try it soon.
Thanks for your help.
Best,
Nanjiang

On 20 Mar 2017, at 13:44, Jyrsa<EMAIL_ADDRESS>wrote:
Ahh, clearly an omission in the documentation. Instead of tag "v3.0.4" you should checkout the master branch instead. Just omitting the --branch option should fix your problem.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/CSC-IT-Center-for-Science/pebbles/issues/587#issuecomment-287748476, or mute the thread https://github.com/notifications/unsubscribe-auth/AFkD-jARkq4EMTs7_r1OmB7gjPWvCeIEks5rnnSZgaJpZM4MfI3u.

I'll mark this as closed. Reopen if it didn't succeed.


fix "attempt to index a number value" issue

Following on https://github.com/rcarriga/nvim-notify/issues/252, this PR implements the solution proposed by @ls-devs in https://github.com/rcarriga/nvim-notify/pull/251#issuecomment-1935492877
The difference in the config data can be seen bellow:

Before fix:

After fix:

@pfernandom Good job and thank you ! 👍🏼

This doesn't seem to work for me in the latest nightly. Latest nightly is built upon 170c890 and the previous one is built upon 32b4944. There must have been some other changes in between that affect this PR.

PR #253 already fixes this.

#253

Sorry @ls-devs , I missed your PR. Since my PR is based on yours, I can close mine. Of if you prefer it, we can merge mine.

@pfernandom Since my solution doesn't provide backwards-compatibility I think we should merge yours.

@ls-devs my PR is identical to yours. util.get_win_config does the same type-checking I was doing in my first draft.

@ls-devs my PR is identical to yours. util.get_win_config does the same type-checking I was doing in my first draft (which means both our PRs are backwards-compatible).

@pfernandom  Sorry I misunderstood something reading your PR. I thought I was missing something but it seems I'm not.
Let's go for #253 if you agree.


Unit tests for the SensitiveDataLogger

Motivation and Context (Why the change? What's the scenario?)
Contributing unit tests for the SensitiveDataLogger to cover a recently fixed bug and prevent future regression.
High level description (Approach, Design)
N/A - unit tests only

These unit tests provide coverage for the problem raised in #851

@microsoft-github-policy-service agree


Avoid crashing due to ErroneousElementX

ErroneousElementX seems to cause random crashes in places where it shouldn't.  We should fix that by having it make up answers instead of crashing.
Mostly, ErroneousElementX is used to tell the backend to generate code that throws an exception.  To all other users, including dartdoc, it is unfortunately a common source of crashes.
I'm pretty sure that 9917 is about ErroneousElementX, and I discovered that if you're using deferred loading, the compiler will crash if a static method is unresolved.
IIRC, Johnni was the original author, so I think he should be deciding to accept this issue or not.
Tentatively marking this as high as I generally raise the priority of crashes.  Not that there's a particular crash here, but it does raise the risk of crashes as the compiler changes.

The model we use now is that [Element]s can be malformed and backend ends have to check for that before asking any questions.


chore: doc hold Client for the app lifetime, log WARN 'connection health check failed'

chore:

doc hold Client for the app lifetime,
log WARN 'connection health check failed'

LGTM


Enable streaming of youtube ... and others

Reasonable if hard drive space is limited...

Ah ... the other guy found it in the configuration file :)
youtube_stream ...
Cannot find that in the devel branch, though.

I copy & pasted the code from master and it works now in my local devel branch.
There is now a new command .ytstream URL so now you can decide per URL whether to stream or to download.


Readme documents old --recursive flag instead of --norecursive

Description
Looks like duroc_hog behavior was changed such that it is recursive by default but the readme and various other files still refer to the old --recursive flag instead of the new --norecursive flag.
Steps to Reproduce
Running duroc_hog with the latest binaries and the documented recursive flag yields the following error:
./duroc_hog --recursive --prettyprint /path/to/repo
error: Found argument '--recursive' which wasn't expected, or isn't valid in this context
	Did you mean --norecursive?

USAGE:
    duroc_hog <FSPATH> --norecursive

For more information try --help

Expected Behaviour
README documentation matches execution behavior.
Relevant Logs / Console output
See above.
Your Environment
      System Version: macOS 10.15.6 (19G73)
      Kernel Version: Darwin 19.6.0
      Boot Volume: Macintosh HD

Additional context
None

Finished. Thanks for submitting this!


[DOCS] Add Translations README

⚠ IMPORTANT: Please fill out this template to give us as much information as possible to consider/implement this update.
Summary
We need to create a README.md file for the /Translations folder of Corona-tracker.
Motivation
This will help explain what the Translations folder represents in terms of the application, and some context to the files contained within. It should include:

[x] A link to the readme file of Corona-tracker
[x] A reference to the lead contact and discord link
[x] A basic description of the translation efforts
[x] A basic description of the repo files, e.g. Translations.md
[x] Links to external docs like translation spreadsheet
[ ] Optional: links to the org-level CoC, Contrib, and Support docs (they also show under "Helpful resources" section on the right of any issue/PR)

Possible Alternatives
Leave it as-is, which doesn't explain the goals of the group and/or this part of the project. It's also confusing if you open Translations.md in an effort to "see how I can help".
Additional Context

[x] update CoronaTracker readme when done with link to README.md instead of Translations.md

Completed this! Thanks @whoabuddy for pointing it out!

Is this good to close then @salvolpe ?

Oh yes! Didnt know if you want to look this over first, but I'm good with it!


✨ Add support of endpoint return type as response_model

Why this PR is open?
Tl;Dr; Generate endpoint response OpenAPI schema based on the endpoint return type.
In order to declare endpoint params' schema, you can just add typing to a param like this:
@app.get("/items/")
async def read_item(skip: int = 0, limit: int = 10):
    return fake_items_db[skip : skip + limit]

and it naturally generates OpenAPi spec for parameter types

The same applies to the request body, which is very convenient and feels natural and expected by the principal of least surprise 👍🏻
However, when you add return type - no response schema generated, return schema stays an arbitrary string.
@app.get("/items/")
async def read_item(skip: int = 0, limit: int = 10) -> List[Item]:
    return Item(fake_items_db[skip : skip + limit])

Unless you specify response_model to generate OpenAPI spec:
@app.get("/items/", response_model=List[Item])
async def read_item(skip: int = 0, limit: int = 10):
    return Item(fake_items_db[skip : skip + limit])

What this PR does?
It aligns behaviour of an endpoint parameter and return type to generate OpenAPI spec.
@app.get("/items/")
async def read_item(skip: int = 0, limit: int = 10) -> List[Item]:
    return Item(fake_items_db[skip : skip + limit])

It is also backward compatible with existing code that has to put response_model in FastAPI route decorator.
How this PR was tested?
pytest tests/test_response_class_is_return_type.py
./scripts/test.sh
and run modified docs_src/query_params/tutorial001.py locally.
WDYT?

oh.. I should have looked at closed PRs, issues and response_model design choice documentation first...
https://github.com/tiangolo/fastapi/issues/101
In general, I upvoted for the same arguments people provided in the comments. I think the Principle of least surprise is violated and it is important to fix it, even considering the tradeoff of having both response_model and return type hint as. it opens the opportunity for better UX and keeps the original solution for many more sophisticated cases like specifying a model with status and description.
Data-wise, it may help to analyze how often people just need anything more sophisticated than just one model type. 🤔
Feel free to close this PR though.

Found a lot more discussions about this already.
My opinion still stays the same, but it may be annoying to go over these issues, again and again, so closing this PR.
If anyone finds this, I would recommend looking at the small utility to get this as you need:
https://fastapi-utils.davidmontague.xyz/user-guide/inferring-router/


ZValues in chart.ContinuousSeries

So I was indeed able to use our features from #33 to plot my z values. But it was pretty ugly. I had to index the z value from the x and y float64 values. The only way I could see to do this was to hash x and y using sprintf("%.3f_%.3f", x, y) and map it to the z value:
zFirst := zSequence[0]
zMin, zMax := zFirst, zFirst
for _, z := range zSequence[1:] {
	if z < zMin {
		zMin = z
	}
	if z > zMax {
		zMax = z
	}
}

getHash := func(x, y float64) string {
	hash := sprintf("%.3f_%.3f", x, y)
	return hash
}

zHash := map[string]float64{}
for i, x := range xSequence {
	y := ySequence[i]
	z := zSequence[i]
	hash := getHash(x, y)
	zHash[hash] = z
}

viridisOfZ := func(xr, yr chart.Range, x, y float64) drawing.Color {
	hash := getHash(x, y)
	z := zHash[hash]
	return chart.Viridis(z, zMin, zMax)
}

I think it would be much better to just add and process an optional ZValues in chart.ContinuousSeries the same way XValues and YValues are processed. Then we would need to change DotColorProvider to have tho following signature:
type DotColorProvider func(xrange, yrange, zrange Range, x, y, z float64) drawing.Color

If the ZValues are not defined, the runtime can just send this function 0 for z and have zrange be from 0 to 0.
With this functionality added, my entire program above could be reduced to
viridisOfZ := func(xr, yr, zr chart.Range, x, y, z float64) drawing.Color {
	return chart.Viridis(z, zr.GetMin(), zr.GetMax())
}

I think you would have a much easier time implementing this one than I since you know the flow of the plotting section of the package.

zMin, zMax := zFirst, zFirst
for _, z := range zSequence[1:] {
	if z < zMin {
		zMin = z
	}
	if z > zMax {
		zMax = z
	}
}```

Can be re-written:

`zmin, zmax := chart.Math.MinAndMax(zSequence...)`

The best fix for your use case, as I understand it (you have a 3rd set of values, zSequence, which is the same length as xvalues and yvalues) would be to feed the `GetValue()` index to the color provider.

ok this has been updated

@wcharczuk Your solution of sending the index would definitely be workable. Are you saying you intended to add this change in #36? I didn't see it there.

i've already added the refactor for the provider into master

@wcharczuk Are you sure? It looks like the old signature is still here in [value_provider.go](https://github.com/wcharczuk/go-chart/blob/master/value_provider.go):
type DotColorProvider func(xrange, yrange Range, x, y float64) drawing.Color

sorry forgot to push, check again

Excellent. That fixes the issue for me.


ENH add install and import failure message

Solves #27
Instead of not raising an error in pip_install_in_env when PRINT_INSTALL_ERROR=False, now the error is always raised, but caught in DependenciesMixin.install.
In a second part of this PR, in  benchopt/runner.py, it prevents the solver from running if the module caught an ImportError in safe_import.

Codecov Report

Merging #29 into master will decrease coverage by 0.29%.
The diff coverage is 37.50%.

@@            Coverage Diff             @@
##           master      #29      +/-   ##
==========================================
- Coverage   54.04%   53.74%   -0.30%
==========================================
  Files           8        8
  Lines         618      627       +9
  Branches       96       97       +1
==========================================
+ Hits          334      337       +3
- Misses        254      261       +7
+ Partials       30       29       -1

PErfect!! Thanks a lot @TomDLT !! :smile:


creating the Appointment UI and installed val-liby

creating the appointment page and installed libraries and yup

changing some error messages


vector literal are incorrectly flagged

According to the vector literal docs, the code below should be correct and the compiler can actually compile it, but the move plugin complains incorrectly with the error described in the comments.
module Scratchpad::blank {
 use std::debug;
 use std::vector;

 #[test]
 public entry fun testit() {
  // Compiles correctly, but plugin gives out the error below:
  // ';', '=', '[' or '{' expected, got '['
  // <expression>, BECH32_ADDRESS, DIEM_ADDRESS, IDENTIFIER, INTEGER_LITERAL, PLACEHOLDER_ADDRESS or POLKADOT_ADDRESS expected, got '0'
  let foo = vector<u8>[0u8,1u8,2u8,3u8];
  debug::print<vector<u8>>(&foo);
 }
}

Using Aptos 0.3.2, on IntelliJ IDEA 2022.2.1 (Ultimate Edition), Build #IU-222.3739.54, built on August 15, 2022.
aptos --version
aptos 0.3.2

Which version of the language plugin is it? Could you update the plugin to the latest one (1.19.0) and check whether the error is still there? I've added support for vector literals in it.


Queue of backend breaking changes

Problem
This issue list a batch of breaking changes to backends that we want to ideally merge around the same time:

We have renamed NpLanguage to ExpressionWidth and removed the partial backend feature.

We need to remove the opcode supported fields in backends and rename language to expression_width in the info.json file

We have removed hash_to_field

We need to remove this from ACIR and regenerate the serialization

We want to add blake3 opcode

We need to add the opcode into ACIR and regenerate the serialization
Happy Case
.
Alternatives Considered
No response
Additional Context
No response
Would you like to submit a PR for this Issue?
No
Support Needs
No response

HashToField is being changed here: https://github.com/AztecProtocol/aztec-packages/pull/3631

This also includes #3840

Adding https://github.com/noir-lang/noir/pull/3909 to he list


Platform should supply mbedtls_platform_context  in different scenarios

Description

Type: Enhancement\Feature Request
Priority: Minor

Enhancement\Feature Request
Justification - why does the library need this feature?
mbedtls_platform_setup() and mbedtls_platform_teardown() receive mbedtls_platform_context as a parameter, but this opaque parameter is not used anywhere. there is a need for the platform to get this context from various locations, according to the different alternative implementations
For example, one platform would hold handles for the HW clocks to be used for crypto accelerations, another platform would hold some initial context as part of the platform context.
Suggested enhancement
There should be some sort of getter function.
Another idea, which involves changing APIs is having the cryptographic init functions receive the platform context as well, and thus the alternative implementation can bind the platform context to the crypto context, if needed.
raised in #1018

ARM Internal Ref: IOTSSL-1937


Allow overriding kwargs in debug_traceTransaction [APE-1272]

Overview
Right now, you are forced to always get memory and stack in debug_traceTransaction, even if you need it for what you are doing.
This can be super annoying for large traces and cause for workarounds and such
Specification
maybe something like:
def get_transaction_trace(memory=False)

Dependencies
Include links to any open issues that must be resolved before this feature can be implemented.

I am leaning on using kwargs or a dict because of how different other nodes like hardhat and foundry are with these kwargs.
Then each plugin can individually raise parsing errors on that dict (rather than ignoring unknown keys and stuff).


fix(subscriber-collection): use callable to remove subscribers

fix #585 #555
possibly fix aurelia/router#456

@bigopon could you describe what the issue was and what the fix is? I'm not clear on the scenario.

disregard my previous comment- I see the issue now. When finding the "rest" index we weren't checking for a match in both contexts and callables.
here's another take- merging some of the conditions... not sure if it's any better or worse.
  let rest = this._contextsRest;
  if (!rest || !rest.length){
    return false;
  }

  let index = 0;
  while ((index = rest.indexOf(context, index)) !== -1) && this._callablesRest[index] !== callable) { }
  if (index === -1) {
    return false;
  }
  rest.splice(index, 1);
  this._callablesRest.splice(index, 1);
  return true;

@jdanyow I think it will work in the same way without specifying a start index. We can use the scenario given in issue #585 to check it:
<!-- simplified version of bug report -->

<div repeat.for='item of items'>
  <my-el config.bind='item.prop[$config]'></my-el>
</div>

the subscriber list of $config will be something like 'sourceContext' * 100. But the callable will not be the same in the way string is. So (index = res.indexOf(context, index)) will be equal 1, instead of 0.

re: edit 2 in https://github.com/aurelia/binding/pull/622#issuecomment-331747634:
ok, let's consider getting rid of indexof altogether. Seems like we could simplify the logic by using while or indexof but not both. Also, since we know context tends to be the same while callable varies more, maybe we could optimize around that heuristic by comparing callable first, followed by context. For example, the && expression in these types of statements would be flipped:  if (this._context0 === context && this._callable0 === callable) { becomes  if (this._callable0 === callable && this._context0 === context) {.
The indexof + while could be replaced with a single while loop, which compares callable first:
const callables = this._callablesRest;
if (callables === undefined || callables.length === 0) {
  return false;
}
const contexts = this._contextsRest;
let i = 0;
while (callables[i] !== callable || contexts[i] !== context) {
  i++;
}
if (i >= callables.length) {
  return false;
}
contexts.splice(i, 1);
callables.splice(i, 1);
return true;

thoughts?
note: this is code I typed into the comments box for illustration purposes, not tested...

That looks a lot better !
I tested it, it runs indefinitely because we merge both conditions together. I think it just need a little tweak:
const callables = this._callablesRest;
if (callables === undefined || callables.length === 0) {
  return false;
}
console.log('callables not undefined');
const callableCount = callables.length;
const contexts = this._contextsRest;
let i = 0;
while (callables[i] !== callable && callableCount > i) {
  i++;
}
while (contexts[i] !== context && callableCount > i) {
  i++;
}
if (i === callables.length) {
  return false;
}
contexts.splice(i, 1);
callables.splice(i, 1);
return true;

In either way, we can't really avoid a length comparison. The reason is if neither callable nor context in parameters matches the callable or context at 0 index, it will quit immediately. I feel really hard to explain why it runs indefinitely but it happens if we merge conditions.

ah the dangers of coding in the comment box :)
const callables = this._callablesRest;
if (callables === undefined || callables.length === 0) {
  return false;
}
const contexts = this._contextsRest;
let i = 0;
while (!(callables[i] === callable && contexts[i] === context)) {
  i++;
}
if (i >= callables.length) {
  return false;
}
contexts.splice(i, 1);
callables.splice(i, 1);
return true;

Awesome, also made me realize I put in wrong test 😰
Doing it this way, however, cause the risk of it running forever if it fail to find a matching pair. should we put a constrain for the length ?

Ha yeah needs len check

Tested, all good with length, even when I tried to call removeSubscriber with invalid params 👍
One last thing though, indexOf seems to have better performance than while, should we resolve around it, or while works better in your benchmark ?

How are you comparing? What are you comparing? Which browsers?
To be thorough we should benchmark the different implantations. A jsperf would be fine. I’m good with merging the solution we’ve discussed and attempting to optimize further in another PR. In the past when I’ve tested these types of things there was a wide variation in indexof perf across browsers we support.

I see. All I did was a quick google for jsperf of for vs indexOf vs while
Let's do it in future, when things get more consistent

Sounds good. That jsperf isn't a good fit for our use-case because it's testing an array of integers which is a special case in some browsers, plus we have a two-dimensional comparison, and their "while" test is quite different than ours. I'll open another issue to track the benchmarking.

Great. Code & Test are ready I think. Please have a final check

looks good- tests passing. Another nice one :+1:


Possible LayoutConstraints issue

Hi Daniele,
Thanks again for sharing this library with the rest of the world :-). Not a big deal but I got these debug messages in my console. Nothing breaks visually but I was wondering if you had this console output as well or if that was specific to my project ?
Thanks again for your feedback.
[LayoutConstraints] Unable to simultaneously satisfy constraints.
	Probably at least one of the constraints in the following list is one you don't want.
	Try this:
		(1) look at each constraint and try to figure out which you don't expect;
		(2) find the code that added the unwanted constraint or constraints and fix it.
(
    "<NSLayoutConstraint:0x1c4299be0 V:|-(25)-[UIStackView:0x1034bf3a0]   (active, names: '|':UIView:0x10343e090 )>",
    "<NSLayoutConstraint:0x1c42987e0 V:[UIStackView:0x1034bf3a0]-(12)-|   (active, names: '|':UIView:0x10343e090 )>",
    "<NSLayoutConstraint:0x1c0292070 V:|-(0)-[UIView:0x10343e090]   (active, names: '|':SwiftToast.SwiftToastView:0x1034a9430 )>",
    "<NSLayoutConstraint:0x1c04823f0 V:[UIView:0x10343e090]-(0)-|   (active, names: '|':SwiftToast.SwiftToastView:0x1034a9430 )>",
    "<NSLayoutConstraint:0x1c029e050 SwiftToast.SwiftToastView:0x1034a9430.height <= 0   (active)>"
)

Will attempt to recover by breaking constraint
<NSLayoutConstraint:0x1c42987e0 V:[UIStackView:0x1034bf3a0]-(12)-|   (active, names: '|':UIView:0x10343e090 )>

Make a symbolic breakpoint at UIViewAlertForUnsatisfiableConstraints to catch this in the debugger.
The methods in the UIConstraintBasedLayoutDebugging category on UIView listed in <UIKit/UIView.h> may also be helpful.
2017-10-02 14:05:12.135542+0200 Sentinel[962:242550] [LayoutConstraints] Unable to simultaneously satisfy constraints.
	Probably at least one of the constraints in the following list is one you don't want.
	Try this:
		(1) look at each constraint and try to figure out which you don't expect;
		(2) find the code that added the unwanted constraint or constraints and fix it.
(
    "<NSLayoutConstraint:0x1c0292070 V:|-(0)-[UIView:0x10343e090]   (active, names: '|':SwiftToast.SwiftToastView:0x1034a9430 )>",
    "<NSLayoutConstraint:0x1c04823f0 V:[UIView:0x10343e090]-(0)-|   (active, names: '|':SwiftToast.SwiftToastView:0x1034a9430 )>",
    "<NSLayoutConstraint:0x1c029e050 SwiftToast.SwiftToastView:0x1034a9430.height <= 0   (active)>",
    "<NSLayoutConstraint:0x1c429a220 UIView:0x10343e090.height >= 64   (active)>"
)

Will attempt to recover by breaking constraint
<NSLayoutConstraint:0x1c429a220 UIView:0x10343e090.height >= 64   (active)>

Toast doesn't use autolayout for positioning. I don't know much about how UIStackView works, but I imagine it's probably not a good idea to add a toast as a subview.


Incompleteness with mutual recursive functions where recursion results are used as arguments to recursive calls

pub spec const NUM_LAYERS: nat = 4;
pub spec const NUM_ENTRIES: nat = 32;

pub enum Entry {
    Directory(Directory),
    Page(nat),
}

pub struct Directory {
    entries: Seq<Entry>,
}

#[verifier(external_body)]
pub struct Data { }

impl Data {

    pub open spec fn fn_one(self, layer: nat) -> Directory
        decreases NUM_LAYERS - layer, NUM_ENTRIES, 2nat
    {
        Directory { entries: self.fn_three(layer, seq![]) }
    }

    pub open spec fn fn_two(self, layer: nat, idx: nat) -> Entry
        decreases NUM_LAYERS - layer, NUM_ENTRIES - idx, 0nat
    {
        if layer + 1 <= NUM_LAYERS {
            Entry::Directory(self.fn_one(layer + 1))
        } else {
            arbitrary()
        }
    }

    pub open spec fn fn_three(self, layer: nat, init: Seq<Entry>) -> Seq<Entry>
        decreases NUM_LAYERS - layer, NUM_ENTRIES - init.len(), 1nat
        via Self::termination_fn_three
    {
        if init.len() >= NUM_ENTRIES {
            init
        } else {
            let entry = self.fn_two(layer, init.len());
            self.fn_three(layer, init.add(seq![entry]))
        }
    }

    #[verifier(decreases_by)]
    proof fn termination_fn_three(self, layer: nat, init: Seq<Entry>) {
        let num_entries: nat = NUM_ENTRIES as nat;
        if init.len() >= num_entries {
            assume(false);
        } else {
            // let entry = self.fn_two(layer, init.len());
            // self.fn_three(layer, init.add(seq![entry]))
        }
    }

}

Fails the recursion check for fn_three, but replacing
            let entry = self.fn_two(layer, init.len());

with
            let _ = self.fn_two(layer, init.len());
            let entry: Entry = arbitrary();

makes it succeed.
Initially reported by @matthias-brun.

[ ] un-ignore test from https://github.com/verus-lang/verus/commit/8ba0e7aad7fdf865ebe1c778c78489f0a6d7d33a

Smaller example:
use vstd::prelude::*;

verus! {

pub spec const NUM_LAYERS: nat = 4;
pub spec const NUM_ENTRIES: nat = 32;

pub open spec fn fn_two(layer: nat, idx: nat) -> nat
    decreases NUM_LAYERS - layer, NUM_ENTRIES - idx, 0nat
{
    if layer + 1 <= NUM_LAYERS {
        fn_three(layer + 1, seq![]).len()
    } else {
        arbitrary()
    }
}

pub open spec fn fn_three(layer: nat, init: Seq<nat>) -> Seq<nat>
    decreases NUM_LAYERS - layer, NUM_ENTRIES - init.len(), 1nat
    via termination_fn_three
{
    if init.len() >= NUM_ENTRIES {
        init
    } else {
        let entry = fn_two(layer, init.len());
        fn_three(layer, init.add(seq![entry]))
    }
}

#[verifier(decreases_by)]
proof fn termination_fn_three(layer: nat, init: Seq<nat>) {
    let num_entries: nat = NUM_ENTRIES as nat;
    if init.len() >= num_entries {
    } else {
        // let entry = fn_two(layer, init.len());
        // fn_three(layer, init.add(seq![entry]))
    }
}

fn main() {}

} // verus!

@utaal-b These tests pass now, and https://github.com/verus-lang/verus/pull/849 changed the termination checking for spec functions to be like that of exec functions, in line with the comment above.  Should this issue be considered fixed?

I think there may had been cases we found that we still wouldn't be able to check, but I am happy to consider this fixed, and then open a new issue if those come up again.
Thank you!


In source code view , ,  is not getting saved

In the source code view when we type <!DOCTYPE html>  and <html> on the top of the code and try to save it. It doesn't get saved. Even when I switch over to 'source' and 'editor' view the <!DOCTYPE html> is disappearing.
All these elements <!DOCTYPE html> , <html> , <body> gets disappear in codeview. Watch this video https://youtu.be/7T-7W5aOyqc
If you get this fixed. Please reply me.

Is there any update to this?  We would really like to use the Jodit Editor; however, with the html, head, body, and other tags being removed, we're limited.

With the html, head, body, and other tags being removed, we're limited.
Soo true

Duplicate of #174


Fixes #179 & avoid blocking state

Ensures clients don't remain blocked in the aggregation phase, in case other clients disconnect before posting their necessary weights for the current round to complete. To do this, asking the server whether the round's model was aggregated can now potentially trigger this very aggregation.
Fixes #179: here is a brief overview of what's going on, where client 3 is "late".  are the selection, training and aggregation phases, respectively, for round . During selection phase, a client periodically asks the server to get selected for the next upcoming round. Then, each client trains locally for a few epochs. Ultimately, selected clients enter aggregation phase during which they periodically ask the server whether the round's model was aggregated yet. Additionally, with this PR, client 3 was "late" and thus not selected, so it directly goes from  to  to ensure it gets the latest model.

looks, ok, but i wonder if the selection phase could be simplified more, taking a slightly more server-centric view? the server could just send a "you are selected" signal to the ones it thinks are alive and selected? (and then sends the model belonging to the round?). anyway i guess it's a push vs pull type of question.
and yes, received model updates which were computed based on an older round can simply be dropped on the server. later we'll slightly relax this when you add the nice buffer based async variant

looks, ok, but i wonder if the selection phase could be simplified more, taking a slightly more server-centric view? the server could just send a "you are selected" signal to the ones it thinks are alive and selected? (and then sends the model belonging to the round?). anyway i guess it's a push vs pull type of question.

Yeah I agree, I feel like this might simplify the code, it would be great if @s314cy could discuss the current implementation in more detail in one of the next meetings, we could then brainstorm the most simple / elegant solution.


Manage user environment

It would be amazing if pipenv could be used to manage the users local environment, that is often used to have python software readily available.
Basically, if I need for example borgbackup I install it using
pip3 install --user borgbackup

This in turn drops the necessary packages into ~/.local/lib/python3.6/ and executables into ~/.local/bin, so I have borg in my $PATH.
If I could now manage this environment with pipenv, that'd be swell.

Why not just use a pipenv project?

i think what you want is pipsi https://github.com/mitsuhiko/pipsi

@mweinelt You may be interested in https://github.com/ofek/hatch where you can choose between a project or environment based work flow.
Your example command would be exactly equal to hatch install borgbackup when not in a virtual env.


Testing for upcoming release

I'd release the v1.5.2 ~today at 18:00 UTC~ very soon. Features are fixed and can be found in the changelog text file.
If anyone discovers a bug, please post it here!
Note that the deprecated commits before july 2016 will get sqashed then, so local repositories will require a rebase.

Haven't had time lately (was focused on cryptocurrencies ;) but I'll try to test this asap.

@FooDeas I enabled the following:
hdmi_monitor_res=640x480
hdmi_disable_overscan=1
console_blank=0
quiet_boot=1
disable_raspberries=1
disable_splash=1
cleanup=1
cleanup_logfiles=1

And something went wrong because the resulting config.txt contains:
[all]
initramfs raspberrypi-ua-netinst/raspberrypi-ua-netinst.cpio.gz
[pi3]
enable_uart=1

# Only for RPi model 3: The following line enables the ability to boot from USB.
# Notice: This flag will be written to OTP and is permanent.
#program_usb_boot_mode=1
disable_overscan=1
disable_overscan=1
disable_overscan=1
disable_overscan=1
disable_overscan=1
disable_overscan=1
disable_overscan=1
disable_overscan=1
disable_overscan=1
disable_overscan=1
disable_overscan=1
disable_overscan=1
disable_overscan=1
disable_overscan=1
disable_overscan=1
disable_overscan=1
disable_overscan=1
disable_overscan=1
disable_overscan=1
disable_overscan=1
disable_overscan=1
disable_overscan=1

And I couldn't find any error log..

any idea @FooDeas?

Yep. Just a bit busy atm.

Finally! The disable_overscan bug is fixed.
Fya: hdmi_monitor_res=640x480 doesn't make sense and gets ignored if hdmi_type isn't set to monitor.

It's a 640x480 VGA monitor with a VGA to HDMI converter..

Tested, and everything looks fine. Files get cleaned up. Logs as well. Please release these goodies, cheers!

One thing though;
I have a master config and slave configs that extend this master file. You suggested to add this at the top of these slave configs:
sanitize_inputfile /bootfs/raspberrypi-ua-netinst/config/installer-config-common.txt
source /bootfs/raspberrypi-ua-netinst/config/installer-config-common.txt

But I'm now seeing an error about sanitize_inputfile not found.

Replacement: inputfile_sanitize

Ok thanks. And about disabling the raspberries; I still see them during installation (but this could be related to inputfile_sanitize and my master config not being read). Will test again.

Boot and kernel config (like the logo) except some device settings will be set to the system only.

Done.


GenomicRanges=>SnaptronQueryBuilder support

It would be great if Snapcount could take a GenomicRanges object of one or more ranges and create a list of SnaptronQueryBuilder objects, one per input range.  As part of this, it would be good if the function to do this could support defining one set of filters (range & sample) and a coordinates modifier (e.g. exact) that would apply to the whole set of ranges.  That way, the user could submit a set of ranges that will all have the same filters and/or modifier applied when queried.

This has already been implemented, albeit slightly different than described here. Snapcount can accept a GenomicRange object and will return a list of results to the user. The user can opt to merge the results into one table or SummarizedExperiment depending on their choice of output.


errors: better godoc

errors package needs better documentation.
much of the package documentation (eg. for NewClient) should be moved or copied down to the corresponding declaration.
cc @jba @bradfitz

Closed via fc7c457e4da4896cd96cc62202d79ba8c68a9d15.


GO:0061611 mannose to fructose-6-phosphate metabolic process

Looking at these 2 terms:
id: GO:0019309
name: mannose catabolic process
namespace: biological_process
def: "The chemical reactions and pathways resulting in the breakdown of mannose, the aldohexose manno-hexose, the C-2 epimer of glucose." [GOC:ai]
synonym: "mannose breakdown" EXACT []
synonym: "mannose catabolism" EXACT []
synonym: "mannose degradation" EXACT []
xref: MetaCyc:MANNCAT-PWY
xref: MetaCyc:PWY3O-1743
intersection_of: GO:0009056 ! catabolic process
intersection_of: has_primary_input CHEBI:37684 ! mannose
property_value: term_tracker_item "https://github.com/geneontology/go-ontology/issues/27107" xsd:anyURI
id: GO:0061611
name: mannose to fructose-6-phosphate metabolic process
namespace: biological_process
def: "The chemical reactions and pathways in which mannose, the aldohexose manno-hexose, is converted to fructose-6-phosphate."  D-fructose 6-phosphate(2-)
intersection_of: starts_with GO:0019158 ! mannokinase activity
created_by: dph
creation_date: 2014-04-03T13:38:50Z
The MetaCyc xrefs currently on GO:0019309 are:
D-mannose degradation I (MANNCAT-PWY)
D-mannose -> fructose-6-phosphate, including transport into cell (via mannokinase & mannose-6-phosphate isomerase)
(archaea and bacteria)
D-mannose degradation II (PWY3O-1743)
D-mannose -> fructose-6-phosphate (via mannokinase & mannose-6-phosphate isomerase)
(eukaryotes, archaea, bacteria)
Given the above, I'd suggest:

[ ] Rename GO:0061611 to 'mannose to fructose-6-phosphate catabolic process'
[ ] Change the LD to make GO:0061611 a child of 'GO:0019309 mannose catabolic process'
[ ] Move the MetaCyc xrefs to GO:0061611.

What do you think @ukemi  and @deustp01 ?
Note that the following term  already classifies as a child of GO:0019309 mannose catabolic process:
id: GO:0061619
name: glycolytic process from mannose through fructose-6-phosphate
namespace: biological_process
def: "The chemical reactions and pathways resulting in the breakdown of mannose into pyruvate, occurring through a fructose-6-phosphate intermediate, with the concomitant production of ATP and NADH." [GOC:dph, ISBN:0201090910, ISBN:0879010479]
intersection_of: GO:0061615 ! glycolytic process through fructose-6-phosphate
intersection_of: has_primary_input CHEBI:37684 ! mannose
intersection_of: starts_with GO:0061611 ! mannose to fructose-6-phosphate metabolic process
created_by: dph
creation_date: 2014-04-14T09:13:13Z
@rozaru

Decision on call was that proposals above are reasonable and should proceed.


How to disable metric collection?

Hi!
I'm sorry for opening the new issue, but I didn't find the answer on my question in docs or in any other issues here.
I want to run Ops Agent only for Logging purposes. I don't need metrics to be collected from my GCE instance. How can I disable the metrics part of the Ops Agent completely?
I'm trying this config, but it doesn't help:
# /etc/google-cloud-ops-agent/config.yaml
# <...>
metrics:
  processors:
    metrics_filter:
      type: exclude_metrics
      metrics_pattern: ["agent.googleapis.com/*"]
  service:
    pipelines:
      default_pipeline:
        receivers: []
        processors: []
        exporters: []

How I validate:
My GCE instance doesn't have the role roles/monitoring.metricWriter, so I see these messages in syslog:
May  3 12:00:06 mbr-stage-mt-24-euw1-c9p1 otelopscol[1053890]: 2022-05-03T12:00:06.431Z#011error#011collector@v0.26.1-0.20220307211504-dc45061a44f9/metrics.go:235#011could not export time series to GCM#011{"error": "rpc error: code = PermissionDenied desc = Permission monitoring.timeSeries.create denied (or the resource may not exist)."}
May  3 12:00:06 mbr-stage-mt-24-euw1-c9p1 otelopscol[1053890]: github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/collector.(*MetricsExporter).exportPendingTimeSerieses
May  3 12:00:06 mbr-stage-mt-24-euw1-c9p1 otelopscol[1053890]: #011/root/go/pkg/mod/github.com/!google!cloud!platform/opentelemetry-operations-go/exporter/collector@v0.26.1-0.20220307211504-dc45061a44f9/metrics.go:235
May  3 12:00:06 mbr-stage-mt-24-euw1-c9p1 otelopscol[1053890]: github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/collector.(*MetricsExporter).exportTimeSeriesRunner
May  3 12:00:06 mbr-stage-mt-24-euw1-c9p1 otelopscol[1053890]: #011/root/go/pkg/mod/github.com/!google!cloud!platform/opentelemetry-operations-go/exporter/collector@v0.26.1-0.20220307211504-dc45061a44f9/metrics.go:103

I want the metric collection to be disabled, so it won't even try to push them anywhere.
Could you please suggest the right configuration for that? Thank you!
P.s. Ideally, it would be good if google-cloud-ops-agent-opentelemetry-collector won't run at all.

otel part of ops-agent can't be turned off completely. You can disable it to not send hostmetrics, it's still there to send out agent self metrics, e.g. agent/uptime： https://cloud.google.com/monitoring/api/metrics_opsagent#agent-agent. Agent metrics is free.

Thank you, @sophieyfang!
Could you please suggest, how the metrics config should look in this case?

I tried using the same configuration which I mentioned in the first message and also added the role roles/monitoring.metricWriter to the instance's service account and now the error message is gone.
Also, it seems that only free agent metrics are sent to Google with that config. Thank you!


testdata/vcstest/bzr: fix unit test with Ubuntu 2020 LTS bzr 3.0.2

As pointed by @bcmills, upstream fixed this in 3.1.0 in https://github.com/breezy-team/breezy/commit/30143860ced0ad9f42cbf18a119f2800117decff
Fixes #62620.

CLA signed.


fix unused test imports on non-nightly, prevent regression

Believe this has been proven out more than sufficiently at this point (failed action runs are obfuscated away from the PR due to squashing but can still be located, e.g. https://github.com/rust-lang/rustfmt/runs/5313881057?check_suite_focus=true)


Create a UnitTest library for Member libraries

Library name(s)
UT.Generators
Description
Create the Unit Testing library

Total: 10m


Fix crash when selecting re-added TreeItem::Cell

Fixes #88889.
When removing a TreeItem from a Tree it doesn't clear the selected property of the cells in the TreeItem, which means that if you remove this TreeItem from the Tree and then add it back again, and then select the same item/cell, you wouldn't pass this condition:
https://github.com/godotengine/godot/blob/a586e860e5fc382dec4ad9a0bec72f7c6684f020/scene/gui/tree.cpp#L2664-L2668
Which then left selected_item at nullptr and would subsequently crash here:
https://github.com/godotengine/godot/blob/a586e860e5fc382dec4ad9a0bec72f7c6684f020/scene/gui/tree.cpp#L3736
This PR fixes this by simply clearing the selected property of all the item's cells when it's removed from the tree.

Thanks!

Cherry-picked for 4.2.2.


fix(runtime-manifest): defer to bundle-level output def sensitivity

What does this change

Updates logic around masking output values.  We still regard step outputs as sensitive by default, but this updates step outputs that are also bundle-level outputs to inherit the bundle-level output setting.  (Note that bundle-level outputs are not sensitive by default.)

What issue does it fix
N/A
Notes for the reviewer
Checklist

[x] Unit Tests
[ ] Documentation
[ ] Schema (porter.yaml)

If this is your first pull request, please add your name to the bottom of our Contributors list. Thank you for making Porter better! 🙇♀

Thanks @carolynvs ; indeed, your recent updates have made this change set much simpler!  Rebased and ready for another look when convenient.


bug: Allow sign and verify to use different hashes than sha256

closes #52

@jcjones, thanks for the r? I'm always happy to get whatever nits or oversights folks are willing to offer, so thank you for your time. Will address and push a new version shortly.


Block producer scaffold

Work towards: #36
Scaffold for the block producer task
Builds on #34.

Closing as outdated; will be implemented later, and differently.


Wait until a nREPL connection is created?

I have developed a plugin that adds some middleware. But if I try to call server.with-conn-and-ops-or-warn then it doesn't crash when the REPL isn't connected. So the call either results in a NOOP or it succeeds. It's random.
So I would like to have some kind of a way to know that it succeeded.
The callback would also be useful when Conjure decides to reconnect to a different REPL. It could notify the child plugins about it.
The alternative could be that I would try to brute-force that function until it "connects" but it's not a good thing to do.

The function takes a opts table which can contain an :else function which will be called if there is no current connection. You could use that to schedule a retry until the REPL is connected. This is pretty internal and undocumented though so YMMV over time if I end up refactoring this code one day. Should be stable for a long time though.
I think you'll just have to check the state of the client and inspect the REPL it's connected to if you want to know if you're still connected to the same one. Not planning on adding plugin-plugin support (plugins for Conjure as a first class thing), I'd rather they piggieback on what is already there or are added to the core through pull requests etc. Some core changes can be made to accommodate plugins but I don't plan on adding notification systems, hooks, extra APIs and deep internal documentation for every internal function (there's a LOT of code I've written over years and it changes, it'd be really hard to maintain that right now).
You can also use this to see if there's a connection https://github.com/Olical/conjure/blob/08236a13164a948fe403f34fe1f593ead0a3c6fe/fnl/conjure/client/clojure/nrepl/server.fnl#L25-L28
Or inspect the state and get the current connection like so: https://github.com/Olical/conjure/blob/08236a13164a948fe403f34fe1f593ead0a3c6fe/fnl/conjure/client/clojure/nrepl/server.fnl#L16
If you're going to be reaching into the internals of Conjure and poking things, might as well reach deeper and access the data you're interested in. It's all there, you have access to it, just have to dig through the code a little to see how I'm doing it. Shouldn't need anything extra added to accommodate further plugins (although I'd probably refer to them as mods since they're piggiebacking and modifying Conjure while it's running?).

I already did what you described:
https://github.com/Invertisment/conjure-clj-additions/blob/master/fnl/conjure-clj-additions/additional-fns.fnl#L47-L52
https://github.com/Invertisment/conjure-clj-additions/blob/master/fnl/conjure-clj-additions/additional-fns.fnl#L55-L56
Even the timer:
https://github.com/Invertisment/conjure-clj-additions/blob/master/fnl/conjure-clj-additions/load-util.fnl#L8-L24
The only problematic thing is that sometimes the test middleware fails to load and then I must restart the REPL as even the editor restarts don't help. That I can't understand yet.
Also this timer loading mechanism sometimes produces multiple messages when it fails to connect and I think it's somehow related to vim.schedule_wrap. When I jump through buffers several times rapidly then it produces some errors as the timer still goes on.


[BUG] AppendChildResource produces incorrect ResourceType

Describe the bug
new ResourceIdentifier("/subscriptions/794307ce-39e3-4505-bce1-23bd9837bcc7/resourceGroups/g1")
  .AppendProviderResource("microsoft.storage", "accounts", "ac1")
  .AppendChildResource("containers", "c1").ResourceType

Expected behavior
Microsoft.Storage/accounts/containers
Actual behavior (include Exception or Stack Trace)
Microsoft.Resources/containers
To Reproduce
See above
Note that calling new ResourceIdentifier(identifer.ToString()).ResourceType  produces the correct result.
Environment:

Name and version of the Library package used: Azure.ResourceManager, 1.0.0-beta.5
Hosting platform or OS and .NET runtime version (dotnet --info output for .NET Core projects): Windows 11, .NET 5.0
IDE and version : VS 2022

Thank you for your feedback.  Tagging and routing to the team member best able to assist.

Hi @aelij we recently did a little bit refactor on the ResourceIdentifier class. We moved it to Azure.Core.
In the latest version of Azure.Core (1.22.0), you could find this class, and your code is working as expected. Therefore I believe the bug should be fixed.
using Azure.Core;

var id = new ResourceIdentifier("/subscriptions/794307ce-39e3-4505-bce1-23bd9837bcc7/resourceGroups/g1").AppendProviderResource("microsoft.storage", "accounts", "ac1").AppendChildResource("containers", "c1");
Console.WriteLine(id.ToString());
Console.WriteLine(id.ResourceType);

Result of this program:

Please give a try with the latest version of Azure.Core to see if this resolves your issue. Thank you!

Thanks, looks like it's fixed


test: Disable GPU on Chrome and Edge

This clones the most useful Chrome options for Edge, and adds --disable-gpu for both.  This should make rendering and layout tests more stable across devices.

This doesn't have the effect I had hoped.


[Bug] pyinstaller build raises "Cannot load native module 'Cryptodome.Util._strxor'"

Hi,
I'm coming to this project after using youtube-dl.exe for a long time. So I download the latest release, and I try running it via command line, but I get the following error.

Traceback (most recent call last):
File "yt_dlp_main_.py", line 16, in
File "", line 991, in _find_and_load
File "", line 975, in _find_and_load_unlocked
File "", line 671, in load_unlocked
File "PyInstaller\loader\pyimod03_importers.py", line 546, in exec_module
File "yt_dlp_init.py", line 16, in
File "", line 991, in _find_and_load
File "", line 975, in _find_and_load_unlocked
File "", line 671, in _load_unlocked
File "PyInstaller\loader\pyimod03_importers.py", line 546, in exec_module
File "yt_dlp\options.py", line 8, in
File "", line 991, in _find_and_load
File "", line 975, in _find_and_load_unlocked
File "", line 671, in _load_unlocked
File "PyInstaller\loader\pyimod03_importers.py", line 546, in exec_module
File "yt_dlp\compat.py", line 154, in
File "", line 991, in _find_and_load
File "", line 975, in _find_and_load_unlocked
File "", line 671, in load_unlocked
File "PyInstaller\loader\pyimod03_importers.py", line 546, in exec_module
File "Cryptodome\Cipher_init.py", line 33, in
File "", line 991, in _find_and_load
File "", line 975, in _find_and_load_unlocked
File "", line 671, in _load_unlocked
File "PyInstaller\loader\pyimod03_importers.py", line 546, in exec_module
File "Cryptodome\Cipher_mode_ccm.py", line 44, in
File "", line 991, in _find_and_load
File "", line 975, in _find_and_load_unlocked
File "", line 671, in _load_unlocked
File "PyInstaller\loader\pyimod03_importers.py", line 546, in exec_module
File "Cryptodome\Util\strxor.py", line 35, in
File "Cryptodome\Util_raw_api.py", line 309, in load_pycryptodome_raw_lib
OSError: Cannot load native module 'Cryptodome.Util._strxor': Not found '_strxor.cp38-win_amd64.pyd', Cannot load '_strxor.pyd': Failed to load dynlib/dll 'C:\Users\\AppData\Local
\Temp\_MEI689362\Cryptodome\Util\..\Util\_strxor.pyd'. Most probably this dynlib/dll was not found when the application was frozen.
[8412] Failed to execute script 'main' due to unhandled exception!

Avast detected "_strxor.pyd" as PUP, might go away in day or two.
https://www.virustotal.com/gui/file/e019ca0334e239970a3ba625bbd24c37d8a3c3b5655015471780d4fb7f5a1a98?nocache=1


[W-11690081] fix some typos in v1.1

ref: W-11690081
fix some typos.
Writer's Quality Checklist
Before merging your PR, did you:

[x] Run spell checker
[ ] Run link checker to check for broken xrefs
[ ] Check for orphan files
[x] Perform a local build and do a final visual check of your content, including checking for:

Broken images
Dead links
Correct rendering of partials if they are used in your content
Formatting issues, such as:

Misnumbered ordered lists (steps) or incorrectly nested unordered lists
Messed up tables
Proper indentation
Correct header levels

[ ] Receive final review and signoff from:

Technical SME
Product Manager
Editor or peer reviewer
Reporter, if this content is in response to a reported issue (internal or external feedback)

[ ] If applicable, verify that the software actually got released

thank you gary!


docs: fixing anti-pattern in one of the hooks state examples

From React.Component setState function documentation:

Subsequent calls will override values from previous calls in the same cycle, so the quantity will only be incremented once. If the next state depends on the current state, we recommend using the updater function form.

In one of the Hooks examples state is updated based on the this.state value which potentially may lead to incorrect result.

@facebook-github-bot check


Add a way for hooks to mutate/write entry Data fields

Hooks should have a way to safely(re goroutine safe) write/update/add fields to entries. There are several issues stemming from the data field not having a lock and hooks not having any way to mutate fields.
One option is to switch from using a hook to using a formatter but you cannot easily wrap formatters while safely modifying fields. ie you cannot directly modify Data due to race conditions, you cannot use entry.WithFields then pass to say json formatter because that screws up entry release/locks/buffer usage, etc.
Use Case
Our use case is a simple hook which adds a hostname to every log entry to help differentiate which server a log came from.
Potential solutions
All current solutions require forks/minor changes if you are using the JSON formatter due to internal fields. They do not all touch this though, only the use a formatter option below(2nd option).
Improve Hooks API(best)
Adding a way for hooks to return a new entry to replace the current one would allow easily modifying/adding new fields. This would probably have the lowest api impact and be the easiest to implement. Hooks could optionally implement a MutatbleHook interface which would take precident over the existing Fire. ie something like FireAndMutate(*Entry) (*Entry, error) where the returned entry is passed onto the next hook and used if non-nil. If any hook returned a new entry then it could be used in place of the current one.
Use a formatter instead(alternative, also very good if formatters could be wrapped easily while modifying entries)
The only real current option around is to modify the json formatter or your target formatter to directly include your new/modified/updated field logic. Even doing this hardly works and requires a fork of the repo due to the Entry.err field being private.
The main issue with this is that Entry.WithField does not create a full copy(ie the error, buffer, and potentially other fields are not copied). It also has issues with releasing entries when calling from within a formatter. I did not fully investigate this although a simple option is to modify the underlying formatter(ie JSON) itself to simply add the field before sending to json.Encoder.Encode. This side steps all the issues/perf with creating entry copies/modifications.
Add Data locks
Another option is to add a mutex around the Data field so that hooks could safely modify them.
See the following related issues.
https://github.com/sirupsen/logrus/issues/1046
https://github.com/sirupsen/logrus/issues/953
https://github.com/vmware-tanzu/velero/issues/599

I've added a PR for a solution which we are using. It happens to be the simplest to implement and also the most performant but requires updating each formatter to support it. Luckily I believe most people use the JSON formatter.
If you are sending logs elsewhere then you can implement a similar feature by copying data fields within a Hook then send the log entry to a third party.

I've added a PR for a solution which we are using. It happens to be the simplest to implement and also the most performant but requires updating each formatter to support it. Luckily I believe most people use the JSON formatter.
If you are sending logs elsewhere then you can implement a similar feature by copying data fields within a Hook then send the log entry to a third party.

Hello @epelc, your analysis is quite complete.
I think your fix would solve your issue, but as you've noticed it won't solve issues on other formatter.
I'd rather not merge that as logrus code is kind of already bloated of several minor features only a few users are using.
I'm currently working on per entry rwlock fix, the lock can then activated at logger configuration level in order to not have a performance drawback for users not having modifying hook.
Just curious, as another work around, why don't you just set up a new entry as the default logger for your application instead of using a hook for setting a value (the hostname) which never changes throughout the life of your application.
defaultLogger := logrus.WithField("hostname", hostname)

You just need then to cascade the logger through your application stack.

@dgsb Thank you for the response! Also very excited to hear about the fixes for lock support.
I agree this may be too specific a use case to accept. The lock support should resolve this without bloat.
The only real thing I'd note is that it'd be nice to be able to implement a modified JSON formatter without forking. ie make the internal APIs it uses public in some way. This fork has been working very well for us for the past month though.
I think the main issue with the defaultLogger suggestion is that not all code has a global logger and it'd be a lot of work to update everything to use it. It would work though if you started a fresh project.

I have a fix which doesn't add any interface changes nor a mutex usage. You may want to check the next release of logrus if you're interested.

@epelc can we close this issue ?

@dgsb yes the locking you added should fix it.


How to get the recommended video list?

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

[X] I understand that I will be blocked if I intentionally remove or skip any mandatory* field

Checklist

[X] I'm asking a question and not reporting a bug or requesting a feature
[X] I've looked through the README
[X] I've verified that I have updated yt-dlp to nightly or master (update instructions)
[X] I've searched known issues and the bugtracker for similar questions including closed ones. DO NOT post duplicates
[X] I've read the guidelines for opening an issue

Please make sure the question is worded well enough to be understood
yt-dlp -v --no-playlist --cookies ./cookies.txt --skip-download --print "%(original_url)s;%(title)s;%(uploader)s" https://www.youtube.com/

I want to use yt-dlp to fetch the recommended video list on the YouTube homepage. However, the efficiency of this operation is low, and the printing speed is slow. It takes more than 1 second on average to print each item. Is there any optimization solution?
Provide verbose output that clearly demonstrates the problem

[X] Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
[X] If using API, add 'verbose': True to YoutubeDL params instead
[X] Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output
yt-dlp -v --no-playlist --cookies ./cookies.txt --skip-download --print "%(original_url)s;%(title)s;%(uploader)s" https://www.youtube.com/
[debug] Command-line config: ['-v', '--no-playlist', '--cookies', './cookies.txt', '--skip-download', '--print', '%(original_url)s;%(title)s;%(uploader)s', 'https://www.youtube.com/']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version<EMAIL_ADDRESS>from yt-dlp/yt-dlp [ff0779267] (zip)
[debug] Python 3.11.2 (CPython aarch64 64bit) - Linux-6.6.20+rpt-rpi-v8-aarch64-with-glibc2.36 (OpenSSL 3.0.11 19 Sep 2023, glibc 2.36)
[debug] exe versions: ffmpeg 5.1.4-0 (setts), ffprobe 5.1.4-0
[debug] Optional libraries: Cryptodome-3.11.0, brotli-1.0.9, certifi-2022.09.24, requests-2.28.1, sqlite3-3.40.1, urllib3-1.26.12
[debug] Proxy map: {}
[debug] Request Handlers: urllib
[debug] Loaded 1810 extractors
[youtube:recommended] Extracting URL: https://www.youtube.com/
[youtube:tab] Extracting URL: https://www.youtube.com/feed/recommended
[youtube:tab] recommended: Downloading webpage
[download] Downloading playlist: recommended
[debug] [youtube:tab] Extracted SAPISID cookie
[debug] [youtube:tab] Copying __Secure-3PAPISID cookie to SAPISID cookie
[youtube:tab] recommended page 1: Downloading API JSON
[youtube:tab] recommended page 2: Downloading API JSON
[youtube:tab] recommended page 3: Downloading API JSON
[youtube:tab] recommended page 4: Downloading API JSON
[youtube:tab] recommended page 5: Downloading API JSON
[youtube:tab] recommended page 6: Downloading API JSON
[youtube:tab] recommended page 7: Downloading API JSON
[youtube:tab] recommended page 8: Downloading API JSON
[youtube:tab] recommended page 9: Downloading API JSON
[youtube:tab] recommended page 10: Downloading API JSON
[youtube:tab] recommended page 11: Downloading API JSON
[youtube:tab] recommended page 12: Downloading API JSON
[youtube:tab] recommended page 13: Downloading API JSON
[youtube:tab] recommended page 14: Downloading API JSON
[youtube:tab] recommended page 15: Downloading API JSON
[youtube:tab] recommended page 16: Downloading API JSON
[youtube:tab] Playlist recommended: Downloading 369 items of 369
[download] Downloading item 1 of 369
[youtube] Extracting URL: https://www.youtube.com/watch?v=1PKxMZXGEbA
[youtube] 1PKxMZXGEbA: Downloading webpage
[debug] [youtube] Extracted SAPISID cookie
[youtube] 1PKxMZXGEbA: Downloading ios player API JSON
[youtube] 1PKxMZXGEbA: Downloading android player API JSON
[youtube] 1PKxMZXGEbA: Downloading m3u8 information
[debug] Sort order given by extractor: quality, res, fps, hdr:12, source, vcodec:vp9.2, channels, acodec, lang, proto
[debug] Formats sorted by: hasvid, ie_pref, quality, res, fps, hdr:12(7), source, vcodec:vp9.2(10), channels, acodec, lang, proto, size, br, asr, vext, aext, hasaud, id
[debug] Default format spec: bestvideo*+bestaudio/best
[info] Testing format 616
[hlsnative] Downloading m3u8 manifest
[hlsnative] Total fragments: 1441
[download] Destination: /tmp/tmpp_ci8p7t.tmp
[info] 1PKxMZXGEbA: Downloading 1 format(s): 616+251
https://www.youtube.com/watch?v=1PKxMZXGEbA;國民姐夫PK賽！具俊曄寵妻無極限像「演韓劇」？小Ｓ熊抱AKIRA喊話林志玲：你老公愛我！✨【#小姐不熙娣】大禮包🎁;DeeGirlsTalk
[download] Downloading item 2 of 369
[youtube] Extracting URL: https://www.youtube.com/watch?v=r0k9jlSUDWk
[youtube] r0k9jlSUDWk: Downloading webpage
[youtube] r0k9jlSUDWk: Downloading ios player API JSON
[youtube] r0k9jlSUDWk: Downloading android player API JSON
[youtube] r0k9jlSUDWk: Downloading m3u8 information
[debug] Sort order given by extractor: quality, res, fps, hdr:12, source, vcodec:vp9.2, channels, acodec, lang, proto
[debug] Formats sorted by: hasvid, ie_pref, quality, res, fps, hdr:12(7), source, vcodec:vp9.2(10), channels, acodec, lang, proto, size, br, asr, vext, aext, hasaud, id
[debug] Default format spec: bestvideo*+bestaudio/best
[info] r0k9jlSUDWk: Downloading 1 format(s): 616+251
https://www.youtube.com/watch?v=r0k9jlSUDWk;【鬼畜實驗室：FC魔界村-街機化完美移植版 一命通關測試】;魔王的直播間
[download] Downloading item 3 of 369
[youtube] Extracting URL: https://www.youtube.com/watch?v=wgzwCIiXtno
[youtube] wgzwCIiXtno: Downloading webpage
[youtube] wgzwCIiXtno: Downloading ios player API JSON
[youtube] wgzwCIiXtno: Downloading android player API JSON
[youtube] wgzwCIiXtno: Downloading m3u8 information
[debug] Sort order given by extractor: quality, res, fps, hdr:12, source, vcodec:vp9.2, channels, acodec, lang, proto
[debug] Formats sorted by: hasvid, ie_pref, quality, res, fps, hdr:12(7), source, vcodec:vp9.2(10), channels, acodec, lang, proto, size, br, asr, vext, aext, hasaud, id
[debug] Default format spec: bestvideo*+bestaudio/best
[info] wgzwCIiXtno: Downloading 1 format(s): 616+251
https://www.youtube.com/watch?v=wgzwCIiXtno;[SUB]樂天新女孩竟然下重手！林襄リンシャン跳跳床跳到快要去了！沈玉琳硬塞Dora吃香蕉？！ 威廉沈歡樂送@williamexpress【網路獨家EP162】20240419 #台湾チア;William Express
[download] Downloading item 4 of 369
[youtube] Extracting URL: https://www.youtube.com/watch?v=g4cxevqAGYs
[youtube] g4cxevqAGYs: Downloading webpage
[youtube] g4cxevqAGYs: Downloading ios player API JSON
[youtube] g4cxevqAGYs: Downloading android player API JSON
[youtube] g4cxevqAGYs: Downloading m3u8 information
[debug] Sort order given by extractor: quality, res, fps, hdr:12, source, vcodec:vp9.2, channels, acodec, lang, proto
[debug] Formats sorted by: hasvid, ie_pref, quality, res, fps, hdr:12(7), source, vcodec:vp9.2(10), channels, acodec, lang, proto, size, br, asr, vext, aext, hasaud, id
[debug] Default format spec: bestvideo*+bestaudio/best
[info] g4cxevqAGYs: Downloading 1 format(s): 313+251
https://www.youtube.com/watch?v=g4cxevqAGYs;户晨风困境：回答是死，不回答也是死｜户晨风｜习近平｜独裁者｜直播｜王局拍案20240422;王志安
[download] Downloading item 5 of 369
[youtube] Extracting URL: https://www.youtube.com/watch?v=5Iw63eqvf_g
[youtube] 5Iw63eqvf_g: Downloading webpage
[youtube] 5Iw63eqvf_g: Downloading ios player API JSON
[youtube] 5Iw63eqvf_g: Downloading android player API JSON
[youtube] 5Iw63eqvf_g: Downloading m3u8 information
[debug] Sort order given by extractor: quality, res, fps, hdr:12, source, vcodec:vp9.2, channels, acodec, lang, proto
[debug] Formats sorted by: hasvid, ie_pref, quality, res, fps, hdr:12(7), source, vcodec:vp9.2(10), channels, acodec, lang, proto, size, br, asr, vext, aext, hasaud, id
[debug] Default format spec: bestvideo*+bestaudio/best
[info] 5Iw63eqvf_g: Downloading 1 format(s): 247+251
https://www.youtube.com/watch?v=5Iw63eqvf_g;【再給金歹笑諧星一次機會！最不好笑藝人排行！下】20150804 綜藝大熱門【完整版】;綜藝大熱門 Hot Door Night
[download] Downloading item 6 of 369
[youtube] Extracting URL: https://www.youtube.com/watch?v=XZ_mjF5un_c
[youtube] XZ_mjF5un_c: Downloading webpage
[youtube] XZ_mjF5un_c: Downloading ios player API JSON
[youtube] XZ_mjF5un_c: Downloading android player API JSON
[youtube] XZ_mjF5un_c: Downloading m3u8 information
[debug] Sort order given by extractor: quality, res, fps, hdr:12, source, vcodec:vp9.2, channels, acodec, lang, proto
[debug] Formats sorted by: hasvid, ie_pref, quality, res, fps, hdr:12(7), source, vcodec:vp9.2(10), channels, acodec, lang, proto, size, br, asr, vext, aext, hasaud, id
[debug] Default format spec: bestvideo*+bestaudio/best
[info] XZ_mjF5un_c: Downloading 1 format(s): 248+251
https://www.youtube.com/watch?v=XZ_mjF5un_c;為什麼有兩個蒙古;歷史衛視 History Channel
[download] Downloading item 7 of 369
[youtube] Extracting URL: https://www.youtube.com/watch?v=7-2ytrnKO7s
[youtube] 7-2ytrnKO7s: Downloading webpage
[youtube] 7-2ytrnKO7s: Downloading ios player API JSON
[youtube] 7-2ytrnKO7s: Downloading android player API JSON
[youtube] 7-2ytrnKO7s: Downloading m3u8 information
[debug] Sort order given by extractor: quality, res, fps, hdr:12, source, vcodec:vp9.2, channels, acodec, lang, proto
[debug] Formats sorted by: hasvid, ie_pref, quality, res, fps, hdr:12(7), source, vcodec:vp9.2(10), channels, acodec, lang, proto, size, br, asr, vext, aext, hasaud, id
[debug] Default format spec: bestvideo*+bestaudio/best
[info] Testing format 616
[hlsnative] Downloading m3u8 manifest
[hlsnative] Total fragments: 182
[download] Destination: /tmp/tmpkixg9298.tmp
[info] 7-2ytrnKO7s: Downloading 1 format(s): 616+251
https://www.youtube.com/watch?v=7-2ytrnKO7s;【聖鬥士懶人包—不死鳥一輝】一輝穿過的聖衣有9件，唯獨沒有變過黃金色/武器除了羽毛飛鏢外還有弒神短劍;不負責任瘋動漫2館
[download] Downloading item 8 of 369
[youtube] Extracting URL: https://www.youtube.com/watch?v=wEdu67mbthk
[youtube] wEdu67mbthk: Downloading webpage
^C
ERROR: Interrupted by user
^C

--lazy-playlist --flat-playlist

--lazy-playlist --flat-playlist

THX!!!

--lazy-playlist --flat-playlist

thanks, it works great. though how to output the "section title" of each recommendation block in the homepage? to my understanding they are not categories, yet not tags as well.


FT Asynchronous should run on platform thread by default

Environment Details

Helidon Version: 4.0.6
Helidon SE or Helidon MP: Both

Problem Description
Fault tolerance asynchronous should always run on physical thread by default.
Currently it is not possible to configure MP for platform thread usage.
Cases for offloading with @Asynchronous:

Blocking code - offloading VT->VT is not needed
Obstructing/long-running code - offloading VT->VT is not practical as carrier thread from the same pool is used
Pinning code - offloading VT->VT doesn't solve the issue

More complicated cases for VT's may exist, configurable option should exist(it doesn't in 4.0.6 MP).
Steps to reproduce
    @GET
    @Path("/platform")
    @Asynchronous
    public CompletionStage<ThreadInfo> getOnPlatformThread() {
        var name = Thread.currentThread().getName();
        var virtual = Thread.currentThread().isVirtual();
        ThreadInfo threadInfo = new ThreadInfo(name, virtual);
        return CompletableFuture.completedFuture(threadInfo);
    }

➜  offloading-mp curl -s localhost:8080/thread-name/platform | jq
{
  "name": "helidon-ft-1: async",
  "virtual": true
}

I agree there should be an option for this, but I'm not sure a platform thread should be default; I see people using asynchronous in 4.x without a good understanding of what they are doing, and suddenly creating a bunch of new platform threads may not be ideal. I will take a look.

@danielkec One possibility would be to introduce a new annotation to control the executor service to use. Something along these lines:
    @Asynchronous
    @InvokedUsing(SingleThreadExecutorSupplier.class)
    CompletableFuture<String> asyncWithExecutor() {
        called.set(!Thread.currentThread().isVirtual());
        FaultToleranceTest.printStatus("AsynchronousBean::asyncWithExecutor", "success");
        return CompletableFuture.completedFuture("success");
    }

    static class SingleThreadExecutorSupplier implements Supplier<ExecutorService> {

        @Override
        public ExecutorService get() {
            return Executors.newSingleThreadExecutor();
        }
    }

@spericas  I like the PR! There is one other thing we should consider. MP FT is designed for runtimes which are sensitive to blocking - all async methods needs to return CompletionStage/@Future. This is no longer needed in Helidon as blocking is no problem any more, I think we should support @Asynchronous also on methods which don't return future.
I thing we should also contest the spec:
The method annotated with @Asynchronous must return a Future or a CompletionStage from the java.util.concurrent package. Otherwise, a FaultToleranceDefinitionException occurs.

@danielkec Maybe we can wait until the spec catches up, otherwise we'll very likely fail some TCKs. Besides that, I'm not sure I agree with you, if you don't mind blocking the calling thread at the time you call the method, then do you really want to use async instead of the same thread? Returning a future gives you the ability to decide when to (potentially) block the calling thread by waiting on the that future at a specific time.

After discussing the use case further with @danielkec, we agreed to proceed with a two-pronged approach: (i) an extension for the FT module to support using a different executor service (this issue!) and (ii) a new, simpler annotation in another module to only handle running arbitrary methods in platform threads. There will be a new issue created for (ii).

Replaced by #8638


Unify donation  loading state

Checklist:

[ ] I have read freeCodeCamp's contribution guidelines.
[ ] My pull request has a descriptive title (not a vague title like Update index.md)
[ ] My pull request targets the main branch of freeCodeCamp.
[ ] I have tested these changes either locally on my machine, or GitPod.

Closes #XXXXX

the main question is, which one of the following scenarios offer a higher perceived performance  and a better user experience:
1)load each button individually (paypal loads faster and some times google/apple pay is not supported)
2)show a spinner until both load (or until one loads and we know the other is not supported)

higher perceived performance

What does Lighthouse say about the content/interactivity?

a better user experience

Personally, one where all buttons appear at same time (2), because as I am reading the description of why/what I am about to do (donate), I do not want my eyes distracted 2-3 times as the buttons appear.

@ShaunSHamilton it is hard to say because it would depend on wether the browser that visits our website has google pay enabled or not.
Since the loader is going to be on the modal as well were there is not much read, it might effect the perceived performance. It is hard to tell if it matter much donation wise, but I think it is a good idea to unify the states at this point and optimize it with A/B testing in upcoming prs.

Since the loader is going to be on the modal

Is there a specific reason we do not just link to /donate on the modal? (no pay buttons)

Since the loader is going to be on the modal

Is there a specific reason we do not just link to /donate on the modal? (no pay buttons)

The longer the user journey the less likely it will be completed. Although I would love to A/B test that to see if the users are affected. That way we can clean up a lot of the components.


Strange behavior of the subsize and start keywords in array datatype

In GitLab by @agueroud on Nov 9, 2021, 16:06
I define my array this way in the yml file and I pass it to deisa plugin (python based plugin).
data:
    local_t:
      type: array
      subtype: double
      size: ['$dsize0', '$dsize1']
      subsize: ['$dsize0 - 2', '$dsize1 - 2']
      start: [1, 1]

When the start : [0,0] is set, the data values are correct. However I noticed strange values in the arrays I share when it is set to start: [1, 1]. It seems that the new pointer to the data is not well set.
I had a quick discussion with Karol, who suggested to try to serialize my data before passing it to deisa. It seems to solve my issue :)
Karol thinks that there is maybe a bug in the to_python function.

In GitLab by @ksiero on Nov 30, 2021, 12:20
mentioned in commit fca2719048508f82b274256376651f07e93412c6

In GitLab by @ksiero on Nov 30, 2021, 12:22
mentioned in commit f2fcc053ab09d44dbf4a4151a08c6cc46bd8b6cf

In GitLab by @ksiero on Nov 30, 2021, 12:18
mentioned in commit 928739ccb7f6f828eae99fd2df370eaf5affa8c2

In GitLab by @ksiero on Nov 30, 2021, 12:43
mentioned in commit 730085e76630a3b3d55f591754efe315e14a36ae