Split TE backup NHG multiple NH into DCGate and POPGate tests

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 53.056%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
1736

Relevant Lines:
3272

💛  - Coveralls

Pull Request Functional Test Report for #2711 / 56e533e735ad73eb9c169916205e6998219b3a0f
Virtual Devices

Device
Test
Test Documentation
Job
Raw Log

Arista cEOS

TE-11.21: Backup NHG: Multiple NH with PBFTE-11.2: Backup NHG: Multiple NH
dcf31490
Log

Cisco 8000E

TE-11.21: Backup NHG: Multiple NH with PBFTE-11.2: Backup NHG: Multiple NH
b8b42dd0
Log

Cisco XRd

TE-11.21: Backup NHG: Multiple NH with PBFTE-11.2: Backup NHG: Multiple NH
6dc7e247
Log

Juniper ncPTX

TE-11.21: Backup NHG: Multiple NH with PBFTE-11.2: Backup NHG: Multiple NH
020111ab
Log

Nokia SR Linux

TE-11.21: Backup NHG: Multiple NH with PBFTE-11.2: Backup NHG: Multiple NH
fc52361c
Log

Openconfig Lemming

TE-11.21: Backup NHG: Multiple NH with PBFTE-11.2: Backup NHG: Multiple NH
8fbc1bf3
Log

Hardware Devices

Device
Test
Test Documentation
Raw Log

Arista 7808

TE-11.21: Backup NHG: Multiple NH with PBFTE-11.2: Backup NHG: Multiple NH

Cisco 8808

TE-11.21: Backup NHG: Multiple NH with PBFTE-11.2: Backup NHG: Multiple NH

Juniper PTX10008

TE-11.21: Backup NHG: Multiple NH with PBFTE-11.2: Backup NHG: Multiple NH

Nokia 7250 IXR-10e

TE-11.21: Backup NHG: Multiple NH with PBFTE-11.2: Backup NHG: Multiple NH

Help


Thumbor version forked and fixed to work on the latest lambda enviroment

Since AWS have updated the lambda execution environment which  broke old deployments (versions <=3.0) of this solution, and  newer versions of this solution doesn't correctly implement the Thumbor interface (they shifted to sharpJS). I decided to Fork version 3.1 (Thumbor version)  and make the required changes for it to work on lambda again
https://github.com/jodevsa/serverless-image-handler
Published solution template  is available in the README

Thank you!

Is this working for other 3rd parties? I tried updating our template to the published one here and CFN complained with:
2019-07-24 12:30:51 UTC-0400	ServerlessImageHandler	UPDATE_ROLLBACK_IN_PROGRESS	The following resource(s) failed to update: [CustomResource].
2019-07-24 12:30:50 UTC-0400	CustomResource	UPDATE_FAILED	Error occurred while GetObject. S3 Error Code: NoSuchBucket. S3 Error Message: The specified bucket does not exist (Service: AWSLambdaInternal; Status Code: 400; Error Code: InvalidParameterValueException; Request ID: REDACTED)

Same for me.  Really need something to work asap.  Multiple production
sites are failing right now.
Jason O Brown
On Wed, Jul 24, 2019 at 11:39 AM Sean Coates<EMAIL_ADDRESS>wrote:

Is this working for other 3rd parties? I tried updating our template to
the published one here and CFN complained with:
2019-07-24 12:30:51 UTC-0400	ServerlessImageHandler	UPDATE_ROLLBACK_IN_PROGRESS	The following resource(s) failed to update: [CustomResource].
2019-07-24 12:30:50 UTC-0400	CustomResource	UPDATE_FAILED	Error occurred while GetObject. S3 Error Code: NoSuchBucket. S3 Error Message: The specified bucket does not exist (Service: AWSLambdaInternal; Status Code: 400; Error Code: InvalidParameterValueException; Request ID: REDACTED)
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/awslabs/serverless-image-handler/issues/127?email_source=notifications&email_token=ACCTAVUAFZ5W6QU4GVC2HP3QBCAUTA5CNFSM4IGQ6ATKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2W5HBA#issuecomment-514708356,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACCTAVQWVLBUSWSZ7UVDSSDQBCAUTANCNFSM4IGQ6ATA
.

@jasonobrown  which region are you deploying to ?

@jodevsa us-east-2

I'm us-east-1, FWIW.

Will it work if I deploy it in us-east-1 but my bucket is in us-east-2?

@jasonobrown @scoates  it should work now for us-east-1, us-east-2, eu-west-1

Thank you. You saved the rest of the week.

It still failed for me in us-east-2

Wrong bucket permission.., should work now :+1:

Negative, failed again.  I really appreciate the help by the way.

To be clear for others who might be following: this is working for me, now, on an updated stack (previously updated in February!), deployed in us-east-1 with a bucket in us-east-1.

@jasonobrown @scoates it should work now for us-east-1, us-east-2 and eu-west-1

I have a stack in us-west-1…any way you could make it deployable there? 👼
I'd also like to nominate @jodevsa as person of the year!

@jodevsa I'm seeing the same thing @jasonobrown is – failing in us-east-2 (and us-west-1). I've confirmed its working in us-east-1, though.

@jodevsa  @timkelty It's failing for me on us-east-1 as well.

Hi guys
The problem is that lambda requires the files to be in the same region which requires me to create multiple buckets for each region. there might be an easier way, not sure
@jasonobrown for some reason I'm unable to create a bucket in us-east-2 ( might be due to the fact i just deleted a bucket with that name in different region. I'll try again later
@timkelty  use-west-1 should work now
working regions:
eu-west-1, us-east-1 and us-west-1
I'll continue on supporting all regions when i come back

@jodevsa you are a saint. Confirmed that us-west-1 is now working.

Can anyone tell me why I can't launch in us-east-1?  Am I supposed to be changing the template url to say us-east-1 or leaving it as eu-west-1?  I've tried both, but obviously something is wrong.

@jasonobrown Should be able to just leave the template url as eu-west-1 (that's what I did).

@timkelty I've tried that.  The bucket should be able to be in a different region right?

@jasonobrown I am having the same issue as you. can not use us-east-1 as an origin. Seeing the rollback error.

I just tried using a bucket in the same region and I'm still getting the error.  I'm not setting an iam policy, but I didn't have one on the previous stack either (it's supposed to default to my user).  Does that line up with what everyone else is doing (those who it has worked for)?

I was able to deploy this to us-east-1 about an hour ago, but I can't anymore using the same steps. Not sure what's changed.

Trying to deploy to us-east-1, I get this error.
Error occurred while GetObject. S3 Error Code: AuthorizationHeaderMalformed. S3 Error Message: The authorization header is malformed; the region 'us-east-1' is wrong; expecting 'us-east-2' (Service: AWSLambdaInternal; Status Code: 400; Error Code: InvalidParameterValueException; Request ID: e935fa07-2bc6-4efc-92b0-1128c970075b)

@jasonobrown what has been working for me is:

go to my CloudFormation Stack
click Update
"replace template"
paste in https://cf-templates-nestrom.s3-eu-west-1.amazonaws.com/serverless-image-handler/1.0/serverless-image-handler.template
so far has worked for me in us-east-1 and us-west-1

@dlosie @jasonobrown  Not sure whats changed us-east-1 used to work

@jasonobrown  us-east-2 should work now

I'm with @jasonobrown - still unable to deploy to us-east-1. I'm getting the same error he mentioned above. It happening when attempting to Create the CustomResource.
@jodevsa When this is all over and I've got it working, I want to know how I can thank you... seriously.

Seeing the same as @jasonobrown and @dyarbrough…although us-east-2 now does work for me, but us-east-1 (which worked earlier) doesn't.

It finally worked for me in us-east-2, but only 1 image worked.  All the other images say key does not exist.

Im trying to update the template but is failing.
Im getting this error:
Error occurred while GetObject. S3 Error Code: AuthorizationHeaderMalformed. S3 Error Message: The authorization header is malformed; the region 'us-east-1' is wrong; expecting 'us-east-2' (Service: AWSLambdaInternal; Status Code: 400; Error Code: InvalidParameterValueException; Request ID: XXXXXX)

@jodevsa I hate to be a pain, especially since your doing this with no obligation, but are you looking into the us-east-1 deployment issue? I only ask because I can potentially deploy to us-east-2 but I'd need to start moving other assets over to match which I'd prefer to avoid.

Not sure if makes sense or helps, but us-east-1 seemed to stop working around the same time us-east-2 started working.

@dyarbrough @timkelty  not sure what really caused it, but i think i have it fixed now

Now is workin on us-east-1!  We all owe you! 🍻

@jodevsa Confirmed that this is working for me in us-east-1. Where can we send money and gifts? Seriously.

@jodevsa  Thank you!!

All of my images in subfolders are not working.   Anyone having the same issue?

@AkivaLandsberg subfolder is working for me: https://dhb07jds0fi8u.cloudfront.net/PjxbmepvudoiJ-P2hkIK6SwY7TA=/380x190/filters:focal(545x412:547x414):quality(82):sharpen(0.5,0.5,true)/development.oak.website%2Ffpo%2Ftest%2FScreen-Shot-2019-07-24-at-5.14.43-PM.png

My images are stored in a subfolder named images
I see the following error in lambda
[ERROR]	2019-07-24T21:20:30.160Z	0f1a8ba7-dbcd-4b9c-af02-0763cde57294	500 GET /unsafe/images/69575688-5ca8-447e-92f9-21f7ccd82772.jpg (<IP_ADDRESS>) 59.61ms

Any clue why /unsafe is being appended?

@timkelty  Thank you!  I now see the issue is not subfolders, rather its an issue with my image names. I have been using uuids as the image name. For some reason it now giving issues even though it worked before.
If anyone has ideas why this is happening, I would really appreciate it.

@AkivaLandsberg maybe check this https://stackoverflow.com/questions/57178748/aws-serverless-image-handler-v3-x-broken-by-changes-to-aws-lamdba-execution-envi

Can somebody provide instructions on how to update the forked version? I wasn't the one who originally installed the image resizer. Would really appreciate if someone can share the steps please

@devmeyster try this? https://github.com/awslabs/serverless-image-handler/issues/127#issuecomment-514757029

@jodevsa @timkelty You rock

@jodevsa Thank you!! You saved a lot of time! <3

Thanks mate! You are a star! @jodevsa

I'm having issues with some of my s3 files that have a Content-Encoding: undefined key set on them. This bad data was previously ignored in v2 of this lib, but causes libcurl to crash in this version.
Trace:
[ERROR] 2019-07-25T10:21:58.10Z b458c57d-a702-4f1b-a3d1-2ccdc215dc03 ERROR: Traceback (most recent call last):
File "/var/task/thumbor/handlers/__init__.py", line 153, in get_image
self.context.request.image_url
File "/var/task/tornado/gen.py", line 1099, in run
value = future.result()
File "/var/task/tornado/concurrent.py", line 260, in result
raise_exc_info(self._exc_info)
File "/var/task/tornado/gen.py", line 1107, in run
yielded = self.gen.throw(*exc_info)
File "/var/task/thumbor/handlers/__init__.py", line 588, in _fetch
loader_result = yield self.context.modules.loader.load(self.context, url)
File "/var/task/tornado/gen.py", line 1099, in run
value = future.result()
File "/var/task/tornado/concurrent.py", line 260, in result
raise_exc_info(self._exc_info)
File "/var/task/tornado/stack_context.py", line 315, in wrapped
ret = fn(*args, **kwargs)
File "/var/task/tornado/concurrent.py", line 531, in run_callback
result = future.result()
File "/var/task/tornado/concurrent.py", line 260, in result
raise_exc_info(self._exc_info)
File "/var/task/tornado/stack_context.py", line 315, in wrapped
ret = fn(*args, **kwargs)
File "/var/task/tornado_botocore/base.py", line 128, in _process_response
parsed = parser.parse(response_dict, operation_model.output_shape)
File "/var/task/botocore/parsers.py", line 207, in parse
if self._is_generic_error_response(response):
File "/var/task/botocore/parsers.py", line 236, in _is_generic_error_response
body = response['body'].strip()
AttributeError: 'NoneType' object has no attribute 'strip'```

Debug:


10:21:58
[DEBUG] 2019-07-25T10:21:58.6Z b458c57d-a702-4f1b-a3d1-2ccdc215dc03 < Content-Encoding: undefined

10:21:58
[DEBUG] 2019-07-25T10:21:58.7Z b458c57d-a702-4f1b-a3d1-2ccdc215dc03 < x-amz-version-id: rp8NOfNtAcWTSa.p617M2yISlJOp3OR.

10:21:58
[DEBUG] 2019-07-25T10:21:58.7Z b458c57d-a702-4f1b-a3d1-2ccdc215dc03 < Accept-Ranges: bytes

10:21:58
[DEBUG] 2019-07-25T10:21:58.7Z b458c57d-a702-4f1b-a3d1-2ccdc215dc03 < Content-Type: image/jpeg

10:21:58
[DEBUG] 2019-07-25T10:21:58.7Z b458c57d-a702-4f1b-a3d1-2ccdc215dc03 < Content-Length: 116548

10:21:58
[DEBUG] 2019-07-25T10:21:58.7Z b458c57d-a702-4f1b-a3d1-2ccdc215dc03 < Server: AmazonS3

10:21:58
[DEBUG] 2019-07-25T10:21:58.7Z b458c57d-a702-4f1b-a3d1-2ccdc215dc03 <

10:21:58
[DEBUG] 2019-07-25T10:21:58.7Z b458c57d-a702-4f1b-a3d1-2ccdc215dc03 Unrecognized content encoding type. libcurl understands deflate, gzip content encodings.
[DEBUG] 2019-07-25T10:21:58.7Z b458c57d-a702-4f1b-a3d1-2ccdc215dc03 Unrecognized content encoding type. libcurl understands deflate, gzip content encodings.

@kjanoudi I got the sameAttributeError: 'NoneType' object has no attribute 'strip' with Content-Encoding: base64.
How do you get DEBUG logs?

I am able to update the stack following #127.  but got "Access Denied" when test in urls. any ideas? thanks a lot!

@davidweica, after you run the update find the "ServerlessImageHandler-ImageHandlerFunction-xyz" Lambda function that it manages and make sure the "TC_AWS_*" values are set correctly.  In my case, after running the template update they got reset to default values.
@jodevsa, thank you for this fork!  You helped save the day for one of the sites we use this on last night.

@kjanoudi I got the sameAttributeError: 'NoneType' object has no attribute 'strip' with Content-Encoding: base64.
How do you get DEBUG logs?

@cxrod Set the log level to DEBUG In the lambda function

@jodevsa @squinn thanks folks. you guys are my hero!

I'm getting a 403 when trying to access the serverless-image-handler.zip in the bucket specified - have permissions changed?

I'm trying to update the template, but get the error on DeployingUI step Failed to update resource. Failed to delete bucket-name/serverless-image-handler-ui/, does anybody experienced such problem?
looks like some S3 permission problem, but I haven't found workable solution

@vitalinfo do you need the UI? Can you set DelpoyUI to no?

@devmeyster Did you get it figured out?

@idioteque1025 definitely make sense to get rid it, thanks

I am late to the party but thank you @jodevsa!

How can i get this to work for eu-west-2?

@jodevsa Is there any chance you could set this up to work in us-west-2 or provide the steps so users can setup the function in a region of their choice? I've tried editing the config values to point to my bucket with the scripts but am still getting a bucket not found error by cloud formation.
I appreciate your work on this. Thanks!

@oakesjosh yes, we are really struggling with this with the eu-west-2 location, similar to your issue. Not had any success yet and been trying different things for the last couple of hours

Also needing to get this to work on eu-west-2
Thanks for your work on this @jodevsa

@all
Thanks everyone  for your kind messages. Glad it worked :+1:
If anyone still having a problem, you can open an issue on  https://github.com/jodevsa/serverless-image-handler
@JayBizzle , @GC-Mark  @oakesjosh  fixed eu-west-2

@JayBizzle  @GC-Mark @oakesjosh eu-west-2 us-west-2 are working

@jodevsa thanks for your solution. Is it possible to get it to work is ap-southeast-2 please?
Cheers!

Can conform this is now working is the eu-west-2 region.
Great work @jodevsa very much appreciated!! 👍

Really well done mate, you are a lifesaver <3

@jodevsa First of all, thanks for the super effort!
I just tried to use it for ap-south-1 and the CloudFormation complained that:
"Error occurred while GetObject. S3 Error Code: NoSuchBucket. S3 Error Message: The specified bucket does not exist (Service: AWSLambdaInternal; Status Code: 400; Error Code: InvalidParameterValueException; Request ID: f7241fad-385f-45e7-b19a-28fb0a7ca491)"
In effect, this is the same issue that @cxrod faced. I tried to debug this a bit and realzied that https://nestrom-production-image-handler-files.s3-eu-west-1.amazonaws.com/serverless-image-handler/1.0/serverless-image-handler.zip is not present:
<Error>
NoSuchKey
<Message>The specified key does not exist.</Message>
<Key>
serverless-image-handler/1.0/serverless-image-handler.zip
</Key>
<RequestId>D24A835966D8348E</RequestId>
<HostId>
NahfqLlKpS20vNfUOZSsHAgiDsjs2UzlkenhHGfNUJzpvpIRhJ6XWq4eNF5W4YVD2+QTsfZ55z0=
</HostId>
</Error>

Any pointers about anything that I can do to fix it.

@jasonobrown what has been working for me is:

go to my CloudFormation Stack
click Update
"replace template"
paste in https://cf-templates-nestrom.s3-eu-west-1.amazonaws.com/serverless-image-handler/1.0/serverless-image-handler.template

so far has worked for me in us-east-1 and us-west-1

@timkelty I'm working on ap-southeast-1, and this solution can not work. Can you please to give advice for that. Thank you!

I am in the same situation... region is ap-southeast-2. Thanks!

@jodevsa Could you help on other regions please. Thanks!

I ran into the issue of my region not being supported the other day. I managed to figure out how to get it setup in my own S3 account and have the template pull from there.

Build the repository or download it already built - Download link (unzip before putting in S3)
Create an S3 bucket with your deployment region appended to then name. So if you're deploying in us-west-2 your bucket name would be my-bucket-name-us-west-2
Upload the 3 zip files into your S3 bucket.
Edit the locations in the .template that reference the bucket/location of the zip files. These references are on lines 135 & 136, 146 & 147, 160 & 161
Now upload this edited template into cloud formation and your deployment should work

@jasonobrown what has been working for me is:

go to my CloudFormation Stack
click Update
"replace template"
paste in https://cf-templates-nestrom.s3-eu-west-1.amazonaws.com/serverless-image-handler/1.0/serverless-image-handler.template

so far has worked for me in us-east-1 and us-west-1

I had changed the original template file, and when I check the old template and this new template I do not see much difference. Could you please explain which part is changed in the template so that I can change it manually?
Thanks.

@oakesjosh Hi, thanks for your sulution. I have tried to upate 135 & 136, 146 & 147, 160 & 161.  put the 3 zip files under the root of the bucket directly. It says
Error occurred while GetObject. S3 Error Code: NoSuchBucket. S3 Error Message: The specified bucket does not exist (Service: AWSLambdaInternal; Status Code: 400; Error Code: InvalidParameterValueException; Request ID: blabla)
Is there other things that I need to change?

@oakesjosh I tried with your solution but I have the same issue like @chuyuou. So sad!

@oakesjosh Hi, thanks for your solution. I have tried to update 135 & 136, 146 & 147, 160 & 161. put the 3 zip files under the root of the bucket directly. It says
Error occurred while GetObject. S3 Error Code: NoSuchBucket. S3 Error Message: The specified bucket does not exist (Service: AWSLambdaInternal; Status Code: 400; Error Code: InvalidParameterValueException; Request ID: blabla)
Is there other things that I need to change?

I am having the same problem from eu-north-1 region.

@oakesjosh  my region is ap-southeast-2, The only difference between my template and the original template is the bucket name.

@oakesjosh Hi, thanks for your solution. I have tried to update 135 & 136, 146 & 147, 160 & 161. put the 3 zip files under the root of the bucket directly. It says
Error occurred while GetObject. S3 Error Code: NoSuchBucket. S3 Error Message: The specified bucket does not exist (Service: AWSLambdaInternal; Status Code: 400; Error Code: InvalidParameterValueException; Request ID: blabla)
Is there other things that I need to change?

oakesjosh I am having the same problem. Did you change something else besides these?

Just wanted to add that I was able to make this work by:
deploying the clouformation stack to us-east-1,
while my bucket is located in ap-southeast-2
I got similar errors like the above 'NoSuchBucket'...
The critical bit was the first 2 Parameters of the temmplate:

Origin S3 Bucket => 'my-bucket-name'
Origin S3 Bucket Region => 'ap-southeast-2'

I used the supplied template as is, including the following bucket and key for the image handler code:
S3Bucket: nestrom-production-image-handler-files
S3Key: serverless-image-handler/1.0/serverless-image-handler.zip

Next I plan to point it to my own bucket, so that I am not dependent on this 3rd party resource that I don't have control over.

@oakesjosh Hi, thanks for your solution. I have tried to update 135 & 136, 146 & 147, 160 & 161. put the 3 zip files under the root of the bucket directly. It says
Error occurred while GetObject. S3 Error Code: NoSuchBucket. S3 Error Message: The specified bucket does not exist (Service: AWSLambdaInternal; Status Code: 400; Error Code: InvalidParameterValueException; Request ID: blabla)
Is there other things that I need to change?

please see my latest comment, if you have not done so already...

Not working for eu-west-3.
Please advice.
Thank you!

Thanks @timkelty, worked for me on us-west-2

Worked out..Thanks @oakesjosh ! This can work!! Saved my tomorrow!!

Worked out..Thanks @oakesjosh ! This can work!! Saved my tomorrow!!

@chuyuou Did you do something else rather than changing the bucket names?

for example, {bucket-name}-ap-southeast-2, then in the template the Origin S3 Bucket should be :{bucket-name} , without region..Follow @oakesjosh instruction and use those three zip files.

@chuyuou You save my life!
Thank you all! @jodevsa @oakesjosh @timkelty

I'm getting this error
Custom Resource failed to stabilize in expected time any thoughts?  followed the above steps.

Thanks a lot for the fix @jodevsa!
By the way, if anyone else here is struggling with errors like invalid elf header, for me building through the docker included in the PR didnt work, I had to start an EC2 running AWS linux and build it there.

Hello all - This version of the solution has been patched and is now available at:
(https://solutions-reference.s3.amazonaws.com/serverless-image-handler/v3.1.1/serverless-image-handler.template)
This patch will fix issues caused by the Lambda Execution Environment updates that have been rolling out. More details on that update available here:(https://aws.amazon.com/blogs/compute/upcoming-updates-to-the-aws-lambda-execution-environment/)

@hayesry Thank you! Solved the problem on us-west-2.

In case anyone else is managing their own template based on the original template: the only fix I needed in my template was to switch all instances of "3.1.0" to "3.1.1". This will target the correct solution image handler zip.

@hayesry Can you share the updated build process? The new solution does not work for us since we have to support animated GIFs :(

@hayesry Thank you for linking that! I used that template with the steps in https://github.com/awslabs/serverless-image-handler/issues/127#issuecomment-514757029 and it solved the issue for me. VERY MUCH appreciate it!

This entire section https://github.com/awslabs/serverless-image-handler#building-distributable-for-customization should be removed until any sort of semblance of a deterministic build can be achieved.
You can brick you entire deployment by simply changing a Tag value and end up replacing your imagehandler function with an architecture that is unsupported. It will allow you do that without a peep from the build process.
My current strategy is deploy the solution, and drift the stack while documenting the changes. To upgrade, I redeploy the solution, reapply my drift, then adjust my dns aliases. I'd be thrilled with just an AMI of environment aws solutions team uses to build https://aws.amazon.com/solutions/serverless-image-handler/
Also thinking of ways to completely decouple the lambda function from this stack completely, way to much risk exposure building image libraries.

We are closing this issue, but please feel free to open the issue again if you any other support.

Has anyone here successfully mitigated the risk posed by AWS no longer supporting botocore.requests through the cfn-response module?
I'm a bit confused as to what actually needs to be done.
This is the email I have from them in January 2021:


Fix 7.x documentation

This PR fixes 7.x documentation in preparation for GA. Once this is merged, https://github.com/elastic/docs/pull/876 can be merged to bring 7.x documentation onto elastic.co site.

https://github.com/elastic/docs/pull/876 is now merged.


Can not delete the tasks at one time

I have to download many large files , but i do not want to continue these tasks after a few minutes later. How can I delete these task at one time ? they are more than one hundred tasks!

First, open the Advanced settings in the Preferences, there is a download.session address, open it in the folder, write down the download path of the task inside, I think you a bunch of tasks should be downloaded to the same directory, you can batch delete them, including *.aria2.
Finally, empty or delete the download.session.

First, open the Advanced settings in the Preferences, there is a download.session address, open it in the folder, write down the download path of the task inside, I think you a bunch of tasks should be downloaded to the same directory, you can batch delete them, including *.aria2.
Finally, empty or delete the download.session.

Thanks a lot


[Feature Request] Top Games Section, Games without covers should display infos by default

In the Top Games Section, you usually select a game by its cover,
Games without a cover use the default "purple twitch logo" picture:
https://static-cdn.jtvnw.net/ttv-static/404_boxart-272x380.jpg
You have to mouseover to actualy discover what game it is,
displaying the infos by default for those games would  be useful,

I'm afraid this is not possible.
It would work if the 404 image link was included in the box attribute of the Game model, but unfortunately those 404 image urls are the result of a http redirect of the CDN webserver when the real images are unavailable.
See the RCT2 record for instance:
{
  "name": "RollerCoaster Tycoon 2",
  "popularity": 1404,
  "_id": 10260,
  "giantbomb_id": 11178,
  "box": {
    "large": "https://static-cdn.jtvnw.net/ttv-boxart/RollerCoaster%20Tycoon%202-272x380.jpg",
    "medium": "https://static-cdn.jtvnw.net/ttv-boxart/RollerCoaster%20Tycoon%202-136x190.jpg",
    "small": "https://static-cdn.jtvnw.net/ttv-boxart/RollerCoaster%20Tycoon%202-52x72.jpg",
    "template": "https://static-cdn.jtvnw.net/ttv-boxart/RollerCoaster%20Tycoon%202-{width}x{height}.jpg"
  },
  "logo": {
    "large": "https://static-cdn.jtvnw.net/ttv-logoart/RollerCoaster%20Tycoon%202-240x144.jpg",
    "medium": "https://static-cdn.jtvnw.net/ttv-logoart/RollerCoaster%20Tycoon%202-120x72.jpg",
    "small": "https://static-cdn.jtvnw.net/ttv-logoart/RollerCoaster%20Tycoon%202-60x36.jpg",
    "template": "https://static-cdn.jtvnw.net/ttv-logoart/RollerCoaster%20Tycoon%202-{width}x{height}.jpg"
  },
  "_links": {}
}

There's no way of implementing such a feature without sending another HEAD/GET http request for each of those images just to check whether the server returns a 3xx status code. This would be terrible solution performance-wise.

In theory, couldn't you look at the image pixels after its loaded with canvas?

In theory, yes, in practice, no...
Also, once this image gets replaced by a different one, this (crazy overkill) approach would fail

Ok, so instead of comparing the images directly, clearly it would be more efficient to just compare file sizes (that having been said canvas is actually very fast, and I do think it would work just fine with that as well).
As for what image to compare it to, you can just load https://static-cdn.jtvnw.net/ttv-static/404_boxart-136x190.jpg and use that for the comparison.
I could write this is you're interested.

Thanks, but it's really not necessary. I appreciate your efforts, though.
Using a stable API endpoint is what ensures the application to work flawlessly with a promise of being some kind of future proof (until certain things get deprecated). Implementing features based on URLs from a CDN on the other hand are not future proof and can break any day. Using just file sizes as a comparison method also doesn't sound like a good implementation. Hashing would be, but with a bigger performance impact.
I honestly don't think that using techniques like canvas can be efficient enough when they are used for many images at the same time. This would increase the rendering time of each PreviewImageComponent and GameItemComponent by too much. Also, using canvas elements instead of simple images could be quite a big hit for Chromium on the drawing speed of the DOM.
Non existent game preview images are such an edge case that I believe it's not worth the effort of programming a way around this. It's Twitch's job to add an image, since the 404 images are displayed on their website, too.

Fair enough, up to you of course :)
With regards to canvas performance, I once made a web-app that loaded images of Dota2 items from the steam market place (size around 300*200 pixels if I recall correctly), cropped them, made them black and white, and applied an alpha mask (this was all basically a single arithmetic operation per pixel). There was no noticeable slowdown even with hundreds of images.

There could be a solution for this once the chrome extensions API becomes available:
https://developer.chrome.com/extensions/devtools_network
Requires #275 to be merged


new options for xrdp.ini disableSSlv3=yes and tls_ciphers=HIGH

New options disableSSlv3 and tls_ciphers for xrdp.ini.
SSlv3 is weak and tls_ciphers option will allow ciphers to be set to high therefore disabling weak ciphers such as RC4 .
SSlv3 and RC4 will show on security scans. These are optional settings in case you have old systems which require these.

+1
I also think SSLv3 should be disabled by default (in xrdp.ini)
בתאריך יום ו׳, 19 באוג׳ 2016 ב-12:15 מאת alexillsley <
<EMAIL_ADDRESS>
New options disableSSlv3 and tls_ciphers for xrdp.ini.
SSlv3 is weak and tls_ciphers option will allow ciphers to be set to high
therefore disabling weak ciphers such as RC4 .
SSlv3 and RC4 will show on security scans. These are optional settings in
case you have old systems which require these.
You can view, comment on, or merge this pull request online at:
https://github.com/neutrinolabs/xrdp/pull/413
Commit Summary

new options for xrdp.ini disableSSlv3=yes and tls_ciphers=HIGH and
code to implement

File Changes

M common/ssl_calls.c
https://github.com/neutrinolabs/xrdp/pull/413/files#diff-0 (22)
M common/ssl_calls.h
https://github.com/neutrinolabs/xrdp/pull/413/files#diff-1 (2)
M common/trans.c
https://github.com/neutrinolabs/xrdp/pull/413/files#diff-2 (4)
M common/trans.h
https://github.com/neutrinolabs/xrdp/pull/413/files#diff-3 (2)
M common/xrdp_client_info.h
https://github.com/neutrinolabs/xrdp/pull/413/files#diff-4 (2)
M libxrdp/xrdp_rdp.c
https://github.com/neutrinolabs/xrdp/pull/413/files#diff-5 (12)
M libxrdp/xrdp_sec.c
https://github.com/neutrinolabs/xrdp/pull/413/files#diff-6 (2)
M xrdp/xrdp.ini
https://github.com/neutrinolabs/xrdp/pull/413/files#diff-7 (4)

Patch Links:

https://github.com/neutrinolabs/xrdp/pull/413.patch
https://github.com/neutrinolabs/xrdp/pull/413.diff

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/neutrinolabs/xrdp/pull/413, or mute the thread
https://github.com/notifications/unsubscribe-auth/ADTH1JzAZlP4NMQK3cXL512i6v0dVir8ks5qhXRFgaJpZM4JoSnL
.

Some code reformatting needed but +1.

-     * SSL_OP_NO_SSLv2:
-     *
-     * We only want SSLv3 and TLSv1, so disable SSLv2.
+     * SSL_OP_NO_SSLv2
      * SSLv3 is used by, eg. Microsoft RDC for Mac OS X.
+     * No SSLv3 if disableSSLv3=yes so only tls used
      */
-    options |= SSL_OP_NO_SSLv2;
+     if (disableSSLv3==1) {
+	options |= SSL_OP_NO_SSLv3;
+     } else {
+	options |= SSL_OP_NO_SSLv2;
+     }

Does this look right?
If disableSSLv3 is set, should both SSL_OP_NO_SSLv3 and SSL_OP_NO_SSLv2 be set?

merged as 47124df
modified a bit

You're right.
SSLv2 should be disabled by default and that option should add v3 to the
denied list.
בתאריך יום ה׳, 25 באוג׳ 2016 ב-21:15 מאת jsorg71<EMAIL_ADDRESS>

* SSL_OP_NO_SSLv2:

*

* We only want SSLv3 and TLSv1, so disable SSLv2.

* SSL_OP_NO_SSLv2
* SSLv3 is used by, eg. Microsoft RDC for Mac OS X.

* No SSLv3 if disableSSLv3=yes so only tls used
*/

options |= SSL_OP_NO_SSLv2;

if (disableSSLv3==1) {

options |= SSL_OP_NO_SSLv3;

} else {

options |= SSL_OP_NO_SSLv2;

}

Does this look right?
If disableSSLv3 is set, should both SSL_OP_NO_SSLv3 and SSL_OP_NO_SSLv2 be
set?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/neutrinolabs/xrdp/pull/413#issuecomment-242488809,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ADTH1EmslET5TFbGFU00ILtArHO22tJWks5qjdvVgaJpZM4JoSnL
.

I changed it to this
    options |= SSL_OP_NO_SSLv2;
    if (disableSSLv3)
    {
        options |= SSL_OP_NO_SSLv3;
    }

Thanks guys, pleased to contribute open source project :D.


Create a basic world

At this point, a world should be created. Also, create 1-2 basic actions that would mutate the world, and then a few actions that would be able to check if it's been successfully mutated or not.

I've had almost a week of doubts, hesitations and back-and-forths in my mind about this.

The biggest point here is that I just don't know what the world should look like at this basic stage.
There's also the issue of me wanting to create a "flawless" world architecture that would account for every single feature being easy to add in the future. Which is obviously impossible.
I'm basically afraid of proceeding with this task because I want to do it perfectly the first time.

Therefore I'm making a difficult decision to proceed with this on an extremely basic level in order to just overcome the "writer's block" and perfectionism issues.  Due to all of that, this is going to be implemented the following way:

The world is going to be an uber struct that includes everything related to the game: the player, the monsters, the locations, the items, the skills, everything.
The Action's execute() method should accept a mut World reference so that every single command is able to mutate the world if necessary.
The World should include the player standing in the Void.
The Void is a Location that has a Description and that's it.
The Player is an Entity that has an optional name but starts nameless. The player can name themself but only once: renaming  yourself is impossible.
The World should have commands to describe the current location and remember their name or name themself. Also, the player has a current location.


[doc] fixing the build

Changes
fix RTD build.
Related Issues
Closes # (add issue numbers)
Checks

[ ] updated CHANGELOG.md
[ ] updated tests
[ ] updated doc/
[ ] update example/tutorial notebooks

Now our own doc pipeline succeeds, but not RTD. I guess its a caching problem of the dependencies.
https://github.com/spatialaudio/nbsphinx/issues/563

thanks for taking a look @marscher , let's keep an eye on this


No Changelog for gulp 3.9.1

There is no changelog for the new patch release 3.9.1, so nobody knows which was changed. Can someone add this info, please?

It seems, that there is no any changes?

@code-chris You can always read commits :) Here's issue
https://github.com/gulpjs/gulp/issues/1518

@TrySound There are many changes between 3.9 and 3.9.1.. So which changes are important?

@code-chris That was only docs changes. Nothing important. 3.9.1 was happened only because previously man pages was accidentally dropped.

@TrySound For me its ok, but its an official release, so there should be an entry in the changelog, or not?

All the gulp team works on this in their free time.  Something was broken and we got a change out yesterday when we had some time.  The community is welcome and encouraged to submit PRs for anything, especially documentation things like this.


Nit in license

You lost the spaces off the start of the first line of https://github.com/twardoch/fonttools-utils/blob/master/LICENSE

Huh? Don't see it. Screenshot please.
Actually, THAT license file is the one provided by Github, I hadnt touched that file myself. I only did inside the MacOSX... tool folder. I made one repo where I'll be collecting more tools, each in a subfolder.
Sent from my mobile phone.

On 20.03.2015, at 04:26, Dave Crossland<EMAIL_ADDRESS>wrote:
You lost the spaces off the start of the first line of https://github.com/twardoch/fonttools-utils/blob/master/LICENSE
—
Reply to this email directly or view it on GitHub.

THAT license file is the one provided by Github

Ah, interesting! I'll file a bug there too

BTW github acknowledged their error and fixed it


AST: Restore unqualified lookup quirk for Swift 3 mode

In Swift 3, unqualified lookup would skip static methods
when performing a lookup from instance context.
In Swift 4 mode, if a module method is shadowed by a static
method, you will need to qualify the module method with the
module name.
It would have been nice to isolate the quirk in Sema and
not AST, but unfortunately UnqualifiedLookup only proceeds
to lookup in the module if scope-based lookup failed to find
anything, and I don't want to change that since it risks
introducing performance regressions.
Fixes rdar://problem/29961715.

@swift-ci Please smoke test

@swift-ci Please smoke test

Is it really correct for this to be an error in Swift 4 mode?


feat!: Update to Capacitor 5

updated using https://github.com/ionic-team/migrate-capacitor-plugin
closes https://github.com/capacitor-community/fcm/issues/143

Same here: https://github.com/ionic-team/migrate-capacitor-plugin/issues/2

Will this get released any time soon? 😅
Kinda stuck on the Capacitor 5 update for our apps right now, waiting for this.

that's also fixed in Capacitor 4.8.0

Thanks! I've skipped v4 and released in v5 to keep up with Capacitor major


beginDrag should allow an array

I'm maintaining a mult-selection model, and when an item is dragged, I'd like to represent that with an array.
i.e.
const spec = {
  beginDrag() {
    return [{id: 1}, {id: 2}];
  }
}

Currently the check uses lodash.isPlainObj which disallows array. There doesn't seem to be a technical reason for this (though maybe best practice is to wrap with a POJO). What about loosening that check?

Would {ids: [1, 2]} work?


auto-publish

@olydis,
Thanks for your contribution as a Microsoft full-time employee or intern. You do not need to sign a CLA.
Thanks,
Microsoft Pull Request Bot

linux-x64 restart

win32-x64 restart

publish job


Unit test in partner.py - cifar10

Add unit tests for the 2 modules partner.py and datasets/dataset_cifar10.py
This could be used as an example to create new tests when adding a feature to the project
I have first wrote test in the tests.py and then implement these constraints in the methods shuffle_labels and corrupt_labels with  if ... raise Exception
These Exception are then caught by with pytest.raises(Exception):
If we plan to add lots of tests, we could consider creating 1 test module for each module to keep it easy to read.
I need to make these tests more automatic. To test all the functions of the projects, we could create a script that will generate a list of scenario, from the datasets and federated learning approaches currenty supported. This list would be used as arguments in tests.py to test more complex functions such as split_data (Functions that need scenario mpl dataset objects as arguments)

solve #191

solve #187
input_shape is still unused


Ingest fails

When ingesting public tagpacks into a local db, I get the following constraint error:
tagpack-tool insert --public packs -u postgresql://$USER:$PASSWORD@$DBHOST:$DBPORT/tagstore
==== TagPack insert starts =====
Path: packs
 Detected repository root in . =
Loaded TagPack schema definition: tagpack_schema.yaml
Loaded taxonomies: ['entity', 'abuse']
Collected 52 TagPack files

1 packs/Alt-Right.yaml: FAILED
insert or update on table "tag" violates foreign key constraint "tag_currency_address_fkey"
DETAIL:  Key (currency, address)=(ETH, 0x87FAd7298aF4346aadEce847a67CDFd97bDAdA14) is not present in table "address".

2 packs/africrypt-hack.yaml: FAILED
current transaction is aborted, commands ignored until end of transaction block

Could this be caused by #39 ?

@cybersec-code could you please check?

@myrho's hunch  was right; _get_currency_and_address() should be the only way to get the address from a tag.


0.9 is available

See https://github.com/MicrosoftGenomics/msgen#release-notes-v0-9-0

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 0d65cda4-f125-43c7-3b46-7eab85fad8b4
Version Independent ID: a7de4399-04af-30ee-7b52-97823d5a2626
Content: Version release history - Microsoft Genomics
Content Source: articles/genomics/version-release-history-genomics.md
Service: genomics
GitHub Login: @grhuynh
Microsoft Alias: grhuynh

@MatthewSteeples Thank you for bringing this to our attention. I have assigned this to the content team to update the Version Release History document with this release detail.

@MatthewSteeples
We will now proceed to close this thread. If there are further questions regarding this matter, please respond here and @YutongTie-MSFT and we will gladly continue the discussion.


[ios + android] Added "width", "height" and "uri" to captured image when target is "temp"

Thanks for this PR.
Why only

when target is "temp"

?

Hey! @npomfret
No special purpose. Just I'm using only "temp"
You want it for all?

Sure.  I don't think it make sense to change the return type for one and not the others.  Maybe images should have the same shape too?

I think so.
I'll add this. I think it's not muck work

Hello,
Width & height attribute is very important for me. I'm curious why these not merge to repo?

Yes, please merge it to repo ASAP.

Why is this not merged yet? looks clean :/


writing an own directive which uses ui-select doesn't work

Hi,
I have a problem with writing an own directive which uses ui-select . e.g. I would like to use an element like
 <m-select ng-model="item.id"  placeholder="Please select item..." list="list"></m-select>
The problem is that the model value is NOT written back after a selection is done.
Example is on plunkr: http://plnkr.co/edit/dJpEtxLsbNWNI8260Q4D?p=preview
BTW, In this example I also used a directive which converted the model.value between object and id...
e.g.
choices in form of objects = [{"id": 20,"label": "abc"},{"id": 21,"label": "def"}]
model value in form of an id: var model = { "testId": 20 }

As mentioned, ng-model requires a dot. See wiki and http://stackoverflow.com/questions/17606936/angularjs-dot-in-ng-model

Closing as a legacy issue over a year old.


TODO: document logging

include logging to file, passing core logs through, log level, etc...

Agreed. I'd even argue that we should log to a file (maybe in a batched manner for performance) by default


how to extract a single file??

I read (multiple times) all examples, docs, searched directly the code from the repo... Sorry I couldn't figure it out.
I also looked at https://github.com/fcorbelli/zpaqfranz/issues/13 but even the example given is not working for me
I am using zpaqfranz v58.6c-NOJIT-L(2023-07-19) on macOS 13.4.1 - installed via Homebrew
Simple example, I have a folder I want to back up:
/Users/luke/important-files/
   file1.txt
   file2.txt
   folderB/
      file3.png

I issue this command:
zpaqfranz a ~/backup.zpaq /Users/luke/important-files

All fine so far... archive is created, and I can also update, seems to work fine
Now I want to restore just file3.png...
I made many failed attempts...
list (ok)
~ $ zpaqfranz l ~/backup.zpaq
zpaqfranz v58.6c-NOJIT-L(2023-07-19)
/Users/luke/backup.zpaq:
1 versions, 6 files, 3.676 bytes (3.59 KB)

- 2023-07-22 20:31:48                   0 d0755 /Users/luke/important-files/
- 2023-07-22 20:32:03               6.148  0644 /Users/luke/important-files/.DS_Store
- 2023-07-22 20:31:38                   4  0644 /Users/luke/important-files/file1.txt
- 2023-07-22 20:31:38                   4  0644 /Users/luke/important-files/file2.txt
- 2023-07-22 20:32:08                   0 d0755 /Users/luke/important-files/folderB/
- 2023-07-22 20:31:53               3.486  0644 /Users/luke/important-files/folderB/file3.png

                9.642 (9.42 KB) of 9.642 (9.42 KB) in 6 files shown
                3.676 compressed

0.001 seconds (00:00:00) (all OK)

extract - 1st try (fail)
~ $ zpaqfranz x ~/backup.zpaq file3.png
zpaqfranz v58.6c-NOJIT-L(2023-07-19)
/Users/luke/backup.zpaq:
1 versions, 6 files, 3.676 bytes (3.59 KB)
Extract 0 bytes (0.00 B) in 0 files (0 folders) / 8 T

0.002 seconds (00:00:00) (all OK)

extract - 2nd try (fail)
~ $ zpaqfranz x ~/backup.zpaq '/Users/luke/important-files/folderB/file3.png' -to /tmp/file3.png
zpaqfranz v58.6c-NOJIT-L(2023-07-19)
/Users/luke/backup.zpaq:
1 versions, 6 files, 3.676 bytes (3.59 KB)
Extract 3.486 bytes (3.40 KB) in 1 files (0 folders) / 8 T
Free space on <<-to /tmp/file3.png>>
is                          0
needed                  3.486
33646: -to is a single file, NOT a folder. Maybe you miss the final /?
Aborting extraction. Use -space to bypass and enforcing.
23013: zpaqfranz error: Not enough free space

0.003 seconds (00:00:00) (with errors)

extract - 3rd try (fail)
~ $ zpaqfranz x ~/backup.zpaq '/Users/luke/important-files/folderB/file3.png' -to /private/tmp/file3.png
zpaqfranz v58.6c-NOJIT-L(2023-07-19)
/Users/luke/backup.zpaq:
1 versions, 6 files, 3.676 bytes (3.59 KB)
Extract 3.486 bytes (3.40 KB) in 1 files (0 folders) / 8 T
Free space on <<-to /private/tmp/file3.png>>
is                          0
needed                  3.486
33646: -to is a single file, NOT a folder. Maybe you miss the final /?
Aborting extraction. Use -space to bypass and enforcing.
23013: zpaqfranz error: Not enough free space

0.003 seconds (00:00:00) (with errors)

extract - 4th try (fail)
~ $ zpaqfranz x ~/backup.zpaq '/Users/luke/important-files/folderB/file3.png' -to /private/tmp/
zpaqfranz v58.6c-NOJIT-L(2023-07-19)
31744: MAGIC: selected 1 file extracting to a folder => merge to /private/tmp/file3.png
/Users/luke/backup.zpaq:
1 versions, 6 files, 3.676 bytes (3.59 KB)
Extract 3.486 bytes (3.40 KB) in 1 files (0 folders) / 8 T
Free space on <<-to /private/tmp/file3.png>>
is                          0
needed                  3.486
33646: -to is a single file, NOT a folder. Maybe you miss the final /?
Aborting extraction. Use -space to bypass and enforcing.
23013: zpaqfranz error: Not enough free space

0.002 seconds (00:00:00) (with errors)

extract - 5th try (fail)
~ $ zpaqfranz x ~/backup.zpaq '/Users/luke/important-files/folderB/file3.png'
zpaqfranz v58.6c-NOJIT-L(2023-07-19)
/Users/luke/backup.zpaq:
1 versions, 6 files, 3.676 bytes (3.59 KB)
00000001 ?existing files skipped (-force overwrites).
Extract 0 bytes (0.00 B) in 0 files (0 folders) / 8 T

0.002 seconds (00:00:00) (all OK)

How do I do it?? 🤷

I figured out that free space detection seems broken on macOS, so if add -space to ignore it, I was able to extract a single file using
zpaqfranz x ~/backup.zpaq '/Users/luke/important-files/folderB/file3.png' -space -to /tmp/file3.png

But, when I try
zpaqfranz x ~/backup.zpaq '/Users/luke/important-files/folderB/file3.png' -to /tmp/ -space -all 4

...it fails. So, new question: is it possible to extract all versions of a single file?

Yes, you can, with ranges
Please read the examples by asking help
zpaqfranz h x

And you can quickly select files with only
-only *.png
-only *somefile.jpg

Hmm, ok well sorry to have bothered you. I can't figure it out, just doesn't work on my system. But I will keep trying.

Relax
Tomorrow I will answer better, it is my fault if something is not easy enough

Extract into folder muz7:            x z:\1.zpaq -to z:\muz7\
0-bytes files (check restoration):   x z:\1.zpaq -to z:\muz7\ -kill
Extract into single directory:       x z:\1.zpaq -to z:\muz7\ -flat
Extract without utf,<255,.eml:       x z:\1.zpaq -to z:\muz7\ -utf -fix255 -fixeml
Extract forcing overwrite:           x z:\1.zpaq -to z:\muz7\ -force
Extract version K:                   x z:\1.zpaq -to z:\muz7\ -until K
Extract last versions of multipart:  x "z:\a_???" -to z:\ugo
Extract into muz7 every versions:    x z:\1.zpaq -to z:\muz7\ -all
Show the filelist (if any):          x z:\1.zpaq -filelist
Show the filelist (if any) of v3:    x z:\1.zpaq -filelist -until 3
Extract all *.xls into new archive:  x z:\1.zpaq *.xls -repack onlyxls.zpaq
Extract from a VSS (Windows):        x z:\1.zpaq \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy1\path -to d:\output
Replace path in extract              x 1.zpaq -find /tank/ -replace z:\uno\
Change path in extract, longpath     x 1.zpaq -replace z:\uno\ -longpath
Create files' tree                   x 1.zpaq -to z:\testdir\ -debug -kill -zero -longpath
Range-extract version                x copia.zpaq -only *comp.pas -to z:\allcomp -all -range 100-1000
Restore datetime and comments        x copia.zpaq -to z:\prova\ -all -comment
Restore huge image w/smart progress  x copia.zpaq -to z:\prova\ -image
Single file to stdout                x copia.zpaq 00000015.zfs >z:\15.zfs
Extract multiple zpaqs               x vu*.zpaq -to z:\pippo\

In your case, all version of a file, something like
zpaqfranz x copia.zpaq -only *comp.pas -to z:\allcomp -all -range 100-1000

You can use whatever you want for range, better NOT using - (I'll fix the help)
Example

-range :10000  (from 1 to 10.000)
-range 1000: (from 1.000 to 99.999.999)
-range 3 (from 3 to 3)
-range 10:100 (from 10 to 100)

The free space test definitely needs to be improved; it doesn't actually work if the folder doesn't exist.
So extracting to /tmp should always go, but extracting to /tmp/nonexistent requires the switch - space
Thanks for the report, I will try to improve

Sorry for little off topic: I sometimes use bandizip (https://en.bandisoft.com/bandizip/) to extract single files. It support zpaq files. It is Windows only, but maybe it will work with wine or something.

Well... I use my own GUI extractor :)
PAKKA

Well... I use my own GUI extractor :)
PAKKA
Bandizip have a poor multiversion support

Please check the attached-prelease (not tested on Mac, only on BSD)
58_7g.zip
You must compile with something like
g++ -Dunix  -O3  zpaqfranz.cpp -o zpaqfranz -pthread

Of course with g++ (/clang) installed. On newer Mac at first invocation everything should be "automagically" download
I am not a regular Mac User, please help me and report :)

Thank you for trying to help @fcorbelli
I did compile the custom build using the exact command you posted, and it runs successfully :pray:
However, it is outputting errors when I try to extract.
$ zpaqfranz x my-test-backup.zpaq -only "/some/dir/foo.png" -to "/tmp/" -all

zpaqfranz v58.7g-JIT-L(2023-07-23)
franz:-only                 /some/dir/foo.png
--------------------------------------------------------------------------------------------------------------------
franz:-all                                      4
my-test-backup.zpaq:
Skipping block at 378669404:  2249: allocx failed for 8.192
Skipping block at 378669404:  2249: allocx failed for 8.192
Skipping block at 378669404:  2249: allocx failed for 8.192
Skipping block at 378669404:  2249: allocx failed for 8.192

(...error repeats 120 more times...)

1 versions, 0 files, 378.669.404 bytes (361.13 MB)
Extract 0 bytes (0.00 B) in 0 files (0 folders) / 8 T

Extracted 0 files (124 errors) using 0 bytes x 8 threads

0.021 seconds (00:00:00) (with warnings)

Also, for extraction purposes, if we have a tree structure where the file basename alone is not enough to guarantee uniqueness, e.g.
/foo
  /bar
    baz123.png
  /quux
    baz123.png

I guess the correct way to exxtract only /foo/quux/baz123.png is to use the full path name instead of -only '*/baz123.png'? I thought about using the excellent fzf utility to filter and select files for extraction, but I can't figure out if there is a switch in zpaqfranz list mode that will output purely filenames without all of the extraneous info, headers etc.

Thank you for trying to help @fcorbelli
I did compile the custom build using the exact command you posted, and it runs successfully 🙏
However, it is outputting errors when I try to extract.
$ zpaqfranz x my-test-backup.zpaq -only "/some/dir/foo.png" -to "/tmp/" -all

(...)
This seems a broken archive.
What kind of CPU do you have?
Please try a
zpaqfranz autotest -all -to /tmp/whatever

running the .sh
BTW what exactly you want to do, with -only and -all?
Extract ALL the versions of a single file?
In this case you should use range x:y

Also, for extraction purposes, if we have a tree structure where the file basename alone is not enough to guarantee uniqueness, e.g.
/foo
  /bar
    baz123.png
  /quux
    baz123.png

I guess the correct way to exxtract only /foo/quux/baz123.png is to use the full path name instead of -only '*/baz123.png'?

You are right. Of course the "right" way is the full path
You can use -only as a quick-and-dirty access

I thought about using the excellent fzf utility to filter and
select files for extraction, but I can't figure out if there is a switch in zpaqfranz list mode that will
output purely filenames without all of the extraneous info, headers etc.

I do not know fzf
You can try the attached pre-relase with the
58_7i.zip
new... fzf command
On Windows there is the text-based GUI, but it takes too much effort on *nix

Asked for help on fzf
https://github.com/junegunn/fzf/discussions/3379

Thank you for the continued effort!
In my situation, my "goal" would be to extract the last 3 versions, for example. I don't think the range x:y can achieve this, right? It would be something like range -3:
I compiled latest test version from 58_7i.zip, tests seem to pass
$ zpaqfranz autotest -all -to /tmp/foo/
zpaqfranz v58.7i-JIT-L(2023-07-23)
franz:-all                                      4
Self-test for correct internal functioning
Creating autotest folder in <</tmp/foo/>>
Iteration 0/9 chunksize    1.000.000
Iteration 1/9 chunksize      333.333
Iteration 2/9 chunksize      111.111
Iteration 3/9 chunksize       37.037
Iteration 4/9 chunksize       12.345
Iteration 5/9 chunksize        4.115
Iteration 6/9 chunksize        1.371
Iteration 7/9 chunksize          457
Iteration 8/9 chunksize          152
Iteration 9/9 chunksize           50
This seems a LITTLE ENDIAN CPU (aka:'normal')
         BLAKE3 : OK
         CRC-32 : OK
        CRC-32C : OK
     HIGHWAY128 : OK
     HIGHWAY256 : OK
      HIGHWAY64 : OK
            MD5 : OK
    SHA-256-PUT : OK
  SHA-256-WRITE : OK
          SHA-3 : OK
       SHA1-PUT : OK
     SHA1-WRITE : OK
      WHIRLPOOL : OK
           XXH3 : OK
       XXHASH64 : OK
      XXHASH64Y : OK
Time 47.62 seconds for bytes 1.748.003.691

The test batchfile is: /tmp/foo/dotest.sh

47.628 seconds (000:00:47) (all OK)

But, with my old, existing .zpaq archive, these new builds can't operate:
$ zpaqfranz fzf my-test-backup.zpaq

Skipping block at 378669404:  2249: allocx failed for 8.192
Skipping block at 378669404:  2249: allocx failed for 8.192
Skipping block at 378669404:  2249: allocx failed for 8.192
Skipping block at 378669404:  2249: allocx failed for 8.192
(repeated 124 times)

The old v58.6c-NOJIT-L(2023-07-19) version works without errors, so yes I guess this is some kind of compilation problem on my end due to macOS, or other code change.

Thank you for the continued effort!
In my situation, my "goal" would be to extract the last 3 versions, for example. I don't think the range x:y can achieve this, right? It would be something like range -3:

The "-" is not good (because confuse the parser)

I compiled latest test version from 58_7i.zip, tests seem to pass
(...)
$ zpaqfranz fzf my-test-backup.zpaq
Skipping block at 378669404:  2249: allocx failed for 8.192
Skipping block at 378669404:  2249: allocx failed for 8.192
Skipping block at 378669404:  2249: allocx failed for 8.192
Skipping block at 378669404:  2249: allocx failed for 8.192
(repeated 124 times)

The old `v58.6c-NOJIT-L(2023-07-19)` version works without errors, so yes I guess this is some kind of compilation problem on my end due to macOS, or other code change.

What kind of CPU do you have? Maybe M1 or M2?
If you DO NOT have an Intel, you MUST insert -DNOJIT when compiling!!
You can see in the header
zpaqfranz v58.6c-NOJIT-L(2023-07-19)
zpaqfranz v58.7i-JIT-L(2023-07-23)
Please try
g++ -Dunix  -O3 -DNOJIT zpaqfranz.cpp -o zpaqfranz -pthread

Ah, yes I am working on M1 and M2 cpus, sorry.
I rebuilt using -DNOJIT and now it is working, no more archive errors when reading!
The initial test of fzf flag output is good, the only thing is: there is a single blank line output at the start. It would be good to eliminate it.

Work in progress... :)

Awesome! I have not played with fzf on Windows, but there are Windows binaries in case you haven't found them, could be helpful in testing

zpaqfranz (for Windows) already include a text-based "GUI" :)

58_7j.zip
Please check this pre-release
Should

do fzf
list-extract the last N versions with    -all -range ::N
(not tested at all)

@fcorbelli this is great!  -range ::N is working!
well, I only have 2 comments:

if a zpaq file has e.g. 6 snapshots, and I ask for last 3 versions of file xxx using -range ::3, but file has not changed since v0002, then there is no extraction. I don't know if this is expected. I would expect to get the 2 versions contained from snapshot 0001 and 0002. Not sure if this is possible or not. Otherwise you will need to really know before hand what the last version is where your file was changed.

I have to prefix all -only flag with a * otherwise it doesn't work. Even if I give the exact full pathname like

zpaqfranz x my-test-archive.zpaq -only "/some/dir/foo.png" -to "/tmp/" -all

It doesn't work, I have to use
zpaqfranz x my-test-archive.zpaq -only "*/some/dir/foo.png" -to "/tmp/" -all

Not sure if it's a bug...

@fcorbelli this is great! -range ::N is working!
well, I only have 2 comments:

if a zpaq file has e.g. 6 snapshots, and I ask for last 3 versions of file xxx using -range ::3, but file has not changed since v0002, then there is no extraction. I don't know if this is expected. I would expect to get the 2 versions contained from snapshot 0001 and 0002. Not sure if this is possible or not. Otherwise you will need to really know before hand what the last version is where your file was changed.

Bit too complex to do, I don't think I will be implement anytime soon.
It is NOT a "selection", but a DE-selection:rather crude, but enough for my needs
if ((version)<g_rangefrom) || (version)>g_rangeto))
   issel=false;

I have to prefix all -only flag with a * otherwise it doesn't work. Even if I give the exact full

Not sure if it's a bug...

Not a bug, at best a feature :)
You can use as many -onlys as you want, not just one, and they normally specify the full filename. With wildcards you can use as a "trick" to do not write the full path
Some time ago I had thought about a regex parser, but in the end the game was not worth the candle.
With "only" you can quickly catch about everthing

Not a bug, at best a feature :) You can use as many -onlys as you want, not just one, and they normally specify the full filename. With wildcards you can use as a "trick" to do not write the full path

Ok fair enough, but why does -only /some/dir/foo.png not work? Do I still misunderstand the syntax? I am trying to be as explicit as possible with the extraction, so it seems odd that I have to add the * even when I am passing an exact matching full pathname,

Not a bug, at best a feature :) You can use as many -onlys as you want, not just one, and they normally specify the full filename. With wildcards you can use as a "trick" to do not write the full path

Ok fair enough, but why does -only /some/dir/foo.png not work? Do I still misunderstand the syntax? I am trying to be as explicit as possible with the extraction, so it seems odd that I have to add the * even when I am passing an exact matching full pathname.

You are extracting a file to a folder
A file should be extracted to a file, with even a different name
C:\zpaqfranz>zpaqfranz l versioni.zpaq
zpaqfranz v58.7k-JIT-GUI-L,HW BLAKE3,SHA1/2,SFX64 v55.1,(2023-07-24)
franz:-hw
versioni.zpaq:
4 versions, 20 files, 23.669 bytes (23.11 KB)

- 2022-10-09 15:11:54                   0 A     1.txt
- 2022-09-17 16:56:17                 570 A     2.txt
- 2022-09-18 15:12:54                 874 A     3.txt
- 2022-09-18 15:12:54                 426 A     4.txt
- 2022-10-07 14:10:08                   1 A     c.txt
- 2022-09-08 15:54:43              13.050 A     cpuz.txt
- 2022-09-08 15:54:43                 153 A     lavoretti.txt
- 2022-09-18 19:25:43                 190 A     news.txt
- 2022-09-08 15:54:43              21.966 A     nz.txt
- 2022-09-08 15:54:43               1.398 A     nz2.txt
- 2022-10-03 15:06:06                   0 D     prova/
- 2022-10-03 15:06:12                   0 D     prova/directory3/
- 2022-10-03 15:06:14                   4 A     prova/directory3/versione3.txt
- 2022-10-03 15:05:15                   4 A     prova/file1.txt
- 2022-10-03 15:05:22                   5 A     prova/filea.txt
- 2022-10-03 15:05:48                  14 A     prova/versione2.txt
- 2022-09-08 15:54:43              14.338 A     provapc.txt
- 2022-09-08 15:54:43                 434 A     thelog.txt

A file to a file
zpaqfranz x versioni.zpaq prova/versione2.txt -to z:\ripristinato_versione2.txt

The file(s) inside a folder to a folder
>zpaqfranz x versioni.zpaq prova/directory3 -to z:\newfolder

A file to a folder, creating subfolders as needed
zpaqfranz x versioni.zpaq -only prova/filea.txt -to z:\thetree

C:\zpaqfranz>dir z:\ /s

 Directory di z:\

03/10/2022  15:06    <DIR>          newfolder
03/10/2022  15:05                14 ripristinato_versione2.txt
24/07/2023  21:01    <DIR>          thetree
               1 File             14 byte

 Directory di z:\newfolder

03/10/2022  15:06    <DIR>          .
03/10/2022  15:06    <DIR>          ..
03/10/2022  15:06                 4 versione3.txt
               1 File              4 byte

 Directory di z:\thetree

24/07/2023  21:01    <DIR>          .
24/07/2023  21:01    <DIR>          ..
24/07/2023  21:01    <DIR>          prova
               0 File              0 byte

 Directory di z:\thetree\prova

24/07/2023  21:01    <DIR>          .
24/07/2023  21:01    <DIR>          ..
03/10/2022  15:05                 5 filea.txt
               1 File              5 byte

     Totale file elencati:
               3 File             23 byte

I understand the examples you give above, but I think some things are not working right on the macOS version, or at least not working as they do on Windows.
For me, with the following syntax, zpaqfranz is ignoring the -to directive and simply extracting the file(s) to the current directory.
$ zpaqfranz x my-test-backup.zpaq '/Users/luke/some/path/file.png' -to /private/tmp/ -all

Instead of creating a 0001 dir in /private/tmp and extracting there:
/private/tmp
          |
          0001
            └── Users
                └── luke
                    └── some
                        └── path
                            └── file.png

It gets created in my current directory ($HOME):
/Users/luke
         |
         0001
           └── Users
               └── luke
                   └── some
                       └── path
                           └── file.png

Also, in your examples, you have e.g. prova/directory3/versione3.txt. In my .zpaq, all paths are shown absolute, I have no relative paths. So it would be shown as /Users/luke/prova/directory3/versione3.txt
This variant with -only does not match any files unless I add a * in front of the filename, and also add the -space option:
fails:
$ zpaqfranz x my-test-backup.zpaq -only '/Users/luke/some/path/file.png' -to /private/tmp -all

succeeds:
$ zpaqfranz x my-test-backup.zpaq -only '*/Users/luke/some/path/file.png' -to /private/tmp -all -space

Lastly, in fzf discussion you show a sample output (maybe theoretical) where files are listed along with their version#s:
0000999945|0001|file.txt
0000777124|0003|file.txt
0000777125|0002|file.txt

Is this output possible? Current fzf mode outputs only filenames.

fzf should show versions with -all
On the rest
For a single file -to does NOT extract to a folder, but to a file
Why? Because dr.Mahoney choose this way
You can find the historic example here
https://mattmahoney.net/dc/zpaqdoc.html
Almost everything is maintained for backward compatibility as a drop-in replacement (not 100% in fact, the output is way different, I do not like zpaq's output at all)

Ok, I have been playing around with the syntax. I understand now that you don't want to change the syntax to stay as close to Dr. Mahoney's spec as possible. I will say that in all my years of working in the shell, this has to be one of the craziest syntax/arg parsers I've ever seen!

I indeed agree :)
And this is NOT the "craziest thing" within zpaq :)
I can tell you that from time to time I discover things that perplex me, even though I have been developing for more than 35 years :)

I still don't understand why this command does not work:
zpaqfranz x my-test-backup.zpaq '/Users/luke/some/path/foo.md' -to '/private/tmp/foo.md' -range 0009

This is a bug on the new "mac-friendly-get-the-free-space"?
Try -space
But do NOT use -range 9, but -until 9 (later more)

For now, I am at least able to extract a specific file @ a specific snapshot version, but there seem to be some things broken here.
Well, I am pretty confident that it is possible to restore everything inside a zpaq, even on systems without utf or longfilename support (the -flat)
Even millions of files and tens of terabytes. There is also a specific function for a third-party certification, with hashdeep :)

Can you please attach some kind of your file to make some tests?
This would make everything much easier
If there is any correction or improvement I am always ready
Unfortunately, on *nix you don't know what a file is, or even a folder, or even "where" a file is (there are mounts)
So figuring out if /tmp/foo.txt is a file, or a folder, is not trivial, not at all.
Just as it is not trivial to figure out whether it is on the computer's hard drive, and not perhaps a mount of an NFS folder or whatever
The "old" zpaq does not have any kind of verification on written data. If space runs out (which happens frequently in planned backups) the .zpaq file will be silently "truncated," but without zpaq giving an error message [data of course non exctractable]
As you can see this is a gigantic problem (for crontab executions)
So zpaqfranz tries to do three things (which succeed well on Windows, not so well on *nix, but better then nothing)
1- Figure out if the target space is non-zero (you try to write to a filled-up filesystem)
2- Figure out how big a file is expected to be written.
3- Verify that the file being written is as large as it was expected to be
On Windows it is quite simple (ignoring symlinks that actually exist, but are rare), not so simple on network shares
Short version: practically everywhere by putting a -space these tests are not done, just to allow starting even in the case of "confusion"
Another note: to restore a single version the switch is -until. Range does a whole range of other things.
-until means it restores a file (or all files) of version X (aka: all files not deleted from 1 to X)
-range extracts within a single folder many different subfolders, each with files in THAT version. I use it for source versioning
Translation: if in version 9 you have 100 files, and in version 10 you have 3 files, with -until (I am simplifying, there are 1000 different cases, nothing is easy with zpaq) you will get a folder with 103 files
With range you will get a folder with 3 files

Thank you @fcorbelli
Indeed, this format works (example)
zpaqfranz x my-test-backup.zpaq "/Users/luke/foo/bar.md" -to "/tmp/restored/bar.md" -space -until 9

I am happy 🙏   confused, but happy... which is fine

Actually, after a couple of years of digging, it starts to make sense :)
PS should work without -space too. Maybe... who knows? :) :)

A bit of explanation on the heuristic
/tmp/restored/bar.md

is /tmp/restored/bar.md an existing folder?  => yes, get free space, no => continue, split the path in "pieces", ad rebuild backward until the n-1 (take the "father" of /tmp/restored/bar.md, that should be /tmp/restored)
If /tmp/restored is a path that does exists, then zpaqfranz will "automagically" get the free space on /tmp/restored
If /tmp/restored does NOT exists, then zpaqfranz throws in the towel and "thinks" there is no space
In fact, it is true, a non existend path does not have free space
I didn't make further attempts, otherwise sooner or later you get to "/," and there the space is (generally) always present
Short version: if you are dealing with non-existent output paths, use -space to bypass any checks.

I think I can close, at least for now, after explaination and examples


Figure design

We want to have consistent colors, legends, etc for figures- need to define some style rules or have someone come in after the fact and make them consistent.

@jread-usgs you are pretty savvy with this- maybe we'll ask you to drop in for one task on this?


Recording in random position from an existing wav file

I am currently building a voice recorder application.
Often they want to go back in the recording and record from that position again.
I am not sure if this is supported through the WaveFileWriter?
How can this be accomplished?
Thank you

This is not a feature of NAudio, but I have done something similar with my own custom version of WaveFileWriter. So long as you are careful about block align, its not too hard to write over the top of the data chunk of an existing file


Limit autocomplete results based on context

Autocomplete results should be limited based on the expected value type for a property or function param instead of results that cannot be used.
for instance,

autocomplete results should not include functions on data fields
autocomplete results should not include strings for a function parameter that is of type object
etc...

### Tasks
- [ ] https://github.com/appsmithorg/appsmith/issues/28106
- [ ] https://github.com/appsmithorg/appsmith/issues/28107

Closing this issue since both the subtasks are completed


[Bug] fix alignment

Description
i want to fix the alignment of below pages.. there are many such
Screenshots

Checklist

[X] I have checked the existing issues.
[X] I follow Contributing Guidelines of this project.
[ ] I am a IWOC'24 contributor.
[X] I am a JWOC'24 contributor.
[ ] I am a SWOC'24 contributor.

Code of Conduct

[X] I agree to follow this project's "Code of Conduct".

What alignment are you talking about, please mention the issue clearly @Anushkabh

The screenshots above shows text animations search text animations are not aligned properly..
So i would fix those.

Do you want to just align the heading at center @Anushkabh

Yes
Kindly assign me this issue.. so i could reach top 10

Proceed and check whether heading is not aligned center or Search bar and Search text @Anushkabh

Provide your PR pls @Anushkabh

JWOC period is over now @Anushkabh


Not getting the image

Thanks for your help and response on the original inactive github project. I did visit your fork and followed your instructions by running
blast-radius --serve "C:\Projects\...........\Terraform - Full Course\weatherapi"
the path contains a terraform initialized project
it runs then I choose Firefox, I get a page with python code as follows (not sure what should I do to have the image insttead of code)

@eliassal Could you provide more details? Does the CLI show any output or errors?

Nothing

and here is the same output

@Ianyliu any feedback please?

@eliassal could you try running via Docker?

I am running the tool in a powershell terminal. in Docker? How?

Thanks @Ianyliu , I have already docker desktop, I ran the command you gave and it did not work for the project I was using with backend error as you can notice in snapshot 1, I tried both init --migrate and the 2nd init reconfigure but it gives a backend error, so is this a terrafoprm error or blast error?

but it worked for another project and I got a graph as you can see in snapshot 1. However,

Also, I tried in terminal running blast-radius for the 2nd project as I throught it ws a terraform issue but gave the same error as the 1st one; it did not display the graph
blast-radius --serve "C:\Projects\AzureDevOpsProjects\terraform\MS-Quickstart\Direct web traffic with Application Gateway"

Another question, when graph was displayed correctly, is it possible to tell the blast docker container to save it to a directory in the host machine?


SortFilterProxyModel with PyQt

How do I use this SortFilterProxyModel which is in c ++ with PyQt??

You can use my patch from https://github.com/oKcerG/SortFilterProxyModel/issues/25 to build as standalone plugin.
After building you get a libqmlqsortfilterproxymodelplugin.so library. You should put it (and qmldir file) into /usr/qml/QQSFPM/ or whereever your QML plugins live. Then you can do "import QQSFPM 0.2" as usual.
qmldir file:
module QQSFPM
plugin qmlqsortfilterproxymodelplugin

i tried your example, I had this error "could not find the executable, please specify one"
Please i am beginner, Can you Explain more ??

Hmm, let's try.

Checkout code.
Create plugin.cpp and SortFilterProxyModel.pro as in #25.
qmake
make
You get libqmlqsortfilterproxymodelplugin.so
Put it into /usr/qml/QQSFPM/
Put qmldir file nearby.
Profit.

Feel free to contact me on skype: iberiozko.

I tried but it did not work!!!! so i show you my file "finale":

-finale
finale.pro
-SortFilterProxyModel
SortFilterProxyModel.pro
-Header
filter.h
qmlqsortfilterproxymodel.h
sorter.h
-Source
filter.cpp
plugin.cpp
qmlqsortfilterproxymodel.cpp
sorter.cpp
-Sources
main.py
-Ressources
-qml.qrc
-/
main.qml

when i try this, i get : "qmlqsortfilterproxymodelplugin.dll" not "libqmlqsortfilterproxymodelplugin.so" so when i put this .dll and qmldir in my folder (stage/finale) and i run my PyQt i get error "module QQSFPM is not installed). so is the bug in the architecture that I showed you ?? or another problem??
thank you

Duplicate of #25


Adjusted ULP precesion for noncentral distribution test

As title. Simple precision adjustment for failing noncentral distribution tests

Build numba_smoketest_cpu_nprgen_31 has started

Build numba_smoketest_cpu_nprgen_31 has started

Was green


fix: toast notification not displaying when table is missing

fixes #68

:tada: This PR is included in version 1.8.4 :tada:
The release is available on:

v1.8.4
GitHub release

Your semantic-release bot :package::rocket:


Handling empty strings for integers & floats in changesets?

Hello!
In a Phoenix Controller I update an Ecto model that has a high number of optional integer fields. From an HTML form these fields will come in as empty strings, "". I'd love to either translate these to nil in Ecto or be treated as if they weren't passed in but I only see two options for doing so now:

Sanitize the params before I pass them into the changeset to make this conversion
Make my own custom Integer/Float types that do this translation but delegate back to the original type otherwise.

The first approach seems contrary to the purpose of changesets, but the second approach seems like overkill.
Unless I am missing a sensible third option here (likely, I am n00b at all this), I think there are two possible ways to modify Ecto to fix this:

Within non-string types, translate empty string to nil during casting. (seems sensible but perhaps there's an edge case I'm not thinking of)
Add some kind of "translation" hook to changesets or schemas that occurs before casting. (I can't think of many other use cases for this though.)

Thoughts?

The purpose of changesets is also to validate parameters, so I think it's not contrary to the purpose of changesets to do something like:
def changeset(model, params) do
  params = Enum.map(params, fn
    {k, ""} -> {k, nil}
    pair -> pair
  ) |> Enum.into(%{})
  # ...
end

or something similar. That said, the fact that HTML forms send blank fields as "" is probably something that could be taken care of by Ecto; however, Ecto has nothing to do with HTML so maybe this should happen at the Phoenix level.

This should be handled by Ecto but I am not sure what is the best way to handle this. Should we scrub those values before calling cast? Maybe by adding a scrub function that removes anything that is an empty binary or an empty list from the map? On the other hand we could say this is a controller responsibility as those attributes are leaking due to HTML. Thoughts?

Ok. This is not an Ecto responsibility. You need a scrub_params/2 function that should go through all parameters replacing any "" or [] by nil. In fact, we will remove the current blank checks from Ecto and discuss about adding scrub_params/2 directly to Phoenix. Thank you!

In my case, I have a search form on an index page.
When the form is submitted, the search param is set. In the case that the page is used as an index page, the search param is not set. In that case, we cannot use the scrub_params plug, since the param raises an error.
What is the best way to handle that case?


Display additional metadata for test results

It's a common problem that users want to include additional metadata about their tests, e.g. the infrastructure config (machine size, OS). You can stuff this in to the JSON payload today but the frontend won't display it. Clicking on a data point should pop up a modal that contains all of this data.

Fixed in #133


Heat map to view statics

it would be amazing to have different points of view for statistics. I would like to have a heatmap type view.

Hi thanks for the suggestion. Do you have an example of this heat map? I'm not sure I'm on the same page as you.

Ah yes. A feature similar to that is on the list. Thank you for the suggestion. I like the term heat map.


discussion

Hello, @xvkdev
Am I right, it is you? I have a question, best to discuss out of the fl.com

not related to this repo

@maxkoryukov sure but if it's a private thing here is probably not the best place either :-)


Directory with space is not supported

When picking a directory for the extraction, if the directory contains a space character (D:\World Flipper) there will be an error even though it's a valid directory. It works with underscore though

This is known issue as stated in documentation. Whitespaces were intentionally blocked since they're causing problems while extraction process, so we need to fix that part first.


Blank packages

For some reason, some packages are blank rows. (see #1)

this issue fixed in version 4.0
https://github.com/HandyOrg/HandyWinGet/releases/tag/v4.0.0-beta.1


How to generate motion masks?

Nice work, Pose free dynamic scene gaussian splatting was something I was waiting on.
Congratulations on being the first one to do it.
I wanted to know how motion masks are generated. Because if it cannot be generated easily for most scenes then this will be a big limitation.
And also any estimate on code release time.

Hi,
Thank you for your comments. For now, we use the G.T. motion mask. As mentioned in our paper, we take developing an efficient way to tackle the motion mask problem as our future goal. Please continue following our research.
The way of generating motion masks in RoDynRF might help you. However, we also tried their way, and we found that their approach always fails, which is also the motivation for our future goal.
For the estimated time of code release, we will release the code soon. Will make comments in this issue once released.
Best,
Fang

Thank you for your reply.
Looking forward for your future work.


[Perf] Use sparse checkout in perf.yml

File /eng/common/pipelines/templates/jobs/perf.yml should use sparse-checkout.yml (excluding test recordings) instead of the checkout task.  .NET clone times improve from 3m25s to 45s.
Example:
- template: /eng/common/pipelines/templates/steps/sparse-checkout.yml
  parameters:
    Paths:
    - '/*'
    - '!sdk/**/test-recordings/*'
    - '!sdk/**/recordings/*'
    - '!sdk/**/SessionRecords/*'
    - '!sdk/**/session-records/*'
    Repositories:
    - Name: $(Build.Repository.Name)
      Commitish: $(Build.SourceVersion)
      WorkingDirectory: $(System.DefaultWorkingDirectory)
    - Name: Azure/azure-sdk-tools
      Commitish: main
      WorkingDirectory: $(System.DefaultWorkingDirectory)/azure-sdk-tools

Fixed by https://github.com/Azure/azure-sdk-tools/pull/3874


Cannot import name 'AccessTokenInfo' from 'azure.core.credentials'

Package Name: az-ai-ml
Package Version:  1.19.0
Operating System: Linux
Python Version:  3.11

Describe the bug

[!NOTE]
Apologies if this issue should have been created in a different repository. Please let me know if I should create this ticket elsewhere.

In this PR, the minimum version of azure-core was updated to include the new SupportTokenInfo protocol and AccessTokenInfo class. However, when attempting to install the Azure ML extension using:
az extension add --name ml

the following error occurs:
cannot import name 'AccessTokenInfo' from 'azure.core.credentials'

As this issue is currently disrupting our CI pipelines, we would be grateful for any guidance you can provide to help us address it promptly.
Possible Cause
This issue seems to arise because the Azure ML extension installs azure_identity-1.18.0, which relies on AccessTokenInfo from azure.core.credentials. However,  azure-cli version 2.64.0 still uses an older version of azure-core (1.28.0), and hence the error above is thrown.
For reference, here’s the azure-cli requirements file.
To Reproduce
Steps to reproduce the behavior:

Install az cli
Run az extension add --name ml

Yes, same for me. It's now a blocker for my CI pipelines.

This is no blocking all my ML pipelines

This is also blocking our pipelines also

Thanks for the pointer to azure-identity @miladh – this works for me as a workaround on CI:
$ az extension add --upgrade --name ml -y
$ sudo /opt/az/bin/python3 -m pip install azure-identity==1.17.1

Also a blocker for me (muliple training pipelines)

azure-cli

This workaround did work on our case.

Thanks for the pointer to azure-identity @miladh – this works for me as a workaround on CI:
$ az extension add --upgrade --name ml -y
$ sudo /opt/az/bin/python3 -m pip install azure-identity==1.17.1

(use at your own risk)

This workaround did work on our case.

Appears that there is some intermingling of dependencies.
Looking at where CLI extension dependencies are installed for me, (~/.azure/cliextensions/ml), I do see that there are compatible versions of azure-core and azure-identity installed. However, the versions of these packages installed in the azure-cli Python virtual environment (e.g. /opt/az/lib/python3.11/site-packages/) seem to take precedence at runtime causing incompatibilities.
I mentioned this in the linked azure-cli issue, but I think something may need updating in the dependency resolution logic when installing an extension. Ideally, azure-cli extension Python requirements should remain compatible with the base azure-cli requirements when the extension is installed (i.e. azure-identity 1.17.1 should automatically have been installed with the ML extension).
I believe the workaround, as mentioned above, is to install an older version of azure-identity in whatever virtual environment azure-cli is using.
Example workarounds:
Linux:
sudo /opt/az/bin/python3 -m pip install azure-identity==1.17.1

Windows (in elevated/administrator terminal):
C:\"Program Files (x86)"\"Microsoft SDKs"\Azure\CLI2\python.exe -m pip install azure-identity==1.17.1

Not sure about Mac, but probably just need to run the python executable somewhere in /opt/homebrew/Cellar/azure-cli/<version>/libexec/ ?
I think you can glean the location of the environment your azure-cli is using based on the error message:
cannot import name 'AccessTokenInfo' from 'azure.core.credentials' (/opt/az/lib/python3.11/site-packages/azure/core/credentials.py) -> Need to use python executable somewhere in /opt/az

@pvaneck for Linux, the workaround works.  On my Windows machine, I ran the command (64-bit version), but it doesn't work.

Same issue here - stopping all of our CI pipelines.

As a temporary solution, you can add the following to your requirements.txt for a quick CI pipelines fix:
azure-identity==1.17.1

@pvaneck for Linux, the workaround works. On my Windows machine, I ran the command (64-bit version), but it doesn't work.

For our azure pipelines the following pipeline works
     C:\"Program Files"\"Microsoft SDKs"\Azure\CLI2\python.exe -m pip install azure-identity==1.17.1

This issue has been resolved. Please retry.
az extension add --name ml --version 2.30.0

please correct me if I am misunderstanding but this solution assumes that we are using / installed the azure-cli package. In our case we are not.
Will there be a fix directly on the newest version of azure-identity to solve the problem ? We would ideally want to stay up to date with the newest version of azure-identity

@ezpzzz19 The latest version of azure-identity already requires the version of azure-core which contains the necessary imports. You shouldn't be encountering this issue if you are installing normally through pip as the dependency resolution will ensure you have compatible versions of azure-core and azure-identity.
If you are seeing this issue outside of the context of Azure CLI or the Azure CLI ML extension, then please open another issue.

The mentioned issue https://github.com/Azure/azure-sdk-for-python/issues/37790 has two fixes that work for me:

Pin azure-identity to 1.17.1 and azure-core to <1.31.0 or
Disable Application Insights for the web app:
Application Insights [Preview]
Collect application monitoring data using Application Insights
--> Disable


Bump ruby supported version

From 2.4 to 2.7 to follow up with #366

Ruby 2.6, end of life: 31 Mar 2022
-- https://endoflife.date/ruby


Hotfix: moved test up to be nested under dev check

Description
Prober tests are failing on AWS staging in part because we do not have an ESRI client and this test expects one. Moving this test up under the isDevelopment check makes it so that this test will not run on staging.

resolved by https://github.com/civiform/civiform/pull/4411


Cells become invisible

Environment data

VS Code version: 1.75.0
Jupyter Extension version (available under the Extensions sidebar): v2023.1.2010391206
Python Extension version (available under the Extensions sidebar): v2023.2.0
SSH to Fedora machine from Windows

Cells become invisible and unclickable and only become available again after collapsing them.

This is fixed now in latest Insiders and will be available in next Stable recovery build.

When is the next stable recovery build and how can i get my hands on it?


enhancement cast expression to the type with parameters

Proposed changes
Issue Number: close #xxx
Problem summary
Describe your changes.
Checklist(Required)

Does it affect the original behavior:

[ ] Yes
[ ] No
[ ] I don't know

Has unit tests been added:

[ ] Yes
[ ] No
[ ] No Need

Has document been added or modified:

[ ] Yes
[ ] No
[ ] No Need

Does it need to update dependencies:

[ ] Yes
[ ] No

Are there any changes that cannot be rolled back:

[ ] Yes (If Yes, please explain WHY)
[ ] No

Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 34.96 seconds
load time: 440 seconds
storage size:<PHONE_NUMBER>1 Bytes
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20221128172442_clickbench_pr_54623.html


Ruleset suggestions

There are certain ESLint equivalent rules that can be replaced by @typescript-eslint equivalent. A good reference can be found in @typescript-eslint/recommended. They are no-use-before-define, no-empty-function.
Also certain rules can be turned off for TypeScript files as they would be checked by TypeScript compiler. A good reference would be @typescript-eslint/eslint-recommended.

Hey @deskoh
This config was using a slightly older version of @typescript-eslint without those rules you mentioned.
Have just updated to the latest version of @typescript-eslint in #17. This PR includes the rules you mentioned.
Great catch and very thorough, please let me know if you find any other missing rules now 👍

Good suggestion about disabling rules checked by TSC.
As eslint-config-airbnb-typescript is configured to work for both js and ts files, this would require using the more complex overrides syntax, e.g.:
overrides: [
  {
    files: ['*.ts', '*.tsx'],
    rules: {
      'example-rule-checked-by-tsc': 'off',
    },
  },
]

However this makes it difficult to make local overrides, an issue discovered previously in #4.
I would also prefer to keep eslint-config-airbnb-typescript closely mirroring eslint-config-airbnb.
You can choose to opt-in to using that config with something like:
{
  "extends": [
    "airbnb-typescript",
    "plugin:@typescript-eslint/eslint-recommended",
    "plugin:@typescript-eslint/recommended"
  ]
}

Hey @deskoh
eslint-config-airbnb-typescript was using a slightly older version of @typescript-eslint, and those roles did not exist.
Have just updated to the latest version of @typescript-eslint in #17 and added those rules you've mentioned.
Great catch and very thorough checking, thank you! Please let me know if you find any other missing rules now 👍

Great update!
Consider the following:

@typescript/no-extra-parens is unnecessary? (See commit https://github.com/airbnb/javascript/commit/56a524a47ed0b71f348d6bfaa42289d562f6bc5b for details).

The same goes for @typescript/no-magic-numbers as the rule is disabled.

quotes can be replaced by @typescript-eslint/quotes.

I wrote the following script to help catch the above 😄
https://github.com/deskoh/typescript-eslint-util

Thanks @deskoh !


update BOEM Wind Planning Areas

from @evsatt:

Hey Taylor and Ben,
I was just looking at this paper and wondering if we have the most up to date BOEM spatial information for the wind call energy areas in our app and in the map? We have 3 polygons whereas this wind energy area map has 2. Any thoughts on this?
Thanks!
Erin

Location of wind energy areas of interest along the California coast,... | Download Scientific Diagram

Yup, thanks @evsatt, I was recently playing with the latest BOEM leases and planning areas here:

marinesensitivity.org/workflows: explore_blocks

I'll update the oceano app accordingly:

Oceano App | CalCOFI.io

I'll actually update the app first via cc_places:

CalCOFI Places — cc_places • calcofi4r

/data-raw/cc_places.R


Flatpak support

Changes:

adds support for flatpak's ZAP
moved addonupdate to additionalOptions. (configVersion , converter & pytest updated)
addonupdate is now enabled by default
Moved the old-schemas to the tests/ directory
additionalOptions.enableUI to enable UI (remove the -cmd option, wont work in podman)

This is currently a proposal. For example :

additionalOptions feels like a long string, which is a little annoying to handle in the code, but I didn't think of something shorter
is it okay to move addonupdate & have to increment the configVersion number ? (or should we leave it as previously to avoid the version change)
addonupdate is now enabled by default, which I think is what we wanted to do. is it ?

additionalOptions vs miscOptions vs otherOptions, which would you choose? :)

is it okay to move addonupdate & have to increment the configVersion number ?
yes, as long as revision '1' => '3' migration can be done without an issue.

addonupdate is now enabled by default, which I think is what we wanted to do. is it ?
yes.

miscOptions is better, indeed.
I also moved it to the bottom in the template-long

revision '1' => '3' migration should work without any issues, although there's no pytest for that (only 1 pytest from 1 to 2, and another for 2 to 3, which I think should be sufficient)


Release version 1.6.1

Codecov Report
Attention: 28 lines in your changes are missing coverage. Please review.

Comparison is base (0c8949a) 83.03% compared to head (175c565) 82.69%.
Report is 6 commits behind head on main.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #284      +/-   ##
==========================================
- Coverage   83.03%   82.69%   -0.35%
==========================================
  Files          48       48
  Lines        3761     3733      -28
==========================================
- Hits         3123     3087      -36
- Misses        638      646       +8

Files
Coverage Δ

eogrow/core/area/base.py
100.00% <100.00%> (+2.29%)
:arrow_up:

eogrow/core/area/batch.py
66.07% <100.00%> (-1.73%)
:arrow_down:

eogrow/core/area/utm.py
100.00% <100.00%> (ø)

eogrow/core/config.py
92.78% <100.00%> (+0.05%)
:arrow_up:

eogrow/core/storage.py
100.00% <100.00%> (ø)

eogrow/pipelines/batch_to_eopatch.py
88.70% <100.00%> (-0.10%)
:arrow_down:

eogrow/pipelines/features.py
95.78% <ø> (ø)

eogrow/pipelines/import_tiff.py
98.07% <ø> (ø)

eogrow/pipelines/import_vector.py
100.00% <ø> (ø)

eogrow/pipelines/prediction.py
89.02% <ø> (ø)

... and 12 more

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Changing an agent logon from the Services snap-in

Under Run as a service, there's a notice that says

If you need to change the agent's logon account, don't do it from the Services snap-in. Instead, see the information below to re-configure the agent.

I haven't been able to find any further information on this stipulation. I have the following questions:

Why shouldn't I use the Services snap-in to change an agent's logon account?
What if I already did?
Does this information apply to YAML pipeline agents (configured as VM resources in Environments)? Or only to deployment groups for classic releases?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 425b68d1-4ba0-5657-4e09-f3aac361ea09
Version Independent ID: f8044e9d-98e2-6e00-b513-a91e772b2ef8
Content: Deploy an Azure Pipelines agent on Windows - Azure Pipelines
Content Source: docs/pipelines/agents/v2-windows.md
Product: devops
Technology: devops-cicd-agents
GitHub Login: @steved0x
Microsoft Alias: sdanie

@isaaclyman -- Isaac, thank you for your feedback and questions.  Please consider these resources:

Azure DevOps Services
Azure DevOps on Stack Overflow

@steved0x -- Steve, please look into this issue.


All metadata should be in the metadata step of the add event wizard

The “source” step of the add event wizard also contains metadata. This doesn't really make sense since there is a whole step dedicated to metadata. We should move “Start date” in there.

The problem: This field should only be shown for a file upload. Not for a scheduled event.
The solution: Make “Source” step 1 and metadata step 2. We then know if we need a start date once we are in the metadata tab.

This fixes #487

Two other fields missing are "location" and "duration" (used for calculating the end date in the Admin UI). I set this for file uploads if I know these files are from a recording (e.g. CA did the recording, but editing was done locally). Is this in the scope of this PR, or should I create another issue?

You mean, you want these two additional fields in the recording metadata section? I think that's unrelated enough to not wait for that when merging this pull request. Nevertheless, I'll try taking a peek at the issue since I now kind of know the code and I hope it's not too hard to do.

You mean, you want these two additional fields in the recording metadata section? I think that's unrelated enough to not wait for that when merging this pull request. Nevertheless, I'll try taking a peek at the issue since I now kind of know the code and I hope it's not too hard to do.

Fair. I created issue #742 to track this.

Fixed the minor issues and merged the different metadata sections. Now they shouldn't seem like different catalogs any longer.

Now they shouldn't seem like different catalogs any longer.

Can confirm, looks good to me now.
I was going to complain that we should need extra code for rendering start_date, as it is part of the standard metadata catalog and therefore should be able to be rendered just like any other metadata field. But it turns the reason we cannot do that is ultimately the backends fault, as the relevant endpoint (admin-ng/event/new/metadata) explicitly excludes start_date (among other fields). Whether or not this is reasonable likely requires additional consideration, so I'm ok with leaving this as it is for now.

Rebased to resolve merge conflicts and fixed the next/previous problem.

Rebased to fix resolve conflicts.

The validation for the main metadata is not happening anymore.

Fixed.

Can confirm, validation is happening again.
But the "Previous" button on the metadata tab does nothing again. Seems to only occur if I want to schedule an event.
Here is a video of me clicking around:
Bildschirmaufzeichnung vom 2024-07-26, 14-48-04.webm