Ethereum ecrecover support

According to this commit, an initial secp256k1 support has been added to plutus-tx recently. I was wondering if there is going to be an implementation of Ethereum "ecrecover" any time soon since we need it in a project we are currently working on and the current secp256k1 builtins in plutus-tx will not be enough.

What do you need exactly and whi can't you implement it yourself?

What do you need exactly and whi can't you implement it yourself?

I need to implement the verifySignature function defined here. The implementation is straightforward except for the last step which uses the ecrecover function. I have implemented ecrecover here which depends on below functions from libsecp256k1 (a library that cardano-base also depends on):

secp256k1_ecdsa_recoverable_signature_parse_compact
secp256k1_ecdsa_recover
secp256k1_ec_pubkey_serialize

The point is that I'm not able to use my implementation directly inside the on-chain part of a plutus-contract since the way I have implemented it and the dependencies I have used, may violate the "INLINABLE" condition. So before starting to modify the implemented function, I had to make sure whether there are any plans to add more utilites to plutus-tx builtins that depend on libsecp256k1 including ecrecover implementation and If not, perhaps the first step would be importing the three functions mentioned above in here.

We won't be adding additional builtins in Vasil. You can make a CIP to propose adding additional ones, see:

The process CIP: https://cips.cardano.org/cips/cip35/
The CIP for the current SECP functions: https://github.com/cardano-foundation/CIPs/pull/250

Such requests should go through the CIP route, hence I'm closing the issue.


Improve tab expansion to handle params for push.

Fix #234  BTW if we decide this is an OK fix (need another set of eyes on my $gitParams regex), I'd like to review the fact that all these switch regex case statements fallthrough to the next (observed while debugging tab expansion).  I don't believe that is necessary.  I'm thinking that most (all) of these should have a break statement.
Started a minimum set of Pester tests for tab completion.  Would be good to flesh these out over time.
BTW the latest PowerShell extension for VS Code release supports "attach to PS host process" which makes debugging tab expansion a breeze.  VS Code FTW!  :-)

@dahlbyk I don't have a good test case for the remote refspec (with : in it).  Can you provide me with such a refspec that returns results?

Stretch goal, since you're already up to your neck in this

Oh sure, goad me on.  :-)  OK last commit addresses this I think.

Stretch goal, since you're already up to your neck in this

Oh sure, goad me on. :-) OK last commit addresses this I think.

Not quite, since ref:branch can be mixed with other forms. I added/adjusted the tests and made the regex dumber between the remote and force/ref, so I think we're good.

Yeah, getting into usages of git I'm not so familiar with.  Well, you've fixed it.  Want me to rebase or merge as is (didn't see any fixup)?

Nevermind, there is a fixup.

@dahlbyk FYI I rebased -i --autosquash this branch and pushed with --force-with-lease in case you want to make more changes.  Otherwise, I'm feeling pretty good about merging this in.


how to require file in plugin init.lua

Please describe the problem you're trying to solve
I want split my plugin file, and use require to connect them. but I can't find a way to make it work.
Describe the solution you'd like
maybe just make require relative path work or provide some local variable as current plugin path. then I can use it to load other file...

You need to place it in the plugins directory, and it has to be a directory containing an init.lua file to work as a Lua module, see https://yazi-rs.github.io/docs/plugins/overview


chore: package updates

:---:|
🧰 Changes
Updates various packaegs including @readme/emojis to v4.0.0
🧬 QA & Testing

Working in this PR app.

This PR was released!
🚀 Changes included in v6.31.0


PermissionStatus.Unknown

Which Plugin does this impact: Xamarin.Plugins.Permissions
Bug
Version Number of Plugin: 1.1.7
Device Tested On: Nexus 7 (Android 6) and Oppo R5 (CM 13)
Simulator Tested On:
Expected Behavior
When callingawait CrossPermissions.Current.RequestPermissionsAsync(new[] { Permission.Storage });a system dialog is shown when the user can grant/deny the requested permission
Actual Behavior
No dialog is shown and the status of the requested Permission is PermissionStatus.Unknown
Steps to reproduce the Behavior
Sample code:
 private async Task<bool> CheckExternalStoragePermission()
        {
            var externalStorageStatus = false;
            try
            {
                var status = await CrossPermissions.Current.CheckPermissionStatusAsync(Permission.Storage);
                if (status != PermissionStatus.Granted)
                {
                    if (await CrossPermissions.Current.ShouldShowRequestPermissionRationaleAsync(Permission.Storage))
                    {
                        await Mvx.Resolve<IUserDialogs>().AlertAsync(Catalog.GetString("We need permission to store the downloaded files"),
                            Catalog.GetString("Permission to access storage"));
                    }

                    var results = await CrossPermissions.Current.RequestPermissionsAsync(new[] { Permission.Storage });
                    status = results[Permission.Storage];
                }

                if (status == PermissionStatus.Granted)
                {
                    externalStorageStatus = true;
                }
                else if (status != PermissionStatus.Unknown)
                {

                    await Mvx.Resolve<IUserDialogs>().AlertAsync(Catalog.GetString("Cannot continue sharing."), Catalog.GetString("Storage Permission Denied"));
                }
            }
            catch (Exception ex)
            {
                Debug.WriteLine("Exception while requesting Storage permission: "+ex);
                externalStorageStatus = false;
            }

            return externalStorageStatus;
        }

I acidentally deleted the MainApplication.cs file that the CurrentActivity plugin includes


Add Instance claim kind

Description of your changes

Adds a VM claim kind and a corresponding default class controller such that infrastructure stacks can add VM resources that can be dynamically provisioned. Because VM configuration can vary greatly across cloud providers (or even within a cloud provider), this initial claim implementation does not surface any configuration details. It is possible we would want OS or some other field surfaced in the future.
Checklist

I have:

[x] Run make reviewable to ensure this PR is ready for review.
[x] Ensured this PR contains a neat, self documenting set of commits.
[x] Updated any relevant documentation, examples, or release notes.
[x] Updated the RBAC permissions in clusterrole.yaml to include any new types.

@muvaf I agree that those are general enough that we should consider supporting. Would be happy to add if all are in agreement. cc @negz @jbw976

@hasheddan Do you have an idea of how far off any Stacks (e.g. Packet) are from using this claim? I'm a little hesitant to ship a new claim kind until at least one Stack actually makes use of it.

@negz official work on Packet stack starts in earnest tomorrow. It might be nice to have a claim type to be able to test with. Still think we need @prasek and @bassam thoughts here.

Closed in favor of #942


Fixing the Codecov coverage file

Description of Changes
Swapping the file we push to codecov so it reports accurate coverage stats
Related Issues

Codecov Report

Merging #11 (265c1c2) into main (d7e4342) will increase coverage by 8.82%.
The diff coverage is n/a.

:exclamation: Current head 265c1c2 differs from pull request most recent head 48f3ecf. Consider uploading reports for the commit 48f3ecf to get more accurate results

@@             Coverage Diff             @@
##             main       #11      +/-   ##
===========================================
+ Coverage   91.17%   100.00%   +8.82%
===========================================
  Files          13        13
  Lines          68        70       +2
  Branches        6         7       +1
===========================================
+ Hits           62        70       +8
+ Partials        6         0       -6

Impacted Files
Coverage Δ

src/axios.ts
100.00% <0.00%> (+14.28%)
:arrow_up:

src/events/types/EventStatus.ts
100.00% <0.00%> (+14.28%)
:arrow_up:

src/config.ts
100.00% <0.00%> (+25.00%)
:arrow_up:

src/events/getManyEvents.ts
100.00% <0.00%> (+25.00%)
:arrow_up:

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


[Question] How should I use Exit Animations

Hi,
I have a qeustion about animations for animate.css. There are some Exit Animations, but they are pretty much useless. How should I or can I use them? I had the idea to exit out animations when going to a next page, but I have no JavaScript/JQuery skills to do that. Thanks.

Hey there, then the best thing to do is to learn JS if you want to use those exit animation. They are not useless it depends on the user who know how to use them based on their experience in development. My suggestion learn JS and start learning how to use these exit animation.

So Exit Animations only can be used using JS? Can you give me one example?

Yeah because all the animation is only working when the page loads. So if you want to trigger an exit animation you need a javascript for the user experience. I will provide a quick example later.

May I close this as it's kinda solved? @WarenGonzaga  and @ThaumicTom

An example is still missing as @WarenGonzaga wanted to give me one, but I think it's done.

Oh, I forget to provide example sorry guys!


Add config option for LWIP TCP_TMR_INTERVAL (IDFGH-1040)

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

@JosuGZ Can you submit your changes against to the master branch?

@JosuGZ Can you submit your changes against to the master branch?

I have modified it here.


update oauth flow for Gitlab 15 compatibility

The PR contains two changes;

OAuth implicit grant flow is being demised
OAuth refresh token flow is being implemented

/easycla


Javalin introduction video

Thanks for doing this @Schachte. There's a pretty handy introduction to Javalin in Java magazine: http://www.javamagazine.mozaicreader.com/MarchApril2019#&pageSet=13&page=0
If you have any questions or need any other assistance at all, please let me know.

@Schachte have you started on this? If not, it might be relevant to you that the first alpha version of Javalin 3 is out: https://javalin.io/future/documentation

@tipsy Haven't started on this yet, but I'll update when I get to that point. Been heavily busy at work, so haven't had free time to dive into this project yet.

No worries, just thought I'd give you a heads up about the new version!

Hi. Is this issue open for anyone? Would you prefer a video walkthrough of what Javalin is, or an in-dept article that walks you through Javalin's core functionalities?

@jjjimenez100 are you asking if you can create a video? If so, yes.

I’m working on a video series about Javalin (and various other tools): https://www.jhu-oose.com/todoose/

That's really cool @leafac, will check it out later.

Sure, I think maybe it would be a good fit for https://javalin.io/tutorials/

@Schachte Are you still planning on doing this? It's completely fine if you decided not too, but I'd like to close the issue if that's the case.


Update to bevy 0.15

These should update to bevy 0.15

Now I'm going to work on porting the semantics to using required components

Hey @Dimchikkk I would like a place to discuss some design changes I think would benefit this project for my own use cases, but there is no thread on the bevy discord and your discord account isn't accepting messages from non-friends
Can you enable the github discussions board? Or, actually, I'll write in the issue

OK, at this point there are only a few outstanding points to me:

No 3D support
This random bug with ui rendering where it starts squished
When bevy_editor_pls gets updated to bevy 0.15 I'll be able to quickly fix the squished ui bug, the 3D support I'll seperate into a different PR because I don't want that to hold back merging bevy 0.15 support

Please give me feedback if the direction the API has gone in all of my major changes is OK! The only type I'm still not sure about is CosmicEditor:

Is the CosmicEditor component an optional extra? Only for focussed CosmicEditBuffers? IDK and I think I would understand this project better if I did know

When bevy_editor_pls gets updated to bevy 0.15 I'll be able to quickly fix the squished ui bug

could you elaborate pls? I don't see how bevy_editor_pls is related to the bug :)

OK @Dimchikkk I've fixed the final UI bug
I have quite a few more improvements, esspecially 3D, that I'll prepare in a seperate PR for. Currently, I have two patch.crates-io entries for dev-dependencies that are just early access to bevy 0.15 supporting libraries (for bevy_editor_pls and transform-gizmo). bevy_editor_pls helped me find the bug (no correctly handling the redraw flag on cosmic_text::Buffer/Editor)
All in all, this can be published

Thank you for your work 👍

error: failed to prepare local package for uploading
Caused by:
failed to select a version for the requirement bevy_editor_pls = "^0.11.0"
candidate versions found which didn't match: 0.10.0, 0.9.0, 0.8.1, ...
location searched: crates.io index
required by package bevy_cosmic_edit v0.26.0

Failed during publishing :/

Can we remove TODO.md and bevy_editor_pls from dev deps?

Oh yeah, though its already merged
I had those in for testing purposes

If you can open new PR pls


Find and remove buckets script failing

Regarding the merging of this PR, I am seeing the Find and remove buckets job failing. The last time I ran into this issue, I re-ran the failed job and it seemed to have resolved itself, but it didn't seem to work this time. Making this issue now because it's happened more than once.
Log error snippet:
delete: s3://pulumi-hugo-origin-pr-3554-87459bd3/yaml/index.html
remove_bucket: pulumi-hugo-origin-pr-3554-87459bd3

An error occurred (ParameterNotFound) when calling the DeleteParameter operation:
make: *** [Makefile:54: ci-pull-request-closed] Error 254
Error: Process completed with exit code 2.

Full log file

@cnunciato Can you help me look into this?

Engin ran into this bug today


Cant connect to DB from host

Hi all,
I have built the docker image, ran it, and connect to it to verify it works. I can use psql in the instance shell and create database, etc. That all seems to work fine.
What I cant do is connect from my Mac host to the running container DB. I have used the IP from docker inspect, the -p 5432:5432 is in my run command, etc. In fact, I cant even ping the container ip listed in inspect.
Ultimately I want to a) connect via my host to the DB and b) have a separate container connect to the DB container.
I am using the latest native docker on Mac now, no longer boot2docker. Is there something special needed to make this work?

I think some of the parameters changed somewhere along the way
docker run --name NAME --expose 5432 -p 5432:5432 -e POSTGRES_USER=dbuser -e POSTGRES_DB=dbname -e POSTGRES_PASSWORD=password -d postgres

This worked for me - note that I used a different port number because I have postgres running locally on my Mac:
docker run -it --rm -p 5433:5432 -e POSTGRES_PASSWORD=12345 -e POSTGRES_USER=bob postgres:latest
psql -p 5433 -h localhost -U postgres
Also, problems can crop up for other reasons such a volume hanging around - see e.g. https://github.com/docker-library/postgres/issues/203#issuecomment-255200501

Looks like this issue was resolved so I'm going to prune.
If you believe this to be in error then let me know and I'll re-open it


Can this be used in production?

I know from the versioning that it's not a stable release. But is someone using it in production right now? Is it advisable to do so, or should I shift to CRA for now?

What is CRA.


Fix Marker position while zooming

If a marker is set to a new latlng while zooming the marker jumps around. The problem is that the this._pixelOrigin (which is used for the calculation of the marker position) is only set after the zoom (_onZoomTransitionEnd), so while zooming the old pixelOrigin is used and the marker jumps.
I tried to create a test but it was not possible.

Fixes #7017
Fixes #5181
Fixes #5505
Fixes #5994 (It describes exactly that issue but I'm not abel to reproduce it with the demo)

New src: link

Fixed:

@Falke-Design In 1.8 leaflet version this problem occurs with permanent tooltip bound to that marker.


Produce and consume records

This PR adds a produce and cosume command to a kafka topic. Flags like the partition, key, limit, and timestamp are optional. This is useful for debugging purposes and currently you need another thrid-party kafka cli for this feature.
Verification Steps

Do x
Do y

Type of change

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] Documentation change
[ ] Other (please specify)

Just to comment on the format of the timestamps...

Format
Value
Worked?

UTC
06/14/2022 @ 2:13pm
No

ISO 8601
2022-06-14T14:13:31+00:00
Yes

RFC 822
Tue, 14 Jun 2022 14:13:31 +0000
No

RFC 2822
Tuesday, 14-Jun-22 14:13:31 UTC
No

Unix timestamp
1655216639
No

If I had to guess ISO is the default and unix could work if I was able to set the timestamp type(currently cannot do this through go-sdk)

I guess we can send this as email for feedback to the admin backend so we can add better sdk docs for it

@wtrocki basic functionality is in just for consuming and producing, tried to stick to design guidelines but I may have missed something

make lint and make format required

@wtrocki thoughts?

That is why we have offset argument set in the consume api. We can control offset :)

Oh okay that makes sense I didn't think of this...

make lint and make format required

rhoas kafka topic examples can be added as well to provide comprehensive documentation for this

Very good job. Not sure about what we can do with table format. I will do more non happy path testing once build will be fixed

Table not working is not that bad anyway, there was some issues with it anyway that I couldn't get around when looking at the library we use so not using make it easier.

As in the design doc. Let's drop our normal output format and provide custom output formats for that command specicically.
Many people would probably want to see key value format/json etc.

custom output formats

Do you mean outputting the messages in a way that only is done for this command? Not using dump

Do you mean outputting the messages in a way that only is done for this command? Not using dump

exactly. It can be some --output-format flag or anything you like to not confuse with general cli -o flag

Left basic comments. I could not verify commands:
[wtrocki@graphapi app-services-cli (produce-consume)]$ rhoas kafka topic produce --name=missing-topic
Missing info that we are waiting for user input. Missing validation on non existent topic.
[wtrocki@graphapi app-services-cli (produce-consume)]$ rhoas kafka topic produce --name=test --file connector.json
❌ 404 Not Found. Run the command in verbose mode using the -v flag to see more information
No API error handling present.

JSON format is unusable. Value needs to be escaped thus it is hard to read.
I do not think we can do anything else than using key, value format as default instead.
[wtrocki@graphapi app-services-cli (produce-consume)]$ rhoas kafka topic consume --name=test
{
"topic": "test",
"key": "",
"value": "{\n  "channel": "stable",\n  "connector": {\n    "kafka_topic": "lb-cos",\n    "log_show_all": true\n  },\n  "connector_type_id": "log_sink_0.1",\n  "id": "c97gajsotd37bufhphu0",\n  "kafka": {\n    "id": "c52ajhf2ek06ri44ife0",\n    "url": "openbridge-c--ajhf-ek--ri--ifea.bf2.kafka.rhcloud.com:443"\n  },\n  "kind": "Connector",\n  "modified_at": "2022-04-07T15:37:51.612514Z",\n  "name": "cos-wtrocki-logs",\n  "namespace_id": "c9f8i3njogcbrlqqlt90",\n  "owner": "wtrocki_kafka_supporting",\n  "schema_registry": {\n    "id": "",\n    "url": ""\n  },\n  "service_account": {\n    "client_id": "srvc-acct-3de38d77-90ff-4480-9a0f-8a8230ccff23",\n    "client_secret": "srvc-acct-3de38d77-90ff-4480-9a0f-8a8230ccff23"\n  }\n}",
"partition": 0,
"offset": 0
}

Key value format seems to be missformatted. Some messages with key will get formatted differently than others:
{
  "schema_registry": {
    "id": "",
    "url": ""
  },
  "service_account": {
    "client_id": "srvc-acct-3de38d77-90ff-4480-9a0f-8a8230ccff23",
    "client_secret": "srvc-acct-3de38d77-90ff-4480-9a0f-8a8230ccff23"
  }
}

Key: test
Value: {
"channel": "stable",
"connector": {
"kafka_topic": "lb-cos",
"log_show_all": true
},
"connector_type_id": "log_sink_0.1",
"id": "c97gajsotd37bufhphu0",
"kafka": {
"id": "c52ajhf2ek06ri44ife0",
"url": "openbridge-c--ajhf-ek--ri--ifea.bf2.kafka.rhcloud.com:443"
},
"kind": "Connector",
"modified_at": "2022-04-07T15:37:51.612514Z",
"name": "cos-wtrocki-logs",
"namespace_id": "c9f8i3njogcbrlqqlt90",
"owner": "wtrocki_kafka_supporting",
"schema_registry": {
"id": "",

I think that we would need always print Key, Message or Message: format to visibly show end of the messages.

Some valid cases return 400 for me:
[wtrocki@graphapi app-services-cli (produce-consume)]$ rhoas kafka topic consume --name=test --partition=1 --offset=1
❌ 400 Bad Request. Run the command in verbose mode using the -v flag to see more information

Some valid cases return 400 for me

Was not able to replicate this so will wait for feedback from others to find the issue

Was not able to replicate this so will wait for feedback from others to find the issue

Will check now

It works now (throws validation error)

Used https://awesomeopensource.com/project/jdorfman/awesome-json-datasets
Tested --wait use case. Tested normal produce consume. This looks very good to be shared. It is just matter of figuring out dates and examples

Message: Binary or non-UTF-8 encoded data cannot be displayed

Nice!

@wtrocki added more examples and made commands hidden, will I squash and merge this now?

Every new change - new review and verification. doing it now

We need more examples for typical use cases:

getting message at offset
jq some part of the message stream only
getting info about current offset
adding same examples to topic root


Test website pr

This PR modifies the test_website PR to make sure only the section labelled with "The task" is stored in the task_info.json and excludes the heading itself.

I'd rather not do it this way. The The metrics section is being removed anyway in #810. If we just remove the The task headings then this achieves the same goal.

No longer needed


Add multithreading support to OCR

This adds support for running the OCR multithreaded to improve the
performance of the output.
Signed-off-by: Ethan Dye<EMAIL_ADDRESS>

And another exciting one @timj, this one should help with a lot of the slowdown, to be clear, my decoder is still slow, but the really slow part is now multithreaded. Let me know if this causes any breaking issues for you.


[QA] Avatar positition stutters when moving on platforms

The avatar shakes up and down when displacing on moving platforms as resolves collisions. Is probable to fall through the platform if the speed is too high.
The character should be parented to the platforms to inherit position?

This issue is still happening:
https://github.com/decentraland/unity-explorer/assets/146749313/89e4a554-619e-4a45-beaf-66b113951d54


App crashing with ConcurrentModificationException in Mixpanel SDK code

Hey,
We are seeing crashes in firebase crashlytics that originate in Mixpanel SDK code.
We use version 5.8.4.
This is the relevant part of the stacktrace:
Fatal Exception: java.util.ConcurrentModificationException
       at java.util.LinkedHashMap$LinkedHashIterator.nextNode(LinkedHashMap.java:757)
       at java.util.LinkedHashMap$LinkedKeyIterator.next(LinkedHashMap.java:780)
       at com.mixpanel.android.mpmetrics.MixpanelAPI.track(MixpanelAPI.java:2885)
       at com.mixpanel.android.mpmetrics.MixpanelAPI.track(MixpanelAPI.java:868)

Could you please take a look?
Thanks in advance!

Hi @ori-6over6 , did you track any properties on the track call? Is there any possibility you modify the properties on another thread during the track call?

I’m closing this issue because it has been inactive for a few months. Please reopen if you still encounter this issue


quic: gquic tests for cyu hash and alerts

pcaps taken from this redmine attachement:
https://redmine.openinfosecfoundation.org/issues/3440#note-8
Previous PR https://github.com/OISF/suricata-verify/pull/368

Cf https://github.com/OISF/suricata/pull/6334

Already merged in https://github.com/OISF/suricata-verify/pull/671 it seems


Added cartographer test library to CMake project

Fake, mock and test helpers are now compiled into a test library.
This should resolve #1128. The bazel build file was updated in #1101.

@wally-the-cartographer merge

Merge requested by authorized user schwoere. Merge queue now has a length of 1.

Thanks for the quick fix!

@schwoere PTAL. It seems like your forgot the follow up change in cartographer_ros.
https://travis-ci.org/googlecartographer/cartographer_ros/builds/373162785?utm_source=email&utm_medium=notification

@kdaun Thanks for letting me know. I fixed the cmake file and cartographer_ros is building now again.


Terraform apply executor job didn't exit after it's finished

✗ k get job
NAME                 COMPLETIONS   DURATION   AGE
aliyun-oss-destroy   0/1           63s        63s

TTL controller is a Kubernetes v1.21 [beta] feature, not enabled in v1.18 Kubernetes.
https://kubernetes.io/docs/concepts/workloads/controllers/ttlafterfinished/


AV1 failing in FFMPEG

Checklist

[ ] I'm reporting a broken site support issue
[X] I've verified that I'm running youtube-dl version 2019.10.22
[X] I've checked that all provided URLs are alive and playable in a browser
[X] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[X] I've searched the bugtracker for similar bug reports including closed ones
[X] I've read bugs section in FAQ

Verbose log

youtube-dl "https://www.youtube.com/watch?v=pBuUkd5uakk" --merge-output-format webm -v
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'https://www.youtube.com/watch?v=pBuUkd5uakk', u'--merge-output-format', u'webm', u'-v']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2019.10.22
[debug] Python version 2.7.16 (CPython) - Darwin-18.6.0-x86_64-i386-64bit
[debug] exe versions: ffmpeg 4.2.1, ffprobe 4.2.1, rtmpdump 2.4
[debug] Proxy map: {}
[youtube] pBuUkd5uakk: Downloading webpage
[youtube] pBuUkd5uakk: Downloading video info webpage
[debug] Default format spec: bestvideo+bestaudio/best
[debug] Invoking downloader on u'https://r3---sn-5ualdnl7.googlevideo.com/videoplayback?expire=1571773653&ei=dQivXb7FHbWdlAPgwJyIBQ&ip=<IP_ADDRESS>&id=o-AAws99svgOPa3Gs9RBkXugRDaiqWmwNFGVL4EDa0MnQG&itag=137&aitags=133%2C134%2C135%2C136%2C137%2C160%2C242%2C243%2C244%2C247%2C248%2C278&source=youtube&requiressl=yes&mm=31%2C29&mn=sn-5ualdnl7%2Csn-5uaeznd7&ms=au%2Crdu&mv=m&mvi=2&pl=16&initcwndbps=755000&mime=video%2Fmp4&gir=yes&clen=171076265&dur=395.027&lmt=1553316821359253&mt=1571751960&fvip=6&keepalive=yes&fexp=23842630&c=WEB&txp=5535432&sparams=expire%2Cei%2Cip%2Cid%2Caitags%2Csource%2Crequiressl%2Cmime%2Cgir%2Cclen%2Cdur%2Clmt&sig=ALgxI2wwRAIgXMAfSN9zVOTUa-x5a-Fsd0eKXOgYBxD_QJgGVi9c-kYCIGLZokIIFfCgDpOXLi3uEF0gUegJ4yr4BJGmR1zP2j1v&lsparams=mm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps&lsig=AHylml4wRgIhANQ6dJKgwxAbhdBPtykDykJim7zpNX-eNAviwINieS8JAiEAw5df2b6YHn4Z1IJzGWsPXGjuSroGKNJkZ4h9ALqhuVg%3D&ratebypass=yes'
[download] FINAL FANTASY XIV - SHADOWBRINGERS Full Trailer-pBuUkd5uakk.f137.mp4 has already been downloaded
[download] 100% of 163.15MiB
[debug] Invoking downloader on u'https://r3---sn-5ualdnl7.googlevideo.com/videoplayback?expire=1571773653&ei=dQivXb7FHbWdlAPgwJyIBQ&ip=<IP_ADDRESS>&id=o-AAws99svgOPa3Gs9RBkXugRDaiqWmwNFGVL4EDa0MnQG&itag=251&source=youtube&requiressl=yes&mm=31%2C29&mn=sn-5ualdnl7%2Csn-5uaeznd7&ms=au%2Crdu&mv=m&mvi=2&pl=16&initcwndbps=755000&mime=audio%2Fwebm&gir=yes&clen=6077321&dur=395.041&lmt=1553319151785813&mt=1571751960&fvip=6&keepalive=yes&fexp=23842630&c=WEB&txp=5511222&sparams=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cmime%2Cgir%2Cclen%2Cdur%2Clmt&sig=ALgxI2wwRAIgMKsc37faYGJEXF_yXVeqNGL4MCqNTOMyBMgNOpqhReYCIF6RXnUacxqGYEUd19w1Wmhr8vwlm6FvU5FspId9Nuya&lsparams=mm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps&lsig=AHylml4wRgIhANQ6dJKgwxAbhdBPtykDykJim7zpNX-eNAviwINieS8JAiEAw5df2b6YHn4Z1IJzGWsPXGjuSroGKNJkZ4h9ALqhuVg%3D&ratebypass=yes'
[download] FINAL FANTASY XIV - SHADOWBRINGERS Full Trailer-pBuUkd5uakk.f251.webm has already been downloaded
[download] 100% of 5.80MiB
[ffmpeg] Merging formats into "FINAL FANTASY XIV - SHADOWBRINGERS Full Trailer-pBuUkd5uakk.webm"
[debug] ffmpeg command line: ffmpeg -y -loglevel 'repeat+info' -i 'file:FINAL FANTASY XIV - SHADOWBRINGERS Full Trailer-pBuUkd5uakk.f137.mp4' -i 'file:FINAL FANTASY XIV - SHADOWBRINGERS Full Trailer-pBuUkd5uakk.f251.webm' -c copy -map '0:v:0' -map '1:a:0' 'file:FINAL FANTASY XIV - SHADOWBRINGERS Full Trailer-pBuUkd5uakk.temp.webm'
ERROR:   Stream #1:0 -> #0:1 (copy)
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 2064, in post_process
    files_to_delete, info = pp.run(info)
  File "/usr/local/bin/youtube-dl/youtube_dl/postprocessor/ffmpeg.py", line 512, in run
    self.run_ffmpeg_multiple_files(info['__files_to_merge'], temp_filename, args)
  File "/usr/local/bin/youtube-dl/youtube_dl/postprocessor/ffmpeg.py", line 235, in run_ffmpeg_multiple_files
    raise FFmpegPostProcessorError(msg)
FFmpegPostProcessorError:   Stream #1:0 -> #0:1 (copy)

Description

I was trying to output a proper webm for use with a HTML5 video element. The mkv version is outputted and works as expected (generated by youtube-dl "https://www.youtube.com/watch?v=pBuUkd5uakk"), but trying to merge it as a webm fails despite it using av1 video & opus audio. I did manage to workaround this by using the mkv and running that through ffmpeg manually through the experimental flag by using ffmpeg directly.
FFMPEG details
ffmpeg version 4.2.1 Copyright (c) 2000-2019 the FFmpeg developers
  built with Apple clang version 11.0.0 (clang-1<IP_ADDRESS>)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/4.2.1_1 --enable-shared --enable-pthreads --enable-version3 --enable-avresample --cc=clang --host-cflags='-I/Library/Java/JavaVirtualMachines/adoptopenjdk-13.jdk/Contents/Home/include -I/Library/Java/JavaVirtualMachines/adoptopenjdk-13.jdk/Contents/Home/include/darwin -fno-stack-check' --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libbluray --enable-libmp3lame --enable-libopus --enable-librubberband --enable-libsnappy --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librtmp --enable-libspeex --enable-libsoxr --enable-videotoolbox --disable-libjack --disable-indev=jack
  libavutil      56. 31.100 / 56. 31.100
  libavcodec     58. 54.100 / 58. 54.100
  libavformat    58. 29.100 / 58. 29.100
  libavdevice    58.  8.100 / 58.  8.100
  libavfilter     7. 57.100 /  7. 57.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  5.100 /  5.  5.100
  libswresample   3.  5.100 /  3.  5.100
  libpostproc    55.  5.100 / 55.  5.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'file:FINAL FANTASY XIV - SHADOWBRINGERS Full Trailer-pBuUkd5uakk.f137.mp4':
  Metadata:
    major_brand     : dash
    minor_version   : 0
    compatible_brands: iso6avc1mp41
    creation_time   : 2019-03-23T04:50:33.000000Z
  Duration: 00:06:35.03, start: 0.000000, bitrate: 3464 kb/s
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 11 kb/s, 29.97 fps, 29.97 tbr, 30k tbn, 59.94 tbc (default)
    Metadata:
      creation_time   : 2019-03-23T04:50:33.000000Z
      handler_name    : ISO Media file produced by Google Inc. Created on: 03/22/2019.
Input #1, matroska,webm, from 'file:FINAL FANTASY XIV - SHADOWBRINGERS Full Trailer-pBuUkd5uakk.f251.webm':
  Metadata:
    encoder         : google/video-file
  Duration: 00:06:35.04, start: -0.007000, bitrate: 123 kb/s
    Stream #1:0(eng): Audio: opus, 48000 Hz, stereo, fltp (default)
[webm @ 0x7f8559004e00] Only VP8 or VP9 or AV1 video and Vorbis or Opus audio and WebVTT subtitles are supported for WebM.
Could not write header for output file #0 (incorrect codec parameters ?): Invalid argument
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #1:0 -> #0:1 (copy)

If I try to output this directly as an mp4 without manually using ffmpeg I get this error from the command generated by youtube-dl.
Command:
 youtube-dl "https://www.youtube.com/watch?v=pBuUkd5uakk" --merge-output-format mp4 -v
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'https://www.youtube.com/watch?v=pBuUkd5uakk', u'--merge-output-format', u'mp4', u'-v']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2019.10.22
[debug] Python version 2.7.16 (CPython) - Darwin-18.6.0-x86_64-i386-64bit
[debug] exe versions: ffmpeg 4.2.1, ffprobe 4.2.1, rtmpdump 2.4
[debug] Proxy map: {}
[youtube] pBuUkd5uakk: Downloading webpage
[youtube] pBuUkd5uakk: Downloading video info webpage
[debug] Default format spec: bestvideo+bestaudio/best
[debug] Invoking downloader on u'https://r3---sn-5ualdnl7.googlevideo.com/videoplayback?expire=1571774757&ei=xQyvXdz_KsPb4QTcgZLIBA&ip=<IP_ADDRESS>&id=o-AKo4t2IRRwn-Xr_YHgFbj2J0Li0Zy_4f_HDKl8PGwbmA&itag=137&aitags=133%2C134%2C135%2C136%2C137%2C160%2C242%2C243%2C244%2C247%2C248%2C278&source=youtube&requiressl=yes&mm=31%2C26&mn=sn-5ualdnl7%2Csn-q4flrnee&ms=au%2Conr&mv=m&mvi=2&pl=16&initcwndbps=858750&mime=video%2Fmp4&gir=yes&clen=171076265&dur=395.027&lmt=1553316821359253&mt=1571753049&fvip=3&keepalive=yes&fexp=23842630&c=WEB&txp=5535432&sparams=expire%2Cei%2Cip%2Cid%2Caitags%2Csource%2Crequiressl%2Cmime%2Cgir%2Cclen%2Cdur%2Clmt&sig=ALgxI2wwRQIhAKvUnjO85GDpdHLuO5mvVunUh3k_b68qEzey0nL2BgZXAiBqQ_h5X2k5GXMYZ5avZwpkgJYiEvSCUoCJOSZXFvEj-A%3D%3D&lsparams=mm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps&lsig=AHylml4wRgIhAMFIZKxSQxBefMh-xFwKm3x3zKXCp-zbwpac9RdPBVlWAiEAymyXhDI9OnoqOMRLuszpV6uU4E0MDXq-Np6ZrgP7nTE%3D&ratebypass=yes'
[download] FINAL FANTASY XIV - SHADOWBRINGERS Full Trailer-pBuUkd5uakk.f137.mp4 has already been downloaded
[download] 100% of 163.15MiB
[debug] Invoking downloader on u'https://r3---sn-5ualdnl7.googlevideo.com/videoplayback?expire=1571774757&ei=xQyvXdz_KsPb4QTcgZLIBA&ip=<IP_ADDRESS>&id=o-AKo4t2IRRwn-Xr_YHgFbj2J0Li0Zy_4f_HDKl8PGwbmA&itag=251&source=youtube&requiressl=yes&mm=31%2C26&mn=sn-5ualdnl7%2Csn-q4flrnee&ms=au%2Conr&mv=m&mvi=2&pl=16&initcwndbps=858750&mime=audio%2Fwebm&gir=yes&clen=6077321&dur=395.041&lmt=1553319151785813&mt=1571753049&fvip=3&keepalive=yes&fexp=23842630&c=WEB&txp=5511222&sparams=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cmime%2Cgir%2Cclen%2Cdur%2Clmt&sig=ALgxI2wwRAIgBT2MM_J4Dj0ojecuSBE4pv8nL6fEiQzFoCwQ-hHs4V0CIDFGwleUIZi5GAiLs1PUCfUvLADL8-pVlPInN-kRhO8S&lsparams=mm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps&lsig=AHylml4wRgIhAMFIZKxSQxBefMh-xFwKm3x3zKXCp-zbwpac9RdPBVlWAiEAymyXhDI9OnoqOMRLuszpV6uU4E0MDXq-Np6ZrgP7nTE%3D&ratebypass=yes'
[download] FINAL FANTASY XIV - SHADOWBRINGERS Full Trailer-pBuUkd5uakk.f251.webm has already been downloaded
[download] 100% of 5.80MiB
[ffmpeg] Merging formats into "FINAL FANTASY XIV - SHADOWBRINGERS Full Trailer-pBuUkd5uakk.mp4"
[debug] ffmpeg command line: ffmpeg -y -loglevel 'repeat+info' -i 'file:FINAL FANTASY XIV - SHADOWBRINGERS Full Trailer-pBuUkd5uakk.f137.mp4' -i 'file:FINAL FANTASY XIV - SHADOWBRINGERS Full Trailer-pBuUkd5uakk.f251.webm' -c copy -map '0:v:0' -map '1:a:0' 'file:FINAL FANTASY XIV - SHADOWBRINGERS Full Trailer-pBuUkd5uakk.temp.mp4'
ERROR:   Stream #1:0 -> #0:1 (copy)
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 2064, in post_process
    files_to_delete, info = pp.run(info)
  File "/usr/local/bin/youtube-dl/youtube_dl/postprocessor/ffmpeg.py", line 512, in run
    self.run_ffmpeg_multiple_files(info['__files_to_merge'], temp_filename, args)
  File "/usr/local/bin/youtube-dl/youtube_dl/postprocessor/ffmpeg.py", line 235, in run_ffmpeg_multiple_files
    raise FFmpegPostProcessorError(msg)
FFmpegPostProcessorError:   Stream #1:0 -> #0:1 (copy)

Error:
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'file:FINAL FANTASY XIV - SHADOWBRINGERS Full Trailer-pBuUkd5uakk.f137.mp4':
  Metadata:
    major_brand     : dash
    minor_version   : 0
    compatible_brands: iso6avc1mp41
    creation_time   : 2019-03-23T04:50:33.000000Z
  Duration: 00:06:35.03, start: 0.000000, bitrate: 3464 kb/s
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 11 kb/s, 29.97 fps, 29.97 tbr, 30k tbn, 59.94 tbc (default)
    Metadata:
      creation_time   : 2019-03-23T04:50:33.000000Z
      handler_name    : ISO Media file produced by Google Inc. Created on: 03/22/2019.
Input #1, matroska,webm, from 'file:FINAL FANTASY XIV - SHADOWBRINGERS Full Trailer-pBuUkd5uakk.f251.webm':
  Metadata:
    encoder         : google/video-file
  Duration: 00:06:35.04, start: -0.007000, bitrate: 123 kb/s
    Stream #1:0(eng): Audio: opus, 48000 Hz, stereo, fltp (default)
[mp4 @ 0x7feb0580d400] track 1: codec frame size is not set
[mp4 @ 0x7feb0580d400] opus in MP4 support is experimental, add '-strict -2' if you want to use it.
Could not write header for output file #0 (incorrect codec parameters ?): Experimental feature
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #1:0 -> #0:1 (copy)

you're trying to merge incompatible formats:
the codec for format 137 is avc1 which is not supported by webm container.
the codec for format 251 is opus which is not supported by mp4 container.
so it's better not to mess with --merge-output-format and let youtube-dl choose the container suitable for the selected formats unless you're sure of what you're doing.

Hey, @remitamine. I've just got the very same error.
I'm trying to download -f bestvideo[height=1080]+bestaudio and --embed-thumbnail. The last command is available only for mp4 files (as per error) so I tried using --merge-output-format mp4 which fails with same error as first post. What video/audio formats are mergeable to mp4? Tried bestvideo[height=1080][ext=mp4]+bestaudio[ext=m4a] with no success

This is in fact a bug, still present in current version.  Note that OP did not specify input formats, it's youtube-dl itself that is selecting incompatible formats.  In my case, I'm tying to use bestvideo+bestaudio with merge-output-format, so youtube-dl should select the best formats that are compatible with my chosen output container

No, it's not. OP specifies incompatible merge output format, once you specify that all bets are of and it's your responsibility to provide compatibility via format selection.


Not possible to define designs in m5o files without joins

Migrated from GitLab: https://gitlab.com/meltano/meltano/-/issues/502
Originally created by @iroussos on 2019-03-19 14:36:09

What is the current bug behavior?
I am reporting this as a bug, but it could also be a feature proposal depending on the understanding we have of what should be supported in .m5o files.
When creating a .topic.m5o file, I want to be able to only check a single table.
Let's say for example that in tap-gitlab's transforms, we generate a stats table gitlab_stats_per_user with various stats grouped together.
This table already has everything I need to generate my reports, both the dimensions (attributes to group by like user name, project or milestone) and the values to be aggregated (e.g. issues authored or assigned, mrs, etc)
I want to be able to generate a design using only that table:
{
  version = 1
  name = gitlab
  connection = postgres_db
  label = Gitlab
  designs {
    gitlab_stats_per_user {
      label = Gitlab Stats
      from = gitlab_stats_per_user
      description = "Gitlab Stats per User, Project and Milestone"
    }
  }
}

Unfortunately, the design above fails to compile as we try to access the joins attribute of the definition:
  File "/home/iroussos/work/meltano/src/meltano/core/m5o/m5o_file_parser.py", line 148, in graph_design
    joins = deepcopy(design["joins"])
KeyError: 'joins'

For a more complete log, check the section bellow.
My understanding is that this type of design should be supported out of the box per the definition of Models. This is also supported by the fact that the .topic.m5o file above passes our syntactical analysis.
What is the expected correct behavior?
Users should be able to create .topic.m5o files with designs that have a single table without joins.
Steps to reproduce
You can use the topic and models defined in the model-gitlab project to test and reproduce this issue. Checkout the branch I work on meltano/model-gitlab!1 as this depends on fixing this issue.
If you want to check it with real data, then create a new project and add the gitlab token in our Vault to extract data from the Gitlab API and check the transformed results in the analytics schema:
meltano init tap-gitlab-project --no_usage_stats
cd tap-gitlab-project

# update .env

source .env
meltano elt tap-gitlab target-postgres --transform run

# Copy the models from the model-gitlab project to the /model/ directory of your project

meltano ui

.env required:
export FLASK_ENV=development
export SQLITE_DATABASE=meltano

export PG_DATABASE=warehouse
export PG_PASSWORD=
export PG_USERNAME=
export PG_ADDRESS=localhost
export PG_PORT=5432
export PG_SCHEMA='tap_gitlab'

export GITLAB_API_TOKEN='OUR GITLAB TOKEN'
export GITLAB_API_GROUPS='meltano'
export GITLAB_API_PROJECTS=''
export GITLAB_API_START_DATE='2018-01-01T00:00:00Z'

Relevant logs and/or screenshots
Traceback (most recent call last):
  File "/home/iroussos/work/meltano/venv/lib/python3.7/site-packages/flask/app.py", line 2309, in __call__
    return self.wsgi_app(environ, start_response)
  File "/home/iroussos/work/meltano/venv/lib/python3.7/site-packages/flask/app.py", line 2295, in wsgi_app
    response = self.handle_exception(e)
  File "/home/iroussos/work/meltano/venv/lib/python3.7/site-packages/flask_restful/__init__.py", line 269, in error_router
    return original_handler(e)
  File "/home/iroussos/work/meltano/venv/lib/python3.7/site-packages/flask_cors/extension.py", line 161, in wrapped_function
    return cors_after_request(app.make_response(f(*args, **kwargs)))
  File "/home/iroussos/work/meltano/venv/lib/python3.7/site-packages/flask/app.py", line 1741, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/home/iroussos/work/meltano/venv/lib/python3.7/site-packages/flask/_compat.py", line 35, in reraise
    raise value
  File "/home/iroussos/work/meltano/venv/lib/python3.7/site-packages/flask/app.py", line 2292, in wsgi_app
    response = self.full_dispatch_request()
  File "/home/iroussos/work/meltano/venv/lib/python3.7/site-packages/flask/app.py", line 1815, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/home/iroussos/work/meltano/venv/lib/python3.7/site-packages/flask_restful/__init__.py", line 269, in error_router
    return original_handler(e)
  File "/home/iroussos/work/meltano/venv/lib/python3.7/site-packages/flask_cors/extension.py", line 161, in wrapped_function
    return cors_after_request(app.make_response(f(*args, **kwargs)))
  File "/home/iroussos/work/meltano/venv/lib/python3.7/site-packages/flask/app.py", line 1718, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/home/iroussos/work/meltano/venv/lib/python3.7/site-packages/flask/_compat.py", line 35, in reraise
    raise value
  File "/home/iroussos/work/meltano/venv/lib/python3.7/site-packages/flask/app.py", line 1813, in full_dispatch_request
    rv = self.dispatch_request()
  File "/home/iroussos/work/meltano/venv/lib/python3.7/site-packages/flask/app.py", line 1799, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/home/iroussos/work/meltano/src/meltano/api/controllers/repos.py", line 143, in sync
    return lint_all(True)
  File "/home/iroussos/work/meltano/src/meltano/api/controllers/repos.py", line 120, in lint_all
    compiler.compile()
  File "/home/iroussos/work/meltano/src/meltano/core/compiler/project_compiler.py", line 42, in compile
    self.m5o_parse.compile(self.topics)
  File "/home/iroussos/work/meltano/src/meltano/core/m5o/m5o_file_parser.py", line 165, in compile
    topic = self.graph_topic(topic)
  File "/home/iroussos/work/meltano/src/meltano/core/m5o/m5o_file_parser.py", line 155, in graph_topic
    design_graph = self.graph_design(design)
  File "/home/iroussos/work/meltano/src/meltano/core/m5o/m5o_file_parser.py", line 148, in graph_design
    joins = deepcopy(design["joins"])
KeyError: 'joins'

Possible fixes
  File "/home/iroussos/work/meltano/src/meltano/core/m5o/m5o_file_parser.py", line 148, in graph_design
    joins = deepcopy(design["joins"])

Fixing the KeyError above is pretty simple and could be as easy as writing something like design.get("joins", None) instead of design["joins"] (or checking the result and skipping the deepcopy operation).
What I am not sure about is whether there are other places in our m5o parser and the UI where we implicitly assume that a design always has a joins key as part of its structure, so we should care this one with care and check our implementation end to end.
Further regression test
We should add at least one test with a .topic.m5o file without joins

[ ] Write additional adequate test cases and submit test results
[ ] Test results should be reviewed by a person from the team

View 17 previous comments from the original issue on GitLab


Move AuthorizationFailure exception to keystoneauth1

We're currently fixing this upstream so that implementation details (such as keystoneauth) don't leak outside of OpenStack SDK. This shouldn't be necessary starting very soon.

That work happened in Map KSA exception to SDK exceptions. I think that made it into the latest release of the openstacksdk.


Appstudio update test-component-pac-bbff

Pipelines as Code configuration proposal
To start the PipelineRun, add a new comment with content /ok-to-test
For more detailed information about running a PipelineRun, please refer to Pipelines as Code documentation Running the PipelineRun
To customize the proposed PipelineRuns after merge, please refer to Build Pipeline customization

Pipelines as Code CI/test-component-pac-bbff-on-pull-request has successfully validated your commit.

StatusDurationName

✅ Succeeded
7 seconds
init

✅ Succeeded
27 seconds
clone-repository

✅ Succeeded
40 seconds
build-container

✅ Succeeded
26 seconds
inspect-image

✅ Succeeded
29 seconds
deprecated-base-image-check

✅ Succeeded
3 minutes
clair-scan

✅ Succeeded
1 minute
clamav-scan

✅ Succeeded
22 seconds
sbom-json-check

✅ Succeeded
7 seconds
label-check

✅ Succeeded
19 seconds
show-summary

✅ Succeeded
7 seconds
show-sbom

Pipelines as Code CI/test-component-pac-bbff-on-pull-request has successfully validated your commit.

StatusDurationName

✅ Succeeded
7 seconds
init

✅ Succeeded
16 seconds
clone-repository

✅ Succeeded
33 seconds
build-container

✅ Succeeded
15 seconds
inspect-image

✅ Succeeded
12 seconds
deprecated-base-image-check

✅ Succeeded
10 seconds
clair-scan

✅ Succeeded
47 seconds
clamav-scan

✅ Succeeded
10 seconds
sbom-json-check

✅ Succeeded
20 seconds
label-check

✅ Succeeded
8 seconds
show-sbom

✅ Succeeded
10 seconds
show-summary


JavaScript Object

Numbers and signleton arrays in JavaScript objects will be mapped to strings if provided as key.
Other datatypes will raise a runtime error.
See #156
Some questions:

Do we want to raise an error while generating the javascript code if the keys are not a string?
Do we want to extend the code that checks if the testplan is compatible with a programming language, to check if the datatypes of the map-keys/... are valid?

In my opinion, the cleanest way would be to do the check in the compatibility tests. That way:

Testplans containing non-string keys are not compatible with JavaScript (which they aren't)
Normally TESTed runs a check to see if a testplan is compatible with a language before running it. This allows us the keep the generation code cleaner (no exceptions), since we can assume if we reach the generation stage, the testplan is compatible.

Suggestion of updating the config.json files for compatibility testing set value- and map keytypes:
{
  ...,
  "datatypes": {
    ...
  },
  "restrictions": {
    "map": {
      "key_only": ["text", "char"]
    },
    "set": {
      "only": ["integer", "rational", "text", "char", "boolean", "tuple"]
    }
  }
}


web-streams-polyfill is not really optional

The way web-streams-polyfill is called makes it a non optional dependancy as it is declared in package.json
if (!global.ReadableStream) {
	try {
		global.ReadableStream = require('web-streams-polyfill/ponyfill/es2018');
	} catch (_) {}
}

Therefore if the environment simply does not have ReadableStreams the dependancy is called. If a developer is not using readable streams they still have to install it. Thus the dependancy is not optional.

It's marked as optional:
https://github.com/sindresorhus/ky-universal/blob/d1e9a5d8d325542f40ed2d09104aa6ccfc3cd065/package.json#L68-L72

I don't see the problem here. We mark it as optional because if the dependency is not installed, we catch the load error and don't do anything.

Yarn 2 throws an error. Stating that an optional import is being required. Not sure if thats an issue?

Sounds like they don't detect the try. Might be worth asking the Yarn folks whether this is intentional. I would assume not, because if an optional dependency can't be required(), then what's the point of having it? They might be using a different definition of "optional" or something, though.


maj tslib to next major

Expected Behavior
We expect not to have any duplicate module after our application build
Current Behavior
We have tslib twice, because of amplitude-js and underlying amplitude utils
## static/js/2.f752eb41.chunk.js
tslib (Found 2 resolved, 2 installed, 13 depended. Latest 2.3.1.)
  1.14.1
    ~/@amplitude/utils/~/tslib
      * Dependency graph
<EMAIL_ADDRESS>-><EMAIL_ADDRESS>-><EMAIL_ADDRESS>-><EMAIL_ADDRESS>      * Duplicated files in static/js/2.f752eb41.chunk.js
        tslib/tslib.es6.js (S, 10845)

  2.3.1
    ~/tslib
      * Dependency graph
<EMAIL_ADDRESS>-><EMAIL_ADDRESS>-><EMAIL_ADDRESS><EMAIL_ADDRESS>-><EMAIL_ADDRESS>-><EMAIL_ADDRESS>-><EMAIL_ADDRESS><EMAIL_ADDRESS>-><EMAIL_ADDRESS>-><EMAIL_ADDRESS>-><EMAIL_ADDRESS><EMAIL_ADDRESS>-><EMAIL_ADDRESS>-><EMAIL_ADDRESS>-><EMAIL_ADDRESS><EMAIL_ADDRESS>-><EMAIL_ADDRESS>-><EMAIL_ADDRESS>-><EMAIL_ADDRESS><EMAIL_ADDRESS>-><EMAIL_ADDRESS>-><EMAIL_ADDRESS>-><EMAIL_ADDRESS>-><EMAIL_ADDRESS><EMAIL_ADDRESS>-><EMAIL_ADDRESS>-><EMAIL_ADDRESS>-><EMAIL_ADDRESS>-><EMAIL_ADDRESS><EMAIL_ADDRESS>-><EMAIL_ADDRESS>-><EMAIL_ADDRESS>-><EMAIL_ADDRESS>-><EMAIL_ADDRESS><EMAIL_ADDRESS>-><EMAIL_ADDRESS>-><EMAIL_ADDRESS>-><EMAIL_ADDRESS><EMAIL_ADDRESS>-><EMAIL_ADDRESS>-><EMAIL_ADDRESS>-><EMAIL_ADDRESS><EMAIL_ADDRESS>-><EMAIL_ADDRESS>-><EMAIL_ADDRESS>-><EMAIL_ADDRESS><EMAIL_ADDRESS>-><EMAIL_ADDRESS>-><EMAIL_ADDRESS>-><EMAIL_ADDRESS>      * Duplicated files in static/js/2.f752eb41.chunk.js
        tslib/tslib.es6.js (S, 12348)

* Understanding the report: Need help with the details? See:
  https://github.com/FormidableLabs/inspectpack/#diagnosing-duplicates
* Fixing bundle duplicates: An introductory guide:
  https://github.com/FormidableLabs/inspectpack/#fixing-bundle-duplicates

Possible Solution
Upgrade tslib in utils to ^2.0.0
Such as what as been done in https://github.com/amplitude/Amplitude-Node/issues/176
Steps to Reproduce

git clone https://github.com/pass-culture/pass-culture-app-native.git
cd pass-culture-app-native
remove this line in package.json : https://github.com/pass-culture/pass-culture-app-native/blob/master/package.json#L336
nvm use
yarn install
yarn build:testing
watch logs for duplicate plugins.

Environment

JS SDK Version: 8.16.1
Installation Method: yarn
Browser and Version: chrome latest

@amplitude/utils is not part of the new Amplitude SDK maintained in this repo. It is part of https://github.com/amplitude/Amplitude-Node
The new @amplitude/analytics-node SDK available in this repo does not depend on @amplitude/utils at all (and it uses tslib 2)

And amplitude-js is the old Amplitude JS SDK, available at https://github.com/amplitude/Amplitude-JavaScript
But I would say that the best solution would be to migrate to the new SDK in your project (using @amplitude/analytics-browser for the browser SDK).

I think the packages in this repo has<EMAIL_ADDRESS>already, right?

@kevinpagtakhan yes, this new SDK already uses<EMAIL_ADDRESS>but the description of the issue shows usages of the old SDK instead.

@stof so for us to help on this issue, we need to update amplitude-js dependency on @amplitude/utils to be the one we just released right? just making sure we're on the same page since there may be some confusion as to why this was opened on this repo

Can you check if this satisfies your proposed changes? https://github.com/amplitude/Amplitude-JavaScript/pull/553 Though I noticed amplitude-js has a yarn.lock. Not sure if you are able to dedupe despite of

the amplitude-js package does not have a lock file. the development repo has one, but it is not part of the npm package.

I see, so if lock files does not make it to npm then the PR should be sufficient

This is fixed in<EMAIL_ADDRESS>


test: additional check to ensure hosts are left in up state

Description
This PR...

Types of changes

[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] New feature (non-breaking change which adds functionality)
[ ] Bug fix (non-breaking change which fixes an issue)
[ ] Enhancement (improves an existing feature and functionality)
[ ] Cleanup (Code refactoring and cleanup, that may add test cases)
[ ] build/CI

Feature/Enhancement Scale or Bug Severity
Feature/Enhancement Scale

[ ] Major
[ ] Minor

Bug Severity

[ ] BLOCKER
[ ] Critical
[ ] Major
[ ] Minor
[ ] Trivial

Screenshots (if appropriate):
How Has This Been Tested?

How did you try to break this feature and the system with this change?

@blueorangutan package

@shwstppr a [SL] Jenkins job has been kicked to build packages. It will be bundled with  KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@blueorangutan package

@shwstppr a [SL] Jenkins job has been kicked to build packages. It will be bundled with  KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

Packaging result [SF]: ✔ el7 ✔ el8 ✔ el9 ✔ debian ✔ suse15. SL-JID 8106

@blueorangutan test

@weizhouapache a [SL] Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

[SF] Trillian test result (tid-8648)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 40634 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr8383-t8648-kvm-centos7.zip
Smoke tests completed. 109 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test
Result
Time (s)
Test File

@weizhouapache there seems some issue with GH action run on test cleanup. Checking that

@weizhouapache there seems some issue with GH action run on test cleanup. Checking that

@shwstppr right..
so we should consider "Unsecure" as a valid and good state for kvm hosts.

@weizhouapache I guess Unsecure is just shown in UI and we don't have such a state in API. In API it returns host.details.secured flag. GH action error was related to the zone variable missing. Fixed it.
With this change, I'm testing a fix for failures seen with test_08_migrate_vm or other migration-related test because the target host is in Connecting state,
https://github.com/apache/cloudstack/pull/8356#issuecomment-1865422925
https://github.com/apache/cloudstack/pull/8374#issuecomment-1865028605
and more

@weizhouapache I guess Unsecure is just shown in UI and we don't have such a state in API. In API it returns host.details.secured flag.

yes, right.

GH action error was related to the zone variable missing. Fixed it.
With this change, I'm testing a fix for failures seen with test_08_migrate_vm or other migration-related test because the target host is in Connecting state, #8356 (comment) #8374 (comment) and more
Great !

Will merge this once GH actions return success


Azure DNS Not Appropriate

Issue Details
The guide recommends Azure DNS: https://github.com/cloudfoundry-incubator/bosh-azure-cpi-release/blob/master/docs/advanced/deploy-azuredns/README.md
However, Azure DNS only provides public DNS, and is not well suited for private/internal routes. It seems inappropriate for it to be the recommended form of DNS on Azure.

@apeschel As the document describes, Azure DNS is only one option. Which DNS server you need to use depends on your environment if you have some specific requirements.
We recommend you to use Azure DNS because it should be most reliable and stable on Azure.


Improve test coverage on decorators/clean up rate limiting a bit.

The first commit is pretty trivial.
The second and third commits are symptomatic of what happens when a decorator has lots of conditional logic in it.  It makes it really difficult to put test coverage on it.  Since the code is already in production, I jumped though all the testing hoops to cover the conditional branches first, and then that allowed me to extract some of the logic to a function that's more easily testable in the future.

Automated message from Dropbox CLA bot
@showell, it looks like you've already signed the Dropbox CLA. Thanks!

Is there a way to force a re-build?  I think my error was a transient Casper thing unrelated to my fixes.

There is a way to force a rebuild with Travis CI admin access by clicking the little spinny thing in the upper-right here:

But I think only some users can do it.
Was the Casper failure this https://github.com/zulip/zulip/issues/933?  Would like to know if we have a new nondeterministic Casper thing, since overall the goal is to not have any in steady state.  We think that one might be fixed by upgrading Casper (#967).

Nice, thanks for adding these tests @showell!  I've merged this branch.
One note, I think we could probably with some work on test helpers decrease the code duplication involved in setting up some of these tests.  For example, we have a class PostRequestMock in test_helpers.py that we might be able to use or adapt to support this sort of use case.  Right now the duplication level isn't too bad:
(zulip-venv) tabbott@zaset:~/zulip$ git grep RequestMock
zerver/lib/test_helpers.py:class POSTRequestMock(object):
zerver/tests/test_events.py:from zerver.lib.test_helpers import AuthedTestCase, POSTRequestMock
zerver/tests/test_events.py:        request = POSTRequestMock(post_data, user_profile, callback)
zerver/tests/test_narrow.py:    AuthedTestCase, POSTRequestMock,
zerver/tests/test_narrow.py:        request = POSTRequestMock(query_params, user_profile)
zerver/tests/test_narrow.py:        request = POSTRequestMock(query_params, user_profile)
(zulip-venv) tabbott@zaset:~/zulip$ git grep 'class Request'
zerver/lib/request.py:class RequestVariableMissingError(JsonableError):
zerver/lib/request.py:class RequestVariableConversionError(JsonableError):
zerver/lib/request.pyi:class RequestVariableMissingError(JsonableError): ...
zerver/tests/test_decorators.py:        class Request(object):
zerver/tests/test_decorators.py:        class Request(object):
zerver/tests/test_decorators.py:        class Request(object):
zerver/tests/test_decorators.py:        class Request(object):
zerver/tests/test_decorators.py:        class Request(object):
zerver/tests/test_decorators.py:        class Request(object):
zerver/tests/test_decorators.py:        class Request(object):
zerver/tests/test_decorators.py:        class Request(object):
zerver/tests/test_decorators.py:    class Request(object):

but it's probably at the point we could shrink the codebase nontrivially by making it possible to just do e.g. request = RequestMock(ip='<IP_ADDRESS>') for some of these.

I would open a ticket for the RequestMock issue.  I'm a little worried that POSTRequestMock has some baggage that would make decorator tests confusing to debug.  For example, POSTRequestMock automatically sets a dummy _tornado_handler attribute, which would make it tough to test the behavior of a decorator where there wasn't set.  For the decorator tests you really want a clean, custom mock that's easy to reason about, whereas most of the other test codebase wants the more functional POSTRequestMock to just magically simulate the request mechanism, because non-decorator tests are not specifically testing our request mechanism.
You and I are in 100% agreement that we can still clean up the decorator tests, but I don't know the best path forward there, and I'm ok with the current code duplication.  This might be a good bite-size ticket for somebody else.
Thanks for merging!

Sounds good re opening a ticket
-Tim Abbott (mobile)

Opened #1211 for this.


前缀选择

开发分很多组，我们组的资源有固定的前缀，现在版本默认以一个下划线区分的前缀进行分组，然后全分到一个组了，能不能自己选择前缀进行分组

@dezng Hi, can you explain in English?

via Google Translator (maybe this helps...)
Title:
Prefix Select
Comment:
Development divided into many groups, resources, our group has a fixed prefix, the current version of the default prefix an underscore to distinguish grouping, and then assigned to a whole group, and can not own prefixes group

sometimes , i always use it ,from example： ”activity_home.xml“  rather than “home_activity.xml",so could you add the  more flexible setting. thanks ！

@dmytrodanylyk Maybe use a config file to let the user define the rule of grouping.

Closing this ticket as it's duplicate of #1


Rename namespace

Currently the namespace for the lib is ffmpeg it should be libFFmpeg. That is better for integration into other projects.

Done in https://github.com/ChristianFeldmann/LibFFmpeg/pull/10


Make DOMSource.elements() always return array, create .element() for single element

Currently, in the DOM driver, DOM.select('.foo').elements() has a type of Element | Element[].
This is because when selecting the root element, the document or the body, there is only one possible result.
However, this has poor user experience, as it means users have to coerce the type depending on what they are selecting, which means we might as well not have types for this usecase.
Additionally, it is common to want to take the first item from the resulting array, in which case you must do DOM.select('.foo').elements().filter(elements => elements > 0).map(elements => elements[0]).
I propose to instead have .elements() always return an Element[], and to create an alternative .element() function that always returns an Element.
This means that users would not have to coerce the type, and could instead use the function appropriate for their use case.
.element() would be effectively equivalent to .elements().filter(elements => elements > 0).map(elements => elements[0]), which would save having to include that relatively nasty looking code.
I don't think this would make the user experience worse in any way, but would make our type information more useful and our code elegant and more readable.
The biggest downside is that this is a breaking change, as I am proposing that when selecting root, document or body, and using .elements(), we return a single item array.

Yes, I think we should just do this. In the case of .elements() for the document or body, it's okay to return an array with one entry.


Add persistrence info to DescribeCluster API call

What changed?
DescribeCluster API call will add PersistenceInfo to response.
To fully implement this feature, will update IDL and handler accordingly.

Why?
If we expose available persistence capabilities, it helps to decide if advanced UI features can be enabled.

How did you test it?
Unit tests

Potential risks

Release notes

Documentation Changes

Pull Request Test Coverage Report for Build aa562b14-99e8-46dd-a91d-5ebd7c91316a

0 of 14 (0.0%)  changed or added relevant lines in 2 files are covered.
98 unchanged lines in 11 files lost coverage.
Overall coverage decreased (-0.05%) to 67.889%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

common/persistence/visibilityWrapper.go
0
4
0.0%

service/frontend/adminHandler.go
0
10
0.0%

Files with Coverage Reduction
New Missed Lines
%

host/signalworkflowTest.go
1
95.53%

service/frontend/adminHandler.go
1
38.52%

common/persistence/executionStore.go
2
74.15%

common/task/weightedRoundRobinTaskScheduler.go
2
89.12%

common/cache/lru.go
3
90.73%

service/history/queue/timer_queue_processor_base.go
4
78.83%

common/persistence/cassandra/cassandraPersistence.go
7
52.8%

service/history/shard/context.go
8
62.55%

service/history/execution/context.go
17
68.01%

common/persistence/cassandra/cassandraTaskPersistence.go
22
59.16%

Totals

Change from base Build e8410958-aaad-47b0-880e-566658b7f31c:
-0.05%

Covered Lines:
98758

Relevant Lines:
145469

💛  - Coveralls


Ver0.1

Added the feature that players can overwrite an area of others when game is paused.
So when and where to overwrite can be critical to win in the pong wars.
Also made some refactoring to make the original version easier to be extended.
Note that there is no restrict to overwrite right now, but I plan to add some in a later version.

@vocdex
Really sorry that I created the pull request in the wrong place.
I will close it and please just ignore it.


Hide sensitive data (password) in module's output

When you create a CSS cluster via https://github.com/opentelekomcloud/ansible-collection-cloud/blob/master/plugins/modules/css_cluster.py and setting db password, it shows in Ansible output as an open text.
For security reasons it need to be hide.

just add  no_log: true in your playbook

Yes, it can help, but only in my current case. And this option will not fix the whole issue, it will fix only the symptom. I think it's better to fix module's code to avoid security leak caused by user's oversight :)

how?

Like it was solved in other our module using 'password' field. Don't worry, I'm working on it

like this? https://github.com/opentelekomcloud/ansible-collection-cloud/pull/218/commits/91ce67f0d989087cfd7a441363598e5a879879e3

No, your commit related with backupStrategy errors, and mine - with password


Equivalent of react's context

Hello deku, sorry if the issue has already been raised elsewhere but is there some kind of context (I mean like the React one, i.e. a way to make some data be propagated to all the descendants without explicit props passing) in deku edge right now?
My issue is the following: I need to pass a state tree instance along the app's hierarchy without having to pass it though props every time because this is really tiresome and because I don't want my app's code to rely on an instance of the state tree. Instead the state tree should be provided as an argument to the app.
// I don't want that
var stateTree = 'whatever';

function renderApp() {
  // Using the stateTree, it's impure
}

renderApp();

// I'd like that
function renderApp(stateTree) {
  // This is pure
}

var stateTree = 'whatever';
renderApp(stateTree);

How is this kind of problem solved currently with deku? I am thinking about react contexts because this is the only option I know.

Yeah I'm thinking about adding context into the next version and removing the app.set concept. Setting data on the app was like setting a context for the entire tree, but there was no way for components to set things on the app too.
Context could make this job easier, but it could be a dangerous way to hide state.

I don't want to hide state :-). Once my composed components are branched to a state tree, they pass data from the tree as props to their "dumb" children. For now I can live with "clever" components relying on a require/import on a state tree's instance but this is hardly ideal when needing to propagate dependencies along an app's hierarchy, especially on the server side. But I also agree that the separation between a useful context and hidden state is very thin.

Yup totally agree. That's similar to what we're planning on doing :)

I just discovered deku and this was one of the first things i looked for :+1:
I would love this feature not for passing state down but to reasonably pass state up without manually passing hooks from the parent container to each of the many components i would like to compose.

+1

This exists in 2.0. You'll just pass a context object to the render function each time. It can't be modified throughout the tree, so you just have the one object, available to all components.


Setter for $options

Parsedown allows setting options through per-option setters (setBreaksEnabled, setMarkupEscaped, ...) which is handy when using the lib with Parsedown::instance()->text($str)
ParsedownExtended does not allow this, effectively forcing us to use (new ParsedownExtended($options))->text($str))
There should be a setOptions() method!

you thinking on having setters for each option or a separate function setOptions(array, $array)
what would you prefer ?

Maybe both, to keep in line with parsedown and offer the more convenient set with array method


Locating Moon Surface Themes

Hello, I'm interested in using Moon with a new Elixir Phoenix project after reading about it on the Elixir forums. Following the getting started guide and it seems like there are a few steps missing. Was able to poke around a bit and get some aspects up and running but having trouble finding/generating the css referenced in the Moon CSS assets adding step.
<link rel="stylesheet" href="<%= Routes.static_path(@conn, "/moon/assets/css/lab-dark.css") %>" />
<link rel="stylesheet" href="<%= Routes.static_path(@conn, "/moon/assets/css/lab-light.css") %>" />

Where do the style sheets /moon/assets/css/lab-dark.css exist / how are they generated? I'm also wondering if there are any sample projects you could point me towards to learn more about setting up a project for Moon?
Thank you.

https://github.com/coingaming/moon/tree/main/priv/static/themes


Cannot use memoized class level method in sub classes

require "memo_wise"

module M1
  module Concern
    def self.included(includer)
      includer.extend ClassMethods
    end
  end

  module ClassMethods
    prepend MemoWise

    # This rarely changed
    # So should be fine caching on first usage
    def class_level_method
      1 + 1
    end
    memo_wise :class_level_method
  end
end

class C1
  include M1::Concern

  def instance_level_method
    self.class.class_level_method + 1
  end
end

class C2 < C1
  # Empty
end

# These are fine
C1.class_level_method
C1.new.instance_level_method

# These are not
C2.class_level_method
C2.new.instance_level_method

I use this pattern in my controllers
C1 would be ApplicationController and C2 would be my any of the used controllers
M1 would be any of the app-wise shared controller mixin modules

Thanks for reporting this, @PikachuEXE ! I'll look into this soon.

Thank you for fixing it 😄

Test with sample code and seem fixed
Will test with real app later

No issue found in real app


Shortcuts don't work on Linux

Using Linux (Mint 17.3 Cinnamon):
Ctrl + z doesn't undo.
The undo button still works; and Ctrl + Shift + z works for redo.
I've confirmed that this is not a problem in Windows 7 (Running the Nightcode jar)
Ctrl + z is working in other Linux programs; but it still could be something odd with my distro clobbering that key combo.  Are any other *nix users experiencing this?

I believe this is a general issue with shortcuts on Linux, so I renamed this issue to reflect that. I'll look into a fix soon.

Seeing same issue under Arch Linux and OpenJDK Runtime Environment (build 1.8.0_92-b14)

Fixed in e6b722257f026d5fe621f27160f13688fe492b7b

The fix is now available in 2.0.2.


Chapter 4 - Early stopping example

In chapter 4 in the section on early stopping, I found an issue with the way the best model is being retained (I'm using python 3.8.3, scikit-learn 0.23.1)
best_epoch = None
best_model = None
for epoch in range(1000):
    sgd_reg.fit(X_train_poly_scaled, y_train)  # continues where it left off
    y_val_predict = sgd_reg.predict(X_val_poly_scaled)
    val_error = mean_squared_error(y_val, y_val_predict)
    if val_error < minimum_val_error:
        minimum_val_error = val_error
        best_epoch = epoch
        best_model = clone(sgd_reg)

Here, the best model is declared as a clone of the SGDRegressor for that epoch. However, I found that sklearn.base.clone does not copy the fitted parameters like coef_ and intercept_, but only the regressor parameters. This means that this cloned instance is not useful for making later predictions on new data.
According to the documentation for sklearn.base.clone, this is indeed the case:

Clone does a deep copy of the model in an estimator without actually copying attached data.

A simple solution is to do the following instead:
from copy import deepcopy
best_model = deepcopy(sgd_reg)

Hi @AhmedOmran0 ,
Nice catch! Indeed, the best_model could not be used directly, it would have to be retrained. That said you generally want to retrain the best model (i.e., the model with the best hyperparameters) on the full dataset, including the training set + the validation set.
Your solution works fine. Another option is to replace best_model = clone(sgd_reg) with:
        best_model = sgd_reg
        sgd_reg = clone(sgd_reg)

I'll update the book & notebook, thanks! 👍


License File?

Hello. This looks to be a derived work of Hazel engine but I don't see the same Apache license. I was curious what license you are releasing your modifications under and I believe there are some other requirements as well listed here - https://github.com/TheCherno/Hazel/blob/master/LICENSE As far as can see, you need to list your modified files as per 4b here - https://www.apache.org/licenses/LICENSE-2.0.html Btw,  think your videos and work are really great! Thanks!

No problem! I would love to use this as a base but the license notice worries me and I don't want you to get into legal trouble. I think per the license you are required to put some sort of notice that you modified a file in any of the original but modified files. See here under 4b - https://www.apache.org/licenses/LICENSE-2.0.html


Define through association before trying to use it

Fixes a rails 5.1 failure below.  The solution is 5.0/5.1 safe.
From openstack provider specs:
  1) ManageIQ::Providers::Openstack::CloudManager::Refresher will perform a fast full legacy refresh against RHOS
     Failure/Error: expect(network.public_network_vms.count).to eq vms.count

     ActiveRecord::HasManyThroughOrderError:
       Cannot have a has_many :through association 'CloudNetwork#public_network_vms' which goes through 'CloudNetwork#public_network_routers' before the through association is defined.
     # /Users/joerafaniello/.gem/ruby/2.5.5/gems/activerecord-5.1.7/lib/active_record/reflection.rb:975:in `check_validity!'
     # /Users/joerafaniello/.gem/ruby/2.5.5/gems/activerecord-5.1.7/lib/active_record/associations/association.rb:25:in `initialize'
     # /Users/joerafaniello/.gem/ruby/2.5.5/gems/activerecord-5.1.7/lib/active_record/associations/has_many_through_association.rb:8:in `initialize'
     # /Users/joerafaniello/.gem/ruby/2.5.5/gems/activerecord-5.1.7/lib/active_record/associations.rb:265:in `new'
     # /Users/joerafaniello/.gem/ruby/2.5.5/gems/activerecord-5.1.7/lib/active_record/associations.rb:265:in `association'
     # /Users/joerafaniello/.gem/ruby/2.5.5/gems/activerecord-5.1.7/lib/active_record/associations/builder/association.rb:111:in `public_network_vms'
     # ./spec/models/manageiq/providers/openstack/cloud_manager/refresh_spec_common.rb:542:in `block in assert_networks'
     # /Users/joerafaniello/.gem/ruby/2.5.5/gems/activerecord-5.1.7/lib/active_record/relation/delegation.rb:39:in `each'
     # /Users/joerafaniello/.gem/ruby/2.5.5/gems/activerecord-5.1.7/lib/active_record/relation/delegation.rb:39:in `each'
     # ./spec/models/manageiq/providers/openstack/cloud_manager/refresh_spec_common.rb:531:in `assert_networks'
     # ./spec/models/manageiq/providers/openstack/cloud_manager/refresh_spec_common.rb:115:in `assert_common'
     # ./spec/models/manageiq/providers/openstack/cloud_manager/refresher_rhos_liberty_keystone_v3_spec.rb:72:in `block (3 levels) in <top (required)>'
     # ./spec/models/manageiq/providers/openstack/cloud_manager/refresher_rhos_liberty_keystone_v3_spec.rb:64:in `times'
     # ./spec/models/manageiq/providers/openstack/cloud_manager/refresher_rhos_liberty_keystone_v3_spec.rb:64:in `block (2 levels) in <top (required)>'

:+1:

It's also more alphabetical 😉

That was totally incidental.  😆


Update JSON tests based on a closed/fixed issues

This updates a few tests that were still marked as xfail despite passing. The underlying issues have been fixed.

build

build


Please clarify necessary scopes

Hi, I want to try your action but the docs don’t define what Oauth scopes are needed to attach to the token
Could you please edit README.md to clarify exactly what scopes are needed?

But the "appropriate" ones are never defined anywhere.


[BUG] Metamask not connecting to my dApp on Mobile (iOS & Android)

Describe the bug
We've been using Connect Kit & Wagmi for a few months in https://playprode.com and it worked like a charm. Yesterday I've been reported that users can't connect their Metamask in mobile phones. I thought maybe you've updated something, so I've installed the latest version of Connect Kit. It still doesn't work. It opens MM but does nothing there, instead of connecting and returning to the dApp like before.
To reproduce
[A CodeSandbox (preferably) or specify which version of ConnectKit you're using along with reproduction steps.](https://codesandbox.io/s/5rhqm0?file=/README.md)
I found this Sandbox from you here: https://github.com/family/connectkit. I've tried in my laptop and it works as expected. Then I've tried it on my phone and has the same behavior described before, it doesn't works.
Expected behavior
Open MM, connect and return to the dApp.
Screenshots
Environment details
I've tested it both in iOS & Android, Chrome & Safari.
Additional context
Add any other context about the problem here.

How did you fix this? I'm having same issue on iOS safari. It opens MetaMask but doesn't ask to connect. Also it's localhost if that could be the issue?

Hi, I wasn't including the walletConnectProjectId when creating the config.
After that it worked.
Best,
Alejandro
const config = createConfig(
getDefaultConfig({
appName: "xx",

alchemyId: process.env.REACT_APP_ALCHEMYID,

walletConnectProjectId: process.env.REACT_APP_WALLETCONNECT_PROJECT_ID,

chains,

}),
);
El lun, 21 ago 2023 a la(s) 16:00, Montana Flynn @.***)
escribió:

How did you fix this? I'm having same issue on iOS safari. It opens
MetaMask but doesn't ask to connect. Also it's localhost if that could be
the issue?
—
Reply to this email directly, view it on GitHub
https://github.com/family/connectkit/issues/279#issuecomment-1686868332,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/A3XMMWXD2SLXJOLMHVWX47LXWOV4LANCNFSM6AAAAAA22L2WOA
.
You are receiving this because you modified the open/close state.Message
ID: @.***>

Hi, I wasn't including the walletConnectProjectId when creating the config. After that it worked.

Thanks for that update @araiczyk-wbi, this makes sense as MetaMask on iOS uses the WalletConnect protocol for linking the wallet to the dApp.


CS:GO Achievements

I resetted my cs go achievements and if I dont launch the game it stays reset, but when I launch it suddenly I unlock about 5-10 achievements immediately in the starting screen, how do I fix this?

If you're able to edit stats for this title, then remove the stats that cause the achievement. Though I do not recommend editing stats.

From: KiteDieHard<EMAIL_ADDRESS>Sent: June 15, 2020 4:01 PM
To: gibbed/SteamAchievementManager<EMAIL_ADDRESS>Cc: Subscribed<EMAIL_ADDRESS>Subject: [gibbed/SteamAchievementManager] CS:GO Achievements (#166)
I resetted my cs go achievements and if I dont launch the game it stays reset, but when I launch it suddenly I unlock about 5-10 achievements immediately in the starting screen, how do I fix this?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHubhttps://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fgibbed%2FSteamAchievementManager%2Fissues%2F166&data=02|01||f551f53804a54ccf46c208d81177abc2|84df9e7fe9f640afb435aaaaaaaaaaaa|1|0|637278552956002276&sdata=2%2Bp%2BQdgBEIelIh88fhZ8FPCrl8pxrt%2FtPIjI%2FhMc%2Fn0%3D&reserved=0, or unsubscribehttps://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FABRPSR44ZY2TVS6RFL462BDRW2K35ANCNFSM4N64AWEQ&data=02|01||f551f53804a54ccf46c208d81177abc2|84df9e7fe9f640afb435aaaaaaaaaaaa|1|0|637278552956002276&sdata=7CkOetn%2BrjnEFdNvsFjm2%2BN1oJyrl5C39lW4GJPSE00%3D&reserved=0.

Why would you not recommend editing stats? I would really like to stop the achievements from unlocking if possible

CSGO has an anti cheat VAC
It's unlikely but editing stats probably breaks a rule and VAC can lead to account wide suspension, unlike other games where you'd only be banned from that one game.

So I have to risk getting banned or just deal with the achievements?

Ya, modifying achievements is not an issue, but I'd be concerned about modifying stats.
What's the point anyways? Just make another account.
I only use this for bugged achievements. My friend only used this for unlocking achievements when playfire use to do sponsored achievement deals.

My account has quite a few games on it so id rather not make a new one and buy them again, so im not really sure what to do

In most games editing stats should be fine, you only seemed interested in modifying csgo, which I wouldn't recommend. CSGO is FTP and you can just make a new account for it. I don't understand why somone would want to remove all achievements.

Achievements mean a bit to me, its something to do if i get bored or finish a game and id like to just have 1 account and have everything on there

how about running SAM while in VAC secured server??


Patch

About

Status

[x] These changes have been tested against Discord API or do not contain API change.
[ ] This PR includes only documentation changes, no code change.
[ ] This PR introduces some Breaking changes.

This PR doesn't really change a thing, there is no need to merge when there are no changes.


TypeError: argument of type 'NoneType' is not iterable

Welcome to Poloniex Lending Bot
2017-06-27 04:21:23 Error: argument of type 'NoneType' is not iterable Requesting returnAvailableAccountBalances
Traceback (most recent call last):
  File "/home/.../poloniexlendingbot/lendingbot.py", line 78, in <module>
    Lending.cancel_all()
  File "/home/.../poloniexlendingbot/modules/Lending.py", line 177, in cancel_all
    available_balances = api.return_available_account_balances('lending')
  File "/home/.../poloniexlendingbot/modules/Poloniex.py", line 168, in return_available_account_balances
    balances = self.api_query('returnAvailableAccountBalances', {"account": account})
  File "/home/.../poloniexlendingbot/modules/Poloniex.py", line 43, in new_method
    return method(self, *arg, **kws)
  File "/home/.../poloniexlendingbot/modules/Poloniex.py", line 125, in api_query
    json_ret = _read_response(ret)
  File "/home/.../poloniexlendingbot/modules/Poloniex.py", line 90, in _read_response
    if 'error' in data:
TypeError: argument of type 'NoneType' is not iterable
Unhandled error, please open a Github issue so we can fix it!

To replicate the issue, set the currencies to analyze to ACTIVE and that's where it fails based on where the error is being generated. I changed it to ALL and it hasn't immediately failed. I'll continue to monitor and report back if there is anything to add. Thanks!

do you mean this?
`# Local Market Analysis: Choose which coins to record the market of in order to analyse them and compute a recommended lending rate.
Enter ALL to analyse all coins, enter ACTIVE to analyse the markets of any coins you have in your lending account when the bot starts, these can be mixed.
analyseCurrencies = ACTIVE`
i just tried but couldn't reproduce

It works now... not sure what happened. I always have some sort of balance in the account so if it happens again I'll add more details.

Might have just needed some time to get what you had in your account. Comment again if it happens again.


Unable to read scheme Monokai.icls

Cannot read scheme Monokai.icls  from '$ROOT_CONFIG$/colors': Unsupported color scheme version: 142
This error came up while restarting the android studio..

I've reexported the jar, can you please try again? Does it read the theme at all?

No luck,
Still showing the same error:
Cannot read scheme Monokai.icls  from '$ROOT_CONFIG$/colors': Unsupported color scheme version: 142

Oh, wich Android Studio Version are u using?
I am using the 2.0 Preview, that might be the issue ...

Maybe, because my Studio version is stable 1.5.1

Ok, I've fixed the jar. Please reimport, to delete your existing file:
rm ~/Library/Preferences/AndroidStudio1.5/colors/Monokai.icls

Thanks a lot,
It works great.


Can we have a method to create *Entity class from base class instance.

Hello,
I am using base class instance with Gson for serialization and I would like to have it quickly transform to *Entity class for data storage.
Something like:
Method: static DataEntity DataEntity.fromData(Data dataOriginal)?
That generated method would pass all fields and create new instance of DataEntity
I could create PR if interested.
Thanks!
Mat

There is no issue with JSON deserialization to DataEntity.class


ps-scroll-x trigger many time

document.addEventListener('ps-scroll-x', function (ev) {     console.log(ev.srcElement); })
I will scroll vertical bar, the ps-scroll-x event will be trigger many time??
please help

Hmmm… that's odd, does ps-scroll-y trigger as well?

If you need to throttle the event, please consider implementing it in each handler. You can find it with keywords like throttle or debounce in JavaScript.


NotificationTexts

phealng does not return NotificationTexts
The NotificationTexts API returns the text message as CDATA. PhealNG cannot parse this correctly, and returns only the NotoficationID - not the notification text.

pheal->requestXML needs to be modified ...
$element = @new \SimpleXMLElement($this->xml, LIBXML_NOCDATA);

rowset->__construct needs to be modified to parse the CDATA correctly. The CDATA is not returned in a "named" attribute which means it is currently being ignored.

Try using __toString() on the notification element.
Example usage: https://github.com/eveseat/eveapi/blob/master/src/Api/Character/NotificationTexts.php#L74

Nice, thanks for the hint. I'll try that :)

just to understand this:
does casting it to string not work?
like $text = (string) $notification;?

as i didn't get a reply anymore, I'm closing this


Add slides about project structure

Add to week 1 slides

@graciellehigino is this done? I can't remember

Yeah, I think we just needed to render it! I wasn't able to render on my end because the code accesses your Google account =D

All set, thanks! The only thing I changed was calling the section you added How to Organize Your Files in R in 3 Months, which is also what I renamed the lesson. However, the lesson isn't currently visible, but I will make it visible during week 1.


Handle node crash situation

We had few users who experienced the node crash on startup upnp bug. They didn't see a Smesher error dialog box and just waited for hours for the app to show it starts to sync...
Proposed fix: monitor the node while the app is running and if it is down (not responding to api calls) then show red error bar telling the user that the smesher has crashed ask the user to restart the app.

this already exists in the app, red tab appeared in my app yesterday after node couldn't connect to any peer after some attempts

Ilya - no red error tab when node crashes on startup - please check this case...

@avive I just saw error bar when node failed to connect after 20 attempts, can't fix something that is working ok


Decoding Error

I get this random error every 2-10 requests, and I cant figure out the reason why.
I think in the OpenAI file, the object should be optional
public struct OpenAI: Codable {
public let object: String
public let model: String?
public let choices: [Choice]
}
ie public let object: String?
Error is...
Swift.DecodingError.keyNotFound(CodingKeys(stringValue: "object", intValue: nil),
Swift.DecodingError.Context(codingPath: [], debugDescription:
"No value associated with key CodingKeys(stringValue: "object", intValue: nil) ("object").", underlyingError: nil)))
Sorry Im not advanced enough to do a pull request and test etc.

OK So I figured out how to edit the package and I added a JSONSerialization to the Error Response and it comes back with
["error": {
code = "";
message = "The server had an error while processing your request. Sorry about that!";
param = "";
type = "server_error";
}]

Hi!
Official documentation doesn't say nothing about optionals, Doc. It seems to not be very well documented at least the responses with/without errors.
I have created a test to try to reproduce the scenario that you have shared and the only way I have found to reproduce it is to provide a wrong auth token, example:
import XCTest
@testable import OpenAISwift

final class OpenAISwiftTests: XCTestCase {
    let fakeAuthToken = "" // wrong auth token

    func testCompletions() async throws {
        do {
            let sut = OpenAISwift.init(authToken: fakeAuthToken)

            let result = try await sut.sendCompletion(with: "Write a haiku")

            XCTAssertFalse(result.choices.isEmpty)
        } catch let OpenAIError.genericError(error) {
            print(error)
        } catch let OpenAIError.decodingError(error) {
            print(error)
        }
    }
}

This test output is similar to what you comment
Test Case '-[OpenAISwiftTests.OpenAISwiftTests testCompletions]' started.
keyNotFound(CodingKeys(stringValue: "object", intValue: nil), Swift.DecodingError.Context(codingPath: [], debugDescription: "No value associated with key CodingKeys(stringValue: \"object\", intValue: nil) (\"object\").", underlyingError: nil))

Said that, have you checked that your auth token is not missing and correct?
You can check it here https://platform.openai.com/account/api-keys
I hope it helps. If you can provide more info if thats not the case it would help to debug the exact problem

The bottom half of my comment provides the response that is returned
“error” and it appears that it occurs when the servers are overloaded,
because if I resend the same request on getting that error, it eventually
returns a response. After 2 to 5 tries.
It’s got nothing to do with my authToken
The error is in the JSON Decode because “error” is not in the OpenAI
codable struct you use in JSONDecode…
I will send a detailed response when I get home
On Fri, 10 Feb 2023 at 8:58 am, Alberto Penas Amor @.***>
wrote:

Hi!
Official documentation doesn't say nothing about optionals, Doc
https://platform.openai.com/docs/api-reference/completions. It seems to
not be very well documented at least the responses with/without errors.
I have created a test to try to reproduce the scenario that you have
shared and the only way I have found to reproduce it is to provide a wrong
auth token, example:
import @.*** import OpenAISwift
final class OpenAISwiftTests: XCTestCase {
let fakeAuthToken = "" // wrong auth token
func testCompletions() async throws {
do {
let sut = OpenAISwift.init(authToken: fakeAuthToken)
        let result = try await sut.sendCompletion(with: "Write a haiku")

        XCTAssertFalse(result.choices.isEmpty)
    } catch let OpenAIError.genericError(error) {
        print(error)
    } catch let OpenAIError.decodingError(error) {
        print(error)
    }
}

}
This test output is similar to what you comment
Test Case '-[OpenAISwiftTests.OpenAISwiftTests testCompletions]' started.
keyNotFound(CodingKeys(stringValue: "object", intValue: nil), Swift.DecodingError.Context(codingPath: [], debugDescription: "No value associated with key CodingKeys(stringValue: "object", intValue: nil) ("object").", underlyingError: nil))
Said that, have you checked that your auth token is not missing and
correct?
You can check it here https://platform.openai.com/account/api-keys
I hope it helps. If you can provide more info if thats not the case it
would help to debug the exact problem
—
Reply to this email directly, view it on GitHub
https://github.com/adamrushy/OpenAISwift/issues/28#issuecomment-1424890013,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AGJCPF6KHMCFU7VZZMF4AGTWWVR7DANCNFSM6AAAAAAUWAHHVY
.
You are receiving this because you authored the thread.Message ID:
@.***>

Hi!
It would be nice if you can share the statusCode of the response, I guess is possible that you get this DecodingError if the OpenAPI returns 200 and the error body that you have shared previously.

Yes the Status Code is 200 and the JSONDecode is failing, which I then did a JSONSerialization on and get the following.
["error": {
code = "";
message = "The server had an error while processing your request. Sorry about that!";
param = "";
type = "server_error";
}]

I'm getting the same error and haven't found a solution yet unfortunately.

I think we have to monitor this and reach out to OpenAI - my gut feeling is that it's to do with some kind of rate limiting on the API or some changes they're making

Yes the Status Code is 200 and the JSONDecode is failing, which I then did a JSONSerialization on and get the following.
["error": { code = ""; message = "The server had an error while processing your request. Sorry about that!"; param = ""; type = "server_error"; }]

It looks like instead of returning a 500 for an internal error they return 200, so the body is tried to be parsed to a succesfull response, so it seems normal the DecodingError.
A possible workaround strategy is to try to parse the success response and catch the DecodingError, if it throws we can parse this error body that you have shared and then throw a new error like internalError, this can contain the message that the API returned.
It can be done easily @adamrushy, what do you think on this approach?

@albertopeam I like this approach :]

I will implement it then, thanks!

PR opened https://github.com/adamrushy/OpenAISwift/pull/33
Take a look, probably I will do some improvements regarding unit tests
@adamrushy
@MarkHoath

Hi!
Any updates on this @adamrushy? :)
https://github.com/adamrushy/OpenAISwift/pull/33

Also I see there is a new model, we should probably add that ? I might play
around with it tonight
On Sat, 4 Mar 2023 at 7:05 am, Alberto Penas Amor @.***>
wrote:

Hi!
Any updates on this @adamrushy https://github.com/adamrushy? :)
#33 https://github.com/adamrushy/OpenAISwift/pull/33
—
Reply to this email directly, view it on GitHub
https://github.com/adamrushy/OpenAISwift/issues/28#issuecomment-1454067064,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AGJCPF6HA43HAJLPLCQOLZDW2JFJLANCNFSM6AAAAAAUWAHHVY
.
You are receiving this because you were mentioned.Message ID:
@.***>

@adamrushy Hi!
Any update on this? We should fix the implementation on OpenAISwift. URLSession does not "auto-fail" when the status code is not 2XX-3XX, we have to implement that behavior, and retrieve the error message properly so it can be presented to the user or properly log it. To do that I would recommend following the same approach as in the official Python OpenAI API:

https://github.com/openai/openai-python/blob/94428401b4f71596e4a1331102a6beee9d8f0bc4/openai/api_requestor.py#L681
https://github.com/openai/openai-python/blob/94428401b4f71596e4a1331102a6beee9d8f0bc4/openai/api_requestor.py#L329

@adamrushy @MarkHoath
Solved conflicts, ready to review and merge!
Could you take a look @adamrushy

I have this error in my project
"2023-03-16 17:41:54.001076+0100 ChatGPT-IOS[56127:1373580] [plugin] AddInstanceForFactory: No factory registered for id <CFUUID 0x600002da68a0> F8BB1C28-BAE8-11D6-9C31-00039315CD46
decodingError(error: Swift.DecodingError.keyNotFound(CodingKeys(stringValue: "object", intValue: nil), Swift.DecodingError.Context(codingPath: [], debugDescription: "No value associated with key CodingKeys(stringValue: "object", intValue: nil) ("object").", underlyingError: nil)))"
Is is the same as you describe ?

I got the same error today. Did you found any solution @davidyannick86 ?

This PR should solve this: https://github.com/adamrushy/OpenAISwift/pull/65


Use Azure Active Directory as an external user store

Azure Active Directory supports SAML to enable SSO. And UAA also supports connecting to external user stores through LDAP and SAML. For SAML, my understanding is configuring a saml_login_server.
Does that mean I can integrate Azure AD with UAA and login-server just by changing configuration?
Thanks.

Closing - duplicate of https://github.com/cloudfoundry/uaa/issues/213