Is it possible to download these subtitles with Yt-Dlp? [Question]

Checklist

[x] I'm asking a question
[x] I've looked through the README and FAQ for similar questions
[x] I've searched the bugtracker for similar questions including closed ones

Verbose log

C:\Downloads>yt-dlp -v "https://vod-akc-na-central-1.media.dsso
tt.com/ps01/disney/33efc89b-fc39-47b8-8151-105a00f9969e/r/composite_es-419_FORCED_3a39e823-5f42-4685
-b6b8-8cdbdddb5568_a3334ef7-8927-4091-9493-ad533469fed6.m3u8" --skip-download --sub-langs all --writ
e-subs
[debug] Command-line config: ['-v', 'https://vod-akc-na-central-1.media.dssott.com/ps01/disney/33efc
89b-fc39-47b8-8151-105a00f9969e/r/composite_es-419_FORCED_3a39e823-5f42-4685-b6b8-8cdbdddb5568_a3334
ef7-8927-4091-9493-ad533469fed6.m3u8', '--skip-download', '--sub-langs', 'all', '--write-subs']
[debug] Encodings: locale cp1252, fs utf-8, out utf-8, pref cp1252
[debug] yt-dlp version 2021.09.25 (exe)
[debug] Python version 3.8.10 (CPython 64bit) - Windows-2012ServerR2-6.3.9600-SP0
[debug] exe versions: none
[debug] Optional libraries: Crypto, mutagen, sqlite, websockets
[debug] Proxy map: {}
[debug] [generic] Extracting URL: https://vod-akc-na-central-1.media.dssott.com/ps01/disney/33efc89b
-fc39-47b8-8151-105a00f9969e/r/composite_es-419_FORCED_3a39e823-5f42-4685-b6b8-8cdbdddb5568_a3334ef7
-8927-4091-9493-ad533469fed6.m3u8
[generic] composite_es-419_FORCED_3a39e823-5f42-4685-b6b8-8cdbdddb5568_a3334ef7-8927-4091-9493-ad533
469fed6: Requesting header
[generic] composite_es-419_FORCED_3a39e823-5f42-4685-b6b8-8cdbdddb5568_a3334ef7-8927-4091-9493-ad533
469fed6: Downloading m3u8 information
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, vcodec:vp9.2(10), acodec, files
ize, fs_approx, tbr, vbr, abr, asr, proto, vext, aext, hasaud, source, id
[debug] Default format spec: best/bestvideo+bestaudio
[info] composite_es-419_FORCED_3a39e823-5f42-4685-b6b8-8cdbdddb5568_a3334ef7-8927-4091-9493-ad533469
fed6: Downloading 1 format(s): 0

C:\Downloads\>yt-dlp -v "https://vod-akc-na-central-1.media.dsso
tt.com/ps01/disney/33efc89b-fc39-47b8-8151-105a00f9969e/r/composite_es-419_NORMAL_c79657c0-bec6-444f
-9dfc-dc51d5c5c46a_6abedef0-7a4f-42cd-b9ad-9ac71fe7888f.m3u8" --skip-download --sub-format best --wr
ite-subs
[debug] Command-line config: ['-v', 'https://vod-akc-na-central-1.media.dssott.com/ps01/disney/33efc
89b-fc39-47b8-8151-105a00f9969e/r/composite_es-419_NORMAL_c79657c0-bec6-444f-9dfc-dc51d5c5c46a_6abed
ef0-7a4f-42cd-b9ad-9ac71fe7888f.m3u8', '--skip-download', '--sub-format', 'best', '--write-subs']
[debug] Encodings: locale cp1252, fs utf-8, out utf-8, pref cp1252
[debug] yt-dlp version 2021.09.25 (exe)
[debug] Python version 3.8.10 (CPython 64bit) - Windows-2012ServerR2-6.3.9600-SP0
[debug] exe versions: none
[debug] Optional libraries: Crypto, mutagen, sqlite, websockets
[debug] Proxy map: {}
[debug] [generic] Extracting URL: https://vod-akc-na-central-1.media.dssott.com/ps01/disney/33efc89b
-fc39-47b8-8151-105a00f9969e/r/composite_es-419_NORMAL_c79657c0-bec6-444f-9dfc-dc51d5c5c46a_6abedef0
-7a4f-42cd-b9ad-9ac71fe7888f.m3u8
[generic] composite_es-419_NORMAL_c79657c0-bec6-444f-9dfc-dc51d5c5c46a_6abedef0-7a4f-42cd-b9ad-9ac71
fe7888f: Requesting header
[generic] composite_es-419_NORMAL_c79657c0-bec6-444f-9dfc-dc51d5c5c46a_6abedef0-7a4f-42cd-b9ad-9ac71
fe7888f: Downloading m3u8 information
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, vcodec:vp9.2(10), acodec, files
ize, fs_approx, tbr, vbr, abr, asr, proto, vext, aext, hasaud, source, id
[debug] Default format spec: best/bestvideo+bestaudio
[info] composite_es-419_NORMAL_c79657c0-bec6-444f-9dfc-dc51d5c5c46a_6abedef0-7a4f-42cd-b9ad-9ac71fe7
888f: Downloading 1 format(s): 0

Description

I am trying to download two subtitles, and I have not been able to download them successfully, I do not know if it is possible to download them with Yt-Dlp
My commands are as follows:
First Subtitle (Spanish "NORMAL Complete")
yt-dlp "https://vod-akc-na-central-1.media.dssott.com/ps01/disney/33efc89b-fc39-47b8-8151-105a00f9969e/r/composite_es-419_NORMAL_c79657c0-bec6-444f-9dfc-dc51d5c5c46a_6abedef0-7a4f-42cd-b9ad-9ac71fe7888f.m3u8" --skip-download --sub-langs all --write-subs
yt-dlp "https://vod-akc-na-central-1.media.dssott.com/ps01/disney/33efc89b-fc39-47b8-8151-105a00f9969e/r/composite_es-419_NORMAL_c79657c0-bec6-444f-9dfc-dc51d5c5c46a_6abedef0-7a4f-42cd-b9ad-9ac71fe7888f.m3u8" --skip-download --sub-format best --write-subs
Second Subtitle (Spanish "FORCED")
yt-dlp "https://vod-akc-na-central-1.media.dssott.com/ps01/disney/33efc89b-fc39-47b8-8151-105a00f9969e/r/composite_es-419_FORCED_3a39e823-5f42-4685-b6b8-8cdbdddb5568_a3334ef7-8927-4091-9493-ad533469fed6.m3u8" --skip-download --sub-langs all --write-subs
yt-dlp "https://vod-akc-na-central-1.media.dssott.com/ps01/disney/33efc89b-fc39-47b8-8151-105a00f9969e/r/composite_es-419_FORCED_3a39e823-5f42-4685-b6b8-8cdbdddb5568_a3334ef7-8927-4091-9493-ad533469fed6.m3u8" --skip-download --sub-format best --write-subs

Those are direct links to subtitle segment lists, which yt-dlp does not recognise correctly. Basically this is the issue I anticipated in https://github.com/yt-dlp/yt-dlp/pull/247#issuecomment-826028909.
If you point yt-dlp to the m3u8 manifest describing all the video and audio formats, yt-dlp should be able to recognise the subtitles present within.

Use the master m3u8

Yes, it is what I tried, use the "master", but only download one of the two subtitles since they have the same language code (es-419), One is full subtitle (NAME="Español (Latinoamericano)") and the other is forced subtitle (NAME="es-419--forced--")
This is the "master"
https://vod-akc-na-central-1.media.dssott.com/ps01/disney/33efc89b-fc39-47b8-8151-105a00f9969e/ctr-all-8eeb158b-ac95-4193-b71f-57176b37c3fe-a4ce99af-b2e0-474f-b07e-fc30331a0c8a.m3u8?r=720&v=1&sxl=ja&hash=6320c97932dec099707adb9a32ce98d6b3d7db35

but only download one of the two subtitles since they have the same language code (es-419), One is full subtitle (NAME="Español (Latinoamericano)") and the other is forced subtitle (NAME="es-419--forced--"

This is a known issue. I don't remember whether there is an issue open for this though
I am working on improving subtitle selection code (which will to fix this). But it wont be complete for a while

I am working on improving subtitle selection code (which will fix this). But it wont be complete for a while

Thanks, I will wait

It seems there has never been a proper issue reported for it. So I am keeping this open to track this

I think #946 already covers this.
Meanwhile, this ticket was originally opened about direct links to subtitle segment lists.

oh, I did not see that. Thanks!
In that case, I am closing this since it's not really possible to support direct subtitle manifests

It is possible, but requires content-type sniffing: at the very least performing a HEAD request at the segments.


Feature Request: Support interactive settings in the common markdown

I found the Interactive settings feature very useful. Because vscode has so many configuration items, I often need to organize some of my own settings in my md notes. If this feature can also be used in normal markdown rendering, then it will be very pleasant!😀
But for now, the feature only seems to support the use of release note.
This is my md note, and I wrote the following following the format in release, but it just render a common style:

editor.stickyScroll.maxLineCount

reference v1_87

We try to keep VS Code lean and we think the functionality you're asking for is great for a VS Code extension. Maybe you can already find one that suits you in the VS Code Marketplace. Just in case, in a few simple steps you can get started writing your own extension. See also our issue reporting guidelines.
Happy Coding!


Typo in sample deployment yaml

Typo in sample deployment yaml

The line should be:
spec:

instead of:
spec::

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 4e19e732-65d3-6f83-7669-cc86f2f71064
Version Independent ID: 9ffe3b59-cd00-a1b1-9e81-f4f19eb817d8
Content: Quickstart: Deploy an AKS cluster with Enclave Confidential Container Intel SGX nodes by using the Azure CLI
Content Source: articles/confidential-computing/confidential-enclave-nodes-aks-get-started.md
Service: virtual-machines
Sub-service: confidential-computing
GitHub Login: @angarg05
Microsoft Alias: ananyagarg

@sturrent
Thanks for your feedback! We will investigate and update as appropriate.


Usdt specify provider ns

Fixes for #328
This should allow for manually specifying a USDT probe's provider namespace.
I've also added:

[x] Documentation for the behavior
[x] Parser and semantic tests

I've manually done a functional test against the example from https://www.joyfulbikeshedding.com/blog/2019-01-31-full-system-dynamic-tracing-on-linux-using-ebpf-and-bpftrace.html#usdt-probe-example, in which @foobarwidget mentions this issue and confirmed that this works as expected.
This will probably cause conflicts with #280 as they touch a lot of the same code.
@brendangregg @mmarchini :eyes: please, since you were in the issue conversation.

I just had a thought that GetProviderFromPath could probably also be made to be a bit smarter.
Eg, it seems reasonable to always chop off a filename suffix. So libc.so should reasonably be inferred to be /lib/libc. It might also make sense to split on - and take the first token as well, in which case /lib/libc-2.27.so would also become libc, which would solve a lot of issues where the namespace name can be reasonably pulled from the path with just a bit more parsing sauce.
I can't think of any libraries that include a - in their base name, but this could be a problem for binaries, where naming is more unpredictable.

GetProviderFromPath is a very rudimentary/naive solution, and probably should be scrapped. The reality is that the provider name is customizable, and with things like libstapsdt (and before it was libusdt) during runtime multiple providers may be instantiated independently of what was included in the binary during compilation.
I think there's also an impedance mismatch between the 4 tuple that dtrace uses, and the tuple that bpftrace uses, that makes specifying provider for usdt a bit complicated.
Also, at the end of the day, regardless of setting the provider here in bpftrace we have a larger problem where bcc can't enable probes of the same name with different providers -- https://github.com/iovisor/bcc/blob/master/src/cc/usdt/usdt.cc#L299 which is probably more common that people realize. So this plumbing for the dispatch of providers and probes goes all the way down.

we have a larger problem where bcc can't enable probes of the same name with different providers

Yes I noticed that code in bcc while doing some related debugging. I agree with your assessment that this should be propagated all the way through.

I think there's also an impedance mismatch between the 4 tuple that dtrace uses, and the tuple that bpftrace uses, that makes specifying provider for usdt a bit complicated.

Can you elaborate?

Thanks @tjfontaine, we should get that bcc limitation fixed.
Thanks @dalehamel -- I think this looks good. Any reason not to merge right now? It might add some extra merge changes to #280.

@brendangregg if you're asking me, I think it should be good to merge if you agree the approach is sound.
There are two things I'm a bit worried about, but I think are both unavoidable:

That I had to add 4 lines to the parser (to cover all combinations of wildcard and string), when I was originally expecting to add just one. I don't see a way around that though, in order for this to be consistent with the semantics of other attach points. Otherwise, it wouldn't follow conventions of other attach_points accepting string parameters in both their quoted and unquoted form.
Also related to the parser, the way that tuples are handled presently means that the 3rd parameter of any 4-tuple in the future will always be assumed to be "provider namespace" in the present implementation. If we start having cases where other probes would use 4-tuples, then this could be an issue. This isn't a problem yet though, and I think it should be easy enough to make the semantic-analyzer more flexible for future cases as is already done for 3-tuples.

And to clarify, I just wanted to give a courtesy heads-up that it might complicate @tjfontaine's rebase in #280 (and I'm also pretty excited for some of the changes in there), but I don't think this should block on that. If this is merged first it should be easy enough to rebase #280 on top of it, even though it will definitely introduce a few conflicts.

Ok, I'll merge, thanks. Sorry for the pain @tjfontaine, we all still want your PR!


CHAOS-7059 Pod JVM CPU stress

Pod JVM CPU stress docs, and fault diagram
Thanks for contributing to the Harness Developer Hub! Our code owners will review your submission.
Description

Please describe your changes: __________________________________
Jira/GitHub Issue numbers (if any): ______________________________
Preview links/images (Internal contributors only): __________________

PR lifecycle
We aim to merge PRs within one week or less, but delays happen sometimes.
If your PR is open longer than two weeks without any human activity, please tag a code owner in a comment.
PRs must meet these requirements to be merged:

[ ] Successful preview build.
[ ] Code owner review.
[ ] No merge conflicts.
[ ] Release notes/new features docs: Feature/version released to at least one prod environment.

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app. Current Draft URL is: https://67516df1ed10ccf63c467cf6--harness-developer.netlify.app

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app. Current Draft URL is: https://67519e1210f710424a9cda61--harness-developer.netlify.app


Add vendored dependencies

Follow the same practice as other projects that I maintain.

No external dependencies at this time.


Answer bubbles switch places sometimes

Occasionally answer bubble buttons would switch places. Which is really wierd.

I actually thought this was a feature, to look non-deterministic :)

Nah, it needs to be fixed or controlled. Definitely not intended.

Closing due to inactivity.


[sram_ctrl] FSM coverage

Description
FSM coverage for sram_ctrl is at 70%. E.g. below 90% threshold
https://reports.opentitan.org/hw/ip/sram_ctrl_ret/dv/latest/report.html

@msfschaffner

I am not 100% certain, but I believe the uncovered state StFlush and associated uncovered transition edges are actually unreachable in this design. I will try to confirm this with UNR.

Ok UNR flags this as unreachable.
I am going to add an exclusion file generated by UNR for sram_ctrl to take care of this.


[WIP] Fix a bug when retrieving some triggers which have no parameters

Yet another of those nasty never ending trigger bug fixes.
Some of the Trigger database objects have parameters attribute set to empty dict
({}) and others don't have this attribute so we need to do double check.
Previously, an exception would be thrown since in some cases (no parameters attribute in the db), Trigger object wasn't found.
Honestly though, we should also do a better job with enforcing schema and data consistency in the database.
Exception:
Traceback (most recent call last):
  File "/data/stanley/st2reactor/st2reactor/bootstrap/rulesregistrar.py", line 78, in _register_rules_from_pack
    rule_db = RuleAPI.to_model(rule_api)
  File "/data/stanley/st2common/st2common/models/api/rule.py", line 149, in to_model
    trigger_db = TriggerService.create_trigger_db_from_rule(rule)
  File "/data/stanley/st2common/st2common/services/triggers.py", line 163, in create_trigger_db_from_rule
    + 'triggertype. Cannot create trigger: %s.' % trigger_dict)
TriggerDoesNotExistException: A simple trigger should have been created when registering triggertype. Cannot create trigger: {'type': 'twitter.matched_tweet', 'parameters': {}, 'pack': 'twitter'}.
2015-02-02 10:13:28,001 INFO [-] Registering rules from pack: examples

DB object:
{ "_id" : ObjectId("54bfec6f0640fd38c707d273"), "name" : "matched_tweet", "pack" : "twitter", "type" : "twitter.matched_tweet" }

/cc #1031

Also backported the fix to v0.7.


Check content of public IP response against a list of valid characters.

This won't actually ensure that the result is a valid IP address, but it should be effective at catching responses that very definitely aren't IP addresses.

Context for this is that users behind a captive portal were getting and html response put into their IP cache, which messed everything up for them.


pyhcarm: fix creation of virtualenv

Closes #6884

[x] I have read the Contributing Guide.

/verify

The issue looks like this.
Pointing to the real location without any symlinks allows pycharm's virtualenv helper to work correctly.
It can be easily reproduced by running something like this:
C:\Users\User\scoop\apps\python\current\python.exe -S C:\Users\User\scoop\apps\pycharm\current\IDE\plugins\python-ce\helpers\virtualenv.pyz C:\Users\User\PycharmProjects\pythonProject1\venv
In short, running bat file causes direct launch of executable bypassing this bug.

Oh, thanks for the detailed explanation!


Update dashboard.json

Switch Energy usage panel to time series. Enable 'centered zero' option to keep zero-points vertically centered.
Add grid_down to energy usage panel.
Remove time zone option from several panels where it has no effect when time zone is set in Grafana preferences and "use browser time" is set in dashboard settings
changed 'group by time(5m)' in queries in Energy Usage panel to 'group by time($__interval)'. It was causing fluctuations to be smoothed over. The graphs looked prettier, but the actual changes in the data were being lost.

Thanks for this @youzer-name  - the json export seems to be tied to your instance.  I'm not able to get it to work. Can you export using "for sharing externally"?

Done. Hopefully this one works.

That worked!  I'm going to merge this.

I noticed that the "Sun altitude" now shows up in the legend (as a max) and tool tips.  It was hidden on the older version since it doesn't add meaningful data.  We could also remove "feels like" as that likely doesn't add anything either. Thoughts?

I went ahead and made the two minor changes and also updated temp units to "number" instead of percentage.  I also created a dashboards folder to declutter things a bit. :)

I'm not sure why the legend and tool tip changed, but good catch.


Task/convert to schemapack

It includes the script to conver the linkml schema to its schemapack representation.

Hi @sbilge, there are merge conflicts, could you maybe resolve them by merging the refactor branch into this one?

Sure, I also introduce a fix for relations with an optional target.


Feature request: iOS / iPadOS support

Description
Hello. As far as I can see, the iOS is providing a network filter functionality which is nearly identical to that of macOS implementation. As a consequence, a lot of applications like VPN clients, as well as Tor and SSH tunnelling tools are working on both iOS and macOS without any problem (see SSH VPN for example). It would be great to see the SpoofDPI working on mobile platforms.

It's mighty relevant issue especially considering the lack of such kind of software on iOS.

It's a mighty relevant issue especially considering the utter lack of such kind of software on iOS.

There's AdGuard and those similar to that but their functionality is rather limited.

There's AdGuard and similar tools but their functionality is rather limited.

Hmm In my last little research I have found no DPI spoofers...

Description
(see SSH VPN for example).

Source code for the reference:
https://github.com/hossinasaadi/sshTunnel-ios

Source code for the reference:
https://github.com/hossinasaadi/sshTunnel-ios

Is it possible without rewriting the app in Swift?

Source code for the reference:

https://github.com/hossinasaadi/sshTunnel-ios

Is it possible without rewriting the app in Swift?

Swift can interact with everything that is able to export symbols. Meaning that integration with Go is not a problem. The sshTunnel itself is a wrapper over Go implementation too.

I haven't checked it myself, but looks like https://github.com/SpoofDPIApp/SpoofDPI-App may do.

~I haven't checked it myself, but looks like https://github.com/SpoofDPIApp/SpoofDPI-App may do.~
An app for macOS won't work on iOS. But I suggest everyone interested ask the author about an iOS app. Maybe they will be willing to create one.

If there are no macOS-specific APIs then it might work after compiling for the iOS target. As it would seem, I can say that the GUI itself is interacting with menu bar so it's unlikely to work as is. In order to make it work, that GUI functionality will need to be removed.


Links functionality to pages of (login) and (create account) for the homepage is added

Description
Create account and login from the side menu of the home page must direct you to their pages , now the name of the page is added and linked.
Fixes # (#8 )
How to test?
npm run dev
Checklist

[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[x] This PR is ready to be merged and not breaking any other functionality

The description and title of this PR can be improved so that reviewers understand what is your intention with the PR.
Can you please add more detail?
I changed and I hope it is clear now ?


Add dns_settings to vnet_config in Core module

We are missing dns_settings at vnet_config to set DNS settings at Vnet level.
https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/virtual_network#dns_servers

@tvuorenmaa89 released: https://github.com/XenitAB/terraform-modules/releases/tag/2021.03.2


fix(mssql): support cte in virtual tables

SUMMARY
Superset was not allowing to run CTE based SQLs on MSSQL while creating virtual tables, because MSSQL is not supporting CTE in subquery. Always the CTE sql should be at the top. Hence added the fix to make the SQL to run as same as CTE, this feature can be used for all the databases. Added a new flag called "allows_cte_in_subquery" , if it's True, Superset will parse CTE as Subquery and run it else Superset will run the CTE as same as it is.
BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

TESTING INSTRUCTIONS
Added unit test case, unit test case can be run against mssql dbengine using below command
tox -e sqlite tests/unit_tests/db_engine_specs/test_mssql.py

Thank you @villebro  for all the help you provided .


Document all and any better

all( .. ) and any( .. ) need a clear indication what they return in case of empty iterators

Yes, that would be a good addition. They are both just passing on to the Iterator methods with the same name (see the standard library), so that's the place to get the full details.
I agree with this addition, but it's not a bug so it will not stay open -- for minor things, we just want a PR.


Alert Policies/Metrics should provide more context

Expected Behavior
For the person "on call", it's important to be able to have some context of what the alert "means"
For example, there's the "blockchain-api-feed-event-counts-absent-alfajores
" policy; whose message is:
logging/user/blockchain-api-feed-events for alfajores [SUM]
Violates when: Any logging.googleapis.com/user/blockchain-api-feed-events stream is absent for greater than 1 day

For that doesn't really tell anything, unless you already know what you are looking for.
You can extract that:

it's about the blockchain api
Some log is missing "logging.googleapis.com/user/blockchain-api-feed-events"
no idea what that log is for, not what it indicates.

Surely, the one defining the policy has more context, and should provide it within the policy documentation

Also, the alert should list the correct corresponding network.
This is not the case right now, for example alert " notification-service-querying-absent-alfajoresstaging" was actually an issue with the integration notification service. (https://portal.victorops.com/client/celo#/incident/3640/alertDetails)


Suggestion: Hide all tweets/replies/etc from verified accounts

Given the changes to twitter's verification system, the type of people to actually be verified are not so much the type that I actually care to see the posts of, and this might be an opinion shared by many others.
An option to filter out anything posted by a verified account would be neat. The feature could also include some sort of whitelisting, but either way would be a good addition I feel.
(Great work btw)

The majority of people that are verified did not pay but got that mark before this change was suggested by the new owner.
It gives credibility to journalists,  activists and institutions and politicians and prevents people from easily impersonate them.
I think this suggestion does more harm than good and I could imagine that twitter won't  implement  this ad planned.

Do you mean the new "Official" label or blue ticks?
You can already use the "Verified accounts" setting in the Experiments section of the options page to highlight or hide blue ticks and interactions with them (which I just noticed was broken, should be fixed in v2.15.6)

This would be useful in tweet threads now Twitter Blue replies are being boosted, sometimes it's just pages of them in no particular order

Bumping because this seems to still be broken? It is at the very least unclear if the setting blindly targets all checkmarks, legacy (from database?) checkmarks or just "blue" checkmarks. I've got it on but I still see Twitter Blue posts.

The current experimental setting was for actual verified accounts, so it does nothing now


Error when trying to build

Because this project doesn't build I was unable to benchmark it.
Example:
>git clone https://github.com/dlang-stdx/collections
Cloning into 'collections'...
remote: Counting objects: 360, done.
remote: Compressing objects: 100% (47/47), done.
remote: Total 360 (delta 62), reused 116 (delta 48), pack-reused 228
Receiving objects: 100% (360/360), 267.44 KiB | 4.69 MiB/s, done.
Resolving deltas: 100% (161/161), done.
>cd collections
>dub test
Generating test runner configuration 'collections-test-library' for 'library' (library).
Excluding package.d file from test due to https://issues.dlang.org/show_bug.cgi?id=11847
Performing "unittest" build using dmd for x86_64.
collections 0.1.0+commit.22.g7e0f719: building configuration "collections-test-library"...
source/stdx/collections/array.d(1518,6): Error: pure function stdx.collections.array.testConcatAndAppend cannot call impure destructor std.experimental.allocator.RCIAllocator.~this
source/stdx/collections/common.d(99,5): Error: pure constructor stdx.collections.common.AllocatorHandler.__ctor!(RCIAllocator, AllocatorHandler).this cannot call impure destructor std.experimental.allocator.RCIAllocator.~this
source/stdx/collections/common.d(128,40): Error: pure constructor stdx.collections.common.AllocatorHandler.__ctor!(RCIAllocator, AllocatorHandler).this cannot call impure function std.experimental.allocator.RCIAllocator.__postblit
source/stdx/collections/common.d(128,18): Error: pure constructor stdx.collections.common.AllocatorHandler.__ctor!(RCIAllocator, AllocatorHandler).this cannot call impure destructor std.experimental.allocator.building_blocks.affix_allocator.AffixAllocator!(RCIAllocator, ulong, void).AffixAllocator.~this
source/stdx/collections/common.d(129,17): Error: pure constructor stdx.collections.common.AllocatorHandler.__ctor!(RCIAllocator, AllocatorHandler).this cannot call impure function std.algorithm.mutation.move!(AffixAllocator!(RCIAllocator, ulong, void)).move
source/stdx/collections/array.d-mixin-39(52,40): Error: template instance `stdx.collections.common.AllocatorHandler.__ctor!(RCIAllocator, AllocatorHandler)` error instantiating
source/stdx/collections/array.d(132,25):        instantiated from here: setAllocator!(RCIAllocator)
source/stdx/collections/array.d(142,27):        instantiated from here: __ctor!(RCIAllocator, Array!int)
source/stdx/collections/array.d(1522,14):        instantiated from here: Array!int
source/stdx/collections/array.d-mixin-39(52,41): Error: pure function stdx.collections.array.Array!int.Array.setAllocator!(RCIAllocator).setAllocator cannot call impure function std.experimental.allocator.RCIAllocator.__postblit
source/stdx/collections/array.d(1571,33): Error: pure delegate stdx.collections.array.__unittest_L1564_C7.__lambda2 cannot call impure function std.experimental.allocator.RCIAllocator.__postblit
source/stdx/collections/array.d(1574,5): Warning: statement is not reachable
source/stdx/collections/array.d(1575,5): Warning: statement is not reachable
source/stdx/collections/array.d(1580,6): Error: pure function stdx.collections.array.testSimple cannot call impure destructor std.experimental.allocator.RCIAllocator.~this
source/stdx/collections/array.d(1629,24): Error: pure delegate stdx.collections.array.__unittest_L1622_C7.__lambda2 cannot call impure function std.experimental.allocator.RCIAllocator.__postblit
source/stdx/collections/array.d(1632,5): Warning: statement is not reachable
source/stdx/collections/array.d(1633,5): Warning: statement is not reachable
source/stdx/collections/array.d(1638,6): Error: pure function stdx.collections.array.testSimpleImmutable cannot call impure destructor std.experimental.allocator.RCIAllocator.~this
dmd failed with exit code 1.

Yes, this project currently requires DMD nightly to build due to recent changes to std.experimental.allocator.
That's the reason for pushing parts of Phobos on code.dlang.org, but sadly this doesn't fully work yet.
This library is far far far from public use anyhow for now.


finish use case 3

I added tests for the use cases in test/integration/input_use_cases Each of the tests runs a use case without chemistry. I checked the output to make sure it was what was expected and saved it as expected_output.csv in each folder. The tests run the model with each use case and compare the results to expected_results.csv.

I like the new organization of the test cases.
I wonder if there might eventually be a reorganization (renaming) of the example use cases as well.

yeah, maybe after they are all done it will clearer how to name them

I had not noticed the changes to MICM_chemistry. I like those as well!

thanks!


[FEATURE] Expose rest rpc metrics

Code of Conduct

[X] I agree to follow this project's Code of Conduct

Search before asking

[X] I have searched in the issues and found no similar issues.

Describe the feature
Expose the metrics likes:
kyuubi.rest.connection.opend
kyuubi.rest.connection.failed
kyuubi.rest.connection.total
kyuubi.rest.connection.${method}.opened
kyuubi.rest.connection.${method}.failed
kyuubi.rest.connection.${method}.total

The method value could be one of the following: get, post, put, delete
Motivation
monitor the rest rpc
Describe the solution
No response
Additional context
No response
Are you willing to submit PR?

[x] Yes I am willing to submit a PR!

cc @yaooqinn

only rest，how about the rest？

kyuubi.rest.connection.opend

Do you mean kyuubi.rest.opend instead of kyuubi.rest.connection.opend?

oh.. i mean only recording the REST frontends? what about the rest frontends

The rest frontends seems special, because it is not interactive connection.
and now there is metrics(THRIFT_HTTP_CONN) for KyuubiTHttpFrontendService.

we need an umbrella then to support all of them, not just for rest, wdyt?

we need an umbrella then to support all of them, not just for rest, wdyt?

Agree with that, will raise an umbrella ticket.

we can also check apache/hive for referecnes

Reference:
https://github.com/apache/hive/blob/9ad5a0b28b136521ded27b4b5418f61b610da788/service/src/java/org/apache/hive/service/cli/thrift/ThriftHttpCLIService.java#L241
https://github.com/apache/hive/blob/7b3ecf617a6d46f48a3b6f77e0339fd4ad95a420/itests/hive-unit/src/test/java/org/apache/hive/jdbc/miniHS2/TestHs2ConnectionMetricsHttp.java


(Gradle) Project fails to import into Eclipse when home directory contains links to other filesystems

Describe the bug
If WINE is installed and remote NFS drives are mounted, importing the Ghidra project into Eclipse will take an inordinate amount of time (ten hours) and fails.
This is due to a Ghidra Gradle script searching for src/main/resources/generic.log4jtest.xml starting from the user's home directory. This should start searching from the top of Ghidra's Git repository.
The offending lines of code are:
// ~/git/ghidra/gradleScripts/test.gradle
def getLogFileUrl() {
	String rootDir = System.properties.get("user.dir")
	String foundFile
	new File(rootDir).eachFileRecurse(groovy.io.FileType.FILES) {
		if (it.path.endsWith('src/main/resources/generic.log4jtest.xml')) {
			foundFile = "file:" + it.path.toString()
			return "file:" + it.path.toString()
		}
	}

	return foundFile
}

One workaround is to comment out the "rootDir" assignment and supply the path to the Ghidra Git clone:
	String rootDir = '/home/philpem/git/ghidra/Ghidra/Framework/Generic/src/main/resources'

To Reproduce
Steps to reproduce the behavior:

Follow the instructions in DevGuide.md to set up Ghidra for Eclipse.
Import Ghidra into Eclipse as a Gradle project.
Eclipse proceeds to "Configuring project: decompiler" then angs on "configuring project".

Expected behavior
Project is imported and builds normally.
Screenshots
FAILURE: Build failed with an exception.

* Where:
Script '/home/philpem/git/ghidra/gradleScripts/test.gradle' line: 126

* What went wrong:
A problem occurred evaluating script.
> The provided File object is not a directory: /home/philpem/.wine/dosdevices/z:/proc/self/task/28449/cwd/.wine/dosdevices/z:/proc/self/task/28449/cwd/.wine/dosdevices/z:/proc/self/task/28449/cwd/.wine/dosdevices/z:/proc/self/task/28449/cwd/.wine/dosdevices/z:/proc/self/task/28449/cwd/.wine/dosdevices/z:/proc/self/task/28449/cwd/.wine/dosdevices/z:/proc/self/task/28449/cwd/.wine/dosdevices/z:/proc/self/task/28449/cwd/.wine/dosdevices/z:/proc/self/task/28449/cwd/.wine/dosdevices/z:/proc/self/task/28449/cwd/.wine/dosdevices/z:/proc/self/task/28449/cwd/.wine/dosdevices/z:/proc/self/task/28449/cwd/.wine/dosdevices/z:/proc/self/task/28449/root/proc/3559/task/12471/root/mnt/zfs2/scratch/veryold_homedir/philpem/JUNK/RESTORE/www/temp/misc/originals/originals_archive/file

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

CONFIGURE FAILED in 9h 58m 21s

Environment (please complete the following information):

OS: Ubuntu 18.04 "xenial"
Java Version: 11.0
Ghidra Version: Git master

Thanks for reporting.

The fix will be available in 9.1.  :dragon:


Geospatial file support

Add support for common geospatial data formats, possibly including:

shapefiles
geojson
geotiff
gdb
km[lz]

Shapefile support can be added using pyshp
GeoTIFFs may be read with rioxarray

The other file formats are less commonly distributed and may have to wait for a second iteration


Reproduction with Synbols_100K

Hi, when I want to reproduce the Synbols_100K，with the script,
synbols-datasets --dataset default --n_samples 100000 --seed 42
I got,
Generating default dataset. Info: Generate the default dataset,
using gradiant as foreground and background.
Preview generated.
12%|#########1                                                                  | 11975/100000 [04:09<28:40, 51.16it/s]/data/examples/entrypoints/synbols/drawing.py:35: RuntimeWarning: divide by zero encountered in double_scalars
font_size = attributes.scale / max(extent.width, extent.height)
12%|#########1                                                                  | 11975/100000 [04:09<30:36, 47.92it/s]
Traceback (most recent call last):
File "entrypoints/generate_datasets.py", line 78, in
main()
File "entrypoints/generate_datasets.py", line 70, in main
write_h5(file_path + ".h5py", ds_generator, args.n_samples)
File "/data/examples/entrypoints/synbols/data_io.py", line 101, in write_h5
for i, (x, mask, y) in enumerate(dataset_generator):
File "/data/examples/entrypoints/synbols/generate.py", line 326, in make_preview
for x, mask, y in generator:
File "/data/examples/entrypoints/synbols/generate.py", line 277, in dataset_generator
mask = attributes.make_mask()
File "/data/examples/entrypoints/synbols/drawing.py", line 430, in make_mask
mask_list.append(symbol.make_mask(self.resolution))
File "/data/examples/entrypoints/synbols/drawing.py", line 515, in make_mask
draw_symbol(ctxt, self)
File "/data/examples/entrypoints/synbols/drawing.py", line 38, in draw_symbol
extent = ctxt.text_extents(char)
cairo.Error: invalid matrix (not invertible)
I found the last ctxt.width and ctxt.height are zero, so did you meet this problem?

Thanks for reporting.
I cannot reproduce with synbols 1.0.3. Can you confirm this is the version you are using?
Are you running in the docker?
I cannot reproduce with the current docker.

Thanks for reporting!
When there is a symbol that is not visible for a given font, it creates a division by zero. Our filters are supposed to catch all of these but seems like there is a specific combination of symbol + font that sneaked through those filters.
I added a quick fix to avoid division by zero. we'll upgrade the version and push a new package.

Fixed version (1.0.2) pushed to pypi (https://pypi.org/project/synbols/1.0.2/). @recursix you can close if you're done.

Thanks for your reply


fix rust-analyzer test error "error[E0463]: can't find crate for test"

Thanks, sorry for the delay.


Ajenti repo is deprecated

According to Ajenti repo link provided in fucking-awesome-python, that repo is deprecated.
The new repo is this. So star feature is deprecated too.

Thank you for poiting this out. It would be verry much appreciated.
I will update on next commit.
Thanks


Globby should skip node_modules

Right now the library will remove files in node_modules after the result of globby, but some performance could be gained by inserting the check to skip node_modules into the glob itself.

Resolving the files is actually really fast. Seems like there's no need for this.


[{{safe}}/collectibles endpoint] status code 500 returned for Goerli safe with NFTs

A status code 500 is returned when trying to fetch the collectibles for one of my test safes with NFTs on Goerli.
 This field  has an schema validation: imageUri: { type: 'string', nullable: true, default: null, format: 'uri' } but the field could contains also a Data URL.
By instance, the Safe in the example request contains an item which has
"imageUri": "data:image/svg+xml;base64,PHN2ZyB3aWR0aD0iMjg5IiBoZWlnaHQ9IjE1MCIgdmlld0JveD0iMCAwIDI4OSAxNTAiIHhtbG5zPSJodHRwOi8vd3d3LnczLm9yZy8yMDAwL3N2ZyI+PHN0eWxlPkBmb250LWZhY2V7Zm9udC1mYW1pbHk6IkNhcHN1bGVzLTUwMCI7c ...

This would be fixed by relaxing the schema requirements by deleting format: 'uri' from the definition.

Besides that case, other malformed URIs are found in the Transaction Service:
https://ipfs.io/ipfs/QmaVqhfo8Ch19JJxbRAhPJYHo3KwUiBjq3EPardLPJRcKU/NFT DE TESTNET.png which are making the CGW to fail as well.

This is caused because validation fails when validating Transaction Service response for Collectible.imageUri. This field has an schema validation: imageUri: { type: 'string', nullable: true, default: null, format: 'uri' }. The requested Safe contains an item with imageUri: https://ipfs.io/ipfs/QmaVqhfo8Ch19JJxbRAhPJYHo3KwUiBjq3EPardLPJRcKU/NFT DE TESTNET.png which is making the CGW to fail because that URL is not well-formed.
I'd say this behavior is correct, since we are validating the correctness of that field (even if the Transaction Service allows that URI). I'd like to know @fmrsabino opinion on this. We need to decide between closing the issue or relaxing the validation for these cases 🤔

I'm leaning towards relaxing the validation on this field to match what was already provided by the old CGW. They should be considered valid URIs where you just have scheme:path. It's hard to say if the path should be considered valid in this example. It applies to both imageUri and uri


[Feature]: Add --debug flag to the CLI dev command

What area(s) will this request affect?
Running your code locally
What type of change do you want to see?
New feature
Overview
Is it possible to support a --debug flag to shopify app dev for integration with IDEs?
https://remix.run/docs/en/main/other-api/dev#remix-dev---debug
This was introduced in hyrdrogen and seemed like a quick fix.
https://github.com/Shopify/hydrogen/discussions/866
https://github.com/Shopify/hydrogen/pull/869/files
Motivation
Improve local development by making it easier for developers to run debuggers for their remix apps building on shopify.

Hi! In case it helps, you can easily debug from VSCode by opening Debug: Debug npm script from the command palette and then choosing dev.
Do you work with any other IDE?


Refactor and Fix Attachment Handling in File Download and Carousel

Details

Pull Request Description
This pull request introduces several changes aimed at improving the handling of attachments in file download processes and the Attachment Carousel component. The key changes are as follows:

Refactor libs/FileUtils: The function getAttachmentName has been removed. It has been replaced by getFileName, which extracts and sanitizes filenames from URLs for filesystem usage. This new function decodes the URL segment, replaces illegal characters with underscores, and includes logging for cases where the filename cannot be determined.

Fix in libs/fileDownload: Modifications have been made to ensure there is a reachable fallback for filenames. This includes changes in various platform-specific files (index.android.ts, index.ios.ts, index.ts) to utilize getFileName for generating fallback filenames when necessary.

Update in AttachmentCarousel: The CarouselItem component now uses the source URL for fallback file names. This change is reflected in extractAttachmentsFromReport.js, where the source and filename are determined based on either the provided attributes or the fallback logic.

PropTypes Enhancements: In AttachmentCarousel/CarouselItem.js and Attachments/propTypes.js, the file property within the attachment prop types has been updated to be required, ensuring better type safety and clarity in the component's expected data structure.

Technical Details:

Files Affected:

AttachmentCarousel/CarouselItem.js
Attachments/propTypes.js
AttachmentCarousel/extractAttachmentsFromReport.js
fileDownload/FileUtils.ts
fileDownload/index.android.ts
fileDownload/index.ios.ts
fileDownload/index.ts

The changes aim to enhance the robustness of attachment handling, ensuring filenames are correctly determined and handled across different components and platforms. Additionally, the update in PropTypes enforces stricter type checking, contributing to the overall reliability of the components.
Fixed Issues

$ https://github.com/Expensify/App/issues/30931
PROPOSAL: https://github.com/Expensify/App/issues/30931#issuecomment-1809743645
Tests

Case 1 - Concierge Drag & Drop Attachments
Objective: Ensure attachments added by Concierge through Drag & Drop are downloadable across platforms.
Prerequisites: A chat session with Concierge.
General:

Initiate a chat with Concierge.
Request Concierge to send inline image attachments using the Drag & Drop method.

On Web / Desktop:

Click on any of the received attachments.
Click the Download button located in the upper right corner.

Note: Since the attachment is sourced directly from Amazon S3 (which is a different origin), it will not directly download but will open in a new browser tab/window.

On iOS / Android (a regression prevents testing this flow in the current PR):

Tap on any of the received attachments.
Tap the Download button located in the upper right corner.
Observation: Unlike web platforms, native apps are unaffected by CORS. As such, the attachment should download directly to the device.

Case 2 - File Names Compliance
Objective: Verify that downloaded file names adhere to the specified naming conventions.
General:

Utilize the files downloaded in Case 1 for this test.

Naming Rules Verification:

For files with original names:

Ensure the downloaded file's name retains its original name, with the download datetime appended.
Example:

Original name: sample.pdf.
Expected download name: sample-2023-11-29 17_42_33.461.pdf.

Note: Replace colons (:) in datetime with underscores (_) as colons are not permitted in filenames.
Note: original file name is the name of the file on sender's device at the time they select it for upload

For files without original names (e.g., Concierge Drag & Drop attachments):

Confirm that the file name is derived from its URL, with the download datetime appended.
Example:

URL: https://s3-us-west-1.amazonaws.com/concierge-responses-expensify-com/uploads%2F1676033736079-1676033736079.png.
Expected download name: uploads_F1676033736079-1676033736079-2023-11-29 17_44_14.113.png.

Case 3 - Prop-Type Warnings for Attachment Carousel
Objective: Ensure there are no prop-type warnings in the console related to file names when opening attachments in the attachment carousel.
General Procedure:

Interaction with Various Attachments:

Engage in different chat scenarios:

Chat with Concierge and open attachments that were added through drag and drop.
Open attachments in regular or group chats.

For each scenario, focus on opening a variety of attachment types.

Observation in Attachment Carousel:

After the attachment carousel opens, inspect the browser's console logs.
Look specifically for any prop-type warnings related to file names.

Expected Outcome:

There should be no warnings in the console regarding missing or incorrect file names for any attachments opened in the carousel.
Ensure that the test covers a diverse set of attachment types to verify the robustness of the implementation.

Offline tests

N/A - attachments cannot be downloaded while offline
QA Steps

The following tests are applicable to both staging and production
Case 1 - Concierge Drag & Drop Attachments
Objective: Ensure attachments added by Concierge through Drag & Drop are downloadable across platforms.
Prerequisites: A chat session with Concierge.
General:

Initiate a chat with Concierge.
Request Concierge to send inline image attachments using the Drag & Drop method.

On Web / Desktop:

Click on any of the received attachments.
Click the Download button located in the upper right corner.

Note: Since the attachment is sourced directly from Amazon S3 (which is a different origin), it will not directly download but will open in a new browser tab/window.

On iOS / Android (a regression prevents testing this flow in the current PR):

Tap on any of the received attachments.
Tap the Download button located in the upper right corner.
Observation: Unlike web platforms, native apps are unaffected by CORS. As such, the attachment should download directly to the device.

Case 2 - File Names Compliance
Objective: Verify that downloaded file names adhere to the specified naming conventions.
General:

Utilize the files downloaded in Case 1 for this test.

Naming Rules Verification:

For files with original names:

Ensure the downloaded file's name retains its original name, with the download datetime appended.
Example:

Original name: sample.pdf.
Expected download name: sample-2023-11-29 17_42_33.461.pdf.

Note: Replace colons (:) in datetime with underscores (_) as colons are not permitted in filenames.
Note: original file name is the name of the file on sender's device at the time they select it for upload

For files without original names (e.g., Concierge Drag & Drop attachments):

Confirm that the file name is derived from its URL, with the download datetime appended.
Example:

URL: https://s3-us-west-1.amazonaws.com/concierge-responses-expensify-com/uploads%2F1676033736079-1676033736079.png.
Expected download name: uploads_F1676033736079-1676033736079-2023-11-29 17_44_14.113.png.

Case 3 - Prop-Type Warnings for Attachment Carousel
Objective: Ensure there are no prop-type warnings in the console related to file names when opening attachments in the attachment carousel.
General Procedure:

Interaction with Various Attachments:

Engage in different chat scenarios:

Chat with Concierge and open attachments that were added through drag and drop.
Open attachments in regular or group chats.

For each scenario, focus on opening a variety of attachment types.

Observation in Attachment Carousel:

After the attachment carousel opens, inspect the browser's console logs.
Look specifically for any prop-type warnings related to file names.

Expected Outcome:

There should be no warnings in the console regarding missing or incorrect file names for any attachments opened in the carousel.
Ensure that the test covers a diverse set of attachment types to verify the robustness of the implementation.

PR Author Checklist

[x] I linked the correct issue in the ### Fixed Issues section above
[x] I wrote clear testing steps that cover the changes made in this PR

[x] I added steps for local testing in the Tests section
[x] I added steps for the expected offline behavior in the Offline steps section
[x] I added steps for Staging and/or Production testing in the QA steps section
[x] I added steps to cover failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[x] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)
[x] I tested this PR with a High Traffic account against the staging or production API to ensure there are no regressions (e.g. long loading states that impact usability).

[x] I included screenshots or videos for tests on all platforms
[x] I ran the tests on all platforms & verified they passed on:

[x] Android: Native
[x] Android: mWeb Chrome
[x] iOS: Native
[x] iOS: mWeb Safari
[x] MacOS: Chrome / Safari
[x] MacOS: Desktop

[x] I verified there are no console errors (if there's a console error not related to the PR, report it or open an issue for it to be fixed)
[x] I followed proper code patterns (see Reviewing the code)

[x] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick)
[x] I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
[x] I verified that comments were added to code that is not self explanatory
[x] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[x] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method

[x] If any non-english text was added/modified, I verified the translation was requested/reviewed in #expensify-open-source and it was approved by an internal Expensify engineer. Link to Slack message:

[x] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[x] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is approved by marketing by adding the Waiting for Copy label for a copy review on the original GH to get the correct copy.
[x] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[x] I verified the JSDocs style guidelines (in STYLE.md) were followed

[x] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[x] I followed the guidelines as stated in the Review Guidelines
[x] I tested other components that can be impacted by my changes (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar are working as expected)
[x] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[x] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[x] I verified that if a function's arguments changed that all usages have also been updated correctly
[x] If any new file was added I verified that:

[x] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[x] If a new CSS style is added I verified that:

[x] A similar style doesn't already exist
[x] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(themeColors.componentBG))

[x] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[x] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[x] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[x] If the PR modifies a component or page that can be accessed by a direct deeplink, I verified that the code functions as expected when the deeplink is used - from a logged in and logged out account.
[x] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[x] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.

Screenshots/Videos

Android: Native

Android: mWeb Chrome

iOS: Native

iOS: mWeb Safari

MacOS: Chrome / Safari

MacOS: Desktop

Update: added specific test steps

@kidroca please complete author checklist. Some platform screenshots are still missing

✔ Done

@kidroca you've got conflicts

@kidroca you've got conflicts

✔ Conflicts resolved - synced with main
✔ Retested after syncing with main - All tests and checks have passed without any issues.

🚀 Deployed to staging by https://github.com/NikkiWines in version: 1.4.8-2 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

🚀 Deployed to production by https://github.com/yuwenmemon in version: 1.4.8-3 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
failure ❌

🕸 web 🕸
success ✅

🚀 Deployed to production by https://github.com/yuwenmemon in version: 1.4.8-3 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅


spring cloud

APPLICATION FAILED TO START

Description:
Parameter 2 of method bindingService in org.springframework.cloud.stream.config.BindingServiceConfiguration required a single bean, but 2 were found:
- defaultSockJsTaskScheduler: defined by method 'defaultSockJsTaskScheduler' in class path resource [org/springframework/web/socket/config/annotation/DelegatingWebSocketConfiguration.class]
- messageBrokerTaskScheduler: defined by method 'messageBrokerTaskScheduler' in class path resource [org/springframework/web/socket/config/annotation/DelegatingWebSocketMessageBrokerConfiguration.class]

I do not see how this is connected to ShedLock but it seems you have multiple task schedulers in the project a Spring does not know which one to use. In such case you have to wire them by hand

sorry ,when
@Bean
public BindingService bindingService(BindingServiceProperties bindingServiceProperties,
BinderFactory binderFactory, TaskScheduler taskScheduler) {
return new BindingService(bindingServiceProperties, binderFactory, taskScheduler);
}
spring cloud stream only one taskScheduler bean , you example
@Bean
public ScheduledLockConfiguration taskScheduler(LockProvider lockProvider) {
return ScheduledLockConfigurationBuilder
.withLockProvider(lockProvider)
.withPoolSize(10)
.withDefaultLockAtMostFor(Duration.ofMinutes(10))
.build();
}
Will prompt
Parameter 2 of method bindingService in org.springframework.cloud.stream.config.BindingServiceConfiguration required a single bean, but 2 were found:
Just change taskScheduler the name.
Can close this issue


Checkstyle: add check for missing @Override

Type of change

Enhancement

Description
Enforce (via checkstyle) @Override on methods that override a superclass or superinterface method.
Checklist

[x] Write tests
[x] Make sure all tests pass
[x] Check coding style
[x] Update documentation
[x] Check RBAC rights for Kubernetes / OpenShift roles
[x] Try your changes from Pod inside your Kubernetes and OpenShift cluster, not just locally
[x] Reference relevant issue(s) and close them after merging

Since this is purely a change to the coding style, I don't think more checks that successful CI completion should be necessary.

@scholzj note the failed CI build. That seems to be a timeout in the waitUntilReadySuccessful() test you added last week.

Does it work? :-o

No, on closer inspection it appears not :-(


Building a reusable microservice platform

This could entirely be our fault, but when trying to use the reusable platform in Chapter 11 we are encountering an error, any help appreciated.  We are using VS2017 RC2 and net core 1.1.
11:41:17 [Error] () Unhandled exception
System.NullReferenceException: Object reference not set to an instance of an object.
at MicroserviceNET.Auth.Authorization.<>c__DisplayClass0_0.b__0(IDictionary2 env)    at MicroserviceNET.Logging.MonitoringMiddleware.Invoke(IDictionary2 env)
at MicroserviceNET.Logging.PerformanceLogging.<>c__DisplayClass0_0.<b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at MicroserviceNET.Logging.RequestLogging.<>c__DisplayClass0_0.<b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at MicroserviceNET.Logging.CorrelationToken.<>c__DisplayClass0_0.<b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at MicroserviceNET.Logging.GlobalErrorLogging.<>c__DisplayClass0_0.<b__0>d.MoveNext()

I think the problem stems from requests that do not have the appropriate auth headers set.
In the Startup of HelloMicroservicesPlatform this code appears:
      app.UseOwin()
         .UseMonitoringAndLogging(ConfigureLogger(), HealthCheck)
         .UseAuthPlatform("test-scope")
         .UseNancy();

The important bit here is the UseAuthPlatform("test-scope") line which configures the auth middleware to require all requests to have an auth header and have the scope test-scope.
Unfortunately the auth middleware throws when there is no auth header instead of failing the request in a controlled manner. The bug is in line 18 of AuthorizationMiddleware.cs, which looks like this:
        if (principal.HasClaim("scope", requiredScope))

When there is no auth header principal is null. Boom.
To fix it change that line to
        if (principal != null && principal.HasClaim("scope", requiredScope))

Now you should get a proper 403 response when there are no auth headers.
Also worth noting is that the sample code in chapter 11 takes some fiddling to run because of the requirement for auth headers. -It's really more of an illustration than a ready to run solution.
You might consider taking out the auth platform bit and then see the other parts of the platform in action - that is the request logging, performance logging and monitor endpoints.

Thanks for the update.  Do you have an example request including auth headers and scope that we can try?


Download rule in the workflow

uses snakemake's http remote to download files from zenodo url
replaces the hippunfold_download_models script
location of files remains the same for now
reduces the docker container size from 12.84GB to 3.37GB  - can try it out in docker://khanlab/hippunfold:download-workflow

TODO:

[x] dockerfile updated
[ ] maybe add a new CLI argument --model-dir
[ ] come up with solution for dealing with nodes without internet access (maybe just a conditional to check connection, and skip the rule otherwise)
[ ] think about changing default location of models - is resources folder a better default than home directory? more likely to work, but space-inefficient if the user never overrides.. Or maybe for reproducibility it is worthwhile having the models appear with the results?

I'll merge this in later, want to test things a bit more and not have to revamp the docs before the resubmission..

quick update -- for whatever reason (have noticed it elsewhere too) the dropbox urls don't work with snakemake http remote... they end up downloading an ~800K file with html info in it, but not the file itself.. This same code was working back in November so not sure what changed..

I'm going to move introduction of the synthseg model to another PR, as this one is stalled for now..


Add support for 3D Touch on iPhone 6S and 6S Plus

This seems like a perfect library for 3D Touch!

In my opinion, it's the best replacement scheme of 3D Touch in the other devices except for iPhone 6s and 6s Plus.

Would love to see this as well... I just tried to make it work without digging into the STPopup implementation, but no gold.

what about "peek" to show a blur preview of the popup, "pop" to show the final view?

The blur preview is unnecessary for the popup. Making the opening experience special on iPhone 6s and 6s Plus is a good idea.

Made http://github.com/kevin0571/STPopupPreview as an alternative for non 3D Touch devices.


/healthz endpoint for http apps no longer functioning as intended?

In spin v0.7.x, it seems like the dedicated /healthz endpoint as mentioned in the docs no longer functions as intended (or at least as related in the docs).
As an example, let's look at an app based on the stock http-rust template.
If the HTTP path is declared as /..., the /healthz endpoint seems absorbed into this rule, returning the same result as the main component.
Spin version: spin 0.7.0 (73d315f 2022-12-15)
$ curl -i localhost:3000
HTTP/1.1 200 OK
foo: bar
content-length: 14
date: Wed, 21 Dec 2022 20:13:08 GMT

Hello, Fermyon

$ curl -i localhost:3000/healthz
HTTP/1.1 200 OK
foo: bar
content-length: 14
date: Wed, 21 Dec 2022 20:13:10 GMT

Hello, Fermyon

Whereas with spin version spin 0.6.0 (12a5037 2022-10-21), we see the behavior as relayed by the docs:
$ curl -i localhost:3000
HTTP/1.1 200 OK
foo: bar
content-length: 14
date: Wed, 21 Dec 2022 20:17:01 GMT

Hello, Fermyon

$ curl -i localhost:3000/healthz
HTTP/1.1 200 OK
content-length: 2
date: Wed, 21 Dec 2022 20:17:04 GMT

OK

If the HTTP path is declared as /hello, the /healthz endpoint returns 404:
Spin version: spin 0.7.0 (73d315f 2022-12-15)
$ curl -i localhost:3000/hello
HTTP/1.1 200 OK
foo: bar
content-length: 14
date: Wed, 21 Dec 2022 20:14:12 GMT

Hello, Fermyon

$ curl -i localhost:3000/healthz
HTTP/1.1 404 Not Found
content-length: 0
date: Wed, 21 Dec 2022 20:14:14 GMT

Whereas with spin version spin 0.6.0 (12a5037 2022-10-21), we see the behavior as relayed by the docs:
$ curl -i localhost:3000/hello
HTTP/1.1 200 OK
foo: bar
content-length: 14
date: Wed, 21 Dec 2022 20:17:58 GMT

Hello, Fermyon

$ curl -i localhost:3000/healthz
HTTP/1.1 200 OK
content-length: 2
date: Wed, 21 Dec 2022 20:18:01 GMT

OK

I do see the following PR which appears to start the process of deprecating the /healthz endpoint -- but perhaps in Spin v0.7.0 we still do expect it to function, though maybe to be fully removed/replaced at a later date: https://github.com/fermyon/spin/pull/836
I suppose since it appears support is fully removed, whether intentionally or not, we use this opportunity to update docs to use the /.well-known/spin/health endpoint?
All looks good with that route when using Spin v0.7.0:
$ curl -i localhost:3000/.well-known/spin/health
HTTP/1.1 200 OK
content-length: 2
date: Wed, 21 Dec 2022 20:30:58 GMT

OK

Yeah I think this is a docs update issue.  At the moment, I think deploy considers both health paths (to allow for a transitional period with Cloud), but spin up no longer exposes the old healthz path (because there are unlikely to be compatibility considerations when running entirely locally).

#870 is where the old route got removed.

Ah, thank you @itowlson!  I'll let the linked PR close this one out when merged.


Redeploy app

Hi, When I run the command catalyze redeploy I get this message Cannot redeploy a service that isn't running. to check this I tried catalyze status and I got this
    rails (ram = 1, storage = 0, behavior = disk_heavy, type = docker, cpu = 1, build status = finished, deploy status = caution) ID: xxxx
	service_proxy (ram = 1, storage = 0, behavior = disk_heavy, type = docker, cpu = 1, image = service_proxy, status = running) ID: xxx
	monitoring (ram = 1, storage = 0, behavior = disk_heavy, type = docker, cpu = 1, image = monitoring, status = running) ID: xxx
	logging (ram = 2, storage = 0, behavior = disk_heavy, type = docker, cpu = 2, image = logging, status = running) ID: xxx
	postgres01 (ram = 1, storage = 0, behavior = disk_heavy, type = docker, cpu = 1, image = postgresql, status = running) ID: xxx
	cache01 (ram = 1, storage = 0, behavior = disk_heavy, type = docker, cpu = 1, image = redis, status = running) ID: xxx

my rails app has status: caution but it's running and the catalyze monitor does not show nothing.
thanks

Hi @aaroalan , we've identified the cause of this and have fixed the issue upstream in our API. We are having a planned maintenance window this coming Monday night and this will be fixed then.

Great, Thanks @jault3