Misleading error message when crates.io is replace-with a local source

Problem
When using a local source via a replace-with in config.toml and there is a missing package, cargo build gives confusing errors saying that it searched crates-io for the package when it only looked in the local file system. The error it gives is identical to the error from an actually missing package:
error: no matching package named `<package name>` found
location searched: registry `crates-io`
required by package `example v0.1.0`

Possible Solution(s)
If you run cargo seach it gives much more specific information:
$ cargo search ahash
error: crates-io is replaced with non-remote-registry source dir /.../cargo-bug/vendor;
include `--registry crates-io` to use crates.io

It would be great if the original error included this "crates-io is replaced with non-remote-registry source" remark.
Steps

Unpack this zip archive: cargo-bug.zip
Run cargo build in root
Get error:error: no matching package named `ahasha` found
location searched: registry `crates-io`
required by package `example v0.1.0 (/home/rchatham/Documents/programming/tmp/cargo-bug)`

As an alternative, instead of unpacking, make an empty folder, cd into it, make the following files and then do the following shell instructions:
Cargo.toml
[package]
name = "example"
version = "0.1.0"
edition = "2021"

[dependencies]
ahash = { version = "^0.8.9" }

.cargo/config.toml
[source.crates-io]
replace-with = "vendored-sources"

[source.vendored-sources]
directory = "vendor"

Shell instructions
Then:
mkdir src
touch src/lib.rs
mkdir vendor
cargo build

Version
cargo 1.84.0-nightly (15fbd2f60 2024-10-08)
release: 1.84.0-nightly
commit-hash: 15fbd2f607d4defc87053b8b76bf5038f2483cf4
commit-date: 2024-10-08
host: x86_64-unknown-linux-gnu
libgit2: 1.8.1 (sys:0.19.0 vendored)
libcurl: 8.9.0-DEV (sys:0.4.74+curl-8.9.0 vendored ssl:OpenSSL/1.1.1w)
ssl: OpenSSL 1.1.1w  11 Sep 2023
os: Ubuntu 22.4.0 (jammy) [64-bit]

Thanks for the report.
It might be improved by using a similar mechanism you pointed out. The relevant code cargo-search handling error resides here and here.
We might want a simpler message for resolver errors, such as
error: no matching package named `ahash` found
location searched: registry `crates-io` (replaced with registry `/path/to/vendor`)
required by package `example v0.1.0 (/home/rchatham/Documents/programming/tmp/cargo-bug)`

Calling SourceId::display_registry_name should give a good enough descriptive name of the source I believe.

Hi,
I did some research on the issue the used Registry::describe_source in resolver/errors to get replacement message, and it showed following messages when building cargo-bug.zip:
error:` no matching package named `ahasha` found
location searched: registry `crates-io`, directory source `/home/todd/Desktop/cargo-bug/vendor` (which is replacing registry `crates-io`)
required by package `example v0.1.0 (/home/todd/Desktop/cargo-bug)`

Since I'm new to both open source and Rust, I'm not sure if this is the right way to approach the issue, and I would greatly appreciate any feedback.
Thanks.

Feel free to claim this issue and submit a pull request!
Personally I lean toward keeping the message concise, and prefer showing the replaced source first. Other may have different preference.

@rustbot claim

@rustbot release-assignment

@rustbot  claim

I did some research on the issue the used Registry::describe_source in resolver/errors to get replacement message, and it showed following messages when building cargo-bug.zip:
error:` no matching package named `ahasha` found
location searched: registry `crates-io`, directory source `/home/todd/Desktop/cargo-bug/vendor` (which is replacing registry `crates-io`)
required by package `example v0.1.0 (/home/todd/Desktop/cargo-bug)`

I just understood what you meant. We already have such a message, so probably we can just apply this a simple fix, and update all test snapshots.
diff --git a/src/cargo/core/resolver/errors.rs b/src/cargo/core/resolver/errors.rs
index 7fbf4e22c0e..64503b9401a 100644
--- a/src/cargo/core/resolver/errors.rs
+++ b/src/cargo/core/resolver/errors.rs
@@ -355,3 +355,6 @@ pub(super) fn activation_error(
         }
-        msg.push_str(&format!("location searched: {}\n", dep.source_id()));
+        msg.push_str(&format!(
+            "location searched {}\n",
+            registry.describe_source(dep.source_id())
+        ));
         msg.push_str("required by ");

Thanks, I'll follow the advice!


External TS input reader

Adresses/Fixes #914.
Implements

A Server-client model based typescript input reader with,

typescript matcher and trigger interpreter.
Tests and test files.

@devonfw/cobigen

There is one conflict on the tsplugin pom.xml. Please check that.


Project: Expanded panel below the optimizer grid for more details about the selected build

Currently a lot of the math is hidden - For example, skill damage increase, def shred, res pen, etc. Part of the reason is to minimize the amount of data transfer from threadworker buffers to the main thread.
Goal: A panel under the optimizer grid to show these hidden numbers would be useful for more detailed investigation into the effect of buffs / etc. To do this would most likely require refactoring the optimizerWorker.js so it can be called from the main thread on each click of the grid, and display the expanded numbers from a single row instead of calculating them all for the whole grid.

Possibly toggle a full width row for this:
https://www.ag-grid.com/angular-data-grid/full-width-rows/


HTTP 500: 'internal error' when restoring, no logs available

My Environment

ArangoDB Version:        3.9.10
Deployment Mode:       Cluster
Deployment Strategy:  Manual Start
Configuration:               Cluster
Infrastructure:               AWS
Operating System:        CentOS7
Total RAM in your machine:        32Gb x 5
Disks in use:        SSD x 5
Used Package:    Debian

Component, Query & Data
Affected feature:
Backup restore
Size of your Dataset on disk:
373G
Replication Factor & Number of Shards (Cluster only):
rep factor: 3, No. of shards: 1
Problem:
Restoring a db dump which contains a particularly large collection 'resumes' approx 175mill doc(s) returns error HTTP 500: 'internal error' when restoring using arangorestore while testing our Disaster Recovery setup. There doesn't seem to be any more information given as to why this error is occurring and I can't find anything in the logs. See Error 1. To get around this issue I rerun the restore with --force enabled however these errors persist and the full collection fails to be restored. I also see another 403 error 'collection is read-only' which I don't understand as I have not set this collection to be read-only in either source/destination.
(NOTE: version 3.9.10, we are tied to 3.9 for now, I know it's reached EOL for some time so any help is greatly appreciated)
Error1:
2023-12-12T11:50:55Z [7219] INFO [75e65] {restore} # Current restore progress: restored 18 of 19 collection(s), read 110.6 GB from datafiles, sent 13196 data batch(es) of 110.5 GB total size, queued jobs: 0, workers: 4
2023-12-12T11:50:59Z [7219] WARNING [a595a] {restore} Error while restoring data into collection 'resumes': got invalid response from server: HTTP 500: 'internal error' while executing restoring data
Error2 (ran arangorestore with --force):
2023-12-14T14:13:42Z [31234] INFO [69a73] {restore} # Loading data into document collection 'resumes', 909.0 GB of 909.0 GB read (99 %)
2023-12-14T14:13:42Z [31234] WARNING [a595a] {restore} Error while restoring data into collection 'resumes': got invalid response from server: HTTP 403: 'collection is read-only' while executing restoring data

Further details

Source Cluster
No. of Nodes: 5
Collections: 19
- Problematic collection: "resumes"
Size of problematic collection: 173million doc(s)
Arangodb version: 3.9.10

backup command:
arangodump --server.endpoint "tcp://$host:8529" --server.username "$USER" --server.password "$PASS" --server.database "$database" --output-directory "$BKUP_DIR/$FILE_NAME" --compress-output false --progress --threads 4

Compression command: tar -I lz4 -cf "$BKUP_DIR/$FILE" --directory "$BKUP_DIR/$FILE_NAME" .
Arangodb cluster start command:
/usr/bin/arangodb \
    --starter.address=${PRIVATEIP} \
    --starter.data-dir=${DATADIR} \
    --starter.join=${STARTERENDPOINTS} \
    --auth.jwt-secret=/etc/arangodb3/jwtSecret \
    --server.storage-engine=rocksdb

Destination Cluster

No. of Nodes: 5
Collections: 19
Arangodb version: 3.9.10
DeCompress command: tar -I lz4 -xf /datadrive/$backupObj --directory "/datadrive/$database-$yyyymmdd/"

Restore command:
arangorestore --server.password "$PASS" --server.endpoint "tcp://$ip:8529" --server.username "root" --server.database "$database" --input-directory "/datadrive/$database-$yyyymmdd/"
Successfully restored: 144million doc(s)

Can we get an update from someone at ArangoDB on this issue?


error NSError * domain: @"com.samsoffes.sskeychain" - code:<PHONE_NUMBER>370951

I get this error with code<PHONE_NUMBER>370951. I find no error code like this. Is there any information could tell me ? thanks.

I have the same issue

I believe that is record not found. This isn't an issue with SAMKeychain, but an issue with the system keychain. Best of luck!


ETL job errors if there are no videos to process and parallel processing is requested

I would expect this to create the output directory and exit instead of erroring.
$ python -m aind_behavior_video_transformation.etl -j '{"input_source": "foo", "output_directory": "bar",  "compression_requested": {"compresison_enum": "gamma fix colorspace"}}'
<frozen runpy>:128: RuntimeWarning: 'aind_behavior_video_transformation.etl' found in sys.modules after import of package 'aind_behavior_video_transformation', but prior to execution of 'aind_behavior_video_transformation.etl'; this may result in unpredictable behaviour
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/home/galen.lynch/Documents/Code/aind-behavior-video-transformation/src/aind_behavior_video_transformation/etl.py", line 183, in <module>
    job_response = job.run_job()
  File "/home/galen.lynch/Documents/Code/aind-behavior-video-transformation/src/aind_behavior_video_transformation/etl.py", line 153, in run_job
    self._run_compression(convert_video_args)
    ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
  File "/home/galen.lynch/Documents/Code/aind-behavior-video-transformation/src/aind_behavior_video_transformation/etl.py", line 87, in _run_compression
    with ProcessPoolExecutor(max_workers=num_jobs) as executor:
         ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/concurrent/futures/process.py", line 666, in __init__
    raise ValueError("max_workers must be greater than 0")
ValueError: max_workers must be greater than 0

https://github.com/AllenNeuralDynamics/aind-behavior-video-transformation/blob/153d898dc500308f6a43b179bdfc9302f48e24c0/src/aind_behavior_video_transformation/etl.py#L87

Closed by #20


Unable to access this over lan

Hello Ridhwaans,
Great initiative by the way, I tried to set this up works good locally on the machine where I set this up but unable to access it on my lan using different machine. Is there any additional configuration required to get this working on lan. Please advise.
The issue I am facing is the app loads up fine, shows all the movies in the catalog but when I open any of it, the player becomes inactive (as if all the controls greys out and I cannot do anything with that). Appreciate your time and thanks in advance.
Regards,
Mohammed Wajee

hi @contactwajeeh. In client-v1.1.0 you would have to search and replace localhost:<port> with the ip address of your computer in the lan.
If that doesnt work, I would check if the fs_path in your movies json is pointing to the right location on the filesystem. See https://github.com/ridhwaans/homehost#naming-conventions

All good now, thanks appreciate your response.


Added: warning, Python 3.6 support will be dropped in future versions

@KleinK0910 please review changelog(PR's name) and ipyvizzu will print a FutureWarning: "Python 3.6 support will be dropped in future versions." if someone using it on python 3.6.


Condor jobs not auto-removed

Describe the bug
CondorProvider sets a leave_in_queue=True flag that causes condor jobs to linger in the queue
indefinitely. This might cause issues in cases where total jobs in queue are limited.
To Reproduce
Steps to reproduce the behavior, for e.g:

Setup Parsl 0.8.0 with Python 3.6 on cluster
Run tests with CondorProvider
Wait to complete
condor_q will show tasks in

Expected behavior
Condor jobs should be removed once Parsl finishes execution.
Environment
?
Reported by Matthew Becker from the DESC community.

Not sure. In many cases the queuing system attaches errors to the job status that are otherwise not accessible. This is an incentive to actually poll the queue for errored jobs when possible. So we might not want to completely abandon that.
I think Swift did something similar by setting leave_in_queue=True until a terminal status was detected, after which it would set leave_in_queue to False. But it seems like that scheme would also leave jobs in the queue when things were abruptly terminated.

leave_in_queue is can be any ClassAd expression that evaluates to a bool, not just a literal True/False.
Maybe there's something interesting that can be done there to attempt to preserve jobs for a while but make them disappear eventually - eg keep for n mins after completion according to ClassAds, but still deleted by parsl when it no longer needs the jobs to be there?
As an example of a more complicated expression, https://research.cs.wisc.edu/htcondor/manual/v8.7.8/labelmancondorsubmitCondorsubmit.html gives:
leave_in_queue = (JobStatus == 4) && ((StageOutFinish =?= UNDEFINED) ||\
                 (StageOutFinish == 0))

Eg leave_in_queue could go false when completion time is more than 3x the parsl polling period.

Yeah, that's what I thought many years ago, but couldn't figure out a solution, but maybe I didn't really try. I see:
on_exit_hold = (time() - JobStartDate) < (60 * $(MINUTE))
So maybe leave_in_queue = (time() - JobEndDate) < (60 * $(MINUTE))


Question about pseudo label and category mismatch.

Pseudo Label
From what I gather, the graphs for target images in the paper are constructed solely based on the pseudo label due to the lack of ground truth labels.
However, in cases where the domain gap between source and target is very large, such as going from a sunny day to a heavy rainy night, relying on the pseudo label can be inadequate and may lead to issues in graph construction.
Assuming my understanding is accurate, are there any potential solutions to address this problem and enhance SIGMA's performance?
Category Mismatch
I have an additional concern regarding the effectiveness of the node completion (DNC) strategy used in the paper. The datasets used for domain adaptation, such as Cityscape, FoggyCityscape, Sim10k, and KITTI, have similar categories.
As a result, I am uncertain whether DNC would perform well if the categories were significantly different between the source and target datasets.

Hi, thanks for your good questions. I am happy that you have such deep and insightful thoughts and show interest in our work.
Pseudo Label
The smoothness property of the graph operations will make it robust for noisy labels. Therefore, we set a very low threshold (0.5) to sample nodes. More insights can be found in several noisy-label-related works, e.g.,  the famous graph propagation in pseudo-labeling (semi-supervised learning) here. This is a theoretically-grounded idea, which hasn't been considered in DAOD.
To further improve the performance, you can consider other scenarios related to noisy labels to find the solution. E.g., introduce Teacher-Student EMA, add CycleGAN interpolated data (Note that this will significantly improve the self-training-based method), and logit engineering.
Category Mismatch
The main function of DNC is to enable the unbiased adaptation signal in each iteration, designed for the cross-domain inconsistent P(Y). Yes, we also found that it will make a more noticeable effect in the scenarios with more classes, e.g., P2C. Moreover, this can be seen as a fine-grained version of the existing prototype-based approach (RPN Prototype Alignment for Domain Adaptive Object Detector).
Hope this can help you a lot.

Thanks for your timely reply. Your response is really helpful to me!


Fixes several TODOs in Credential-related code.

Fixes issue #769
Fixes issue #770
Fixes issue #771

This change is

One more nit with the copyright. Otherwise looks good to go.


Implement private class gui::views::ControlView::_CtrlRecord (cpp)

This is a sub-task of #237 .

Partially implemented; Compiles.
Still to be completed then tested.


An error occured when I tried to do this operation:   Project is required.

Hi, I've installed and configured the bot but when I'm trying to create a new issue it displays the following error.
 `Component/s is required.`

Chat log
jirabot APP [11:20 AM]
Ok! Give me the `Summary` of the story ?

António Carvalho [11:20 AM]
test

jirabot APP [11:20 AM]
Component/s ?
1 `DevOps`   2 `Web`   3 `Example`

António Carvalho [11:20 AM]
1

jirabot APP [11:20 AM]
Product ?
1 `Web1`   2 `Web2`

António Carvalho [11:20 AM]
1

jirabot APP [11:20 AM]
I am working on it

bot APP [11:20 AM]
An error occured when I tried to do this operation:
 `Project is required.`

Application log
11:50:16.769 [ForkJoinPool.commonPool-worker-1] INFO  c.i.b.channel.slack.SlackChannel - Found a conversation catalyst command for action: action_jira_comment_issue .Entering catalyst conversation.
11:50:16.771 [ForkJoinPool.commonPool-worker-1] INFO  c.i.b.channel.slack.SlackChannel - Handling action action=action_jira_create, user=antonio.carvalho
11:50:16.949 [AsyncHttpClient-2-1] INFO  a.s.rtm.SlackRealTimeMessagingClient - Slack RTM message : {"ok":true,"reply_to":0,"ts":"1542109816.017800","text":"I am working on it \n"}
11:50:17.289 [ForkJoinPool.commonPool-worker-1] ERROR c.i.benten.jira.BentenJiraClient - com.intuit.benten.jira.exceptions.BentenJiraException: Project is required.
java.lang.RuntimeException: com.intuit.benten.jira.exceptions.BentenJiraException: Project is required.
        at com.intuit.benten.jira.http.JiraHttpClient.createIssue(JiraHttpClient.java:207)
        at com.intuit.benten.jira.BentenJiraClient.createIssue(BentenJiraClient.java:161)
        at com.intuit.benten.jira.actionhandlers.JiraCreateIssueActionHandler.handle(JiraCreateIssueActionHandler.java:61)
        at com.intuit.benten.channel.slack.SlackChannel.lambda$null$2(SlackChannel.java:246)
        at java.util.concurrent.CompletableFuture.uniAccept(CompletableFuture.java:656)
        at java.util.concurrent.CompletableFuture$UniAccept.tryFire(CompletableFuture.java:632)
        at java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:443)
        at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
        at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
        at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
        at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
Caused by: com.intuit.benten.jira.exceptions.BentenJiraException: Project is required.
        at com.intuit.benten.jira.http.JiraHttpClient.handleJiraException(JiraHttpClient.java:227)
        at com.intuit.benten.jira.http.JiraHttpClient.createIssue(JiraHttpClient.java:200)
        ... 10 common frames omitted

Okay, found out that there is a custom field required in my case cf[12402] which corresponds to projects name.

Any guidance on how to add custom fields to requests and responses? Thank you :smiley_cat:

I assume you were trying the create jira story. The way it works for required fields is BenTen calls Jira API to figure out what are all the required fields and then convert them into a conversation. Do you have another required field called Project in you Jira create issue UI ? Can you give provide a screenshot. That will help me fixing this.

Released a new version 0.1.4 with the fix
please try it out and let me know
you can update the benten.version in pom.xml of benten-starter
and that should do it.

Got a confirmation from @ampc  Antonio that the issue has been resolved in Release 0.1.5  in support-channel. Closing the issue as resolved.


[Bug] Swipe notification doesnt trigger password view

Nothing happens after swiping, doesnt happen all the time.

I get this as well. On alpha 4. I'm yet to test alpha 5.

@samedgagger @sween64 Ima aware of this issue and it will be reworked in alpha 6. its caused by my attempt at fixing the constant prompting for users password when scrolling through notifications.


Timeout Querying Undocumented API

Govee Device SKU
H6056
Govee2MQTT Version
2024.01.11-9564506d
Describe the issue
When I turned on the add on, it seemed to get hung up on the undocumented API call for a while before finally timing out and shutting down. Please see logs.
Startup Diagnostics
s6-rc: info: service s6rc-oneshot-runner: starting
s6-rc: info: service s6rc-oneshot-runner successfully started
s6-rc: info: service fix-attrs: starting
s6-rc: info: service fix-attrs successfully started
s6-rc: info: service legacy-cont-init: starting
s6-rc: info: service legacy-cont-init successfully started
s6-rc: info: service legacy-services: starting
s6-rc: info: service legacy-services successfully started
GOVEE_API_KEY=REDACTED
GOVEE_MQTT_HOST=core-mosquitto
GOVEE_EMAIL=REDACTED
GOVEE_PASSWORD=REDACTED
GOVEE_MQTT_PASSWORD=REDACTED
GOVEE_MQTT_USER=addons
GOVEE_MQTT_PORT=1883
++ cd /app
++ exec /app/govee serve
[2024-01-11T18:57:08 INFO  govee::commands::serve] Starting service. version 2024.01.11-9564506d
[2024-01-11T18:57:08 INFO  govee::commands::serve] Querying platform API for device list
[2024-01-11T18:57:08 INFO  govee::commands::serve] Querying undocumented API for device + room list
Error: timeout connecting to IoT in AWS

Caused by:
    deadline has elapsed

Stack backtrace:
   0: std::backtrace_rs::backtrace::libunwind::trace
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/../../backtrace/src/backtrace/libunwind.rs:104:5
   1: std::backtrace_rs::backtrace::trace_unsynchronized
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
   2: std::backtrace::Backtrace::create
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/backtrace.rs:331:13
   3: govee::service::iot::start_iot_client::{{closure}}
   4: govee::commands::serve::ServeCommand::run::{{closure}}
   5: govee::Args::run::{{closure}}
   6: tokio::runtime::park::CachedParkThread::block_on
   7: tokio::runtime::runtime::Runtime::block_on
   8: govee::main
   9: std::sys_common::backtrace::__rust_begin_short_backtrace
  10: std::rt::lang_start::{{closure}}
  11: core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/ops/function.rs:284:13
  12: std::panicking::try::do_call
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:552:40
  13: std::panicking::try
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:516:19
  14: std::panic::catch_unwind
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panic.rs:142:14
  15: std::rt::lang_start_internal::{{closure}}
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:148:48
  16: std::panicking::try::do_call
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:552:40
  17: std::panicking::try
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:516:19
  18: std::panic::catch_unwind
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panic.rs:142:14
  19: std::rt::lang_start_internal
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:148:20
  20: std::rt::lang_start
  21: main
s6-rc: info: service legacy-services: stopping
s6-rc: info: service legacy-services successfully stopped
s6-rc: info: service legacy-cont-init: stopping
s6-rc: info: service legacy-cont-init successfully stopped
s6-rc: info: service fix-attrs: stopping
s6-rc: info: service fix-attrs successfully stopped
s6-rc: info: service s6rc-oneshot-runner: stopping
s6-rc: info: service s6rc-oneshot-runner successfully stopped

Additional Logs
No response
Home Assistant Logs
No response
Anything else?
No response

The most likely cause is some kind of firewall configuration on your local network.
Can your home assistant instance reach amazon's IoT servce on port 8883?
If you don't want it to talk to the public internet, you will have to remove both the govee API key and the govee credentials from your configuration so that only the LAN API will be used instead.

Looks like the HA update and the recent update to the MQTT fixed whatever the issue was. Didn’t change anything with the network but it’s working now.

let's close this then!


no mitochondrial genes are included in loom file (bug fixed, velocyto team please check this)

Hello velocyto,
I transfer 10X bam file into loom file by velocyto.py and use Scanpy to do cell clustering. However, when doing the data processing by Scanpy, I found there are no mito genes included in loom file, see below.

There is a similar question raised here (https://github.com/velocyto-team/velocyto.R/issues/96). I understand that "mito genes do not have introns, so no velocyto-type analysis can be done on these genes.----carlosf79". But the data processing by Scanpy is the pre-operation before RNA velocity analysis.
In my understanding, transferring bam file to loom file by velocyto.py is just re-doing the alignment to the reference genome by including the information of spliced and unspliced mRNAs.
If no mito genes were included in loom file, does it mean that the method to make loom file by velocyto is wrong, becasue the strategy of velocyto.py seems like "we check whether the genes have spliced and unspliced mRNAs, if not, removes the genes", rather than "we align the genes, and then check whether the genes have spliced or unspliced mRNAs, if not, mark this gene's spliced and unspliced mRNAs=0". If the first strategy is the truth, it means that velocyto.py removes thousands of genes because only 15-25% genes' unspliced RNAs can be detected in scRNA-seq, which means this strategy is a little biased.
So velocyto generated loom file cannot be used for Scanpy??
Thanks!
Best,
YJ

Hello Velocyto,
When running velocyto.py, it shows:
"WARNING - The .bam file refers to a chromosome 'MT+' not present in the annotation (.gtf) file"
"WARNING - The .bam file refers to a chromosome 'MT-' not present in the annotation (.gtf) file"
BAM file is from 10X generated by Cell Ranger v3.1. Reference genome is refdata-gex-mm10-2020-A.tar.gz from 10X. Repeat annotation file is mm10_rmsk.gtf from UCSC.
Is it because cell ranger v3.1 is too old and doesn't have the same chromosome name as refdata-gex-mm10-2020-A.tar.gz and mm10_rmsk.gtf? But when I check the chromosome names in BAM, I found it is ChrM, the same as refdata-gex-mm10-2020-A.tar.gz and mm10_rmsk.gtf? Then why the hint shows the .bam file refers to a chromosome 'MT-'?
Here is the code to check chromosome names in bam.
$ samtools view /home/hyjforesight/mybam.bam | cut -f3 | perl -ne 'print unless $seen{$_}++' -
chr1
chr10
chr11
chr12
chr13
chr14
chr15
chr16
chr17
chr18
chr19
chr2
chr3
chr4
chr5
chr6
chr7
chr8
chr9
chrM
chrX
chrY

https://github.com/samtools/samtools/issues/1545

@hyjforesight  --- Thank you so much for looking into this!
I noticed your picture above showing the command for samtools, but that is only to view the cellsorted_... bam file after you reran run10x with the new genome annotation gtf file (chrM changed to chrMT).
Would you mind giving me some more instruction on how you used VS Code to change the naming of chrM to chrMT in the reference genome folder, including genome.fa, genome.fa.fai, gene.gtf, chrName.txt, chrNamelength.txt files?
Also, how did you change chrM to chrMT in rmsk.gtf?
Sorry for basic questions. Thank you so much for pointing me here in my issue #320 (I still have issue of it not finding the index file for the cellsorted bam yet it still generated the loom file --- #321 🥵).

Updated:
For (1) --- Use VS Code to Find and Replace. And, yes, there is nothing "chrM" in any other files, e.g., genes.pickle etc.
For (2) --- Do similarly as (1)

Hi @denvercal1234GitHub,
for (3), cellranger 6.1.2 outputs have no outs/analys folder, which is required for run10x command in velocyto.py. so i use this command velocyto run -b filtered_barcodes.tsv -o output_path -m repeat_msk_srt.gtf possorted_genome_bam.bam mm10_annotation.gtf with the new bam file, and chrM replaced gtf.
Thanks!
Best,
YJ

Thank you very much, @hyjforesight! Did you do some additional filtering of the barcodes.tsv? Because usually cellranger will only name it barcodes.tsv, and not filtered_barcodes.
Given the memory limitation, I needed to samtools sort the possorted_.. bam file first before running velocyto run. Do you by chance know if I can still use the index file (bam.bai) that came originally with the possorted_ bam file? Or I will need to index the manually sorted bam?

@denvercal1234GitHub
filtered matrices files are at \SC_RNA_COUNTER_CS\SC_MULTI_CORE\MULTI_GEM_WELL_PROCESSOR\COUNT_GEM_WELL_PROCESSOR_BASIC_SC_RNA_COUNTER\FILTER_BARCODES\fork0\files\filtered_matrices_mex
check the velocyto.py tutorial. no need to sort Cell Ranger generated bam file. it's already sorted. just change the name to cellsorted_yourbamname.bam
The index file is not needed for velocyto.py

Humm... I think the bam file from cellranger is actually only sorted by position, and velocyto then sorts it by cell (CB). That is why it renames it to cellsorted_...bam. That is why if memory is an issue, then it is recommended to cellsorted first then run run10x.

hello @denvercal1234GitHub
Yor are right. cellranger did position sorting. the CB sorting will be done by velocyto. but we use velotcyto command, not velocyto 10x command, which required the filtered barcodes.tsv. so we believe the CB sorting is done internally. To make it sure, let me try CD sorting individually and let you know whether the results are consistent.

updata: use the individual cell sorted bam file generates errors

hope velocyto team can fix this BUG in the next release

Hi,
I rename all chM to chMT in the reference genome folder,and run cellranger count to get a new bam file.
After velocyto,I use scanpy to analyse the loom file,but the gene expression does not match the 10x results well.
There is still some chromosome not found in gtf(refdata-gex-GRCh38-2020-A,download from 10x Genomics ) during velocyto

2022-03-22 11:18:44,717 - WARNING - The .bam file refers to a chromosome 'KI270442.1+' not present in the annotation (.gtf) file
2022-03-22 11:18:44,717 - WARNING - The .bam file refers to a chromosome 'KI270442.1-' not present in the annotation (.gtf) file
2022-03-22 11:18:44,725 - DEBUG - Marking up chromosome KI270729.1
2022-03-22 11:18:44,725 - WARNING - The .bam file refers to a chromosome 'KI270729.1+' not present in the annotation (.gtf) file
2022-03-22 11:18:44,725 - WARNING - The .bam file refers to a chromosome 'KI270729.1-' not present in the annotation (.gtf) file
2022-03-22 11:18:44,726 - DEBUG - Marking up chromosome GL000225.1
2022-03-22 11:18:44,726 - WARNING - The .bam file refers to a chromosome 'GL000225.1+' not present in the annotation (.gtf) file
2022-03-22 11:18:44,726 - WARNING - The .bam file refers to a chromosome 'GL000225.1-' not present in the annotation (.gtf) file
2022-03-22 11:18:44,729 - DEBUG - Marking up chromosome KI270743.1
2022-03-22 11:18:44,729 - WARNING - The .bam file refers to a chromosome 'KI270743.1+' not present in the annotation (.gtf) file
2022-03-22 11:18:44,729 - WARNING - The .bam file refers to a chromosome 'KI270743.1-' not present in the annotation (.gtf) file
2022-03-22 11:18:44,731 - DEBUG - Marking up chromosome GL000009.2
2022-03-22 11:18:44,731 - WARNING - The .bam file refers to a chromosome 'GL000009.2+' not present in the annotation (.gtf) file
2022-03-22 11:18:44,733 - DEBUG - Marking up chromosome KI270722.1
2022-03-22 11:18:44,734 - WARNING - The .bam file refers to a chromosome 'KI270722.1+' not present in the annotation (.gtf) file
2022-03-22 11:18:44,734 - WARNING - The .bam file refers to a chromosome 'KI270722.1-' not present in the annotation (.gtf) file
2022-03-22 11:18:44,734 - DEBUG - Marking up chromosome GL000194.1
2022-03-22 11:18:44,734 - WARNING - The .bam file refers to a chromosome 'GL000194.1+' not present in the annotation (.gtf) file
2022-03-22 11:18:44,834 - DEBUG - Marking up chromosome KI270742.1
2022-03-22 11:18:44,834 - WARNING - The .bam file refers to a chromosome 'KI270742.1+' not present in the annotation (.gtf) file
2022-03-22 11:18:44,834 - WARNING - The .bam file refers to a chromosome 'KI270742.1-' not present in the annotation (.gtf) file
2022-03-22 11:18:44,837 - DEBUG - Marking up chromosome GL000205.2
2022-03-22 11:18:44,837 - WARNING - The .bam file refers to a chromosome 'GL000205.2+' not present in the annotation (.gtf) file
2022-03-22 11:18:44,877 - DEBUG - Marking up chromosome GL000195.1
2022-03-22 11:18:44,901 - DEBUG - Marking up chromosome KI270736.1
2022-03-22 11:18:44,901 - WARNING - The .bam file refers to a chromosome 'KI270736.1+' not present in the annotation (.gtf) file
2022-03-22 11:18:44,901 - WARNING - The .bam file refers to a chromosome 'KI270736.1-' not present in the annotation (.gtf) file
2022-03-22 11:18:44,902 - DEBUG - Marking up chromosome KI270733.1
2022-03-22 11:18:44,902 - WARNING - The .bam file refers to a chromosome 'KI270733.1+' not present in the annotation (.gtf) file
2022-03-22 11:18:44,902 - WARNING - The .bam file refers to a chromosome 'KI270733.1-' not present in the annotation (.gtf) file

Do you have any suggestion?

@AY-LIANG I am not sure, but between which that the gene expression did not match? What do you meant by 10X results?

@denvercal1234GitHub
10x result is the output of CellRanger,which is, these three files
barcodes.tsv.gz  features.tsv.gz  matrix.mtx.gz
I use scanpy to call the highest expression genes,and there are some differences between the loom file and CellRanger result.
It seems that some gene information is missing during velocyto.I guess these warnings might be the reason.

hi @AY-LIANG
we can omit these GL KL chromosomes. it's useless.
velocyto removes some counts, check this https://github.com/theislab/scvelo/discussions/813
I don't use loom file for Scanpy. I use 10X matrics. when I need unspliced and spliced information. I just copy them from loom to the matrix.
ldata = scv.read(filename.loom, cache=True)
adata = scv.utils.merge(adata, ldata)

@hyjforesight
Thanks!It's very helpful

Acoording to this comment, and cellranger has been updated for serveral years, incongruences between .gtf and .bam are now removed, thus this issue may be solved by deleting two line of code of counter.py:
if chrom == "M":
    chrom = "MT"
https://github.com/velocyto-team/velocyto.py/blob/0963dd2d/velocyto/counter.py#L282
https://github.com/velocyto-team/velocyto.py/blob/0963dd2d/velocyto/counter.py#L283

Hi, I rename all chM to chMT in the reference genome folder,and run cellranger count to get a new bam file. After velocyto,I use scanpy to analyse the loom file,but the gene expression does not match the 10x results well. There is still some chromosome not found in gtf(refdata-gex-GRCh38-2020-A,download from 10x Genomics ) during velocyto

2022-03-22 11:18:44,717 - WARNING - The .bam file refers to a chromosome 'KI270442.1+' not present in the annotation (.gtf) file
2022-03-22 11:18:44,717 - WARNING - The .bam file refers to a chromosome 'KI270442.1-' not present in the annotation (.gtf) file
2022-03-22 11:18:44,725 - DEBUG - Marking up chromosome KI270729.1
2022-03-22 11:18:44,725 - WARNING - The .bam file refers to a chromosome 'KI270729.1+' not present in the annotation (.gtf) file
2022-03-22 11:18:44,725 - WARNING - The .bam file refers to a chromosome 'KI270729.1-' not present in the annotation (.gtf) file
2022-03-22 11:18:44,726 - DEBUG - Marking up chromosome GL000225.1
2022-03-22 11:18:44,726 - WARNING - The .bam file refers to a chromosome 'GL000225.1+' not present in the annotation (.gtf) file
2022-03-22 11:18:44,726 - WARNING - The .bam file refers to a chromosome 'GL000225.1-' not present in the annotation (.gtf) file
2022-03-22 11:18:44,729 - DEBUG - Marking up chromosome KI270743.1
2022-03-22 11:18:44,729 - WARNING - The .bam file refers to a chromosome 'KI270743.1+' not present in the annotation (.gtf) file
2022-03-22 11:18:44,729 - WARNING - The .bam file refers to a chromosome 'KI270743.1-' not present in the annotation (.gtf) file
2022-03-22 11:18:44,731 - DEBUG - Marking up chromosome GL000009.2
2022-03-22 11:18:44,731 - WARNING - The .bam file refers to a chromosome 'GL000009.2+' not present in the annotation (.gtf) file
2022-03-22 11:18:44,733 - DEBUG - Marking up chromosome KI270722.1
2022-03-22 11:18:44,734 - WARNING - The .bam file refers to a chromosome 'KI270722.1+' not present in the annotation (.gtf) file
2022-03-22 11:18:44,734 - WARNING - The .bam file refers to a chromosome 'KI270722.1-' not present in the annotation (.gtf) file
2022-03-22 11:18:44,734 - DEBUG - Marking up chromosome GL000194.1
2022-03-22 11:18:44,734 - WARNING - The .bam file refers to a chromosome 'GL000194.1+' not present in the annotation (.gtf) file
2022-03-22 11:18:44,834 - DEBUG - Marking up chromosome KI270742.1
2022-03-22 11:18:44,834 - WARNING - The .bam file refers to a chromosome 'KI270742.1+' not present in the annotation (.gtf) file
2022-03-22 11:18:44,834 - WARNING - The .bam file refers to a chromosome 'KI270742.1-' not present in the annotation (.gtf) file
2022-03-22 11:18:44,837 - DEBUG - Marking up chromosome GL000205.2
2022-03-22 11:18:44,837 - WARNING - The .bam file refers to a chromosome 'GL000205.2+' not present in the annotation (.gtf) file
2022-03-22 11:18:44,877 - DEBUG - Marking up chromosome GL000195.1
2022-03-22 11:18:44,901 - DEBUG - Marking up chromosome KI270736.1
2022-03-22 11:18:44,901 - WARNING - The .bam file refers to a chromosome 'KI270736.1+' not present in the annotation (.gtf) file
2022-03-22 11:18:44,901 - WARNING - The .bam file refers to a chromosome 'KI270736.1-' not present in the annotation (.gtf) file
2022-03-22 11:18:44,902 - DEBUG - Marking up chromosome KI270733.1
2022-03-22 11:18:44,902 - WARNING - The .bam file refers to a chromosome 'KI270733.1+' not present in the annotation (.gtf) file
2022-03-22 11:18:44,902 - WARNING - The .bam file refers to a chromosome 'KI270733.1-' not present in the annotation (.gtf) file

Do you have any suggestion?

Hi @AY-LIANG, I know this post is quite old, but I just want to add that this warning happens because the code for masking will check if every read in bam file and see if the chromosome it is mapped is in the gtf file or not. In the case of reads mapped to regions like GL000194.1, it is in fasta file, but not gtf file, so it will generate the warning.


Update to LiveView 0.15+

Noticed that the library is not compatible with 0.15+ version of LV due to number of changes to pubsub and LV itself. Not sure what is the minimum version needed setting it 0.15 for now.

I updated the library to LiveView 1.15.7 before I saw this PR. Can we close this PR or are you missing something from the main repo?

Do you want to replace Poison with Jason as I did here ? It seem to be the community way forward as it's superior parser.

Sounds good! Will do.

Poison was replaced with Jason in the main repo


Upgrade compilesdkVersion Android to 29

compile and target sdk upgrade to 29.
Solves Issue #156

I've tried myself and it does indeed work. Good job ;)

is it still working? Tried and failed, it says using deprecated version of Android.


Change props of previous view

Version

react-native-router-flux v3.?.?
react-native v0.24.1

Expected behaviour
I'm doing this:
<Scene key="shop" type="replace">
    <Scene key="home" component={Home} title="Home"/>
    <Scene key="products" component={Products} />
</Scene>

OnPress on a Button in "Home"-View I push data and go to "Products"-View. I want to change in my "Products"-View the props of my "Home"-View on button press.
For example:
Home.js has a state loader: true. I want to change the state in my Products.js to loader: false. If I press the < Button on the top left and I'm on my "Home"-View I want that my state loader is set to false.
Is this possible? Let me know if you need more informations for this issue(?).
Thank you a lot!
albo1337

Is that even possible?
kind regards
albo1337

There is different ways to solve this issue, I recommend you check other people issues (open/closed) about custom navigation bar, and/or integration with flux/redux.

I don't have problems with my navigation bar. I want to change props of my root scene while being on a child scene.
Kind regards and thank you for your response

Have you tried Actions.refresh({key:SCENE_KEY, ..data}) ?

I'm actually trying to change the props of my parent view while being on a child view.
My problem is, that if I go to a child view from a parent view, my parent view is still running. If I could set it onPause state (or something like that) while being on child view my problem would get solved...

http://stackoverflow.com/questions/37180919/change-props-of-parent-view-in-react-native
Here I have opened a question where I explain it with more details.
Thanks for your respond @aksonov

I'll close this issue since it was answered on Stack Overflow.


What's the difference between Eval and Compile Run?

Eval consists of two steps: Compile and Run.
If your expression is not changed frequently, you can precompile it and keep in memory resulting program. This will save some time on runtime.

Eval

Compile

Parse
Type check
Compile to bytecode

Run bytecode


ShaderMaterial: keep uniqueId in serialize()

Like Material here: https://github.com/BabylonJS/Babylon.js/blob/c654c7303965a43f340f8c1e0eab3fc804c566f1/packages/dev/core/src/Materials/material.ts#L1719
Related commit: https://github.com/BabylonJS/Babylon.js/commit/738236a7578d1c6dfcae4f93cfffeb768ea5a826

I think this is a bug in the material class, uniqueId is a session-based unique id, (at least it used to be), so should not be serialized at all

They are not session based now :) so this PR is good to go


Paypal checkout button on Cart skips Confirmation page

I tried changing the options, but it still skips that last step for confirming when the user clicks on Continue on the Address page.
Is there anyway that I can bring back the confirmation page, in the case paypal checkout button was used?

@damianlegawiec it is not a bug, this was removed deliberately to avoid overriding checkout flow (in commit https://github.com/spree-contrib/spree_braintree_vzero/commit/917db8b5641c84dcbf39e032c5b6a550c4fcb291)
@0bserver07
You have to remove this from your checkout flow:
https://github.com/spree/spree/blob/master/core/app/models/spree/order.rb#L52
and it should work.

@damianlegawiec @Mafi88  I tried removing the line you referred to above and it results in Spree losing the value of the transaction (it becomes zero) on the confirmation page. The order still goes through, but then there is also no transaction ID on the backend to process. Any ideas?

This explanation given here may help:
https://github.com/spree-contrib/spree_braintree_vzero/issues/81


White Screen?

Hi Dustin,
love the design and the idea.
I bought all the products you recommended, pcb from pcbway ILI9488 screen and ESP32-WROOM-32D and ive followed your youtube tutorial https://www.youtube.com/watch?v=s2X4BQ9VmEU&t=515s&ab_channel=DustinWatts.
I've got an issue which im sure is on my end with my config.
I have a white screen it strobes a little bit my user_setup.h is exactly as the video any suggestions on how to fix?

I assume this issue is resolved after so many months, so closing it.

Bonjour
impossible de faire la calibration l'écran reste figé sur Touch corners as indicated
ILI9341_DRIVER
merci


[ENHANCEMENT] Package installed indicator.

Please confirm these before moving forward

[X] I have searched for my feature proposal and not found a work-in-progress/duplicate/resolved/discarded issue.
[X] This improvements refers to an existing feature. If you want to suggest a new feature, please use this template
[X] This improvements is not a bug. If you want to report a bug, please use this template

Describe the improvement
Having the installation status could help avoid confusion when installing a package without having to check whether it is already installed.
Describe how this improvement could help users.
Knowing whether a package is already installed could save you the eventual reinstallation, or make sure it isn't already installed and can improve the accessibility. The image below gives a brief idea of what this might look like.

I tried a system of labeling, but it turned out to be so laggy that it was unusable. I like this approach of the icons

I use a label from Microsoft fluent icon's font

Tooltips and colored icons will do the job.


Error: Page does not load if AUTH_USER_MODEL does not have id field

In my Django project I have overridden AUTH_USER_MODEL so that it its id field is renamed to person_id.
This causes the line 90 in explorer/views/list.py to break:
query_favorites_for_user = QueryFavorite.objects.filter(user_id=self.request.user.id).values_list

The line should be updated so that user.pk would be used instead of user.id.
query_favorites_for_user = QueryFavorite.objects.filter(user_id=self.request.user.pk).values_list

Since pk refers to the models's primary key regardless of what the actual field name is, it is guaranteed to always be present.

Yeah, valid point 👍
Happy to accept a PR on this.

PR is here: https://github.com/groveco/django-sql-explorer/pull/544


Pen + Issues with transparency

Transparent pngs have a white background, and the transparency value doesn't really work.

What extension was this in?

Pen+

@David-Orangemoon

It's because there is no background its wierd  because I though webgl would mix it in with the black bg but I am adding a fix for that next update.

#535 Resolved this.


Main Window: Fix "Quit Spyder" action on OS X

Fixes #3288

We had a couple of actions that contained the strings "quit" and
"exit" in their names, and that was interfering with the "Quit
Spyder" action.
That's because Qt creates automatically that action based
on any action in the main application menus that contains those
substrings.

@goanpeca, just FYI in case you face something like this in Navigator in the future. Another crazy Qt issue on Mac!!

And it took almost three hours to figure this one out ;-p

Thanks for the heads up!


Previous cookiecutter lost git history. Try again with Chris M.

See also

#655

[ ] Existing module paths won't be the same unless addressed
[ ] Existing CLI tools and utility code won't work until addressed (placed back in a publishable directory, exposed under the module path that existing users expect)

Mike N uses XXX
May require adding some package dependencies back in
Is this an opportunity to understand how to add utilities to the published package in general?

[ ] Existing GH raw/download URLs (for YAML, JSON schema, etc.) won't unless addressed

Currently only generating a few artifacts... should document the performance and outcome of the others
Is this a good opportunity to retry linkml-sqldb dump? with some of the exhaustive example data files?

excel
graphql
jsonld
jsonldcontext
jsonschema
markdown
owl
prefixmap
proto
python
shacl
shex
sqlddl


Make TPE import data more robust

Description
The exported data format and/or resuming metrics format has changed sometime before and they are mismatch now.
I really don't want to investigate into those messy stuffs. So make TPE tolerate any metrics format.
It seems this problem applies to other tuners as well. But I don't want to touch them before a systematic refactor.
Checklist

test case
doc

How to test
Run an HPO experiment. Export then import.

better to pick the logic of data format out of import_data, leave to the next pr


Is axios necessary?

We have the one file issuing requests other than the API itself https://github.com/qdrant/qdrant-web-ui/blob/5444af009a0d174bb1f2cc89049d51152ecbdd1b/src/components/CodeEditorWindow/config/RequesFromCode.js#L4, what's the benefit of using axios and thus loading more KBs on user's browsers when the same could be done with window.fetch()?

Axios has much more functionalities and it is scalable.


Attempt to fetch one chunk prior on seeking

As per issue #330, manifests are only required to be approx.
correct with respect to their description of the media segments.
In order to avoid stalling when seeking through content like this,
fetch the chunk prior to the one that supposedly contains the
current time.
This is an attempt to apply a similar fix to Ia7bb74d0120.

I'm not convinced this is ideal or complete but I'd like to see if we can back port this fix to 1.6.x because we will still be relying on 1.6.x for some time and this is causing problems for us. Notably I'm not sure how to properly handle the nudgeNeeded issue. If I take it out then seeking seems to work fine for me but I'm not sure how to reproduce the other conditions that require this nudge.

The nudge was required on older versions of Chromium.  They have since fixed the bug on their end, so we got rid of the nudge in Shaka v2.  Let me discuss with the team.  It may be cleaner to have you completely remove the nudge in a separate PR first.

Can you rebase on top of the other PR and resolve conflicts?


Create Project to track status of Windows impementation

Hey, it would be great, if there was a project here https://github.com/ROCmSoftwarePlatform/MIOpen/projects?type=classic for windows support aswell. I can assume some are waiting to use it and it would be great to know, how much work still needs to be done and how much has already been done to implement it.

Done. Next step is to associate all Windows Enablement related PRs to that project :)

Done. Next step is to associate all Windows Enablement related PRs to that project :)

Thanks for that :D It´ll definitely help people, once all the related PRs are associated

@junliume Will (or better when will) the PRs get associated with the Project?


[Adaptive Card: Link]: Screen reader is announcing link and button both while keyboard focus lands on actionable item 'Publish adaptive card schema'.

https://microsoft.visualstudio.com/OS/_workitems/edit/36485740

possible invalid test case since it's using unexpanded template for testing

This is an authoring issue. The text in the TextBlock is a link - "text": "${Title}" while at the same time there is a "selectAction" put on the same column - "selectAction": {"type": "Action.Submit",...}.
The link on the text causes a link to be created and the selectAction on the column causes the button to be created. Pick the one to keep, remove the other and the issue will be fixed.


allow html

allow html to be inserted into default toast component when enabled

not doing this. easy to replace toast component

@scttcper can you explain this? i want to put html inside the toast

@OscarGalindo you can check out the (pink toast in the demo)[https://github.com/scttcper/toastr-ng2/blob/master/src/app/pink.toast.ts].
It inherits the controller from the original toast and only replaces the aesthetics. You'll need to pass this component to the individual toast options and it will need to be declared and an entry component in your module https://github.com/scttcper/toastr-ng2/blob/master/src/app/app.module.ts#L19

You can also pass it as a global option when you setup the toastr


Help output format error

When communicating with hubot through hipchat, I sent the 'hubot help' command and the format is very messy, I believe due to hubot replacing a '>' in the help text with '>'. It would be nice to clean this up a bit but I had trouble figuring out how.
Output from help command :
Events:
debug - {user: <user object to send message to>}
HuBro <user> doesn't have <role> role - Removes a role from a user
HuBro <user> has <role> role - Assigns a role to a user
HuBro <user> is a badass guitarist - assign a role to a user
HuBro <user> is not a badass guitarist - remove a role from a user
HuBro animate me <query> - The same thing as `image me`, except adds a few parameters to try to return an animated GIF instead.
HuBro die - End HuBro process
HuBro echo <text> - Reply back with <text>
HuBro fake event <event> - Triggers the <event> event for debugging reasons
HuBro help - Displays all of the help commands that HuBro knows about.
HuBro help <query&g
t; - Displays all help commands that match <query>.
HuBro image me <query> - The Original. Queries Google Images for <query> and returns a random top result.
HuBro map me <query> - Returns a map view of the area returned by `query`.
HuBro mustache me <query> - Searches Google Images for the specified query and mustaches it.
HuBro mustache me <url> - Adds a mustache to the specified URL.
HuBro ping - Reply with pong
HuBro pug bomb N - get N pugs
HuBro pug me - Receive a pug
HuBro show storage - Display the contents that are persisted in the brain
HuBro show users - Display all users that HuBro knows about
HuBro the rules - Make sure HuBro still knows the rules.
HuBro time - Reply with current time
HuBro translate me <phrase> - Searches for a translation for the <phrase> and then prints that bad boy out.
HuBro translate me from <source> into <target> <phrase> - Translates <phrase> from <source> into <target>. Both <source> and <target> are optional
HuBro what role does <user> have - Find out what roles are assigned to a specific user
HuBro who has admin role - Find out who's an admin and can assign roles
HuBro who is <user> - see what roles a user has
HuBro youtube me <query> - Searches YouTube for the query and returns the video embed link.
ship it - Display a motivation squirrel

I can't reproduce this using the latest hubot/hubot-hipchat combo.  e.g.
Hubot help <query> - Displays all help commands that match <query>.


Mbuild integration

Description
Added in a few simple routines to convert from the System class to an mBuild Compound.
Todos
Notable points that this PR has either accomplished or will accomplish.

[ X] Add in functions to convert to mbuild Compound
[X] preserve basic hierarchy of System class when converting
[X] updated documentation

Status

[X ] Ready to go

Codecov Report

Merging #2 (8c6964b) into main (3b02d80) will increase coverage by 0.37%.
The diff coverage is 100.00%.

Additional details and impacted files

CI on windows is failing during package setup (likely due to including mbuild in the environment yaml).  I disabled the windows testing as failure in that was causing the other tests to abort.


Support ES6

Are you going to rewrite your library to ES6 syntax using classes?

@siarheipashkevich then we would have to use a transpiler, to make this code work in older browsers.
Or do you just want to write code in this lib in ES6 and ship it transpiled?

I'm willing to write this lib in ES6. Ideas on a roadmap?

@peter-pogocarshare I am open to suggestions :-)

My use-case is for React-Native so I can start by writing this es6. We can figure how to ship it after?

@peter-pogocarshare I would prefer if we built this project in ES6 but distributed in ES5, for this we would need to have babel transpiler and prepublish npm hook.
May be I am gonna some free time this weekends.


Save autosave is not working

We have written the needed data into your clipboard because it was too large to send. Please paste.On Ubuntu 19.04. Issue with saving changes in documents to the point that I can't use VSCode. When I try to save a document  I get: " Failed to save 'index.js': The content of the file is newer. Please compare your version with the file contents."
Unistalled all the git extentions at this point issue remains.
Autosave settings don't work. When autosave is  on, I can't type. The cursor jumps to the end of the document after each character. Also solograph and ruby lang support + rubocope extensions are not working ...

/needsMoreInfo


[Login page form layout plus the responsiveness is not aligned]

What happened?
Image and form should have the ratio specified as 3:7 so that it looks good even if  it gets to the minimized scrolling window and even at the current layout the image is bigger in height ration because of which the size of the window get expanded.
I would like to propose a new view of the current scenario

How can we reproduce this bug?
There is no css given for the image,
We can implement that and add flexing to both img and form as it is defined in divs and can align them to  create responsiveness
Desktop Information (Optional)
No response
Urgency (Optional)
None

@agamjotsingh18 can i start working on this issue

Sure @shuddhi-jain


Update documentation and turn off notGCed by default.

Also, this PR adds streetsidesoftware.code-spell-checker as recommended extension and adds initial dictionary of the repo-specific exceptions.

I added a comment on the release mode bug for this, so that can just be handled separately. My concerns have been addressed, but I think @kenzieschmoll or @devoncarew should probably LGTM this.


NiFi-924: Camel integration

working version of nifi-camel integration.
It Supports to load camel context from File, Classpath as well as to define and modify Routes on the fly.
Overall it's an implementation of feature request NIFI-924 [https://issues.apache.org/jira/browse/NIFI-924#]
Sample NiFi Template:
CamelProcessorTestingTemplate.xml.zip
Over and above all, this component will mitigate the need of creating  many Processors as camel integrate excellently with a lot of other products, protocols and standards available in the market, e.g. activemq, rabitmq, aws, rules engine, rest services etc. A larger list of available components/protocols is available at :

http://camel.apache.org/component-list-grouped.html

http://camel.apache.org/components.html

Commented on NIFI-924 to reference https://issues.apache.org/jira/browse/NIFI-1571 and consider this as resolved by that.

@joewitt  @olegz Thank you all and sorry for the delay in reply, was dead busy.


Schema - Flow: icon "AzureDataExplorer" does not show up

According to https://github.com/microsoft/BotFramework-Composer/blob/main/docs/ui-schema.md#header-widgets
Fluent icon name. List of available icons.
Seems not all available icon can show up, e.g., the following AzureDataExplorer does not show up.
    "flow": {
        "widget": "ActionCard",
        "body": {
            "widget": "ResourceOperation",
            "operation": "Host",
            "resource": "=coalesce(action.skillEndpoint, \"?\")",
            "singleline": true
        },
        "header": {
            "widget": "ActionHeader",
            "colors": {
                "theme": "#2F2C69",
                "color": "#FFFFFF",
                "icon": "#FFFFFF"
            },
            "icon": "AzureDataExplorer"
        },
        "footer": {
            "widget": "PropertyDescription",
            "property": "=action.resultProperty",
            "description": "= Result"
        },
        "hideFooter": "=!action.resultProperty"
    }

@a-b-r-o-w-n Could you take a look at this ticket?

@dawwa We actually only have support for a subset of the icons at https://uifabricicons.azurewebsites.net/. I just became aware of this. I will update the docs accordingly.
The list of icons that are supported can be found here: https://developer.microsoft.com/en-us/fluentui#/styles/web/icons


MINOR: More efficient midpoint calc for AbstractIndex

retest this please

retest this please

@cmccabe I pushed a commit. If that looks good to you, you can merge.

retest this please

retest this please

retest this please

This PR causes LogCleanerTest .testRecoveryAfterCrash to fail in an infinite loop in binarySearch with lo=15 and hi=16.

Fixed via https://github.com/apache/kafka/pull/7702, thanks @hachikuji


Hour Strike App not working :(

Hi! Thanks for the app, but for some reason it doesn't work for me on Bangle.js.
Any idea why? Could there be conflicts with other apps?

Strange. Let me take a look perhaps during the weekend! Thanks for reporting.

Looks like it is indeed an app conflict issue. The app stopped working for me from time to time. I have a bunch of apps (boo loaders, clocks, and more) installed. When I reset the watch and installed some apps back, it seems to work just fine.
Could you let me know a list of your apps? Maybe I can try to re-create the situation that led to its malfunctioning. Or at least, pin point the app that breaks it.

The app is now working!
What I did: I updated the firmware to 2v11 and applications requiring an update (including Bootloader). But it started working specifically after updating the applications on Bangle.js
Thanks again for the useful app :)


Externally Visible Server

Codecov Report

Merging #6264 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #6264   +/-   ##
=======================================
  Coverage   77.02%   77.02%
=======================================
  Files          64       64
  Lines        9504     9504
=======================================
  Hits         7320     7320
  Misses       2184     2184

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update da24b0e...71d1d37. Read the comment docs.


docs: access custom props for a NuxtLayout

I documented how one can access props passed to a NuxtLayout, in the layout

🔗 Linked issue

❓ Type of change

[x] 📖 Documentation (updates to the documentation, readme or JSdoc annotations)
[ ] 🐞 Bug fix (a non-breaking change that fixes an issue)
[ ] 👌 Enhancement (improving an existing functionality like performance)
[ ] ✨ New feature (a non-breaking change that adds functionality)
[ ] 🧹 Chore (updates to the build process or auxiliary tools and libraries)
[ ] ⚠ Breaking change (fix or feature that would cause existing functionality to change)

📚 Description

📝 Checklist

[ ] I have linked an issue or discussion.
[x] I have updated the documentation accordingly.

Nice work, thank you ❤


Setting the getBody function approach.

Codecov Report

Merging #151 into master will decrease coverage by 0.05%.
The diff coverage is 25%.

@@            Coverage Diff             @@
##           master     #151      +/-   ##
==========================================
- Coverage   62.63%   62.58%   -0.06%
==========================================
  Files         100      100
  Lines        9504     9515      +11
==========================================
+ Hits         5953     5955       +2
- Misses       2911     2919       +8
- Partials      640      641       +1

Impacted Files
Coverage Δ

wrp/wrphttp/fanout.go
93.52% <25%> (-6.48%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 900be56...5704e87. Read the comment docs.


Test against more Django and Python versions

Test against Django 2.1 & 2.2
Install Django in tox environment from package rather than git archive
Update default Python version in Travis to 3.7
Test against all supported versions of Python for latest Django LTS release
Slight Travis file cleanup (move matrix things all to one place, jobs is an alias for matrix)

This is a large-ish change to the Travis file I've made without any consultation, so I'd be totally OK if you wanted me to make fewer changes or to pick out some of these changes yourself.

Codecov Report

Merging #215 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #215   +/-   ##
=======================================
  Coverage   73.92%   73.92%
=======================================
  Files          32       32
  Lines         349      349
=======================================
  Hits          258      258
  Misses         91       91

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0ac7282...fa59038. Read the comment docs.

@JayH5 thanks for the contribution. Good idea, I didn't know that we haven't updated the test suite in so long. Please drop 1.11, since 2.2 is a new LTS. I will issue a new major release dropping 1.11 support.
I am a bit confused, that you didn't run the test suite in all Python version. Is there any particular reason. It would seem plausible to me, to use the Django/Python version combinations that are actually supported by Django.

@codingjoe

Please drop 1.11, since 2.2 is a new LTS. I will issue a new major release dropping 1.11 support.

Done.

I am a bit confused, that you didn't run the test suite in all Python version. Is there any particular reason.

Just didn't want to create too many build jobs. Since this is now only tested against two Django versions, testing against all supported Python versions definitely makes sense.

@JayH5 nudge to resolve conflicts and a 👍 for the work

I will close this one, since its outdated. We already test the current master against, Py37 and Django 2.2.


Documentation: Update example in writing-custom-queries.md to include…

… timestamps.

might be worthwhile explaining why spec.inputs is necessary for the period_end, period_start columns in the schema definition.

@timflannagan1 This looks great. Can you squash your commits then I'll approve it.

@chancez done

/lgtm
/approve

/retest

/override ci/prow/metering-e2e-aws

/override ci/prow/e2e-aws-metering


[CNNArticle] Unable to extract cnn url

Checklist

[X] I'm reporting a broken site
[X] I've verified that I'm running yt-dlp version 2022.02.04. (update instructions)
[X] I've checked that all provided URLs are alive and playable in a browser
[X] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[X] I've searched the bugtracker for similar issues including closed ones. DO NOT post duplicates
[X] I've read the guidelines for opening an issue
[X] I've read about sharing account credentials and I'm willing to share it if required

Region
No response
Description
Can't download videos from CNN anymore. e.g.:
https://www.cnn.com/travel/article/parrot-steals-gopro-scli-intl
Verbose log
[debug] Command-line config: ['-vU', 'https://www.cnn.com/travel/article/parrot-steals-gopro-scli-intl']
[debug] Encodings: locale cp1252, fs utf-8, out utf-8 (No ANSI), err utf-8 (No ANSI), pref cp1252
[debug] yt-dlp version 2022.02.04 [c1653e9] (win_exe)
[debug] Python version 3.8.10 (CPython 64bit)
[debug] exe versions: ffmpeg git-2020-07-30-134a48a
[debug] Optional libraries: Cryptodome, mutagen, sqlite, websockets
[debug] Proxy map: {}
Latest version: 2022.02.04, Current version: 2022.02.04
yt-dlp is up to date (2022.02.04)
[debug] [CNNArticle] Extracting URL: https://www.cnn.com/travel/article/parrot-steals-gopro-scli-intl
[CNNArticle] parrot-steals-gopro-scli-intl: Downloading webpage
ERROR: [CNNArticle] Unable to extract cnn url; please report this issue…
  File "yt_dlp\extractor\common.py", line 612, in extract
  File "yt_dlp\extractor\cnn.py", line 145, in _real_extract
  File "yt_dlp\extractor\common.py", line 1195, in _html_search_regex
  File "yt_dlp\extractor\common.py", line 1186, in _search_regex

bump... just came across another example:
URL:  https://www.cnn.com/2018/05/21/entertainment/jada-pinkett-hair-loss/index.html

yt-dlp.py -v "https://www.cnn.com/2018/05/21/entertainment/jada-pinkett-hair-loss/index.html"
[debug] Command-line config: ['-v', 'https://www.cnn.com/2018/05/21/entertainment/jada-pinkett-hair-loss/index.html']
[debug] Encodings: locale cp1252, fs utf-8, out utf-8, err utf-8, pref cp1252
[debug] yt-dlp version 2<IP_ADDRESS> [c0c2c57d3] (zip)
[debug] Python version 3.6.6 (CPython 64bit) - Windows-10-10.0.14393-SP0
[debug] exe versions: ffmpeg 4.3.2-2021-02-02-full_build-www.gyan.dev, ffprobe 4.3.2-2021-02-02-full_build-www.gyan.dev, rtmpdump 2.4
[debug] Optional libraries: sqlite
[debug] Proxy map: {}
[debug] [CNNArticle] Extracting URL: https://www.cnn.com/2018/05/21/entertainment/jada-pinkett-hair-loss/index.html
[CNNArticle] index.html: Downloading webpage
ERROR: [CNNArticle] Unable to extract cnn url; please report this issue on  https://github.com/yt-dlp/yt-dlp , filling out the "Broken site" issue template properly. Confirm you are on the latest version using  yt-dlp -U
File "C:\Transmogrifier\yt-dlp.py\yt_dlp\extractor\common.py", line 617, in extract
ie_result = self._real_extract(url)
File "C:\Transmogrifier\yt-dlp.py\yt_dlp\extractor\cnn.py", line 145, in _real_extract
cnn_url = self._html_search_regex(r"video:\s*'([^']+)'", webpage, 'cnn url')
File "C:\Transmogrifier\yt-dlp.py\yt_dlp\extractor\common.py", line 1201, in _html_search_regex
res = self._search_regex(pattern, string, name, default, fatal, flags, group)
File "C:\Transmogrifier\yt-dlp.py\yt_dlp\extractor\common.py", line 1192, in _search_regex
raise RegexNotFoundError('Unable to extract %s' % _name)

Thanks
Ringo

Hello, i'm have the extractor that can download OP url however it didn't download the url in test and this url. I always get error for this kind url (in the browser)

Error Message
There was an issue with accessing the network for media playback. The sub-error code will contain what specific resource was unable to be accessed over the network. | NETWORK_ERROR encountered, code: 1400

In dev tools, it said CORS Missing Allow Origin for this kind of url manifest. Using media url in json give 403 Error. I appreciated if someone can confirm this url or http://www.cnn.com/2014/12/21/politics/obama-north-koreas-hack-not-war-but-cyber-vandalism/ works

and this url. I always get error for this kind url (in the browser)

It plays in browser for me

The webpage JS makes an API call to:
https://fave.api.cnn.io/v1/video?id=world/2022/02/04/parrot-steals-gopro-new-zealand-lon-orig-na.cnn&customer=cnn&edition=domestic&env=prod
which contains the metadata and format info. The ID (world/2022/02/04/parrot-steals-gopro-new-zealand-lon-orig-na.cnn) can be found in the JSON LD block.
Same M.O. for the webpage from #6167, API call to:
https://fave.api.cnn.io/v1/video?id=health/2021/06/18/how-to-improve-your-breathing-lbb-orig.cnn&customer=cnn&edition=domestic&env=prod
with the ID found in JSON LD

With this information which URL would I use to donwload the ID?

The CNN extractors seem to be generally broken (yt-dl is essentially the same, but lacks CNNIndonesiaIE).

CNNIE uses the XML API inherited from TurnerBaseIE but it produces manifest URLs that mostly give 400 Bad Request with the resulting media links giving 403. Presumably the JSON API above should be used instead.
Applying the logic used in TurnerBaseIE._extract_cvp_info() to the results from the JSON API also throws up 400 errors, resulting from _extract_f4m_formats() and _extract_akamai_formats(). Are these methods now obsolete, with the demise of Flash?
The inherited _extract_timestamp() method was disabled, presumably because the base class method failed to to scale the epoch value from ms to s.
The extractor did not detect expired URLs that silently redirect to a home page (don't bother with 404, will you, CNN).
money.cnn.com doesn't seem to have any videos now: they're under edition.cnn.com/business/ instead.
It's not clear whether there should still be any dependency on TurnerBaseIE (and so AdobePassIE) but perhaps this helps NA cable TV subscribers in some way.

Taking most of those into account gives something like this (tested in yt-dl but  making the canonical changes):
--- old/yt_dlp/extractor/cnn.py
+++ new/yt_dlp/extractor/cnn.py
@@ -1,6 +1,22 @@
+import re
+
 from .common import InfoExtractor
 from .turner import TurnerBaseIE
-from ..utils import merge_dicts, try_call, url_basename
+from ..utils import (
+    determine_ext,
+    extract_attributes,
+    ExtractorError,
+    float_or_none,
+    HEADRequest,
+    int_or_none,
+    parse_duration,
+    strip_or_none,
+    traverse_obj,
+    update_url_query,
+    url_basename,
+    url_or_none,
+)
+from ..compat import compat_urlparse

 class CNNIE(TurnerBaseIE):
@@ -18,7 +34,7 @@
             'duration': 135,
             'upload_date': '20130609',
         },
-        'expected_warnings': ['Failed to download m3u8 information'],
+        'skip': 'Redirect to home: content expired?',
     }, {
         'url': 'http://edition.cnn.com/video/?/video/us/2013/08/21/sot-student-gives-epic-speech.georgia-institute-of-technology&utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+rss%2Fcnn_topstories+%28RSS%3A+Top+Stories%29',
         'md5': 'b5cc60c60a3477d185af8f19a2a26f4e',
@@ -29,18 +45,18 @@
             'description': "A Georgia Tech student welcomes the incoming freshmen with an epic speech backed by music from \"2001: A Space Odyssey.\"",
             'upload_date': '20130821',
         },
-        'expected_warnings': ['Failed to download m3u8 information'],
+        'skip': 'Redirect to home: content expired?',
     }, {
         'url': 'http://www.cnn.com/video/data/2.0/video/living/2014/12/22/growing-america-nashville-salemtown-board-episode-1.hln.html',
-        'md5': 'f14d02ebd264df951feb2400e2c25a1b',
+        'md5': '5aa0b7170c275d9772e1cfa5c29a6f5f',
         'info_dict': {
             'id': 'living/2014/12/22/growing-america-nashville-salemtown-board-episode-1.hln',
             'ext': 'mp4',
             'title': 'Nashville Ep. 1: Hand crafted skateboards',
             'description': 'md5:e7223a503315c9f150acac52e76de086',
+            'timestamp':<PHONE_NUMBER>,
             'upload_date': '20141222',
         },
-        'expected_warnings': ['Failed to download m3u8 information'],
     }, {
         'url': 'http://money.cnn.com/video/news/2016/08/19/netflix-stunning-stats.cnnmoney/index.html',
         'md5': '52a515dc1b0f001cd82e4ceda32be9d1',
@@ -55,6 +71,7 @@
             # m3u8 download
             'skip_download': True,
         },
+        'skip': 'Redirect to home: content expired?',
     }, {
         'url': 'http://cnn.com/video/?/video/politics/2015/03/27/pkg-arizona-senator-church-attendance-mandatory.ktvk',
         'only_matching': True,
@@ -70,6 +87,7 @@
         # http://edition.cnn.com/.element/apps/cvp/3.0/cfg/spider/cnn/expansion/config.xml
         'edition': {
             'data_src': 'http://edition.cnn.com/video/data/3.0/video/%s/index.xml',
+            'json_src': 'https://fave.api.cnn.io/v1/video?id=%s&customer=cnn&edition=domestic&env=prod',
             'media_src': 'http://pmd.cdn.turner.com/cnn/big',
         },
         # http://money.cnn.com/.element/apps/cvp2/cfg/config.xml
@@ -79,17 +97,169 @@
         },
     }

-    def _extract_timestamp(self, video_data):
-        # TODO: fix timestamp extraction
-        return None
+    def _extract_f4m_formats(self, *args, **kwargs):
+        return []
+
+    def _extract_akamai_formats(self, *args, **kwargs):
+        return []
+
+    def _extract_cvp_info(self, url, video_id, path_data={}, ap_data={}, fatal=False):
+
+        video_data = self._download_json(url, video_id)
+        urls = set()
+        formats = []
+        thumbnails = []
+        subtitles = {}
+        VIDEO_NAME_RE = r'(?P<width>[0-9]+)x(?P<height>[0-9]+)(?:_(?P<bitrate>[0-9]+))?'
+        # Possible formats locations: files/fileUri, groupFiles/.../files/fileUri
+        # and maybe others
+        for video_url in traverse_obj(
+                video_data,
+                ((('groupFiles', Ellipsis), None), 'files', Ellipsis, 'fileUri'),
+                expected_type=lambda x: x if '/' in x else None):
+            video_id = video_data.get('id') or video_id
+            if video_url.startswith('/mp4:protected/'):
+                continue
+                # TODO Correct extraction for these files
+                # protected_path_data = path_data.get('protected')
+                # if not protected_path_data or not rtmp_src:
+                #     continue
+                # protected_path = self._search_regex(
+                #     r'/mp4:(.+)\.[a-z0-9]', video_url, 'secure path')
+                # auth = self._download_webpage(
+                #     protected_path_data['tokenizer_src'], query={
+                #         'path': protected_path,
+                #         'videoId': content_id,
+                #         'aifp': aifp,
+                #     })
+                # token = xpath_text(auth, 'token')
+                # if not token:
+                #     continue
+                # video_url = rtmp_src + video_url + '?' + token
+            elif video_url.startswith('/secure/'):
+                secure_path_data = path_data.get('secure')
+                if not secure_path_data:
+                    continue
+                video_url = self._add_akamai_spe_token(
+                    secure_path_data['tokenizer_src'],
+                    secure_path_data['media_src'] + video_url,
+                    video_id, ap_data)
+            v_url = url_or_none(video_url)
+            if (not v_url) and video_url.startswith('/'):
+                video_url = traverse_obj(
+                    path_data, ('default', 'media_src'),
+                    expected_type=lambda x: url_or_none('%s%s' % (x, video_url)))
+            if (not video_url) or video_url in urls:
+                continue
+            urls.add(video_url)
+            ext = determine_ext(video_url)
+            if ext in ('scc', 'srt', 'vtt'):
+                subtitles.setdefault('en', []).append({
+                    'ext': ext,
+                    'url': video_url,
+                })
+            elif ext == 'png':
+                thumbnails.append({
+                    # 'id': format_id,
+                    'url': video_url,
+                })
+            elif ext == 'smil':
+                formats.extend(self._extract_smil_formats(
+                    video_url, video_id, fatal=False))
+            elif re.match(r'https?://[^/]+\.akamaihd\.net/[iz]/', video_url):
+                formats.extend(self._extract_akamai_formats(
+                    video_url, video_id, {
+                        'hds': path_data.get('f4m', {}).get('host'),
+                        # nba.cdn.turner.com, ht.cdn.turner.com, ht2.cdn.turner.com
+                        # ht3.cdn.turner.com, i.cdn.turner.com, s.cdn.turner.com
+                        # ssl.cdn.turner.com
+                        'http': traverse_obj(
+                            path_data, ('default', 'media_src'),
+                            expected_type=lambda x: compat_urlparse.urlparse(x).host),
+                    }))
+            elif ext == 'm3u8':
+                m3u8_formats = self._extract_m3u8_formats(
+                    video_url, video_id, 'mp4',
+                    m3u8_id='hls', entry_protocol='m3u8_native',
+                    fatal=False)
+                if '/secure/' in video_url and '?hdnea=' in video_url:
+                    for f in m3u8_formats:
+                        f['_seekable'] = False
+                formats.extend(m3u8_formats)
+            elif ext == 'f4m':
+                formats.extend(self._extract_f4m_formats(
+                    update_url_query(video_url, {'hdcore': '3.7.0'}),
+                    video_id, f4m_id='hds', fatal=False))
+            else:
+                f = {
+                    'url': video_url,
+                    'ext': ext,
+                }
+                mobj = re.search(VIDEO_NAME_RE, video_url)
+                if mobj:
+                    f.update({
+                        'format_id': mobj.group(),
+                        'width': int(mobj.group('width')),
+                        'height': int(mobj.group('height')),
+                        'tbr': int_or_none(mobj.group('bitrate')),
+                    })
+                formats.append(f)
+
+        sttl_fmts = {
+            'scc': 'scc',
+            'webvtt': 'vtt',
+            'smptett': 'tt',
+        }
+        for source in traverse_obj(
+                video_data,
+                ('closedCaptions', 'types',
+                 lambda _, v: v['format'] in sttl_fmts)):
+            track = traverse_obj(source, 'track', expected_type=dict)
+            track_url = url_or_none(track.get('url'))
+            if not track_url or track_url.endswith('/big'):
+                continue
+            lang = traverse_obj(track, 'lang', 'label') or 'en'
+            subtitles.setdefault(lang, []).append({
+                'url': track_url,
+                'ext': sttl_fmts.get(source.get('format'))
+            })
+
+        thumbnails.extend(
+            traverse_obj(video_data, ('images', Ellipsis, {
+                'id': 'name',
+                'url': 'uri',
+                'width': 'imageWidth',
+                'height': 'imageHeight',
+            })))
+
+        return {
+            'id': video_id,
+            'title': video_data['headline'],
+            'formats': formats,
+            'subtitles': subtitles,
+            'thumbnails': thumbnails,
+            'description': strip_or_none(video_data.get('description')),
+            'duration': parse_duration(video_data.get('length')) or int_or_none(video_data.get('trt')),
+            'timestamp': traverse_obj(video_data, ('dateCreated', 'uts'), expected_type=lambda x: float_or_none(x, 1000)),
+        }

     def _real_extract(self, url):
-        sub_domain, path, page_title = self._match_valid_url(url).groups()
+        sub_domain, path, page_title = re.match(self._VALID_URL, url).groups()
         if sub_domain not in ('money', 'edition'):
             sub_domain = 'edition'
+
+        urlh = self._request_webpage(
+            HEADRequest(url), page_title, fatal=False, expected_status=404)
+        if not urlh:
+            raise ExtractorError('URL inaccessible')
+        elif urlh.getcode() == 404:
+            raise ExtractorError('URL not found (404)')
+        elif compat_urlparse.urlparse(urlh.geturl()).path in ('/business/' if sub_domain == 'money' else '/videos/'):
+            raise ExtractorError('Redirect to home page: content expired?')
+
         config = self._CONFIG[sub_domain]
         return self._extract_cvp_info(
-            config['data_src'] % path, page_title, {
+            config['json_src'] % path, page_title, {
                 'default': {
                     'media_src': config['media_src'],
                 },
@@ -103,12 +273,13 @@
     _VALID_URL = r'https?://[^\.]+\.blogs\.cnn\.com/.+'
     _TEST = {
         'url': 'http://reliablesources.blogs.cnn.com/2014/02/09/criminalizing-journalism/',
-        'md5': '3e56f97b0b6ffb4b79f4ea0749551084',
+        'md5': '8738943bc67afb23f02d8a116a13370f',
         'info_dict': {
             'id': 'bestoftv/2014/02/09/criminalizing-journalism.cnn',
             'ext': 'mp4',
             'title': 'Criminalizing journalism?',
             'description': 'Glenn Greenwald responds to comments made this week on Capitol Hill that journalists could be criminal accessories.',
+            'timestamp':<PHONE_NUMBER>,
             'upload_date': '20140209',
         },
         'expected_warnings': ['Failed to download m3u8 information'],
@@ -125,19 +296,28 @@
     _VALID_URL = r'https?://(?:(?:edition|www)\.)?cnn\.com/(?!videos?/)'
     _TEST = {
         'url': 'http://www.cnn.com/2014/12/21/politics/obama-north-koreas-hack-not-war-but-cyber-vandalism/',
-        'md5': '689034c2a3d9c6dc4aa72d65a81efd01',
+        'md5': 'ad618091beda9eb5afc80bb62c8cdc3a',
         'info_dict': {
             'id': 'bestoftv/2014/12/21/ip-north-korea-obama.cnn',
             'ext': 'mp4',
             'title': 'Obama: Cyberattack not an act of war',
             'description': 'md5:0a802a40d2376f60e6b04c8d5bcebc4b',
+            'timestamp':<PHONE_NUMBER>,
             'upload_date': '20141221',
         },
-        'expected_warnings': ['Failed to download m3u8 information'],
+        'params': {
+            'user_agent': 'Mozilla/5.0',
+        },
         'add_ie': ['CNN'],
     }

     def _real_extract(self, url):
-        webpage = self._download_webpage(url, url_basename(url))
-        cnn_url = self._html_search_regex(r"video:\s*'([^']+)'", webpage, 'cnn url')
-        return self.url_result('http://cnn.com/video/?/video/' + cnn_url, CNNIE.ie_key())
+        video_id = re.sub(r'/index\.html$', '', compat_urlparse.urlparse(url).path)
+        webpage = self._download_webpage(url, video_id)
+        cnn_url = self._html_search_regex(r"video:\s*'([^']+)'", webpage, 'cnn url', default=None)
+        if not cnn_url:
+            video_resource = self._search_regex(
+                r'''(<div\s+(?:[\w-]+\s*=\s*"[^"]*"\s+)*?data-featured-video\s*=\s*('|")true\2(?:\s+[\w-]+\s*=\s*"[^"]*")*\s*>)''',
+                webpage, 'featured video resource')
+            cnn_url = extract_attributes(video_resource)['data-video-id']
+        return self.url_result('http://cnn.com/videos/' + cnn_url, CNNIE.ie_key())

How do we get the supported sites list updated? It's been over 1 year.
In my case it falls back on generic extractor and ends "unsupported site".
Am I doing something wrong as to why it isn't finding the CNN extractor?
I was using --ap-mso and cookies in my config
https://www.cnn.com/videos/title-980446

log
yt-dlp -v -S "hasaud, height:720, vext" -o "GPS 2023.10.08 720p.mp4" https://www.cnn.com/videos/title-980446

[debug] Command-line config: ['-v', '-S', 'hasaud, height:720, vext', '-o', 'GPS 2023.10.08 720p.mp4', 'https://www.cnn.com/videos/title-980446']
[debug] User config "C:\Users\Scott\AppData\Roaming\yt-dlp\config.txt": ['-v', '--ap-mso', 'Comcast_SSO', '--ap-username', 'PRIVATE', '--ap-password', 'PRIVATE', '--cookies', 'C:\\Users\\Scott\\Videos\\Downloaded Videos\\youtube-dl and ffmpeg\\cookies-cnn-com.txt']
[debug] Encodings: locale cp1252, fs utf-8, pref cp1252, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version<EMAIL_ADDRESS>[088add956] (win_exe)
[debug] Python 3.8.10 (CPython AMD64 64bit) - Windows-10-10.0.19045-SP0 (OpenSSL 1.1.1k  25 Mar 2021)
[debug] exe versions: ffmpeg 4.3.1, ffprobe 5.1.2-essentials_build-www.gyan.dev
[debug] Optional libraries: Cryptodome-3.19.0, brotli-1.1.0, certifi-2023.07.22, mutagen-1.47.0, sqlite3-3.35.5, websockets-11.0.3
[debug] Proxy map: {}
[debug] Loaded 1886 extractors
[generic] Extracting URL: https://www.cnn.com/videos/title-980446
[generic] title-980446: Downloading webpage
WARNING: [generic] Falling back on generic information extractor
[generic] title-980446: Extracting information
[debug] Looking for embeds
ERROR: Unsupported URL: https://www.cnn.com/videos/title-980446
Traceback (most recent call last):
  File "yt_dlp\YoutubeDL.py", line 1567, in wrapper
  File "yt_dlp\YoutubeDL.py", line 1702, in __extract_info
  File "yt_dlp\extractor\common.py", line 715, in extract
  File "yt_dlp\extractor\generic.py", line 2568, in _real_extract
yt_dlp.utils.UnsupportedError: Unsupported URL: https://www.cnn.com/videos/title-980446

@RutD0g that is unrelated to this issue (CNN Article videos). Open a new issue. I don't that type of CNN URL has ever been supported

How do we get the supported sites list updated? It's been over 1 year.

You wait for volunteer developers to decide to work on site support in their own unpaid free time

CNN is a funny one but you have to find an API call it makes for the manifest and the address will be in there. If you search fave.api in the network tab you'll find it

Patch seems to work fine BUT with many warnings. Some adjustments are needed to how requests are made (if I understand correctly).  Here's a test link:  https://edition.cnn.com/2024/03/27/opinions/gaza-israel-resigning-state-department-sheline/index.html
Here's a log:
[CNNArticle] Extracting URL: https://edition.cnn.com/2024/03/27/opinions/gaza-israel-resigning-state-department-sheline/index.html
[CNNArticle] /2024/03/27/opinions/gaza-israel-resigning-state-department-sheline: Downloading webpage
[CNN] Extracting URL: http://cnn.com/videos/tv/2024/03/28/amanpour-sheline-gaza-protest.cnn
[CNN] amanpour-sheline-gaza-protest: Downloading webpage
ERROR: Passing a urllib.request.Request to _create_request() is deprecated. Use yt_dlp.networking.common.Request instead.; please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
File "", line 198, in _run_module_as_main
File "", line 88, in _run_code
File "./yt-dlp/main.py", line 17, in
yt_dlp.main()
File "./yt-dlp/yt_dlp/init.py", line 1071, in main
_exit(*variadic(_real_main(argv)))
File "./yt-dlp/yt_dlp/init.py", line 1061, in _real_main
return ydl.download(all_urls)
File "./yt-dlp/yt_dlp/YoutubeDL.py", line 3578, in download
self.__download_wrapper(self.extract_info)(
File "./yt-dlp/yt_dlp/YoutubeDL.py", line 3553, in wrapper
res = func(*args, **kwargs)
File "./yt-dlp/yt_dlp/YoutubeDL.py", line 1599, in extract_info
return self.__extract_info(url, self.get_info_extractor(key), download, extra_info, process)
File "./yt-dlp/yt_dlp/YoutubeDL.py", line 1610, in wrapper
return func(self, *args, **kwargs)
File "./yt-dlp/yt_dlp/YoutubeDL.py", line 1766, in __extract_info
return self.process_ie_result(ie_result, download, extra_info)
File "./yt-dlp/yt_dlp/YoutubeDL.py", line 1845, in process_ie_result
return self.extract_info(
File "./yt-dlp/yt_dlp/YoutubeDL.py", line 1599, in extract_info
return self.__extract_info(url, self.get_info_extractor(key), download, extra_info, process)
File "./yt-dlp/yt_dlp/YoutubeDL.py", line 1610, in wrapper
return func(self, *args, **kwargs)
File "./yt-dlp/yt_dlp/YoutubeDL.py", line 1745, in __extract_info
ie_result = ie.extract(url)
File "./yt-dlp/yt_dlp/extractor/common.py", line 734, in extract
ie_result = self._real_extract(url)
File "./yt-dlp/yt_dlp/extractor/cnn.py", line 250, in _real_extract
urlh = self._request_webpage(
File "./yt-dlp/yt_dlp/extractor/common.py", line 890, in _request_webpage
return self._downloader.urlopen(self._create_request(url_or_request, data, headers, query, extensions))
File "./yt-dlp/yt_dlp/extractor/common.py", line 824, in _create_request
self._downloader.deprecation_warning(
File "./yt-dlp/yt_dlp/YoutubeDL.py", line 1064, in deprecation_warning
deprecation_warning(
File "./yt-dlp/yt_dlp/utils/_utils.py", line 1452, in deprecation_warning
File "./yt-dlp/yt_dlp/YoutubeDL.py", line 1077, in report_error
self.trouble(f'{self._format_err("ERROR:", self.Styles.ERROR)} {message}', *args, **kwargs)
File "./yt-dlp/yt_dlp/YoutubeDL.py", line 1005, in trouble
tb_data = traceback.format_list(traceback.extract_stack())
ERROR: Response.getcode() is deprecated, use Response.status; please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
ERROR: Response.geturl() is deprecated, use Response.url; please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
[CNN] amanpour-sheline-gaza-protest: Downloading JSON metadata
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7), vcodec:vp9.2(10), channels, acodec, size, br, asr, proto, vext, aext, hasaud, source, id
[info] Available formats for tv/2024/03/28/amanpour-sheline-gaza-protest.cnn:
ID             EXT RESOLUTION │   FILESIZE   TBR PROTO │ VCODEC  ACODEC
────────────────────────────────────────────────────────────────────────
1920x1080_8000 mp4 1920x1080  │ ≈672.34MiB 8000k https │ unknown unknown


[BSO] Thieving Skillcape Perk not included in Thieving Master Cape or derivatives

🐛 Bug Report
The perk for "+10% chance of success from Thieving cape" from the level 99 thieving skillcape is not included in the Thieving Master Cape / Support Cape / Completionist Cape. You need to equip both types of capes to get all boosts, which doesn't make sense for a higher tier item.

Fixed in next update


feat: support shareable Preview links

Shareable links were not working with the initial alpha release of @prismicio/next
This PR makes it possible to use shareable links.
Types of changes

[ ] Chore (a non-breaking change which is related to package maintenance)
[ ] Bug fix (a non-breaking change which fixes an issue)
[x] New feature (a non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Description
The update to <PrismicPreview /> checks for Prismic Cookies and/or if Next Preview mode is enabled. It triggers Preview mode accordingly.
Checklist:

[ ] My change requires an update to the official documentation.
[x] All TSDoc comments are up-to-date and new ones have been added where necessary.
[x] All new and existing tests are passing.

Changes submitted, lemme know if I missed anything @angeloashmore


Model observability for Anthropic

Add instrumentation to Anthropic integration to support observability for chat models.

merged in 3b7522b6c0545553a51d188d87c15560eb088094  :bow:


React Cheat sheet

Create a cheat sheet for React JS

Yes, you are right. Are you familiar with React?

I am learning React, i can help you as am learning i could create the cheat sheet

Yes this would be great :)

Here it is :-)
https://github.com/LeCoupa/awesome-cheatsheets/blob/master/frontend/react.js


Support PSR7Session?

Just dropping it here: https://github.com/Ocramius/PSR7Session
Maybe interesting?
Note: requirement is PHP ^7.0 (won't be downgraded)

I know the project and It's really interesting. The only problem is php7 (this library requires ^5.5 and I'm not sure whether it's a good idea to have different requirements for different middlewares).

I'm working in the v4.x that requires PHP 7.0+. The aim is to be fully compatible with 3.x but adding other PHP7-only middlewares, like this one.

Because PSR7Session can work as a middleware itself, there's no need to create an adapter in this package.
So I close this issue.

Ah, sorry, I probably misinterpreted the scope of this project, thinking that it was aggregating initialization of different middlewares spread across the interwebs.

There's a way to aggregate new middlewares from a 3rd party library, so you can provide an adapter. For example:
namespace PSR7Session\Middleware;

//Register this namespace that contains more middleware pieces:
\Psr7Middlewares\Middleware::registerNamespace(__NAMESPACE__);

//Define the middleware:
class Psr7Session
{
    // ...
}

To use it:
$middleware[] = Middleware::Psr7Session();

If you like it, I can create a pull request.

Ah, I see, so this would go the other way around. My bad, I really misunderstood how this project works, sorry :D


Docs not working

@kenhyuwa I think your domain expired, could you add the docs somewhere? I need them pretty badly..

You can find an archive here : https://web.archive.org/web/20211216002245/http://litepie.com/
If you host it elsewhere, you can share it :-D
This project is abandoned, maintainer doesn't answer anymore.

https://web.archive.org/web/20211216002245/http://litepie.com/
this is not working


Fix missing class issue with H2 version 2.0.202

With older versions H2 returned a proprietary  instance from .
We used to support the conversion of that to an .
With the most recent versions  is no longer part of the H2 driver and we only register the converter when we encounter older versions of H2.
Closes #1114
See h2database/h2database#1359

Thanks for the review. That's merged.


Turkish Translation

reviewing Turkish translation

Please consider #792 before merging this PR.

#792 is merged. Thank you for your contribution @duchampdev


Add customTitle to OPTION_NAMES_TO_HASH

Problem
While we have multiple runs using this action, only one comment gets updated instead of writing multiple comments for each run.
Solution

Add customTitle  along with other options as basis to check if the comment is unique
Also improve replace startsWith with includes as the former does not verify if the comment is update manually with a text at the start. (P. S Might happen very rarely but still a good to have)

Hey @vikasperi :wave:,
Thank you for your PR. Everything looks good!
@all-contributors please add @vikasperi for code


[Telegram] Is it possible to edit sent messages?

Bot Info

SDK Platform: Node.js
SDK Version: 3.11.0
Active Channels: Telegram

Issue Description
Is it possible to edit a message bot sent in Telegram with NodeJS SDK?

@MohammadMDSA Great question.
You can edit a "sent" message using the Telegram Bot API. For more information, check out the interface for editMessageText:
https://core.telegram.org/bots/api#editmessagetext
Then use the channelData property of your bot's message to send this data to the Telegram conversation.