Indexing Issues with UMLS?

Hey there,
I'm having an issue that's causing the indexing process that's done in the umls-similarity.pl script to take an incredibly long time.
I've looked into the Perl source code and I understand that the UMLS::Similarity and UMLS::Interface modules need to first create a set of indexes in order to speed up subsequent path dependent semantic similarity measures. I've checked their mailing list, which has been unfortunately inactive since 2019, and I know that this process is expected to take several hours and maybe even days. However, when I ran it in my machine, the first time I tried it, I left it running over the weekend and after 48+ hours just about 1000 CUIs had been indexed which means that at that rate it would take over 17 YEARS to index all of the 3.31 million concepts within the UMLS... The machine I'm running it in should definitely not be having these issues, at least from a hardware perspective (96 cores, 488 GB RAM, 2TB SSD).
I've been experimenting with the MySQL my.ini config file and was able to reduce that down to 3 years. I used the parameters delineated here as a starting point: https://www.nlm.nih.gov/research/umls/implementation_resources/scripts/README_RRF_MySQL_Output_Stream.html. A good improvement but it is still unreasonably long... I'm aware of the --realtime flag which I've turned on, however, that can take a VERY long time to process some concept pairs and I'd rather go through the long process of setting up the indexes once in order to significantly speed up any subsequent calculations.
The routine that's causing these delays seems to be the subroutine "_initializeDepthFirstSearch" that can be found here: https://github.com/bmcinnes/UMLS-Interface/blob/master/lib/UMLS/Interface/PathFinder.pm
Do you have any recommendations/suggestions? Could you share with me what is the exact configuration you used for your UMLS build as well as your MySQL parameters?
Here's some system info in case it helps:
OS: Windows Server 2019 Datacenter (64-bit OS)
MySQL version : 8.1.0
UMLS Version: 2023AA
Perl Version: Strawberry Perl <IP_ADDRESS>-64bit
UMLS Interface Version: 1.51
I'm including some screenshots of what I'm seeing in case it helps:
Here you can see that the routine successfully connected to MySQL database:

Here you can see that after a successful connection to the umls database in MySQL, there is a new schema called "umlsinterfaceindex"

Here are the contents of the file containing example CUI pairs that I'm looking to get a semantic similarity measure for:

Here you can see some of the outputs from running the routine with VERBOSE mode turned on:

Here you can see the contents of a table file that gets created alongside verbose mode that tracks the progress of the indexing process:

Please let me know if you have any thoughts on this.
Thanks!

I resolved this and I'll include my solution here in case anyone runs into this issue:

The UMLS database in MySQL for some reason had not been indexed during installation. When you run the Metamorphosys application to build the UMLS you have the option of generating load scripts for your UMLS configuration. The load scripts are called mysql_tables.sql and mysql_indexes.sql. In addition to that, the Metamorphosys process creates both .sh and .bat files that allow you to load your configuration using those scripts. The mysql_indexes.sql routine didn't run and that caused the _initializeDepthFirstSearch subroutine to take FOREVER. After that modification, the routine works quite fast!

I had to make some further adjustments to the MySQL configuration file. You'll have to experiment with your own system, but I found that in addition to using the parameters suggested by the UMLS that I linked in my above query, you'll generally want to increase the innodb_buffer_pool_size parameter to 70-80% of your total memory capacity. Keeping it at the default value of 128 MB, caused delays at times. In my case, I set it to 100GB and that was more than sufficient for the calculations I'm running.

There are other modifications I'm making to the Perl modules and I'm building my own Python routine for this to further speed up things, but those two things above, are what lead to the largest performance bottlenecks.


SCG SLURM hg38 permissions error?

I am getting an error that I believe is related to permission to access hg38 on SCG. At the call-bowtie2 step, the pipeline fails.
Stderr:
Traceback (most recent call last):
File "/labs/smontgom/mdegorte/ATAC/atac-seq-pipeline/src/encode_bowtie2.py", line 207, in
main()
File "/labs/smontgom/mdegorte/ATAC/atac-seq-pipeline/src/encode_bowtie2.py", line 151, in main
untar(tar, args.out_dir)
File "/labs/smontgom/mdegorte/ATAC/atac-seq-pipeline/src/encode_common.py", line 113, in untar
run_shell_cmd(cmd)
File "/labs/smontgom/mdegorte/ATAC/atac-seq-pipeline/src/encode_common.py", line 230, in run_shell_cmd
os.killpg(pgid, signal.SIGKILL)
OSError: [Errno 3] No such process
ln: failed to access ‘.bam’: No such file or directory
ln: failed to access ‘.bai’: No such file or directory
ln: failed to access ‘.align.log’: No such file or directory
ln: failed to access ‘.flagstat.qc’: No such file or directory
Stdout:
[2018-07-05 12:39:20,742 INFO] ['/labs/smontgom/mdegorte/ATAC/atac-seq-pipeline/src/encode_bowtie2.py', '/labs/smontgom/mdegorte/ATAC/atac-seq-pipeline/cromwell-executions/atac/03a44c57-75aa-44b0-b167-2ca44e06407f/call-bowtie2/shard-0/inputs/-1037641694/GRCh38_no_alt_analysis_set_GCA_000001405.15.fasta.tar', '/labs/smontgom/mdegorte/ATAC/atac-seq-pipeline/cromwell-executions/atac/03a44c57-75aa-44b0-b167-2ca44e06407f/call-bowtie2/shard-0/inputs/-1978335035/merge_fastqs_R1_SCS47_merge_R1.trim.merged.fastq.gz', '/labs/smontgom/mdegorte/ATAC/atac-seq-pipeline/cromwell-executions/atac/03a44c57-75aa-44b0-b167-2ca44e06407f/call-bowtie2/shard-0/inputs/-1978335035/merge_fastqs_R2_SCS47_merge_R2.trim.merged.fastq.gz', '--paired-end', '--multimapping', '4', '--nth', '4']
[2018-07-05 12:39:20,742 INFO] Initializing and making output directory...
[2018-07-05 12:39:20,742 INFO] Generating read length file...
[2018-07-05 12:39:41,015 INFO] Unpacking bowtie2 index tar...
[2018-07-05 12:39:41,020 INFO] run_shell_cmd: PID=198613, CMD=tar xvf /labs/smontgom/mdegorte/ATAC/atac-seq-pipeline/cromwell-executions/atac/03a44c57-75aa-44b0-b167-2ca44e06407f/call-bowtie2/shard-0/inputs/-1037641694/GRCh38_no_alt_analysis_set_GCA_000001405.15.fasta.tar -C .
PID=198613: tar: /labs/smontgom/mdegorte/ATAC/atac-seq-pipeline/cromwell-executions/atac/03a44c57-75aa-44b0-b167-2ca44e06407f/call-bowtie2/shard-0/inputs/-1037641694/GRCh38_no_alt_analysis_set_GCA_000001405.15.fasta.tar: Cannot open: Permission denied
PID=198613: tar: Error is not recoverable: exiting now
[2018-07-05 12:39:41,092 ERROR] Unknown exception caught. Killing process group 198613...
Traceback (most recent call last):
File "/labs/smontgom/mdegorte/ATAC/atac-seq-pipeline/src/encode_common.py", line 224, in run_shell_cmd
p.returncode, cmd)
CalledProcessError: Command 'tar xvf /labs/smontgom/mdegorte/ATAC/atac-seq-pipeline/cromwell-executions/atac/03a44c57-75aa-44b0-b167-2ca44e06407f/call-bowtie2/shard-0/inputs/-1037641694/GRCh38_no_alt_analysis_set_GCA_000001405.15.fasta.tar -C .' returned non-zero exit status 2
In the .json file I am using this line to specify the genome:
"atac.genome_tsv" : "/labs/smontgom/mdegorte/ATAC/atac-seq-pipeline/genome/scg/hg38_scg.tsv",
I've started to install my own version of hg38 as a work around, but would prefer to use the common files as advised in the documentation - any help would be appreciated. Thanks!
Note: I am using my own installation of Cromwell (cromwell-33.1) as it did not appear to be present in the dependencies installation.

You don't have to install genome data on SCG. There is a shared genome data for hg38. Please try with a genome TSV file genome/scg/hg38_scg.tsv.
Also, please post issues on https://github.com/ENCODE-DCC/atac-seq-pipeline/ next time.

Thanks Jin. I tried again with the following line in the .json file:
"atac.genome_tsv" : "genome/scg/hg38_scg.tsv",
The pipeline fails with this output to the screen:
[2018-07-06 12:13:48,53] [info] DispatchedConfigAsyncJobExecutionActor [579835bcatac.trim_adapter:0:1]: executing: sbatch
--export=ALL
-J cromwell_579835bc_trim_adapter
-D /labs/smontgom/mdegorte/ATAC/atac-seq-pipeline/cromwell-executions/atac/579835bc-3ce3-4489-bd44-250bf952540b/call-trim_adapter/shard-0
-o /labs/smontgom/mdegorte/ATAC/atac-seq-pipeline/cromwell-executions/atac/579835bc-3ce3-4489-bd44-250bf952540b/call-trim_adapter/shard-0/execution/stdout
-e /labs/smontgom/mdegorte/ATAC/atac-seq-pipeline/cromwell-executions/atac/579835bc-3ce3-4489-bd44-250bf952540b/call-trim_adapter/shard-0/execution/stderr
-t 1440
-n 1
--ntasks-per-node=1
--cpus-per-task=2
--mem=12000

--account smontgom
--wrap "/bin/bash /labs/smontgom/mdegorte/ATAC/atac-seq-pipeline/cromwell-executions/atac/579835bc-3ce3-4489-bd44-250bf952540b/call-trim_adapter/shard-0/execution/script"
[2018-07-06 12:13:48,75] [info] DispatchedConfigAsyncJobExecutionActor [579835bcatac.trim_adapter:0:1]: job id: 2129106
[2018-07-06 12:13:48,75] [info] DispatchedConfigAsyncJobExecutionActor [579835bcatac.read_genome_tsv:NA:1]: job id: 2129105
[2018-07-06 12:13:48,77] [info] DispatchedConfigAsyncJobExecutionActor [579835bcatac.trim_adapter:0:1]: Status change from - to WaitingForReturnCodeFile
[2018-07-06 12:13:48,77] [info] DispatchedConfigAsyncJobExecutionActor [579835bcatac.read_genome_tsv:NA:1]: Status change from - to WaitingForReturnCodeFile
[2018-07-06 12:14:26,69] [info] DispatchedConfigAsyncJobExecutionActor [579835bcatac.read_genome_tsv:NA:1]: Status change from WaitingForReturnCodeFile to Done
[2018-07-06 12:52:36,69] [info] DispatchedConfigAsyncJobExecutionActor [579835bcatac.trim_adapter:0:1]: Status change from WaitingForReturnCodeFile to Done
[2018-07-06 12:52:37,29] [error] WorkflowManagerActor Workflow 579835bc-3ce3-4489-bd44-250bf952540b failed (during ExecutingWorkflowState): cromwell.backend.standard.StandardAsyncExecutionActor$$anon$2: Failed to evaluate job outputs:
Bad output 'read_genome_tsv.genome': java.nio.file.NoSuchFileException: /labs/smontgom/mdegorte/ATAC/atac-seq-pipeline/cromwell-executions/atac/579835bc-3ce3-4489-bd44-250bf952540b/call-read_genome_tsv/execution/genome/scg/hg38_scg.tsv
at cromwell.backend.standard.StandardAsyncExecutionActor.$anonfun$handleExecutionSuccess$1(StandardAsyncExecutionActor.scala:836)
at scala.util.Success.$anonfun$map$1(Try.scala:251)
at scala.util.Success.map(Try.scala:209)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:288)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:29)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:29)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:60)
at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55)
at akka.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:91)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12)
at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:81)
at akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:91)
at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:40)
at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:43)
at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

Please use an absolute path for genome TSV file.
Jin
On Fri, Jul 6, 2018 at 3:00 PM, mkdegorter<EMAIL_ADDRESS>wrote:

Thanks Jin. I tried again with the following line in the .json file:
"atac.genome_tsv" : "genome/scg/hg38_scg.tsv",
The pipeline fails with this output to the screen:
[2018-07-06 12:13:48,53] [info] DispatchedConfigAsyncJobExecutionActor
[579835bcatac.trim_adapter:0:1]: executing: sbatch
--export=ALL
-J cromwell_579835bc_trim_adapter
-D /labs/smontgom/mdegorte/ATAC/atac-seq-pipeline/cromwell-
executions/atac/579835bc-3ce3-4489-bd44-250bf952540b/call-trim_adapter/shard-0
-o /labs/smontgom/mdegorte/ATAC/atac-seq-pipeline/cromwell-
executions/atac/579835bc-3ce3-4489-bd44-250bf952540b/call-
trim_adapter/shard-0/execution/stdout
-e /labs/smontgom/mdegorte/ATAC/atac-seq-pipeline/cromwell-
executions/atac/579835bc-3ce3-4489-bd44-250bf952540b/call-
trim_adapter/shard-0/execution/stderr
-t 1440
-n 1
--ntasks-per-node=1
--cpus-per-task=2
--mem=12000
--account smontgom
--wrap "/bin/bash /labs/smontgom/mdegorte/ATAC/atac-seq-pipeline/cromwell-
executions/atac/579835bc-3ce3-4489-bd44-250bf952540b/call-
trim_adapter/shard-0/execution/script"
[2018-07-06 12:13:48,75] [info] DispatchedConfigAsyncJobExecutionActor
[579835bcatac.trim_adapter:0:1]: job id: 2129106
[2018-07-06 12:13:48,75] [info] DispatchedConfigAsyncJobExecutionActor
[579835bcatac.read_genome_tsv:NA:1]: job id: 2129105
[2018-07-06 12:13:48,77] [info] DispatchedConfigAsyncJobExecutionActor
[579835bcatac.trim_adapter:0:1]: Status change from - to
WaitingForReturnCodeFile
[2018-07-06 12:13:48,77] [info] DispatchedConfigAsyncJobExecutionActor
[579835bcatac.read_genome_tsv:NA:1]: Status change from - to
WaitingForReturnCodeFile
[2018-07-06 12:14:26,69] [info] DispatchedConfigAsyncJobExecutionActor
[579835bcatac.read_genome_tsv:NA:1]: Status change from
WaitingForReturnCodeFile to Done
[2018-07-06 12:52:36,69] [info] DispatchedConfigAsyncJobExecutionActor
[579835bcatac.trim_adapter:0:1]: Status change from
WaitingForReturnCodeFile to Done
[2018-07-06 12:52:37,29] [error] WorkflowManagerActor Workflow
579835bc-3ce3-4489-bd44-250bf952540b failed (during
ExecutingWorkflowState): cromwell.backend.standard.
StandardAsyncExecutionActor$$anon$2: Failed to evaluate job outputs:
Bad output 'read_genome_tsv.genome': java.nio.file.NoSuchFileException:
/labs/smontgom/mdegorte/ATAC/atac-seq-pipeline/cromwell-
executions/atac/579835bc-3ce3-4489-bd44-250bf952540b/call-
read_genome_tsv/execution/genome/scg/hg38_scg.tsv
at cromwell.backend.standard.StandardAsyncExecutionActor.$anonfun$
handleExecutionSuccess$1(StandardAsyncExecutionActor.scala:836)
at scala.util.Success.$anonfun$map$1(Try.scala:251)
at scala.util.Success.map(Try.scala:209)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:288)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:29)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:29)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:60)
at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(
BatchingExecutor.scala:55)
at akka.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(
BatchingExecutor.scala:91)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12)
at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:81)
at akka.dispatch.BatchingExecutor$BlockableBatch.run(
BatchingExecutor.scala:91)
at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:40)
at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(
ForkJoinExecutorConfigurator.scala:43)
at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.
runTask(ForkJoinPool.java:1339)
at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(
ForkJoinWorkerThread.java:107)
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/kundajelab/atac_dnase_pipelines/issues/130#issuecomment-403157267,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AIOd_CawxuR3XPTuw3Nf3JbDOTZu-2qxks5uD94PgaJpZM4VEc4R
.

Hi Jin,
That's what I had tried in my first post,
In the .json file I am using this line to specify the genome:
"atac.genome_tsv" : "/labs/smontgom/mdegorte/ATAC/atac-seq-pipeline/genome/scg/hg38_scg.tsv"
With the absolute path it fails a little further along the pipeline.

How further did it go? Please post a new log.

With the full path it gets to the same step in the pipeline as in the first post. This is the entire output to the screen:
--wrap "/bin/bash /labs/smontgom/mdegorte/ATAC/atac-seq-pipeline/cromwell-executions/atac/449accb2-e994-40e1-912f-753255616799/call-trim_adapter/shard-0/execution/script"
[2018-07-09 15:14:28,55] [info] DispatchedConfigAsyncJobExecutionActor [449accb2atac.trim_adapter:0:1]: job id: 2131269
[2018-07-09 15:14:28,57] [info] DispatchedConfigAsyncJobExecutionActor [449accb2atac.trim_adapter:0:1]: Status change from - to WaitingForReturnCodeFile
[2018-07-09 15:14:57,68] [info] DispatchedConfigAsyncJobExecutionActor [449accb2atac.read_genome_tsv:NA:1]: Status change from WaitingForReturnCodeFile to Done
[2018-07-09 15:54:17,48] [info] DispatchedConfigAsyncJobExecutionActor [449accb2atac.trim_adapter:0:1]: Status change from WaitingForReturnCodeFile to Done
[2018-07-09 15:54:19,17] [info] WorkflowExecutionActor-449accb2-e994-40e1-912f-753255616799 [449accb2]: Starting atac.bowtie2
[2018-07-09 15:54:19,21] [warn] DispatchedConfigAsyncJobExecutionActor [449accb2atac.bowtie2:0:1]: Unrecognized runtime attribute keys: preemptible, disks
[2018-07-09 15:54:19,40] [warn] Localization via hard link has failed: /labs/smontgom/mdegorte/ATAC/atac-seq-pipeline/cromwell-executions/atac/449accb2-e994-40e1-912f-753255616799/call-bowtie2/shard-0/inputs/-1037641694/GRCh38_no_alt_analysis_set_GCA_000001405.15.fasta.tar -> /ifs/scratch/leepc12/pipeline_genome_data/hg38/bowtie2_index/GRCh38_no_alt_analysis_set_GCA_000001405.15.fasta.tar: Invalid cross-device link
[2018-07-09 15:54:19,42] [info] DispatchedConfigAsyncJobExecutionActor [449accb2atac.bowtie2:0:1]: python $(which encode_bowtie2.py)
/labs/smontgom/mdegorte/ATAC/atac-seq-pipeline/cromwell-executions/atac/449accb2-e994-40e1-912f-753255616799/call-bowtie2/shard-0/inputs/-1037641694/GRCh38_no_alt_analysis_set_GCA_000001405.15.fasta.tar
/labs/smontgom/mdegorte/ATAC/atac-seq-pipeline/cromwell-executions/atac/449accb2-e994-40e1-912f-753255616799/call-bowtie2/shard-0/inputs/1145178294/merge_fastqs_R1_SCS47_merge_R1.trim.merged.fastq.gz /labs/smontgom/mdegorte/ATAC/atac-seq-pipeline/cromwell-executions/atac/449accb2-e994-40e1-912f-753255616799/call-bowtie2/shard-0/inputs/1145178294/merge_fastqs_R2_SCS47_merge_R2.trim.merged.fastq.gz
--paired-end
--multimapping 4

--nth 4
[2018-07-09 15:54:19,44] [info] DispatchedConfigAsyncJobExecutionActor [449accb2atac.bowtie2:0:1]: executing: sbatch
--export=ALL
-J cromwell_449accb2_bowtie2
-D /labs/smontgom/mdegorte/ATAC/atac-seq-pipeline/cromwell-executions/atac/449accb2-e994-40e1-912f-753255616799/call-bowtie2/shard-0
-o /labs/smontgom/mdegorte/ATAC/atac-seq-pipeline/cromwell-executions/atac/449accb2-e994-40e1-912f-753255616799/call-bowtie2/shard-0/execution/stdout
-e /labs/smontgom/mdegorte/ATAC/atac-seq-pipeline/cromwell-executions/atac/449accb2-e994-40e1-912f-753255616799/call-bowtie2/shard-0/execution/stderr
-t 2880
-n 1
--ntasks-per-node=1
--cpus-per-task=4
--mem=20000

--account smontgom
--wrap "/bin/bash /labs/smontgom/mdegorte/ATAC/atac-seq-pipeline/cromwell-executions/atac/449accb2-e994-40e1-912f-753255616799/call-bowtie2/shard-0/execution/script"
[2018-07-09 15:54:23,53] [info] DispatchedConfigAsyncJobExecutionActor [449accb2atac.bowtie2:0:1]: job id: 2131354
[2018-07-09 15:54:23,54] [info] DispatchedConfigAsyncJobExecutionActor [449accb2atac.bowtie2:0:1]: Status change from - to WaitingForReturnCodeFile
[2018-07-09 15:55:41,41] [info] DispatchedConfigAsyncJobExecutionActor [449accb2atac.bowtie2:0:1]: Status change from WaitingForReturnCodeFile to Done
[2018-07-09 15:55:41,82] [error] WorkflowManagerActor Workflow 449accb2-e994-40e1-912f-753255616799 failed (during ExecutingWorkflowState): Job atac.bowtie2:0:1 exited with return code 1 which has not been declared as a valid return code. See 'continueOnReturnCode' runtime attribute for more details.
Check the content of stderr for potential additional information: /labs/smontgom/mdegorte/ATAC/atac-seq-pipeline/cromwell-executions/atac/449accb2-e994-40e1-912f-753255616799/call-bowtie2/shard-0/execution/stderr.
Traceback (most recent call last):
File "/labs/smontgom/mdegorte/ATAC/atac-seq-pipeline/src/encode_common.py", line 224, in run_shell_cmd
p.returncode, cmd)
subprocess.CalledProcessError: Command 'tar xvf /labs/smontgom/mdegorte/ATAC/atac-seq-pipeline/cromwell-executions/atac/449accb2-e994-40e1-912f-753255616799/call-bowtie2/shard-0/inputs/-1037641694/GRCh38_no_alt_analysis_set_GCA_000001405.15.fasta.tar -C .' returned non-zero exit status 2
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/labs/smontgom/mdegorte/ATAC/atac-seq-pipeline/src/encode_bowtie2.py", line 207, in
main()
File "/labs/smontgom/mdegorte/ATAC/atac-seq-pipeline/src/encode_bowtie2.py", line 151, in main
untar(tar, args.out_dir)
File "/labs/smontgom/mdegorte/ATAC/atac-seq-pipeline/src/encode_common.py", line 113, in untar
run_shell_cmd(cmd)
File "/labs/smontgom/mdegorte/ATAC/atac-seq-pipeline/src/encode_common.py", line 230, in run_shell_cmd
os.killpg(pgid, signal.SIGKILL)
ProcessLookupError: [Errno 3] No such process
ln: failed to access ‘.bam’: No such file or directory
ln: failed to access ‘.bai’: No such file or directory
ln: failed to access ‘.align.log’: No such file or directory
ln: failed to access ‘.flagstat.qc’: No such file or directory
[2018-07-09 15:55:41,83] [info] WorkflowManagerActor WorkflowActor-449accb2-e994-40e1-912f-753255616799 is in a terminal state: WorkflowFailedState
[2018-07-09 15:55:49,39] [info] SingleWorkflowRunnerActor workflow finished with status 'Failed'.
[2018-07-09 15:55:53,53] [info] Workflow polling stopped
[2018-07-09 15:55:53,54] [info] Shutting down WorkflowStoreActor - Timeout = 5 seconds
[2018-07-09 15:55:53,55] [info] Aborting all running workflows.
[2018-07-09 15:55:53,57] [info] Shutting down WorkflowLogCopyRouter - Timeout = 5 seconds
[2018-07-09 15:55:53,57] [info] Shutting down JobExecutionTokenDispenser - Timeout = 5 seconds
[2018-07-09 15:55:53,57] [info] WorkflowStoreActor stopped
[2018-07-09 15:55:53,57] [info] WorkflowLogCopyRouter stopped
[2018-07-09 15:55:53,57] [info] Shutting down WorkflowManagerActor - Timeout = 3600 seconds
[2018-07-09 15:55:53,57] [info] JobExecutionTokenDispenser stopped
[2018-07-09 15:55:53,57] [info] WorkflowManagerActor All workflows finished
[2018-07-09 15:55:53,58] [info] Connection pools shut down
[2018-07-09 15:55:53,58] [info] WorkflowManagerActor stopped
[2018-07-09 15:55:53,58] [info] Shutting down SubWorkflowStoreActor - Timeout = 1800 seconds
[2018-07-09 15:55:53,58] [info] Shutting down JobStoreActor - Timeout = 1800 seconds
[2018-07-09 15:55:53,58] [info] Shutting down CallCacheWriteActor - Timeout = 1800 seconds
[2018-07-09 15:55:53,58] [info] JobStoreActor stopped
[2018-07-09 15:55:53,58] [info] SubWorkflowStoreActor stopped
[2018-07-09 15:55:53,58] [info] CallCacheWriteActor Shutting down: 0 queued messages to process
[2018-07-09 15:55:53,58] [info] CallCacheWriteActor stopped
[2018-07-09 15:55:53,58] [info] Shutting down ServiceRegistryActor - Timeout = 1800 seconds
[2018-07-09 15:55:53,58] [info] Shutting down DockerHashActor - Timeout = 1800 seconds
[2018-07-09 15:55:53,58] [info] Shutting down IoProxy - Timeout = 1800 seconds
[2018-07-09 15:55:53,58] [info] DockerHashActor stopped
[2018-07-09 15:55:53,58] [info] WriteMetadataActor Shutting down: 0 queued messages to process
[2018-07-09 15:55:53,58] [info] KvWriteActor Shutting down: 0 queued messages to process
[2018-07-09 15:55:53,58] [info] IoProxy stopped
[2018-07-09 15:55:53,59] [info] ServiceRegistryActor stopped
[2018-07-09 15:55:53,61] [info] Database closed
[2018-07-09 15:55:53,61] [info] Stream materializer shut down
Workflow 449accb2-e994-40e1-912f-753255616799 transitioned to state Failed
[2018-07-09 15:55:53,68] [info] Automatic shutdown of the async connection
[2018-07-09 15:55:53,68] [info] Gracefully shutdown sentry threads.
[2018-07-09 15:55:53,69] [info] Shutdown finished
The call-bowtie2 folder is created, and in it, the stderr file reads:
Traceback (most recent call last):
File "/labs/smontgom/mdegorte/ATAC/atac-seq-pipeline/src/encode_common.py", line 224, in run_shell_cmd
p.returncode, cmd)
subprocess.CalledProcessError: Command 'tar xvf /labs/smontgom/mdegorte/ATAC/atac-seq-pipeline/cromwell-executions/atac/449accb2-e994-40e1-912f-753255616799/call-bowtie2/shard-0/inp
uts/-1037641694/GRCh38_no_alt_analysis_set_GCA_000001405.15.fasta.tar -C .' returned non-zero exit status 2
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/labs/smontgom/mdegorte/ATAC/atac-seq-pipeline/src/encode_bowtie2.py", line 207, in
main()
File "/labs/smontgom/mdegorte/ATAC/atac-seq-pipeline/src/encode_bowtie2.py", line 151, in main
untar(tar, args.out_dir)
File "/labs/smontgom/mdegorte/ATAC/atac-seq-pipeline/src/encode_common.py", line 113, in untar
run_shell_cmd(cmd)
File "/labs/smontgom/mdegorte/ATAC/atac-seq-pipeline/src/encode_common.py", line 230, in run_shell_cmd
os.killpg(pgid, signal.SIGKILL)
ProcessLookupError: [Errno 3] No such process
ln: failed to access ‘.bam’: No such file or directory
ln: failed to access ‘.bai’: No such file or directory
ln: failed to access ‘.align.log’: No such file or directory
ln: failed to access ‘.flagstat.qc’: No such file or directory
And the stdout file reads:
Traceback (most recent call last):
File "/labs/smontgom/mdegorte/ATAC/atac-seq-pipeline/src/encode_common.py", line 224, in run_shell_cmd
p.returncode, cmd)
subprocess.CalledProcessError: Command 'tar xvf /labs/smontgom/mdegorte/ATAC/atac-seq-pipeline/cromwell-executions/atac/449accb2-e994-40e1-912f-753255616799/call-bowtie2/shard-0/inp
uts/-1037641694/GRCh38_no_alt_analysis_set_GCA_000001405.15.fasta.tar -C .' returned non-zero exit status 2
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/labs/smontgom/mdegorte/ATAC/atac-seq-pipeline/src/encode_bowtie2.py", line 207, in
main()
File "/labs/smontgom/mdegorte/ATAC/atac-seq-pipeline/src/encode_bowtie2.py", line 151, in main
untar(tar, args.out_dir)
File "/labs/smontgom/mdegorte/ATAC/atac-seq-pipeline/src/encode_common.py", line 113, in untar
run_shell_cmd(cmd)
File "/labs/smontgom/mdegorte/ATAC/atac-seq-pipeline/src/encode_common.py", line 230, in run_shell_cmd
os.killpg(pgid, signal.SIGKILL)
ProcessLookupError: [Errno 3] No such process
ln: failed to access ‘.bam’: No such file or directory
ln: failed to access ‘.bai’: No such file or directory
ln: failed to access ‘.align.log’: No such file or directory
ln: failed to access ‘.flagstat.qc’: No such file or directory
[mdegorte@hppsl230s-rcf-412-02-l execution]$ more stdout
[2018-07-09 15:54:22,907 INFO] ['/labs/smontgom/mdegorte/ATAC/atac-seq-pipeline/src/encode_bowtie2.py', '/labs/smontgom/mdegorte/ATAC/atac-seq-pipeline/cromwell-executions/atac/449a
ccb2-e994-40e1-912f-753255616799/call-bowtie2/shard-0/inputs/-1037641694/GRCh38_no_alt_analysis_set_GCA_000001405.15.fasta.tar', '/labs/smontgom/mdegorte/ATAC/atac-seq-pipeline/crom
well-executions/atac/449accb2-e994-40e1-912f-753255616799/call-bowtie2/shard-0/inputs/1145178294/merge_fastqs_R1_SCS47_merge_R1.trim.merged.fastq.gz', '/labs/smontgom/mdegorte/ATAC/
atac-seq-pipeline/cromwell-executions/atac/449accb2-e994-40e1-912f-753255616799/call-bowtie2/shard-0/inputs/1145178294/merge_fastqs_R2_SCS47_merge_R2.trim.merged.fastq.gz', '--paire
d-end', '--multimapping', '4', '--nth', '4']
[2018-07-09 15:54:22,907 INFO] Initializing and making output directory...
[2018-07-09 15:54:22,907 INFO] Generating read length file...
[2018-07-09 15:54:47,489 INFO] Unpacking bowtie2 index tar...
[2018-07-09 15:54:47,493 INFO] run_shell_cmd: PID=164718, CMD=tar xvf /labs/smontgom/mdegorte/ATAC/atac-seq-pipeline/cromwell-executions/atac/449accb2-e994-40e1-912f-753255616799/ca
ll-bowtie2/shard-0/inputs/-1037641694/GRCh38_no_alt_analysis_set_GCA_000001405.15.fasta.tar -C .
PID=164718: tar: /labs/smontgom/mdegorte/ATAC/atac-seq-pipeline/cromwell-executions/atac/449accb2-e994-40e1-912f-753255616799/call-bowtie2/shard-0/inputs/-1037641694/GRCh38_no_alt_a
nalysis_set_GCA_000001405.15.fasta.tar: Cannot open: Permission denied
PID=164718: tar: Error is not recoverable: exiting now
[2018-07-09 15:54:47,666 ERROR] Unknown exception caught. Killing process group 164718...
Traceback (most recent call last):
File "/labs/smontgom/mdegorte/ATAC/atac-seq-pipeline/src/encode_common.py", line 224, in run_shell_cmd
p.returncode, cmd)
subprocess.CalledProcessError: Command 'tar xvf /labs/smontgom/mdegorte/ATAC/atac-seq-pipeline/cromwell-executions/atac/449accb2-e994-40e1-912f-753255616799/call-bowtie2/shard-0/inp
uts/-1037641694/GRCh38_no_alt_analysis_set_GCA_000001405.15.fasta.tar -C .' returned non-zero exit status 2
Could this be related to how Cromwell handles links? (https://github.com/ENCODE-DCC/atac-seq-pipeline/issues/18) I'm using Cromwell-33.1.
Thanks for your help with this!

Those errors :
ln: failed to access ‘.bam’: No such file or directory
ln: failed to access ‘.bai’: No such file or directory
ln: failed to access ‘.align.log’: No such file or directory
ln: failed to access ‘.flagstat.qc’: No such file or directory

are coming from the file script in your call-bowtie2 . There is no output files so the "ln -l" comands that want to link the ouputs in the glob directory are failing.
This is because this command fails :
tar xvf /labs/smontgom/mdegorte/ATAC/atac-seq-pipeline/cromwell-executions/atac/449accb2-e994-40e1-912f-753255616799/call-bowtie2/shard-0/inputs/-1037641694/GRCh38_no_alt_analysis_set_GCA_000001405.15.fasta.tar -C

because the fiel might not be there due to a link problem here :

There was a permission problem and it's fixed now.
Can you check if you can see this file (/ifs/scratch/leepc12/pipeline_genome_data/hg38/bowtie2_index/GRCh38_no_alt_analysis_set_GCA_000001405.15.fasta.tar) on SCG?

Yes, I can see the file, and the call-bowtie2 step completed. Thank you for your help with this!


/v1/register-yourself call to be included in the index.js

Assertion failure details
In RO Individual Services , following are the assertions that are failed because of this problem,

/v1/register-application:: Attribute correctness / client-update-operation checked?
/v1/register-application:: Update LTP notified? / embedding-operation update ::  Check /core-model-1-4:control-construct
/v1/register-application :: Update LTP notified? /operation-client-update-operation update::  Check /core-model-1-4:control-construct
/v1/relay-server-replacement::Forwardings triggered? / ServerReplacementBroadcast
/v1/relay-operation-update:: Request notified?

Reason for the failure :
Because initial load file does not contain op-c of RegistryOffice  in TypeApprovalCasuesRequestForEmbedding produces undefined for embedding-operation , this shall be solved once register-yourself is sent automatically while deploying the application.


Jenkins fix version change errors after Android 2.22.0

LEARNER-7832
Notes

Update Appium version in "trigger_aws.yml" file
Update Android release (.apk) name in jenkins file

@nedo please review this PR ?

Jenkins run test


Bot is not making new channels or properly repsonding

the bot, when Dm'ed says "Hello! If you wish to start a new ticket, please use !start." but then I say !start, the bot says
"Hello! If you wish to start a new ticket, please use !start.
Hello, please describe your issue briefly, the more details, the faster it will be for the staff team to help you. Type cancel to cancel."
I then proceed to give the feedback, and it happens all over again.

Hello, thank you for your report. I will be looking forward the issue.


Feat/XRPL connector v2

Before submitting this PR, please make sure:

[x] Your code builds clean without any errors or warnings
[x] You are using approved title ("feat/", "fix/", "docs/", "refactor/")

A description of the changes proposed in the pull request:

This PR reworks the Ripple DEX connector so that It implements the CLOBish class
Add a brand new order tracking system to help tracking and persisting trade orders submitted to Ripple DEX

Tests performed by the developer:

Tested posting and cancelling orders as integration tests
Tested tracking the state of orders as integration tests
Tested the CLOB Spot endpoints for Ripple DEX as routes tests

Tips for QA testing:

Issue connected to this PR: https://github.com/hummingbot/gateway/issues/103
Proposal: https://snapshot.org/#/hbot-ip.eth/proposal/0x07b027fc420274d26add346ed65a2f7e3e5a662bd5317d2ddb2dd02562a7b2d2
Currently I'm finishing the integration tests where it will simulate trading operations with order tracking to meet the required % coverage, I will update the PR with those tests in upcoming days.

Planned Tests To Add:

[x] Connector routes test (test/connectors/xrpl/xrpl.routes.test.ts)
[ ] Connector integration test
[ ] Chain routes test
[ ] Chain controller tests

Steps:

Cloned and installed feature branch and Client development
Connected testnet XRPL wallet
Check balance command

Actual:
2023-06-15 13:48:44,687 - 847867 - hummingbot.core.utils.async_utils - ERROR - Unhandled error in background task: 'NoneType' object is not subscriptable
Traceback (most recent call last):
  File "/home/ubuntu/development/hummingbot/core/utils/async_utils.py", line 9, in safe_wrapper
    return await c
  File "/home/ubuntu/development/hummingbot/client/command/balance_command.py", line 84, in show_balances
    all_ex_bals = await asyncio.wait_for(
  File "/home/ubuntu/miniconda3/envs/hummingbot/lib/python3.8/asyncio/tasks.py", line 483, in wait_for
    return fut.result()
  File "/home/ubuntu/development/hummingbot/user/user_balances.py", line 140, in all_balances_all_exchanges
    await self.update_exchanges(client_config_map)
  File "/home/ubuntu/development/hummingbot/user/user_balances.py", line 136, in update_exchanges
    results = await safe_gather(*tasks)
  File "/home/ubuntu/development/hummingbot/core/utils/async_utils.py", line 22, in safe_gather
    return await asyncio.gather(*args, **kwargs)
  File "/home/ubuntu/development/hummingbot/user/user_balances.py", line 110, in update_exchange_balance
    return await self.add_exchange(exchange_name, client_config_map, **api_keys)
  File "/home/ubuntu/development/hummingbot/user/user_balances.py", line 86, in add_exchange
    market = UserBalances.connect_market(exchange, client_config_map, **api_details)
  File "/home/ubuntu/development/hummingbot/user/user_balances.py", line 25, in connect_market
    init_params = conn_setting.conn_init_parameters(
  File "/home/ubuntu/development/hummingbot/client/settings.py", line 263, in conn_init_parameters
    connector_name=connector_spec["connector"],
TypeError: 'NoneType' object is not subscriptable

@mlguys Could you please tell if you have Any updates on this bounty PR?

@mlguys Could you please resolve branch conflicts?

hi @mlguys.
Please be informed about hummingbot ongoing migration to python3.10.
The new PR referring to the https://github.com/hummingbot/hummingbot/pull/6389 will be created and merged before all the connectors PRs to prevent connectors conflicts with python3.10 environment

Hey @nikspz, sorry for being unresponsive. I've been busy fixing the order tracker logic since I found some bugs during making integration tests. Fortunately I have resolved the logic issue and pushed integration tests which should cover more than 75% cov of connector code.
I will now proceed to fix the bug you mentioned above and make sure this PR is conflict free.

Hey guys, could you re-run the unit tests again? I have pushed fixes to address the test errors.

Tests performed:

cloned and installed XRPL connector: Client PR6535 + gateway PR128
connected testnet succesfully
created pureMM startegy using USD/VND pair
started the strategy
review order creation is inconsistent (logs fails to show orders not created on the exchange/ Client strategy created one order instead of pair of two, when balance is sufficient to place both)

gw128.zip
logs_puremm1.log
logs_gateway_app-xrpl.log

Hey @nikspz, thanks for testing, currently I'm busy with moving to new home so I'm unable to work on this yet. Will resolve this next week.

Hey @nikspz, I'm still working on this but due to tight schedule with other connector and limited work hours during my moving to another living space, I'm unable to finish this yet. However, I get more time for this week so I will push this, hopefully will be able to push some fixes by today.

hi @mlguys Could you please provide updates and fix branch conflicts?

Hey @nikspz, I've resolved the conflicts and added fix to your issue, could you check it?

Tests performed:

cloned and installed XRPL connector: Client PR6535 + gateway PR128
connected testnet succesfully
created pureMM startegy using USD/VND pair
started the strategy
review order creation is inconsistent (Client failed to created orders for both sides, created only one)

xrpl.zip
logs_xrp4test.log
2023-11-13 20:48:43,407 - 3824428 - hummingbot.client.hummingbot_application - INFO - start command initiated.
2023-11-13 20:48:43,511 - 3824428 - hummingbot.connector.gateway.clob_spot.gateway_clob_spot.GatewayCLOBSPOT - INFO - Network status has changed to NetworkStatus.CONNECTED. Starting networking...
2023-11-13 20:48:43,931 - 3824428 - hummingbot.core.data_type.order_book_tracker - INFO - Initialized order book for USD-VND. 1/1 completed.
2023-11-13 20:48:45,001 - 3824428 - hummingbot.strategy.pure_market_making.pure_market_making - INFO - (USD-VND) Creating 1 bid orders at (Size, Price): ['4 USD, 21532.5 VND']
2023-11-13 20:48:45,002 - 3824428 - hummingbot.strategy.pure_market_making.pure_market_making - INFO - (USD-VND) Creating 1 ask orders at (Size, Price): ['4 USD, 21967.5 VND']
2023-11-13 20:48:45,801 - 3824428 - hummingbot.core.data_type.order_book_tracker - NETWORK - Unexpected error while fetching last trade price.
Traceback (most recent call last):
  File "/home/nikita/xrpl6535/hummingbot/core/data_type/order_book_tracker.py", line 160, in _update_last_trade_prices_loop
    last_prices = await self._data_source.get_last_traded_prices(**args)
  File "/home/nikita/xrpl6535/hummingbot/connector/gateway/clob_spot/gateway_clob_api_order_book_data_source.py", line 26, in get_last_traded_prices
    last_traded_prices = {
  File "/home/nikita/xrpl6535/hummingbot/connector/gateway/clob_spot/gateway_clob_api_order_book_data_source.py", line 27, in <dictcomp>
    trading_pair: float(await self._api_data_source.get_last_traded_price(trading_pair=trading_pair))
  File "/home/nikita/xrpl6535/hummingbot/connector/gateway/clob_spot/data_sources/xrpl/xrpl_api_data_source.py", line 196, in get_last_traded_price
    ticker_data = await self._get_ticker_data(trading_pair=trading_pair)
  File "/home/nikita/xrpl6535/hummingbot/connector/gateway/clob_spot/data_sources/xrpl/xrpl_api_data_source.py", line 346, in _get_ticker_data
    return ticker_data["markets"][trading_pair]
TypeError: list indices must be integers or slices, not str
2023-11-13 20:48:46,348 - 3824428 - hummingbot.core.rate_oracle.rate_oracle - INFO - Network status has changed to NetworkStatus.CONNECTED. Starting networking...
2023-11-13 20:48:50,054 - 3824428 - hummingbot.core.event.event_reporter - EVENT_LOG - {"timestamp":<PHONE_NUMBER>.0, "order_id": "0x0000000000000000000998584e8f07792895a60a1fb1e1ee4fefdb10a1468da2", "exchange_order_id": "33596625", "event_name": "OrderCancelledEvent", "event_source": "xrpl_xrpl_testnet"}
2023-11-13 20:48:50,054 - 3824428 - hummingbot.connector.client_order_tracker - INFO - Successfully canceled order 0x0000000000000000000998584e8f07792895a60a1fb1e1ee4fefdb10a1468da2.
2023-11-13 20:48:50,132 - 3824428 - hummingbot.connector.client_order_tracker - INFO - Created LIMIT SELL order 0x0000000000000000000998584e8f07792895a60a1fb1e1ee4fefdb10a1469001 for 4.000000000000000 USD-VND.
2023-11-13 20:48:50,176 - 3824428 - hummingbot.core.event.event_reporter - EVENT_LOG - {"timestamp":<PHONE_NUMBER>.0, "type": "OrderType.LIMIT", "trading_pair": "USD-VND", "amount": "4.000000000000000", "price": "21967.500000000000000", "order_id": "0x0000000000000000000998584e8f07792895a60a1fb1e1ee4fefdb10a1469001", "creation_timestamp":<PHONE_NUMBER>.0, "exchange_order_id": "33596626", "leverage": 1, "position": "NIL", "event_name": "SellOrderCreatedEvent", "event_source": "xrpl_xrpl_testnet"}
2023-11-13 20:49:16,430 - 3824428 - hummingbot.core.data_type.order_book_tracker - NETWORK - Unexpected error while fetching last trade price.
Traceback (most recent call last):
  File "/home/nikita/xrpl6535/hummingbot/core/data_type/order_book_tracker.py", line 160, in _update_last_trade_prices_loop
    last_prices = await self._data_source.get_last_traded_prices(**args)
  File "/home/nikita/xrpl6535/hummingbot/connector/gateway/clob_spot/gateway_clob_api_order_book_data_source.py", line 26, in get_last_traded_prices
    last_traded_prices = {
  File "/home/nikita/xrpl6535/hummingbot/connector/gateway/clob_spot/gateway_clob_api_order_book_data_source.py", line 27, in <dictcomp>
    trading_pair: float(await self._api_data_source.get_last_traded_price(trading_pair=trading_pair))
  File "/home/nikita/xrpl6535/hummingbot/connector/gateway/clob_spot/data_sources/xrpl/xrpl_api_data_source.py", line 196, in get_last_traded_price
    ticker_data = await self._get_ticker_data(trading_pair=trading_pair)
  File "/home/nikita/xrpl6535/hummingbot/connector/gateway/clob_spot/data_sources/xrpl/xrpl_api_data_source.py", line 346, in _get_ticker_data
    return ticker_data["markets"][trading_pair]
TypeError: list indices must be integers or slices, not str
2023-11-13 20:49:46,777 - 3824428 - hummingbot.core.data_type.order_book_tracker - NETWORK - Unexpected error while fetching last trade price.
Traceback (most recent call last):
  File "/home/nikita/xrpl6535/hummingbot/core/data_type/order_book_tracker.py", line 160, in _update_last_trade_prices_loop
    last_prices = await self._data_source.get_last_traded_prices(**args)
  File "/home/nikita/xrpl6535/hummingbot/connector/gateway/clob_spot/gateway_clob_api_order_book_data_source.py", line 26, in get_last_traded_prices
    last_traded_prices = {
  File "/home/nikita/xrpl6535/hummingbot/connector/gateway/clob_spot/gateway_clob_api_order_book_data_source.py", line 27, in <dictcomp>
    trading_pair: float(await self._api_data_source.get_last_traded_price(trading_pair=trading_pair))
  File "/home/nikita/xrpl6535/hummingbot/connector/gateway/clob_spot/data_sources/xrpl/xrpl_api_data_source.py", line 196, in get_last_traded_price
    ticker_data = await self._get_ticker_data(trading_pair=trading_pair)
  File "/home/nikita/xrpl6535/hummingbot/connector/gateway/clob_spot/data_sources/xrpl/xrpl_api_data_source.py", line 346, in _get_ticker_data
    return ticker_data["markets"][trading_pair]
TypeError: list indices must be integers or slices, not str

Hey @nikspz, Thanks for testing. I've added a fix on https://github.com/hummingbot/hummingbot/pull/6535 to fix your issue, could you check?

Steps to reproduce:

cloned and installed XRPL connector: Client PR6535 + gateway PR128
connected testnet succesfully
created pureMM startegy using USD/VND pair
started the strategy

Actual:
Order creation is inconsistent (failed to created orders for both sides, one order cancelled immediately and only one created)
Available balance in status not updated (should be changed depending on order amount)
6635.zip
logs_xrp2test.log
2023-11-14 17:43:53,388 - 4125298 - hummingbot.client.hummingbot_application - INFO - start command initiated.
2023-11-14 17:43:53,532 - 4125298 - hummingbot.connector.gateway.clob_spot.gateway_clob_spot.GatewayCLOBSPOT - INFO - Network status has changed to NetworkStatus.CONNECTED. Starting networking...
2023-11-14 17:43:53,908 - 4125298 - hummingbot.core.data_type.order_book_tracker - INFO - Initialized order book for USD-VND. 1/1 completed.
2023-11-14 17:43:55,002 - 4125298 - hummingbot.strategy.pure_market_making.pure_market_making - INFO - (USD-VND) Creating 1 bid orders at (Size, Price): ['3 USD, 19710.5 VND']
2023-11-14 17:43:55,003 - 4125298 - hummingbot.strategy.pure_market_making.pure_market_making - INFO - (USD-VND) Creating 1 ask orders at (Size, Price): ['3 USD, 19789.5 VND']
2023-11-14 17:43:56,833 - 4125298 - hummingbot.core.rate_oracle.rate_oracle - INFO - Network status has changed to NetworkStatus.CONNECTED. Starting networking...
2023-11-14 17:44:00,189 - 4125298 - hummingbot.core.event.event_reporter - EVENT_LOG - {"timestamp":<PHONE_NUMBER>.0, "order_id": "0x000000000000000000138a60579a4cd19823a2c7ba8c9766c835fbd1fe59f870", "exchange_order_id": "33596726", "event_name": "OrderCancelledEvent", "event_source": "xrpl_xrpl_testnet"}
2023-11-14 17:44:00,189 - 4125298 - hummingbot.connector.client_order_tracker - INFO - Successfully canceled order 0x000000000000000000138a60579a4cd19823a2c7ba8c9766c835fbd1fe59f870.
2023-11-14 17:44:00,194 - 4125298 - hummingbot.connector.client_order_tracker - INFO - Created LIMIT SELL order 0x000000000000000000138a60579a4cd19823a2c7ba8c9766c835fbd1fe59fbd9 for 3.000000000000000 USD-VND.
2023-11-14 17:44:00,218 - 4125298 - hummingbot.core.event.event_reporter - EVENT_LOG - {"timestamp":<PHONE_NUMBER>.0, "type": "OrderType.LIMIT", "trading_pair": "USD-VND", "amount": "3.000000000000000", "price": "19789.500000000000000", "order_id": "0x000000000000000000138a60579a4cd19823a2c7ba8c9766c835fbd1fe59fbd9", "creation_timestamp":<PHONE_NUMBER>.0, "exchange_order_id": "33596727", "leverage": 1, "position": "NIL", "event_name": "SellOrderCreatedEvent", "event_source": "xrpl_xrpl_testnet"}
2023-11-14 17:44:55,002 - 4125298 - hummingbot.strategy.pure_market_making.pure_market_making - INFO - (USD-VND) Canceling the limit order 0x000000000000000000138a60579a4cd19823a2c7ba8c9766c835fbd1fe59fbd9. [clock=2023-11-14 17:44:55+00:00]
2023-11-14 17:45:00,090 - 4125298 - hummingbot.core.event.event_reporter - EVENT_LOG - {"timestamp":<PHONE_NUMBER>.0, "order_id": "0x000000000000000000138a60579a4cd19823a2c7ba8c9766c835fbd1fe59fbd9", "exchange_order_id": "33596727", "event_name": "OrderCancelledEvent", "event_source": "xrpl_xrpl_testnet"}
2023-11-14 17:45:00,090 - 4125298 - hummingbot.connector.client_order_tracker - INFO - Successfully canceled order 0x000000000000000000138a60579a4cd19823a2c7ba8c9766c835fbd1fe59fbd9.
2023-11-14 17:45:01,001 - 4125298 - hummingbot.strategy.pure_market_making.pure_market_making - INFO - (USD-VND) Creating 1 bid orders at (Size, Price): ['3 USD, 19710.5 VND']
2023-11-14 17:45:01,002 - 4125298 - hummingbot.strategy.pure_market_making.pure_market_making - INFO - (USD-VND) Creating 1 ask orders at (Size, Price): ['3 USD, 19789.5 VND']
2023-11-14 17:45:10,131 - 4125298 - hummingbot.core.event.event_reporter - EVENT_LOG - {"timestamp":<PHONE_NUMBER>.0, "order_id": "0x000000000000000000138a60579a4cd19823a2c7ba8c9766c835fbd20249086f", "exchange_order_id": "33596729", "event_name": "OrderCancelledEvent", "event_source": "xrpl_xrpl_testnet"}
2023-11-14 17:45:10,132 - 4125298 - hummingbot.connector.client_order_tracker - INFO - Successfully canceled order 0x000000000000000000138a60579a4cd19823a2c7ba8c9766c835fbd20249086f.
2023-11-14 17:45:10,137 - 4125298 - hummingbot.connector.client_order_tracker - INFO - Created LIMIT SELL order 0x000000000000000000138a60579a4cd19823a2c7ba8c9766c835fbd202490a93 for 3.000000000000000 USD-VND.
2023-11-14 17:45:10,189 - 4125298 - hummingbot.core.event.event_reporter - EVENT_LOG - {"timestamp":<PHONE_NUMBER>.0, "type": "OrderType.LIMIT", "trading_pair": "USD-VND", "amount": "3.000000000000000", "price": "19789.500000000000000", "order_id": "0x000000000000000000138a60579a4cd19823a2c7ba8c9766c835fbd202490a93", "creation_timestamp":<PHONE_NUMBER>.0, "exchange_order_id": "33596730", "leverage": 1, "position": "NIL", "event_name": "SellOrderCreatedEvent", "event_source": "xrpl_xrpl_testnet"}
2023-11-14 17:46:01,002 - 4125298 - hummingbot.strategy.pure_market_making.pure_market_making - INFO - (USD-VND) Canceling the limit order 0x000000000000000000138a60579a4cd19823a2c7ba8c9766c835fbd202490a93. [clock=2023-11-14 17:46:01+00:00]
2023-11-14 17:46:05,118 - 4125298 - hummingbot.core.event.event_reporter - EVENT_LOG - {"timestamp":<PHONE_NUMBER>.0, "order_id": "0x000000000000000000138a60579a4cd19823a2c7ba8c9766c835fbd202490a93", "exchange_order_id": "33596730", "event_name": "OrderCancelledEvent", "event_source": "xrpl_xrpl_testnet"}
2023-11-14 17:46:05,118 - 4125298 - hummingbot.connector.client_order_tracker - INFO - Successfully canceled order 0x000000000000000000138a60579a4cd19823a2c7ba8c9766c835fbd202490a93.
2023-11-14 17:46:06,002 - 4125298 - hummingbot.strategy.pure_market_making.pure_market_making - INFO - (USD-VND) Creating 1 bid orders at (Size, Price): ['3 USD, 19710.5 VND']
2023-11-14 17:46:06,003 - 4125298 - hummingbot.strategy.pure_market_making.pure_market_making - INFO - (USD-VND) Creating 1 ask orders at (Size, Price): ['3 USD, 19789.5 VND']
2023-11-14 17:46:15,102 - 4125298 - hummingbot.core.event.event_reporter - EVENT_LOG - {"timestamp":<PHONE_NUMBER>.0, "order_id": "0x000000000000000000138a60579a4cd19823a2c7ba8c9766c835fbd20628de4b", "exchange_order_id": "33596732", "event_name": "OrderCancelledEvent", "event_source": "xrpl_xrpl_testnet"}
2023-11-14 17:46:15,102 - 4125298 - hummingbot.connector.client_order_tracker - INFO - Successfully canceled order 0x000000000000000000138a60579a4cd19823a2c7ba8c9766c835fbd20628de4b.
2023-11-14 17:46:15,106 - 4125298 - hummingbot.connector.client_order_tracker - INFO - Created LIMIT SELL order 0x000000000000000000138a60579a4cd19823a2c7ba8c9766c835fbd20628e01a for 3.000000000000000 USD-VND.
2023-11-14 17:46:15,116 - 4125298 - hummingbot.core.event.event_reporter - EVENT_LOG - {"timestamp":<PHONE_NUMBER>.0, "type": "OrderType.LIMIT", "trading_pair": "USD-VND", "amount": "3.000000000000000", "price": "19789.500000000000000", "order_id": "0x000000000000000000138a60579a4cd19823a2c7ba8c9766c835fbd20628e01a", "creation_timestamp":<PHONE_NUMBER>.0, "exchange_order_id": "33596733", "leverage": 1, "position": "NIL", "event_name": "SellOrderCreatedEvent", "event_source": "xrpl_xrpl_testnet"}
2023-11-14 17:47:06,007 - 4125298 - hummingbot.strategy.pure_market_making.pure_market_making - INFO - (USD-VND) Canceling the limit order 0x000000000000000000138a60579a4cd19823a2c7ba8c9766c835fbd20628e01a. [clock=2023-11-14 17:47:06+00:00]
2023-11-14 17:47:15,088 - 4125298 - hummingbot.core.event.event_reporter - EVENT_LOG - {"timestamp":<PHONE_NUMBER>.0, "order_id": "0x000000000000000000138a60579a4cd19823a2c7ba8c9766c835fbd20628e01a", "exchange_order_id": "33596733", "event_name": "OrderCancelledEvent", "event_source": "xrpl_xrpl_testnet"}
2023-11-14 17:47:15,088 - 4125298 - hummingbot.connector.client_order_tracker - INFO - Successfully canceled order 0x000000000000000000138a60579a4cd19823a2c7ba8c9766c835fbd20628e01a.
2023-11-14 17:47:16,001 - 4125298 - hummingbot.strategy.pure_market_making.pure_market_making - INFO - (USD-VND) Creating 1 bid orders at (Size, Price): ['3 USD, 19710.5 VND']
2023-11-14 17:47:16,001 - 4125298 - hummingbot.strategy.pure_market_making.pure_market_making - INFO - (USD-VND) Creating 1 ask orders at (Size, Price): ['3 USD, 19789.5 VND']
2023-11-14 17:47:20,544 - 4125298 - hummingbot.core.event.event_reporter - EVENT_LOG - {"timestamp":<PHONE_NUMBER>.0, "order_id": "0x000000000000000000138a60579a4cd19823a2c7ba8c9766c835fbd20a54f7e3", "exchange_order_id": "33596735", "event_name": "OrderCancelledEvent", "event_source": "xrpl_xrpl_testnet"}
2023-11-14 17:47:20,544 - 4125298 - hummingbot.connector.client_order_tracker - INFO - Successfully canceled order 0x000000000000000000138a60579a4cd19823a2c7ba8c9766c835fbd20a54f7e3.
2023-11-14 17:47:21,021 - 4125298 - hummingbot.connector.client_order_tracker - INFO - Created LIMIT SELL order 0x000000000000000000138a60579a4cd19823a2c7ba8c9766c835fbd20a54f9ad for 3.000000000000000 USD-VND.
2023-11-14 17:47:21,034 - 4125298 - hummingbot.core.event.event_reporter - EVENT_LOG - {"timestamp":<PHONE_NUMBER>.0, "type": "OrderType.LIMIT", "trading_pair": "USD-VND", "amount": "3.000000000000000", "price": "19789.500000000000000", "order_id": "0x000000000000000000138a60579a4cd19823a2c7ba8c9766c835fbd20a54f9ad", "creation_timestamp":<PHONE_NUMBER>.0, "exchange_order_id": "33596736", "leverage": 1, "position": "NIL", "event_name": "SellOrderCreatedEvent", "event_source": "xrpl_xrpl_testnet"}
2023-11-14 17:48:16,003 - 4125298 - hummingbot.strategy.pure_market_making.pure_market_making - INFO - (USD-VND) Canceling the limit order 0x000000000000000000138a60579a4cd19823a2c7ba8c9766c835fbd20a54f9ad. [clock=2023-11-14 17:48:16+00:00]
2023-11-14 17:48:20,045 - 4125298 - hummingbot.core.event.event_reporter - EVENT_LOG - {"timestamp":<PHONE_NUMBER>.0, "order_id": "0x000000000000000000138a60579a4cd19823a2c7ba8c9766c835fbd20a54f9ad", "exchange_order_id": "33596736", "event_name": "OrderCancelledEvent", "event_source": "xrpl_xrpl_testnet"}
2023-11-14 17:48:20,046 - 4125298 - hummingbot.connector.client_order_tracker - INFO - Successfully canceled order 0x000000000000000000138a60579a4cd19823a2c7ba8c9766c835fbd20a54f9ad.
2023-11-14 17:48:21,001 - 4125298 - hummingbot.strategy.pure_market_making.pure_market_making - INFO - (USD-VND) Creating 1 bid orders at (Size, Price): ['3 USD, 19710.5 VND']
2023-11-14 17:48:21,002 - 4125298 - hummingbot.strategy.pure_market_making.pure_market_making - INFO - (USD-VND) Creating 1 ask orders at (Size, Price): ['3 USD, 19789.5 VND']
2023-11-14 17:48:21,747 - 4125298 - hummingbot.core.gateway.gateway_http_client - NETWORK - Server disconnected
Traceback (most recent call last):
  File "/home/nikita/xrpl6535/hummingbot/core/gateway/gateway_http_client.py", line 184, in api_request
    response = await client.get(url, params=params)
  File "/home/nikita/miniconda3/envs/hummingbot/lib/python3.10/site-packages/aiohttp/client.py", line 586, in _request
    await resp.start(conn)
  File "/home/nikita/miniconda3/envs/hummingbot/lib/python3.10/site-packages/aiohttp/client_reqrep.py", line 905, in start
    message, payload = await protocol.read()  # type: ignore[union-attr]
  File "/home/nikita/miniconda3/envs/hummingbot/lib/python3.10/site-packages/aiohttp/streams.py", line 616, in read
    await self._waiter
aiohttp.client_exceptions.ServerDisconnectedError: Server disconnected
2023-11-14 17:48:21,815 - 4125298 - hummingbot.core.gateway.gateway_http_client - WARNING - Call to https://localhost:15888/clob/ticker failed. See logs for more details.
2023-11-14 17:48:21,815 - 4125298 - hummingbot.core.data_type.order_book_tracker - NETWORK - Unexpected error while fetching last trade price.
Traceback (most recent call last):
  File "/home/nikita/xrpl6535/hummingbot/core/data_type/order_book_tracker.py", line 160, in _update_last_trade_prices_loop
    last_prices = await self._data_source.get_last_traded_prices(**args)
  File "/home/nikita/xrpl6535/hummingbot/connector/gateway/clob_spot/gateway_clob_api_order_book_data_source.py", line 26, in get_last_traded_prices
    last_traded_prices = {
  File "/home/nikita/xrpl6535/hummingbot/connector/gateway/clob_spot/gateway_clob_api_order_book_data_source.py", line 27, in <dictcomp>
    trading_pair: float(await self._api_data_source.get_last_traded_price(trading_pair=trading_pair))
  File "/home/nikita/xrpl6535/hummingbot/connector/gateway/clob_spot/data_sources/xrpl/xrpl_api_data_source.py", line 196, in get_last_traded_price
    ticker_data = await self._get_ticker_data(trading_pair=trading_pair)
  File "/home/nikita/xrpl6535/hummingbot/connector/gateway/clob_spot/data_sources/xrpl/xrpl_api_data_source.py", line 340, in _get_ticker_data
    ticker_data = await self._get_gateway_instance().get_clob_ticker(
  File "/home/nikita/xrpl6535/hummingbot/core/gateway/gateway_http_client.py", line 921, in get_clob_ticker
    resp = await self.api_request(method="get", path_url="clob/ticker", params=request_payload)
  File "/home/nikita/xrpl6535/hummingbot/core/gateway/gateway_http_client.py", line 222, in api_request
    raise e
  File "/home/nikita/xrpl6535/hummingbot/core/gateway/gateway_http_client.py", line 184, in api_request
    response = await client.get(url, params=params)
  File "/home/nikita/miniconda3/envs/hummingbot/lib/python3.10/site-packages/aiohttp/client.py", line 586, in _request
    await resp.start(conn)
  File "/home/nikita/miniconda3/envs/hummingbot/lib/python3.10/site-packages/aiohttp/client_reqrep.py", line 905, in start
    message, payload = await protocol.read()  # type: ignore[union-attr]
  File "/home/nikita/miniconda3/envs/hummingbot/lib/python3.10/site-packages/aiohttp/streams.py", line 616, in read
    await self._waiter
aiohttp.client_exceptions.ServerDisconnectedError: Server disconnected
2023-11-14 17:48:25,390 - 4125298 - hummingbot.core.event.event_reporter - EVENT_LOG - {"timestamp":<PHONE_NUMBER>.0, "order_id": "0x000000000000000000138a60579a4cd19823a2c7ba8c9766c835fbd20e34ca93", "exchange_order_id": "33596738", "event_name": "OrderCancelledEvent", "event_source": "xrpl_xrpl_testnet"}
2023-11-14 17:48:25,390 - 4125298 - hummingbot.connector.client_order_tracker - INFO - Successfully canceled order 0x000000000000000000138a60579a4cd19823a2c7ba8c9766c835fbd20e34ca93.
2023-11-14 17:48:25,843 - 4125298 - hummingbot.connector.client_order_tracker - INFO - Created LIMIT SELL order 0x000000000000000000138a60579a4cd19823a2c7ba8c9766c835fbd20e34cc96 for 3.000000000000000 USD-VND.
2023-11-14 17:48:25,869 - 4125298 - hummingbot.core.event.event_reporter - EVENT_LOG - {"timestamp":<PHONE_NUMBER>.0, "type": "OrderType.LIMIT", "trading_pair": "USD-VND", "amount": "3.000000000000000", "price": "19789.500000000000000", "order_id": "0x000000000000000000138a60579a4cd19823a2c7ba8c9766c835fbd20e34cc96", "creation_timestamp":<PHONE_NUMBER>.0, "exchange_order_id": "33596739", "leverage": 1, "position": "NIL", "event_name": "SellOrderCreatedEvent", "event_source": "xrpl_xrpl_testnet"}
2023-11-14 17:49:21,002 - 4125298 - hummingbot.strategy.pure_market_making.pure_market_making - INFO - (USD-VND) Canceling the limit order 0x000000000000000000138a60579a4cd19823a2c7ba8c9766c835fbd20e34cc96. [clock=2023-11-14 17:49:21+00:00]
2023-11-14 17:49:30,088 - 4125298 - hummingbot.core.event.event_reporter - EVENT_LOG - {"timestamp":<PHONE_NUMBER>.0, "order_id": "0x000000000000000000138a60579a4cd19823a2c7ba8c9766c835fbd20e34cc96", "exchange_order_id": "33596739", "event_name": "OrderCancelledEvent", "event_source": "xrpl_xrpl_testnet"}
2023-11-14 17:49:30,089 - 4125298 - hummingbot.connector.client_order_tracker - INFO - Successfully canceled order 0x000000000000000000138a60579a4cd19823a2c7ba8c9766c835fbd20e34cc96.

hi @mlguys, kindly Please add your updates here

Ran test on multiple order levels:

At first instances its working ok with creating and cancellation of orders
Start again with order levels 3 and this is where the issue we found:

Order refresh time of the strategy is 30mins
On first cancellation, other orders got stuck on Canceling active order..2023-11-29 12:52:14,057 - 7789 - hummingbot.core.event.event_reporter - EVENT_LOG - {"timestamp":<PHONE_NUMBER>.0, "order_id": "VGHBBEPUT60b42e101d3f53c74215300", "exchange_order_id": "33efe7870c3082a256bd6acd4dde9e3ef6995b0927fd8fe9d1f49dcd44de99cf", "event_name": "OrderCancelledEvent", "event_source": "vega_perpetual"}
2023-11-29 12:52:14,062 - 7789 - hummingbot.connector.client_order_tracker - INFO - Successfully canceled order VGHBBEPUT60b42e101d3f53c74215300.
2023-11-29 12:52:14,062 - 7789 - hummingbot.core.event.event_reporter - EVENT_LOG - {"timestamp":<PHONE_NUMBER>.0, "order_id": "VGHBBEPUT60b42e101d5073c74215300", "exchange_order_id": "6288ba37cea094c4d3ecf1a763244177c8ce7fd735411c46aa127fe528f24dea", "event_name": "OrderCancelledEvent", "event_source": "vega_perpetual"}
2023-11-29 12:52:14,065 - 7789 - hummingbot.connector.client_order_tracker - INFO - Successfully canceled order VGHBBEPUT60b42e101d5073c74215300.
2023-11-29 12:52:14,066 - 7789 - hummingbot.core.event.event_reporter - EVENT_LOG - {"timestamp":<PHONE_NUMBER>.0, "order_id": "VGHBSEPUT60b42e101d6373c74215300", "exchange_order_id": "51b9c7a62751e42f2b1dafbe33d3986167c712f8868dd8174c26671c80d01d8b", "event_name": "OrderCancelledEvent", "event_source": "vega_perpetual"}
2023-11-29 12:52:14,068 - 7789 - hummingbot.connector.client_order_tracker - INFO - Successfully canceled order VGHBSEPUT60b42e101d6373c74215300.
2023-11-29 12:52:15,001 - 7789 - hummingbot.strategy.perpetual_market_making.perpetual_market_making - INFO - Canceling active order VGHBBEPUT60b42e101d4bb3c74215300.
2023-11-29 12:52:15,001 - 7789 - hummingbot.strategy.perpetual_market_making.perpetual_market_making - INFO - Canceling active order VGHBSEPUT60b42e101d67b3c74215300.
2023-11-29 12:52:15,001 - 7789 - hummingbot.strategy.perpetual_market_making.perpetual_market_making - INFO - Canceling active order VGHBSEPUT60b42e101d6ba3c74215300.

Eventually got a WARNING and then continues to try cancel the order:2023-11-29 12:53:12,005 - 7789 - hummingbot.connector.derivative.vega_perpetual.vega_perpetual_derivative.VegaPerpetualDerivative - WARNING - Not canceling order due to state OrderState.PENDING_CANCEL for VGHBBEPUT60b42e101d4bb3c74215300
2023-11-29 12:53:12,005 - 7789 - hummingbot.connector.derivative.vega_perpetual.vega_perpetual_derivative.VegaPerpetualDerivative - WARNING - Not canceling order due to state OrderState.PENDING_CANCEL for VGHBSEPUT60b42e101d67b3c74215300
2023-11-29 12:53:12,005 - 7789 - hummingbot.connector.derivative.vega_perpetual.vega_perpetual_derivative.VegaPerpetualDerivative - WARNING - Not canceling order due to state OrderState.PENDING_CANCEL for VGHBSEPUT60b42e101d6ba3c74215300

Got Failed transaction submission for cancel...2023-11-29 12:54:14,044 - 7789 - hummingbot.strategy.perpetual_market_making.perpetual_market_making - INFO - Canceling active order VGHBBEPUT60b42e101d4bb3c74215300.
2023-11-29 12:54:14,044 - 7789 - hummingbot.strategy.perpetual_market_making.perpetual_market_making - INFO - Canceling active order VGHBSEPUT60b42e101d67b3c74215300.
2023-11-29 12:54:14,044 - 7789 - hummingbot.strategy.perpetual_market_making.perpetual_market_making - INFO - Canceling active order VGHBSEPUT60b42e101d6ba3c74215300.
2023-11-29 12:54:14,248 - 7789 - hummingbot.connector.derivative.vega_perpetual.vega_perpetual_derivative.VegaPerpetualDerivative - WARNING - Failed transaction submission for cancel of VGHBSEPUT60b42e101d67b3c74215300 with {'success': True, 'txHash': '5AB6BC704B4E48267BB9F894BEC88DE2C96681B1C4855AD8E9C20B0E4E6FFDC0', 'code': 89, 'data': '70617274792069732062616E6E65642066726F6D2073656E64696E67207472616E73616374696F6E73', 'log': '', 'height': '0'}
2023-11-29 12:54:14,295 - 7789 - hummingbot.connector.derivative.vega_perpetual.vega_perpetual_derivative.VegaPerpetualDerivative - WARNING - Failed transaction submission for cancel of VGHBBEPUT60b42e101d4bb3c74215300 with {'success': True, 'txHash': 'C28A30318EB6AE0136E22F471F7B7A31F1A7866DA7E7DEDF782C237B2DB470F4', 'code': 89, 'data': '70617274792069732062616E6E65642066726F6D2073656E64696E67207472616E73616374696F6E73', 'log': '', 'height': '0'}
2023-11-29 12:54:14,295 - 7789 - hummingbot.connector.derivative.vega_perpetual.vega_perpetual_derivative.VegaPerpetualDerivative - WARNING - Failed transaction submission for cancel of VGHBSEPUT60b42e101d6ba3c74215300 with {'success': True, 'txHash': '85E22A490AC87BD6D3FDDC954DC88CC8A35A1CD9E3D5C18E454D18FA29722CE1', 'code': 89, 'data': '70617274792069732062616E6E65642066726F6D2073656E64696E67207472616E73616374696F6E73', 'log': '', 'height': '0'}

The stuck on canceling active order has been happening every tick and the failed transaction submittion every minute.
Tried to run stop command, got a display of failed orders:

Tried to relaunch the client and start again and it recognized the stuck orders yet not able to cancel them

This issue resulted to the connector not able to submit orders with an error of RuntimeError: No active exception to reraise, see attached log file2023-11-29 13:10:15,001 - 10740 - hummingbot.strategy.perpetual_market_making.perpetual_market_making - INFO - (ETHUSDPERP-USDT) Creating 3 LIMIT bid orders at (Size, Price): ['0.007 ETHUSDPERP, 2049.6 USDT', '0.007 ETHUSDPERP, 2045.49 USDT', '0.007 ETHUSDPERP, 2041.38 USDT'] to OPEN position.
2023-11-29 13:10:15,001 - 10740 - hummingbot.strategy.perpetual_market_making.perpetual_market_making - INFO - (ETHUSDPERP-USDT) Creating 3  LIMIT ask orders at (Size, Price): ['0.007 ETHUSDPERP, 2061.94 USDT', '0.007 ETHUSDPERP, 2066.05 USDT', '0.007 ETHUSDPERP, 2070.16 USDT'] to OPEN position.
2023-11-29 13:10:17,684 - 10740 - hummingbot.connector.derivative.vega_perpetual.vega_perpetual_derivative.VegaPerpetualDerivative - NETWORK - Error submitting buy LIMIT order to Vega_perpetual for 0.007 ETHUSDPERP-USDT 2041.38.
Traceback (most recent call last):
  File "/Users/rapcomia/github/hummingbot/6628/hummingbot/connector/exchange_py_base.py", line 452, in _create_order
    await self._place_order_and_process_update(order=order, **kwargs,)
  File "/Users/rapcomia/github/hummingbot/6628/hummingbot/connector/derivative/vega_perpetual/vega_perpetual_derivative.py", line 339, in _place_order_and_process_update
    exchange_order_id, update_timestamp = await self._place_order(
  File "/Users/rapcomia/github/hummingbot/6628/hummingbot/connector/derivative/vega_perpetual/vega_perpetual_derivative.py", line 432, in _place_order
    raise
RuntimeError: No active exception to reraise
2023-11-29 13:10:17,687 - 10740 - hummingbot.connector.derivative.vega_perpetual.vega_perpetual_derivative.VegaPerpetualDerivative - WARNING - Failed to submit BUY order to Vega_perpetual. Check API key and network connection.
2023-11-29 13:10:17,687 - 10740 - hummingbot.core.event.event_reporter - EVENT_LOG - {"timestamp":<PHONE_NUMBER>.0, "order_id": "VGHBBEPUT60b438ce82310ce72fff9ee", "order_type": "OrderType.LIMIT", "event_name": "MarketOrderFailureEvent", "event_source": "vega_perpetual"}
2023-11-29 13:10:17,690 - 10740 - hummingbot.connector.client_order_tracker - INFO - Order VGHBBEPUT60b438ce82310ce72fff9ee has failed. Order Update: OrderUpdate(trading_pair='ETHUSDPERP-USDT', update_timestamp=1701234616.0, new_state=<OrderState.FAILED: 6>, client_order_id='VGHBBEPUT60b438ce82310ce72fff9ee', exchange_order_id=None, misc_updates=None)

Tried to manually close the stuck orders but got stuck as well on Unknown error occurred on exchange

Was able to cancel these orders after not using the exchange for couple of minutes, could we possible be rate limited or a delay on the tx request happened? 🤔
Tried to start the strategy again and stuck orders are recognized and noted as successfully cancelled. Connector was able to create and cancel orders at the same time too. ✅

Steps to reproduce

Setup a strategy or script with order multiple levels
Run for couple for hours and observe the behavior of connector

logs_perp-vega1.log
perp-vega1.yml.log

LGTM!


Display planned outage announcement on the home page

Ticket Link
Ticket 974
List of squashed commits

Change endpoint name
Console logs and getting rid of head message
Get rid of white spacing on top
Message at top
Finalize position of alert message
Rename endpoint and functions
Fix home page display
Commented out files
Display planned outage announcement on the home page
Messages are aligned and within page dimensions
Clean up code and add API error modal
Add getOutageMessages test
Fix test for announcements path
Separate alert messages and add close functionality
Dismissable alerts and test for controller
Get rid of white space between alerts
Add back call to the endpoint
Merge conflicts with main

Test Checklist

[x] Exhibits Clear Code Structure:
[x] Project Unit Tests Passed:
[ ] Project Jasmine Tests Passed:
[x] Executes Expected Functionality:
[x] Tested For Incorrect/Unexpected Inputs:

The Jasmine tests should be passing but because my origin main isn't syncing with the project's main, the test seems to be failing. Confirmed during the team meeting today that my "main" branch has a failing HomeController on it and all my old commits for this ticket (that shouldn't be in the main branch yet).

The Jasmine tests should be passing but because my origin main isn't syncing with the project's main, the test seems to be failing. Confirmed during the team meeting today that my "main" branch has a failing HomeController on it and all my old commits for this ticket (that shouldn't be in the main branch yet).

No worries, I can run the Jasmine test before I merge this PR to make sure it's passing.

Rebranched and made new pr: #879


[crypto] support Secp256k1RecoverableSignature in crypto.rs to further support recovering Ethereum signature to Rooch signature and vice versa

In PR #551 in file https://github.com/rooch-network/rooch/pull/551/files#diff-e2b13bff975b6a03ca701eef7fd8ca0dc1c002cdb5a9aac583dfcabcbf8142fc, I have removed Ethereum signature to directly implement and unify all signatures to Rooch signature scheme in authenticator.
In PR #532 in file https://github.com/rooch-network/rooch/pull/532/files#diff-4b575a0dfd3a01413c3287d253b6021e8dadd8d7d97902923e51e6c9ca2e1430, I have added converting Etheneum's RSV 65-byte signature to a recoverable Secp256k1 signature at Rooch for use cases in Ethereum blockchain L1 to Rooch L2. However, it doesn't work as there's no support for a Secp256k1RecoverableSignature in crypto.rs.
To further support ECDSA validator validation, Ethereum signature to Rooch signature, and vice versa, it needs to be added and supported Secp256k1RecoverableSignature from fastcrypto repo, since it's not yet implemented in BuiltinScheme and RoochSignature.

@jolestar Please add this issue to Rooch roadmap. We need a solution to automatically adding "closed issue as completed" to the roadmap.


Reddit jokes

196k Reddit Jokes dataset
Dataset link- https://raw.githubusercontent.com/taivop/joke-dataset/master/reddit_jokes.json

@lhoestq would you please rerun the test,

I re-started the test.
@lhoestq let's hold off on merging for now though, having a conversation on Slack about some of the offensive content in the dataset and how/whether we want to present it.


Escape Sequences in Strings

Challenge Escape Sequences in Strings has an issue.
User Agent is: Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.143 Safari/537.36.
Please describe how to reproduce this issue, and include links to screenshots if possible.
My code:

var myStr="\n \\ \\ \r " ; // Change this line

The issue tracker is for reporting bugs only. Please use the help chat room or try looking through the forum for help with a specific challenge.
Happy Coding.


New testset header not working on chrome

The new testset recently implemented is working only on firefox.
This is the same testset and this is how it looks on chrome:

This is how it looks on firefox/

@witehound @MohammedMaaz @mmabrouk

@mmabrouk Let's remember to always test on both Firefox and Chrome :)
Let's have this as high priority. @bekossy Can you take this over

@bekossy, @aakrem fixed this let me know if you want me to open a pr as it’s high priority!

Yes @witehound, please go ahead and open a pull request.


The exit button does not have any function.

Describe the bug 🐛
The exit button in board component does not have any function. It does not navigate to any other route.
To Reproduce
Steps to reproduce the behavior:

Go to '/board'
Scroll down to 'Exit'
See error

Observed behavior
The Exit button does not navigate to any other route.
Expected behavior
The Exit button when clicked should navigate to the previous route that is folder/id
Screenshots

Desktop (please complete the following information; delete this section if the issue does not arise on desktop):

OS: [e.g. iOS]
Browser [e.g. chrome, safari]
Version [e.g. 22]

Smartphone (please complete the following information; delete this section if the issue does not arise on smartphones):

Device: [e.g. iPhone6]
OS: [e.g. iOS8.1]
Browser [e.g. stock browser, safari]
Version [e.g. 22]

Additional context
Add any other context about the problem here.

I have solved this issue as now after clicking the exit button, it navigates to the previous route , i.e., /folder/id


Improve qualifier documentation

Add a section on qualifier combinations. Also added graph illustrations.

@wilzbach awesome, thanks. Somewhat miraculously a simple solution exists for the LaTeX version as well - see the new commit.

Somewhat miraculously a simple solution exists for the LaTeX version as well - see the new commit.

Not so sure - you would need to define LATEX in dlang.org.ddoc, due to many people using undefined macros, we check for them since a couple of weeks:
/dev/shm/dtest/work/repo/dlang.org/web/spec/const3.html:425:<!--UNDEFINED MACRO: "LATEX"-->
/dev/shm/dtest/work/repo/dlang.org/web/spec/const3.html:441:<!--UNDEFINED MACRO: "LATEX"-->

@wilzbach oh, of course. I thought it was defined already - apparently it wasn't used anywhere yet.

Could you please add the GraphViz source files for the graphs? Just place them in the images directory alongside the renders.

@CyberShadow cool, done


Revert real_mobile on 0.1.x

The commit https://github.com/karma-runner/karma-browserstack-launcher/commit/85c4764199c9c78c7a725385d055b9d1293f6272 shouldn't have landed in a patch release as it's a breaking change.
The true default is IMO wrong - BrowserStack doesn't have all devices available as real mobiles so just updating the karma-browserstack-launcher makes a lot of browsers fail.
The default should be changed to false.

@mzgol, care to review?

@donato reviewed

Published as a patch release, thanks!


[Suggestion] Liquid XP -> Mending

I would like to be able to use my massive supply of Liquid XP to repair items that have the Mending enchantment.  Compatibility with Tinkers' Construct Mending Moss would also be appreciated.  I can think of three possible implementations.

Glass bottles can be filled with liquid XP to make Bottle o' Enchanting bottles.  The Minecraft wiki states that a Bottle o' Enchanting will drop somewhere between 3 and 11 experience, and it would be quite reasonable for this to be a slightly inefficient method of mending tools.  I would suggest each bottle costs around 10 to 15 experience points worth of liquid xp (Ideally 11+ to ensure no cases where you might get more xp than you started out with).

Experience Obelisk gets 'drop as XP orbs' button(s).

Liquid XP can be used for mending directly in certain Ender IO machines.  This could be added to the Experience Obelisk; or a new machine could be added, like a Liquid XP Infuser.  The Killer Joe (or an upgraded version of it) could also accept liquid XP for mending and remove all experience orb attracting as an alternative fix for #3823.

The Tank could treat items with mending like fluid containers that take liquid XP.

When this gets implemented it would be awesome if it also worked with TConstruct tools with Mending Moss!

For reference, my go-to assumption in 1.10.2 was to try to find something like OpenBlocks' XP Shower. Not saying that EIO should necessarily take this route (I assume it's quite lag prone), but that is an established user experience to consider.
Regardless of implementation, would greatly appreciate this.

Our version of the XP shower was in Ender IO Addons, which hasn't been ported to 1.10 yet because I was drafted to do the Ender IO porting. And now that I finally could find some time, 1.11 has arrived...

This works with everything that has the vanilla enchantment, I don#t know how that moss is implemented.
More uses for liquid XP may come later.

I guess you would need to use the TConstruct API for the moss.


[bug] Global shortcuts are never triggered on Linux

Describe the bug
Similar to Tao issue #307, global shortcuts never trigger on the Linux platform
Perhaps a requirement that isn't specified somewhere?
Tested on 2 fresh installs of Ubuntu 20.04 running on the vmware, and VirtualBox platforms, and also tested through WSL2
Reproduction
let mut gsm = app_handle.global_shortcut_manager();
gsm.register("CmdOrCtrl+1", || {
    println!("here");
});

Expected behavior
Registered handler should be triggered, printing to the console
Platform and versions
cargo tauri info

Operating System - Ubuntu, version 20.04 X64

Node.js environment
  Node.js - 17.8.0
  @tauri-apps/cli - 1.0.0-rc.6 (outdated, latest: 1.0.0-rc.7)
  @tauri-apps/api - 1.0.0-rc.1 (outdated, latest: 1.0.0-rc.2)

Global packages
  npm - 8.5.5
  pnpm - Not installed
  yarn - Not installed

Rust environment
  rustup - 1.24.3
  rustc - 1.59.0
  cargo - 1.59.0
  toolchain - stable-x86_64-unknown-linux-gnu

App directory structure
/node_modules
/src-tauri
/src
/.git
/public
/.vscode

App
  tauri - 1.0.0-rc.4
  tauri-build - 1.0.0-rc.4
  tao - 0.6.4
  wry - 0.13.3
  build-type - bundle
  CSP - unset
  distDir - ../build
  devPath - http://localhost:3000/
  framework - React

Stack trace
No response
Additional context
No response

Was not aware it had not made it to the RC builds
Thanks - any timeline on it?

The progress has halted for now but we sure need to fix it before the tauri v1 stable release.

Thanks


fix: added mapping for python client-MI version compatibility

https://modelonproducts.atlassian.net/browse/FLOW-24

I like this very much. It would be appreciated if this was reflected here as well: https://modelon-impact-client.readthedocs.io/en/latest/install.html

I like this very much. It would be appreciated if this was reflected here as well: https://modelon-impact-client.readthedocs.io/en/latest/install.html

@maraberg  This will be published when merged. Or do you mean adding a pointer to this text from the first section?

Hi @maraberg , @iakovn , Thanks for suggestion! I have now made the suggested fixes. Could you please take another look?

Looks good!


Does Zenject work with Unity 5.4?

Is Zenject tested with Unity 5.4 yet and will it work?
Thanks
Patrick

I just tested it with 5.4 and it worked fine after fixing some minor compiler errors.  I'm not aware of any incompatibilities, feel free to report any though if you find some.  You can try it with version 3.11

Great. Thanks for fixing these so quickly!


Hook pre-upgrade catalog/templates/pre-migration-job.yaml failed

Bug Report
What happened:
During upgrade of service-catalog with helm, the following error was thrown:
helm upgrade --install service-catalog svc-cat/catalog --wait --namespace catalog --version v0.3.0 --set controllerManager.resources.requests.memory=100Mi --set controllerManager.resources.limits.memory=100Mi
Error: UPGRADE FAILED: pre-upgrade hooks failed: warning: Hook pre-upgrade catalog/templates/pre-migration-job.yaml failed: object is being deleted: persistentvolumeclaims "service-catalog-catalog-migration-storage" already exists

This happens not every time. Most of the time the installation succeeds.
What you expected to happen:
Upgrade is always successful
How to reproduce it (as minimally and precisely as possible):
Run the command helm upgrade --install .... many times on different days.
Anything else we need to know?:
Environment:

Kubernetes version (use kubectl version):

Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.6", GitCommit:"dff82dc0de47299ab66c83c626e08b245ab19037", GitTreeState:"clean", BuildDate:"2020-07-16T00:04:31Z", GoVersion:"go1.14.4", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.12", GitCommit:"17c50ce2d686f4346924935063e3a431360e0db7", GitTreeState:"clean", BuildDate:"2020-06-26T03:33:27Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}

service-catalog version: 0.3.0
Cloud provider or hardware configuration: AWS
Do you have api aggregation enabled?

Do you see the configmap in kube-system? No
Does it have all the necessary fields? yes

kubectl get cm -n kube-system extension-apiserver-authentication -o yaml and look for requestheader-XXX fields

Install tools:

Did you use helm? What were the helm arguments? Did you --set any extra values? See above

Are you trying to use ALPHA features? Did you enable them? No

same here. Any news about it?

/remove-lifecycle stale

we are seeing the same behaviour every upgrade as the resource already exists. this seems to be a very common problem when using Kubernetes Deployments, Helm & dynamically provisioned volumes from StorageClasses as the only reference to the PersistentVolume is inside the PersistentVolumeClaim.


Required properties snippet with parameters (for resources and modules)

Is your feature request related to a problem? Please describe.
I really like the new required-properties snippet, it's really helpful and saves a lot of time. In my case I end up adding parameters the property values most of the time. It would be awesome if you could add another snippet function.
Describe the solution you'd like
I would like to see an required-properties-with-params in the VS Code extension.
Today if i pick declare a storage account and pick the required-properties snippet I get the following in my template:
resource storage 'Microsoft.Storage/storageAccounts@2021-02-01' = {
  name:
  location:
  sku: {
    name:
  }
  kind:

}

I would like to see a required-properties-with-params snippet that would add this to my template:
param name string
param location string
param sku string
param kind string

resource storage 'Microsoft.Storage/storageAccounts@2021-02-01' = {
  name: name
  location: location
  sku: {
    name: sku
  }
  kind: kind

}

Rather than having duplicate snippets, the way I'd like to see this solved is with a "Extract as parameter" code action which will convert (cursor marked |):
resource storage 'Microsoft.Storage/storageAccounts@2021-02-01' = {
  name: na|me
  ...
}

To:
param name string

resource storage 'Microsoft.Storage/storageAccounts@2021-02-01' = {
  name: name
  ...
}

That way we just have the one snippet, but I can quickly tab through the stops and decide quickly what to parameterize myself - plus it would be useful in scenarios outside of snippets. I think this was suggested under #934.


 Version 110 is 2 years old,
We may be able to override the ENV VAR for the Go version for a quick win. Ideally updating Goreleaser and the yml isn't overly difficult.
https://hub.docker.com/layers/goreleaser/goreleaser/v0.110/images/sha256-ae12ac8d7b360a180dd42d3aa79b58610c15b86f56bf20c73966fa76d73e2344?context=explore

@zchee is there any reason we need to use a Docker image for this action?
https://github.com/cuelang/cue/blob/760aa115cf9b62ed51c2d2ee4153d8654c0f231e/.github/workflows/release.yml#L22
I'm wondering whether we don't simply use what is suggested as a default workflow here:
https://github.com/goreleaser/goreleaser-action/tree/e64074c2ea82c84abced0496ae4c8b28cd20469f#workflow
Notes when we come to look at this:

Use fetch-depth: 0 on actions/checkout@v2
Specify a specific version of go-releaser in the goreleaser/goreleaser-action@v2 action

will check and comment. but IIRC when I wrote them, actions have some limitations. therefore I use docker.

@zchee, thanks.

IIRC when I wrote them, actions have some limitations. therefore I use docker.

That was exactly my guess. i.e. back then Docker was the only option.
I suspect we can move to a non-docker version now.

@myitcv
yes, if already no limitations on actions, we can avoid use docker container.
Could you investigate it? I think needs check only current cue used opions.

@zchee please can you take a look at https://cue-review.googlesource.com/c/cue/+/8666?

@myitcv will do.

@zchee we submitted the change on the basis we're looking to cut another release soon so at least this is one way of trying it out!
If however you have any comments please let me know and I'll happily fix things up.

This issue has been migrated to https://github.com/cue-lang/cue/issues/737.
For more details about CUE's migration to a new home, please see https://github.com/cue-lang/cue/issues/1078.


not converting coco to yolo

Hi,
The code used
from pylabel import importer
path_to_annotations = "my_data/modified.json"
path_to_images = "/my_data/myVOC/JPEGImages/"
dataset = importer.ImportCoco(path=path_to_annotations, path_to_images=path_to_images, name="strawberry")
dataset.export.ExportToYoloV5(dataset)

The error msg says

Traceback (most recent call last):
  File "/home/ash/Ash/repo/object_detection/convert_coco2yolo/coco2yolo.py", line 5, in <module>
    dataset.export.ExportToYoloV5(dataset)
  File "/home/ash/anaconda3/envs/condaEnv3.8/lib/python3.8/site-packages/pylabel/exporter.py", line 434, in ExportToYoloV5
    path = PurePath(output_path)
  File "/home/ash/anaconda3/envs/condaEnv3.8/lib/python3.8/pathlib.py", line 651, in __new__
    return cls._from_parts(args)
  File "/home/ash/anaconda3/envs/condaEnv3.8/lib/python3.8/pathlib.py", line 683, in _from_parts
    drv, root, parts = self._parse_args(args)
  File "/home/ash/anaconda3/envs/condaEnv3.8/lib/python3.8/pathlib.py", line 667, in _parse_args
    a = os.fspath(a)
TypeError: expected str, bytes or os.PathLike object, not Dataset

Hello @ashBabu . I notice that you are passing the parameter dataset as a parameter to the ExportToYoloV5 command. It is necessary to pass the dataset object. Please see the documentation at https://pylabel.readthedocs.io/en/latest/pylabel.html#pylabel.exporter.Export.ExportToYoloV5
If you change the last line to this dataset.export.ExportToYoloV5() you should get some output and then you can customize it further.

thanks. It worked. Closing.


WHY existing engines are not extendable? What about Open-closed Principle in SOLID?

Hello
what is the reason that you don't allow to extend
DefaultRule
AbstractRulesEngine
DefaultRulesEngine
We would like to implement feature that will allow to return specific result when
public Facts execute(Facts facts) throws Exception
will be executed.
Because of that, this code is not extendable. We have to copy paste whole code from AbstractRulesEngine, DefaultRulesEngine.

https://github.com/j-easy/easy-rules/pull/62
It had been discussed here , hope useful for u


"pip install plum" creates a "tests" directory in site-packages

Ideally, each package should create only one folder.
Also, "tests" is a very common name, and this is very likely to collide with local folders.

It's due to the autodiscovery of the package and subpackage.. it's an easy fix

Hey @erezsh! Sorry for the radio silence.
I've amended setup.py to exclude tests and all subdirectories. Thanks for catching this. :)


envoy: Create and configure the caches for NPDS and NPHDS

This PR is required to complete https://github.com/cilium/cilium/pull/2875.
And this PR should be rebased and merged only after https://github.com/cilium/cilium/pull/2781 is merged, as the Envoy API packages have changed.
Signed-off-by: Romain Lenglet<EMAIL_ADDRESS>

test-me-please

test-me-please

test-me-please

test-me-please

test-me-please

test-me-please


Add date to front matter in new posts

When making new posts, I'd like to auto-generate date/timestamp to front matter. Thoughts?

This is the very first feature I found myself wanting when first trying jekyll-compose. +1

Bump ?

@parkr any chances for this? :)

LGTM


NullPointerException

Hello,
One of my users got a NullPointerException (reported via Rollbar, I don't have more context on what he was doing with the keyboard, sadly).
He is running a Staraddict 4, a rebranded ZTE Skate, with android 4.4.2
His app is running ionic rc3, and I'm not calling the plugin in my code
java.lang.NullPointerException
at com.ionic.keyboard.IonicKeyboard$2.run (IonicKeyboard.java:75)
at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1112)
at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:587)
at java.lang.Thread.run (Thread.java:841)

What version of the plugin are you using? I believe this was fixed in https://github.com/driftyco/ionic-plugin-keyboard/commit/bd1adc9b532512e07c4b6fd5d29a73915d50f64c

Ah yes, it seems I'm not up to date. Thanks!
Btw, any news on the screen flickering issue on Android? I'm seeing more reports of this bug, has anyone found a workaround?

Yeah the flickering thing is really annoying.  Unfortunately no updates yet, are there any specific versions of Android or devices that you've been seeing it on?

Someone mentioned here #70 it appeared with an update from crosswalk 8 to 9, if I remember correctly the bug started appearing around the same time for me, but I'm not 100% positive.
I'm testing on a Nexus 5 running 5.0.1 and a Galaxy S2 running 4.2, both with crosswalk (I'm at version 12 now), and both have the bug

Ah the Crosswalk bug, that's probably this: https://crosswalk-project.org/jira/browse/XWALK-3547, there are several links to discussions about it in there.

This is it, thanks for the link!


final red-left/blue-right updates

From test on 2/15

red-left is not pulling far enough forward to reach cone stack
blue-right is not backing up enough after grabbing a cone causing it to knock over the stack while turning


Events are not properly listed under their stating date

Hi,
I've been using your plugin and found a weird issue. I wasn't able to track down completely the problem, but I hope I'll guide you through my findings.
I set two events, the first one with a starting day on 11-01-2016 and an ending day on 11-02-2016 and the second on with a starting day on 12-02-2016 and and ending day on 12-02-2016.
The problem is that both events are shown both on their months and each others. Additionally, the first event is still rendered in December even though there is no indication in the html.
The data.js file:

The calendar is shown correctly ...

... but on November 2nd, there are 2 events

also in December 2nd, 2 events

and a non-existent event in December 1st

What I've found is that you set and search curData based on the numeric date. This leads to a problem for non-unique indices, thus setting the same event (or another one) on a wrong date.

I did encounter such problems while updating to the latest version, and if I remember I did fix this problem. Can you make sure that you are using the latest version?

Yes, I do use the latest version (5.0.0) and the problem is still there.

Fixing this will take time, Downgrade back to 4.1.0, Thanks for informing this is a very buggy version.


Clarify these features are not related to Next

There was some discussion online that the warning mentioning Next may give the impression this is a NextJS feature.
What if we leave it just as a blank warning for now? I can see the appeal of linking to something vs. nothing but I do hope to get these fleshed out soon. What do you think?

@sophiebits @lunaleaps yeah, and when other frameworks add support and docs, we can also add links to those docs.

This API can be used in any framework that supports React Server Components:

Next.js Server Components

More coming soon


[Infra] Include GitHub workflow name in each task

Which Delta project/connector is this regarding?

[ ] Spark
[ ] Standalone
[ ] Flink
[ ] Kernel
[X] Other (Infra)

Description
I'd like to do some data science using the GitHub REST API on our PR runtimes.
Unfortunately, using the GitHub CHECKs API (https://docs.github.com/en/rest/checks/runs?apiVersion=2022-11-28#list-check-runs-for-a-git-reference), I only get the name of the individual job. Getting the name of the parent workflow requires an additional API call.
This is because our job names are the default job names right now for most of our GitHub workflows. This doesn't give us any info as to which workflow it is for. For example, today I'd get result test (2.13.13, 0) but I am unable to determine if this is for Delta Spark Latest or Delta Spark Master.
This PR updates the workflow job name to include the workflow name so now we can uniquely identify the jobs.
How was this patch tested?
CI tests.
Does this PR introduce any user-facing changes?
No

It looks like I can see the new names in the CI that's running on this PR? Unfortunately this feels redundant - many have names like Delta Kernel Tests / Delta Kernel - any idea if we could alias somehow to shorten? if not then oh well probably worth the naming redundancy to do your science

@zachschuermann - no way that I know of. If this redundancy escalates to something more serious/concerning, we can revert this PR

The names in the UI are getting very very long and cut off. This aint great. Especially this Delta Spark Local Publishing and Examples Compilation / Delta Spark Local Publishing and Examples Compilation: Scala 2.13.13 (pull_request)

@tdas - ACK, I pushed a new commit with shortened names


Can't spin up session on datum GCP qhub

Seems there is a permissions error when spinning up an instance:
Server requested
2020-05-14 14:08:08+00:00 [Normal] Successfully assigned dev/jupyter-t-2dpotts to gke-datum-gcp-qhub-dev-user-fce1d8cf-2mtk
2020-05-14 14:08:08+00:00 [Normal] Container image "busybox:1.31" already present on machine
2020-05-14 14:08:09+00:00 [Normal] Created container init-nfs
2020-05-14 14:08:09+00:00 [Normal] Started container init-nfs
2020-05-14 14:08:09+00:00 [Normal] Container image "quansight/qhub-jupyterlab:fb3489522cde4b0fe6b2b0d4f24522d0fffcabc8" already present on machine
2020-05-14 14:08:10+00:00 [Normal] Created container notebook
2020-05-14 14:08:10+00:00 [Normal] Started container notebook
2020-05-14 14:08:10+00:00 [Warning] Exec lifecycle hook ([/bin/sh -c echo 'costrouc:x:100000000:20000000:costrouc:/home/jovyan:/bin/bash t-potts:x:100000001:20000000:t-potts:/home/jovyan:/bin/bash dharhas:x:100000002:20000000:dharhas:/home/jovyan:/bin/bash' > /tmp/passwd && echo 'users:x:20000000: admin:x:20000001:' > /tmp/group && ln -sfn /home/shared /home/jovyan/shared]) for Container "notebook" in Pod "jupyter-t-2dpotts_dev(563d6f38-95ec-11ea-aedb-42010a80000b)" failed - error: command '/bin/sh -c echo 'costrouc:x:100000000:20000000:costrouc:/home/jovyan:/bin/bash t-potts:x:100000001:20000000:t-potts:/home/jovyan:/bin/bash dharhas:x:100000002:20000000:dharhas:/home/jovyan:/bin/bash' > /tmp/passwd && echo 'users:x:20000000: admin:x:20000001:' > /tmp/group && ln -sfn /home/shared /home/jovyan/shared' exited with 1: ln: failed to create symbolic link '/home/jovyan/shared': Permission denied , message: "ln: failed to create symbolic link '/home/jovyan/shared': Permission denied\n"
2020-05-14 14:08:10+00:00 [Normal] FailedPostStartHook
2020-05-14 14:08:15+00:00 [Warning] Back-off restarting failed container
Spawn failed: Timeout

Once this times out I can't spin up an instance

Seems to be related to my github username being t-potts. I ended up changing to a different github accout tpottstesting as a work-around
Closing out


cancel grpc service.AddxxxHandler return error

Describe the solution you'd like
Almost go router design do not return error, suggest to AddxxxHandler cancels error return, avoid many error deal, may panic when handler is invalid.
Additional context
Refer Design
net/http HandleFunc
Gin Router
fasthttp recommand third-party routers
I do not know too much about dapr design. If I'm wrong, please just ignore it.

@li1234yun can you elaborate on this? Point maybe to code where this applies?

Code position:
event
topic

Good catch, thanks.

checks for nil handler added

Sorry for my previous expressions, what I want to express is service.AddxxxHandler no return result(now return is error). if any invalid handler may be panic like gin.Router before service start.


Use postgres/notify to invalidate query cache on DDL

This makes it possible to use different EdgeDB servers connected
to the same Postgres cluster.  With this change, when a DDL command
is committed, a NOTIFY command is issued which all EdgeDB
connections listen to.

I'm unsure how to test this though. We should probably have a test for this in our cloud infra project.

We can test this locally by starting a second server with —postgres-dsn of the first one.

I'll let Elvis approve this but it looks pretty clean. Postgres is a great base for this.

@elprans Please review again.


reel_board: running tests/subsys/power/power_mgmt_soc failed

Describe the bug

Test test_pm_dummyinit fail, happens ***** MPU FAULT *****
That can be related to #30192 , but reel_board board and the Microchip board from #30192 issue have different error types and failed tests.

To Reproduce
Steps to reproduce the behavior:

west build -p -b reel_board tests/subsys/power/power_mgmt_soc
west flash
Open terminal and See error

Expected behavior
Test must pass.
Impact
Low pass rate.
Logs and console output
*** Booting Zephyr OS build zephyr-v2.4.0-1755-g0a0404df29ed  ***
Running test suite test_power_mgmt
===================================================================
START - test_pm_dummyinit
I: PM dummy single-thread test started for one cycle
I: About to enter light sleep
PM >
I: PM sleep entry latency E: ***** MPU FAULT *****
E:   Stacking error (context area might be not valid)
E:   Data Access Violation
E:   MMFAR Address: 0x200015c8
E: r0/a1:  0x20000ca0  r1/a2:  0x00000000  r2/a3:  0x04010102
E: r3/a4:  0x00002800 r12/ip:  0x00000000 r14/lr:  0x00000000
E:  xpsr:  0x21000000
E: Faulting instruction address (r15/pc): 0x000001d0
E: >>> ZEPHYR FATAL ERROR 2: Stack overflow on CPU 0
E: Current thread: 0x20000400 (idle 00)
E: Halting system

Environment (please complete the following information):

OS: Linux Fedora 28
Toolchain Zephyr SDK 0.11.4
Commit SHA 0a0404d

According to the platform exclusion in the yaml file, reel board should be skipped for this case. In the latest mainline daily test report, I didn't see this failing on reel board.

based on the comment above, this should not be run on reel_board. Also not sure how this is related to memory protection. It is just a stack overflow.