the rule of barcode correction

Hello,
I am using TRUST4 on my spatial-RNAseq data, which has 20bp barcode. However, I am not clear, how TRUST4 handels the single-base error within these 20bp-barcode? In other words, when two barcodes standing for a same spatial spot have a single-base difference, will trust4 looks them as a same barcode?
Thank you so much!
Best wishes,
Yuyu

The correction is to match with a barcode whitelist. It corrects one mismatch, and if there is a tie, TRUST4 will use the whitelist barcode frequency observed in the dataset from those perfect barcodes to break ties.
If there is no whitelist given, TRUST4 will not conduct error correction.

Thank you so much for your answer.
When I give the barcodeTranslate file, where two different barcodes stand for a same spot, will trust4 also look them as a same spot?
Thanks again.
Yuyu

Do you mean that in the barcode translate table, two barcode say A, B will be translated into the same barcode C? In that case, TRUST4 will just output barcode C.

Thank you so much for your answer.
I think that I have make you confused, sorry.
For example, folloing is my barcode translate table:
A   ATCGATCGATCG      <-
A   TTCGTTCGATCG      <-
B   ATCGTTTTCCCC
......
In other words, two barcodes (marked by arrow) with maybe two-base difference stand for same spot A. Will TRUST4 output A also?
Thank you so much!
Yuyu

Yes, both barcodes will be translated into A, and A will be in the final output.

Just want to confirm. Do the two barcodes ATCGATCGATCG and TTCGTTCGATCG still correspond to two cells in your data, and it just mean they are from the same spatial spot A?

Thank you so much for your answer.
According to my data, two barcodes shoube be the same spatial spot A, which also can be looked as a cell.
Therefore, I can just set the barcode translate table instead of barcode whitelist, so that similar barcodes will be translated as a same spot and the correction step can be skipped?
Thanks again. and I wish you a Happy New Year in advance!
Yuyu

Yes, the correction step will be skipped and the output will report the representative TCR or BCR for spot A.
Just curious, why do you have two barcodes for one spot?

Thanks a lot.
Cause my original single spots are smaller, and I wanna merge several samll spots to a bigger one (similar to a cell), so that it runs faster.
Yuyu

Hello,
I have attempted to utilize TRUST4 for analyzing my spatial-mRNA data. However, I am concerned that I might be misunderstanding the outputs.
1.*_report.tsv: I have noticed that the first column in this file records the number of barcodes for the CDR3, instead of the read count. However, I have observed that the cid-column only contains a "cid/barcode". Could you please clarify what this "cid" represents?

An alternative mumber following the Translated barcode: In my TranslatedBarcode list, I have translated several cid sequences into a new "cid" label, which follows a coordinate format (e.g., x_y). However, I have come across translated cids that have an alternative number appended to them, like x_y_num. I am unsure of the significance of this alternative number.

3.The number of reads used for barcode assembly: Is there an output file where I can find the read count used for assembling a CDR3 for each barcode?
At present, these are the three points where I am seeking clarification. I greatly appreciate your assistance in addressing these questions.
Best regards,
Yuyu

Just now, I have found a warning during annotation:
Use of uninitialized value $tmpGermline[43] in string ne at ./TRUST4/trust-airr.pl line 368,  line 423598. (There are many rows of the same warning.)
I am not sure, where does this error cone from?
By the way, I really want to ask you if there is any way to assemble like 500,000 barcodes efficiently, and each of them has about 10,000 reads?
Thank you so much for your patience.
Best wishes,
Yuyu

1,2. cid is the contig id/name. It has the barcode information if it is single-cell data. Since there could be multiple contigs assembled for a cell, the last number in the ID provides the information to distinguish the contigs from the same cell. They are useful to link the CDR3s to the underlying sequences in the annot.fa file.
3. I think you can look at the barcode_report or barcode_airr file, which list the clonotype information for each barcode, and has the number of reads supporting the CDR3s.
4. This could relate to some file's format errors? Could you please download the newest version of TRUST4 and give it a try?
5.  Are these VDJ-targed amplified 10K reads?

Thank you very much for your detailed answer.
According to the fifth point, yes, they are all VDJ-targed amplified reads. However, I have not performed an alignment on them. Do you recommend that these reads should be performed alignment firstly? And is STAR a suitable/useable tool to do it? Or which alignment tool do you suggest to use for VDJ-data?
Thank you so much again!
Best wishes,
Yuyu

I don't think you need to do alignment first. Since these are enriched VDJ reads, you may try the --repseq option to speed up the process.

Thank you so much for your help. I will try them again with this option.
Best wishes,
Yuyu

Hello,
I recently downloaded the latest version of TRUST4 via conda. However, I got an issue when attempting to assemble with the parameter --readFormat bc:0:15. The error message was: Unknown parameter --readFormat.
I was wondering if you could kindly assist me in troubleshooting this problem?
Thank you so much!
Best wishes,
Yuyu

Could you please show me your full command?

YES. Following is my full command:
run-trust4
-u ./test_assemble_1_read_2.fq.gz
-t 8
--barcode ./test_assemble_1_read_1.fq.gz
--readFormat bc:0:15
--barcodeTranslate ./test_assemble_1_barcodeTranslate.tsv
-f ./TRUST4/mouse/GRCm38_bcrtcr.fa
--ref ./TRUST4/mouse/mouse_IMGT+C.fa
--repseq
-o ./test_assemble_1_res
*The format of test_assemble_1_barcodeTranslate.tsv is: "x_y" \t "15bp_barcode"
*The newest version of TRUST4 was downloaded with command "conda install -c bioconda trust4"
*When I just downloaded .zip and unzip it, the command worked good, just with the warning as I have mentioned.
Moreover, I want to ask about the output again.

As you have introduced that the count reported in the report.tsv (the 1st column) is the number of barcode/cell containing the corresponding CDR3. However, why there is only one CID listed in the 'cid' column? Or is the cid with the highest score?
Is the 'consensus_count' of the barcode_airr.tsv the readscount used to assemble the corresponidng CDR3 of a cell/barcode?

Thank you so much again!
Best wishes,
Yuyu

Could you please run "./run-trust4" without any parameters? It will show the version number. Some times you need to specify the version on conda to get the newest version.
For other questions:

That cid is the representative contig. It should be the contig with the most reads covering the corresponding CDR3 region.
Yes.

Here is the feedback with "run-trust4"

I am sorry, I have checked the version of trust4 just now, which is <IP_ADDRESS>. I will try again to install the latest one.
Thank you so much again for your patience!
Best wishes,
Yuyu

This is indeed an older version of 1.0.5. You can try something like conda install -c bioconda trust4=1.0.13 to install the latest version. Otherwise, you can also download the github version. TRUST4 does not have many dependencies, it is straightforward to compile from source code.

Okay! Thank you so much for your tips! :)
Yuyu

Hello, I apologize for any confusion caused, but I have come across another question regarding the results of assembling based on multi-barcodes assemble.
I have attempted to perform the assembly of sequences from a specific barcode. However, I have noticed that the final read count used for assembling the same V-gene, J-gene, and CDR3 (colontype), differs from the corresponding barcode's "consensus_count" recorded in the barcode_airr.tsv though the final assembled colontype is same.
Following is my result:
Single-Barcode assemble:  (Top1 assembly with 2009 reads)

Multiple-Barcode assemble:  (The corresponding consensus_count is 827 reads)

Can you please assist me in explaining these differences? I would like to know which read count would be more suitable for determining the reliability of the final assembly.
Thank you so much again for your assistance.
Best wishes,
Yuyu

Does the run of "single-barcode assemble" mean that you extract all the reads from that barcode and run it in the bulk RNA-seq mode? The assembly procedure is slightly different in single-cell mode and bulk mode, such as contig overlap criteria. DIfferent underlying contigs will result in different abundance estimations.

The Single-Barcode assembly was performed in bulk mode.
Could you please explain how the assembly procedure differs between single-cell mode and bulk mode (maybe with a simple example)? I am particularly using the read count used for assembly to determine the reliability of the final assembly. For instance, if there are 100 reads associated with a specific barcode, but only 1 read is used to determine the CDR3, I would consider this resulting CDR3 as potentially inaccurate due to the possibility of bias.
Thank you so much.
Best wishes,
Yuyu

There could be many subtle differences. One is read soring will be different. Using the whole data set will have a different k-mer abundance distribution, so the read sorting and the downstream assembly will be different. Furthermore, in single-cell mode, we use a more lenient threshold when determining whether a read overlaps with a contig. In bulk, the threshold is at least 21bp (also depending on the read length), in single-cell mode, it can be as low as 13.

Thank you so much for your explanation.
Then, I am facing a slight challenge in deciding which mode is suitable for the spatial sequencing data. Should I still use the single-cell mode or assemble each spot as a bulk (but I have more than 5k spots)? I think the final top 1 assembly for each barcode should be same, while the number of reads used to assemble is calculated differently.
However, I am confused as to why I am observing a lower number of reads used to assemble the same clonotype for the specific barcode in single-cell mode than in the bulk mode, even when the threshold (used to determine whether a read overlaps with a contig) is set low?
Best wishes,
Yuyu

Does one spot contain a single cell or multiple cells in your data? Ideally, it shall be in single-cell mode.
I think it is mainly affected by the read order, which may put some other contigs first to be assembled, which will drag away some alignments. Do you see this abundance discrepancy in many other spots as well?

Yes, each spot should be considered as a single cell.
I have recently made a comparison for a particular barcode. However, I plan to examine more barcodes to gain a deeper understanding of their differences.
Once again, thank you sincerely for providing such a detailed explanation.
Best regards,
Yuyu

Dear Prof. Li,
Thanks again for your detailed explantion.
II still have several confusion about the results generated by TRUST4 in single-cell mode, when I have performed TRUST4-assembly on my spatial RNA-seq data and I tried defining different size of spots as a cell (barcode).

I have compared the result of "assemble_reads.fq" with the alignment result generated by bowtie2/STAR. The assembled reads (containing both CDR3 reads, and reads that may full contained in the V gene region or C gene region) are much fewer than the aligned reads to the VDJC genes found by STAR and bowtie2. As I have thought that, the assembled reads extracted by TRUST4 should be more than that found by general alignment tools, because TRUST4 can "save" more reads containing CDR3 motif, even though they were thought as "unmapped" by the general alignment tools.
I have compared the number of the assembled reads extracted respectively by 2 times

Dear Professor Li,
Thank you once again for your detailed explanation.
Firstly, I would like to explain an alternative step I have done for my spatial RNAseq dat. Before I performed TRUST4 assembly on my data in single-cell mode,  I have separated this large data set (.fq) in 5 parts, according to the barcodes, to speed up the assemble step. Then I would perform the TRUST4-assemble in single-cell mode to these 5 split data set respectively. Finally, I would merge the assemble results together.
However, I have several questions regarding the results obtained from TRUST4 in single-cell mode.

I noticed that the number of assembled reads (merged by 5 assembled_reads.fa files) was significantly lower compared to the alignment results obtained from bowtie2/STAR. These assembled reads, according to your explaination in the other issue, should include both CDR3 reads and reads that may be fully contained within the V gene or C gene regions. I initially expected that the number of assembled reads extracted by TRUST4 would be higher than those identified by general alignment tools, since TRUST4 can potentially "save" more reads containing the CDR3 motif that are typically considered as "unmapped" by general alignment tools.

I performed two separate TRUST4 assemblies on the same data set but using different spot definitions (different sizes of spot). Upon comparing the total number of assembled reads between these two assemblies, I noticed slight variations. This leads me to wonder whether the step of extracting candidate reads also takes the barcode into consideration.

Thank you so much for your assistance.
Best wishes,
Yuyu

How do you count the alignment results from bowtie2/STAR? Since TRUST4 only assembled the first a few hundred bases of the C genes, many reads in the later part of the C gene will not be included in the assembly. Therefore, depending on your kit, such as has some 3' bias, it's possible that the number of reads from V, CDR3, J region is much smaller than the reads from C genes.
I'm not sure I understand the spot definition here. The candidate reads extraction step does not take the barcode into account.

Dear Prof. Li,
Thank you so much for your response.

I have just merged 5 assembled_reads.fa files and counted the reads. Next, I will verify if the majority of my reads are mapped to the C region.

Based on the spot definition, for a large spot, I defined more barcode sequences as a cell-id in the barcodeTranslate.tsv, whereas for smaller spots, fewer barcode sequences were assigned as a cell-id. Due to this difference, there is a slight bias in the number of assembled reads (e.g. 4,148,542 and 4,150,315). Although this bias is minimal, I would like to ensure that I have correctly processed TRUST4 previously.

Thank you once again for your patience and assistance.
Best regards,
Yuyu

Thanks the quick answer and tips.
Just now, I have checked them. There is no "missing_barcode" in all toassemble_bc.fa files. I think it is reasonable, cause I have defined all given barcode sequences in the barcodeTranslate.tsv. :)

Dear Prof. Li,
I am sorry that I have to disturb you again about the alignment step. I noticed that some of the reads were identified as VDJC genes by bowtie2 but not by TRUST4. Upon further investigation, I found that most of these reads are not in the C-regions, but rather in the V-regions. And the mapping scores were also relatively high.I have a few examples of these reads and I was wondering if you could help me understand them better. The species I'm working with is human, and I used the hg38_bcrtcr.fa reference file provided in the TRUST4 package.

read1 (mapped to TRAV9-2 recognized by bowtie2 but TRUST4 no)
CTTAGTATCTCTGATACCCTTACTGCTTGGAAGAACCCGTGGAAATTCAGTGACCCAGATGGAAGGGCCAGTGACTCTCTCAGAAGAGGC
TRAV9-2
CACTGTGATTTCTTCATGTTAAGGATCAAGACCATTATTTGGGTAACACACTAAAGATGAACTATTCTCCAGGCTTAGTATCTCTGATACTCTTACTGCTTGGAAGAACCCGTGGAGATTCAGTGACCCAGATGGAAGGGCCAGTGACTCTCTCAGAAGAGGCCTTCCTGACTATAAACTGCACGTACACAGCCACAGGATACCCTTCCCTTTTCTGGTATGTCCAATATCCTGGAGAAGGTCTACAGCTCCTCCTGAAAGCCACGAAGGCTGATGACAAGGGAAGCAACAAAGGTTTTGAAGCCACATACCGTAAAGAAACCACTTCTTTCCACTTGGAGAAAGGCTCAGTTCAAGTGTCAGACTCAGCGGTGTACTTCTGTGCTCTGAGTGA

read2 (mapped to TRAV13-1 recognized by bowtie2 but TRUST4 no)
GGAGGGAGACAGCGCTGTTATCAAGTGTACTTATTCAGACAGTGCCTCAAACTACTTCCCTTGGTATAAGCAAGAACTTGGAAAAGGACC
gene
GATCTTAATTGGGAAGAACAAGGATGACATCCATTCGAGCTGTATTTATATTCCTGTGGCTGCAGCTGGACTTGGTGAATGGAGAGAATGTGGAGCAGCATCCTTCAACCCTGAGTGTCCAGGAGGGAGACAGCGCTGTTATCAAGTGTACTTATTCAGACAGTGCCTCAAACTACTTCCCTTGGTATAAGCAAGAACTTGGAAAAGGACCTCAGCTTATTATAGACATTCGTTCAAATGTGGGCGAAAAGAAAGACCAACGAATTGCTGTTACATTGAACAAGACAGCCAAACATTTCTCCCTGCACATCACAGAGACCCAACCTGAAGACTCGGCTGTCTACTTCTGTGCAGCAAGTA

I have observed that Bowtie2 identifies mapped reads twice as often as TRUST4 does. This discrepancy is concerning to me, and I would appreciate your input on this matter. Thank you!
Best wishes,
Yuyu

Thank you for scrutinizing this step. If you directly put the two reads as input to TRUST4, they will be recognized. How did you search the reads? Is your data paired-end? TRUST4 will merge read pairs, so if you are searching the read sequence, you may not be able to find it directly. Another possibility is the read quality. There are some read quality trimming inside of TRUST4. This will make either the read too short to be used in the assembly stage, or affect your read search (if you are searching using read sequence).

Dear Prof. Li,
thank you so much for the answer.

How did you search the reads?
I performed alignment using bowtie2 and the complete TRUST4 pipeline (alignment + assembly + annotation) on the same .fq file. Then, I summarized the mapped reads (fqid) identified by bowtie2 and the candidate reads (fqid) identified by TRUST4 (in the assembled_reads.fa). Finally, I found the reads identified as VDJC genes by bowtie2 but not by TRUST4 in the difference set between these two sets of fqid.

Is your data paired-end?
No, my data is single-end RNAseq, where read1 only contains the barcode sequence.

Another possibility is the read quality.
I have not examined the read quality yet. Thank you for the suggestion. I will consider the read quality in the next step. By the way, could you please explain more about quality trimming, such as the threshold for trimming reads based on quality? Then, I can check , whether these reads do trimmed in the TRUST4.

Thanks again for your assistance and patience.
Best wishes,
Yuyu

I just checked the running command you posted before, I think the command should be:
run-trust4
-u ./test_assemble_1_read_2.fq.gz
-t 8
--barcode ./test_assemble_1_read_1.fq.gz
--readFormat bc:0:15,r1:16:-1
--barcodeTranslate ./test_assemble_1_barcodeTranslate.tsv
-f ./TRUST4/mouse/GRCm38_bcrtcr.fa
--ref ./TRUST4/mouse/mouse_IMGT+C.fa
--repseq
-o ./test_assemble_1_res

You shall also add r1:16:-1 in readFormat, otherwise the read itself will include the barcode sequence. Though the command before is for mouse data, I guess you used the same command this time?

Oh, yes, I have always used thus commond.
To make it clear: my data looks like: read1 15bp (barcode), read2 90bp (only TCR)
Therefore, although my read2 does not include barcode sequence, I have to also add r1:16:-1 to make clear it?
Thank you so much! This is definietly important to me!
Yuyu

I see. I thought both of them were from read1. You don't need the r1:16:-1 then.
I just remembered you used the "repseq" option. There might be a bug in processing TCR region. Let me look into this issue. Thank you for providing the example!

Thank you for your continued support and feedback too!  I will also follow the read quality! :)
I would like to provide my example fq file (1000 reads) to you to test the processes.
(ps. What I have done to the reads are trimming the adapters, deduplicating completely same reads in a spot)
test_fq.txt
Best wishes,
Yuyu

I think I've found the issue and pushed an update to the main branch. Could you pull the github repo, recompile trust4 and give it a try?

Thank you once again for your support.
No problem. I'll let you know as soon as possible if it gets better.

Dear Prof. Li,
By the way, is it possible that I can only run the first step, namely reads extraction?
Thanks

If you have the log/output on screen from the run-trust4, you can find the command for each step. You can get the command of running "fastq-extractor" there. The updated code does not affect the read extraction stage though.

Dear Dr. Li,
I have tested the updated code. Initially, there were approximately 4M assembled reads, but now there are around 9M reads, slightly exceeding the recognition of bowtie2. Based on these results, I believe the outcome is reasonable.
Thanks a lot for your support throughout this process.
Thank you once again!
Best regards,
Yuyu

How's the assembly speed after relaxing the filtering? I recalled that I removed those reads because in TCR assembly we only need to know their V, J assignment and there was no need to infer the full-length sequence.
The conda version requires the releasing a new version of TRUST4. I want to make sure there are no more other urgent issues before drafting a new release.

Dear Prof. Li,
I have reviewed the time cost and it seems acceptable. Originally, it took about 22 minutes to complete the entire process for approximately 9M reads, while the updated version now takes about 30 minutes. I hope these data can help you.
For my project, I aim to assemble the full length of VDJ for TCR. Therefore, it would greatly benefit me if TRUST4 could retain all essential reads mapped to VDJ-regions. And it would be even greater if TRUST4 could provide users with an overview of their inpuit raw data, specifically indicating the number of reads mapped to VDJ-regions or obtaining CDR3-motifs. This feature would greatly support us in assessing the reliability of the final assembly result.
Thank you once again for your support.
Best wishes,
Yuyu

Thank you for the testing. It's not bad and makes sense to have the full-length TCR assembly.

t would be even greater if TRUST4 could provide users with an overview of their inpuit raw data, specifically indicating the number of reads mapped to VDJ-regions or obtaining CDR3-motifs.

What's the difference between this and the abundance of the CDR3?

Dear Prof. Li,
Although I have not looked into the abundance of the CDR3, there were more spots, in which CDR3 were successfully assembled.
This week, I will continue looking into the abundance of the CDR3. Then, I will keep updating my information to you.
Best wishes,
Yuyu

Dear Prof. Li,
I have double checked the reads used to assemble again with the newest TRUST4 (the new update to the main branch). However, I have noticed that there are still certain reads, potentially mapping to TCR genes (TRAV and TRAJ), which were not included in the assembly. I can provide you with these reads in fq format, along with the mapping results obtained from STAR in sam format.
By the way,Additionally, I would like to mention that these reads were sequenced from human samples and only for TRA enrichment. Before to assembly, I performed several quality control steps, including the use of fastp to filter out low-quality reads (Q30>80%) and cutadapt to trim adapter sequences. As a result, the lengths of these reads vary.
Regarding the reasons why these reads were not used, I have a few hypotheses:

Could it be due to a discrepancy between the threshold of quality filtering I applied and that used by TRUST4?
Were the reads excluded because they were too short in length (most of them were shorter than 50bp)?
Is it possible that the mapping results differ from those provided by TRUST4? I noticed that a majority of these reads were mapped to the reference with Skip (S or N) flags.

I kindly request your assistance in investigating these matters further.
star_mapped2tra_read_2.fq.gz
mapped2tra.sam.zip
Best wishes,
Yuyu

Sorry for the delayed reply. You are right that the majority of the alignments have long introns "xxxN" or soft clips in the CIGAR field, so I think these reads will be aligned poorly to the TRA genes as a whole read and will be filtered by TRUST4. For the remaining reads, I checked some of them manually, and it seems they overlap a lot with the UTR regions, which are not part of the reference sequences. These reads will be filtered during the assembly stage if there is no valid contig for them to anchor on, i.e. the V gene is lowly expressed/used.
The read length and the quality filtering should be fine.

Thank you sincerely for your answer. Then I think TRUST4 works truly fine. I am grateful once again for this useful tool and your support.

Dear Prof. Li,
I am happy to find the newest release (v1.1.0) of TRUST4. I was excited to give it a try. However, I have encountered an issue during the installation process every time I attempt to install it directly using the zip file. I am consistently facing an error related to the missing 'zlib.h' file, although I have already installed zlib using conda. Last time, when I attempted to install TRUST4 in Linux using the zip file, I also encountered the same issue. Cause I have found that only samtools depends on zlib, I have commented out the steps related to bamextractor involving samtools (I do not need this step also) in the Makefile so that the installation could be done. However, I am unable to apply the same solution this time.
Therefore, I would like to kindly request your assistance. Could you please provide any insights or guidance on how to resolve this 'zlib.h' error during installation? Cause I have no permission to install zlib in the hpc direcly.
Additionally, I wanted to inquire about the possibility of updating TRUST4 on bioconda, as I can install TRUST4 successfully through conda.
Thank you for your attention to this matter, and I look forward to your guidance.
Best wishes,
Yuyu
The error:

What I have done last time, I have successfully installed:


Permit current Vagrant version.

The playbook currently is known to work on versions of vagrant up to 2.1.2 (the current). But it involves manually editing the Vagrantfile to spin it up unless you have an old version of Vagrant.
I've done this in a way that we don't specify a "max" allowed version, because I don't think it's necessary until something is known to break. This way people who are just starting with vagrant don't have to re-install an older version.

Whoa, just wound up doing this myself recently.  @rosiel++


properly extract 2.0 ranges. comment out scrolling for now.

closes #94, #933
TODO: fix 2.0 test and scrolling

Coverage increased (+0.2%) to 47.293% when pulling 3040d2bca404ae06954380580f32dcf94cd5000b on 94-20-ranges into 5f898b8d12fb9da035a7c4e2d07bb9e020390b83 on release2.1.


Fix encoding/accent raising  ActsAsTaggableOn::DuplicateTagError

Creating a tag named "spécifique", then a tag named "specifique" would raise ActsAsTaggableOn::DuplicateTagError
SQL server being correctly configured recognizes that "spécifique" == "specifique", but acts-as-taggable-on does not.

I implemented 2 things in this PR, sorry for the confusion but I was obliged to fix that spec bug in order to correctly deliver my PR. Here are the details:
1) fix rspec bug
Several specs was setting ActsAsTaggableOn.strict_case_match = true without reinitializing it after, which may input invalid settings for the other specs, and all depending on the order specs are run. I cleaned that by replacing all instances of
it 'should do something' do
  ActsAsTaggableOn.strict_case_match = true
  ...
end

by
it 'should do something', strict_case_match: true do
  ...
end

The strict_case_match flag correctly sets ActsAsTaggableOn.strict_case_match = true before the examples and reset it to false after the examples.
2) Use I18n transliterate for unicode comparison
The main reason of this PR: In order to correctly compare strings such as "spécifique" and "specifique" I added I18n.transliterate to Tag#comparable_name. This also has the advantage to support I18n, so users of acts-as-taggable-on can use I18n to precisely define unicode comparison according to their language.
Example:
In english "Jürgen" would be compared to ASCII "Jurgen"
In german "Jürgen" would be compared to ASCII "Juergen"
And this can be set in locale yml files
More info: http://api.rubyonrails.org/classes/ActiveSupport/Inflector.html#method-i-transliterate

Seems to be also a fix for #227
Let me know what you guys think

+1 for this fix

:+1:

I've also run into this, my tag list has "montreal" and "montréal". I'm also not able to remove "montréal" from the tag list using tag_list.remove so we've been doing it manually with SQL queries.
Has anyone come up with a fix/workaround for this?

@seuros as a collaborator, can you advise/help ?
Thanks in advance

Hi @nburt, are you using MySql? If so, you can enable binary collation.
Via a rake task:
rake acts_as_taggable_on_engine:tag_names:collate_bin
Or via an initializer:
ActsAsTaggableOn.force_binary_collation = true

Thanks @rikettsie.
We are using MySql, I'll check this out tomorrow morning and let you know if that fixes it.


Snapshot splitting

When shipping snapshots to raft followers, the leader raft will return a single message containing the snapshot data. When sending large snapshots over gRPC, we would need to set large message size limits and also large timeouts. This can make the system unpredictable.
While users of etcd/raft can implement their own streaming, it might be more suited to provide the ability to set max message size when initializing raft and using that to split large snapshots (or other message types) into smaller messages.

@xiang90 LMK what you think about this. If we get consensus that this will be useful to have, I would be interested in working on this.

@anshulpundir
We do have streaming snapshot support in etcd. The suggested way to handle the problem you described is to always only include metadata into the raft snapshot message, and use a side-channel to supply the actual application snapshot.
The basic workflow would be:

send raft snapshot message with metadata
after receive the snapshot message with metadata, the follower fetches the application snapshot from the leader though streaming protocol or whatever
the follower save the application snapshot
the follower apply the raft snapshot
the follower apply the application snapshot

Ahh, I didn't realize that this was the case. I thought I saw some code in etcd which sent snapshots over http. But anyway, the way you suggested works. Wondering why you decided to go with different semantics (pull vs push for general raft flow) ? Can you please share the motivation behind this approach and the other approaches you considered ?
@xiang90

BTW, wondering why you decided to go with different semantics (pull vs push for general raft flow) ?

I do not think pull vs push matters a lot in theory. But for us, we have a use case that an out cluster component might pull a snapshot from leader. So we wanted to build that path anyway.

Can you please share the motivation behind this approach and the other approaches you considered ?

The motivation for why the snapshot works this way? The main principle is to keep application logic separated from raft logic as much as possible. There is no need to couple application snapshot sending logic into raft snapshot sending logic.

@anshulpundir
Can you help to improve our documentation on raft about snapshot sending if possible?

A quick question about snapshot pull logic, if you don't mind.

The basic workflow would be:

send raft snapshot message with metadata
after receive the snapshot message with metadata, the follower fetches the application snapshot from the leader though streaming protocol or whatever

If I get an error, timeout or any other issue here, I should drop this message with snapshot meta data on the follower side. And raft will resend new snapshot (maybe from new leader). Am I right?
Something like this using etcdserver/api/rafthttp api:
func (rn *raftNode) Process(ctx context.Context, m raftpb.Message) error {
	if m.Type == raftpb.MsgSnap {
		if err := rn.catchUp(ctx, m.Snapshot); err != nil {
			return err
		}
	}

	return rn.node.Step(ctx, m)
}

@xiang90


Failed to copy: update getinfo: object not found

What is your rclone version (eg output from rclone -V)

rclone v1.39
- os/arch: linux/amd64
- go version: go1.9.2

Which OS you are using and how many bits (eg Windows 7, 64 bit)

ubuntu16.04.3

Which cloud storage system are you using? (eg Google Drive)

ftp

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone -vv sync /www/wwwroot/anime/* woot:/public_html/ni-co.moe/anime/public_html

A log from the command with the -vv flag (eg output from rclone -vv copy /tmp remote:tmp)

2018/01/17 03:31:16 INFO  : 黄金拼图/[KTXP][Kiniro_mosaic][05][GB][720P].[卜卜酱].mp4: Copied (new)
2018/01/17 03:31:37 INFO  : 黄金拼图2/[YUI-7][Hello!! Kin-iro Mosaic][04][GB][X264_AAC][720P].mp4: Copied (new)
2018/01/17 03:31:40 INFO  : 魔装学园HxH/[Array Stars][Masou Gakuen HxH][06][720P][GB&JAP].mp4: Copied (new)
2018/01/17 03:31:58 ERROR : [CASO&SumiSora][Sound!Euphonium2][01-13][GB][720P]/[CASO&SumiSora][Sound!Euphonium2][03][GB][720P].mp4: Failed to copy: update getinfo: object not found
2018/01/17 03:32:08 INFO  :
Transferred:   91.528 GBytes (17.160 MBytes/s)
Errors:                 5
Checks:               526
Transferred:          561
Elapsed time:   1h31m1.7s
Transferring:
 *                                魔法禁书目录Ⅱ/10.mkv: 70% /300.690M, 3.480M/s, 25s
 *                     魔法禁书目录/16-dmxz.zerodm.mp4: 35% /279.863M, 3.110M/s, 58s
 *      ... the Bottom][BDrip X264 AAC 720P].mkv: 100% /351.495M, 4.876M/s, 0s
 *      ...esub][Kuromukuro][04v2][720P][GB].mp4: 16% /231.927M, 2.304M/s, 1m24s

2018/01/17 03:32:09 ERROR : Wake Up, Girls! Beyond the Bottom[720P]/[漏勺rip][Wake Up, Girls! Beyond the Bottom][BDrip X264 AAC 720P].mkv: Failed to copy: update getinfo: object not found
2018/01/17 03:32:30 INFO  : 魔法禁书目录Ⅱ/10.mkv: Copied (new)

Sorry, I didn't reply so late (LOL).
All files in question are present and usable (can be requested and accessed through http)
I think this problem is caused by some special characters in the filename.

It would be worth retrying with the latest beta since this query is quite old now!
Is it a public server this? Could I try it?

hi
I have sent you the server information by mail.
Please note the email from<EMAIL_ADDRESS>

I get the same error with latest rclone, ftp-ing files from an ubuntu server 18.04.
Tons of 'Failed to copy: update getinfo: object not found' errors in the log. Even if the files are actually created correctly at the destination. The problem is that with rclone retrying the files several times, it adds 30 minutes and tens of GBs of useless traffic to my backups.

Which ftp server are you using on the ubuntu server?
Can you paste a log with -vv of the problem?

Rclone is running on ubuntu server. The FTP server is Pure-FTPd running on a NAS.
$ rclone -vv copy /mnt/tank/music/artists/America\ -\ History\ \[Vinyl\]/ nsa:/public/backup/music/artists/America\ -\ History\ \[Vinyl\]/ 2018/12/30 09:20:45 DEBUG : rclone: Version "v1.45" starting with parameters ["rclone" "-vv" "copy" "/mnt/tank/music/artists/America - History [Vinyl]/" "nsa:/public/backup/music/artists/America - History [Vinyl]/"] 2018/12/30 09:20:45 DEBUG : Using config file from "/home/stefan/.config/rclone/rclone.conf" 2018/12/30 09:20:45 DEBUG : ftp://<IP_ADDRESS>:21/public/backup/music/artists/America - History [Vinyl]: Connecting to FTP server 2018/12/30 09:20:46 INFO  : ftp://<IP_ADDRESS>:21/public/backup/music/artists/America - History [Vinyl]: Waiting for checks to finish                                           2018/12/30 09:20:46 INFO  : ftp://<IP_ADDRESS>:21/public/backup/music/artists/America - History [Vinyl]: Waiting for transfers to finish                                                                                2018/12/30 09:20:46 DEBUG : ftp://<IP_ADDRESS>:21/public/backup/music/artists/America - History [Vinyl]: Connecting to FTP server                                                                                2018/12/30 09:20:46 DEBUG : ftp://<IP_ADDRESS>:21/public/backup/music/artists/America - History [Vinyl]: Connecting to FTP server                                                                                2018/12/30 09:20:46 DEBUG : ftp://<IP_ADDRESS>:21/public/backup/music/artists/America - History [Vinyl]: Connecting to FTP server                                                                                2018/12/30 09:20:46 ERROR : AlbumArtSmall.jpg: Failed to copy: update getinfo: object not found                          2018/12/30 09:20:48 ERROR : HMV.nfo: Failed to copy: update getinfo: object not found                                    2018/12/30 09:20:48 ERROR : HMV.jpg: Failed to copy: update getinfo: object not found                                    2018/12/30 09:20:48 ERROR : Thumbs.db: Failed to copy: update getinfo: object not found                                  2018/12/30 09:20:54 ERROR : a2.i.need.you.flac: Failed to copy: update getinfo: object not found   [...] 2018/12/30 09:21:45 ERROR : Attempt 3/3 failed with 17 errors and: update getinfo: object not found                      2018/12/30 09:21:45 Failed to copy: update getinfo: object not found

If it helps, it's always the same files that "fail" to copy btw.

Is it because there are [ and ] in the directory?

Thanks, that was the issue. What is the explanation, what's wrong with the square brackets in the directory name?

Thanks, that was the issue. What is the explanation, what's wrong with the square brackets in the directory name?

I managed to replicate this with a local copy of pureftpd
It is only directories with [ ] in that it doesn't like - files are fine.
Doing a bit of poking around it appears pureftp is interpreting those [ as ] glob characters in the MLSD command rclone is using to list directories whereas it shouldn't according to the rfc: https://tools.ietf.org/html/rfc3659#section-2.2.2
I'll email the author and see what they say.

@ncw The problem with [ ] in the directory name should not be fixed in the 1.52.0, right?

The problem with [ ] in the directory name should not be fixed in the 1.52.0, right?

It is a bug in pureftpd alas...

Ok, thank you very much

@ncw
Do you think this pureftp bug deserves to be added in the FTP Caveats documentation?

Closing after 6 months of inactivity


How to construct graph object

Say I have the following CSV:
id, title, geometry
1, 'feature 1', 'POLYGON(...)'
2, 'feature 2', 'POLYGON(...)'

I'm trying to turn this into a geosparql compatible graph. Each row in the CSV would have to look like:
ex:1
    a geo:Feature ;
    skos:prefLabel "feature 1" ;
    geo:hasGeometry [
        geo:asWKT "POLYGON(...)"^^geo:wktLiteral ;
    ] ;

The graph would look like
graph LR
    A((eg:1)) -->|rdf:type| B((geo:Feature))
    A -->|skos:prefLabel| C(("feature 1"))
    A -->|geo:hasGeometry| D((BlankNode))
    D -->|geo:asWKT| E(("POLYGON (...)"))

I'm stuck on how I would generate a blank node that points to the WKT
@prefix rr: <http://www.w3.org/ns/r2rml#> .
@prefix over: <http://overturemaps.org#> .
@prefix skos: <http://www.w3.org/2004/02/skos/core#> .
@prefix geo: <http://www.opengis.net/ont/geosparql#> .
@prefix ex: <http://example.org/#> .

<feature> a rml:TriplesMap;

    rml:logicalSource [
        a rml:LogicalSource;
        rr:sqlVersion rr:SQL2008;
    ];

    rml:subjectMap [
        rml:template "http://example.org/#{id}" ;
        rml:class geo:Feature;
    ];

    rml:predicateObjectMap [
        rml:predicate skos:prefLabel ;
        rml:objectMap [ rml:reference "title" ]
    ];

    rr:predicateObjectMap [
        rr:predicate geo:hasGeometry;
        rr:objectMap [
#.           TODO: How would this work?
#            rr:parentTriplesMap <#GeometryMap>
          rml:termType: rml:BlankNode
        ]
    ]
.

How would I generate the triple?
<blank_node from objectMap> geo:asWKT "POLYGON(...)"^^geo:wktLiteral ;

Sorry, this is not an issue for the spec. it's a question about how to do things. Please, open it in the proper repository (rml-questions --> discussions)

Ok will do, haven't seen a separate repo for Discussions only before so didn't know where to put it.


win32: vimdll: Fix multibyte input

I had a report that the VIMDLL patch still has a problem.
Some multibyte characters which include 0x80 cannot be input on gvim.exe
when compiled with VIMDLL=yes.  E.g. 、 and 。 cannot be input when
'enc' is utf-8.  This fixes it.

Codecov Report

Merging #4330 into master will increase coverage by <.01%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #4330      +/-   ##
==========================================
+ Coverage   79.99%   79.99%   +<.01%
==========================================
  Files         108      108
  Lines      141965   141965
==========================================
+ Hits       113558   113569      +11
+ Misses      28407    28396      -11

Impacted Files
Coverage Δ

src/getchar.c
76.41% <ø> (ø)
:arrow_up:

src/gui_gtk_x11.c
48.5% <0%> (-0.55%)
:arrow_down:

src/window.c
86.57% <0%> (-0.04%)
:arrow_down:

src/terminal.c
80.66% <0%> (+0.03%)
:arrow_up:

src/term.c
71.18% <0%> (+0.05%)
:arrow_up:

src/screen.c
80.7% <0%> (+0.08%)
:arrow_up:

src/ex_cmds2.c
87.31% <0%> (+0.11%)
:arrow_up:

src/syntax.c
79.74% <0%> (+0.12%)
:arrow_up:

src/gui.c
57.99% <0%> (+0.51%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 92fd599...a753098. Read the comment docs.


Em dash for decrement

On some devices (particularly iPhones), -- persistently becomes —. Allowing — instead of -- would make it easier to write code on these devices, particularly in the Discord.

I'm in favor of this. It could be an easy PR for anyone interested in contributing.


Added clarification to key distributions

Added clarification to key distributions.
Original wording just said "in your repository" - this could have referred to the one triggering the Github action or the one being pulled from. A few extra words can clear this ambiguity up and make the documentation non-ambiguous.

Thank indeed makes things more understandable. Thank you @rorcores!


Document --silent flag

See https://github.com/foundry-rs/foundry/pull/1820#event-6752318750

@onbjerg Would --silent it be part of Watch Options or Compiler Options ? And I am assuming the option will be the same for all build, test, and create?

It would be part of this file: https://github.com/foundry-rs/book/blob/master/src/reference/forge/core-build-options.md

Would this look ?

@Tirthnp Ideally it should be but practically it does show up for all of those commands:
$ forge snapshot --help | grep silent
        --silent

@onbjerg So does the params next to --silent makes sense ?

Oh sorry I missed that, no the path doesn't make sense since it doesn't take an argument

@onbjerg I created PR https://github.com/foundry-rs/book/pull/384


Support/dev portal links

📝 Description
I did this modification yesterday, but I needed to move the paragraph elsewhere so it's not deleted when the doc is generated again.
❓ Context

Impacted projects: ``Third-party devs using this documentation

✅ Checklist

[ ] Test coverage
[x] Atomic delivery
[x] No breaking changes

📸 Demo
No demo

:x: @github-actions
[Test] Libraries / Test Docs (pull_request) Failing after 1m — Test Docs

somehow it still fails :'(

https://github.com/LedgerHQ/ledger-live/runs/7741236346?check_suite_focus=true

Ah too bad, well I will remove this part from the auto-generated doc and only update the other READMEs.

It's just because there are two extra space at the end I think.
I think the command you can run locally is:
pnpm turbo run doc

over your current PR and it will fix them all


Add typescript definitions for react-jsx-highstock

This builds on #303 and extends types to also cover the
react-jsx-highstock package.
Please verify that the type definitions makes sense, @anajavi and and
@whawker. I used the types you defined, @anajavi, as reference and tried
to compare props of RJH components with Highcharts config types.
It works for our usage of RJH, but not all components are tested.

Hey this is great!
I tested it a bit and it seems to work nicely.
Although there are a couple of extensions in node_modules/highcharts/modules/stock.d.ts  to base highcharts types, including them would make this a bit too complex for my taste. We can add them later if somebody needs them.
@whawker I think we could merge this as is?

@anajavi Thanks for the quick feedback!
Regarding the extensions in stock.d.ts, as far as I can see those are merged in already when you import Highcharts from "highcharts/highstock".

@whawker do you mind if I merge this and release as an alpha?

Yeah, go for it!

released as<EMAIL_ADDRESS>

I guess there aren't any problems with this, so I am gonna release this.


Support web

Can you support web?
JavaScript seems to support this feature
https://developer.mozilla.org/en-US/docs/Web/API/Picture-in-Picture_API

The API you have linked only concerns the <video> DOM elements, which Flutter doesn't use.


Next airing episode data

Is your feature request related to a problem? Please describe.
On hianime.to, let's say One Piece, I can see that there's data for when the next episode will be released, is this possible to add in the api? I searched /episodes and /info and didn't find any data related to this.
Describe alternatives you've considered
Using consumet api and the anilistId provided by aniwatch api for this feature

An API endpoint for this already exists. The docs.

I saw that, but it requires a date. I was probably not clear enough, but I meant a property in seconds or so that tells when a specific anime's next episode will be released.
This is how consumet shows it (at the bottom):

Ohh got it. Yeah, I think we could add this feature like consumet.

All righty, thank you!

@yusuftuncay Hey, I've added the airingTimestamp and secondsUntilAiring properties to the /schedule endpoint response schema.
Closing this as completed. Ff you have any other queries then feel free to open another issue.

@ghoshRitesh12 I have a question and couldn't ask it anywhere else, I don't see the options for all posibilites of "sort" values are when searching, I tried score and it works fine, but I want the "most watched" which I found that hianime has, tried "most-watched", "mostWatched" and other things but nothing works for that, is there even a value for sorting on most watched?

@ghoshRitesh12 I have a question and couldn't ask it anywhere else, I don't see the options for all posibilites of "sort" values are when searching, I tried score and it works fine, but I want the "most watched" which I found that hianime has, tried "most-watched", "mostWatched" and other things but nothing works for that, is there even a value for sorting on most watched?

I recently checked and its not working on hianime.to

Oh okay, it does work now in your api. Just checked hianime and it works there as well. Thanks a lot! By the way, I implemented so many functions on my website using your api, I'm thankful that you created this! Everything loads so fast and has literally no bugs compared to consumet. Oh and this is the website if you feel like checking it out.

Sorry for disturbing you, but what is the quality of an episode by default? Because I can see it's not 1080p comparing to hianime.to, same episode, same timeframe.

Oh okay, it does work now in your api. Just checked hianime and it works there as well. Thanks a lot! By the way, I implemented so many functions on my website using your api, I'm thankful that you created this! Everything loads so fast and has literally no bugs compared to consumet. Oh and this is the website if you feel like checking it out.

Thank you so much ❤


Python API, REL TABLE GROUP & DELETE ⇒ Segmentation fault

Hello,
While testing Kuzu 0.3.2, I observed that:
Deleting nodes and relation which include at least one REL TABLE GROUP implies a segmentation fault.
A way to reproduce this observation is included in this archive: kuzu_segfault.tar.gz
Cheers,
PHF

Here is a way to reproduce this observation:
python3 experiment.py

Result:
Segmentation fault

Where experiment.py is:
from db import DB
from shutil import rmtree

db_path = "./kuzu_db"
rmtree(db_path)
db = DB(db_path)
db.reset()

Where db.py is:
from pathlib import Path
import kuzu
from kuzu import QueryResult

class DB():

    def __init__(self, db_path):
        self._db = kuzu.Database(Path(db_path).resolve())
        self._conn = kuzu.Connection(self._db)
        self._schema_is_defined()
        self._add_a_few_nodes()
        self._add_a_few_edges()

    def reset(self):
        query = "MATCH ()-[r]->() DELETE r;"
        self._execute(query, {})
        query = "MATCH (n) DELETE n;"
        self._execute(query, {})
        return self

    def _add_a_few_nodes(self):
        self._store_user({ 'id': 'user-id', 'name': 'user-name' })
        self._store_book({ 'id': 'book-id-1', 'name': 'book-name-1' })
        self._store_book({ 'id': 'book-id-2', 'name': 'book-name-2' })
        self._store_book({ 'id': 'book-id-3', 'name': 'book-name-3' })

    def _add_a_few_edges(self):
        self._store_own_user_book({ 'id': 'user-id' }, { 'id': 'book-id-1' })
        self._store_own_user_book({ 'id': 'user-id' }, { 'id': 'book-id-2' })
        self._store_own_user_book({ 'id': 'user-id' }, { 'id': 'book-id-3' })

    def _execute(self, query:str, parameters:dict) -> QueryResult:
        return self._conn.execute(query, parameters)

    def _schema_is_defined(self) -> None:
        queries = [
            'CREATE NODE TABLE User (id String, name String, PRIMARY KEY (id));',
            'CREATE NODE TABLE Book (id String, name String, PRIMARY KEY (id));',
            'CREATE NODE TABLE Article (id String, name String, PRIMARY KEY (id));',
            'CREATE REL TABLE GROUP own (FROM User TO Book, FROM User TO Article);',
        ]
        for query in queries:
            try:
                self._write(query)
            except RuntimeError:
                # The table already exists.
                pass

    def _write(self, query, params={}):
        if isinstance(query, str):
            self._execute(query, params)

        if isinstance(query, list):
            for q in query:
                self._execute(q, params)

    def _execute(self, query:str, parameters:dict) -> QueryResult:
        return self._conn.execute(query, parameters)

    def _store_user(self, user:dict):
        query="""
        MERGE (u:User {id: $id})
        ON CREATE SET u.name = $name
        ON MATCH SET u.name = $name;
        """
        self._write(query, user)
        return self

    def _store_book(self, book:dict):
        query="""
        MERGE (b:Book {id: $id})
        ON CREATE SET b.name = $name
        ON MATCH SET b.name = $name;
        """
        self._write(query, book)
        return self

    def _store_own_user_book(self, user:dict, book:dict):
        parameters={ 'user_id': user['id'], 'book_id': book['id'] }
        query="""
        MATCH (u:User), (b:Book) WHERE u.id = $user_id AND b.id = $book_id
        MERGE (u)-[:own]->(b);
        """
        self._write(query, parameters)
        return self

I guess that this counts as a workaround:
def reset(self):
    if ISSUE_3552_IS_RESOLVED:
        self._execute("MATCH ()-[r]->() DELETE r;", {})
        self._execute("MATCH (n) DELETE n;", {})
    else:
        db_path = self._db.database_path
        del self._db
        import gc
        gc.collect()
        import shutil
        shutil.rmtree(db_path)
        self._db = kuzu.Database(db_path)
        self._conn = kuzu.Connection(self._db)
        self._schema_is_defined()
    return self

One may observe a relation to the issue #3268. If it was resolved then we could replace this:
del self._db
import gc
gc.collect()

with this:
self._db.close()

Hi PHF,
The issue has already been resolved in our latest master. You can download and try our nightly build python package by pip install kuzu==0.3.3.dev45.
Thanks for reporting the bug to us!
Cheers,
Ziyi


Receiver class io.grpc.netty.shaded.io.grpc.netty.NettyClientTransport does not define or inherit an implementation of the resolved method 'abstract io.grpc.internal.ClientStream newStream(io.grpc.MethodDescriptor, io.grpc.Metadata, io.grpc.CallOptions, io.grpc.ClientStreamTracer[])' of interface io.grpc.internal.ClientTransport

Using com.google.cloud:google-cloud-pubsub:1.114.7 in a Spring Boot (version 2.5.6) application with Java 17 throws this error upon subscription and handling of messages:

Exception in thread "grpc-default-executor-0" java.lang.AbstractMethodError: Receiver class io.grpc.netty.shaded.io.grpc.netty.NettyClientTransport does not define or inherit an implementation of the resolved method 'abstract io.grpc.internal.ClientStream newStream(io.grpc.MethodDescriptor, io.grpc.Metadata, io.grpc.CallOptions, io.grpc.ClientStreamTracer[])' of interface io.grpc.internal.ClientTransport.
at io.grpc.internal.MetadataApplierImpl.apply(MetadataApplierImpl.java:74)
at io.grpc.CallCredentials2$1.apply(CallCredentials2.java:61)
at io.grpc.auth.GoogleAuthLibraryCallCredentials$1.onSuccess(GoogleAuthLibraryCallCredentials.java:133)
at com.google.auth.oauth2.OAuth2Credentials$FutureCallbackToMetadataCallbackAdapter.onSuccess(OAuth2Credentials.java:549)
at com.google.auth.oauth2.OAuth2Credentials$FutureCallbackToMetadataCallbackAdapter.onSuccess(OAuth2Credentials.java:540)
at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1080)
at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30)
at com.google.common.util.concurrent.ExecutionList.executeListener(ExecutionList.java:141)
at com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:130)
at com.google.common.util.concurrent.ListenableFutureTask.done(ListenableFutureTask.java:107)
at java.base/java.util.concurrent.FutureTask.finishCompletion(FutureTask.java:381)
at java.base/java.util.concurrent.FutureTask.set(FutureTask.java:232)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:272)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:833)

Downgrading to com.google.cloud:google-cloud-pubsub:1.114.6 solves the issue.

Hi @lpellegr,
Are you using maven or gradle? And are you using a google-cloud-libraries BOM (https://github.com/GoogleCloudPlatform/cloud-opensource-java/wiki/The-Google-Cloud-Platform-Libraries-BOM)?
Can you share your dependency tree?

@hannahrogers-google Thanks for your answer.
I am using Gradle 7.2. No google-cloud-libraries BOM is used directly. Here are the dependencies:
https://gist.github.com/lpellegr/f6cc11802b8d6c254b5d4d9b034de364

Thanks for providing that info! I am trying to reproduce this issue, can you share steps to reproduce? How are you configuring the Subscriber, and when do you see this error (upon creating the subscriber, upon starting the subscriber, etc.)?

This happens to me too. I can see this error when trying to publish a message from Google Cloud Function.
I downgraded to 1.114.6 and it seems to be fine now.

I am seeing dependency conflicts in the dependency tree that you provided. Specifically:
io.grpc:grpc-core:1.39.0 -> 1.41.0
io.grpc:grpc-protobuf:1.39.0 -> 1.41.0
io.grpc:grpc-context:1.39.0 -> 1.41.0
io.grpc:grpc-api:1.39.0 -> 1.41.0
io.grpc:grpc-stub:1.39.0 -> 1.41.0
It looks like the NettyClientTransport class has changed between grpc versions 1.39.0 to 1.41.0. It looks like this specific change may be causing the dependency conflict: https://github.com/grpc/grpc-java/pull/8355. A fourth argument has been added to the newStream implementation.
For resolving dependency conflicts we typically recommend using a google-cloud-libraries BOM. Please find instructions here: https://cloud.google.com/java/docs/bom
If you are still running into issues after resolving the dependency conflicts, please let me know.


fix(ui-components): apply lost background to splitter

vscode color regression after - https://github.com/SAP/open-ux-tools/pull/759
after we removed vscuie variables - splitter lost its background because vscuie variable name was not matching vscode variable:

Adapted code to use previous variable --vscode-debugToolBar-background

@815are Hold on, splitter background and debugtoolbar background don't sound semantically the same. I believe there is a variable in vscode that should fit better here.
Where are we using this variable? What is the component? Is it the area splitter?

Because that is var(--vscode-editorWidget-border) and on hover var(--vscode-focusBorder)
In any case, there should be a better match for this as I don't think splitter should go into debugtoolbar. It sounds too random. 😄

Where are we using this variable? What is the component? Is it the area splitter?

Your screenshot is correct.
It looks following:

--vscode-debugToolBar-background:

--vscode-editorWidget-border

Should we still apply --vscode-editorWidget-border? Or we need adjust color for inner dots? currently it is '--vscode-icon-foreground'

@gpeuc
Regarding debug menu, I am not sure if I was author of that usage or it was passed by someone else, but maybe idea was that debug toolbar have splitter dots as well:

The dots might be ok, since they have an exact match, but the separator background is strange.
Native vscode is using this:

for various separators

can you please see if there is something that's a good match in our webview for this?

can you please see if there is something that's a good match in our webview for this?

--vscode-panel-border: rgba(128, 128, 128, 0.35);
--vscode-panelSection-border: rgba(128, 128, 128, 0.35);
--vscode-settings-headerBorder: rgba(128, 128, 128, 0.35);
--vscode-settings-sashBorder: rgba(128, 128, 128, 0.35);
--vscode-terminal-border: rgba(128, 128, 128, 0.35);
--vscode-notebook-cellToolbarSeparator: rgba(128, 128, 128, 0.35);
If I use --vscode-panel-border:
Light:

Dark:

Should we use --vscode-panel-border?

--vscode-panelSection-border
Because it is a border that splits sections. And it seems RGBA values match as well.

--vscode-panelSection-border
Because it is a border that splits sections. And it seems RGBA values match as well.

ok, pushed change to use --vscode-panelSection-border


Add jumping mod back

Jumping mod was removed because not working on 0.4.13 stable.
The fact is it was working, we just needed to put higher values (for example, 10 intead of 0.10). I could experience it on VanessaE's Dreambuilder server and @crabman77 confirmed me that it was working.
Could we add it back and find good values to it ?
Also, could we add it to the Creative server ?
(nice for races and traps)

It won't be added on the HG server
But yes, it's a good thing to re-add it to the Classic and Creative server :)

Edited the post

https://github.com/MinetestForFun/server-minetestforfun/commit/497fdfef65c025c163505c411da8e52e32d9c433
And https://github.com/MinetestForFun/server-minetestforfun-creative/commit/6af5afe81dab9967c631a7acb2d58dc865e6cfe3


Cache cleaner policy

Hi there,
I have a quick question. Which policy does the cache cleaner use?
LRU, LFU, or something else?
Thanks.

These are the queries used for the cleaning:
"SELECT * FROM cache WHERE size IS NOT NULL AND owner='orphans' AND creation_date < ?",
"SELECT * FROM cache WHERE size IS NOT NULL AND creation_date < ? ORDER BY last_access ASC",

So it removes the "orphan" entries and then the least recently accessed ones until the total size goes below the limit.

Thanks!


Trie

Type
Data-Structure
Directory
Data-Structures => Tree => Trie => Trie.extension
Description
Trie is a non-linear data structure that is used for locating specific keys from within a set.
Note : Just simply demonstrate insert and search operations for a dict.
Programming Language

[X] c++
[X] c
[X] python
[ ] java
[ ] kotlin
[ ] javascript
[ ] typescript
[X] c#
[ ] swift
[ ] ruby
[ ] scala
[ ] go
[ ] php
[ ] haskell
[ ] bash

I to have fix your issue if you want then assignee

I to have fix your issue if you want then assignee

Hi @SudipMukhejee, please mention your preferred language.

Hi @ahampriyanshu Can I pick this up for Java, Javascript and typescript?

Javascript i used


standalone binary without macaroons: connection closed after restart

In a setup with
export TDEX_NO_MACAROONS=true
tdexd

and tdex config as
tdex config
fee_wallet: {"privateKey": "mykey", "blindingKey": "anotherkey"}
macaroons_path:
network: liquid
no_macaroons: true
rpcserver: localhost:9000
tls_cert_path:
explorer_url: http://<IP_ADDRESS>:3002

after a restart seems to have error on tdex unlock
tdex unlock --password xxxxx
[tdex] rpc error: code = Unavailable desc = connection closed

level=info msg="Replaying file id: 0 at offset: 0\n"
 level=info msg="Replay took: 13.014µs\n"
level=info msg="Restoring internal wallet's utxo set. This could take a while..."
level=info msg="starting daemon"
level=info msg=Done
level=info msg="wallet unlocker interface is listening on :9000"
level=info msg="operator interface is listening on :9000"
level=info msg="trade interface is listening on :9945"

I'm able to replicate this.
I tried on both macOS and Linux platforms and the feeder updates price on the target daemon as expected since their configurations are ok.


Add "Getting Involved" section to docs

A new "Getting Involved" section needs to be added to the Meshery docs. It should include the same links and invitations found in the Community section here - https://layer5.io/meshery/#community.
Also, a link to the welcome guide needs to be added to the Meshery project page under the Community section.

@arghac14 would you like to give this one a go as well?

@leecalcote  sure! but exactly where should I put the 'Getting involved' html file? Is it here https://github.com/layer5io/meshery/tree/master/docs/_layouts ?
And yes, I'll add the link to the welcome guide under the Community section.

@arghac14 thank you. Good question. You're looking in the right place. Instead of the "Getting Involved" (or maybe it should be called "Contributing") being a link to one document, it'll be a new section on the Overview page. This new section is to contain the same Community information -

@leecalcote review the PR.

Fixed by #182

@arghac14 good deal. Jump into Slack here -- http://slack.layer5.io, if you would.

@raghavaggarwal99, there's a misspelling in the Community section - "callendar" that needs to be corrected. Could you please fix this?

@raghavaggarwal99 seeing now that this issue was fixed above (https://github.com/layer5io/meshery/issues/176#issuecomment-517721505), I'll close this issue and open a new one for the misspelling issue.


Ionide v3.10.1 - '>F#: New Project' does not work

I'm using VSCode on Ubuntu 16.04 with mono, and recently I have run into issues using the 'New Project' shortcut. When used it shows the following in the output panel:

forge new project -n example.fsproj -t console --folder  --no-prompt
Unhandled Exception:
System.BadImageFormatException: Could not resolve field token 0x04000008
File name: 'Forge'
[ERROR] FATAL UNHANDLED EXCEPTION: System.BadImageFormatException: Could not resolve field token 0x04000008
File name: 'Forge'

I just realized that this issue is caused by an issue with forge itself.


Add agenda/notes for monthly meeting

Add an author: Stephen tag to the front matter too.

Good call, updated now.

I'll update with actual meeting notes and submit a new request.


[feature request] - Output coloring

At the moment I have "java test runner" output window. I tried to install the Output colourizer, but it didn't help. I saw a Todo Tree plugin, which has got a special user setting, where you were able to define a regular expression for colouring purpose for TODO lines.
My java log has timestamps in each row. I would like to be able to set
"[2018-05-06 12:45.45000] - [INFO] - blablabla" to purple and bold font till the [INFO] with yellow background and blablabla just leave it as it is. (white colour with black background)
"[2018-05-06 12:45.45000] - [ERROR] - abrakadabra" to red and bold font till the [ERROR] with green background and abrakadabra just leave it as it is. (white colour with black background)
I would like to be able to set the colouring by project type in the user/workspace settings. (probably to be able to define by name of the items of the output dropdown list)
It's just an idea, but it would be very flexible with regex settings, so if I have a custom log, I can set the colouring. (the output colourizer plugin  probably works, if it finds a "warn" word at the beginning of the line, but who use that kind of logs without timestamps?)
At the moment I need to copy over the output log to a different editor to see what's going on, as everything is white font black background.
The TODO line colouring is not perfect, but I got, what you can really achieve with it, unluckily the logging is very customized many times. (mine has got a logback.xml, so hardcoded coloring won't solve the issue)

Unfortunately, this plugin does not exist.


fix #3867 IoT list_principal_things

This pull request will fix #3867
Fix iot list_principal_things response list of thingnames.

Codecov Report

Merging #3868 (95b8f1c) into master (9b3e932) will decrease coverage by 0.00%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #3868      +/-   ##
==========================================
- Coverage   94.84%   94.84%   -0.01%
==========================================
  Files         447      447
  Lines       48893    48893
==========================================
- Hits        46373    46371       -2
- Misses       2520     2522       +2

Flag
Coverage Δ

servertests
55.87% <ø> (ø)

unittests
94.71% <ø> (-0.01%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

moto/iot/models.py
90.34% <ø> (ø)

moto/iam/models.py
95.13% <0.00%> (-0.15%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9b3e932...95b8f1c. Read the comment docs.


Symbols for logical expressions

Hey, I could use some more logical expression symbols.

[ ] \models, but for equivalence. In some cases, \equiv is used, but not in general. There is the \models symbol, but only from left to right. Here, logical equivalence is denoted by the \models symbol followed by the \models symbol mirrored along the y-axis.
[ ] \sslash (from stmaryrd package). A double slash for function declarations in first order logic. The regular slashes are too far apart from each other to be readable in this context.

These two would be awesome. I am sure I will find more over time, but I am missing especially the first one pretty much daily. Thank you for the great work!

A backwards \models could be obtained if we supported \reflectbox (via \mathop{\reflectbox{\models}}), which I feel like might be relatively easy to do via CSS transform, and quite broadly useful.
stmaryrd seems to implement \sslash as a custom font symbol, but we could also consider a simple implementation like /\hspace{-0.25em}/ which looks like this:
. You could do this now via custom macros.


The deployment encountered an exception

i make python3 brc20_light_client_sqlite.py to get the following.
Trying to get events from http://<IP_ADDRESS>:8000
Event count:  182
Ticks refreshed in 0.29262495040893555 seconds
Traceback (most recent call last):
  File "brc20_light_client_sqlite.py", line 1303, in <module>
    if index_block(current_block):
  File "brc20_light_client_sqlite.py", line 751, in index_block
    transfer_inscribe(block_height, inscr_id, source_pkScript, source_wallet, tick, amount)
  File "brc20_light_client_sqlite.py", line 326, in transfer_inscribe
    execute_with_params('''insert into brc20_events (event_type, block_height, inscription_id, event)
  File "brc20_light_client_sqlite.py", line 51, in execute_with_params
    cur.execute(sql, obj)
sqlite3.OperationalError: near "returning": syntax error
rolling back

You should update your python. Some of the features that we've used are very new so they are not included in default SQLite library of older pythons.


Mechanical Turk Integration

Integration of CVAT with MTurk for deploying work as HITs would be very useful for such projects. Need to integrate Turkic framework of VATIC with CVAT.
I would also like to contribute to your project. Please help me in setting up the development environment for the same.

Hi,
It is a feature in our mind which we don't have resources to implement till EOY for sure. If you can help it will be awesome. Development environment is described here: https://github.com/opencv/cvat/issues/18.
Several notes:

Need to implement a separate Django application for the feature
It isn't good to use turkic from VATIC because it isn't supported anymore. I would suggest using an official API (boto3?) - http://boto3.readthedocs.io/en/latest/reference/services/mturk.html
Need to think how split a task on many HITs (by frame/image?) because HITs should be trivial. Also need to automatically create "verification" tasks to check results. In many case it will be necessary to merge annotation from several HITs.
GUI to publish, view progress, ... Need to have sketch of it.
The application should have REST API.

In general it is a big feature which cannot be easily implemented.

General requirements:

Easy interface to create/review/accept an annotation task for MTurk (separate Django app).
Probably need to have a separate instance on AWS for such tasks.
Automatic authorization from MTurk, special group to access only links from MTurk django app
Email notifications about deadlines to verify annotation tasks, etc...

Requirements (image annotation):

One image per a segment. Thus it is easy to post such job as HIT.
Several jobs for the same segment.
Automatically correlate annotations in different jobs for the same segment. Automatically find "trash" in annotations and discard such HIT.
Minimum number of jobs for a segment is 3.
Maximum number of jobs for a segment is 7. After that need to verify or discard HITs due to low quality.
One user cannot do annotations for the same segment (several similar jobs). Need to improve auth/permissions to access only specific jobs.

Hi, Wanted to know if any progress has been made in this direction :)

@nik13 , we are not working at the direction at the moment.

Nice overview of using a crowd sourcing platform (Toloka): https://www.youtube.com/watch?v=yyybisXHFc0

Has there been any progress on the integration with crowdsourcing platforms?

I hope have some news in Q2-Q3 on the topic...

Very interested in this too!


Sort by severity is inconsistent

1 is the highest, but 0 appears even before

And after 3 years this issue, which requires a few one-line fixes, is still there.
I think i will take the liberty to spend just one day to fix it in every scanner at once, until we implement that better GUI we were talking about that long.


Unable to publish to Azure with HTTP Error 502.5 - Process Failure

I am having issues deploying to Azure, even though the publish "succeeded." This repository can be used as an example. I've had similar issues on multiple version of Visual Studio 2017 (include 15.6.2 and 15.5.2) using the default ASP.NET Core templates.
The issue only seems to be related to the latest version of Visual Studio 2017, which is 15.6.2 using the default ASP.NET Core templates. Specifically, using the NuGet package Microsoft.AspNetCore.All (2.0.6). In addition, downloads to previous versions of Microsoft.AspNetCore.All cause the build to fail.
#406 is similar to my issue but I was told to open a new one to include a repository.

Is this a scenario where you are git pushing to Azure (i.e. build on Azure), or you are deploying from VS via WebDeploy (i.e. build locally)?
Also, when you say "cause the build to fail", can you provide to complete error that you are getting?

I just tried git pushing this repo to a clean new Azure Web App, and everything works fine. It builds without errors, and I can then access /api/values in the running app.

@davidebbo I am deploying directly through Visual Studio using the Publish menu.
This repository was built and published with the older version of Visual Studio (15.5.2) that seems to work fine (which is why I put a line through the "multiple versions of Visual Studio" sentence).
This issue could be related to Microsoft.AspNetCore.All (2.0.6) not being available on Azure until today. I will upgrade my version of Visual Studio to the latest and try again.

I'll let core experts like @Tratcher comment on some of this. My expectation is that the build artifact produced by VS cannot be influenced by the version of Core that happens to be on Azure. Instead, it will produce whatever artifacts it thinks are right for the project, and will deploy that. Then how that runs on Azure can depend on what's available.

Updated Visual Studio 2017 to 15.6.2; created a new API project using ASP.NET Core Web Application and published to Azure without issue. In addition, I attempted to downgrade Microsoft.AspNetCore.All to 2.0.5 and rebuilt without issue.
I am going to run a repair on my other installation of Visual Studio and see if that resolves the issues I was seeing yesterday. I am closing this issue.
Thanks for your help!

Hello - I am encountering this issue with VS2017 15.7.1 Enterprise.  I have created a .net core 2.0 mvc project.  made some cosmetic changes to it and published to azure.  the publish was successful but got 502.5 error.
i have tried the wwwroot wipe and not successful.
in the log, i got this
8000808c.100010Keywords1129163281ApplicationRD00155DFE9F35Application 'MACHINE/WEBROOT/APPHOST/DEVARCOSACOM' with physical root 'D:\home\site\wwwroot' failed to start process with commandline 'dotnet .\xyz.com.dll', ErrorCode = '0x80004005 : 8000808c.100420Keywords1129293625ApplicationRD00155DFE9F35Warning: Could not create stdoutLogFile \?\D:\home\site\wwwroot\logs\stdout_8344_201851116448.log, ErrorCode = -2147024893.100010Keywords1129293921ApplicationRD00155DFE9F35Application 'MACHINE/WEBROOT/APPHOST/DEVARCOSACOM' with physical root 'D:\home\site\wwwroot' failed to start process with commandline 'dotnet .\xyz.com.dll', ErrorCode = '0x80004005 : 8000808c.
Is there any other information i can post there so that we can have better idea?
Thank you,
Henry

I experienced that too.. Azure with HTTP Error 502.5 - Process Failure
In my case, I resolved the problem by updating my .net core sdk to the latest 2.1 SDK (v2.1.302).

It looks like the issue is that the VS 2017 templates use the latest updated NuGet packages which is right, it's good. But Azure isn't prepared for these so Azure will not run the latest. I had to downgrade packages to a lower version to where the App then did run. This wasn't a framework downgrade, this was NuGet package minor version downgrades.

You can configure the publish profile in VS2017 to use Self-Contained mode.
It will publish all dependencies, so you will be able to run your web app in Azure.
Make sure the target runtime corresponds to the configuration of web app in Azure.

@ru-sh for new people creating new templates and publishing those apps like directed from VS 2017, those are extra steps that might not be obvious. I can try them the next time I play around and run into the above issues but hopefully Azure is now prepared for the latest packages. At least by now.

It's not ready; I had to publish as a self contained app last night, took me hours to figure it out also.
Incredibly frustrating.

@James-Jackson-South I still need to remember to try this. Why wouldn't self-contained mode be on by default I wonder? Or at least could it be explained in the process that 2017 provides us.

Thank you @ru-sh - self-contained did the trick.

I experienced that too.. Azure with HTTP Error 502.5 - Process Failure
In my case, I resolved the problem by updating my .net core sdk to the latest 2.1 SDK (v2.1.302).

Same issue with me, updating the .net core sdk also resolved my issue. Thanks!


updated travis yaml to test on python3

added python3 tests.

Coverage remained the same at 97.143% when pulling d398793398ef2cb7636e44d2dc58ca0a6291fac7 on python3 into d8bc3d8745c774e2241b598800da503fefc0de41 on master.

Coverage remained the same at 97.143% when pulling af1e8d9953953cf4c52b5cab58e61e48ec4a2c56 on python3 into d8bc3d8745c774e2241b598800da503fefc0de41 on master.

Coverage remained the same at 97.143% when pulling af26b6767e1661c64131d855e9c2ac61da6a57fb on python3 into d8bc3d8745c774e2241b598800da503fefc0de41 on master.

Coverage remained the same at 97.143% when pulling 7047dc6640a147e396e7f4f595283b196e67c9d4 on python3 into d8bc3d8745c774e2241b598800da503fefc0de41 on master.

Coverage remained the same at 97.143% when pulling 7047dc6640a147e396e7f4f595283b196e67c9d4 on python3 into d8bc3d8745c774e2241b598800da503fefc0de41 on master.


WIP : combine_evoked

that's a follow up on a conversation with @mshamalainen in Helsinki about contrasting evoked. Weighted contrasts should be an option and weights with nave is not always the good approach eg. in an oddball task.

This one is too far behind to rebase appropriately, closing for follow-up PR.

I just discovered this issue after a painful bug tracking.
I think we should warn the user if +/- are used and nave aren't equal, and provides clearer documentations in + / -.

where do you suggest we put this doc?
See
https://github.com/mne-tools/mne-python/issues/1934

Something like:
    def __sub__(self, evoked):
       """ Applies a subtraction of evoked datasets
            /!\ The subtraction is weighted by the number of epochs originally
           averaged (evoked.nave) """
        this_evoked = deepcopy(evoked)
        this_evoked.data *= -1.
        if self.nave != this_evoked.nave:
             warnings.warn('The evokeds do not have identical nave! '
                        'This will generate a weighted subtraction.')
        out = combine_evoked([self, this_evoked])
        if self.comment is None or this_evoked.comment is None:
            warnings.warn('evoked.comment expects a string but is None')
            out.comment = 'unknown'
        else:
            out.comment = self.comment + " - " + this_evoked.comment
        return out

I would not make it a warning which are displayed only once anyway. I'd use
the logger. I think it's valuable info for users.
On Tue, Apr 7, 2015 at 10:05 AM, J-R King<EMAIL_ADDRESS>wrote:

Something like:
def __sub__(self, evoked):
   """ Applies a subtraction of evoked datasets
        /!\ The subtraction is weighted by the number of epochs originally
       averaged (evoked.nave) """
    this_evoked = deepcopy(evoked)
    this_evoked.data *= -1.
    if self.nave != this_evoked.nave:
         warnings.warn('The evokeds do not have identical nave! '
                    'This will generate a weighted subtraction.')
    out = combine_evoked([self, this_evoked])
    if self.comment is None or this_evoked.comment is None:
        warnings.warn('evoked.comment expects a string but is None')
        out.comment = 'unknown'
    else:
        out.comment = self.comment + " - " + this_evoked.comment
    return out

—
Reply to this email directly or view it on GitHub
https://github.com/mne-tools/mne-python/pull/461#issuecomment-90445270.

I would not make it a warning which are displayed only once anyway. I'd use
the logger. I think it's valuable info for users."
Perhaps both?

On second thought, I agree with what 's been said above: the very large majority of naive users will think that '-' does a classic '-' operation, not a weighted one, so I vote for an important change there, even if this means changing a lot of code based on the assumption that the subtraction is weighted.
In the meantime, we could also explicit the code:
def __sub__():
...
out = combine_evoked([self, this_evoked], , weights='nave')

not using nave by defaults means breaking inverse modeling params by
default... that's my concern

I understand the issue indeed. Do you have an example where this would break? Could we think of a test that checks the space (sensor or source) of the subtraction?

I understand the issue indeed. Do you have an example where this would
break? Could we think of a test that checks the space (sensor or source) of
the subtraction?

basically the amplitudes of sLORETA / dSPM would be wrong and the
regularization using SNRs will be bad too. Now in practice it might not
change much on some data the output. It just opens the door to improper and
uncontrolled modeling.

Adding verbosity should do the job.
Log sth like:
Computing weighted average, nave ...

On 07 Apr 2015, at 11:22, Alexandre Gramfort<EMAIL_ADDRESS>wrote:

I understand the issue indeed. Do you have an example where this would
break? Could we think of a test that checks the space (sensor or source) of
the subtraction?

basically the amplitudes of sLORETA / dSPM would be wrong and the
regularization using SNRs will be bad too. Now in practice it might not
change much on some data the output. It just opens the door to improper and
uncontrolled modeling.
—
Reply to this email directly or view it on GitHub.

I think you overestimate users' reading ability / attention capacity...

so how do you make sure they properly use nave to control noise level?

But it's wrong not to do it for probably most of the cases and it corresponds to the traditional MNE behavior. We would break conistency with code that's been around for more than a decade.

I would vote for adding verbosity and maybe a contrast tutorial which points out differences for sensor space and source space handling and highlights the use of the function in certain cases.

deprecating the +/- operators will be expansive. It's used everywhere. Changing their meaning is not good either.

Yes, it's a classic conservative versus logic debate. Is it not possible to test for the type of subtraction one does (i.e. source or sensor space)?

Is it not possible to test for the type of subtraction one does (i.e. source or sensor space)?

this is not the question, it's always in sensor space when this operation is performed. It's about getting the sensor space contrast right for later source localization.

damned

So traditionally MNE has been a source localization software and I think the Python package followed these goals. In addition to the classical use case which has been exisiting for more than a decade and is reflected in our long term user population, we now have folks that do MEG and EEG in sensor space. We would need to deprecate the +/- for evoked and point everyone to using the function and educate about different use cases, depending on experimental protocol and the intention to source localize.

+1 for deprecation. This comes up repeatedly, and even with clear logging and/or documentation users can still easily make an error, which is unsafe. Forcing them to use a function should be safer.

Forcing them to use a function should be safer.

Thinking about it a bit this would probably be worth the pain on the long run. We need defaults that make sense to all of our users.

Thinking about it a bit this would probably be worth the pain on the long run. We need defaults that make sense to all of our users.

+1


net: ipv6_nbr: don't count incoming Neighbour Advertisements in sent stats

Fixes #54361.

@arbrauns A bug report is required in order for this PR to be merged during the feature freeze. Please file a bug report issue for the problem that this PR is trying to fix and link the issue to this PR.

I've opened the issue.
I also found another issue in the ND stats counting, I've added a second commit.


New release please... or is this project dead?

Hello,
can you release the new version, please?
It seems that the last release was out about seven years ago.
Thank you.

Hi,
if you met any issue when using this package, or if you have any feature request, as the author I'd like to help in fixing/implementing it.    I believe it's still in Quicklisp distribution, and I will try to make sure it's always buildable with updates of supported CL systems.
This project was created when many years ago I was working as a Linux system administrator, and I was having the needs to retrieve information from various servers and network devices through their SNMP interface.  But now I don't work on these things any more, and that's why I stopped contributing on this project.
Regards,
Chun Tian


Cannot convert IsoCode or language string into the actual type

Since the Language class and IsoCode classes are Rust classes, it would be nice to provide the from_str methods back so I can dynamically import iso codes.

This feature has been implemented in https://github.com/pemistahl/lingua-rs/issues/410 now and will be part of the next Rust release 1.7.0 and the next Python release 2.1.0.


Use urllib downloads instead of curl

urllib must manually decode responses to utf-8, which may cause errors on special characters, e.g. ñ, ö, ß . Can be ignored to pass tests, but may have unforeseen consequences.

urllib must manually decode responses to utf-8, which may cause errors on special characters, e.g. ñ, ö, ß . Can be ignored to pass tests, but may have unforeseen consequences.

Resolved by https://github.com/spack/spack/pull/21398

I have a WIP branch where all 'fetch_url' tests are passing on Windows --> https://github.com/BetsyMcPhail/spack/pull/new/urllib-windows
A few notes:

This almost certainly breaks Linux, there's no switching on OS.
For the most part, the changes deal with handling Windows filepaths
The changes are just good enough to get the tests passing, I have not thought about how robust they will be in other situations

FYI @loulawrence if you want to give the branch a try on your system

Thanks for testing @loulawrence, hopefully all of the tests should be passing on Windows now!

PR #21594 has the changes necessary for url_fetch test to pass on Windows.
To test, the Windows system must have 1. curl available and 2. #20412
Command to run tests: spack unit-test lib/spack/spack/test/url_fetch.py


Some suggested documentation updates based on my install experience.

I teased some of these out while trying to get the code to compile.

I did build an image, but only locally. I haven’t pushed it to Docker Hub.

On Sep 6, 2018, at 17:04, Dheepak Krishnamurthy<EMAIL_ADDRESS>wrote:
@kdheepak commented on this pull request.
@activeshadow you are everywhere! Thanks for the updates! They look great! Did you build a docker image as well? I've been meaning to build one so that users don't have to deal with installations to try out HELICS and possibly other HELICS compatible tools.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

I'll leave this open for now. I'll merge this along with some documentation changes that I have on my plate.

I am hoping we can get a 1.3.1 release at some point next week.  We should merge it before then.

Sounds good!

@kdheepak feel free to use the Dockerfile I included in the documentation as the basis for a Dockerfile in the project that can be automatically built by Docker Hub or Quay or whatever Docker image host you want to use.


Help wanted with dynamic kustomize resource configuration.

I am trying to set a dynamic configuration of manifests using Argocd, kustomize, helm and the lovely plugin. For example  want to dynamically change the host of an ingress resource.
This is a reference kustomization file that uses grafana helm chart:
# kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

helmCharts:
- name: grafana
  includeCRDs: true
  releaseName: grafana
  version: 7.3.0
  repo: https://grafana.github.io/helm-charts
  valuesFile: values.yaml
  # namespace: demo

I would like to dynamically configure the ingress host name through the argo application or application set. I saw on the docs that I can use LOVELY_KUSTOMIZE_PATCH env variable to set kustomization patches. My issue is that I cant understand how to define the target resource (eg. kind :Ingress, name: grafana-ingress).
An example Argocd Application is the following
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: kustomize-grafana-helm
  namespace: argocd
  finalizers:
    - resources-finalizer.argocd.argoproj.io
spec:
  destination:
    namespace: demo
    server: https://kubernetes.default.svc
  project: default
  source:
    path: resources/kustomize/grafana # <- points to the above kustomization file
    repoURL: <eg-repo>
    targetRevision: HEAD
    plugin:
      name: lovely-vault-plugin
      env:
        - name: LOVELY_KUSTOMIZE_PATCH
          value: '[{"op": "replace", "path": "/spec/rules/0/host", "value": "grafana-test.demo.com"}]'

The problem with the above application is that I don't see how can I reference the Ingress resource and naturally argocd throws an error of :
Failed to load target state: failed to generate manifest for source 1 of 1: rpc error: code = Unknown desc = Manifest generation error (cached): plugin sidecar failed. error generating manifests in cmp: rpc error: code = Unknown desc = error generating manifests: `argocd-lovely-plugin` failed exit status 1: 2024/02/24 15:30:22 replace operation does not apply: doc is missing path: /spec/rules/0/host: missing value

I might not be in the right path to do this, so any suggestions would be much appreciated.
Thanks in advance.

LOVELY_KUSTOMIZE_PATCH is patching the kustomization.yaml file.
It can do what you want, but you need to write the correct patch. Start by writing what you need into a kustomization.yaml without the patching happening. This will need to identify the object needed to be patched (an ingress). You can then convert this into an appropriate patch.
What you have written there is patching kustomization.yaml as though it were the ingress object.


specific errors are not available when validation fails

This issue is explained upstream: https://github.com/Automattic/mongoose/issues/4113
@joshkopecek Would you mind testing a code update that throws a different type of error besides CastError to see if that exposes the raw error messages?
Maybe a MongooseError ?

Pushed a change which throws mongoose.Error for each problem. It now tests for specific strings in the error reasons. We'll wait for CastError to be modified in the Mongoose repo then we can use a proper CastError, but this will do for the meantime.


[BUG] Using 66% of a CPU when idle

Environment
Self-Hosted (Docker)
System
Ubuntu 22.0.4
Version
2.1.1
Describe the problem
With no browser windows taking to dashy, the "node server" process is using 66% of a CPU core at minimum. Presumably this is background processing such as the docker healthcheck. There's no rebuild running. It's not going to zero at any point. With any browser window open, it goes up to 125% and stays there or above. However, this all seems very excessive for what's logically happening.
This was a straight-forward docker install using Portainer on a Raspberry Pi 4 running Ubuntu 22.04. This is enough CPU use to trigger the cooling fan on this box, hence why I've noticed it.
Additional info
partial output of pstree -a -p:
  ├─containerd-shim,654085 -namespace moby -id fcfa6e2acda69da0fbd4ca458e75243f3b7b4563f64322e231e6566505c71980 -address /run/containerd/containerd.sock
  │   ├─tini,654108 -- yarn build-and-start
  │   │   └─node,654156 /opt/yarn-v1.22.15/bin/yarn.js build-and-start
  │   │       ├─node,654179 /app/node_modules/.bin/npm-run-all --parallel build-watch start
  │   │       │   ├─node,654190 /opt/yarn-v1.22.15/bin/yarn.js run build-watch
  │   │       │   │   ├─node,654212 /app/node_modules/.bin/vue-cli-service build --watch --mode production
  │   │       │   │   │   ├─{node},654214
  │   │       │   │   │   ├─{node},654215
  │   │       │   │   │   ├─{node},654216
  │   │       │   │   │   ├─{node},654217
  │   │       │   │   │   ├─{node},654218
  │   │       │   │   │   ├─{node},654224
  │   │       │   │   │   ├─{node},654232
  │   │       │   │   │   ├─{node},654233
  │   │       │   │   │   ├─{node},654234
  │   │       │   │   │   └─{node},654235
  │   │       │   │   ├─{node},654192
  │   │       │   │   ├─{node},654193
  │   │       │   │   ├─{node},654194
  │   │       │   │   ├─{node},654195
  │   │       │   │   ├─{node},654196
  │   │       │   │   ├─{node},654202
  │   │       │   │   ├─{node},654204
  │   │       │   │   ├─{node},654205
  │   │       │   │   ├─{node},654206
  │   │       │   │   └─{node},654207
  │   │       │   ├─node,654191 /opt/yarn-v1.22.15/bin/yarn.js run start
  │   │       │   │   ├─node,654213 server
  │   │       │   │   │   ├─{node},654219
  │   │       │   │   │   ├─{node},654220
  │   │       │   │   │   ├─{node},654221
  │   │       │   │   │   ├─{node},654222
  │   │       │   │   │   ├─{node},654223
  │   │       │   │   │   ├─{node},654225
  │   │       │   │   │   ├─{node},654226
  │   │       │   │   │   ├─{node},654227
  │   │       │   │   │   ├─{node},654228
  │   │       │   │   │   └─{node},654229
  │   │       │   │   ├─{node},654197
  │   │       │   │   ├─{node},654198
  │   │       │   │   ├─{node},654199
  │   │       │   │   ├─{node},654200
  │   │       │   │   ├─{node},654201
  │   │       │   │   ├─{node},654203
  │   │       │   │   ├─{node},654208
  │   │       │   │   ├─{node},654209
  │   │       │   │   ├─{node},654210
  │   │       │   │   └─{node},654211
  │   │       │   ├─{node},654180
  │   │       │   ├─{node},654181
  │   │       │   ├─{node},654182
  │   │       │   ├─{node},654183
  │   │       │   ├─{node},654184
  │   │       │   ├─{node},654185
  │   │       │   ├─{node},654186
  │   │       │   ├─{node},654187
  │   │       │   ├─{node},654188
  │   │       │   └─{node},654189
  │   │       ├─{node},654167
  │   │       ├─{node},654168
  │   │       ├─{node},654169
  │   │       ├─{node},654170
  │   │       ├─{node},654171
  │   │       ├─{node},654172
  │   │       ├─{node},654175
  │   │       ├─{node},654176
  │   │       ├─{node},654177
  │   │       └─{node},654178
  │   ├─{containerd-shim},654086
  │   ├─{containerd-shim},654087
  │   ├─{containerd-shim},654088
  │   ├─{containerd-shim},654089
  │   ├─{containerd-shim},654090
  │   ├─{containerd-shim},654091
  │   ├─{containerd-shim},654092
  │   ├─{containerd-shim},654093
  │   ├─{containerd-shim},654094
  │   ├─{containerd-shim},654375
  │   └─{containerd-shim},657704

Please tick the boxes

[X] You have explained the issue clearly, and included all relevant info
[X] You are using a supported version of Dashy
[X] You've checked that this issue hasn't already been raised
[X] You've checked the docs and troubleshooting guide
[X] You agree to the code of conduct

I had the same problem, so I built a new docker image from master, and that sort of seems to have fixed it.
However, I also noticed that when I had the status check enabled, it would interfere with my pi-alert container (Note that pi-alert is VERY buggy). And both would consume a lot of cpu.

AFAIK, nothing has changed. Sounds like Lissy93 is swamped.

No stale

Still relevant. @Lissy93, any chance you'll be able to look at this? If not, perhaps you have some hints as to how you'd diagnose it?

Hey,
Really sorry for the delay in replying, (I have so many unread GH notifications at the moment, that it's hard to spot the important ones! 😬)
This definitely doesn't sound good at all, weirdly I'm running a very similar setup (Pi 4, default Docker, etc) and no issues.
Looking at those logs, I'm wondering if it's something gone wrong with the healthcheck. If you disable that, do you still see issues?

@ohmantics, do you have an example config file to reproduce this issue? Also, does it still happen if you change the container command to run yarn start instead?
@Lissy93, is there any reason for the ARM-based Docker images to have yarn build-and-serve as default CMD, while the Docker image for x86 is just yarn start?

No stale

It is still relevant, no stale

Same issue with docker compose, Dashy v2.1.1 on Debian 11 in a raspberry pi4. I was searching what was using 100% of CPU constantly until I stopped the stack. It does't go down when Dashy isn't accessed.

Same issue with docker compose, Dashy v2.1.1 on Debian 11 in a raspberry pi4. I was searching what was using 100% of CPU constantly until I stopped the stack. It does't go down when Dashy isn't accessed.

Turning off Enable status check in page settings and restarting the container fixed it. I think I'm going to use uptime kuma for now

Still relevant.

Still relevant

I also run Dashy with many other things on my Rpi 3B+ with 1GB RAM and Dashy is consuming very few MB and barely any CPU even when starting up.
Can you confirm if you are running x64 or x32 ?

Auto-closing bots are a project management smell.

Auto-closing bots are a project management smell.

I haven't gotten any confirmation that it is still relevant, so the bot did everything correct.

Still relevant

Then please answer the question I asked above.
Instead of ranting on the bot!

In my case x64

In my case x64

Hi
Could you test this with the :latest tag by repulling and rebuilding the app?
Keep in mind V3 has breaking changes:
https://github.com/Lissy93/dashy/releases/tag/3.0.0

Not seeing the original issue now. It did take several re-pulls for the build to complete without errors; not sure why since it says "possible error, check logs" and there are no logs to see.

Not seeing the original issue now. It did take several re-pulls for the build to complete without errors; not sure why since it says "possible error, check logs" and there are no logs to see.

Awesome!
Then I think we can close this.
If you have changed the bind mounts and the port those errors shouldn't appear, yeah.
Did you wait about 3 minutes longer and refreshed the page, after the error appears?
The first build takes a bit longer and on a raspberry pi it takes even longer, so it may exceed the expected time and then the "possible error" popps up.


New release version for "0.3.0"?

Are there any plans to upgrade the "latest" release from version "0.2.0" to "0.3.0" or something else?
I see the Readme mentions using "operatehappy/packer-github-actions@master"  to use "master" instead of a "release".  For stability, it would be better to have a designated release to leverage against.

Hey @senterfd-jrg,
thank you for creating this issue, We'll have an updated release tag available soon!

Hi @ksatirli,
Do you have a news about a new tag based the current version in master?
Thanks

Hi @ksatirli Since it's been a while, might we be able to get a release please?
Thanks

A new version with Packer 1.7.8 was just released.
Thanks for bringing this back from the dead, @yachub