gemini query

I've got a 12g gemini db from cohort calling. Doing some basic queries didn't seem to work. I checked the  the variant table and a large number of columns such as gene/lof are NONE. The info field seems to have all the annotation from the loaded vcf including gene/lof. Is this expected?
I can move the issue to gemini but I wanted to check here first.
gemini region --gene MYH7 batch1-freebayes.db
# nothing

gemini lof_sieve batch1-freebayes.db
chrom	start	end	ref	alt	highest_impact	aa_change	var_trans_pos	trans_aa_length	var_trans_pct	sample	genotype	gene	transcript	trans_type
# empty

gemini stats --gts-by-sample batch1-freebayes.db
sample	num_hom_ref	num_het	num_hom_alt	num_unknown	total
# empty

Hi Daryl,
That’s definitely not expected, generally that happens when the VEP or snpEff annotation isn’t correct. In the log file, what did the GEMINI loading command look like? Can you paste a line from the VCF file?
Thanks!

On Feb 22, 2015, at 8:41 PM, Daryl Waggott<EMAIL_ADDRESS>wrote:
I've got a 12g gemini db from cohort calling. Doing some basic queries didn't seem to work. I checked the the variant table and a large number of columns such as gene/lof are NONE. The info field seems to have all the annotation from the loaded vcf including gene/lof. Is this expected?
I can move the issue to gemini but I wanted to check here first.
gemini region --gene MYH7 batch1-freebayes.db
nothing
gemini lof_sieve batch1-freebayes.db
chrom   start   end ref alt highest_impact  aa_change   var_trans_pos   trans_aa_length var_trans_pct   sample  genotype    gene    transcript  trans_type
empty
gemini stats --gts-by-sample batch1-freebayes.db
sample  num_hom_ref num_het num_hom_alt num_unknown total
empty
—
Reply to this email directly or view it on GitHub https://github.com/chapmanb/bcbio-nextgen/issues/773.

Hey,
From the  bcbio-nextgen-commands.log in the final output, these are the gemini commands. I'll try and run them manually to see what happens.
Do you have any gemini test queries to figure out if it loaded correctly?
015-02-17T18:35Z] sh-5-16.local: gatk-framework -Xms500m -Xmx4000m -XX:+UseSerialGC -U LENIENT_VCF_PROCESSING --read_filter BadCigar --read_filter NotPrimaryAlignment -T SelectVariants -R /share/PI/euan/apps/bcbio/genomes/Hsapiens/GRCh37/seq/GRCh37.fa --variant /scratch/PI/euan/projects/athletes/bcbio/gemini/batch1-freebayes.vcf.gz --out /scratch/PI/euan/projects/athletes/bcbio/gemini/tx/tmpjVMQDj/batch1-freebayes-biallelic.vcf.gz -restrictAllelesTo BIALLELIC
[2015-02-17T18:37Z] sh-5-17.local: /share/PI/euan/apps/bin/tabix -f -p vcf /scratch/PI/euan/projects/athletes/bcbio/freebayes/tx/tmp7fdSZl/3_57-effects-filter.vcf.gz
[2015-02-17T18:35Z] sh-5-16.local: gatk-framework -Xms500m -Xmx4000m -XX:+UseSerialGC -U LENIENT_VCF_PROCESSING --read_filter BadCigar --read_filter NotPrimaryAlignment -T SelectVariants -R /share/PI/euan/apps/bcbio/genomes/Hsapiens/GRCh37/seq/GRCh37.fa --variant /scratch/PI/euan/projects/athletes/bcbio/gemini/batch1-freebayes.vcf.gz --out /scratch/PI/euan/projects/athletes/bcbio/gemini/tx/tmpjVMQDj/batch1-freebayes-biallelic.vcf.gz -restrictAllelesTo BIALLELIC
[2015-02-17T18:37Z] sh-5-17.local: /share/PI/euan/apps/bin/tabix -f -p vcf /scratch/PI/euan/projects/athletes/bcbio/freebayes/tx/tmp7fdSZl/3_57-effects-filter.vcf.gz
[2015-02-17T18:37Z] sh-5-17.local: gatk-framework -Xms500m -Xmx4000m -XX:+UseSerialGC -U LENIENT_VCF_PROCESSING --read_filter BadCigar --read_filter NotPrimaryAlignment -T SelectVariants -R /share/PI/euan/apps/bcbio/genomes/Hsapiens/GRCh37/seq/GRCh37.fa --variant /scratch/PI/euan/projects/athletes/bcbio/gemini/batch1-freebayes.vcf.gz --out /scratch/PI/euan/projects/athletes/bcbio/gemini/tx/tmp1hxEwz/batch1-freebayes-biallelic.vcf.gz -restrictAllelesTo BIALLELIC
[2015-02-17T19:05Z] sh-5-16.local: gatk-framework -Xms500m -Xmx4000m -XX:+UseSerialGC -XX:+UseSerialGC -U LENIENT_VCF_PROCESSING --read_filter BadCigar --read_filter NotPrimaryAlignment -T SelectVariants -R /share/PI/euan/apps/bcbio/genomes/Hsapiens/GRCh37/seq/GRCh37.fa --variant /scratch/PI/euan/projects/athletes/bcbio/gemini/batch1-freebayes.vcf.gz --out /scratch/PI/euan/projects/athletes/bcbio/gemini/tx/tmpRrm1GM/batch1-freebayes-multiallelic.vcf.gz -restrictAllelesTo MULTIALLELIC
[2015-02-17T19:07Z] sh-5-17.local: gatk-framework -Xms500m -Xmx4000m -XX:+UseSerialGC -XX:+UseSerialGC -U LENIENT_VCF_PROCESSING --read_filter BadCigar --read_filter NotPrimaryAlignment -T SelectVariants -R /share/PI/euan/apps/bcbio/genomes/Hsapiens/GRCh37/seq/GRCh37.fa --variant /scratch/PI/euan/projects/athletes/bcbio/gemini/batch1-freebayes.vcf.gz --out /scratch/PI/euan/projects/athletes/bcbio/gemini/tx/tmpa7W2wl/batch1-freebayes-multiallelic.vcf.gz -restrictAllelesTo MULTIALLELIC
[2015-02-17T19:18Z] sh-5-16.local: vt decompose /scratch/PI/euan/projects/athletes/bcbio/gemini/batch1-freebayes-multiallelic.vcf.gz | awk '{ gsub("./-65", "./."); print $0 }'| bgzip -c > /scratch/PI/euan/projects/athletes/bcbio/gemini/tx/tmpBvXbLU/batch1-freebayes-multiallelic-decompose.vcf.gz
[2015-02-17T19:19Z] sh-5-16.local: /share/PI/euan/apps/bin/tabix -f -p vcf /scratch/PI/euan/projects/athletes/bcbio/gemini/tx/tmp_YzChB/batch1-freebayes-multiallelic-decompose.vcf.gz
[2015-02-17T19:19Z] sh-5-16.local: /share/PI/euan/apps/bin/snpEff -Xms750m -Xmx6g eff -dataDir /share/PI/euan/apps/bcbio/genomes/Hsapiens/GRCh37/snpeff -formatEff -noLog -i vcf -o vcf GRCh37.75 /scratch/PI/euan/projects/athletes/bcbio/gemini/batch1-freebayes-multiallelic-decompose.vcf.gz | /share/PI/euan/apps/bin/bgzip -c > /scratch/PI/euan/projects/athletes/bcbio/gemini/tx/tmpuP1h0x/batch1-freebayes-multiallelic-decompose-effects.vcf.gz
[2015-02-17T19:20Z] sh-5-17.local: /share/PI/euan/apps/bin/snpEff -Xms750m -Xmx6g eff -dataDir /share/PI/euan/apps/bcbio/genomes/Hsapiens/GRCh37/snpeff -formatEff -noLog -i vcf -o vcf GRCh37.75 /scratch/PI/euan/projects/athletes/bcbio/gemini/batch1-freebayes-multiallelic-decompose.vcf.gz | /share/PI/euan/apps/bin/bgzip -c > /scratch/PI/euan/projects/athletes/bcbio/gemini/tx/tmpM5KrEj/batch1-freebayes-multiallelic-decompose-effects.vcf.gz
[2015-02-17T19:29Z] sh-5-16.local: /share/PI/euan/apps/bin/tabix -f -p vcf /scratch/PI/euan/projects/athletes/bcbio/gemini/tx/tmpbsvddn/batch1-freebayes-multiallelic-decompose-effects.vcf.gz
[2015-02-17T19:29Z] sh-5-16.local: /share/PI/euan/apps/bin/gatk-framework -Xms500m -Xmx52417m -XX:+UseSerialGC -XX:+UseSerialGC -XX:+UseSerialGC -U LENIENT_VCF_PROCESSING --read_filter BadCigar --read_filter NotPrimaryAlignment -T CombineVariants -R /share/PI/euan/apps/bcbio/genomes/Hsapiens/GRCh37/seq/GRCh37.fa --out /scratch/PI/euan/projects/athletes/bcbio/gemini/tx/tmpavfrWb/batch1-freebayes-nomultiallelic.vcf.gz --variant:v0 /scratch/PI/euan/projects/athletes/bcbio/gemini/batch1-freebayes-biallelic.vcf.gz --variant:v1 /scratch/PI/euan/projects/athletes/bcbio/gemini/batch1-freebayes-multiallelic-decompose-effects.vcf.gz --rod_priority_list v0,v1 --suppressCommandLineHeader --setKey null -L /scratch/PI/euan/projects/athletes/bcbio/bedprep/134_2015-01-22_project_n143-fb-sort-callable-callableblocks.bed --interval_set_rule INTERSECTION -nt 4
[2015-02-17T19:31Z] sh-5-17.local: /share/PI/euan/apps/bin/tabix -f -p vcf /scratch/PI/euan/projects/athletes/bcbio/gemini/tx/tmppGablU/batch1-freebayes-multiallelic-decompose-effects.vcf.gz
[2015-02-17T19:31Z] sh-5-17.local: /share/PI/euan/apps/bin/tabix -f -p vcf /scratch/PI/euan/projects/athletes/bcbio/gemini/tx/tmptSmbrT/batch1-freebayes-multiallelic-decompose-effects.vcf.gz
[2015-02-17T19:31Z] sh-5-17.local: /share/PI/euan/apps/bin/gatk-framework -Xms500m -Xmx52417m -XX:+UseSerialGC -XX:+UseSerialGC -XX:+UseSerialGC -U LENIENT_VCF_PROCESSING --read_filter BadCigar --read_filter NotPrimaryAlignment -T CombineVariants -R /share/PI/euan/apps/bcbio/genomes/Hsapiens/GRCh37/seq/GRCh37.fa --out /scratch/PI/euan/projects/athletes/bcbio/gemini/tx/tmpCyAClY/batch1-freebayes-nomultiallelic.vcf.gz --variant:v0 /scratch/PI/euan/projects/athletes/bcbio/gemini/batch1-freebayes-biallelic.vcf.gz --variant:v1 /scratch/PI/euan/projects/athletes/bcbio/gemini/batch1-freebayes-multiallelic-decompose-effects.vcf.gz --rod_priority_list v0,v1 --suppressCommandLineHeader --setKey null -L /scratch/PI/euan/projects/athletes/bcbio/bedprep/134_2015-01-22_project_n143-fb-sort-callable-callableblocks.bed --interval_set_rule INTERSECTION -nt 4
[2015-02-17T20:47Z] sh-5-16.local: /share/PI/euan/apps/bin/gemini load  --passonly --skip-gerp-bp  -v /scratch/PI/euan/projects/athletes/bcbio/gemini/batch1-freebayes-nomultiallelic.vcf.gz -t snpEff --cores 16 /scratch/PI/euan/projects/athletes/bcbio/gemini/tx/tmp1HyQZi/batch1-freebayes.db
[2015-02-17T20:49Z] sh-5-17.local: /share/PI/euan/apps/bin/gemini load  --passonly --skip-gerp-bp  -v /scratch/PI/euan/projects/athletes/bcbio/gemini/batch1-freebayes-nomultiallelic.vcf.gz -t snpEff --cores 16 /scratch/PI/euan/projects/athletes/bcbio/gemini/tx/tmpozR7h1/batch1-freebayes.db

Hi Daryl,
How come each line is repeated twice? Was there more than one version of the pipeline running simultaneously?
Best,
Rory

On Feb 22, 2015, at 11:26 PM, Daryl Waggott<EMAIL_ADDRESS>wrote:
Hey,
From the bcbio-nextgen-commands.log in the final output, these are the gemini commands. I'll try and run them manually to see what happens.
Do you have any gemini test queries to figure out if it loaded correctly?
015-02-17T18:35Z] sh-5-16.local: gatk-framework -Xms500m -Xmx4000m -XX:+UseSerialGC -U LENIENT_VCF_PROCESSING --read_filter BadCigar --read_filter NotPrimaryAlignment -T SelectVariants -R /share/PI/euan/apps/bcbio/genomes/Hsapiens/GRCh37/seq/GRCh37.fa --variant /scratch/PI/euan/projects/athletes/bcbio/gemini/batch1-freebayes.vcf.gz --out /scratch/PI/euan/projects/athletes/bcbio/gemini/tx/tmpjVMQDj/batch1-freebayes-biallelic.vcf.gz -restrictAllelesTo BIALLELIC
[2015-02-17T18:37Z] sh-5-17.local: /share/PI/euan/apps/bin/tabix -f -p vcf /scratch/PI/euan/projects/athletes/bcbio/freebayes/tx/tmp7fdSZl/3_57-effects-filter.vcf.gz
[2015-02-17T18:35Z] sh-5-16.local: gatk-framework -Xms500m -Xmx4000m -XX:+UseSerialGC -U LENIENT_VCF_PROCESSING --read_filter BadCigar --read_filter NotPrimaryAlignment -T SelectVariants -R /share/PI/euan/apps/bcbio/genomes/Hsapiens/GRCh37/seq/GRCh37.fa --variant /scratch/PI/euan/projects/athletes/bcbio/gemini/batch1-freebayes.vcf.gz --out /scratch/PI/euan/projects/athletes/bcbio/gemini/tx/tmpjVMQDj/batch1-freebayes-biallelic.vcf.gz -restrictAllelesTo BIALLELIC
[2015-02-17T18:37Z] sh-5-17.local: /share/PI/euan/apps/bin/tabix -f -p vcf /scratch/PI/euan/projects/athletes/bcbio/freebayes/tx/tmp7fdSZl/3_57-effects-filter.vcf.gz
[2015-02-17T18:37Z] sh-5-17.local: gatk-framework -Xms500m -Xmx4000m -XX:+UseSerialGC -U LENIENT_VCF_PROCESSING --read_filter BadCigar --read_filter NotPrimaryAlignment -T SelectVariants -R /share/PI/euan/apps/bcbio/genomes/Hsapiens/GRCh37/seq/GRCh37.fa --variant /scratch/PI/euan/projects/athletes/bcbio/gemini/batch1-freebayes.vcf.gz --out /scratch/PI/euan/projects/athletes/bcbio/gemini/tx/tmp1hxEwz/batch1-freebayes-biallelic.vcf.gz -restrictAllelesTo BIALLELIC
[2015-02-17T19:05Z] sh-5-16.local: gatk-framework -Xms500m -Xmx4000m -XX:+UseSerialGC -XX:+UseSerialGC -U LENIENT_VCF_PROCESSING --read_filter BadCigar --read_filter NotPrimaryAlignment -T SelectVariants -R /share/PI/euan/apps/bcbio/genomes/Hsapiens/GRCh37/seq/GRCh37.fa --variant /scratch/PI/euan/projects/athletes/bcbio/gemini/batch1-freebayes.vcf.gz --out /scratch/PI/euan/projects/athletes/bcbio/gemini/tx/tmpRrm1GM/batch1-freebayes-multiallelic.vcf.gz -restrictAllelesTo MULTIALLELIC
[2015-02-17T19:07Z] sh-5-17.local: gatk-framework -Xms500m -Xmx4000m -XX:+UseSerialGC -XX:+UseSerialGC -U LENIENT_VCF_PROCESSING --read_filter BadCigar --read_filter NotPrimaryAlignment -T SelectVariants -R /share/PI/euan/apps/bcbio/genomes/Hsapiens/GRCh37/seq/GRCh37.fa --variant /scratch/PI/euan/projects/athletes/bcbio/gemini/batch1-freebayes.vcf.gz --out /scratch/PI/euan/projects/athletes/bcbio/gemini/tx/tmpa7W2wl/batch1-freebayes-multiallelic.vcf.gz -restrictAllelesTo MULTIALLELIC
[2015-02-17T19:18Z] sh-5-16.local: vt decompose /scratch/PI/euan/projects/athletes/bcbio/gemini/batch1-freebayes-multiallelic.vcf.gz | awk '{ gsub("./-65", "./."); print $0 }'| bgzip -c > /scratch/PI/euan/projects/athletes/bcbio/gemini/tx/tmpBvXbLU/batch1-freebayes-multiallelic-decompose.vcf.gz
[2015-02-17T19:19Z] sh-5-16.local: /share/PI/euan/apps/bin/tabix -f -p vcf /scratch/PI/euan/projects/athletes/bcbio/gemini/tx/tmp_YzChB/batch1-freebayes-multiallelic-decompose.vcf.gz
[2015-02-17T19:19Z] sh-5-16.local: /share/PI/euan/apps/bin/snpEff -Xms750m -Xmx6g eff -dataDir /share/PI/euan/apps/bcbio/genomes/Hsapiens/GRCh37/snpeff -formatEff -noLog -i vcf -o vcf GRCh37.75 /scratch/PI/euan/projects/athletes/bcbio/gemini/batch1-freebayes-multiallelic-decompose.vcf.gz | /share/PI/euan/apps/bin/bgzip -c > /scratch/PI/euan/projects/athletes/bcbio/gemini/tx/tmpuP1h0x/batch1-freebayes-multiallelic-decompose-effects.vcf.gz
[2015-02-17T19:20Z] sh-5-17.local: /share/PI/euan/apps/bin/snpEff -Xms750m -Xmx6g eff -dataDir /share/PI/euan/apps/bcbio/genomes/Hsapiens/GRCh37/snpeff -formatEff -noLog -i vcf -o vcf GRCh37.75 /scratch/PI/euan/projects/athletes/bcbio/gemini/batch1-freebayes-multiallelic-decompose.vcf.gz | /share/PI/euan/apps/bin/bgzip -c > /scratch/PI/euan/projects/athletes/bcbio/gemini/tx/tmpM5KrEj/batch1-freebayes-multiallelic-decompose-effects.vcf.gz
[2015-02-17T19:29Z] sh-5-16.local: /share/PI/euan/apps/bin/tabix -f -p vcf /scratch/PI/euan/projects/athletes/bcbio/gemini/tx/tmpbsvddn/batch1-freebayes-multiallelic-decompose-effects.vcf.gz
[2015-02-17T19:29Z] sh-5-16.local: /share/PI/euan/apps/bin/gatk-framework -Xms500m -Xmx52417m -XX:+UseSerialGC -XX:+UseSerialGC -XX:+UseSerialGC -U LENIENT_VCF_PROCESSING --read_filter BadCigar --read_filter NotPrimaryAlignment -T CombineVariants -R /share/PI/euan/apps/bcbio/genomes/Hsapiens/GRCh37/seq/GRCh37.fa --out /scratch/PI/euan/projects/athletes/bcbio/gemini/tx/tmpavfrWb/batch1-freebayes-nomultiallelic.vcf.gz --variant:v0 /scratch/PI/euan/projects/athletes/bcbio/gemini/batch1-freebayes-biallelic.vcf.gz --variant:v1 /scratch/PI/euan/projects/athletes/bcbio/gemini/batch1-freebayes-multiallelic-decompose-effects.vcf.gz --rod_priority_list v0,v1 --suppressCommandLineHeader --setKey null -L /scratch/PI/euan/projects/athletes/bcbio/bedprep/134_2015-01-22_project_n143-fb-sort-callable-callableblocks.bed --interval_set_rule INTERSECTION -nt 4
[2015-02-17T19:31Z] sh-5-17.local: /share/PI/euan/apps/bin/tabix -f -p vcf /scratch/PI/euan/projects/athletes/bcbio/gemini/tx/tmppGablU/batch1-freebayes-multiallelic-decompose-effects.vcf.gz
[2015-02-17T19:31Z] sh-5-17.local: /share/PI/euan/apps/bin/tabix -f -p vcf /scratch/PI/euan/projects/athletes/bcbio/gemini/tx/tmptSmbrT/batch1-freebayes-multiallelic-decompose-effects.vcf.gz
[2015-02-17T19:31Z] sh-5-17.local: /share/PI/euan/apps/bin/gatk-framework -Xms500m -Xmx52417m -XX:+UseSerialGC -XX:+UseSerialGC -XX:+UseSerialGC -U LENIENT_VCF_PROCESSING --read_filter BadCigar --read_filter NotPrimaryAlignment -T CombineVariants -R /share/PI/euan/apps/bcbio/genomes/Hsapiens/GRCh37/seq/GRCh37.fa --out /scratch/PI/euan/projects/athletes/bcbio/gemini/tx/tmpCyAClY/batch1-freebayes-nomultiallelic.vcf.gz --variant:v0 /scratch/PI/euan/projects/athletes/bcbio/gemini/batch1-freebayes-biallelic.vcf.gz --variant:v1 /scratch/PI/euan/projects/athletes/bcbio/gemini/batch1-freebayes-multiallelic-decompose-effects.vcf.gz --rod_priority_list v0,v1 --suppressCommandLineHeader --setKey null -L /scratch/PI/euan/projects/athletes/bcbio/bedprep/134_2015-01-22_project_n143-fb-sort-callable-callableblocks.bed --interval_set_rule INTERSECTION -nt 4
[2015-02-17T20:47Z] sh-5-16.local: /share/PI/euan/apps/bin/gemini load  --passonly --skip-gerp-bp  -v /scratch/PI/euan/projects/athletes/bcbio/gemini/batch1-freebayes-nomultiallelic.vcf.gz -t snpEff --cores 16 /scratch/PI/euan/projects/athletes/bcbio/gemini/tx/tmp1HyQZi/batch1-freebayes.db
[2015-02-17T20:49Z] sh-5-17.local: /share/PI/euan/apps/bin/gemini load  --passonly --skip-gerp-bp  -v /scratch/PI/euan/projects/athletes/bcbio/gemini/batch1-freebayes-nomultiallelic.vcf.gz -t snpEff --cores 16 /scratch/PI/euan/projects/athletes/bcbio/gemini/tx/tmpozR7h1/batch1-freebayes.db
—
Reply to this email directly or view it on GitHub https://github.com/chapmanb/bcbio-nextgen/issues/773#issuecomment-75488050.

this is from the log:
015-02-17T18:37Z] sh-5-26.local: Timing: ensemble calling
[2015-02-17T18:37Z] sh-5-26.local: Timing: validation summary
[2015-02-17T18:37Z] sh-5-26.local: Timing: structural variation
[2015-02-17T18:37Z] sh-5-26.local: ipython: detect_sv
[2015-02-17T18:37Z] sh-5-26.local: Timing: population database
[2015-02-17T18:37Z] sh-5-26.local: ipython: prep_gemini_db
[2015-02-17T22:09Z] sh-5-16.local: Uncaught exception occurred
Traceback (most recent call last):
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/bcbio/provenance/do.py", line 21, in run
    _do_run(cmd, checks, log_stdout)
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/bcbio/provenance/do.py", line 95, in _do_run
    raise subprocess.CalledProcessError(exitcode, error_msg)
CalledProcessError: Command 'set -o pipefail; /share/PI/euan/apps/bin/gemini load  --passonly --skip-gerp-bp  -v /scratch/PI/euan/projects/athletes/bcbio/gemini/batch1-freebayes-nomultiallelic.vcf.gz -t snpEff --cores 16 /scratch/PI/euan/projects/athletes/bcbio/gemini/tx/tmp1HyQZi/batch1-freebayes.db
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_merge_chunks.py", line 190, in merge_chunks
    merge_db_chunks(args)
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_merge_chunks.py", line 176, in merge_db_chunks
    append_sample_genotype_counts(main_curr, db)
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_merge_chunks.py", line 40, in append_sample_genotype_counts
    main_curr.execute(cmd)
sqlite3.OperationalError: no such table: toMerge.sample_genotype_counts
Traceback (most recent call last):
  File "/share/PI/euan/apps/bin/gemini", line 6, in <module>
    gemini.gemini_main.main()
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 1105, in main
    args.func(parser, args)
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 244, in mergechunk_fn
    gemini_merge_chunks.merge_chunks(parser, args)
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_merge_chunks.py", line 190, in merge_chunks
    merge_db_chunks(args)
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_merge_chunks.py", line 176, in merge_db_chunks
    append_sample_genotype_counts(main_curr, db)
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_merge_chunks.py", line 40, in append_sample_genotype_counts
    main_curr.execute(cmd)
sqlite3.OperationalError: no such table: toMerge.sample_genotype_counts
Traceback (most recent call last):
  File "/share/PI/euan/apps/bin/gemini", line 6, in <module>
    gemini.gemini_main.main()
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 1105, in main
    args.func(parser, args)
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 244, in mergechunk_fn
    gemini_merge_chunks.merge_chunks(parser, args)
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_merge_chunks.py", line 190, in merge_chunks
    merge_db_chunks(args)
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_merge_chunks.py", line 176, in merge_db_chunks
    append_sample_genotype_counts(main_curr, db)
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_merge_chunks.py", line 40, in append_sample_genotype_counts
    main_curr.execute(cmd)
sqlite3.OperationalError: no such table: toMerge.sample_genotype_counts
Traceback (most recent call last):
  File "/share/PI/euan/apps/bin/gemini", line 6, in <module>
    gemini.gemini_main.main()
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 1105, in main
    args.func(parser, args)
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 244, in mergechunk_fn
    gemini_merge_chunks.merge_chunks(parser, args)
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_merge_chunks.py", line 190, in merge_chunks
    merge_db_chunks(args)
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_merge_chunks.py", line 176, in merge_db_chunks
    append_sample_genotype_counts(main_curr, db)
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_merge_chunks.py", line 40, in append_sample_genotype_counts
    main_curr.execute(cmd)
sqlite3.OperationalError: no such table: toMerge.sample_genotype_counts
Traceback (most recent call last):
  File "/share/PI/euan/apps/bin/gemini", line 6, in <module>
    gemini.gemini_main.main()
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 1105, in main
    args.func(parser, args)
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 244, in mergechunk_fn
    gemini_merge_chunks.merge_chunks(parser, args)
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_merge_chunks.py", line 190, in merge_chunks
    merge_db_chunks(args)
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_merge_chunks.py", line 176, in merge_db_chunks
    append_sample_genotype_counts(main_curr, db)
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_merge_chunks.py", line 40, in append_sample_genotype_counts
    main_curr.execute(cmd)
sqlite3.OperationalError: no such table: toMerge.sample_genotype_counts
Indexing /scratch/PI/euan/projects/athletes/bcbio/gemini/batch1-freebayes-nomultiallelic.vcf.gz with grabix.
Loading 3429621 variants.
Breaking /scratch/PI/euan/projects/athletes/bcbio/gemini/batch1-freebayes-nomultiallelic.vcf.gz into 16 chunks.
Loading chunk 0.
Loading chunk 1.
Loading chunk 2.
Loading chunk 3.
...
Loading chunk 14.
Loading chunk 15.
Done loading 3429621 variants in 16 chunks.
2015-02-17 14:08:34 merging 16 chunks.
Traceback (most recent call last):
  File "/share/PI/euan/apps/bin/gemini", line 6, in <module>
    gemini.gemini_main.main()
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 1105, in main
    args.func(parser, args)
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 136, in load_fn
    gemini_load.load(parser, args)
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_load.py", line 50, in load
    load_multicore(args)
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_load.py", line 75, in load_multicore
    merge_chunks_multicore(chunks, args.db)
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_load.py", line 137, in merge_chunks_multicore
    cleanup_temp_db_files(chunks)
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_load.py", line 304, in cleanup_temp_db_files
    os.remove(chunk_db)
OSError: [Errno 2] No such file or directory: '/scratch/PI/euan/projects/athletes/bcbio/gemini/batch1-freebayes-nomultiallelic.vcf.chunk1.db'
' returned non-zero exit status 1
[2015-02-17T22:09Z] sh-5-16.local: Unexpected error
Traceback (most recent call last):
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/bcbio/distributed/ipythontasks.py", line 38, in _setup_logging
    yield config
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/bcbio/distributed/ipythontasks.py", line 230, in prep_gemini_db
    return ipython.zip_args(apply(population.prep_gemini_db, *args))
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/bcbio/variation/population.py", line 33, in prep_gemini_db
    gemini_db = create_gemini_db(gemini_vcf, data, gemini_db)
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/bcbio/variation/population.py", line 72, in create_gemini_db
    do.run(cmd, "Create gemini database for %s" % gemini_vcf, data)
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/bcbio/provenance/do.py", line 21, in run
    _do_run(cmd, checks, log_stdout)
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/bcbio/provenance/do.py", line 95, in _do_run
    raise subprocess.CalledProcessError(exitcode, error_msg)
CalledProcessError: Command 'set -o pipefail; /share/PI/euan/apps/bin/gemini load  --passonly --skip-gerp-bp  -v /scratch/PI/euan/projects/athletes/bcbio/gemini/batch1-freebayes-nomultiallelic.vcf.gz -t snpEff --cores 16 /scratch/PI/euan/projects/athletes/bcbio/gemini/tx/tmp1HyQZi/batch1-freebayes.db
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_merge_chunks.py", line 190, in merge_chunks
    merge_db_chunks(args)
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_merge_chunks.py", line 176, in merge_db_chunks
    append_sample_genotype_counts(main_curr, db)
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_merge_chunks.py", line 40, in append_sample_genotype_counts
    main_curr.execute(cmd)
sqlite3.OperationalError: no such table: toMerge.sample_genotype_counts
Traceback (most recent call last):
  File "/share/PI/euan/apps/bin/gemini", line 6, in <module>
    gemini.gemini_main.main()
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 1105, in main
    args.func(parser, args)
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 244, in mergechunk_fn
    gemini_merge_chunks.merge_chunks(parser, args)
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_merge_chunks.py", line 190, in merge_chunks
    merge_db_chunks(args)
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_merge_chunks.py", line 176, in merge_db_chunks
    append_sample_genotype_counts(main_curr, db)
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_merge_chunks.py", line 40, in append_sample_genotype_counts
    main_curr.execute(cmd)
sqlite3.OperationalError: no such table: toMerge.sample_genotype_counts
Traceback (most recent call last):
  File "/share/PI/euan/apps/bin/gemini", line 6, in <module>
    gemini.gemini_main.main()
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 1105, in main
    args.func(parser, args)
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 244, in mergechunk_fn
    gemini_merge_chunks.merge_chunks(parser, args)
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_merge_chunks.py", line 190, in merge_chunks
    merge_db_chunks(args)
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_merge_chunks.py", line 176, in merge_db_chunks
    append_sample_genotype_counts(main_curr, db)
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_merge_chunks.py", line 40, in append_sample_genotype_counts
    main_curr.execute(cmd)
sqlite3.OperationalError: no such table: toMerge.sample_genotype_counts
Traceback (most recent call last):
  File "/share/PI/euan/apps/bin/gemini", line 6, in <module>
    gemini.gemini_main.main()
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 1105, in main
    args.func(parser, args)
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 244, in mergechunk_fn
    gemini_merge_chunks.merge_chunks(parser, args)
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_merge_chunks.py", line 190, in merge_chunks
    merge_db_chunks(args)
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_merge_chunks.py", line 176, in merge_db_chunks
    append_sample_genotype_counts(main_curr, db)
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_merge_chunks.py", line 40, in append_sample_genotype_counts
    main_curr.execute(cmd)
sqlite3.OperationalError: no such table: toMerge.sample_genotype_counts
Traceback (most recent call last):
  File "/share/PI/euan/apps/bin/gemini", line 6, in <module>
    gemini.gemini_main.main()
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 1105, in main
    args.func(parser, args)
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 244, in mergechunk_fn
    gemini_merge_chunks.merge_chunks(parser, args)
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_merge_chunks.py", line 190, in merge_chunks
    merge_db_chunks(args)
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_merge_chunks.py", line 176, in merge_db_chunks
    append_sample_genotype_counts(main_curr, db)
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_merge_chunks.py", line 40, in append_sample_genotype_counts
    main_curr.execute(cmd)
sqlite3.OperationalError: no such table: toMerge.sample_genotype_counts
Indexing /scratch/PI/euan/projects/athletes/bcbio/gemini/batch1-freebayes-nomultiallelic.vcf.gz with grabix.
Loading 3429621 variants.
Breaking /scratch/PI/euan/projects/athletes/bcbio/gemini/batch1-freebayes-nomultiallelic.vcf.gz into 16 chunks.
Loading chunk 0.
Loading chunk 1.
Loading chunk 2.
Loading chunk 3.
Loading chunk 4.
Loading chunk 5.
Loading chunk 6.
Loading chunk 7.
Loading chunk 8.
Loading chunk 9.
Loading chunk 10.
Loading chunk 11.
Loading chunk 12.
Loading chunk 13.
Loading chunk 14.
Loading chunk 15.
Done loading 3429621 variants in 16 chunks.
2015-02-17 14:08:34 merging 16 chunks.
Traceback (most recent call last):
  File "/share/PI/euan/apps/bin/gemini", line 6, in <module>
    gemini.gemini_main.main()
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 1105, in main
    args.func(parser, args)
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 136, in load_fn
    gemini_load.load(parser, args)
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_load.py", line 50, in load
    load_multicore(args)
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_load.py", line 75, in load_multicore
    merge_chunks_multicore(chunks, args.db)
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_load.py", line 137, in merge_chunks_multicore
    cleanup_temp_db_files(chunks)
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_load.py", line 304, in cleanup_temp_db_files
    os.remove(chunk_db)
OSError: [Errno 2] No such file or directory: '/scratch/PI/euan/projects/athletes/bcbio/gemini/batch1-freebayes-nomultiallelic.vcf.chunk1.db'
' returned non-zero exit status 1
[2015-02-17T23:53Z] sh-5-26.local: Timing: quality control
[2015-02-17T23:53Z] sh-5-26.local: ipython: pipeline_summary
[2015-02-17T23:53Z] sh-5-17.local: Generating summary files: ['', '4_78']
[2015-02-17T23:57Z] sh-5-17.local: There is no qsignature for this species: GRCh37
[2015-02-17T23:57Z] sh-5-17.local: Generating summary files: ['', '4_119']
[2015-02-18T00:00Z] sh-5-17.local: There is no qsignature for this species: GRCh37
[2015-02-18T00:00Z] sh-5-17.local: Generating summary files: ['', '1_06']
[2015-02-18T00:04Z] sh-5-17.local: There is no qsignature for this species: GRCh37
[2015-02-18T00:04Z] sh-5-17.local: Generating summary files: ['', '4_241']
[2015-02-18T00:08Z] sh-5-17.local: There is no qsignature for this species: GRCh37
[2015-02-18T00:08Z] sh-5-17.local: Generating summary files: ['', '4_107']
[2015-02-18T00:13Z] sh-5-17.local: There is no qsignature for this species: GRCh37
...

If you run:
gemini query -q "select * from variants limit 1" batch1-freebayes.db

what does it look like?

Yup, the running twice is probably the issue. I must have restarted this pipeline 10x due to various memory issues. At one point I think I submitted twice.  Sorry for all of the trouble. That database looks like the annotations didn't load properly, thats why there are so many None columns. If you nuke it and load it by hand, do the annotations get parsed correctly?

k, this may take a moment.

Were the two bcbio-nextgen runs running simultaneously for a long time? That might have introduced some corrupted files which might have been the source of the truncated file in #771. Do you know how far back two were running?

Hard to tell. I'm rerunning the pipeline on two different datasets to see
what happens.  The biggest issue is unobvious jobs being killed due to
memory, specifically the submission and controller jobs.
On Sun, Feb 22, 2015 at 8:56 PM, Rory Kirchner<EMAIL_ADDRESS>wrote:

Were the two bcbio-nextgen runs running simultaneously for a long time?
That might have introduced some corrupted files which might have been the
source of the truncated file in #771
https://github.com/chapmanb/bcbio-nextgen/issues/771. Do you know how
far back two were running?
—
Reply to this email directly or view it on GitHub
https://github.com/chapmanb/bcbio-nextgen/issues/773#issuecomment-75489676
.

Lot's of these when manually trying to load the file. I can see all the snpeff annotation so not sure what's going on. I'll post a test vcf.

WARNING: The input VCF has no snpEFF annotations. Variant impact will be set to unknown
WARNING: The input VCF has no snpEFF annotations. Variant impact will be set to unknown
WARNING: The input VCF has no snpEFF annotations. Variant impact will be set to unknown
WARNING: The input VCF has no snpEFF annotations. Variant impact will be set to unknown
WARNING: The input VCF has no snpEFF annotations. Variant impact will be set to unknown
WARNING: The input VCF has no snpEFF annotations. Variant impact will be set to unknown
WARNING: The input VCF has no snpEFF annotations. Variant impact will be set to unknown
WARNING: The input VCF has no snpEFF annotations. Variant impact will be set to unknown
WARNING: The input VCF has no snpEFF annotations. Variant impact will be set to unknown
WARNING: The input VCF has no snpEFF annotations. Variant impact will be set to unknown

I'm going to just assume this run was compromised and not worry about it. Let's see what happens when I rerun the entire pipeline. Close please.

A parallel run using joint instead of cohort calling just finished. The resulting db looks empty
gemini query -q "select * from variants limit 1" batch1-freebayes-joint.db gives nothing and the file is only 150mb.
Part of the log:
log

Daryl;
Thanks for the detailed log files, this was a big help in debugging the issues. There are two things going on:

There is an issue with GEMINI loading CADD scores on some of the input variants. This caused the empty/truncated database.
There is an issue merging split multi-allelic inputs back with bi-allelic. We do some work to split up multi-allelic inputs since GEMINI can't handle these, but the subsequent combine step using GATK's CombineVariants would, in some cases, re-merge the split multi-allelic inputs dropping the effects predictions. This caused the WARNING: The input VCF has no snpEFF annotations. messages.

I pushed fixes I hope will handle both if you update both gemini and bcbio to the latest development versions:
bcbio_nextgen.py upgrade -u development
gemini update --devel

Please let us know if this doesn't fix the underlying problem and we can dig more.

If I blow away the contents of the gemini and final directory will it try
restart from that point?
On Thu, Feb 26, 2015 at 2:58 AM, Brad Chapman<EMAIL_ADDRESS>wrote:

Daryl;
Thanks for the detailed log files, this was a big help in debugging the
issues. There are two things going on:

There is an issue with GEMINI loading CADD scores on some of the
input variants. This caused the empty/truncated database.
There is an issue merging split multi-allelic inputs back with
bi-allelic. We do some work to split up multi-allelic inputs since GEMINI
can't handle these, but the subsequent combine step using GATK's
CombineVariants would, in some cases, re-merge the split multi-allelic
inputs dropping the effects predictions. This caused the WARNING: The
input VCF has no snpEFF annotations. messages.

I pushed fixes I hope will handle both if you update both gemini and bcbio
to the latest development versions:
bcbio_nextgen.py upgrade -u development
gemini update --devel
Please let us know if this doesn't fix the underlying problem and we can
dig more.
—
Reply to this email directly or view it on GitHub
https://github.com/chapmanb/bcbio-nextgen/issues/773#issuecomment-76159124
.

Daryl;
That's right on -- remove the gemini directory and also checkpoints_parallel/multicore2.done and re-run. It'll re-do the VCF and gemini database preparation using multiple machine/cores. Hope this works cleanly now.

Not sure it did. The query results were the same as before and both batch1-freebayes-nomultiallelic.vcf.gz and batch1-freebayes-multiallelic.vcf.gz have lot's of lines with multiple ALTs.
i.e.
1       952494  .       AT      ATT,ATTT,A      11929.4 PASS
I'll run again incase there was problem with updating the code.

Daryl;
Sorry about the issues, that is unexpected. batch1-freebayes-multiallelic.vcf.gz should have multple ALTs, since that is prior to splitting, but they should be removed in batch1-freebayes-multiallelic-decompose-effects.vcf.gz and also stay decomposed in the final file batch1-freebayes-nomultiallelic.vcf.gz. If you check the run command lines in log/bcbio-nextgen-commands.log, is it still using GATK CombineVariants? If so, that would indicate the code didn't get updated for some reason. Hope this helps some.

Sigh, looks like the code didn't update. Rerunning after deleting final/gemini and multicore2.done now fails on an assertion. Might there be something else I should delete?
[2015-03-02T17:46Z] sh-2-22.local: Resource requests: sambamba, samtools; memory: 2.00; cores: 1, 16
[2015-03-02T17:46Z] sh-2-22.local: Configuring 20 jobs to run, using 16 cores each with 32.1g of memory reserved for each job
[2015-03-02T17:46Z] sh-2-22.local: run local -- checkpoint passed: multicore
[2015-03-02T17:46Z] sh-2-22.local: Timing: organize samples
[2015-03-02T17:46Z] sh-2-22.local: multiprocessing: organize_samples
[2015-03-02T17:46Z] sh-2-22.local: Using input YAML configuration: /scratch/PI/euan/projects/athletes/bcbio/project_n143-fb-joint/config/project_n143-fb-joint.yaml
Traceback (most recent call last):
  File "/share/PI/euan/apps/bin/bcbio_nextgen.py", line 216, in <module>
    main(**kwargs)
  File "/share/PI/euan/apps/bin/bcbio_nextgen.py", line 42, in main
    run_main(**kwargs)
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/main.py", line 45, in run_main
    fc_dir, run_info_yaml)
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/main.py", line 81, in _run_toplevel
    for xs in pipeline.run(config, run_info_yaml, parallel, dirs, samples):
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/main.py", line 143, in run
    [x[0]["description"] for x in samples]]])
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/bcbio/distributed/multi.py", line 28, in run_parallel
    return run_multicore(fn, items, config, parallel=parallel)
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/bcbio/distributed/multi.py", line 84, in run_multicore
    for data in joblib.Parallel(parallel["num_jobs"])(joblib.delayed(fn)(x) for x in items):
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/joblib/parallel.py", line 653, in __call__
    self.dispatch(function, args, kwargs)
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/joblib/parallel.py", line 400, in dispatch
    job = ImmediateApply(func, args, kwargs)
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/joblib/parallel.py", line 138, in __init__
    self.results = func(*args, **kwargs)
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/bcbio/utils.py", line 149, in wrapper
    return apply(f, *args, **kwargs)
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/bcbio/distributed/multitasks.py", line 179, in organize_samples
    return run_info.organize(*args)
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/run_info.py", line 38, in organize
    run_details = _run_info_from_yaml(dirs["flowcell"], run_info_yaml, config, sample_names)
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/run_info.py", line 504, in _run_info_from_yaml
    item["vrn_file"] = vcfutils.bgzip_and_index(genome.abs_file_paths(item["vrn_file"]), config)
  File "/share/PI/euan/apps/bcbio/anaconda/lib/python2.7/site-packages/bcbio/variation/vcfutils.py", line 441, in bgzip_and_index
    assert not in_file == out_file, "Input file is bgzipped but not found: %s" % in_file
AssertionError: Input file is bgzipped but not found: /home/dwaggott/projects1/athletes/bcbio/project_n143-fb/final/4_78/4_78-freebayes.vcf.gz

Querying the intermediate db chunks shows similar unparsed INFO tags. I don't see any multiallelic variants in the input vcf or CombineVariants calls in the command logs.

Using a small subset of the nomultiallelic vcf doesn't seem to load properly. Does it work for you?
tmp.vcf.gz
My quick check is for gene names:
bcftools view batch1-freebayes-nomultiallelic.vcf.gz -O z -r 1:1-1000000 > tmp.vcf.gz
/share/PI/euan/apps/bin/gemini load  --passonly --skip-gerp-bp  -p $ /share/PI/euan/apps/bin/gemini load  --passonly --skip-gerp-bp  -p /scratch/PI/euan/projects/athletes/bcbio/project_n143-fb/work/gemini/batch1-freebayes-nomultiallelic.ped -v tmp.vcf.gz -t snpEff tmp.db
CADD scores are being loaded (to skip use:--skip-cadd).
pid 7896: 887 variants processed.
pid 7896: 0 skipped due to having the FILTER field set.
$ gemini query -q "select gene from variants "   tmp.db  | uniq
None

Daryl;
Ugh, apologies. I made a mistake when running back-compatibility mode on snpEff 4.1 and forgot one of the required flags to make the new snpEff work with GEMINI loading. So although everything gets annotated with snpEff, it's not in the form GEMINI expects so it skips adding the information. If you do one more update:
bcbio_nextgen.py upgrade -u development

then remove the existing gemini and effects prediction files:
rm -rf gemini/ && rm -f freebayes/*-effects* && rm -f checkpoints_parallel/multicore2.done

and re-run, it should load correctly now.
Sorry again about all the re-starts on this and thank you for your patience in debugging. Hope this fixes it for you now.

Works! Please close.

Nice one, thanks for all the patience getting this debugged.


Production build

ng build --environment=production

generate dist with dev versions of angular and systemjs, and my app files with .ts and source maps.

Are you getting a specific error? I was able to run this it generates to dist/
Make sure you've got node > 4.1.0.

No, i don't say it fails, i say it not 'production'. Production build should contain .min lib files, no ts or sourcemaps and etc

Prod builds (or any custom environment builds) are currently not supported. It's one of the many things that need to happen in order to make this project production ready.

Looks like the /dist directory contains the exact copy of src directory including typescript files. I think the typescript files are not required in /dist. The /dist/vendor directory is clean, which has only the javascript files.

In order to do the production build we can take the current build and:

[ ] remove livereload references in the index.html
[ ] stop copying the ts files
[ ] stop copying the tsconfig.json file
[ ] add a CNAME?
[ ] enableProdMode()

What else?

Assure that pre-compiled style (LESS/SASS) files are not copied to the dist directory

Config files perhaps, with API addresses and 3rd party services tokens.

I think this list looks good @cironunes. Will we also be minifying/mangling?
Really, this production flow is the only thing missing for me to start using this hard right now.

Picked this as the official production build issue to follow.

Still missing:

[ ] add a CNAME?
[ ] enableProdMode()
[ ] inlining HTML and CSS resources

Added 2 points to your todo list above.

So we are bundling now? That would be the best news I've heard all week.

I'd love to continue working on that, but I just don't have the time at the moment :(
I'm glad that Filipe is taking care of it. :smile:

You are talking about adding a CNAME file, which implies that Github Pages is a "first citizen". If you directly support gh-pages, than you also want to copy dist/index.html to dist/404.html. This will enable browser-reloads together with the PathLocationStrategy.

I know still WIP here, but am I missing a flag for using the production vendor libs or is it just unfinished? I can only get angular2.dev.js for instance. That ~1 second bootstrapping time is maddening. Cheers!

That bit is the next one I'm working on, to be able  to bundle stuff.

Is there any way to add a custom environment by creating a file named for instance config/environment.stage.ts and then execute ng build --environment=stage ? It looks like it's defaulting to dev now.

@cdarken no, it is not currently possible to have more environments other than dev and prod. It's one of my next action items though.
I should mention though (since I haven't yet updated the readme), all builds using -prod use bundling and tree shaking at the moment.

Thanks, I will wait patiently then.

hello @filipesilva do we have an approximate timeline for this? I have a project that I will need to deploy to production soon, is there anything you recommend we should use until this is completed?

@mashhoodr you can already do production builds, the thing that is missing is more than 2 environments.

What about Service Workers?
Also images are minifyed and filerevved? (for permanent caching)
Thanks guys!!!

Currently service workers are only enabled for mobile projects, but we're looking at having them in all prod builds.
Images are not processed at the moment either.

I think this issue is obsolete by now. All of the features mentioned so far have been addressed, with the possible exception of having service workers in all prod builds. That warrants discussion though so probably should be another issue.

can anyone tell plz how to run angular cli production mode forever. i want to deploy production build on live server. But i am not getting it.

Run ng build -prod

Currently ng build --prod still generates source maps
How can I build without suorce maps?
Thanks

Is there a plan to disable source maps in production? My bundle source map is 6.3 MB which is just too much

@neilhem is there an option to skip sourcemaps generation when building using ng build -prod?

Hey @Misiu, I'm using ng build --prod --aot --no-sourcemap  --no-vendor-chunk for my production build.
Still, I get too many files that I believe are unnecessary (but maybe they're not). Would be nice to have all the production deployment options such as: How to deploy gzipped version, how to deploy not gzipped version, with sourcemaps, without sourcemaps...
This are all the files I get when I execute that command. Would appreciate a way to tidy up the production build, maybe include folders to separate files. I also would like to know if I can upload to my cloudfront the .gz without uploading the .js

my angular-cli.json
{   "project": {     "version": "1.0.0-beta.22-1",     "name": "webapp-21b"   },   "apps": [     {       "root": "src",       "outDir": "dist",       "assets": [         "assets",         "download",         ".well-known",         "robots.txt",         "sitemap.xml"       ],       "index": "index.html",       "main": "main.ts",       "test": "test.ts",       "tsconfig": "tsconfig.json",       "prefix": "app",       "mobile": false,       "styles": [         "../node_modules/bootstrap/dist/css/bootstrap.css",         "./css/main.scss",         "./css/buttons.scss",         "./css/forms.scss",         "./css/navbar.scss",         "./css/nouislider.scss",         "./css/mixins.scss",         "./css/spinner.scss",         "./css/utils.scss",         "./css/vars.scss"       ],       "scripts": [         "./assets/js/noUiSlider.9.0.0/nouislider.min.js",         "./assets/js/wNumb.js"       ],       "environments": {         "source": "environments/environment.ts",         "dev": "environments/environment.ts",         "prod": "environments/environment.prod.ts"       }     }   ],   "addons": [   ],   "packages": [   ],   "e2e": {     "protractor": {       "config": "./protractor.conf.js"     }   },   "test": {     "karma": {       "config": "./karma.conf.js"     }   },   "defaults": {     "styleExt": "scss",     "prefixInterfaces": false,     "inline": {       "style": false,       "template": false     },     "spec": {       "class": false,       "component": true,       "directive": true,       "module": false,       "pipe": true,       "service": true     }   } }

How to run forever with angular-cli?


FileNotFoundError: No such file or directory: './StorySalon/image_inpainted_finally_checked'

I encountered a FileNotFoundError while trying to run the dataset.py script. It seems that the directory ./StorySalon/image_inpainted_finally_checked is missing. Below is the detailed error message:
 File "/projects/StoryGen/dataset.py", line 143, in __init__     video_folders = sorted(os.listdir(self.video_image_dir)) # 00001 FileNotFoundError: [Errno 2] No such file or directory: './StorySalon/image_inpainted_finally_checked'

We have provided the download link summary of the dataset (meta-data.json) in the repo and provided the pipeline to obtain the corresponding text and mask.
The dataset has already uploaded to HuggingFace, you can download it from https://huggingface.co/datasets/haoningwu/StorySalon
Please refer to https://github.com/haoningwu3639/StoryGen/issues/20#issuecomment-2076365114 for more details.


application deployed on Device gets into unknown state, cannot remove it anymore

Even though I removed the application from the fleet and the device doesn't have it anymore in the yaml spec, it seems it is not able to sync and actually remove it from the device.
This is the device spec:
apiVersion: v1alpha1
kind: Device
metadata:
  annotations:
    device-controller/console: b5edb81e-c2d6-42b0-9917-90aad8b41cc3
    device-controller/renderedVersion: "23"
    fleet-controller/templateVersion: "2024-12-17T11:42:29.809138975Z"
  creationTimestamp: "2024-12-16T15:45:10.437628Z"
  generation: 18
  labels:
    alias: x1-lmf-centos9
    host: x1-lmf
    location: home
  name: u1vob2sb732feu7jma8i04tnrhvc0kie5at0t4r0j29uddsa46gg
  owner: Fleet/my-fleet
  resourceVersion: "1699"
spec:
  config: []
  os:
    image: quay.io/luferrar/centos-bootc-flightctl:v1
status:
  applications:
  - name: quay.io/luferrar/websockets-echo-server:latest
    ready: 0/0
    restarts: 0
    status: Unknown
  applicationsSummary:
    info: 'Not started: quay.io/luferrar/websockets-echo-server:latest'
    status: Degraded
  conditions:
  - lastTransitionTime: "2024-12-17T11:42:40.227127595Z"
    message: 'Updated to desired renderedVersion: 23'
    reason: Updated
    status: "False"
    type: Updating
  - lastTransitionTime: "2024-12-16T15:45:10.458843259Z"
    message: ""
    reason: Valid
    status: "True"
    type: SpecValid
  config:
    renderedVersion: "23"
  integrity:
    summary:
      status: Unknown
  lastSeen: "2024-12-17T11:58:39.778527562Z"
  os:
    image: quay.io/luferrar/centos-bootc-flightctl:v1
    imageDigest: sha256:ffddd93dd1a478d75976c129245c4e5ee0ac4d459badc20cff4a6c5b6b64a986
  resources:
    cpu: Healthy
    disk: Healthy
    memory: Healthy
  summary:
    status: Online
  systemInfo:
    architecture: amd64
    bootID: |
      94b79142-cf02-49b7-95ce-5f490bf72d4b
    operatingSystem: linux
  updated:
    status: Unknown

and I can still see it in the /etc/compose/manifest directory on the device

tracked here https://issues.redhat.com/browse/EDM-939


Problem installing gometalinter

cd C:\GoWorkspace\src\github.com\alecthomas\gometalinter; git pull --ff-only
There is no tracking information for the current branch.
Please specify which branch you want to merge with.
See git-pull(1) for details.
git pull <remote> <branch>

If you wish to set tracking information for this branch you can do so with:
git branch --set-upstream-to=origin/<branch> master

package github.com/alecthomas/gometalinter: exit status 1

This is not an issue with gometalinter... Use go get -u github.com/alecthomas/gometalinter to update.


Performance: Refactor to slots for class instance variables

Note that slots can't be added to the Timer class as the _instance and _lock_init will throw errors like this:
AttributeError: 'Timer' object attribute '_lock_init' is read-only


React v16 Not Compatible With react-text-mask

I updated my code to react v16 react-dom v16, using your library which is amazing. But I keep on getting this warning message. Which is as follows.
warning.js:33 Warning: Stateless function components cannot be given refs. Attempts to access this ref will fail
Either we shouldn't use the ref props in the component or convert it into a smart component. What do you think? I really look forward to your response.

Are you sure it's because of react-text-mask, we only apply a ref to an input here https://github.com/text-mask/text-mask/blob/master/react/src/reactTextMask.js#L38
We do not use a stateless function component.

Hi,
Sorry no, this was a materiel-ui thing. It is resolved now.


Implement filtering

Todo

[ ] Filter tests
[ ] Index operation tests
[ ] Boolean filter

Coverage decreased (-1.2%) to 97.345% when pulling 19fb269cb9c11e86b7538b574d0333762b043c73 on feature/filters into ecf7e332ff14fbc0c85139c7ed6df621a97290c2 on master.

Coverage decreased (-51.9%) to 46.693% when pulling 19fb269cb9c11e86b7538b574d0333762b043c73 on feature/filters into ecf7e332ff14fbc0c85139c7ed6df621a97290c2 on master.

Coverage increased (+0.05%) to 98.594% when pulling b5b0e05f88ff0654e6db76247fbc2016b5250b17 on feature/filters into 50964886b2e23eef8076ee3bc6adff0bd0691689 on master.

Coverage increased (+0.05%) to 98.594% when pulling 754dbe7025df14e10a49bb0f04ef5fb07cda3eab on feature/filters into 50964886b2e23eef8076ee3bc6adff0bd0691689 on master.

Coverage increased (+0.05%) to 98.594% when pulling 754dbe7025df14e10a49bb0f04ef5fb07cda3eab on feature/filters into 50964886b2e23eef8076ee3bc6adff0bd0691689 on master.


"Ion Creangă", "Averești", "Izvoru", "Muncelu", "Recea" and "Stejaru" are listed in the drop-down menu

Description:
"Ion Creangă", "Averești", "Izvoru", "Muncelu", "Recea" and "Stejaru" are listed in the drop-down menu alongside its corresponding county and commune, according to the law
Precondition:
The website is up an running.

Step 2
Press "x" button.

Expected results
The location was deleted from the search bar.

Step 3
Type "Averești" in the search bar.

Expected results
The location is listed in the drop-down menu as "Sat Averești, NEAMȚ (Ion Creangă)".

Step 6
Press "x" button.

Expected results
The location was deleted from the search bar.

Step 7
Type "Muncelu" in the search bar.

Expected results
The location is listed in the drop-down menu as "Sat Muncelu, NEAMȚ (Ion Creangă)".

Step 8
Press "x" button.

Expected results
The location was deleted from the search bar.

Step 9
Type "Recea" in the search bar.

Expected results
The location is listed in the drop-down menu as "Sat Recea, NEAMȚ (Ion Creangă)".

Step 10
Press "x" button.

Expected results
The location was deleted from the search bar.

Step 11
Type "Stejaru" in the search bar.

Expected results
The location is listed in the drop-down menu as "Sat Stejaru, NEAMȚ (Ion Creangă)".


unlocking without unlatching?

Is it possible to unlock the door without unlatching it automatically?

I am also interested in the feature. An answer would be great.


Position-only Inverse Kinematics

I'd like to move the end-effector along a given trajectory. I only have the 3d positions of the end-effector along the trajectory, not the orientations of the end-effector.
When I try doing IK with the rotation fixed and a small delta in the position, it often does not converge (the error plateaus), which I think is an indication that no valid configuration exists.
How can i do position-only IK?
From  #1336:

if you only want to reach some position, just consider the three rows of the Jacobian related to the translation, the three first rows.

I tried zero-ing out the last three rows of the Jacobian (which is of shape [6, 15]), but this did not seem to converge (in fact, the error started increasing). How can I only consider the three rows of the Jacobian related to the translation?

Hello @arjung128 !
Could you provide a tiny example?
PS: politeness is more than welcome when you ask for advises

Hello @jcarpent,
I'm very sorry if I came across as a bit rude, that was not my intention at all. I really appreciate your prompt responses -- the prompt replies are extremely helpful, especially when stuck on something!
I have the robot end-effector pose at t=0:
R =
  0.70356  0.633152   0.32268
0.0875032 -0.527802  0.844848
 0.705228 -0.566166 -0.426743
  p =  0.45032 -0.10624 0.961281

And I want to move the end-effector position a little bit along the x-direction, e.g. to p = 0.50032 -0.10624 0.961281.
When I do IK with the new desired position and the same rotation from t=0 (as shown below) using code from the tutorial, IK is not able to converge:
R =
  0.70356  0.633152   0.32268
0.0875032 -0.527802  0.844848
 0.705228 -0.566166 -0.426743
  p =  0.50032 -0.10624 0.961281

Because the offset is only 0.05 in the x-direction, it seems like this should be reachable for the robot end-effector (based on visualizing the robot in meshcat), but perhaps not reachable with the specified rotation.
Is there a way to do IK given just the desired 3D position (allowing for any rotation of the end-effector)? Either find a valid rotation and then use that for IK, or not consider error in rotation when doing IK?
Thanks in advance for your help!

Given that you refer to https://github.com/stack-of-tasks/pinocchio/issues/1336, are you sure that you are using the pin.LOCAL_WORLD_ALIGNED jacobian as described in https://github.com/stack-of-tasks/pinocchio/issues/1336#issuecomment-723419697 ?
If I am not wrong, that is the only jacobian that, if multiplied by the robot velocity, returns a quantity where the three rows of the jacobian related to translation actually are the derivative of the origin of the frame expressed in the world frame.

@traversaro I am using pinocchio.computeJointJacobian() as used in the IK tutorial rather than pinocchio.getJointJacobian(), which doesn't have a localFrame parameter.

@traversaro I am using pinocchio.computeJointJacobian() as used in the IK tutorial rather than pinocchio.getJointJacobian(), which doesn't have a localFrame parameter.

I am definitely not an expert of Pinocchio internals (so @jcarpent feel free to correct me if I am wrong) but, under this assumptions:

we have the frame a (inertial/absolute frame) and e (the joint frame of interest), and the transform between this two frames is represented by ᵃRₑ ∈ SO(3)  ᵃoₑ ∈ ℝ³
(.)ᵛ represents the mapping between the skew-symmetrix matrix and the related vector in ℝ³, i.e. such that if u,v ∈ ℝ³, u = (U)ᵛ s.t. U v = u × v
the dot represent the usual mathematical analysis dot operator (time derivative) and not the Featherstone one then the different velocities are:

I think that the translational part of the jacobian you are using is the jacobian of the quantity ᵃRₑᵀ ᵃȯₑ, while you are interested in the jacobian of ᵃȯₑ . Can you try to pre-multiply the jacobian by ᵃRₑ to obtain the jacobian of ᵃȯₑ and check if that is working? This should be equivalent of using the pin.LOCAL_WORLD_ALIGNED jacobian.

@traversaro Thanks for such a detailed response!
How can I get ᵃRₑ? Sorry, I am new to Pinocchio and this area in general.
Also, I tried using pinocchio.getJointJacobian() with pin.LOCAL_WORLD_ALIGNED similar to #1336:
while i < n_max_iter:

    pin.forwardKinematics(model, data, x)

    # dummy current SE3 with identity rotation
    cur = SE3.Identity()
    cur.translation = data.oMi[j_idx].translation

    dMi = goal.actInv(cur)
    err = pin.log(dMi).vector / DT
    mu = lm_damping * max(1e-3, err.dot(err))

    jac = pin.getJointJacobian(model, data, j_idx, pin.LOCAL_WORLD_ALIGNED) # pin.WORLD)

    ATA = np.zeros((model.nv, model.nv))
    ATb = np.zeros((model.nv,))
    ATA += jac.T @ jac + mu * np.eye(model.nv)
    ATb += -err.T.dot(jac)

    dv = -solve_qp(ATA, ATb)
    x = pin.integrate(model, x, dv * DT)
    x[:7] = q0[:7]

    if not i % 1000:
        print('%d: error = %s' % (i, norm(err*DT))) # err.T))
    i += 1

However, this does not seem to converge either (in fact, doesn't seem to converge for the end-effector pose at t=0 either, let alone the modified pose at t=1). Any ideas what could be going wrong?

Ack, my intuition failed then. As asked before, then it would be great if you could provide a tiny (self-contained) example. It is not possible to understand what is going on just by the loop you posted, see https://stackoverflow.com/help/minimal-reproducible-example for more details.

You can see this example : https://github.com/stack-of-tasks/pinocchio/blob/master/examples/inverse-kinematics.py
Thanks @traversaro for your kind help.

@traversaro Thanks for your support! Much appreciated.
@jcarpent This is very helpful. I didn't realize setting the rotation to identity would get rid of this constraint. Thanks a lot for your help!


Add cram input option to samtools fastq pipeline

Also, we're on 1.12 so I think we can bump to 1.20 for a new pipeline.

Supported cram files in dragen somatic and germline steps instead


Improvements to evolve + config md updates

Separate negated and included pokemon
Cleared up config variable ambiguity
Updated configuration_files.md

I'm headed to bed shortly, so if folks could help me test. Idea now is, we can specify all in evolve_list or specifically list certain pokemon. If we set evolve_list to all, we ignore those listed in donot_evolve_list.

The config md updates started off as one or two things. Next thing I know I'm pretty much doing a different PR. I'll leave up.

:+1:

Closes #4873 and some recent complaints about our documentation.

:+1:


fix: don't crash when item's Author is nil

Creating the item's body crashed when accessing items[i].Author without checking whether it's nil.

Oh, I overlooked the possibility that the items[i].Author would be nil. thank you!✨


CSS container queries aren't supported

Type: Bug
All container related properties aren't recognised, the rule itself isn't recognised and now recently there are no suggestions in container queries at all, even though it worked before. They were released a year ago and work just like media queries, although I think you can nest them.


Issue installing cli-premium

When running composer global require laravel-shift/cli-premium (or even in a single project), I get the following error:
Could not find a matching version of package laravel-shift/cli-premium. Check the package spelling, your version constraint and that the package is available in a stability which matches your minimum-stability (dev).

Using PHP 8.1 and Laravel 10.10
Using "minimum-stability": "dev"
I have my auth.json set up correctly with the username/password containing a valid UUID license (valid until July 2024).
I have no issue installing laravel-shift/cli in my project or globally.

Did the cli-premium path changed recently? It used to work a month ago.

I needed to update my composer.json with the following bit:
"repositories": [
    {
        "type": "composer",
        "url": "https://satis.laravelshift.com"
    }
],

Glad you got it. The welcome email (or email I sent to existing license holders) should have noted that addition to composer.json. Let me know if not so I can make it easier for future users.

@jasonmccreary I your mail with subject "Introducing the Shift CLI" from 25.09.2023 I can not find any information about adding a repository to composer.json. Can you add this to the FAQ at https://laravelshift.com/cli?


no relationship of type ... in collection error

Hi, I was trying python-docx with a .docx file which I got from a .doc file 'Saved as' a Strict Open XML Document with Microsoft Office Standard 2013. Unfortunately, when I'm trying to open the .docx file with python-docx, I get the error below. Does anybody have any idea what the problem is ?
Traceback (most recent call last):
  File "fillin_subrep_contract.py", line 39, in <module>
    fillin_contract_docx(sys.argv[1])
  File "fillin_subrep_contract.py", line 20, in fillin_contract_docx
    document = Document(wordfile)
  File "/var/tmp/mow/git/virtualenvs/webdev/lib/python2.7/site-packages/docx/api.py", line 25, in Document
    document_part = Package.open(docx).main_document_part
  File "/var/tmp/mow/git/virtualenvs/webdev/lib/python2.7/site-packages/docx/opc/package.py", line 108, in main_document_part
    return self.part_related_by(RT.OFFICE_DOCUMENT)
  File "/var/tmp/mow/git/virtualenvs/webdev/lib/python2.7/site-packages/docx/opc/package.py", line 127, in part_related_by
    return self.rels.part_with_reltype(reltype)
  File "/var/tmp/mow/git/virtualenvs/webdev/lib/python2.7/site-packages/docx/opc/rel.py", line 63, in part_with_reltype
    rel = self._get_rel_of_type(reltype)
  File "/var/tmp/mow/git/virtualenvs/webdev/lib/python2.7/site-packages/docx/opc/rel.py", line 116, in _get_rel_of_type
    raise KeyError(tmpl % reltype)
KeyError: u"no relationship of type 'http://schemas.openxmlformats.org/officeDocument/2006/relationships/officeDocument' in collection"

I expect you'll see a difference in the .rels part of the document package. I expect it uses a different URI for the main document relationship. If you dump the .rels part with opc-diag you should be able to see it.
If you do that on a regular Word document you get something like this:
$ opc browse default.docx .rels
<?xml version='1.0' encoding='UTF-8' standalone='yes'?>
<Relationships xmlns="http://schemas.openxmlformats.org/package/2006/relationships">
  <Relationship Id="x" Type="http://schemas.openxmlformats.org/officeDocument/2006/relationships/extended-properties" Target="docProps/app.xml"/>
  <Relationship Id="x" Type="http://schemas.openxmlformats.org/officeDocument/2006/relationships/officeDocument" Target="word/document.xml"/>
  <Relationship Id="x" Type="http://schemas.openxmlformats.org/package/2006/relationships/metadata/core-properties" Target="docProps/core.xml"/>
  <Relationship Id="x" Type="http://schemas.openxmlformats.org/package/2006/relationships/metadata/thumbnail" Target="docProps/thumbnail.jpeg"/>
</Relationships>

The second relationship item is the one python-docx is looking for to start the "open" process. Apparently there's change in there somewhere when saving as strict OpenXML.

Thanks @scanny, my bad for using a 'Strict Open XML Document'. Re-generated my input document as a standard 'Word document' .docx and that seems to have fixed the problem.

Closing as resolved.

Is there any way to read such Strict Open XML document or service needs to generate the docx without strict open XML ? Please guide.

i  have same issue , could you have other way using the python-docx  to open the docx when saving as the strict open xml ？

i have same issue , could you have other way using the python-docx to open the docx when saving as the strict open xml ？

@scanny Hi, is there a way we can open this?

Hi you guys： try save AS a new file and read it..


resolve bg symlink to absolute path

faced a problem with symlinks and wal-telegram
my files structure is like
├── wallpaper -> /home/mrt0rtikize/Pictures/wallpapers/a_girl_red_circle.jpg
└── wallpapers
   ├── a_girl_blue_sky.jpeg
   ├── a_girl_red_circle.jpg
   ├──  ...

so in my case wal-telegram doesn't work (cuz symlink has no extension) and i think it's a common case for pywal users
i think what resolving symlinks to absolute path with readlink is easiest solution:

if bg is symlink - we will have absolute path instead
if bg is path - it will not change

Thanks for the MR


Introducing the song tokenizer and update of the .gitignore

-Introducing the tokenizer, for use with songs
The tokenizer reads a file by giving it the path to it. Then it performs a series of operations which result in the content of the file being turned into a List, separated word by word and each verse, lyric and repetition marked by a token.
The tokens are as follows:
||, marks the beginning of a verse
||, marks the beginning of a lyric
||, marks the beginning of a repetition(repetition begins on the following lyric)*
||, marks the end of a repetition(repetition ends on the following lyric)*
*if the two are side by side inside the list, it means that it is only the following lyric that is repeated.

The tokenizer is implemented in the class SongReader, which contains the following methods:
main, which is used to run the SongReader as a standalone application
-tokenizer, which reads the file and places it in a String, performs cleanup tasks (removing
                       punctuation marks and unnecessary tokens), calls the separationIgnition method and
                       returns the List<String> containing the tokenized song.
-separationIgnition, which calls the listHandler method and performs tasks on the list to contain the
                                     final tokenized song
-listHandler, which runs through the list to identify any given tokens and keep them as is and then
                         calls a breaker(frontBreaker and backBreaker) depending on the way the string
                         element in each list's position has to be split
-backBreaker, which performs a split tasks on a String(done for each element of the above list) and
                            splits it at the end(used for marking verses, lyrics and repetition ends)
-frontBreaker, which performs a split tasks on a String(done for each element of the above list) and
                            splits it towards  the front(used for marking repetition start)

-Update .gitignore to ignore the .csv file generated after an experiment run as a report of the training and testing on the selected resources.

Seems good. We can use javadoc, make some refactors , do some testing and u[datethe read-me as well.


Make our release-notes great and share them with the rest of the world

I am thinking about ways to improve the user experience of cert-manager, and one of the areas of possible improvement is the cert-manager release notes. From my experience, I think users are facing two pain points:

People skip reading the release notes due to not being super user-friendly,
People are not aware of these new releases (unless they either watch releases on GitHub or follow us carefully on the Kubernetes Slack).

⚠ Note that these come from my own experience; I did not apply standard product management techniques in order to have "proper" user-centric feedback which would allow me to show proof of this issue and to prioritize things. Please let me know if I should rather do a survey before proceeding to gather more feedback.

Objective
I was thinking about setting ourselves the following objective:

The cert-manager release notes are so great that people cannot wait to read the next release's notes.

Goals
I propose the following goals to achieve this objective:

Precision & Clarity: Users are able to learn enough about the release within 10 seconds and do not need to be experts of either cert-manager or Kubernetes to be able to understand whether a particular change is of interest to them.
Interesting to read: Users find the release notes interesting enough that they often learn something new.
Source of information is clearly identified: Users understand the difference between the notes in (1) GitHub Releases, (2) on https://cert-manager.io, and (3) the upgrade notes, and can easily navigate to them.
Spread the word: Users are informed of new releases in 5 different mediums (e.g., mailing list, GitHub Releases, Twitter, LinkedIn, Kubernetes Slack).

Actions

[x] @maelvls: Talk about the idea on the biweekly dev meeting on 21 Apr 2021.
[ ] @maelvls: Write a set of examples and guidelines on how to write good release notes.
[x] @maelvls: Find out who owns the @jetstack handle on Twitter. We want to be able to tweet (or just retweet) using the Jetstack handle.
[x] @maelvls: Add a "advertize the release" section to https://cert-manager.io/docs/contributing/release-process/ that explains how to (1) send an email to the mailing list, and (2) send a tweet.

cc @jetstack/team-cert-manager

A lot of work has gone into the release notes 1.4 page. I am now confident that our release notes are great, although they still need more context as to why each feature was added.


carbon gray 10 combinators menu text is not visible

User has to highlight text in order to read

no longer an issue, as we have deprecated this plugin


Entitas for Go has moved

The README should be updated with the new repository URL for the Go version of Entitas. It has moved to https://github.com/wooga/go-entitas

Done.

Thanks! :bow:


Forfeit Tasks

Introduce a mechanism where a fleet adapter can notify the dispatcher that it can no longer perform a previously assigned task. The dispatcher should then re-initiate the bidding process for this task.

This seems to be the design for https://github.com/open-rmf/rmf_task/pull/39/files#diff-1d39b8181282eab5320c93dd653c0aa2eb1d5270a3c86cf7a2816ca9cf973547R245

The cancel is a bit different from a forfeit. Cancelling is requested by the operator. A forfeit would be if the active task itself determines that it cannot be completed by the robot.
The redesign will introduce a Failed and a Blocked condition status for task phases. This could be watched to determine when a task cannot be completed.
It's not obvious to me when/how the handling of task failures should be automated. I wonder if maybe the Request should include a parameter that says whether the task should be automatically reassigned on failure vs brought to the attention of an operator to decide what to do about the failure.

In addition to task failures/blockages, tasks may need to be forfeited if all the robots in the fleet become unavailable/ get removed.
This intention of this ticket is to brainstorm the topics/messages that will need to be introduced along with implementation changes within the Dispatcher node and FleetUpdateHandle.


Publish GPU manager documentation

Original issue: https://charm.cs.illinois.edu/redmine/issues/813

mprobson/gpu-manager-doc-cleanup

Original date: 2015-11-11 03:06:52

~~https://charm.cs.illinois.edu/gerrit/806~~ https://github.com/UIUC-PPL/charm/commit/5ad8c5d52cab740f3330c115e195f40a84561d24


pretrained CPU model on opennmt.net is very bad at summarization

Hi, I have been testing the text summarization feature with pre trained model given here http://opennmt.net/Models/ . It gives me a lot of <unk> s. Some example predictions are:
<unk> <unk>
a petting zoo
community hall <unk> <unk> <unk> tractor fans
<unk> <unk> <unk> <unk> <unk> <unk> <unk> <unk> <unk> <unk> <unk> <unk> <unk>
another independence scare for <unk> day

<unk> <unk>
<unk> activist group says <unk> people have died
<unk> <unk> <unk>
<unk> <unk> <unk>
<unk> among those killed in <unk>

<unk> <unk> meeting
no further changes for <unk>
<unk> the course
<unk> <unk>
<unk> <unk> <unk>
<unk> to be a corporation
<unk> <unk> <unk>
churches to be represented in parishes
<unk> future for church life
<unk>
<unk> reform plan for parishes
<unk> <unk> <unk>
<unk>
<unk> <unk>
<unk> <unk> <unk>
<unk> <unk> <unk> <unk>
<unk> <unk> <unk> <unk> <unk>
<unk> to be confirmed in spring meetings
<unk> <unk> <unk>
electronic device rules to stay in force for now
<unk> airline passengers to go on sale
<unk> <unk> <unk> <unk>
<unk> ca n't stop <unk>

<unk> crude falls to $ ### a barrel
still hopeful candidate still the candidate still the candidate still still candidate <unk> <unk> still candidate still candidate still candidate still candidate still candidate still candidate still candidate still candidate still candidate still candidate still candidate still candidate still candidate still candidate still candidate <unk> still candidate still candidate still candidate still candidate still candidate still candidate <unk> still candidate still candidate <unk> still candidate still candidate <unk> still candidate still candidate <unk> still candidate still candidate still candidate <unk> still candidate still candidate <unk> still candidate still candidate <unk> still candidate still candidate <unk> still candidate still candidate <unk> still candidate <unk> still candidate still candidate <unk> still candidate still candidate <unk> still candidate <unk> still candidate still candidate <unk> still candidate <unk> still candidate still candidate <unk> still candidate <unk> still candidate still candidate <unk> still candidate <unk> still candidate still candidate <unk> still candidate <unk> still candidate still candidate <unk> still candidate <unk> still candidate <unk> still candidate still candidate <unk> still candidate <unk> still candidate still candidate <unk> still candidate <unk> still candidate still candidate <unk> still candidate <unk> still candidate <unk> still candidate still candidate <unk> still candidate <unk> still candidate still candidate <unk> still candidate <unk> still candidate still candidate <unk> still candidate <unk> still candidate still candidate <unk> still candidate <unk> still candidate still candidate <unk> still candidate <unk> still candidate still candidate <unk> still candidate <unk> still candidate still candidate <unk> still candidate <unk> still candidate still candidate <unk> still candidate <unk>
new mayoral candidate <unk>
<unk> <unk> is a banker

I think the training is not enough for the CPU model. it will be great if someone could convert the GPU model for CPU.

@freakeinstein How you loaded the model and feed articles to it to generate text summarization?


[docs] Add global override demos

Closes #15946

The preview link https://deploy-preview-16067--material-ui.netlify.com/customization/components/#5-global-theme-variation


[WIP] Add Category entity to test reference field for entire object

Refs #26.

[x] Add test proving the issue
[ ] Solve the issue

So just some food for thought...  Generating an interceptor is probably the best way to handle this on a GET request. I actually have a serializer type that can help get the flat id's off of nested objects, but I personally am not a big fan of this route - it's an extra step to configure. In my demo app I actually use this. On the POST side, in v.0.7.0 you can actually post an id to a relationship and it will load the reference.  So, person.address = 1 will load the proper address record during persistence.  There is a new test in 0.7.0 that shows what I mean here.  I haven't tested it with an array of records, but I have that on my todo. It should work the same though, and I think that solves most of the save operation issues with ng-admin.

Fine for the POST route. But we also need the id for GET routes as ng-admin can deal only with ids. So, we still need an interceptor for GET routes.
It is non optimal for the moment, as we are going to fetch the related entity, extract its id, and then do another useless request to fetch the already retrieved entity. But that's more a ng-admin issue.
So, for the moment, let's use an interceptor for GET requests.

Superseeded by #32.


Implicit call to -[RLMArrayLinkView copyWithZone:] unavailable and crashing app

I have an RLMObject:
class RipOrganization: RLMObject {
    // MARK: Properties
    dynamic var orgId: String = ""
    dynamic var name: String = ""

    dynamic var repositories: RLMArray = RLMArray(objectClassName: "RipRepository")
}

And I'm trying to access its repositories:
guard let org = RipOrganization.allObjects().sortedResultsUsingProperty("name", ascending: true).objectAtIndex(UInt(section)) else { return -1 }

print(org.repositories)

Which errors out with an implicit call to copyWithZone
-[RLMArrayLinkView copyWithZone:]: unrecognized selector sent to instance 0x7f839272f540

I expanded this out to a full test case, and it worked in Xcode 7 for me:
import XCTest
import Realm

class RipRepository: RLMObject {
    dynamic var name: String = ""
}

class RipOrganization: RLMObject {
    dynamic var orgId: String = ""
    dynamic var name: String = ""

    dynamic var repositories: RLMArray = RLMArray(objectClassName: "RipRepository")
}

class TestStuff: XCTestCase {
    func testStuff() {
        let section = 0
        let realm = RLMRealm.defaultRealm()
        realm.transactionWithBlock {
            RipOrganization.createInRealm(realm, withValue: ["a", "b", [["repo"]]])
        }

        guard let org = RipOrganization.allObjects().sortedResultsUsingProperty("name", ascending: true).objectAtIndex(UInt(section)) else { return }

        print(org.repositories)
    }
}

Do you maybe have an extension on RLMArray adding debugDescription or something similar to that which could be causing problems?

I couldn't get it to replicate outside of my project either, but it would appear that adding a cast to
guard let org = RipOrganization.allObjects().sortedResultsUsingProperty("name", ascending: true).objectAtIndex(UInt(section)) else { return }

, i.e.:
guard let org = RipOrganization.allObjects().sortedResultsUsingProperty("name", ascending: true).objectAtIndex(UInt(section)) as? RipOrganization else { return }

resolves the issue. Perhaps guard let is doing some copying of objects?

@pcperini could you please share the project that exhibited this issue with us, either here or privately at<EMAIL_ADDRESS>so we can try and see what's happening? Thanks!

Closing due to inactivity. @pcperini if you manage to craft a reproduction case, please reopen this issue and share the offending code with us.


Add a dependencies install file (requirements.txt)

Expected Behavior
Need to have a requirements.txt file to store dependencies. This is especially important because we have some odd dependencies (like pyserial rather than serial module)
Actual Behavior
You just have to run things and find out what dependencies you are missing.

There actually is a requirements.txt file, but just need to add pyserial


Release Spanner libraries version 5.0.0-beta03

Changes in Google.Cloud.Spanner.Data version 5.0.0-beta03:
New features

Add support for FLOAT32 (commit 83700dd)

Breaking changes

The default mapping for values of CLR type decimal was FLOAT64 and it is now Numeric. (commit 9824a9c)
The default mapping for values of CLR type float was FLOAT64 and it is now FLOAT32. (commit 83700dd)
Support for specifying a maximum commit delay was added in v5.0.0-beta02. Unfortunately properties and methods for this feature were erroneously named using "commit delay" instead of "max commit delay". All such names are being corrected in this commit. (commit 03e66d7)
Remove Obsolete code that had been introduced before v5.0.0-beta01. (commit fdc4bff)

Packages in this release:

Release Google.Cloud.Spanner.Admin.Database.V1 version 5.0.0-beta03
Release Google.Cloud.Spanner.Admin.Instance.V1 version 5.0.0-beta03
Release Google.Cloud.Spanner.Common.V1 version 5.0.0-beta03
Release Google.Cloud.Spanner.Data version 5.0.0-beta03
Release Google.Cloud.Spanner.V1 version 5.0.0-beta03

Created release for Google.Cloud.Spanner.Admin.Database.V1-5.0.0-beta03
Created release for Google.Cloud.Spanner.Admin.Instance.V1-5.0.0-beta03
Created release for Google.Cloud.Spanner.Common.V1-5.0.0-beta03
Created release for Google.Cloud.Spanner.Data-5.0.0-beta03
Created release for Google.Cloud.Spanner.V1-5.0.0-beta03


clusterctl fails with cert-manager installation when moving from bootstrap to management cluster

/kind bug
What steps did you take and what happened:
I followed the steps in the quickstart up to the step where one moves cluster-api objects to the control plane cluster. I chose the hetzner-baremetal-control-planes flavor with 3 control plane nodes and left the worker count at the default of 0.
It failed with this bug (Certmanager pods not starting because of untolerated taint control-plane #7074) previously reported on the cluster-api issue tracker: https://github.com/kubernetes-sigs/cluster-api/issues/7074
What did you expect to happen:
I expected to move the cluster-api objects to the control plane.
Anything else you would like to add:
This is either a bug, or the documentation needs to be updated to clarify whether one can transfer cluster-api objects to the control plane or if one needs to keep the cluster-api objects in a management cluster.
Environment:

cluster-api-provider-hetzner version: v1.0.0-beta.18 (I'm assuming the latest, how do I check?)
Kubernetes version: v1.27.2
OS (e.g. from /etc/os-release): Arch Linux

The installation of cert-manager and cluster-api components like core, bootstrap, control-plane is handled by the cluster-api/clusterctl project. So it's not a bug of this provider.
It's usually not recommended to start the controllers on the control-plane itself. You would need to patch the toleration. I'm not sure if this is supported by clusterctl. If you were using the cluster-api operator, you could patch the resource in the appropriate custom resource object. That way it should be possible to start the cluster-api stack on a control plane as well.

It would be helpful if the documentation can be explicit about which configurations or flavors apply to the section "Moving components" in the quick start and not assume that it will work for all configurations.


Changes for 1044

Well I see Travis is feeling particularly slow today.

@Lukasa Mac OS has been way over-worked as of late on Travis and it doesn't seem like they're adding any new Mac OS workers to the pool... It looks like the tests are now running though!

Current coverage is 100% (diff: 100%)

Merging #1050 into master will not change coverage

@@           master   #1050   diff @@
=====================================
  Files          21      21
  Lines        1881    1881
  Methods         0       0
  Messages        0       0
  Branches        0       0
=====================================
  Hits         1881    1881
  Misses          0       0
  Partials        0       0

Powered by Codecov. Last update 78e8b5b...c4a5cfb


Escaped resolver modifying _root_ doesn't work

Describe the bug
I have a resolver, that modifies _root_. When the resolver is unescaped, it works fine, but if it's escaped, it doesn't behave as expected.
This could be interesting for /related to #1113 @odelalleau
To Reproduce
Please provide a minimal repro.
from omegaconf.omegaconf import OmegaConf as oc

def modify_root(key, value, _root_):
    _root_[key] = value
    return "done"

test = {
    "a": "a",
    "b": "b",
    "modify_a": "${modify_root:a,MODIFIED}",
    "modify_b": "\${modify_root:b,MODIFIED}"
}

oc.register_new_resolver("modify_root", modify_root)

config = oc.create(test)
oc.resolve(config)
obj = oc.to_object(config) # resolving second time
print(obj)
# >>> {'a': 'MODIFIED', 'b': 'b', 'modify_a': 'done', 'modify_b': 'done'}

We see, that both times the resolver got executed, but only the unescaped one could modify its key (a).
Expected behavior
I would expect, that after resolving twice, the key b gets MODIFIED too.
Additional context

[X] OmegaConf version: master (<IP_ADDRESS>.dev)
[X] Python version: 3.10.12
[X] Operating system: Linux Mint
[X] Please provide a minimal repro

Thanks for the fix!
I don't see why the resolving behaves different in oc.resolve() and oc.to_object() and why the _root_ object is affected by this. Because #1113 is just handling the escaping of resolvers, I thought. And this time both resolvers executes...
Maybe you can give me a little insight on that...


Background worker parallel impl

What kind of change does this PR introduce?
Adds

a parallel implementation for WALRUS in a background worker
a realtime transport layer
a partial implementation of a replacement for pg_recvlogical

@soedirgo can you please review the Rust portions of this PR whenever you have the bandwidth?

@w3b6x9


Unable to associate two WebACLs with the CustomIdentityComponent's API Gateway at the same time.

An API Gateway can associate only one AWS WAF WebACL according to https://docs.aws.amazon.com/waf/latest/developerguide/how-aws-waf-works-resources.html#:~:text=Restrictions on multiple resource associations.
There are two WebACLs for an API Gateway. One is for a managed rule group and the other is for a rate limit rule.
When we deploy the stack, it looks like all WebACLs are associated successfully and the stack deployed successfully either. In reality, only one WebACL which is associated later will be associated. It is caused by the property of associateWebACL api that replace WebACL of a resource without any error.
We need to make a WebACL with multiple rules to associate those two rule for an API Gateway resource.

Thanks for spotting this! Now fixed in this commit: https://github.com/aws-solutions-library-samples/guidance-for-custom-game-backend-hosting-on-aws/commit/44924dcd55c143d5a6f5b5e94ec7d08de70fe40c


Db2demo

DB2 chart with injectable startup script for demos to allow users to prepopulate the database by mounting a script to /var/custom

met with @hollisc who recommended using kustomize+configMapGenerator to inject the script

https://github.com/cp4d-pattern-b/multi-tenancy-gitops/blob/master/0-bootstrap/single-cluster/2-services/argocd/instances/ibm-sfg-db2-init.yaml


Which mode is occlum in？Is it secure mode？

Hai
If the application runs in hyperenclave with occlum，Which mode is occlum and application in？Is it secure mode？

Hi @Ariel08081214 ，Occlum and application run in the TEE environment which is provided by HyperEnclave.
We do not have a specific name to describe such mode. You can understand that Occlum and application is secured by HyperEnclave.


[New game]: Menja Game

🎮 Game Request
This exhilarating game consists of multi color cubes that jump from beneath the screen and the user needs to slice the cube using the mouse controls before the cube crosses the bottom of the screen. the game also has a slow-mo mode which gets activated when a n user slices a electric blue cube.
I hope everyone enjoys this game..!
Point down the features

Beautiful Graphics
Slow-Mo  Mode
Fun to Play
Game can be Paused

Select program in which you are contributing
GSSoC23
Code of Conduct

[X] I follow CONTRIBUTING GUIDELINE of this project.

/assign

Hey @trishanu-init ! Thank you so much for your raising the issue💗
It’s all yours, you can come anytime again and make some contributions! 🚀
Alone, we can do little, but together we can do so much! 😇


Invalid argument(s): Failed to load dynamic library

es_compression: ^2.0.12 (I've cleaned my project), seems like it's missing ARM64 macOS binaries.
ArgumentError (Invalid argument(s): Failed to load dynamic library '/Users/m/.pub-
cache/hosted/pub.dev/es_compression-2.0.12/lib/src/brotli/blobs/esbrotli-mac64.dylib': dlopen(/Users/m/.pub-
cache/hosted/pub.dev/es_compression-2.0.12/lib/src/brotli/blobs/esbrotli-mac64.dylib, 0x0001): tried: '/Users/m/.pub-
cache/hosted/pub.dev/es_compression-2.0.12/lib/src/brotli/blobs/esbrotli-mac64.dylib' (mach-o file, but is an
incompatible architecture (have 'x86_64', need 'arm64e' or 'arm64')),
'/System/Volumes/Preboot/Cryptexes/OS/Users/m/.pub-cache/hosted/pub.dev/es_compression-
2.0.12/lib/src/brotli/blobs/esbrotli-mac64.dylib' (no such file), '/Users/m/.pub-cache/hosted/pub.dev/es_compression-
2.0.12/lib/src/brotli/blobs/esbrotli-mac64.dylib' (mach-o file, but is an incompatible architecture (have 'x86_64', need
'arm64e' or 'arm64')))

flutter doctor:
[✓] Flutter (Channel stable, 3.22.2, on macOS 14.5 23F79 darwin-arm64, locale
    en-US)
[✓] Xcode - develop for iOS and macOS (Xcode 15.4)
[✓] Chrome - develop for the web
[!] Android Studio (not installed)
[✓] VS Code (version 1.90.2)
[✓] Connected device (4 available)
[✓] Network resources

Hardware: MacBook M3

Currently the prebuilt library for Mac is Mac/Intel.  When I get a Mac/Arm available and setup, I will be switching it over.
In the meantime you can compile Mac/Arm using the build script: https://github.com/instantiations/es_compression/blob/master/tool/blob_builder/CMakeLists.txt


Master -> Live

Docs Build status updates of commit 62723d0:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

mixed-reality-docs/direct-manipulation.md
:white_check_mark:Succeeded
View

mixed-reality-docs/gaze-and-dwell.md
:white_check_mark:Succeeded
View

mixed-reality-docs/hands-and-tools.md
:white_check_mark:Succeeded
View

mixed-reality-docs/mrlearning-base-ch1.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


[Feature] Add ExecReload to systemd service

Use case
This will allow reloading ACLs via systemctl reload headscale
Description
Add ExecReload=/bin/kill -HUP $MAINPID to systemd service.
Contribution

[X] I can write the design doc for this feature
[X] I can contribute this feature

How can it be implemented?
Add ExecReload=/bin/kill -HUP $MAINPID to systemd service.

That would be great.


Data Table in kichen sink crashes the app when Scrolled on

Description of the Bug
Why does the data Table cause the app to crash when I scroll on it (I was about using it on my project when I just decided to test the one in kitchen sink)
Code and Logs(Below is the final error Traceback I get)
 File "kivy/weakproxy.pyx", line 299, in kivy.weakproxy.WeakProxy.__str__
 RecursionError: maximum recursion depth exceeded while getting the str of an object

Screenshots
Add images to explain us this bug. Paste urls here.
Remove this section if no images here
Versions

OS:
Python:
Kivy:
KivyMD:

@kengoon Unfortunately, I was unable to reproduce the bug you are describing ...

I also got the same error in my app. I couldn't even scroll down too

@kengoon @hemarle To find out the cause of the error, you must first specify the specifications:

You also need to attach a minimal example of code that reproduces the error you are describing. If the bug occurs on an Android device, attach the APK package for testing.

My Version:
OS: Windows 8
Python: 3.7
Kivy: 1.11.1
KivyMD: 0.104.2
The py file:
from kivymd.app import MDApp
from kivymd.uix.screen import Screen
from kivymd.uix.datatables import MDDataTable
from kivy.metrics import dp
class MyApp(MDApp):
def build(self):
self.screen = Screen()
data = MDDataTable(column_data=[('Column1', dp(30)), ('Column2', dp(30))],
row_data=[('boy', 'girl'), ('boy', 'girl'), ('boy', 'girl'), ('boy', 'girl'),
('boy', 'girl'), ('boy', 'girl'), ('boy', 'girl'), ('boy', 'girl'),
('boy', 'girl'), ('boy', 'girl'), ('boy', 'girl'), ('boy', 'girl'),
('boy', 'girl'), ('boy', 'girl'), ('boy', 'girl'), ('boy', 'girl'),
('boy', 'girl'), ('boy', 'girl'), ('boy', 'girl'), ('boy', 'girl'),
('boy', 'girl'), ('boy', 'girl'), ('boy', 'girl'), ('boy', 'girl'),
('boy', 'girl'), ('boy', 'girl'), ('boy', 'girl'), ('boy', 'girl'),
('boy', 'girl'), ('boy', 'girl'), ('boy', 'girl'), ],
pos_hint={'center_x':0.5,'center_y':0.5},
rows_num=30
                       )
    self.screen.add_widget(data)
    return self.screen

MyApp().run()

The indentation is actually correct, it got mumbled up when I copied and paste it. The code ran but crashed after I tried clicking (especially when double clicked) and I also can't scroll down

@HeaTTheatR Your error is due to the stiffscroll module not working correctly in Windows.

@HeaTTheatR
OS: Parrot Linux
Python: 3.7.7
Kivy: 1.11.1
Kivymd: 1.104.2.dev

@kengoon
This StiffScroll bug seems to be fixed in kivy 2.0.0rc3.
Updated my kivy, and my code don't produce that error anymore.

Updating Kivy to 2.0.0rc3