GEMINI errors when running the pipeline (143 columns but 155 values were supplied)

Using today's master, data should be fully up to date.
Some samples are failing with errors like these:
Exception: Attempted workaround for SQLite locking issue on NFS drives has failed. One possible reason is that the temp directory /home/shared-tmp/bcbiotx/a7a9714e-1708-4537-9a5f-7e31be42038a/tmpiEKzGt is also on an NFS drive. Error messages f
rom SQLite: table variants has 143 columns but 155 values were supplied table variants has 143 columns but 155 values were supplied
```
(/home/shared-tmp is local for each node, BTW)

Luca;
Sorry about the issues. That's definitely unexpected and I'm not sure what would cause this. Are you running GEMINI 0.17.0 or a different version? Is there any way you could provide a test case that demonstrates this so we can dig into it further? Simple tests with GEMINI here work fine but I could be failing to replicate due to not understanding the underlying cause. Thanks much for reporting this and hope this helps some.

I have update to the develp version of bcbio_netgen. I obtain the same error message.
I can share with you one bam file producing the issue
Thanks!
[2015-09-11T17:05Z] Timing: coverage
[2015-09-11T17:05Z] Timing: structural variation initial
[2015-09-11T17:05Z] Resource requests: freebayes, gatk, gatk-haplotype, mutect, picard, vardict; memory: 2.00, 3.50, 3.50, 2.50, 3.50, 3.00; cores: 16, 16, 16, 1, 1, 1
[2015-09-11T17:05Z] Configuring 6 jobs to run, using 1 cores each with 3.50g of memory reserved for each job
[2015-09-11T17:05Z] run local -- checkpoint passed: full
[2015-09-11T17:05Z] Timing: alignment post-processing
[2015-09-11T17:05Z] multiprocessing: piped_bamprep
[2015-09-11T17:05Z] Timing: variant calling
[2015-09-11T17:05Z] multiprocessing: variantcall_sample
[2015-09-11T17:06Z] multiprocessing: concat_variant_files
[2015-09-11T17:06Z] Resource requests: bamtools, bcbio_variation, fastqc, gatk, gatk-vqsr, gemini, qsignature, samtools; memory: 2.00, 0.42, 2.00, 3.50, 3.50, 2.00, 0.33, 2.00; cores: 16, 1, 16, 16, 16, 16, 1, 16
[2015-09-11T17:06Z] Configuring 1 jobs to run, using 6 cores each with 21.1g of memory reserved for each job
[2015-09-11T17:06Z] Timing: joint squaring off/backfilling
[2015-09-11T17:06Z] Timing: variant post-processing
[2015-09-11T17:06Z] multiprocessing: postprocess_variants
[2015-09-11T17:06Z] Finalizing variant calls: 60, freebayes
[2015-09-11T17:06Z] Calculating variation effects for 60, freebayes
[2015-09-11T17:06Z] Filtering for 60, freebayes
[2015-09-11T17:06Z] Prioritization for 60, freebayes
[2015-09-11T17:06Z] Create gemini database for /ifs/e63data/sander-lab/pastore/project/HZL/hzl_noaln/work/freebayes/60T-vepeffects-ploidyfix-filter.vcf.gz : 60
[2015-09-11T17:06Z] CADD scores are being loaded (to skip use:--skip-cadd).
[2015-09-11T17:07Z]
[2015-09-11T17:07Z] warning: variant with multiple alternate alleles found.
[2015-09-11T17:07Z]          in order to reduce the number of false negatives
[2015-09-11T17:07Z]          we recommend to split multiple alts. see:                 http://gemini.readthedocs.org/en/latest/content/preprocessing.html#preprocess
[2015-09-11T17:07Z]
[2015-09-11T17:07Z] warning: variant with multiple alternate alleles found.
[2015-09-11T17:07Z]          in order to reduce the number of false negatives
[2015-09-11T17:07Z]          we recommend to split multiple alts. see:                 http://gemini.readthedocs.org/en/latest/content/preprocessing.html#preprocess
[2015-09-11T17:07Z] pid 24854: 249 variants processed.
[2015-09-11T17:07Z] pid 24854: 15 skipped due to having the FILTER field set.
[2015-09-11T17:07Z] pid 24848: 250 variants processed.
[2015-09-11T17:07Z] pid 24848: 14 skipped due to having the FILTER field set.
[2015-09-11T17:07Z] pid 24863: 250 variants processed.
[2015-09-11T17:07Z] pid 24863: 15 skipped due to having the FILTER field set.
[2015-09-11T17:07Z] pid 24860: 250 variants processed.
[2015-09-11T17:07Z] pid 24860: 14 skipped due to having the FILTER field set.
[2015-09-11T17:07Z] pid 24857: 252 variants processed.
[2015-09-11T17:07Z] pid 24857: 12 skipped due to having the FILTER field set.
[2015-09-11T17:07Z] pid 24851: 249 variants processed.
[2015-09-11T17:07Z] pid 24851: 15 skipped due to having the FILTER field set.
[2015-09-11T17:13Z] sqlite3.OperationalError: table variants has 143 columns but 155 values were supplied
[2015-09-11T17:13Z] sqlite3.OperationalError: table variants has 143 columns but 155 values were supplied
[2015-09-11T17:13Z] sqlite3.OperationalError: table variants has 143 columns but 155 values were supplied
[2015-09-11T17:13Z] sqlite3.OperationalError: table variants has 143 columns but 155 values were supplied
[2015-09-11T17:13Z] Traceback (most recent call last):
[2015-09-11T17:13Z]   File "/home/pastore/data/bcbio/anaconda/bin/gemini", line 6, in
[2015-09-11T17:13Z]     gemini.gemini_main.main()
[2015-09-11T17:13Z]   File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 1155, in main
[2015-09-11T17:13Z] sqlite3.OperationalError: table variants has 143 columns but 155 values were supplied
[2015-09-11T17:13Z] Traceback (most recent call last):
[2015-09-11T17:13Z]   File "/home/pastore/data/bcbio/anaconda/bin/gemini", line 6, in
[2015-09-11T17:13Z]     gemini.gemini_main.main()
[2015-09-11T17:13Z]   File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 1155, in main
[2015-09-11T17:13Z]     args.func(parser, args)
[2015-09-11T17:13Z]   File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 312, in mergechunk_fn
[2015-09-11T17:13Z]     args.func(parser, args)
[2015-09-11T17:13Z]   File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 312, in mergechunk_fn
[2015-09-11T17:13Z]     gemini_merge_chunks.merge_chunks(parser, args)
[2015-09-11T17:13Z]   File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_merge_chunks.py", line 238, in merge_chunks
[2015-09-11T17:13Z]     gemini_merge_chunks.merge_chunks(parser, args)
[2015-09-11T17:13Z]   File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_merge_chunks.py", line 238, in merge_chunks
[2015-09-11T17:13Z]     % (args.tempdir, " ".join(errors)))
[2015-09-11T17:13Z] Exception: Attempted workaround for SQLite locking issue on NFS drives has failed. One possible reason is that the temp directory /ifs/e63data/sander-lab/pastore/project/HZL/hzl_noaln/work/freebayes/tx/tmpSWfaik is also on an NFS drive. Error messages from SQLite: table variants has 143 columns but 155 values were supplied table variants has 143 columns but 155 values were supplied
[2015-09-11T17:13Z]     % (args.tempdir, " ".join(errors)))
[2015-09-11T17:13Z] Exception: Attempted workaround for SQLite locking issue on NFS drives has failed. One possible reason is that the temp directory /ifs/e63data/sander-lab/pastore/project/HZL/hzl_noaln/work/freebayes/tx/tmpSWfaik is also on an NFS drive. Error messages from SQLite: table variants has 143 columns but 155 values were supplied table variants has 143 columns but 155 values were supplied
[2015-09-11T17:13Z] sqlite3.OperationalError: table variants has 143 columns but 155 values were supplied
[2015-09-11T17:13Z] Traceback (most recent call last):
[2015-09-11T17:13Z]   File "/home/pastore/data/bcbio/anaconda/bin/gemini", line 6, in
[2015-09-11T17:13Z]     gemini.gemini_main.main()
[2015-09-11T17:13Z]   File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 1155, in main
[2015-09-11T17:13Z]     args.func(parser, args)
[2015-09-11T17:13Z]   File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 312, in mergechunk_fn
[2015-09-11T17:13Z]     gemini_merge_chunks.merge_chunks(parser, args)
[2015-09-11T17:13Z]   File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_merge_chunks.py", line 238, in merge_chunks
[2015-09-11T17:13Z]     % (args.tempdir, " ".join(errors)))
[2015-09-11T17:13Z] Exception: Attempted workaround for SQLite locking issue on NFS drives has failed. One possible reason is that the temp directory /ifs/e63data/sander-lab/pastore/project/HZL/hzl_noaln/work/freebayes/tx/tmpSWfaik is also on an NFS drive. Error messages from SQLite: table variants has 143 columns but 155 values were supplied table variants has 143 columns but 155 values were supplied
[2015-09-11T17:13Z] Indexing /ifs/e63data/sander-lab/pastore/project/HZL/hzl_noaln/work/freebayes/60T-vepeffects-ploidyfix-filter.vcf.gz with grabix.
[2015-09-11T17:13Z] Loading 1585 variants.
[2015-09-11T17:13Z] Breaking /ifs/e63data/sander-lab/pastore/project/HZL/hzl_noaln/work/freebayes/60T-vepeffects-ploidyfix-filter.vcf.gz into 6 chunks.
[2015-09-11T17:13Z] Loading chunk 0.
[2015-09-11T17:13Z] Loading chunk 1.
[2015-09-11T17:13Z] Loading chunk 2.
[2015-09-11T17:13Z] Loading chunk 3.
[2015-09-11T17:13Z] Loading chunk 4.
[2015-09-11T17:13Z] Loading chunk 5.
[2015-09-11T17:13Z] Done loading 1585 variants in 6 chunks.
[2015-09-11T17:13Z] 2015-09-11 13:13:36 merging 6 chunks.
[2015-09-11T17:13Z] Traceback (most recent call last):
[2015-09-11T17:13Z]   File "/home/pastore/data/bcbio/anaconda/bin/gemini", line 6, in
[2015-09-11T17:13Z]     gemini.gemini_main.main()
[2015-09-11T17:13Z]   File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 1155, in main
[2015-09-11T17:13Z]     args.func(parser, args)
[2015-09-11T17:13Z]   File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 192, in load_fn
[2015-09-11T17:13Z]     gemini_load.load(parser, args)
[2015-09-11T17:13Z]   File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_load.py", line 49, in load
[2015-09-11T17:13Z]     load_multicore(args)
[2015-09-11T17:13Z]   File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_load.py", line 75, in load_multicore
[2015-09-11T17:13Z]     merge_chunks_multicore(chunks, args)
[2015-09-11T17:13Z]   File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_load.py", line 149, in merge_chunks_multicore
[2015-09-11T17:13Z]     wait_until_finished(procs)
[2015-09-11T17:13Z]   File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_load.py", line 334, in wait_until_finished
[2015-09-11T17:13Z]     raise ValueError("Processing failed on GEMINI chunk load")
[2015-09-11T17:13Z] ValueError: Processing failed on GEMINI chunk load
[2015-09-11T17:13Z] Uncaught exception occurred
Traceback (most recent call last):
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/bcbio/provenance/do.py", line 21, in run
_do_run(cmd, checks, log_stdout)
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/bcbio/provenance/do.py", line 95, in _do_run
raise subprocess.CalledProcessError(exitcode, error_msg)
CalledProcessError: Command 'set -o pipefail; /home/pastore/data/bcbio/anaconda/bin/gemini  load  --passonly --skip-gerp-bp  -v /ifs/e63data/sander-lab/pastore/project/HZL/hzl_noaln/work/freebayes/60T-vepeffects-ploidyfix-filter.vcf.gz -t VEP --cores 6 --tempdir /ifs/e63data/sander-lab/pastore/project/HZL/hzl_noaln/work/freebayes/tx/tmpSWfaik /ifs/e63data/sander-lab/pastore/project/HZL/hzl_noaln/work/freebayes/tx/tmpSWfaik/60T-vepeffects-ploidyfix-filter.db
CADD scores are being loaded (to skip use:--skip-cadd).
warning: variant with multiple alternate alleles found.
in order to reduce the number of false negatives
we recommend to split multiple alts. see:                 http://gemini.readthedocs.org/en/latest/content/preprocessing.html#preprocess
warning: variant with multiple alternate alleles found.
in order to reduce the number of false negatives
we recommend to split multiple alts. see:                 http://gemini.readthedocs.org/en/latest/content/preprocessing.html#preprocess
pid 24854: 249 variants processed.
pid 24854: 15 skipped due to having the FILTER field set.
pid 24848: 250 variants processed.
pid 24848: 14 skipped due to having the FILTER field set.
pid 24863: 250 variants processed.
pid 24863: 15 skipped due to having the FILTER field set.
pid 24860: 250 variants processed.
pid 24860: 14 skipped due to having the FILTER field set.
pid 24857: 252 variants processed.
pid 24857: 12 skipped due to having the FILTER field set.
pid 24851: 249 variants processed.
pid 24851: 15 skipped due to having the FILTER field set.
sqlite3.OperationalError: table variants has 143 columns but 155 values were supplied
sqlite3.OperationalError: table variants has 143 columns but 155 values were supplied
sqlite3.OperationalError: table variants has 143 columns but 155 values were supplied
sqlite3.OperationalError: table variants has 143 columns but 155 values were supplied
Traceback (most recent call last):
File "/home/pastore/data/bcbio/anaconda/bin/gemini", line 6, in
gemini.gemini_main.main()
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 1155, in main
sqlite3.OperationalError: table variants has 143 columns but 155 values were supplied
Traceback (most recent call last):
File "/home/pastore/data/bcbio/anaconda/bin/gemini", line 6, in
gemini.gemini_main.main()
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 1155, in main
args.func(parser, args)
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 312, in mergechunk_fn
args.func(parser, args)
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 312, in mergechunk_fn
gemini_merge_chunks.merge_chunks(parser, args)
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_merge_chunks.py", line 238, in merge_chunks
gemini_merge_chunks.merge_chunks(parser, args)
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_merge_chunks.py", line 238, in merge_chunks
% (args.tempdir, " ".join(errors)))
Exception: Attempted workaround for SQLite locking issue on NFS drives has failed. One possible reason is that the temp directory /ifs/e63data/sander-lab/pastore/project/HZL/hzl_noaln/work/freebayes/tx/tmpSWfaik is also on an NFS drive. Error messages from SQLite: table variants has 143 columns but 155 values were supplied table variants has 143 columns but 155 values were supplied
% (args.tempdir, " ".join(errors)))
Exception: Attempted workaround for SQLite locking issue on NFS drives has failed. One possible reason is that the temp directory /ifs/e63data/sander-lab/pastore/project/HZL/hzl_noaln/work/freebayes/tx/tmpSWfaik is also on an NFS drive. Error messages from SQLite: table variants has 143 columns but 155 values were supplied table variants has 143 columns but 155 values were supplied
sqlite3.OperationalError: table variants has 143 columns but 155 values were supplied
Traceback (most recent call last):
File "/home/pastore/data/bcbio/anaconda/bin/gemini", line 6, in
gemini.gemini_main.main()
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 1155, in main
args.func(parser, args)
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 312, in mergechunk_fn
gemini_merge_chunks.merge_chunks(parser, args)
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_merge_chunks.py", line 238, in merge_chunks
% (args.tempdir, " ".join(errors)))
Exception: Attempted workaround for SQLite locking issue on NFS drives has failed. One possible reason is that the temp directory /ifs/e63data/sander-lab/pastore/project/HZL/hzl_noaln/work/freebayes/tx/tmpSWfaik is also on an NFS drive. Error messages from SQLite: table variants has 143 columns but 155 values were supplied table variants has 143 columns but 155 values were supplied
Indexing /ifs/e63data/sander-lab/pastore/project/HZL/hzl_noaln/work/freebayes/60T-vepeffects-ploidyfix-filter.vcf.gz with grabix.
Loading 1585 variants.
Breaking /ifs/e63data/sander-lab/pastore/project/HZL/hzl_noaln/work/freebayes/60T-vepeffects-ploidyfix-filter.vcf.gz into 6 chunks.
Loading chunk 0.
Loading chunk 1.
Loading chunk 2.
Loading chunk 3.
Loading chunk 4.
Loading chunk 5.
Done loading 1585 variants in 6 chunks.
2015-09-11 13:13:36 merging 6 chunks.
Traceback (most recent call last):
File "/home/pastore/data/bcbio/anaconda/bin/gemini", line 6, in
gemini.gemini_main.main()
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 1155, in main
args.func(parser, args)
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 192, in load_fn
gemini_load.load(parser, args)
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_load.py", line 49, in load
load_multicore(args)
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_load.py", line 75, in load_multicore
merge_chunks_multicore(chunks, args)
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_load.py", line 149, in merge_chunks_multicore
wait_until_finished(procs)
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_load.py", line 334, in wait_until_finished
raise ValueError("Processing failed on GEMINI chunk load")
ValueError: Processing failed on GEMINI chunk load
' returned non-zero exit status 1
Traceback (most recent call last):
File "/home/pastore/data/bcbio/anaconda/bin/bcbio_nextgen.py", line 226, in
main(**kwargs)
File "/home/pastore/data/bcbio/anaconda/bin/bcbio_nextgen.py", line 43, in main
run_main(**kwargs)
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/main.py", line 37, in run_main
fc_dir, run_info_yaml)
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/main.py", line 80, in _run_toplevel
for xs in pipeline.run(config, run_info_yaml, parallel, dirs, samples):
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/main.py", line 174, in run
samples = run_parallel("postprocess_variants", samples)
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/bcbio/distributed/multi.py", line 28, in run_parallel
return run_multicore(fn, items, config, parallel=parallel)
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/bcbio/distributed/multi.py", line 86, in run_multicore
for data in joblib.Parallel(parallel["num_jobs"])(joblib.delayed(fn)(x) for x in items):
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/joblib/parallel.py", line 657, in call
self.dispatch(function, args, kwargs)
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/joblib/parallel.py", line 404, in dispatch
job = ImmediateApply(func, args, kwargs)
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/joblib/parallel.py", line 142, in init
self.results = func(*args, **kwargs)
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/bcbio/utils.py", line 49, in wrapper
return apply(f, *args, **kwargs)
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/bcbio/distributed/multitasks.py", line 90, in postprocess_variants
return variation.postprocess_variants(*args)
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/variation.py", line 28, in postprocess_variants
data["vrn_file"] = prioritize.handle_vcf_calls(data["vrn_file"], data)
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/bcbio/variation/prioritize.py", line 33, in handle_vcf_calls
gemini_db = population.create_gemini_db(vcf_file, data)
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/bcbio/variation/population.py", line 79, in create_gemini_db
do.run(cmd, "Create gemini database for %s" % gemini_vcf, data)
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/bcbio/provenance/do.py", line 21, in run
_do_run(cmd, checks, log_stdout)
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/bcbio/provenance/do.py", line 95, in _do_run
raise subprocess.CalledProcessError(exitcode, error_msg)
subprocess.CalledProcessError: Command 'set -o pipefail; /home/pastore/data/bcbio/anaconda/bin/gemini  load  --passonly --skip-gerp-bp  -v /ifs/e63data/sander-lab/pastore/project/HZL/hzl_noaln/work/freebayes/60T-vepeffects-ploidyfix-filter.vcf.gz -t VEP --cores 6 --tempdir /ifs/e63data/sander-lab/pastore/project/HZL/hzl_noaln/work/freebayes/tx/tmpSWfaik /ifs/e63data/sander-lab/pastore/project/HZL/hzl_noaln/work/freebayes/tx/tmpSWfaik/60T-vepeffects-ploidyfix-filter.db
CADD scores are being loaded (to skip use:--skip-cadd).
warning: variant with multiple alternate alleles found.
in order to reduce the number of false negatives
we recommend to split multiple alts. see:                 http://gemini.readthedocs.org/en/latest/content/preprocessing.html#preprocess
warning: variant with multiple alternate alleles found.
in order to reduce the number of false negatives
we recommend to split multiple alts. see:                 http://gemini.readthedocs.org/en/latest/content/preprocessing.html#preprocess
pid 24854: 249 variants processed.
pid 24854: 15 skipped due to having the FILTER field set.
pid 24848: 250 variants processed.
pid 24848: 14 skipped due to having the FILTER field set.
pid 24863: 250 variants processed.
pid 24863: 15 skipped due to having the FILTER field set.
pid 24860: 250 variants processed.
pid 24860: 14 skipped due to having the FILTER field set.
pid 24857: 252 variants processed.
pid 24857: 12 skipped due to having the FILTER field set.
pid 24851: 249 variants processed.
pid 24851: 15 skipped due to having the FILTER field set.
sqlite3.OperationalError: table variants has 143 columns but 155 values were supplied
sqlite3.OperationalError: table variants has 143 columns but 155 values were supplied
sqlite3.OperationalError: table variants has 143 columns but 155 values were supplied
sqlite3.OperationalError: table variants has 143 columns but 155 values were supplied
Traceback (most recent call last):
File "/home/pastore/data/bcbio/anaconda/bin/gemini", line 6, in
gemini.gemini_main.main()
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 1155, in main
sqlite3.OperationalError: table variants has 143 columns but 155 values were supplied
Traceback (most recent call last):
File "/home/pastore/data/bcbio/anaconda/bin/gemini", line 6, in
gemini.gemini_main.main()
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 1155, in main
args.func(parser, args)
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 312, in mergechunk_fn
args.func(parser, args)
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 312, in mergechunk_fn
gemini_merge_chunks.merge_chunks(parser, args)
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_merge_chunks.py", line 238, in merge_chunks
gemini_merge_chunks.merge_chunks(parser, args)
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_merge_chunks.py", line 238, in merge_chunks
% (args.tempdir, " ".join(errors)))
Exception: Attempted workaround for SQLite locking issue on NFS drives has failed. One possible reason is that the temp directory /ifs/e63data/sander-lab/pastore/project/HZL/hzl_noaln/work/freebayes/tx/tmpSWfaik is also on an NFS drive. Error messages from SQLite: table variants has 143 columns but 155 values were supplied table variants has 143 columns but 155 values were supplied
% (args.tempdir, " ".join(errors)))
Exception: Attempted workaround for SQLite locking issue on NFS drives has failed. One possible reason is that the temp directory /ifs/e63data/sander-lab/pastore/project/HZL/hzl_noaln/work/freebayes/tx/tmpSWfaik is also on an NFS drive. Error messages from SQLite: table variants has 143 columns but 155 values were supplied table variants has 143 columns but 155 values were supplied
sqlite3.OperationalError: table variants has 143 columns but 155 values were supplied
Traceback (most recent call last):
File "/home/pastore/data/bcbio/anaconda/bin/gemini", line 6, in
gemini.gemini_main.main()
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 1155, in main
args.func(parser, args)
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 312, in mergechunk_fn
gemini_merge_chunks.merge_chunks(parser, args)
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_merge_chunks.py", line 238, in merge_chunks
% (args.tempdir, " ".join(errors)))
Exception: Attempted workaround for SQLite locking issue on NFS drives has failed. One possible reason is that the temp directory /ifs/e63data/sander-lab/pastore/project/HZL/hzl_noaln/work/freebayes/tx/tmpSWfaik is also on an NFS drive. Error messages from SQLite: table variants has 143 columns but 155 values were supplied table variants has 143 columns but 155 values were supplied
Indexing /ifs/e63data/sander-lab/pastore/project/HZL/hzl_noaln/work/freebayes/60T-vepeffects-ploidyfix-filter.vcf.gz with grabix.
Loading 1585 variants.
Breaking /ifs/e63data/sander-lab/pastore/project/HZL/hzl_noaln/work/freebayes/60T-vepeffects-ploidyfix-filter.vcf.gz into 6 chunks.
Loading chunk 0.
Loading chunk 1.
Loading chunk 2.
Loading chunk 3.
Loading chunk 4.
Loading chunk 5.
Done loading 1585 variants in 6 chunks.
2015-09-11 13:13:36 merging 6 chunks.
Traceback (most recent call last):
File "/home/pastore/data/bcbio/anaconda/bin/gemini", line 6, in
gemini.gemini_main.main()
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 1155, in main
args.func(parser, args)
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 192, in load_fn
gemini_load.load(parser, args)
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_load.py", line 49, in load
load_multicore(args)
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_load.py", line 75, in load_multicore
merge_chunks_multicore(chunks, args)
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_load.py", line 149, in merge_chunks_multicore
wait_until_finished(procs)
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_load.py", line 334, in wait_until_finished
raise ValueError("Processing failed on GEMINI chunk load")
ValueError: Processing failed on GEMINI chunk load
' returned non-zero exit status 1

Luca and Alessandro;
Sorry about these problems. I've been trying on some sample runs and am unable to replicate this. Apologies, would it be possible to get a minimal VCF file that reproduces the issue? You should be able to find the gemini load command run in log/bcbio-nextgen-commands.log. If you can create a smaller file that reproduces the issue from the commandline that would be a big help to try to isolate and fix the issue. Thanks again for the help debugging.

Hi Brad, this is the command that produce the issues I am reporting on bcbio v 0.9.3a0.
I can provide you a vcf file. You should be able to access the file a this link
https://drive.google.com/open?id=0B2qoICnODXDqV3hJQnUtaGpKd0E
Thanks!
Alessandro
2015-09-13T14:39Z] /home/pastore/data/bcbio/anaconda/bin/gemini  load  --passonly --skip-gerp-bp  -v /ifs/e63data/sander-lab/pastore/project/HZL/hzl_noaln/work/freebayes/60T-vepeffects-ploidyfix-filter.vcf.gz -t VEP --cores 12 --tempdir /ifs/e63data/sander-lab/pastore/project/HZL/hzl_noaln/work/freebayes/tx/tmpSLBapt /ifs/e63data/sander-lab/pastore/project/HZL/hzl_noaln/work/freebayes/tx/tmpSLBapt/60T-vepeffects-ploidyfix-filter.db
[2015-09-13T14:39Z] tabix index 60T-vepeffects-ploidyfix.vcf.gz
[2015-09-13T14:39Z] Hard filtering /ifs/e63data/sander-lab/pastore/project/HZL/hzl_noaln/work/freebayes/60T-vepeffects-ploidyfix.vcf.gz with (AF[0] <= 0.5 && (DP < 4 || (DP < 13 && %QUAL < 10))) || (AF[0] > 0.5 && (DP < 4 && %QUAL < 50)) : 60
[2015-09-13T14:39Z] tabix index 60T-vepeffects-ploidyfix-filter.vcf.gz
[2015-09-13T14:39Z] Prioritization for 60, freebayes
[2015-09-13T14:39Z] Create gemini database for /ifs/e63data/sander-lab/pastore/project/HZL/hzl_noaln/work/freebayes/60T-vepeffects-ploidyfix-filter.vcf.gz : 60
[2015-09-13T14:39Z] CADD scores are being loaded (to skip use:--skip-cadd).
[2015-09-13T14:39Z]
[2015-09-13T14:39Z] warning: variant with multiple alternate alleles found.
[2015-09-13T14:39Z]          in order to reduce the number of false negatives
[2015-09-13T14:39Z]          we recommend to split multiple alts. see:                 http://gemini.readthedocs.org/en/latest/content/preprocessing.html#preprocess
[2015-09-13T14:39Z]
[2015-09-13T14:39Z] warning: variant with multiple alternate alleles found.
[2015-09-13T14:39Z]          in order to reduce the number of false negatives
[2015-09-13T14:39Z]          we recommend to split multiple alts. see:                 http://gemini.readthedocs.org/en/latest/content/preprocessing.html#preprocess
[2015-09-13T14:39Z] pid 28741: 122 variants processed.
[2015-09-13T14:39Z] pid 28741: 11 skipped due to having the FILTER field set.
[2015-09-13T14:39Z] pid 28717: 122 variants processed.
[2015-09-13T14:39Z] pid 28717: 10 skipped due to having the FILTER field set.
[2015-09-13T14:39Z] pid 28720: 123 variants processed.
[2015-09-13T14:39Z] pid 28720: 9 skipped due to having the FILTER field set.
[2015-09-13T14:39Z] pid 28732: 121 variants processed.
[2015-09-13T14:39Z] pid 28732: 11 skipped due to having the FILTER field set.
[2015-09-13T14:39Z] pid 28708: 123 variants processed.
[2015-09-13T14:39Z] pid 28708: 9 skipped due to having the FILTER field set.
[2015-09-13T14:39Z] pid 28723: 126 variants processed.
[2015-09-13T14:39Z] pid 28723: 6 skipped due to having the FILTER field set.
[2015-09-13T14:39Z] pid 28738: 128 variants processed.
[2015-09-13T14:39Z] pid 28738: 4 skipped due to having the FILTER field set.
[2015-09-13T14:39Z] pid 28729: 125 variants processed.
[2015-09-13T14:39Z] pid 28729: 7 skipped due to having the FILTER field set.
[2015-09-13T14:39Z] pid 28711: 127 variants processed.
[2015-09-13T14:39Z] pid 28711: 5 skipped due to having the FILTER field set.
[2015-09-13T14:39Z] pid 28714: 127 variants processed.
[2015-09-13T14:39Z] pid 28714: 5 skipped due to having the FILTER field set.
[2015-09-13T14:39Z] pid 28735: 129 variants processed.
[2015-09-13T14:39Z] pid 28735: 3 skipped due to having the FILTER field set.
[2015-09-13T14:39Z] pid 28726: 127 variants processed.
[2015-09-13T14:39Z] pid 28726: 5 skipped due to having the FILTER field set.
[2015-09-13T14:41Z] sqlite3.OperationalError: table variants has 143 columns but 155 values were supplied
[2015-09-13T14:41Z] sqlite3.OperationalError: table variants has 143 columns but 155 values were supplied
[2015-09-13T14:41Z] sqlite3.OperationalError: table variants has 143 columns but 155 values were supplied
[2015-09-13T14:41Z] sqlite3.OperationalError: table variants has 143 columns but 155 values were supplied
[2015-09-13T14:41Z] sqlite3.OperationalError: table variants has 143 columns but 155 values were supplied
[2015-09-13T14:41Z] sqlite3.OperationalError: table variants has 143 columns but 155 values were supplied
[2015-09-13T14:41Z] sqlite3.OperationalError: table variants has 143 columns but 155 values were supplied
[2015-09-13T14:41Z] Traceback (most recent call last):
[2015-09-13T14:41Z]   File "/home/pastore/data/bcbio/anaconda/bin/gemini", line 6, in
[2015-09-13T14:41Z]     gemini.gemini_main.main()
[2015-09-13T14:41Z]   File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 1155, in main
[2015-09-13T14:41Z]     args.func(parser, args)
[2015-09-13T14:41Z]   File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 312, in mergechunk_fn
[2015-09-13T14:41Z]     gemini_merge_chunks.merge_chunks(parser, args)
[2015-09-13T14:41Z]   File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_merge_chunks.py", line 238, in merge_chunks
[2015-09-13T14:41Z] sqlite3.OperationalError: table variants has 143 columns but 155 values were supplied
[2015-09-13T14:41Z] sqlite3.OperationalError: table variants has 143 columns but 155 values were supplied
[2015-09-13T14:41Z] Traceback (most recent call last):
[2015-09-13T14:41Z]   File "/home/pastore/data/bcbio/anaconda/bin/gemini", line 6, in
[2015-09-13T14:41Z] Traceback (most recent call last):
[2015-09-13T14:41Z]   File "/home/pastore/data/bcbio/anaconda/bin/gemini", line 6, in
[2015-09-13T14:41Z]     gemini.gemini_main.main()
[2015-09-13T14:41Z]   File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 1155, in main
[2015-09-13T14:41Z]     gemini.gemini_main.main()
[2015-09-13T14:41Z]   File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 1155, in main
[2015-09-13T14:41Z]     args.func(parser, args)
[2015-09-13T14:41Z]   File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 312, in mergechunk_fn
[2015-09-13T14:41Z]     args.func(parser, args)
[2015-09-13T14:41Z]   File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 312, in mergechunk_fn
[2015-09-13T14:41Z]     gemini_merge_chunks.merge_chunks(parser, args)
[2015-09-13T14:41Z]     gemini_merge_chunks.merge_chunks(parser, args)
[2015-09-13T14:41Z]   File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_merge_chunks.py", line 238, in merge_chunks
[2015-09-13T14:41Z]   File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_merge_chunks.py", line 238, in merge_chunks
[2015-09-13T14:41Z]     % (args.tempdir, " ".join(errors)))
[2015-09-13T14:41Z] Exception:     % (args.tempdir, " ".join(errors)))
[2015-09-13T14:41Z] Attempted workaround for SQLite locking issue on NFS drives has failed. One possible reason is that the temp directory /ifs/e63data/sander-lab/pastore/project/HZL/hzl_noaln/work/freebayes/tx/tmpSLBapt is also on an NFS drive. Error messages from SQLite: table variants has 143 columns but 155 values were supplied table variants has 143 columns but 155 values were supplied
[2015-09-13T14:41Z] % (args.tempdir, " ".join(errors)))
[2015-09-13T14:41Z] Exception: Exception: Attempted workaround for SQLite locking issue on NFS drives has failed. One possible reason is that the temp directory /ifs/e63data/sander-lab/pastore/project/HZL/hzl_noaln/work/freebayes/tx/tmpSLBapt is also on an NFS drive. Error messages from SQLite: table variants has 143 columns but 155 values were supplied table variants has 143 columns but 155 values were supplied
[2015-09-13T14:41Z] Attempted workaround for SQLite locking issue on NFS drives has failed. One possible reason is that the temp directory /ifs/e63data/sander-lab/pastore/project/HZL/hzl_noaln/work/freebayes/tx/tmpSLBapt is also on an NFS drive. Error messages from SQLite: table variants has 143 columns but 155 values were supplied table variants has 143 columns but 155 values were supplied
[2015-09-13T14:41Z] sqlite3.OperationalError: table variants has 143 columns but 155 values were supplied
[2015-09-13T14:41Z] Traceback (most recent call last):
[2015-09-13T14:41Z]   File "/home/pastore/data/bcbio/anaconda/bin/gemini", line 6, in
[2015-09-13T14:41Z]     gemini.gemini_main.main()
[2015-09-13T14:41Z]   File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 1155, in main
[2015-09-13T14:41Z]     args.func(parser, args)
[2015-09-13T14:41Z]   File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 312, in mergechunk_fn
[2015-09-13T14:41Z]     gemini_merge_chunks.merge_chunks(parser, args)
[2015-09-13T14:41Z]   File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_merge_chunks.py", line 238, in merge_chunks
[2015-09-13T14:41Z]     % (args.tempdir, " ".join(errors)))
[2015-09-13T14:41Z] Exception: Attempted workaround for SQLite locking issue on NFS drives has failed. One possible reason is that the temp directory /ifs/e63data/sander-lab/pastore/project/HZL/hzl_noaln/work/freebayes/tx/tmpSLBapt is also on an NFS drive. Error messages from SQLite: table variants has 143 columns but 155 values were supplied table variants has 143 columns but 155 values were supplied
[2015-09-13T14:41Z] sqlite3.OperationalError: table variants has 143 columns but 155 values were supplied
[2015-09-13T14:41Z] Traceback (most recent call last):
[2015-09-13T14:41Z]   File "/home/pastore/data/bcbio/anaconda/bin/gemini", line 6, in
[2015-09-13T14:41Z]     gemini.gemini_main.main()
[2015-09-13T14:41Z]   File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 1155, in main
[2015-09-13T14:41Z]     args.func(parser, args)
[2015-09-13T14:41Z]   File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 312, in mergechunk_fn
[2015-09-13T14:41Z]     gemini_merge_chunks.merge_chunks(parser, args)
[2015-09-13T14:41Z]   File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_merge_chunks.py", line 238, in merge_chunks
[2015-09-13T14:41Z]     % (args.tempdir, " ".join(errors)))
[2015-09-13T14:41Z] Exception: Attempted workaround for SQLite locking issue on NFS drives has failed. One possible reason is that the temp directory /ifs/e63data/sander-lab/pastore/project/HZL/hzl_noaln/work/freebayes/tx/tmpSLBapt is also on an NFS drive. Error messages from SQLite: table variants has 143 columns but 155 values were supplied table variants has 143 columns but 155 values were supplied
[2015-09-13T14:41Z] sqlite3.OperationalError: table variants has 143 columns but 155 values were supplied
[2015-09-13T14:41Z] Traceback (most recent call last):
[2015-09-13T14:41Z]   File "/home/pastore/data/bcbio/anaconda/bin/gemini", line 6, in
[2015-09-13T14:41Z]     gemini.gemini_main.main()
[2015-09-13T14:41Z]   File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 1155, in main
[2015-09-13T14:41Z]     args.func(parser, args)
[2015-09-13T14:41Z]   File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 312, in mergechunk_fn
[2015-09-13T14:41Z]     gemini_merge_chunks.merge_chunks(parser, args)
[2015-09-13T14:41Z]   File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_merge_chunks.py", line 238, in merge_chunks
[2015-09-13T14:41Z]     % (args.tempdir, " ".join(errors)))
[2015-09-13T14:41Z] Exception: Attempted workaround for SQLite locking issue on NFS drives has failed. One possible reason is that the temp directory /ifs/e63data/sander-lab/pastore/project/HZL/hzl_noaln/work/freebayes/tx/tmpSLBapt is also on an NFS drive. Error messages from SQLite: table variants has 143 columns but 155 values were supplied table variants has 143 columns but 155 values were supplied
[2015-09-13T14:41Z] Indexing /ifs/e63data/sander-lab/pastore/project/HZL/hzl_noaln/work/freebayes/60T-vepeffects-ploidyfix-filter.vcf.gz with grabix.
[2015-09-13T14:41Z] Loading 1585 variants.
[2015-09-13T14:41Z] Breaking /ifs/e63data/sander-lab/pastore/project/HZL/hzl_noaln/work/freebayes/60T-vepeffects-ploidyfix-filter.vcf.gz into 12 chunks.
[2015-09-13T14:41Z] Loading chunk 0.
[2015-09-13T14:41Z] Loading chunk 1.
[2015-09-13T14:41Z] Loading chunk 2.
[2015-09-13T14:41Z] Loading chunk 3.
[2015-09-13T14:41Z] Loading chunk 4.
[2015-09-13T14:41Z] Loading chunk 5.
[2015-09-13T14:41Z] Loading chunk 6.
[2015-09-13T14:41Z] Loading chunk 7.
[2015-09-13T14:41Z] Loading chunk 8.
[2015-09-13T14:41Z] Loading chunk 9.
[2015-09-13T14:41Z] Loading chunk 10.
[2015-09-13T14:41Z] Loading chunk 11.
[2015-09-13T14:41Z] Done loading 1585 variants in 12 chunks.
[2015-09-13T14:41Z] 2015-09-13 10:41:37 merging 12 chunks.
[2015-09-13T14:41Z] Traceback (most recent call last):
[2015-09-13T14:41Z]   File "/home/pastore/data/bcbio/anaconda/bin/gemini", line 6, in
[2015-09-13T14:41Z]     gemini.gemini_main.main()
[2015-09-13T14:41Z]   File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 1155, in main
[2015-09-13T14:41Z]     args.func(parser, args)
[2015-09-13T14:41Z]   File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 192, in load_fn
[2015-09-13T14:41Z]     gemini_load.load(parser, args)
[2015-09-13T14:41Z]   File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_load.py", line 49, in load
[2015-09-13T14:41Z]     load_multicore(args)
[2015-09-13T14:41Z]   File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_load.py", line 75, in load_multicore
[2015-09-13T14:41Z]     merge_chunks_multicore(chunks, args)
[2015-09-13T14:41Z]   File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_load.py", line 149, in merge_chunks_multicore
[2015-09-13T14:41Z]     wait_until_finished(procs)
[2015-09-13T14:41Z]   File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_load.py", line 334, in wait_until_finished
[2015-09-13T14:41Z]     raise ValueError("Processing failed on GEMINI chunk load")
[2015-09-13T14:41Z] ValueError: Processing failed on GEMINI chunk load
[2015-09-13T14:41Z] Uncaught exception occurred
Traceback (most recent call last):
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/bcbio/provenance/do.py", line 21, in run
_do_run(cmd, checks, log_stdout)
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/bcbio/provenance/do.py", line 95, in _do_run
raise subprocess.CalledProcessError(exitcode, error_msg)
CalledProcessError: Command 'set -o pipefail; /home/pastore/data/bcbio/anaconda/bin/gemini  load  --passonly --skip-gerp-bp  -v /ifs/e63data/sander-lab/pastore/project/HZL/hzl_noaln/work/freebayes/60T-vepeffects-ploidyfix-filter.vcf.gz -t VEP --cores 12 --tempdir /ifs/e63data/sander-lab/pastore/project/HZL/hzl_noaln/work/freebayes/tx/tmpSLBapt /ifs/e63data/sander-lab/pastore/project/HZL/hzl_noaln/work/freebayes/tx/tmpSLBapt/60T-vepeffects-ploidyfix-filter.db
sqlite3.OperationalError: table variants has 143 columns but 155 values were supplied
sqlite3.OperationalError: table variants has 143 columns but 155 values were supplied
Traceback (most recent call last):
File "/home/pastore/data/bcbio/anaconda/bin/gemini", line 6, in
gemini.gemini_main.main()
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 1155, in main
args.func(parser, args)
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 312, in mergechunk_fn
gemini_merge_chunks.merge_chunks(parser, args)
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_merge_chunks.py", line 238, in merge_chunks
sqlite3.OperationalError: table variants has 143 columns but 155 values were supplied
sqlite3.OperationalError: table variants has 143 columns but 155 values were supplied
Traceback (most recent call last):
File "/home/pastore/data/bcbio/anaconda/bin/gemini", line 6, in
Traceback (most recent call last):
File "/home/pastore/data/bcbio/anaconda/bin/gemini", line 6, in
gemini.gemini_main.main()
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 1155, in main
gemini.gemini_main.main()
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 1155, in main
args.func(parser, args)
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 312, in mergechunk_fn
args.func(parser, args)
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 312, in mergechunk_fn
gemini_merge_chunks.merge_chunks(parser, args)
gemini_merge_chunks.merge_chunks(parser, args)
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_merge_chunks.py", line 238, in merge_chunks
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_merge_chunks.py", line 238, in merge_chunks
% (args.tempdir, " ".join(errors)))
Exception:     % (args.tempdir, " ".join(errors)))
Attempted workaround for SQLite locking issue on NFS drives has failed. One possible reason is that the temp directory /ifs/e63data/sander-lab/pastore/project/HZL/hzl_noaln/work/freebayes/tx/tmpSLBapt is also on an NFS drive. Error messages from SQLite: table variants has 143 columns but 155 values were supplied table variants has 143 columns but 155 values were supplied
% (args.tempdir, " ".join(errors)))
Exception: Exception: Attempted workaround for SQLite locking issue on NFS drives has failed. One possible reason is that the temp directory /ifs/e63data/sander-lab/pastore/project/HZL/hzl_noaln/work/freebayes/tx/tmpSLBapt is also on an NFS drive. Error messages from SQLite: table variants has 143 columns but 155 values were supplied table variants has 143 columns but 155 values were supplied
Attempted workaround for SQLite locking issue on NFS drives has failed. One possible reason is that the temp directory /ifs/e63data/sander-lab/pastore/project/HZL/hzl_noaln/work/freebayes/tx/tmpSLBapt is also on an NFS drive. Error messages from SQLite: table variants has 143 columns but 155 values were supplied table variants has 143 columns but 155 values were supplied
sqlite3.OperationalError: table variants has 143 columns but 155 values were supplied
Traceback (most recent call last):
File "/home/pastore/data/bcbio/anaconda/bin/gemini", line 6, in
gemini.gemini_main.main()
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 1155, in main
args.func(parser, args)
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 312, in mergechunk_fn
gemini_merge_chunks.merge_chunks(parser, args)
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_merge_chunks.py", line 238, in merge_chunks
% (args.tempdir, " ".join(errors)))
Exception: Attempted workaround for SQLite locking issue on NFS drives has failed. One possible reason is that the temp directory /ifs/e63data/sander-lab/pastore/project/HZL/hzl_noaln/work/freebayes/tx/tmpSLBapt is also on an NFS drive. Error messages from SQLite: table variants has 143 columns but 155 values were supplied table variants has 143 columns but 155 values were supplied
sqlite3.OperationalError: table variants has 143 columns but 155 values were supplied
Traceback (most recent call last):
File "/home/pastore/data/bcbio/anaconda/bin/gemini", line 6, in
gemini.gemini_main.main()
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 1155, in main
args.func(parser, args)
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 312, in mergechunk_fn
gemini_merge_chunks.merge_chunks(parser, args)
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_merge_chunks.py", line 238, in merge_chunks
% (args.tempdir, " ".join(errors)))
Exception: Attempted workaround for SQLite locking issue on NFS drives has failed. One possible reason is that the temp directory /ifs/e63data/sander-lab/pastore/project/HZL/hzl_noaln/work/freebayes/tx/tmpSLBapt is also on an NFS drive. Error messages from SQLite: table variants has 143 columns but 155 values were supplied table variants has 143 columns but 155 values were supplied
sqlite3.OperationalError: table variants has 143 columns but 155 values were supplied
Traceback (most recent call last):
File "/home/pastore/data/bcbio/anaconda/bin/gemini", line 6, in
gemini.gemini_main.main()
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 1155, in main
args.func(parser, args)
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 312, in mergechunk_fn
gemini_merge_chunks.merge_chunks(parser, args)
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_merge_chunks.py", line 238, in merge_chunks
% (args.tempdir, " ".join(errors)))
Exception: Attempted workaround for SQLite locking issue on NFS drives has failed. One possible reason is that the temp directory /ifs/e63data/sander-lab/pastore/project/HZL/hzl_noaln/work/freebayes/tx/tmpSLBapt is also on an NFS drive. Error messages from SQLite: table variants has 143 columns but 155 values were supplied table variants has 143 columns but 155 values were supplied
Indexing /ifs/e63data/sander-lab/pastore/project/HZL/hzl_noaln/work/freebayes/60T-vepeffects-ploidyfix-filter.vcf.gz with grabix.
Loading 1585 variants.
Breaking /ifs/e63data/sander-lab/pastore/project/HZL/hzl_noaln/work/freebayes/60T-vepeffects-ploidyfix-filter.vcf.gz into 12 chunks.
Loading chunk 0.
Loading chunk 1.
Loading chunk 2.
Loading chunk 3.
Loading chunk 4.
Loading chunk 5.
Loading chunk 6.
Loading chunk 7.
Loading chunk 8.
Loading chunk 9.
Loading chunk 10.
Loading chunk 11.
Done loading 1585 variants in 12 chunks.
2015-09-13 10:41:37 merging 12 chunks.
Traceback (most recent call last):
File "/home/pastore/data/bcbio/anaconda/bin/gemini", line 6, in
gemini.gemini_main.main()
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 1155, in main
args.func(parser, args)
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 192, in load_fn
gemini_load.load(parser, args)
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_load.py", line 49, in load
load_multicore(args)
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_load.py", line 75, in load_multicore
merge_chunks_multicore(chunks, args)
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_load.py", line 149, in merge_chunks_multicore
wait_until_finished(procs)
File "/home/pastore/data/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_load.py", line 334, in wait_until_finished
raise ValueError("Processing failed on GEMINI chunk load")
ValueError: Processing failed on GEMINI chunk load
' returned non-zero exit status 1

Alessandro;
Thanks for the file and additional details. This helped a ton to isolate the issue, which is due to extra annotation columns in VEP prepared scripts from bcbio. We load in additional CADD, LoF and other annotations, and there was a problem with this mechanism when running distributed loading with GEMINI 0.17.0. I pushed a fix for this to the development branch so you have a couple of choices for fixing this:

Update gemini to the development version (gemini update --devel)
Do annotations on your run with effects: snpeff instead of VEP.
Run on a single core to get through GEMINI loading (-n 1).

Once we roll out a new version of GEMINI this will hopefully work cleanly again without the need for these workarounds. Thanks much for the heads up and detailed debugging information.

Thanks Brad! I did not realize that the error was associated with VEP!
Sent from my iPhone

On Sep 15, 2015, at 5:55 AM, Brad Chapman<EMAIL_ADDRESS>wrote:
Closed #1011.
—
Reply to this email directly or view it on GitHub.


Add Grakn

https://github.com/graknlabs/awesome
Grakn is an open-source knowledge graph database with built-in rule-based inferencing
By submitting this pull request I confirm I've read and complied with the below requirements 🖖
Please read it multiple times. I spent a lot of time on these guidelines and most people miss a lot.
Requirements for your pull request
[x] Don't waste my time. Do a good job, adhere to all the guidelines, and be responsive.
[x] You have to review at least 2 other open pull requests.
Try to prioritize unreviewed PRs, but you can also add more comments to reviewed PRs. Go through the below list when reviewing. This requirement is meant to help make the Awesome project self-sustaining. Comment here which PRs you reviewed. You're expected to put a good effort into this and to be thorough. Look at previous PR reviews for inspiration. Just commenting “looks good” or simply marking the pull request as approved does not count! You have to actually point out mistakes or improvement suggestions.
[x] You have read and understood the instructions for creating a list.
[x] This pull request has a title in the format Add Name of List.
- ✅ Add Swift
- ✅ Add Software Architecture
- ❌ Update readme.md
- ❌ Add Awesome Swift
- ❌ Add swift
- ❌ add Swift
- ❌ Adding Swift
- ❌ Added Swift
[x] Your entry here should include a short description about the project/theme of the list. It should not describe the list itself. The first character should be uppercase and the description should end in a dot. It should be an objective description and not a tagline or marketing blurb.
- ✅ - [iOS](…) - Mobile operating system for Apple phones and tablets.
- ✅ - [Framer](…) - Prototyping interactive UI designs.
- ❌ - [iOS](…) - Resources and tools for iOS development.
- ❌ - [Framer](…)
- ❌ - [Framer](…) - prototyping interactive UI designs
[x] Your entry should be added at the bottom of the appropriate category.
[x] The suggested Awesome list complies with the below requirements.
Requirements for your Awesome list
[x] Has been around for at least 30 days.That means 30 days from either the first real commit or when it was open-sourced. Whatever is most recent.
[x] Don't open a Draft / WIP pull request while you work on the guidelines. A pull request should be 100% ready and should adhere to all the guidelines when you open it.
[x] Run awesome-lint on your list and fix the reported issues. If there are false-positives or things that cannot/shouldn't be fixed, please report it.
[x] The default branch should be named main, not master.
[x] Includes a succinct description of the project/theme at the top of the readme. (Example)
- ✅ Mobile operating system for Apple phones and tablets.
- ✅ Prototyping interactive UI designs.
- ❌ Resources and tools for iOS development.
- ❌ Awesome Framer packages and tools.
[x] It's the result of hard work and the best I could possibly produce.
If you have not put in considerable effort into your list, your pull request will be immediately closed.
[x] The repo name of your list should be in lowercase slug format: awesome-name-of-list.
- ✅ awesome-swift
- ✅ awesome-web-typography
- ❌ awesome-Swift
- ❌ AwesomeWebTypography
[x] The heading title of your list should be in title case format: # Awesome Name of List.
- ✅ # Awesome Swift
- ✅ # Awesome Web Typography
- ❌ # awesome-swift
- ❌ # AwesomeSwift
[x] Non-generated Markdown file in a GitHub repo.
[x] The repo should have awesome-list & awesome as GitHub topics. I encourage you to add more relevant topics.
[x] Not a duplicate. Please search for existing submissions.
[x] Only has awesome items. Awesome lists are curations of the best, not everything.
[x] Does not contain items that are unmaintained, has archived repo, deprecated, or missing docs. If you really need to include such items, they should be in a separate Markdown file.
[ ] Includes a project logo/illustration whenever possible.
- Either centered, fullwidth, or placed at the top-right of the readme. (Example)
- The image should link to the project website or any relevant website.
- The image should be high-DPI. Set it to maximum half the width of the original image.
[x] Entries have a description, unless the title is descriptive enough by itself. It rarely is though.
[x] Includes the Awesome badge.
[x] Should be placed on the right side of the readme heading.
- Can be placed centered if the list has a centered graphics header.
[x] Should link back to this list.
[x] Has a Table of Contents section.
[x] Should be named Contents, not Table of Contents.
[x] Should be the first section in the list.
[x] Should only have one level of nested lists, preferably none.
[x] Must not feature Contributing or Footnotes sections.
[x Has an appropriate license.
[x] We strongly recommend the CC0 license, but any Creative Commons license will work.
- Tip: You can quickly add it to your repo by going to this URL: https://github.com/<user>/<repo>/community/license/new?branch=main&template=cc0-1.0 (replace <user> and <repo> accordingly).
[x] A code license like MIT, BSD, Apache, GPL, etc, is not acceptable. Neither are WTFPL and Unlicense.
[x] Place a file named license or LICENSE in the repo root with the license text.
[x] Do not add the license name, text, or a Licence section to the readme. GitHub already shows the license name and link to the full text at the top of the repo.
- To verify that you've read all the guidelines, please comment on your pull request with just the word unicorn.
[x] Has contribution guidelines.
[x] The file should be named contributing.md. Casing is up to you.
[x] It can optionally be linked from the readme in a dedicated section titled Contributing, positioned at the top or bottom of the main content.
[x] The section should not appear in the Table of Contents.
[x] All non-important but necessary content (like extra copyright notices, hyperlinks to sources, pointers to expansive content, etc) should be grouped in a Footnotes section at the bottom of the readme. The section should not be present in the Table of Contents.
[x] Has consistent formatting and proper spelling/grammar.
[x] The link and description are separated by a dash. Example: - [AVA](…) - JavaScript test runner.
[x] The description starts with an uppercase character and ends with a period.
[x] Consistent and correct naming. For example, Node.js, not NodeJS or node.js.
[x] Doesn't use hard-wrapping.
[x] Doesn't include a Travis badge.You can still use Travis for list linting, but the badge has no value in the readme.
[x] Doesn't include an Inspired by awesome-foo or Inspired by the Awesome project kinda link at the top of the readme. The Awesome badge is enough.
Go to the top and read it again.

Thanks for making an Awesome list! 🙌
It looks like you didn't read the guidelines closely enough. I noticed multiple things that are not followed. Try going through the list point for point to ensure you follow it. I spent a lot of time creating the guidelines so I wouldn't have to comment on common mistakes, and rather spend my time improving Awesome.

Thanks for making an Awesome list! 🙌
It looks like you didn't read the guidelines closely enough. I noticed multiple things that are not followed. Try going through the list point for point to ensure you follow it. I spent a lot of time creating the guidelines so I wouldn't have to comment on common mistakes, and rather spend my time improving Awesome.

Reviews:
https://github.com/sindresorhus/awesome/pull/1926#issuecomment-766755640
https://github.com/sindresorhus/awesome/pull/1921#issuecomment-766759043

Reviews:
https://github.com/sindresorhus/awesome/pull/1926#issuecomment-766755640
https://github.com/sindresorhus/awesome/pull/1921#issuecomment-766759043

unicorn

unicorn

The list is a little light on content.

[x] Includes a succinct description of the project/theme at the top of the readme. (Example)

I would like to see more than just libraries. Include some blog posts, tutorials, links to community resources (Stack Overflow tag, IRC, chat, forums, communities, Reddit, etc. Example: https://github.com/sindresorhus/awesome-electron#community), videos of talks, book, learning material, etc.

@sindresorhus thank you for your review! I've tried to address both comments in https://github.com/graknlabs/awesome/commit/47475de8035ce36784fa3cc1169e2b40b821baa4

Include some blog posts, tutorials

Still not seeing any blog posts

Linkify Grakn in Grakn is a database; one that empowers complex systems to be more to its website.

@sindresorhus

Still not seeing any blog posts

We've linked our blog in Community section :slightly_smiling_face:

Linkify Grakn in Grakn is a database; one that empowers complex systems to be more to its website.

Done in https://github.com/graknlabs/awesome/commit/bf5b04c0d1b0f483416a96d28d9b61d6990dc1a5

@sindresorhus added some blogs and videos on https://github.com/graknlabs/awesome/commit/7d6f54be6dd1d1f4cfa779cbfc44267f0b70a50d


Toggle button

Question
how to do custom form (toggle button) if the toggle is turned on and submitted to execute the recipe?
トグルがオンになっていて、レシピが実行のために送信された場合、カスタムフォーム（トグルボタン）をどのように実行しますか？
Plugin Version
2.8.0

Set the trigger of the recipe to Form>(Recipename)> button

when you create the button you get a message like {form.data[0]} <- true ; false) you can use that I think with the script> add condition> script> compare to the character string.

Thanks but I don't understand after click "compare to the character string"

Thanks but I don't understand after click "compare to the character string"

what should i write in "compare to the character string"

Thanks :)

how to make "Input Form" Execute with the words we set how?

No problem


DataChannel open message missing

In current webrtc state dataChannel working only if they are opened from javascript.
Datachannel open from golang does nothing (even when I call SendOpenChannelMessage manually), also manual negoitaion is not possible

Yes, you are correct. We are actively working on filling these gaps in our implementation.
In the mean time: The data-channels-create example has a workaround for this that works with Chrome. (In my testing it doesn't work when connecting to NodeJS thought). On main.go#L42 you can see we manually call SendOpenChannelMessage when the ICEConnectionState changes to Connected. Again, this is obviously a temporary workaround until we can complete the underlying implementation.

Yeah, I've tried that workaround but without success

Hi @Bogdan-D, can you please test again with the last master branch. We're now sending the ChannelOpen messages when creating data channels in Pions. These changes also make Pions to Pions and Pions to NodeJS data channels work in my testing.

Thanks for implementing this @backkem  :)
Closing for now, feel free to re-open @Bogdan-D if it is still an issue, thanks!


Skipping file missing DWARF data

Expected behavior
Upload success
Observed behavior
Uploading files to https://upload.bugsnag.com
Preparing to upload /var/folders/zx/1j0vlwt55s53zzjtqy425v6m0000gn/T/bugsnag-dsym-upload.XXX.qsJitINp/*.dSYM
Skipping file missing DWARF data: /var/folders/zx/1j0vlwt55s53zzjtqy425v6m0000gn/T/bugsnag-dsym-upload.XXX.qsJitINp/*.dSYM
1 files failed to upload

Steps to reproduce

build iOS app using https://appcenter.ms
Download symbols.zip from appcenter
Run bugsnag-dsym-upload -v symbols.zip

Version
Commit 8327d1f4e9224bb83ee20ce56791209af0c8448a
Additional information

Hi @mifi
Could you contact<EMAIL_ADDRESS>with these details including a copy of your symbols.zip so we can take a look?

Hi @mifi
Thanks for sending in the zip file. It appears that the zip files downloaded from appcenter.ms differ from the format we expect which is the format from App Store Connect (formerly iTunes Connect).
Inside the symbols.zip file are one or more xxx.dSYM folders. If you extract these folders somewhere yourself and then use the upload tool on that containing folder:
bugsnag-dsym-upload path/to/dsyms

then it should work.
We plan to take a look at updating the tool to also work with dSYMs downloaded from appcenter.ms in due course.

Ok thanks. Nice if you could update the documentation about this for future confused devs :)

+1 -- I'm getting the same thing currently. I got this both with dSYM's that I got from iTunes Connect and from Xcode. Clarification on expected procedure would be great.

Ok. I've got it working. In my case it's not working if I reference a zip file. But it's working if I reference a folder.

what @crspybits said: this is no longer working with dSYMs downloaded from iTunes either:
    ..
    upload_to_testflight
    download_dsyms(version: "latest", wait_for_dsym_processing: true)
    upload_symbols_to_bugsnag
    ..

Logs say success:
[23:37:39]: ----------------------------
[23:37:39]: --- Step: download_dsyms ---
[23:37:39]: ----------------------------
[23:37:39]: Login to App Store Connect<EMAIL_ADDRESS>[23:37:41]: Login successful
[23:37:46]: Looking for latest build...
[23:37:47]: Looking for dSYM files for 'xyz' on platform IOS v4.3 (454)
[23:38:17]: Waiting for dSYM file to appear...
[23:38:48]: Waiting for dSYM file to appear...
[23:39:23]: 🔑  Successfully downloaded dSYM file for 4.3 - 454 to 'xyz-4.3-454.dSYM.zip'
[23:39:24]: ---------------------------------------
[23:39:24]: --- Step: upload_symbols_to_bugsnag ---
[23:39:24]: ---------------------------------------
[23:39:24]: Using the API Key from ./xyz/Info.plist
[23:39:30]: Uploaded dSYMs in /a/b/c/d/xyz-4.3-454.dSYM.zip

yet Bugsnag UI says missing dSYMs for the exact same build.

👋 . we've released https://github.com/bugsnag/bugsnag-dsym-upload/releases/tag/v2.1.0 of the bugsnag-dsym-upload tool which now takes care of zips, and directories with dSYMs at varying depth levels within. Let us know if you encounter any issues with this!


just a test1

Describe the bug
A clear and concise description of what the bug is. If you are developing a new diagnostic script, please provide a link to the code/branch on GitHub that you are working in.
Please attach

The recipe that you are trying to run, you can find a copy in the run directory in the output directory
The main_log_debug.txt file, this can also be found in the run directory in the output directory

my first comment

my second comment

@pleasetest everything

@esmvalbot bla

something


Conv Transpose doesn't exactly transpose

When checking if ConvTranspose is actually computing the transpose operation, it seems to be failing. I've tried different weight matrix shapes, but I'm uncertain as to why this is failing:
from jax import numpy as jnp
import jax
import equinox as eqx
key = jax.random.PRNGKey(0)
x = jax.random.uniform(key, shape=(2, 6, 6))
a = jax.random.uniform(jax.random.PRNGKey(24), shape=x.shape)

cnn = eqx.nn.Conv2d(in_channels=2, out_channels=2, kernel_size=3, stride=1, padding='SAME', key=key, use_bias=False)
cnn_t = eqx.nn.ConvTranspose2d(in_channels=2, out_channels=2, kernel_size=3, stride=1, padding='SAME', key=key, use_bias=False)
#cnn_t = eqx.tree_at(lambda x: x.weight, cnn_t, jnp.transpose(cnn.weight, (1, 0, 3, 2)))

fx = cnn(x)
ga = cnn_t(a)
dot_product_1 = jnp.sum(a * fx)
dot_product_2 = jnp.sum(x * ga)

assert jnp.allclose(dot_product_1, dot_product_2)

The following TF code succeeds as expected:
import tensorflow as tf
import numpy as np

x = tf.random.normal((1, 5, 5, 2))
a = tf.random.normal((1, 5, 5, 2))

filter_tensor = tf.random.normal((3, 3, 2, 2))

stride = 1
padding = 'SAME'
fx = tf.nn.conv2d(x, filter_tensor, strides=[1, stride, stride, 1], padding=padding)

u_height, u_width, u_depth = a.shape[1], a.shape[2], a.shape[3]
output_shape = [1, u_height, u_width, u_depth]

ga = tf.nn.conv2d_transpose(a, filter_tensor, output_shape=output_shape, strides=[1, stride, stride, 1], padding=padding)

dot_product_1 = tf.math.reduce_sum(a * fx)
dot_product_2 = tf.math.reduce_sum(x * ga)

assert np.allclose(dot_product_1, dot_product_2)

What's the best way to get the actual transpose of a convolution in equinox?

Very similar thread in flax: https://github.com/google/flax/issues/2577. However, their solution of manually transposing doesn't work: cnn_t = eqx.tree_at(lambda x: x.weight, cnn_t, jnp.flip(jnp.array(cnn.weight), (0, 1)).swapaxes(-2, -1)) still fails. And the approach of transpose_kernel=True doesn't work since conv_general_dilated is used not conv_transpose

Forgot that flax had weights reversed. cnn_t = eqx.tree_at(lambda x: x.weight, cnn_t, jnp.flip(jnp.array(cnn.weight), (2, 3)).swapaxes(0, 1)) Actually does work. Maybe the tooling for this should be documented somewhere? Or we could add it as a flag like flax did https://github.com/google/flax/pull/2578?

I think the flip + swapaxes combination would make sense to document somewhere.
I probably wouldn't add it as a flag. Transposed convolutions are already very complicated, I'd prefer not to add yet another complexity to them!
FWIW let's not forget that jax.linear_transpose exists. Back when I first wrote ConvTranspose I considered using jax.linear_transpose(Conv(...)) as its implementation. Sometimes I wonder if that's what I should have done!

Makes sense, these signatures could quickly be overwhelming. I will add it to the documentation.
I definitely did not forget about jax.linear_transpose, because I didn't even know it existed! So thanks for also brining that to my attention


Fire Projectile

Consider object pooling for the raycasts / projectiles for this feature. It will save allot of performance when allot of projectiles are on the screen.
https://www.raywenderlich.com/136091/object-pooling-unity

Added in commit b1b53f386c78d925e5d0942646b2d7ba1a31f92d. Opened #41 for projectile pools.

Awesome :D


GSSOC EXT NOTICE: Improper Labeling in the Project

Dear Project Admin @sharmavikas4
It has come to our attention that your repository not applying the necessary labels to issues and merged PRs, which is affecting contributor scores on the leaderboard.
Please ensure you use the correct labels:

gssoc-ext
level1, level2, level3
hacktoberfest-accepted (only if your repository have Hacktoberfest topic in description)

NOTE: Level labels should not have any dashes between them. Spaces are allowed, but no dashes should be used.
Kindly review and update the labels on your issues and PRs to ensure fairness for all contributors. Failure to address this may result in your project being flagged.
Thank you,
Core Team
GSSoC '24 Extended

Thanks for giving your feedback. I will take necessary steps
On Mon, 7 Oct, 2024, 11:40 github-actions[bot], @.***>
wrote:

👋 Thank you for raising an issue! We appreciate your effort in helping us
improve. Our MERN Blog team will review it shortly. Stay tuned!
—
Reply to this email directly, view it on GitHub
https://github.com/sharmavikas4/MERN_BLOG/issues/21#issuecomment-2395987404,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AYR74VEPPUPUFE63FJF643TZ2IQVXAVCNFSM6AAAAABPPEVGVWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGOJVHE4DONBQGQ
.
You are receiving this because you were mentioned.Message ID:
@.***>

@Meetjain1 can you check now whether the label are correct i have updated the labels.

@Meetjain1 can you check now whether the label are correct i have updated the labels.

Yes perfect ! @sharmavikas4


Incorrect comment for Google_Service_Appengine_AppsServicesResource::patch

In the comment of Google_Service_Appengine_AppsServicesResource::patch, the type of $postBody is Google_Service but Google_Service_Appengine_Service is a subclass of Google_Model

fixed in #10


Understanding the different sanitisation behaviour of the translate directive and filter

As fas as I see, the sanitisation of the translate directive and the filter can nearly never be equal if used in templates.
If you use the filter in an ng-bind expression like {{ 'HEADLINE' | translate }} the ng-bind will always escape the result. If you enable sanitisation in angular-translate, the result will be escaped twice (see http://plnkr.co/edit/gmkba8ksJPXIbQmkYqm6?p=preview).
With this behaviour it seems like it's never possible to use the directive and the filter in the same application. What are the thoughts behind this behaviour? Is it really advantageous?
I currently bypassed this behaviour with an additional translateRaw filter and a custom sanitizer. The filter tells my sanitizer to skip the next sanitisation and therefore the results of the directive and my filter are equal. Even if this gets the job done, it just doesn't feel right.

I've created a sheet some weeks ago at http://plnkr.co/edit/8BHONOjhsxBxxO9oVYOK?p=preview We aren't finished with this, any ideas are welcome!

With this behaviour it seems like it's never possible to use the directive and the filter in the same application.

The filter's value is escaped by AJS itself. We do not have this under control. If you use filters in AJS, you always should be aware of this.
Again: Suggestions and ideas are welcome, please have a look at the #1101


Allow disabling authorization

At the moment if auth is disabled, authorization checks fail. To fix that, a configurable authorizer component should be configured in the infrastructure which will authorize anyone if --authentication-disabled is passed.

Relevant JIRA ticket: https://issues.redhat.com/browse/TC-387


Evaluate TinyMCE editor

https://github.com/isotoma/wagtailtinymce

Enables:
#42
#38

The extensibility in wagtailtinymce is a little iffy, but I'll see if I can do something about it.

Evaluated and merged via #53; closing.


INTERSECT and EXCEPT don't return an error when 2 sets have the different number of columns

Describe the bug
A clear and concise description of what the bug is.
To Reproduce
Steps to reproduce the behavior:
❯ select 1 as a, 2 as b
intersect
select 1 as a;
+---+---+
| a | b |
+---+---+
| 1 | 2 |
+---+---+
1 row in set. Query took 0.009 seconds.
❯ select 1 as a, 2 as b
except
select 2 as a;
+---+---+
| a | b |
+---+---+
| 1 | 2 |
+---+---+

Expected behavior
A clear and concise description of what you expected to happen.
We should return an error, which is the behaviour in the Postgresql: https://onecompiler.com/postgresql/3yhau7n8a
Additional context
Add any other context about the problem here.

I looked into this issue and have some questions.
The below check already seems to do the job. The below function is called in the function intersect_or_except which seems to be the common function for intersect and expect. Is there any other place we need to look into?
https://github.com/apache/arrow-datafusion/blob/6d2b417def2d536e81eb80e082d9cc3ca4e7f5fc/datafusion/expr/src/logical_plan/builder.rs#L490-L495

I guess the problem is in the intersect_or_except where we should check the number of columns before we zip the the join_keys: https://github.com/apache/arrow-datafusion/blob/6d2b417def2d536e81eb80e082d9cc3ca4e7f5fc/datafusion/expr/src/logical_plan/builder.rs#L784-L794

I see. So the zip works when the iterators are of different size. I'll fix this.


feat(config): add HomeSys HomeMech-2001/2

Add new device config for HomeSys HomeMech-2001 and HomeMech-2002 dimmers

Here, let me fix that for you

Here, let me fix that for you

Amazing, thank you!  That lint error was breaking my brain as it wasn't throwing in the local env.
Appreciate you!

That lint error was breaking my brain as it wasn't throwing in the local env.

You probably ran the wrong check (semantic instead of just formatting). Command Palette -> Format Document would have done it btw.


TreeItem getActionButton results in a TypeError: Cannot read properties of null (reading 'indexOf')

I have run into an issue, not 100% sure that I have identified the root cause correctly yet though, so this may be a red-herring.
However, this change did help me move forward.
The getActionButton function in the TreeItem class in pageObjects/sidebar/viewItem.js appears to throw because it is unable to find an action Button.
Current:
    async getActionButton(label) {

        const actions = await this.getActionButtons()
        if (actions.length > 0) {
            return actions.find((item) => item.getLabel().indexOf(label) > -1)
        }
        return undefined
    }

Suggested Change:

    async getActionButton(label) {
        const actions = await this.getActionButtons()
        if (actions.length > 0) {
            for (let item of actions) {
                let itemLabel = await item.getLabel() ?? await item.elem.getAttribute('aria-label');
                if (itemLabel.indexOf(label) > -1) {
                    return item;
                }
            }
        }
        return undefined;
    }

This basically, checks both the label and the aria-label.
However, that begs the question, why (in my case) is it not finding the button by label currently? I am still not sure yet.
This is what the DOM ends up looking like. The label prop passed to getActionButton(label) is the text in the aria-label such as Action Item 1 or Action Item 2.

@tylor-metrics thanks for the fix. Please confirm if the change fixes the issue.

Closing due to inactivity.


Days of zero hits not visible on graphs

I'm trying to analyse some rarely occurring events with GoAccess. At first everything seemed to make sense but then I noticed that there was no days with 0 events visible. (Same applies to hours in the Time Distribution.)
I would expect such days (and hours) to report 0 hits instead of not being visible at all. This is especially important in the charts as not all data points have a label on the X axis.

Thanks for the feedback. Currently it outputs dates contained in your log. Thus if there's no data for a particular date, it won't show it. I can look into printing out those dates/times.

That would be great - thanks!


Implement buffered IO for distributed filesystems

This is a first pass at implementing buffered IO. There seem to be nice performance boosts for azure, will be running benchmarks to fine tune the upload and download chunk sizes, but solicit more ideas on what can be done here.

Codecov Report

Merging #72 (b2ba2b4) into develop (e50ebce) will increase coverage by 0.06%.
The diff coverage is 65.90%.

@@             Coverage Diff             @@
##           develop      #72      +/-   ##
===========================================
+ Coverage    61.31%   61.38%   +0.06%
===========================================
  Files           52       54       +2
  Lines        16786    16932     +146
===========================================
+ Hits         10293    10393     +100
- Misses        6493     6539      +46

Impacted Files
Coverage Δ

core/src/fragment/write_state.cc
66.26% <50.00%> (+0.65%)
:arrow_up:

core/src/storage_manager/storage_buffer.cc
50.00% <50.00%> (ø)

core/include/storage_manager/storage_buffer.h
71.42% <71.42%> (ø)

core/include/storage_manager/storage_fs.h
84.61% <83.33%> (-15.39%)
:arrow_down:

core/src/fragment/book_keeping.cc
62.30% <100.00%> (ø)

core/src/fragment/buffer.cc
78.57% <100.00%> (-0.90%)
:arrow_down:

core/src/fragment/read_state.cc
70.58% <100.00%> (+0.43%)
:arrow_up:

core/src/storage_manager/storage_azure_blob.cc
74.91% <100.00%> (+0.16%)
:arrow_up:

... and 5 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e50ebce...b2ba2b4. Read the comment docs.


Remove upper bound on lens with active-<IP_ADDRESS> (#2496)

I have also removed active-<IP_ADDRESS> watchpoint.

Despite the release of active-<IP_ADDRESS> (which I have unpacked) my attempts to run stackage curator locally gives me:
lens-4.15.2 (Edward Kmett<EMAIL_ADDRESS>@ekmett) is out of bounds for:
- [ ] active-<IP_ADDRESS> (>=4.0 && < 4.15.2). Brent Yorgey<EMAIL_ADDRESS>@byorgey. @bergey @byorgey @fryguybob @jeffreyrosenbluth. Used by: library, test-suite

So I am raising a PR.

Looks good, and travis is happy. Did you run stack update locally?

👍  all good


precompute tags generates redundant tags

Something splits reference subject role and just "role" into two separate tags

Dropping in to offer an eye:
That would be the expand tags stepping up the tree (reference subject role -> participant under investigation role -> role, Ontobee) (code responsible).
If it's not being dropped it's because there's (at least) one experiment annotated with "role" and not "reference subject role", otherwise no new unique chain would be found (code) and it would be filtered. I guess it's also possible that it expanded from something outside of Uberon too since "role" is used in quite a few ontologies (Ontobee).
Should be able to track this down by just grepling ("role") in cf.Base and cf.Val (or whatever they're called).

CHEBI: http://purl.obolibrary.org/obo/CHEBI_50906


PSUseApprovedVerbs doesn't like functions that start with an _

Greetings.
A while back another PowerShell script got me in the habit of placing the _ in front of the verb-noun used for function names. I asked him why he did that and he said it was so he could easily differentiate between true cmdlets and created functions that look like cmdlets.
Is there any way to tell PSScriptAnalyzer to overlook the preceeding "_" in a functions verb-noun checking?

@sheehand You can suppress all the violation instances of verb-noun functions that start with "_" by adding the following suppression statement before the param block in your script.
[Diagnostics.CodeAnalysis.SuppressMessage("PSUseApprovedVerbs", Scope="function", Target="_*" )]

I assume this has been answered and will close this issue for now. If you have any issues with this approach please feel free to reopen the issue or raise a new one.

Sorry for not responding - I don't think I got a notice that this was responded to for some reason, and I am running up against this today.
If I understand your code syntax, you are telling it to not perform the verb check at all for anything that starts with "". If so then that is not my desire. Specifically I would much rather have it still analyze the verb in the function name, just ignore/skip over the "". Is there no way to do that?

Hi, the original maintainer is not working on this repository any more, therefore I will jump in. The proposed solution will make PSSA ignore only the functions starting with _, which would solve your problem. Here is an example of a .ps1 file:
[Diagnostics.CodeAnalysis.SuppressMessage("PSUseApprovedVerbs", Scope="function", Target="_*" )]
Param()

function _Create-Something() { }

function Create-SomethingElse() { }

Hi, the original maintainer is not working on this repository any more, therefore I will jump in. The proposed solution will make PSSA ignore only the functions starting with _, which solves your problem exactly as you desire as it suppresses the rule only for functions starting with _. Here is an example of a .ps1 file:
[Diagnostics.CodeAnalysis.SuppressMessage("PSUseApprovedVerbs", Scope="function", Target="_*" )]
Param()

function _Create-Something() { }

function Create-SomethingElse() { }

I want to be clear, I don't want any function that starts with "" to be ignored, I want "" character in the first character of the function named ignored. I.E. I would like "Write-Log" checked against the rules as "Write-Log". I.E. Just ignore that the "" exists altogether.

In that case, I think the answer to your question is no. The specificity of the request is such that I'm not sure it makes sense to add a configuration to the existing rule to do so.
Instead, you could write a custom rule in script. In the particular case of UseApprovedVerbs it should just be a matter of finding all FunctionDefinitionAsts, stripping any preceding underscores off the function name, comparing the verb part of the name against Get-Verb, and emitting a diagnostic if it's not in there.

In that case, I think the answer to your question is no. The specificity of the request is such that I'm not sure it makes sense to add a configuration to the existing rule to do so.
Instead, you could write a custom rule in script. In the particular case of UseApprovedVerbs it should just be a matter of finding all FunctionDefinitionAsts, stripping any preceding underscores off the function name, comparing the verb part of the name against Get-Verb, and emitting a diagnostic if it's not in there.

How would one write a custom rule to override the UseApprovedVerbs?

How would one write a custom rule to override the UseApprovedVerbs?

Take a look at:

https://devblogs.microsoft.com/scripting/psscriptanalyzer-deep-dive-part-4-of-4/
https://mathieubuisson.github.io/create-custom-rule-psscriptanalyzer/
https://gist.github.com/Jaykul/e58b5820d7ca3d3d681f5605d8e136cd

Thank you. I will try to research that and if I get something good I will post it back here to share.


Nav Bar Icons

If there are less than 5 items in the nav bar, keep them left instead of centered on the page.

I'm not sure which update changed it, but this is now the default behavior.


Style the Contact Us page

I would like to add style to the Contact Us page, which is at this point not styled much. @niloysikdar Please assign me this under LGMSOC'21

Check #139


Lights turn orange after deploy

Whenever I deploy node-red my WLED lights turn orange and are disconnected and unreachable for a bit. Any ideay why this is? or how to debug it to see what the issue is?
I am running node red in HomeAssistant
and WLED version 0.14.0-b1

Sounds like for some reason your WLED controller is resetting. The lights turning orange you can disable by going into WLED settings for that device and unchecking the option to power on the lights after restart. Not sure why the controller would reset though when you deploy node-red.


Pg_restore unsupported version (1.14) in file header

Hello,
I am trying to restore a database, but I get an error that Pg_restore unsupported version (1.14) in file header
host is local host, port is 5432, Dbeaver is version 7.1.1
version pg_restore (PostgreSQL) 11.7 (Debian 11.7-0+deb10u1)
I work with a Chromebook
How can I fix that?
(I should probably add that I am a superbeginner  - installed postgres and Dbeaver 2 days ago)
Let me know!

As I see it is not DBeaver issue, please try to use a client that matches the database version.


Errors in rack when running inside docker container

When we try to use this gem within a docker container,
and configuring the gem to redirect /FooBar => /foobar
and then visit a url that would trigger the case insensitive redirect, we get one of these 2 errors seemingly at random.
Rack::Lint::LintError at /FooBar
a header value must be a String, but the value of 'Location' is a ActiveSupport::Multibyte::Chars

ThreadError at /FooBar
deadlock; recursive locking

However we get no exception if a parameter is tacked onto the end of the url
examples:
mydomain.com/foobar -- works as expected
mydomain.com/FooBar -- fails with one of the 2 exceptions
mydomain.com/FooBar?BAZ -- works as expected
NOTE: no issues when config.redirect = false

What version of Ruby and Rack is running inside the container? Anything you can provide me with, that can help reproduce?
Can you provide a stack-trace for this error?

Sorry, the stacks aren't nicely formatted, as they are copied from the browser error page (nothing shows up in the rails log file)
Please let me know anything else that would help you.
ruby --version
ruby 2.3.6p384 (2017-12-14 revision 61254) [x86_64-linux]

rack (1.6.8)
from gemfile.lock

Stack # 1
Rack::Lint::LintError at /NARUTO
a header value must be a String, but the value of 'Location' is a ActiveSupport::Multibyte::Chars

/usr/local/bundle/gems/rack-1.6.8/lib/rack/lint.rb: in assert
         20 raise LintError, message
/usr/local/bundle/gems/rack-1.6.8/lib/rack/lint.rb: in block in check_headers
       645 assert("a header value must be a String, but the value of " +
/usr/local/bundle/gems/rack-1.6.8/lib/rack/utils.rb: in block in each
         501 yield(k, v.respond_to?(:to_ary) ? v.to_ary.join("\n") : v)
/usr/local/bundle/gems/rack-1.6.8/lib/rack/utils.rb: in each
       500 super do |k, v|
/usr/local/bundle/gems/rack-1.6.8/lib/rack/utils.rb: in each
       500 super do |k, v|
/usr/local/bundle/gems/rack-1.6.8/lib/rack/lint.rb: in check_headers
     629 header.each { |key, value|
/usr/local/bundle/gems/rack-1.6.8/lib/rack/lint.rb: in _call
     53 check_headers headers
/usr/local/bundle/gems/rack-1.6.8/lib/rack/lint.rb: in call
     37 dup._call(env)
/usr/local/bundle/gems/rack-1.6.8/lib/rack/showexceptions.rb: in call
     24 @app.call(env)
/usr/local/bundle/gems/rack-1.6.8/lib/rack/commonlogger.rb: in call
     33 status, header, body = @app.call(env)
/usr/local/bundle/gems/sinatra-1.4.8/lib/sinatra/base.rb: in call
       219 call_without_check(env)
/usr/local/bundle/gems/rack-1.6.8/lib/rack/chunked.rb: in call
     54 status, headers, body = @app.call(env)
/usr/local/bundle/gems/rack-1.6.8/lib/rack/content_length.rb: in call
     15 status, headers, body = @app.call(env)
/usr/local/bundle/gems/unicorn-5.3.0/lib/unicorn/http_server.rb: in process_client
   606 status, headers, body = @app.call(env = @request.read(client))
/usr/local/bundle/gems/unicorn-5.3.0/lib/unicorn/http_server.rb: in worker_loop
         702 process_client(client)
/usr/local/bundle/gems/unicorn-5.3.0/lib/unicorn/http_server.rb: in spawn_missing_workers
       549 worker_loop(worker)
/usr/local/bundle/gems/unicorn-5.3.0/lib/unicorn/http_server.rb: in start
    142 spawn_missing_workers
/usr/local/bundle/gems/unicorn-5.3.0/bin/unicorn: in <top (required)>
126 Unicorn::HttpServer.new(app, options).start.join
/usr/local/bundle/bin/unicorn: in load
29 load Gem.bin_path("unicorn", "unicorn")
/usr/local/bundle/bin/unicorn: in <top (required)>
29 load Gem.bin_path("unicorn", "unicorn")
/usr/local/lib/ruby/site_ruby/2.3.0/bundler/cli/exec.rb: in load
     75 Kernel.load(file)
/usr/local/lib/ruby/site_ruby/2.3.0/bundler/cli/exec.rb: in kernel_load
     75 Kernel.load(file)
/usr/local/lib/ruby/site_ruby/2.3.0/bundler/cli/exec.rb: in run
         28 return kernel_load(bin_path, *args)
/usr/local/lib/ruby/site_ruby/2.3.0/bundler/cli.rb: in exec
     424 Exec.new(options, args).run
/usr/local/lib/ruby/site_ruby/2.3.0/bundler/vendor/thor/lib/thor/command.rb: in run
       27 instance.__send__(name, *args)
/usr/local/lib/ruby/site_ruby/2.3.0/bundler/vendor/thor/lib/thor/invocation.rb: in invoke_command
       126 command.run(self, *args)
/usr/local/lib/ruby/site_ruby/2.3.0/bundler/vendor/thor/lib/thor.rb: in dispatch
     387 instance.invoke_command(command, trailing || [])
/usr/local/lib/ruby/site_ruby/2.3.0/bundler/cli.rb: in dispatch
     27 super do |i|
/usr/local/lib/ruby/site_ruby/2.3.0/bundler/vendor/thor/lib/thor/base.rb: in start
       466 dispatch(nil, given_args.dup, nil, config)
/usr/local/lib/ruby/site_ruby/2.3.0/bundler/cli.rb: in start
     18 super
/usr/local/lib/ruby/gems/2.3.0/gems/bundler-1.16.1/exe/bundle: in block in <top (required)>
 30 Bundler::CLI.start(args, :debug => true)
/usr/local/lib/ruby/site_ruby/2.3.0/bundler/friendly_errors.rb: in with_friendly_errors
   122 yield
/usr/local/lib/ruby/gems/2.3.0/gems/bundler-1.16.1/exe/bundle: in <top (required)>
22 Bundler.with_friendly_errors do
/usr/local/bundle/bin/bundle: in load
 104 load Gem.bin_path("bundler", "bundle")
/usr/local/bundle/bin/bundle: in <main>
 104 load Gem.bin_path("bundler", "bundle")

Stack # 2
ThreadError at /NARUTO
deadlock; recursive locking

/usr/local/bundle/gems/rack-1.6.8/lib/rack/lock.rb: in lock
    16  @mutex.lock
/usr/local/bundle/gems/rack-1.6.8/lib/rack/lock.rb: in call
     16 @mutex.lock
/usr/local/bundle/gems/actionpack-4.2.10/lib/action_dispatch/middleware/static.rb: in call
     120 @app.call(env)
/usr/local/bundle/gems/rack-1.6.8/lib/rack/sendfile.rb: in call
     113 status, headers, body = @app.call(env)
/usr/local/bundle/gems/railties-4.2.10/lib/rails/engine.rb: in call
     518 app.call(env)
/usr/local/bundle/gems/railties-4.2.10/lib/rails/application.rb: in call
     165 super(env)
/usr/local/bundle/gems/rack-1.6.8/lib/rack/tempfile_reaper.rb: in call
     15 status, headers, body = @app.call(env)
/usr/local/bundle/gems/rack-1.6.8/lib/rack/lint.rb: in _call
     49 status, headers, @body = @app.call(env)
/usr/local/bundle/gems/rack-1.6.8/lib/rack/lint.rb: in call
     37 dup._call(env)
/usr/local/bundle/gems/rack-1.6.8/lib/rack/showexceptions.rb: in call
     24 @app.call(env)
/usr/local/bundle/gems/rack-1.6.8/lib/rack/commonlogger.rb: in call
     33 status, header, body = @app.call(env)
/usr/local/bundle/gems/sinatra-1.4.8/lib/sinatra/base.rb: in call
       219 call_without_check(env)
/usr/local/bundle/gems/rack-1.6.8/lib/rack/chunked.rb: in call
     54 status, headers, body = @app.call(env)
/usr/local/bundle/gems/rack-1.6.8/lib/rack/content_length.rb: in call
     15 status, headers, body = @app.call(env)
/usr/local/bundle/gems/unicorn-5.3.0/lib/unicorn/http_server.rb: in process_client
   606 status, headers, body = @app.call(env = @request.read(client))
/usr/local/bundle/gems/unicorn-5.3.0/lib/unicorn/http_server.rb: in worker_loop
         702 process_client(client)
/usr/local/bundle/gems/unicorn-5.3.0/lib/unicorn/http_server.rb: in spawn_missing_workers
       549 worker_loop(worker)
/usr/local/bundle/gems/unicorn-5.3.0/lib/unicorn/http_server.rb: in start
   142 spawn_missing_workers
/usr/local/bundle/gems/unicorn-5.3.0/bin/unicorn: in <top (required)>
126 Unicorn::HttpServer.new(app, options).start.join
/usr/local/bundle/bin/unicorn: in load
29 load Gem.bin_path("unicorn", "unicorn")
/usr/local/bundle/bin/unicorn: in <top (required)>
29 load Gem.bin_path("unicorn", "unicorn")
/usr/local/lib/ruby/site_ruby/2.3.0/bundler/cli/exec.rb: in load
    75  Kernel.load(file)
/usr/local/lib/ruby/site_ruby/2.3.0/bundler/cli/exec.rb: in kernel_load
     75 Kernel.load(file)
/usr/local/lib/ruby/site_ruby/2.3.0/bundler/cli/exec.rb: in run
        28  return kernel_load(bin_path, *args)
/usr/local/lib/ruby/site_ruby/2.3.0/bundler/cli.rb: in exec
     424 Exec.new(options, args).run
/usr/local/lib/ruby/site_ruby/2.3.0/bundler/vendor/thor/lib/thor/command.rb: in run
       27 instance.__send__(name, *args)
/usr/local/lib/ruby/site_ruby/2.3.0/bundler/vendor/thor/lib/thor/invocation.rb: in invoke_command
       126 command.run(self, *args)
/usr/local/lib/ruby/site_ruby/2.3.0/bundler/vendor/thor/lib/thor.rb: in dispatch
     387 instance.invoke_command(command, trailing || [])
/usr/local/lib/ruby/site_ruby/2.3.0/bundler/cli.rb: in dispatch
     27 super do |i|
/usr/local/lib/ruby/site_ruby/2.3.0/bundler/vendor/thor/lib/thor/base.rb: in start
       466 dispatch(nil, given_args.dup, nil, config)
/usr/local/lib/ruby/site_ruby/2.3.0/bundler/cli.rb: in start
     18 super
/usr/local/lib/ruby/gems/2.3.0/gems/bundler-1.16.1/exe/bundle: in block in <top (required)>
 30 Bundler::CLI.start(args, :debug => true)
/usr/local/lib/ruby/site_ruby/2.3.0/bundler/friendly_errors.rb: in with_friendly_errors
   122 yield
/usr/local/lib/ruby/gems/2.3.0/gems/bundler-1.16.1/exe/bundle: in <top (required)>
22 Bundler.with_friendly_errors do
/usr/local/bundle/bin/bundle: in load
 104 load Gem.bin_path("bundler", "bundle")
/usr/local/bundle/bin/bundle: in <main>
 104 load Gem.bin_path("bundler", "bundle")

I haven't been able to reproduce this. As far as I can tell, you're running

Route Downcaser 1.2.1
Rack 1.6.8
Sinatra 1.4.8
Unicorn 5.3.0

When I set up an app with that stack it seems to work fine:
$ curl http://<IP_ADDRESS>:8080/HeLlO/WoRlD
Hello world%

Any chance you can set up a basic reproduction app, that I could work from?

Oh, I got it - I missed the part of you having RouteDowncaser.redirect = true, fix incoming.


Question: Is it possible to use a custom scala type for query param, headers etc?

I'd like to use specific types for the path params, query params, headers etc to avoid getting them mixed up. I currently have methods that take up to 5 string arguments.

@eli-jordan Currently we don't support plain value wrappers, though this is definitely on the roadmap.

Yes, you can use x-scala-type (or x-jvm-type if it's usable in more than scala) in your OpenAPI spec file.  Depending on what it's used for, you'll also need to provide circe encoders/decoders and add customImports to your guardrail maven/sbt plugin configuration to bring them into scope.

Actually I guess what you actually need are Show[A] implementations, if it's a client, and they're path/query/header params.

Ok, I though I had tried this before and not had the scala types be present in the generated code. I just tried it again though, and they are there. So I must have done something wrong before. I'll see if I can define the right implicits to get it to work.
Thanks!

@eli-jordan sorry... @blast-hardcheese and I chatted a bit offline about this.  You can do what I've suggested, but be careful and note that Show is a sort of guardrail implementation detail (which could change at any time in a future version), and also implementing Show (and using x-scala-type) is a bit of an escape hatch that can allow you to violate the contract you've specified in your OpenAPI spec, if you're not careful.

Ok thanks. I'll stick with the supported path for now :)
It looks like I need an implicit AddArg for a query parameter and AddPath for a path parameter.

@eli-jordan You may get those for free if you implement Show, as @kelnos mentioned. Glad you were able to get a workable solution though!


Endtime of query windows of buckets slightly increasing

I'm using 5s for run_every and 1s for bucket_interval in the metric_aggregation rule. Below are some EA output and I'm wondering why the end-time of the query window is slightly increasing.
INFO:elastalert:Ran CPU_User from 2017-05-05 00:23:41.516443 EDT to 2017-05-05 00:23:46.569775 EDT: 10 query hits (0 already seen), 5 matches, 0 alerts sent
INFO:elastalert:Sleeping for 4.97194 seconds
INFO:elastalert:Ran CPU_User from 2017-05-05 00:23:46.516443 EDT to 2017-05-05 00:23:51.574753 EDT: 10 query hits (0 already seen), 5 matches, 0 alerts sent
INFO:elastalert:Sleeping for 4.972751 seconds
INFO:elastalert:Ran CPU_User from 2017-05-05 00:23:51.516443 EDT to 2017-05-05 00:23:56.579921 EDT: 10 query hits (0 already seen), 5 matches, 0 alerts sent
INFO:elastalert:Sleeping for 4.97243 seconds
INFO:elastalert:Ran CPU_User from 2017-05-05 00:23:56.516443 EDT to 2017-05-05 00:24:01.584863 EDT: 10 query hits (0 already seen), 5 matches, 0 alerts sent
INFO:elastalert:Sleeping for 4.972708 seconds

The end time is the current time when the query is generated. The 5 second interval is a just a best attempt by elastalert, it's not going to be exact. If some piece of code runs that takes a long time, it can be even further off.
Do you have a use case where exact intervals are required?

Thanks for the answer.
As for the use case, wondering if it's related to this issue I asked before: https://github.com/Yelp/elastalert/issues/1070
If I use bucket_interval 1s with the above configuration, sometimes (not always) it makes 6 queries with the last one having no hits - that could generate a false match in our use case.
FYI, we don't want to use realert as we need state-based raise and clear of the alerts.

Might the sync_bucket_interval setting be of any use here it should round the buckets to the nearest second and eliminate tiny precision errors.

I actually thing I spotted a bug today, the code that is supposed to align the buckets with the start and end query times isn't working at all. This would definitely explain the partial buckets and probably the time drift as well. I'll submit a fix over the next few days.