Installing latest stable version fails with --nodata

I'm trying to install the latest stable version and it's failing, something to do with a badly formatted YAML file. Here's what I'm running:
bcbio_path=/export/home/ncit/external/a.mizeranschi/bcbio_nextgen
mkdir ${bcbio_path}
cd ${bcbio_path}
wget https://raw.github.com/chapmanb/bcbio-nextgen/master/scripts/bcbio_nextgen_install.py
python bcbio_nextgen_install.py ${bcbio_path} -u stable --tooldir=${bcbio_path}/tools --nodata --isolate

and this is the result:
Downloading and Extracting Packages
python-3.6.10        | 34.1 MB   | ######################################################################################################################################################################## | 100%
certifi-2<IP_ADDRESS>   | 151 KB    | ######################################################################################################################################################################## | 100%
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
Installing data and third party dependencies
Upgrading bcbio
Traceback (most recent call last):
  File "/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda/bin/bcbio_nextgen.py", line 228, in <module>
    install.upgrade_bcbio(kwargs["args"])
  File "/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda/lib/python3.6/site-packages/bcbio/install.py", line 70, in upgrade_bcbio
    _update_conda_packages()
  File "/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda/lib/python3.6/site-packages/bcbio/install.py", line 285, in _update_conda_packages
    channels = _get_conda_channels(conda_bin)
  File "/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda/lib/python3.6/site-packages/bcbio/install.py", line 268, in _get_conda_channels
    config = yaml.safe_load(subprocess.check_output([conda_bin, "config", "--show"]))
  File "/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda/lib/python3.6/site-packages/yaml/__init__.py", line 162, in safe_load
    return load(stream, SafeLoader)
  File "/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda/lib/python3.6/site-packages/yaml/__init__.py", line 114, in load
    return loader.get_single_data()
  File "/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda/lib/python3.6/site-packages/yaml/constructor.py", line 49, in get_single_data
    node = self.get_single_node()
  File "/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda/lib/python3.6/site-packages/yaml/composer.py", line 36, in get_single_node
    document = self.compose_document()
  File "/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda/lib/python3.6/site-packages/yaml/composer.py", line 58, in compose_document
    self.get_event()
  File "/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda/lib/python3.6/site-packages/yaml/parser.py", line 118, in get_event
    self.current_event = self.state()
  File "/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda/lib/python3.6/site-packages/yaml/parser.py", line 193, in parse_document_end
    token = self.peek_token()
  File "/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda/lib/python3.6/site-packages/yaml/scanner.py", line 129, in peek_token
    self.fetch_more_tokens()
  File "/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda/lib/python3.6/site-packages/yaml/scanner.py", line 223, in fetch_more_tokens
    return self.fetch_value()
  File "/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda/lib/python3.6/site-packages/yaml/scanner.py", line 579, in fetch_value
    self.get_mark())
yaml.scanner.ScannerError: mapping values are not allowed here
  in "<byte string>", line 19, column 15:
            GitHub:  https://github.com/QuantStack ...
                  ^
Traceback (most recent call last):
  File "bcbio_nextgen_install.py", line 290, in <module>
    main(parser.parse_args(), sys.argv[1:])
  File "bcbio_nextgen_install.py", line 52, in main
    subprocess.check_call([bcbio, "upgrade"] + _clean_args(sys_argv, args))
  File "/usr/lib64/python3.6/subprocess.py", line 311, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda/bin/bcbio_nextgen.py', 'upgrade', '-u', 'stable', '--tooldir=/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/tools', '--isolate']' returned non-zero exit status 1.

Hi @amizeranschi !
Sorry about the issue.
It seems we have solved it recently in the development:
https://github.com/bcbio/bcbio-nextgen/issues/3180
Could you please try with
python3 bcbio_nextgen_install.py \
/bcbio -u development \
--tooldir=/bcbio/tools \
--nodata \
--isolate

Sergey

Yes, the development version made it through. Thanks for mentioning it. Would be useful to get a new stable release out, though, if the current one has that issue.
However, I ran into another problem now. After installing bcbio with --nodata, I added a custom genome via bcbio_setup_genome.py. This seemed to work fine, but the bcbio_nextgen/galaxy/tool-data directory did not get created. To be more precise, the bcbio_nextgen/galaxy directory only contains the file bcbio_system.yaml.
This renders the custom genome unusable, as bcbio complains about missing loc files:
[2020-05-17T16:21Z] Using input YAML configuration: /export/home/ncit/external/a.mizeranschi/automated-VC-test-BRF/testingVC/config/testingVC.yaml
[2020-05-17T16:21Z] Checking sample YAML configuration: /export/home/ncit/external/a.mizeranschi/automated-VC-test-BRF/testingVC/config/testingVC.yaml
Running bcbio version: 1.2.3
global config: /export/home/ncit/external/a.mizeranschi/automated-VC-test-BRF/testingVC/work/bcbio_system.yaml
run info config: /export/home/ncit/external/a.mizeranschi/automated-VC-test-BRF/testingVC/config/testingVC.yaml
Traceback (most recent call last):
  File "/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/tools/bin/bcbio_nextgen.py", line 245, in <module>
    main(**kwargs)
  File "/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/tools/bin/bcbio_nextgen.py", line 46, in main
    run_main(**kwargs)
  File "/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda/lib/python3.6/site-packages/bcbio/pipeline/main.py", line 50, in run_main
    fc_dir, run_info_yaml)
  File "/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda/lib/python3.6/site-packages/bcbio/pipeline/main.py", line 91, in _run_toplevel
    for xs in pipeline(config, run_info_yaml, parallel, dirs, samples):
  File "/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda/lib/python3.6/site-packages/bcbio/pipeline/main.py", line 128, in variant2pipeline
    [x[0]["description"] for x in samples]]])
  File "/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda/lib/python3.6/site-packages/bcbio/distributed/multi.py", line 28, in run_parallel
    return run_multicore(fn, items, config, parallel=parallel)
  File "/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda/lib/python3.6/site-packages/bcbio/distributed/multi.py", line 86, in run_multicore
    for data in joblib.Parallel(parallel["num_jobs"], batch_size=1, backend="multiprocessing")(joblib.delayed(fn)(*x) for x in items):
  File "/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda/lib/python3.6/site-packages/joblib/parallel.py", line 1029, in __call__
    if self.dispatch_one_batch(iterator):
  File "/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda/lib/python3.6/site-packages/joblib/parallel.py", line 847, in dispatch_one_batch
    self._dispatch(tasks)
  File "/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda/lib/python3.6/site-packages/joblib/parallel.py", line 765, in _dispatch
    job = self._backend.apply_async(batch, callback=cb)
  File "/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda/lib/python3.6/site-packages/joblib/_parallel_backends.py", line 206, in apply_async
    result = ImmediateResult(func)
  File "/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda/lib/python3.6/site-packages/joblib/_parallel_backends.py", line 570, in __init__
    self.results = batch()
  File "/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda/lib/python3.6/site-packages/joblib/parallel.py", line 253, in __call__
    for func, args, kwargs in self.items]
  File "/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda/lib/python3.6/site-packages/joblib/parallel.py", line 253, in <listcomp>
    for func, args, kwargs in self.items]
  File "/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda/lib/python3.6/site-packages/bcbio/utils.py", line 55, in wrapper
    return f(*args, **kwargs)
  File "/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda/lib/python3.6/site-packages/bcbio/distributed/multitasks.py", line 459, in organize_samples
    return run_info.organize(*args)
  File "/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda/lib/python3.6/site-packages/bcbio/pipeline/run_info.py", line 81, in organize
    item = add_reference_resources(item, remote_retriever)
  File "/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda/lib/python3.6/site-packages/bcbio/pipeline/run_info.py", line 177, in add_reference_resources
    data["dirs"]["galaxy"], data)
  File "/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda/lib/python3.6/site-packages/bcbio/pipeline/genome.py", line 233, in get_refs
    galaxy_config, data)
  File "/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda/lib/python3.6/site-packages/bcbio/pipeline/genome.py", line 180, in _get_ref_from_galaxy_loc
    (genome_build, os.path.normpath(loc_file)))
ValueError: Did not find genome build sacCer3_BRF in bcbio installation: /export/home/ncit/external/a.mizeranschi/bcbio_nextgen/galaxy/tool-data/sam_fa_indices.loc

Glad it worked!
I think since you have not installed any data, then tool-data was not created.
Can you try installing with --genomes hg38 --aligners bwa and then install the custom genome?
Or just create manually the loc files:
https://bcbio-nextgen.readthedocs.io/en/latest/contents/configuration.html#reference-genome-files

OK, I tried installing the hg38 genome. This crashed due to a failed download (http://www.cs.jhu.edu/~genomics/GeneSplicer/GeneSplicer.tar.gz). It looks like that URL isn't valid anymore and it caused an error with a GGD recipe (hg38 genesplicer 2004.04.03).
--2020-05-18 08:18:21--  http://www.cs.jhu.edu/~genomics/GeneSplicer/GeneSplicer.tar.gz
Resolving www.cs.jhu.edu (www.cs.jhu.edu)... <IP_ADDRESS>
Connecting to www.cs.jhu.edu (www.cs.jhu.edu)|<IP_ADDRESS>|:80... connected.
HTTP request sent, awaiting response... 404 Not Found
2020-05-18 08:18:21 ERROR 404: Not Found.

Upgrading bcbio
Detected 1.2.3 as latest version of bcbio-nextgen on bioconda.
bcbio version 1.2.3 is newer than the conda version 1.2.3, skipping upgrade from conda
Upgrading bcbio-nextgen to latest development version
Upgrade of bcbio-nextgen development code complete.
Upgrading third party tools to latest versions
Reading packages from /export/home/ncit/external/a.mizeranschi/bcbio_nextgen/tmpbcbio-install/cloudbiolinux/contrib/flavor/ngs_pipeline_minimal/packages-conda.yaml
Creating conda environment: python3
Creating conda environment: samtools0
Creating conda environment: dv
Creating conda environment: python2
Creating conda environment: r36
Creating conda environment: htslib1.10
Checking for problematic or migrated packages in default environment
Initalling initial set of packages for default environment with mamba
# Installing into conda environment default: age-metasv, arriba, ataqv, bamtools=2.4.0, bamutil, bbmap, bcbio-prioritize, bcbio-variation, bcbio-variation-recall, bcftools, bedops, bedtools=2.27.1, bio-vcf, biobambam, bowtie, bowtie2, break-point-inspector, bwa, bwakit, cage, cancerit-allelecount, chipseq-greylist, cnvkit, coincbc, cramtools, cufflinks, cyvcf2, deeptools, delly, duphold, ensembl-vep=99.*, express, extract-sv-reads, fastp, fastqc>=0.11.8=1, fgbio, freebayes=<IP_ADDRESS>, gatk, gatk4, geneimpacts, genesplicer, gffcompare, goleft, grabix, gridss, gsort, gvcfgenotyper, h5py, hmftools-amber, hmftools-cobalt, hmftools-purple, hmmlearn, hts-nim-tools, htslib, impute2, kallisto>=0.43.1, kraken, ldc>=1.13.0, lofreq, macs2, maxentscan, mbuffer, minimap2, mintmap, mirdeep2=<IP_ADDRESS>, mirtop, moreutils, multiqc, multiqc-bcbio, ngs-disambiguate, novoalign, octopus>=0.5.1b, oncofuse, optitype>=1.3.4, parallel, pbgzip, peddy, perl-sanger-cgp-battenberg, picard, pindel, pizzly, pyloh, pysam>=0.14.0, pythonpy, qsignature, qualimap, rapmap, razers3=3.5.0, rtg-tools, sailfish, salmon, sambamba, samblaster, samtools, scalpel, seq2c<2016, seqbuster, seqcluster, seqtk, sickle-trim, simple_sv_annotation, singlecell-barcodes, snap-aligner=1.0dev.97, snpeff=4.3.1t, solvebio, spades, staden_io_lib, star=2.6.1d, stringtie, subread, survivor, tdrmapper, tophat-recondition, trim-galore=0.6.2, ucsc-bedgraphtobigwig, ucsc-bedtobigbed, ucsc-bigbedinfo, ucsc-bigbedsummary, ucsc-bigbedtobed, ucsc-bigwiginfo, ucsc-bigwigsummary, ucsc-bigwigtobedgraph, ucsc-bigwigtowig, ucsc-fatotwobit, ucsc-gtftogenepred, ucsc-liftover, ucsc-wigtobigwig, umis, vardict, vardict-java, variantbam, varscan, vcfanno, vcflib, verifybamid2, viennarna, vqsr_cnn, vt, wham, anaconda-client, awscli, bzip2, ncurses, nodejs, p7zip, readline, s3gof3r, xz, perl-app-cpanminus, perl-archive-extract, perl-archive-zip, perl-bio-db-sam, perl-cgi, perl-dbi, perl-encode-locale, perl-file-fetch, perl-file-sharedir, perl-file-sharedir-install, perl-ipc-system-simple, perl-lwp-protocol-https, perl-lwp-simple, perl-statistics-descriptive, perl-time-hires, perl-vcftools-vcf, bioconductor-annotate, bioconductor-apeglm, bioconductor-biocgenerics, bioconductor-biocinstaller, bioconductor-biocstyle, bioconductor-biostrings, bioconductor-biovizbase, bioconductor-bsgenome.hsapiens.ucsc.hg19, bioconductor-bsgenome.hsapiens.ucsc.hg38, bioconductor-bubbletree, bioconductor-cn.mops, bioconductor-copynumber, bioconductor-degreport, bioconductor-deseq2, bioconductor-dexseq, bioconductor-dnacopy, bioconductor-genomeinfodbdata, bioconductor-genomicranges, bioconductor-iranges, bioconductor-limma, bioconductor-rtracklayer, bioconductor-snpchip, bioconductor-titancna, bioconductor-vsn>=3.50.0, r-base, r-basejump=0.7.2, r-bcbiornaseq>=0.2.7, r-cghflasso, r-chbutils, r-devtools, r-dplyr, r-dt, r-ggdendro, r-ggplot2, r-ggrepel>=0.7, r-gplots, r-gsalib, r-knitr, r-pheatmap, r-plyr, r-pscbs, r-reshape, r-rmarkdown, r-rsqlite, r-sleuth, r-snow, r-stringi, r-viridis>=0.5, r-wasabi, r=3.5.1, xorg-libxt
# Installing into conda environment dv: deepvariant
# Installing into conda environment htslib1.10: mosdepth
# Installing into conda environment python2: bismark=0.22.1, cpat, cutadapt=1.16, dkfz-bias-filter, gemini, gvcf-regions, hap.py, hisat2, htseq=0.9.1, lumpy-sv, manta, metasv, mirge, phylowgs, platypus-variant, sentieon, smcounter2, smoove, strelka, svtools, svtyper, theta2, tophat, vawk, vcf2db
# Installing into conda environment python3: atropos, crossmap
# Installing into conda environment r36: bioconductor-purecn>=1.16.0
# Installing into conda environment samtools0: ericscript
Creating manifest of installed packages in /export/home/ncit/external/a.mizeranschi/bcbio_nextgen/manifest
Third party tools upgrade complete.
Upgrading bcbio-nextgen data files
List of genomes to get (from the config file at '{'genomes': [{'dbkey': 'hg38', 'name': 'Human (hg38) full', 'indexes': ['seq', 'twobit', 'bwa', 'hisat2'], 'annotations': ['ccds', 'capture_regions', 'coverage', 'prioritize', 'dbsnp', 'hapmap_snps', '1000g_omni_snps', 'ACMG56_genes', '1000g_snps', 'mills_indels', '1000g_indels', 'clinvar', 'qsignature', 'genesplicer', 'effects_transcripts', 'varpon', 'vcfanno', 'viral', 'transcripts', 'RADAR', 'rmsk', 'salmon-decoys', 'fusion-blacklist', 'mirbase'], 'validation': ['giab-NA12878', 'giab-NA24385', 'giab-NA24631', 'platinum-genome-NA12878', 'giab-NA12878-remap', 'giab-NA12878-crossmap', 'dream-syn4-crossmap', 'dream-syn3-crossmap', 'giab-NA12878-NA24385-somatic', 'giab-NA24143', 'giab-NA24149', 'giab-NA24694', 'giab-NA24695']}], 'genome_indexes': ['bwa', 'bowtie2', 'hisat2', 'rtg'], 'install_liftover': False, 'install_uniref': False}'): Human (hg38) full
Running GGD recipe: hg38 seq 1000g-20150219_1
Running GGD recipe: hg38 bwa 1000g-20150219
Moving on to next genome prep method after trying ggd
GGD recipe not available for hg38 bowtie2
Downloading genome from s3: hg38 bowtie2
Moving on to next genome prep method after trying s3
No pre-computed indices for hg38 bowtie2
Preparing genome hg38 with index bowtie2
Running GGD recipe: hg38 hisat2 12-07-2015
Moving on to next genome prep method after trying ggd
GGD recipe not available for hg38 rtg
Downloading genome from s3: hg38 rtg
Moving on to next genome prep method after trying s3
No pre-computed indices for hg38 rtg
Preparing genome hg38 with index rtg
Running GGD recipe: hg38 ccds r20
Running GGD recipe: hg38 capture_regions 20161202
Running GGD recipe: hg38 coverage 2018-10-16
Running GGD recipe: hg38 prioritize 20181227
Running GGD recipe: hg38 dbsnp<PHONE_NUMBER>5
Running GGD recipe: hg38 hapmap_snps 20160105
Running GGD recipe: hg38 1000g_omni_snps 20160105
Running GGD recipe: hg38 ACMG56_genes 20160726
Running GGD recipe: hg38 1000g_snps 20160105
Running GGD recipe: hg38 mills_indels 20160105
Running GGD recipe: hg38 1000g_indels 2.8_hg38_20150522
Running GGD recipe: hg38 clinvar 20190513
Running GGD recipe: hg38 qsignature 20160526
Running GGD recipe: hg38 genesplicer 2004.04.03
Traceback (most recent call last):
  File "/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda/bin/bcbio_nextgen.py", line 228, in <module>
    install.upgrade_bcbio(kwargs["args"])
  File "/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda/lib/python3.6/site-packages/bcbio/install.py", line 107, in upgrade_bcbio
    upgrade_bcbio_data(args, REMOTES)
  File "/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda/lib/python3.6/site-packages/bcbio/install.py", line 359, in upgrade_bcbio_data
    args.cores, ["ggd", "s3", "raw"])
  File "/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/tmpbcbio-install/cloudbiolinux/cloudbio/biodata/genomes.py", line 354, in install_data_local
    _prep_genomes(env, genomes, genome_indexes, ready_approaches, data_filedir)
  File "/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/tmpbcbio-install/cloudbiolinux/cloudbio/biodata/genomes.py", line 480, in _prep_genomes
    retrieve_fn(env, manager, gid, idx)
  File "/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/tmpbcbio-install/cloudbiolinux/cloudbio/biodata/genomes.py", line 875, in _install_with_ggd
    ggd.install_recipe(os.getcwd(), env.system_install, recipe_file, gid)
  File "/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/tmpbcbio-install/cloudbiolinux/cloudbio/biodata/ggd.py", line 30, in install_recipe
    recipe["recipe"]["full"]["recipe_type"], system_install)
  File "/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/tmpbcbio-install/cloudbiolinux/cloudbio/biodata/ggd.py", line 62, in _run_recipe
    subprocess.check_output(["bash", run_file])
  File "/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda/lib/python3.6/subprocess.py", line 356, in check_output
    **kwargs).stdout
  File "/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda/lib/python3.6/subprocess.py", line 438, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['bash', '/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/genomes/Hsapiens/hg38/txtmp/ggd-run.sh']' returned non-zero exit status 8.
Checking required dependencies
Installing isolated base python installation
Installing mamba
Installing conda-build
Installing bcbio-nextgen
Installing data and third party dependencies
Traceback (most recent call last):
  File "bcbio_nextgen_install.py", line 290, in <module>
    main(parser.parse_args(), sys.argv[1:])
  File "bcbio_nextgen_install.py", line 52, in main
    subprocess.check_call([bcbio, "upgrade"] + _clean_args(sys_argv, args))
  File "/usr/lib64/python3.6/subprocess.py", line 311, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/anaconda/bin/bcbio_nextgen.py', 'upgrade', '-u', 'development', '--tooldir=/export/home/ncit/external/a.mizeranschi/bcbio_nextgen/tools', '--genomes', 'hg38', '--datatarget', 'variation', '--datatarget', 'rnaseq', '--datatarget', 'smallrna', '--aligners', 'bwa', '--aligners', 'bowtie2', '--aligners', 'hisat2', '--isolate', '--cores', '2', '--data']' returned non-zero exit status 1.

I also found an earlier thread (https://github.com/bcbio/bcbio-nextgen/issues/3165) where @chapmanb suggested that these errors can sometimes happen intermittently and retrying the install/upgrade can get things running.
I retried this a couple of times and each time it crashed at that particular step. I'm guessing the URL http://www.cs.jhu.edu/~genomics/GeneSplicer/GeneSplicer.tar.gz isn't working anymore and hg38 is uninstallable as a result.

Thanks, genesplicer is an easy fix - they moved to FTP server:
https://github.com/chapmanb/cloudbiolinux/blob/master/ggd-recipes/hg38/genesplicer.yaml
Please try again.
S.

Thanks for the fix. That got things moving forward, but the SnpEffect database for hg38 was taking forever to download (estimated 15 hours for 1.5 GB), so I canceled it. I don't actually need the hg38 data, anyway.
Instead, I installed the sacCer3 yeast genome, which I remembered was on the order of a couple hundred MB for all the data. Much more manageable, when its only purpose is to get the tool-data directory created.
Things worked fine and I could then install and use the custom genome. Thanks a lot for your help!


Use LogClientService in external Java Project?

Hi,
I want to use the LogClientService to retrieve CT entries. Is that possible from outside of the project within a Java project?
I found no way so far.
Thanks!

Hi @Peter0x48,
You are right you cannot access LogClientService directly.
Instead you should access it through LogClientFactory, however, I marked it with Kotlin's internal modifier where it should be public.
Effectively this means from Java code you should be able to access this:
//noinspection KotlinInternalInJava
LogClient client = LogClientFactory.INSTANCE.create(baseUrl);

The next release of the library will have this code public in its own module. As such I'm closing this issue for now.


Why url.Parse ???

Why url.Parse ???
https://github.com/HMS-Core/hms-push-serverdemo-go/blob/5c37e3e91cf2debe9d97fe9dc49daf098b9d6112/src/push/authention/auth.go#L74
I do not understand. is appSecret just a string? Is this parameter validation? Couldn't it be easier?
body := fmt.Sprintf("grant_type=client_credentials&client_secret=%s&client_id=%s", ac.appSecret, ac.appId)

This issue has been solved in the latest commit, please refer to afaab2fe7dd4cfcaa82a043cd65bbc5677117f6c


[Broker] Always let system topic TRANSACTION_BUFFER_SNAPSHOT be auto created

Fixes https://github.com/apache/pulsar/issues/10871
Motivation
From the linked issue:

While testing the 2.8.0rc2 I found that if you start a Pulsar broker with transactionCoordinatorEnabled=true and allowAutoTopicCreation=false the broker does not work.

The TRANSACTION_BUFFER_SNAPSHOT topic is a system topic, and it should be auto created as needed.
Modifications

Updated the logic in the isAllowAutoTopicCreation method to ensure that TRANSACTION_BUFFER_SNAPSHOT is considered a system topic.
Updated the logic in checkTopicIsEventsNames to take a TopicName and then check for equality in stead of check endsWith. That could have technically led to unexpected behavior if the topic name was something like my__change_events or my__transaction_buffer_snapshot.
Check for isSystemTopicEnabled first to prevent potentially unnecessary string equality checks.

Verifying this change
This is a simple fix that has limit impact. We'll want to verify that the logic is correct regarding allowing TRANSACTION_BUFFER_SNAPSHOT to be auto created. Otherwise, there should be tests covering the uses of this checkTopicIsEventsNames method.
Note that this change will impact the PersistentTopic and the PersistentSubscription classes. Based on reading through the usage of the checkTopicIsEventsNames method, I think this is the right change for those classes. Here are the affected usages:
https://github.com/apache/pulsar/blob/85a60b0de8c7e53747ee491350f7b21fba90099e/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/PersistentTopic.java#L317-L330
https://github.com/apache/pulsar/blob/85a60b0de8c7e53747ee491350f7b21fba90099e/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/PersistentSubscription.java#L143-L151
Does this pull request potentially affect one of the following parts:
If yes was chosen, please highlight the changes

Dependencies (does it add or upgrade a dependency): no
The public API: no
The schema: no
The default values of configurations: no
The wire protocol: no
The rest endpoints: no
The admin cli options: no
Anything that affects deployment: no

Documentation
I don't think we need new documentation for this, but if someone more familiar with transactions wants docs, please let me know where I can add docs.

@congbobo184 @codelipenghui @sijie @merlimat - PTAL. I am not familiar with the intricacies of the transaction code base within the brokers, but I think this change should fix the issue that @eolivelli discovered in https://github.com/apache/pulsar/issues/10871.

/pulsarbot run-failure-checks

@eolivelli - done. I didn't see a good way to test the other usages of the checkTopicIsEventsNames method. If someone more familiar with transactions can provide a good way to test this method's usage in those cases, please let me know.

ummm, Similar to the https://github.com/apache/pulsar/pull/10850

LGTM
it would be great to have a new 2.8.0rc3


Out Of Memory Error

When I use quantize_ function and run the model, then comment it out in next running time, the original model will appear OOM, and it seems that the pytorch GPU memory automatic reuse function is no longer available. As the running continues the memory to grow until OOM.(My model contains a stack of some modules)
mymodel = MyModel().eval().cuda().half()
quantize_(mymodel, int4_weight_only()) #then comment it in next running

#gen random input
input_ = torch.rand ...
#run model forward
output_ = mymodel(input_)
print(output_)

Is there any cache config be saved in the torchao quantize process and it couldn't be clear?

Hi @bianxuxuxu can you give a repro so we can debug the issue?

Hi, @jerryzh168
There may be something wrong with my environment, I'll check again. Thank you


[FEATURE] Tristate toggle

Is your feature request related to a problem? Please describe.
Normal checkboxes  support an indeterminate state via JavaScript.

Describe the solution you'd like

Add support to indeterminate state.
Add styling to indeterminated toggle elements to show when they're indeterminated .

Describe alternatives you've considered
😞
Additional context
Original from:

https://github.com/minhur/bootstrap-toggle/issues/66 by @paulomcnally
https://github.com/minhur/bootstrap-toggle/issues/175 by @loginetsolutions
https://github.com/minhur/bootstrap-toggle/issues/202 by @solsticedhiver
https://github.com/minhur/bootstrap-toggle/issues/212 by @JaiKumaresh
https://github.com/gitbrent/bootstrap4-toggle/issues/38 by @Mina-R-Meshriky

:ballot_box_with_check: Task list

[x] Develop feature/fix jQuery
[x] Develop feature/fix ECMAS
[x] Add cypress test coverage
[x] Update docs (is needed)

@palcarazm was this feature added? I do not see any examples

Is this still the demo site for this?
https://www.bootstraptoggle.com/

No, the demo page is https://palcarazm.github.io/bootstrap5-toggle/
The website you refer is for bootstrap 4 who has not support


Slack problem

This:
curl -X POST -d @dev/slack-compatible-notification-example.json localhost:4321/slack
works from local
the same json post not working from external source
500
failed to parse alert body

Hi @ticktostick. I don't use that Slack-compatible handler to be honest. So i need some more information from you to work on that issue. Can you tell me what type of Webhook in Slack you are using (And what method if there are different ones)? A Link to the Slack documentation and/or a example-request (Headers + Body) would be awesome!


Remove "value" as required element of address_level

Category
What kind of change is this?
Please select one of the following four options.
Consult Pull request merging criteria for a description of each category.

[ ] Cosmetic change.
[ ] Documentation change by member.
[ ] Documentation change by Overture tech writer.
[x] Material change.

Description
This fixes a bug in the schema where unknown address_levels were disallowed. Allow them by still requiring the
addressLevelContainer struct, but make the "value" field optional.
Reference
List of relevant links to GitHub issues, PRs, and other documentation.

TODO.

Testing
jv
Checklist
Checklist of tasks commonly-associated with schema pull requests. Please review the relevant checklists and ensure you do all the tasks that are required for the change you made.

[x] Add relevant examples.
[ ] Add relevant counterexamples.
[ ] Update any counterexamples that became obsolete. For example, if a counterexample uses property A but is not intended to test property A's validity, and you made a schema change that invalidates property A in that counterexample, fix the counterexample to align it with your schema change.
[x] Update in-schema documentation using plain English written in complete sentences, if an update is required.
[ ] Update Docusaurus documentation, if an update is required.
[ ] Review change with Overture technical writer to ensure any advanced documentation needs will be taken care of, unless the change is trivial and would not affect the documentation.

Documentation Website
Update the hyperlink below to put the pull request number in.
Docs preview for this PR.

Note to myself: We had meant to allow null but never did. Now using {} placeholder instead of allowing null.

Note to myself: We had meant to allow null but never did. Now using {} placeholder instead of allowing null.

Reason for {} is that arrays with explicit null value are somewhat uncommon/surprising. We feel that an existing object/structure with missing fields is a more common/less surprising data structure.
Agreed 2024-10-02 Schema TF meeting: We'll follow this approach and try to avoid explicit null value in array.
There's an open point about whether address_levels  should be empty if it has no meaningful content. Punting on that.


Add RawMutex::is_locked()

Resolves #232

For consistency, could you add a similar API to ReentrantMutex and RwLock?

I'm not sure about the RwLock naming, but I think it's consistent with the difference between RawRwLock::lock_{shared,exclusive} and RwLock::{read,write}

Missing an implementation of is_locked for RawRwLock. You should check if WRITER_BIT | READERS_MASK is non-zero.

Thanks!


Improve model download speed, progress display, etc

I was reading the node-llama-cpp docs, and they mention that the ipull package can be useful for improved model download speeds:

https://withcatai.github.io/node-llama-cpp/guide/#getting-a-model-file

For improved download speeds, you can use ipull to download the model

https://github.com/ido-pluto/ipull

iPull
Super fast file downloader with multiple connections

Features

Download using parallels connections
Pausing and resuming downloads
Node.js and browser support
Smart retry on fail
CLI Progress bar
Download statistics (speed, time left, etc.)

I can see that the current download command calls downloadModel:
https://github.com/jehna/humanify/blob/14e4ae536588935edada4438cf28b3b5553b2596/src/commands/download.ts#L1-L6
Which is defined here, and seems to just use fetch currently, as well as implementing its own download progress tracking in showProgress:
https://github.com/jehna/humanify/blob/14e4ae536588935edada4438cf28b3b5553b2596/src/local-models.ts#L38-L65
https://github.com/jehna/humanify/blob/14e4ae536588935edada4438cf28b3b5553b2596/src/progress.ts#L4-L13

I wonder if using iPull might make sense, both in increased download speed, as well as better download progress visibility/etc.

Related:

I can see that the instructions are in the README here:

https://github.com/jehna/humanify#local-mode

Which suggests I need to run humanify download 2b first.
I wonder if it might make more sense to have the local model download as a sub-command of humanify local, as that's where I was first looking for help for how to download the models, and it didn't even occur to me to check the root level command, since things local things seemed to be 'scoped' under the local command:
⇒ npx humanifyjs local -h
(node:97623) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
Usage: humanify local [options] <input>

Use a local LLM to unminify code

Arguments:
  input                     The input minified Javascript file

Options:
  -m, --model <model>       The model to use (default: "2b")
  -o, --outputDir <output>  The output directory (default: "output")
  -s, --seed <seed>         Seed for the model to get reproduceable results (leave out for random seed)
  --disableGpu              Disable GPU acceleration
  --verbose                 Show verbose output
  -h, --help                display help for command

There also seems to be very minimal information output during the download. It might be nice to know a bit more about which model is being downloaded, from where, where it's being saved, how large it is, etc:
 ⇒ npx humanifyjs download 2b
(node:97932) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
Downloaded 1.63 GB

I guess it does provide slightly more info when the download is completed:
⇒ npx humanifyjs download 2b
(node:97932) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
                  Model "2b" downloaded to /Users/devalias/.humanifyjs/models/Phi-3.1-mini-4k-instruct-Q4_K_M.gguf

I can see it's downloaded here:
⇒ ls ~/.humanifyjs/models
Phi-3.1-mini-4k-instruct-Q4_K_M.gguf

And the code for that is here:
https://github.com/jehna/humanify/blob/85d17e73d6b760bf896d01dd99fa3a5f98ea2848/src/local-models.ts#L13-L25
I also notice that MODEL_DIRECTORY is hardcoded currently. I wonder if that would be something useful to be able to specify/customize via a CLI arg/env variable/etc.
It seems the humanify local command uses getModelPath:
https://github.com/jehna/humanify/blob/85d17e73d6b760bf896d01dd99fa3a5f98ea2848/src/plugins/local-llm-rename/llama.ts#L19-L22
Which only seems to work for model aliases defined in MODELS:
https://github.com/jehna/humanify/blob/85d17e73d6b760bf896d01dd99fa3a5f98ea2848/src/local-models.ts#L69-L75
Even though the error text for humanify download sounds as though it would be capable of downloading any named model:
https://github.com/jehna/humanify/blob/85d17e73d6b760bf896d01dd99fa3a5f98ea2848/src/local-models.ts#L77-L85
And usually for LLM apps, the --model param would let us specify arbitrary models from huggingface or similar.

I ve been downloading the 8b for almost an hour now... So yeah any improvement in this regard will be great. Or maybe allow to just download the models then place them (I don't know where the model will be located as I didn't analyse the source code yet) But having them shared with other softwares like Ollama would be great (Symbolic link)

I don't know where the model will be located as I didn't analyse the source code yet

~/.humanifyjs/models
https://github.com/jehna/humanify/blob/14e4ae536588935edada4438cf28b3b5553b2596/src/local-models.ts#L13-L25

I don't know where the model will be located as I didn't analyse the source code yet

~/.humanifyjs/models
https://github.com/jehna/humanify/blob/14e4ae536588935edada4438cf28b3b5553b2596/src/local-models.ts#L13-L25

Yeah I figured that out and updated to Phi 3.5 but the PR shows an error (unrelated I guess as many dependabot PRs are rejected as well)

It looks like @jehna opted for directly using ipull in this PR:

https://github.com/jehna/humanify/pull/166

Rather than the createModelDownloader / combineModelDownloaders abstractions from node-llama-cpp mentioned above in https://github.com/jehna/humanify/issues/58#issuecomment-2381885999


[Query/suggestion] While loop in cross_validation_wrapper

So the reason no one is working on the issue of while loop in cross_validation _wrapper is because we are still not clear about the functionality of cross validation? Because if not, then isn't it a simple break or return statement after the if statement will break the loop after converging.

What do you exactly mean by 'converging' ?
Also, as much I understand, the protocol is, the code should closely follow the pseudocode in the book, unless confirmed by @norvig sir.
So my guess is, people are waiting for an update from sir, or a change in the pseudocode in the book.

https://github.com/aimacode/aima-python/blob/5e5b51c47a6573c636c424e67bcae8dbbbebc87e/learning.py#L1101-L1104
The if condition from above snippet is to check for convergence (new error is extremely close to previous error). This is what i meant by 'converging'.
So there is no use in working on cross validation until there is an update.

The convergence is checked on training error, and then, best size if determined according to the minimum validation error, as it is in the book.
Refer this to see the pseudocode which is followed in the code.
I hope I'm able to clarify this.


0.0.14 Push issues

Know that we are injecting the push instance, how do we send PushOptions to the constructor?

constructor(deps: PushDependencies, options?: PushOptions);

I'm getting console errors in the browser I wasn't getting before:

Push no longer has an onReady function, is it not needed?

Here's the code i'm using:
  this.platform.ready().then(() => {
      //this.push.onReady(() => {
        this.push.register().then((token) => {
          console.log("Device token:", token.token);
          this.push.saveToken(token);
        });
      //});
    });

Push options are part of the config now. But, there aren't many options. If you're looking for where to put your onNotification handler, it's done through the event emitter (for now... later it may be an observable).
const cloudSettings: CloudSettings = {
  'core': {
    'app_id': "YOUR-APP-ID"
  },
  'push': {} // PushOptions
};

import {EventEmitter} from '@ionic/cloud-angular';

constructor(emitter: EventEmitter)

emitter.on('push:notification': (data) => { });

The onReady function/callback is not needed, correct.

The console errors in the plugin are also correct -- you cannot do native push notifications in the browser.

Thanks for the detailed response @dwieeb, much appreciated.
It works nicely :)
Regarding number 3, till proper mocking or better yet an ionic-browser push implementation is created catching the errors our selves is ok.
Generally speaking I think that all the ionic-native cordova plugins should come with a browser mock.

I agree. We're very interested in implementing browser pushes & deploy functionality for the browser. The browser is a proper platform for Ionic and I think it will do us well to treat it as one for Ionic Cloud.
It sounds like these issues are resolved. If they are not, no worries. I'll reopen.

Thanks @dwieeb, not related to this issue, but Ionic and Ionic Platform have given me a lot of headaches over the past two years :) (to be fare a lot of the headache is Apple/Cordova related), but the dedication and responsiveness from you guys has kept me going and I do believe things are on the right track - keep it up and thanks for the hard work 👍

@sagivf Haha, yep. Ionic is venturing into uncharted territories and bringing a bunch of people with! I also think things are on the right track and I believe it's because of people like you. Thanks so much for working with me. This is the feedback we've needed. 🍻


Remove SourceLink package reference

We've been seeing some hot reload failures of the form: error ENC0003: Updating 'attribute' requires restarting the application
This is caused by SourceLink, which wants to write the git hash to the AssemblyInfo obj file.
Trail starts here: https://github.com/dotnet/sourcelink/pull/1144
SourceLink is included starting with the .NET 8 SDK, and I've tested that SourceLink is still working - dotnet pack and check the .nuspec in the contained package:
sh     $ cat ppy.osu.Framework.nuspec | grep rep     <repository type="git" url="https://github.com/ppy/osu-framework" commit="270b29d58b736deca2b6d76fe9907b2554d5979f" />

Interestingly, this has PrivateAssets="all" so it might not fix it...


Add wifi functionality to systemd server

Description of changes
wifi server is added to systemd server on net-vm
wifi AP list and connection nmcli queries are added
Testing
AP List:

Connect to gui-vm via SSH
Use grpcurl to query of wifi AP list: grpcurl -cacert /run/givc/ca-cert.pem -cert /etc/givc/gui-vm.ghaf-cert.pem -key /etc/givc/gui-vm.ghaf-key.pem <IP_ADDRESS>:9000 systemd.WifiService.ListNetwork
Observe the network list with in-use, SSID, signal strength and security lists.

Connect:

Connect to gui-vm via SSH
Replace ssid and password in the command and run grpcurl to connect to AP: grpcurl '{"SSID":"ssid","Password":"password"}' -cacert /run/givc/ca-cert.pem -cert /etc/givc/gui-vm.ghaf-cert.pem -key /etc/givc/gui-vm.ghaf-key.pem <IP_ADDRESS>:9000 systemd.WifiService.ConnectNetwork
Observe the network connection on the Waybar.

Tested this on Ghaf, wifi list and connect commands work fine.
grpcurl -d '{"SSID":"my-ssid","Password":"my-password"}' ...


Is cacheable-flash enabled?

I would like to be able to tell whether or not cacheable-flash is active and working in my app.
I am transitioning from in-memory session storage to cookie-based storage of the flash object but I don't necessarily want to delete my old code. I would rather have both options available. In the future for other reasons I may transition back to serving flash messages inside the HTML directly.
Something like CacheableFlash.active? which is set to true when the middleware is used or when included in a controller. I'd rather not simply detect the presence of the CacheableFlash constant is it may be included but not used.

Also, I noticed that when I tried to explicitly set Rails.application.config.session_store to ActionDispatch::Session::CacheStore, rails still has ActionDispatch::Session::CookieStore set. Does this gem override whatever default is set?

That would be a useful tool to have.  I don't have much time to work on this gem anymore myself, and would appreciate any help you can render.  I will look into the session store override issue.

I can't find any code in this project that would override the session store.  However, this gem does require a cookie store, so perhaps something in Rails is being "too smart" about that?  You should stop including this in the application controller, and/or unload the middleware.  This code will not work at all without a cookie store set.


feat: add salt_state_health gaugeVec

Add a new metric salt_state_health that will return 0 or 1 depending of the last time a state was run on the node.
This will allow to easily create alerting rules based on this metric.
For example, in a classic monitoring system, if the last state.highstate failed, it will generate an alert. Once it is back to success, it's going to automatically stop the alarm.

hi @arsiesys,
Thanks for the PR.
I had similar metric in the past, but I had to remove it because metrics cardinality was exploding.
I was thinking about adding similar feature, but with an option to filter on the state to watch.
If you are willing to adapt your PR, I would suggest to add a filter in the settings.
Something like: watchedFunctionsPerMinion, watchedStatePerMinion. (there might be a better name)
Values would be a slice of string.
What do you think?

I updated my branch to add some configurations options.
The best would be to make it possible to configure a set of association of "functions" with "states". I just did a basic filter for both of them. So this could be improved.

Thanks for the PR @arsiesys !!! :tada: :rocket:

released in v0.1.5

I missed one detail, the name of the metric is too precise and does not cover everything.
The metric concerns states, but also functions. I'll update the metric name from salt_state_health to salt_job_health in the next release.


剪切板管理和快捷便签功能Clipboard management and quick notes

Description of the new feature / enhancement
剪切板管理Clipboard management:
记录所有的剪切复制内容，快捷调出内容列表可以进行管理，且点击进行粘贴
Record all cut and copied content, quickly access the content list for management, and click to paste
快捷便签Quick notes:
快捷调出便签界面，输入内容后关闭便签或者关机也不会删除。界面功能应当简洁，建议仅有输入和删除字符功能
Quickly call up the note interface, and closing or shutting down the note after entering content will not delete it. The interface functions should be concise, and it is recommended to only have input and delete character functions
Scenario when this would be used?
every time
Supporting information
No response

I hope using this will provide me with a similar excellent experience as using Alfred on macOS.

Clipboard management is built-in into Windows (try Windows key + V). Quick notes is a good idea.

Clipboard management is built-in into Windows (try Windows key + V). Quick notes is a good idea.

It has, but not as good as Alfred's

Clipboard management is built-in into Windows (try Windows key + V). Quick notes is a good idea.

Win+V is too simple,and max count is short,and clear empty when sytem shutdown,and it use inconveniently for use

no search


Adapt widget_utils and messagesoverlay to new LCD overlay code for remove callbacks

Adapt to changes made for #3417.

[x] Update changelogs/metadata
[x] Try to merge exports.overlayRemoved and exports.offset in widget_utils

Looks great to me - thanks for your work on this!

I'll merge this if you're still happy with it @halemmerich?

Yes, I'm using this code without problems since then.


Binder

Hello, would it be possible to connect binder https://mybinder.org/ to this repo so one can have a live OCaml jupyter session in binder without installing anything?

Ah, it’s been already done. Somehow I missed it, sorry. 🤦♂


Don't scrape vcpu metrics when instance is in a wrong state

Feb 25 11:07:07 somehost.example.com libvirt-exporter[780603]: 2021/02/25 11:07:07 Failed to scrape metrics: virError(Code=55, Domain=10, Message='Requested operation is not valid: cannot retrieve vcpu information for inactive domain')

Fixed #10


Develop

``

well done @HubleGiter. Thanks for fixes, I've done  exactly the same for my another project and didn't have a time to commit changes.


Firebase Analytics & Installation Improvements

Created separate installation variables for iOS and Android app parameters
Initial Firebase Analytics Support
Fixed bug when uninstalling the plugin on the iOS platform (previously uninstalling the plugin would delete the entire app directory)
Fixed issue on iOS where GoogleService-Info.plist would have to be manually added to the build target.
GoogleService-Info.plist now resides within the Resources directory on iOS
Updated README with more details on installation

thx for the ios fixes. I have integrated most of your changes, except:

I dropped the conf variables to make it easier to install. Instead the plugin will simply look for GoogleService-Info.plist and google-services.json in the project root and copy them to the right place

on IOS, [FIRApp configure] is the entrypoint for the entire SDK, without it the FCM handlers are not triggered etc. Therefore analytics is now enabled as default on both platforms as part of pluginInitialize, and startAnalytics has been removed

@robertarnesson Great idea copying the files. WAY easier to setup. One issue though. We will still need the ANDROID_GOOGLE_APP_ID parameter on plugin installation. In plugin.xml this line copies the google app id to the android resources/vales/strings.xml file:
<config-file parent="/*" target="res/values/strings.xml">
    <string name="google_app_id">$ANDROID_GOOGLE_APP_ID</string>
</config-file>

This is needed on android for analytics to function properly. If you try to init analytics without this parameter on android you will get this error message in logcat:
06-27 09:33:08.460 E/FA      (25199): GoogleService failed to initialize, status: 10, Missing google app id value from from string resources with name google_app_id.
06-27 09:33:08.460 E/FA      (25199): Scheduler not set. Not logging error/warn.
06-27 09:33:08.480 V/FA      (25199): Registered activity lifecycle callback
06-27 09:33:08.510 E/FA      (25199): Measurement Service called without google_app_id
06-27 09:33:08.520 E/FA      (25199): Uploading is not possible. App measurement disabled

@Chuckv01 yes you are right. I'll update the hook. we can read the app id from google-services.json in there and write it to strings.xml

@robertarnesson I initially tried to do that from the hook. However, I ran into trouble parsing the xml. ><
When you want to add/remove the plugin, or the project is copied from a previous source, the app_id gets duplicated or out-of-date. If you're good at regex'es you might be able to hack together something relatively stable.
IMHO it is reasonable to ask people to provide that parameter at installation and use the plugin.xml config-file to copy it to the strings.xml file. That's the purpose of config-file after all :)

@robertarnesson Unless you want to include an xml parser in the hook script. Don't know about that though... sounds hacky.

@Chuckv01 I'm thinking I could use plugin.xml to insert a placeholder value then just search/replace in the hook. I'll play around with it. An extra variable is not a dealbreaker but if its possible to keep it clean without too much effort I'd prefer that

@robertarnesson Yeah great to have less installation work. I was running into problems with the insert then find/replace approach. Cordova was duplicating the app tag on prepare. I was probably missing something though.

@Chuckv01 seems like this did the trick a189c1c

@robertarnesson You're the man. Working great for me.


Allow neon_superuser to create event triggers

Problem
Only superuser is allows in Postgres to create event triggers.
See https://github.com/neondatabase/neon/issues/7570
Summary of changes
Allow creation of event triggers to neon_superuser
Checklist before requesting a review

[ ] I have performed a self-review of my code.
[ ] If it is a core feature, I have added thorough tests.
[ ] Do we need to implement analytics? if so did you add the relevant metrics to the dashboard?
[ ] If this PR requires public announcement, mark it with /release-notes label and add several sentences in this section.

Checklist before merging

[ ] Do not forget to reformat commit message to not include the above checklist

the Postgres submodule seems to have 200+ files updated...? probably you're not on the right branch :(


fixed proximity mine effect to match errata

Modified the effect text to match the errata as of rules v1.0.2 (Sept 19) https://images-cdn.fantasyflightgames.com/filer_public/2a/57/2a57fdb2-e13f-4d58-b9db-bb082dd23dcc/swzrulesreference_v102.pdf

Didn't realize that FFG also applied their "errata" text on page 20 of that rules reference. I'll go with that version instead because that's probably the page that everybody else is going to print out.


Cleanup the apisnoop box to only have the essential folders.

GIVEN That apisnoop developers want to be able to easily reason about our file structure and data flow
WHEN a developer ssh's into the apisnoop server
THEN they see a well-organized set of relevant files and nothing else
AND can easily reason about both the code and its infrastructure

After discussion, it was decided that it makes more sense to develop the ability to quickly build a new apisnoop.cncf.ci box, with all necessary tools and data, then setup pattern of maintaining the existing one.  This should be the focus for automating data flow, and so this issue will be closed to not be distracting.


Fix usage of PULSAR_EXTRA_OPTS/BOOKIE_EXTRA_OPTS in startup scripts

Motivation
The changes in #11285 broke pulsar-perf and it fails with error
root@3b36a0ea475b:/pulsar# ./bin/pulsar-perf produce
[0.000s][error][logging] Error opening log file 'logs/pulsar_gc_14.log': No such file or directory
[0.000s][error][logging] Initialization of output 'file=logs/pulsar_gc_%p.log' using options 'filecount=10,filesize=20M' failed.
Invalid -Xlog option '-Xlog:gc*:logs/pulsar_gc_%p.log:time,uptime:filecount=10,filesize=20M', see error log for details.
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.

When analyzing the problem, it seems that the usage of PULSAR_EXTRA_OPTS/BOOKIE_EXTRA_OPTS environment variables isn't correctly used. It should be possible to use PULSAR_EXTRA_OPTS/BOOKIE_EXTRA_OPTS to add additional environment variables. This PR fixes that issue.
Modifications

don't combine BOOKIE_MEM, BOOKIE_GC and BOOKIE_GC_LOG in BOOKIE_EXTRA_OPTS
don't add PULSAR_GC_LOG to PULSAR_EXTRA_OPTS
make BOOKIE_GC_LOG default to PULSAR_GC_LOG for consistency
apply PULSAR_EXTRA_OPTS and BOOKIE_EXTRA_OPTS after other settings so that it is possible to override
settings defined by PULSAR_MEM / PULSAR_GC / BOOKIE_MEM / BOOKIE_GC.

Documentation

[x] no-need-doc

Please make sure this change is referenced in the 2.10 release notes as it is a breaking change for OpenMessaging/Benchmarks (at least for me)


GPT3_1_3B configuration for a v3-32 TPU

Hi,
Thank you for releasing the training code for GPT-NEO. We want to train a new model from  scratch using own vocabulary and corpus.
Due to time constraints and resources we want to train a 1.3 billion parameters (GPT_1.3B) model.
We are not sure which config suits best our needs https://github.com/EleutherAI/gpt-neo/tree/master/configs  or if we have to modify an existing config for training the 1.3B parameters model.

The XL Config is a 1.3B parameter model. The naming scheme is admittedly confusing, but chosen to be consistent with GPT-3's. I regularly consult Table 2.1 from that paper as a cheat sheet:

Note that you will likely need to modify some arguments such as train_batch_size and train_steps based on your dataset and mesh_shape and layout based on your hardware. These values for mesh shape and layout are for a v3-256 TPU.
If you are interested in training on GPUs instead of TPUs, I recommend using our recent GPU codebase instead. We haven't released any models trained using it yet, but it's been highly performant in internal testing and we hope to release some models soon.

Thank you for the quick response :). We have a 32 TPUv3 server,that s the reason why we use gpt_neo not gpt_neox.

Gotcha! I haven't experimented with a v3-32 in over a year at this point, but I think we found that a N x 2 mesh is generally the most effective. That said, there's only a handful of combinations for you to check so I would run them all for a couple hours and see what gets the best performance.
Best of luck with training your models!


fix fatfs_create_spiflash_image() feature (IDFGH-6383) (IDFGH-6518)

Open the files in binary mode, and remove the redundant encode.
Fixes #8042

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Thanks for your contribution.

sha=df2ae4689cd1056bfda67c9dbe08c015da1202aa


Doesn't work on Internet Explorer??

Has anyone noticed that react-id-swiper simply doesn't work or load on Internet Explorer. I've tried and it works everywhere, except on IE I get a blank page.
Even if you load the demo site on IE it's broken.
https://kidjp85.github.io/react-id-swiper/
Is there anyway to make this compatible with IE?

Anyone gonna look into this problem? Meeting with the same issue

@oelbaganwg  Any solution or work around ?

@oelbaganwg @wuzinong did you take a look at issue #273 - I just commented there...

@skaltenegger i use rollup? how would one accomplish this? For now i reverted back to older version until i can get this working in ie11.
Thanks in advance.

Hi @oelbaganwg , does this show any error ? And which version of 'react-id-swiper' are u using ?

I think the issue may be unrelated, because I tried using react id swiper on a clean project without any other plugins and it worked on IE. I still can't pinpoint where the clash would be with other plugins.

There is another option I think you guys can try is building your own Swiper locally by changing Gulp config in Swiper's package and put it in your code base.  More detail here


Can't perform schema changes

Environment information
System:
  OS: macOS 14.3.1
  CPU: (10) arm64 Apple M1 Pro
  Memory: 686.28 MB / 32.00 GB
  Shell: /bin/zsh
Binaries:
  Node: 18.19.0 - ~/.nvm/versions/node/v18.19.0/bin/node
  Yarn: undefined - undefined
  npm: 10.2.3 - ~/.nvm/versions/node/v18.19.0/bin/npm
  pnpm: undefined - undefined
NPM Packages:
  @aws-amplify/backend: 0.13.0-beta.14
  @aws-amplify/backend-cli: 0.12.0-beta.16
  aws-amplify: 6.0.26
  aws-cdk: 2.135.0
  aws-cdk-lib: 2.135.0
  typescript: 5.4.3
AWS environment variables:
  AWS_STS_REGIONAL_ENDPOINTS = regional
  AWS_NODEJS_CONNECTION_REUSE_ENABLED = 1
  AWS_SDK_LOAD_CONFIG = 1
No CDK environment variables

Description
I run the command:
npx amplify sandbox --profile amplify-event-registration
And I get the following output:
✨  Synthesis time: 0.03s
⚠ The --hotswap and --hotswap-fallback flags deliberately introduce CloudFormation drift to speed up deployments
⚠ They should only be used for development - never use them for your production Stacks!
amplify-eventregistration-carskoch-sandbox-c6933caeee: deploying... [1/1]
✨ hotswapping resources:
✨ AWS::AppSync::FunctionConfiguration 'MutationcreateRegistrationauth0Function'
✨ AWS::AppSync::FunctionConfiguration 'MutationupdateRegistrationauth0Function'
✨ AWS::AppSync::GraphQLSchema 'm3oq6glwevgi5jozts3kjw3ioyGraphQLSchema'
✨ Contents of S3 Bucket 'amplify-eventregistration-amplifydataamplifycodege-vw3zkgzxx4av'
✨ Contents of S3 Bucket 'amplify-eventregistration-amplifydataamplifycodege-vw3zkgzxx4av' hotswapped!
The CloudFormation deployment has failed.
Caused By: ❌ Deployment failed: AccessDeniedException: User: arn:aws:iam::711568858726:user/amplify-dev is not authorized to perform: appsync:ListFunctions on resource: arn:aws:appsync:eu-west-1:711568858726:/v1/apis/m3oq6glwevgi5jozts3kjw3ioy/functions
Resolution: Find more information in the CloudFormation AWS Console for this stack.
[Sandbox] Watching for file changes...
This problem happened after I was trying to add a field to an existing model.

I checked the CloudTrail and couldn't find a record where the appsync:ListFunctions API has been called.

Hey @cabcookie, thank you for reaching out. From the error message, it appears the IAM user profile being utilized does not allow "appsync:ListFunctions",. Does the IAM user amplify-dev have AmplifyBackendDeployFullAccess policy attached?

Yes, that actually solved it. I only had AdministratorAccess-Amplify assigned to it.


Extract Only Year from text

Thanks for this great project.
Currently I am able to extract the dates, but for only year i.e for eample "In year 2011 the incident happened." The program retrieves "2011-01-01 00:00:00+00".
But we need to retrieve it as "2011-01-01 12:14:12+00"
Can you please let me know how should I change in the library to achieve this.
The basic Aim is to differentiate the original "1st Jan 2011" and "2011".
Thanks

Great question. Give me 24 hours and I'll have a solution for you :)

Thanks man!!!!!!
waiting for your reply :)

Hey, I thought about it a lot and this is what I came up with.  You can set return_precision to True and the functions will return a tuple of (date, precision).  Precision can be "year", "month", or "day".  So precision is "day" for  "1st Jan 2011" and "year" for "2011".  Consult the Readme for a full example.  Let me know if this doesn't work for you and we can work on another solution!  Thanks for your interest!

Hi Daniel,
Thanks for the immediate update, I tried it and its working perfectly fine for our requirement. One more issue which I am facing is in normalisation of dates in the format "98".
For example my text is :

He was selected by the Sacramento Kings in the 2nd round (48th overall) of the 2004 NBA_Draft. A 6'4' guard from Morehead State University, Minard was signed by the Kings in July 2004, but they waived him in November the same year, and so far he has never appeared in an NBA game.

Ouput :

[(datetime.datetime(1948, 1, 1, 0, 0, tzinfo=), 'year'), (datetime.datetime(2004, 1, 1, 0, 0, tzinfo=), 'year'), (datetime.datetime(2004, 7, 1, 0, 0, tzinfo=), 'month')]

So here the 1948 year should not have been fetched.
I think we can solve this issue if we implement a login to only normalise those 2 digits which are preceded by "-" and not followed by "th"|"st"|"nd"|rd.
Please let any know if you have any other solution  to resolve this.
Thanks & Regards,
M Swathi Mithran

@swathimithran, thanks for the example.  Your help is sincerely appreciated!  As a quick fix, I made it so it won't capture ordinal numbers that end in th.  You can view the change here: https://github.com/DanielJDufour/date-extractor/commit/68deab4d342cec33e690e19cb0562e9ae0b52b44
However, we will need more discussion on what rule should be used for what precedes the 2 digits.  Here's a few examples of 2 digit years:

12/23/09
15-11-21
9/1/99 22:00
paper_170120 (in a filename)
taxes_16.docx

Basically, I'm afraid that if we make the rule too strict, people won't be able to parse dates out of filenames.
Here's a few possible solutions.  What would you like?
Option 1
Add a parameter source_type, which could be filename, filepath, text, html, or javascript.  This way you could customize it, so you can restrict the rules to certain types of sources.  Here's an example of what this could look like
from date_extractor import extract_date
string = "I went to my first basketball game in 1990.  My favorite player had number 34."
date = extract_date(string, source_type="text")

string = "my_resume_091216.pdf"
date = extract_date(string, source_type="filename")

Option 2
The second option could be allowing users to override existing patterns.
import date_extractor
date_extractor.patterns['y'] = "\d{4}"
string = "The author asserts that the earliest encounter never happened (43)"
date_extractor.extract_dates(string)

Option 3
The third option could be returning a confidence level, low, medium, or high.  You could then filter depending on your need.
from date_extractor import extract_dates
string = "He was selected by the Sacramento Kings in the 2nd round (48th overall) of the 2004 NBA_Draft. A 6'4' guard from Morehead State University, Minard was signed by the Kings in July 2004, but they waived him in November the same year, and so far he has never appeared in an NBA game."
dates = extract_dates(string, return_confidence)
# dates = [(datetime.datetime(1948, 1, 1, 0, 0, tzinfo=), 'low'), (datetime.datetime(2004, 1, 1, 0, 0, tzinfo=), 'medium'), (datetime.datetime(2004, 7, 1, 0, 0, tzinfo=), 'high')]

Option 4
I'm open to suggestions as long as it doesn't prevent users from extracting years out of filenames.
Which do you prefer? What do you think?


Add containers on windows server 2022 workflows

Add containers on windows server 2022 workflows
/cc bkatyl

Close the PR as per discussion with Peter.


Feature/equipment measurement

This is an amended PR for https://github.com/PoseiDAT/schemas/pull/7
The changes compared to the original PR are:

measurement required/key refactor
renamed vessel-location to vessel-section
add contact details schema
enum spelling and word breaks
linting fixes and export ordering

The measurement change is the main focus for this and the original PR. The proposed structured here prevents value.value paths. It also allow for property existence checks to determine the measurement type.
So if a .position exists it is a position measurement. This also leaves us the option to register combined measurements but the requirement remains it should contain at least one.
The vessel-location schema was a little confusion with the positional schema names. I propose to rename it to vessel section.
Contact details schema was added for reusability. I imagine more persona schemas will be added which will want contact details
The rest are spelling and linting fixes. I also added words breaks with underscore (_) in the enums for readability. It's always a shame when a spelling mistake makes it into a spec because you are stuck with it for a long time.

@Qwerios thanks for the update. The change you made to the measurement looks very good to me as it was what I intended to do in my schema. I had not yet extensively used JSON schema in the past and could not find a very good solution. It was this construction I did not know about:
    "type",	   { "required": [ "type", "numeric" ] },
    "value"	   { "required": [ "type", "position" ] }
  ],	 ],

If I understand correctly this means that a measurement  must have at least one of the properties numeric or position. This is exactly what we do now in the concentrator.
Thank you for the spelling update, I had overlooked the CIRCUITBRAKER one. The underscores where intentionally left out by me as I had not seen them in the other enums and I thought is was by design. I will try to look if we can extend the linter with a UK english spelling check, this might come in handy in the future I think.
For my this pull request can be merged and the other one closed.

@Qwerios thanks for the update. The change you made to the measurement looks very good to me as it was what I intended to do in my schema. I had not yet extensively used JSON schema in the past and could not find a very good solution. It was this construction I did not know about:
    "type",	   { "required": [ "type", "numeric" ] },
    "value"	   { "required": [ "type", "position" ] }
  ],	 ],

If I understand correctly this means that a measurement  must have at least one of the properties numeric or position. This is exactly what we do now in the concentrator.
Thank you for the spelling update, I had overlooked the CIRCUITBRAKER one. The underscores where intentionally left out by me as I had not seen them in the other enums and I thought is was by design. I will try to look if we can extend the linter with a UK english spelling check, this might come in handy in the future I think.
For my this pull request can be merged and the other one closed.

@daanpape that JSON schema construct does exactly that and is very handy. We can keep amending it with new measurement types in the future.
Are you also okay with the vessel section change? I wasn't 100% sure that name matches your intent

@daanpape that JSON schema construct does exactly that and is very handy. We can keep amending it with new measurement types in the future.
Are you also okay with the vessel section change? I wasn't 100% sure that name matches your intent

@Qwerios I'm very okay with the change. I had also my doubts about the vessel-location name because it could indeed be very confusing. vessel-section seems like a good terminology to cover what it's ment for.

@Qwerios I'm very okay with the change. I had also my doubts about the vessel-location name because it could indeed be very confusing. vessel-section seems like a good terminology to cover what it's ment for.


PR #4 : Update OTA PAL

Checklist:

[ ] I have tested my changes. No regression in existing tests.
[ ] My code is Linted.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

/bot run checks

/bot run checks

/bot run checks


ci: add release workflow

This PR adds a release workflow using https://github.com/googleapis/release-please.
There are few things to do to make it work:

You would need to set a NPM_TOKEN secret in this GitHub repo.
You may need to set "Allow GitHub Actions to create and approve pull requests" under repository Settings > Actions > General.
When merging this pull request, ensure that the commit message body (not commit message header) contains Release-As: 0.0.8 to trigger the very first release.

Thanks!