Minimal reproducible example

Before I can start refactoring into a package I need a minimal reproducible example that can be used as a regression test.
@jjriley1 has provided data and tweaked things on the slam_3UIs jjriley1/1-tidy-up branch.
Issues serves as a place for me to document progress and understanding of how the pipeline works as I move towards putting an initial regression test in place.

Running the pipeline
The cgat tools are a pipeline for gluing things together, useful commands (via @jjriley1).
 This is invoked by running the library as a module with python -m pipeline_slam_3UIs <options> where <options> control how the cgat tools are run.
@jjriley1 has annotated the functions on this branch with basic docstrings describing the braod purpose of the function.
It is worth noting that external tools appear to be required such as samtools and the Python is acting as a wrapper to calling these.
--help
Shows a bunch of help on the options that are available. Not clear to me at present whether these are coming from ruffus ( also ruffus (ReadTheDocs) but less useful) or cgat.
❱ python pipeline_slam_3UIs.py --help
Usage:
usage: pipeline_slam_3UIs.py [OPTIONS] [CMD] [target]

Execute pipeline pipeline_slam_3UIs.py.

Commands can be any of the following

make <target>
   run all tasks required to build *target*

show <target>
   show tasks required to build *target* without executing them

plot <target>
   plot image (using inkscape) of pipeline state for *target*

debug <target> [args]
   debug a method using the supplied arguments. The method <target>
   in the pipeline is run without checking any dependencies.

config
   write new configuration files pipeline.yml with default values

dump
   write pipeline configuration to stdout

touch
   touch files only, do not run

regenerate
   regenerate the ruffus checkpoint file

check
   check if requirements (external tool dependencies) are satisfied.

clone <source>
   create a clone of a pipeline in <source> in the current
   directory. The cloning process aims to use soft linking to files
   (not directories) as much as possible.  Time stamps are
   preserved. Cloning is useful if a pipeline needs to be re-run from
   a certain point but the original pipeline should be preserved.

Options:
  --version             show program's version number and exit
  -h, --help            show this help message and exit
  --pipeline-action=PIPELINE_ACTION
                        action to take [default=none].
  --pipeline-format=PIPELINE_FORMAT
                        pipeline format [default=svg].
  -n, --dry-run         perform a dry run (do not execute any shell commands)
                        [default=False].
  -c CONFIG_FILE, --config-file=CONFIG_FILE
                        benchmark configuration file [default=pipeline.yml].
  -f FORCE_RUN, --force-run=FORCE_RUN
                        force running the pipeline even if there are up-to-
                        date tasks. If option is 'all', all tasks will be
                        rerun. Otherwise, only the tasks given as arguments
                        will be rerun. [default=False].
  -p MULTIPROCESS, --multiprocess=MULTIPROCESS
                        number of parallel processes to use on submit host
                        (different from number of jobs to use for cluster
                        jobs) [default=none].
  -e, --exceptions      echo exceptions immediately as they occur
                        [default=True].
  -i, --terminate       terminate immediately at the first exception
                        [default=none].
  -d, --debug           output debugging information on console, and not the
                        logfile [default=False].
  -s VARIABLES_TO_SET, --set=VARIABLES_TO_SET
                        explicitely set paramater values [default=[]].
  --input-glob=INPUT_GLOBS, --input-glob=INPUT_GLOBS
                        glob expression for input filenames. The exact format
                        is pipeline specific. If the pipeline expects only a
                        single input, `--input-glob=*.bam` will be sufficient.
                        If the pipeline expects multiple types of input, a
                        qualifier might need to be added, for example
                        `--input-glob=bam=*.bam` --input-glob=bed=*.bed.gz`.
                        Giving this option overrides the default of a pipeline
                        looking for input in the current directory or
                        specified the config file. [default=[]].
  --checksums=RUFFUS_CHECKSUMS_LEVEL
                        set the level of ruffus checksums[default=0].
  -t, --is-test         this is a test run[default=False].
  --engine=ENGINE       engine to use.[default=local].
  --always-mount        force mounting of arvados keep [False]
  --only-info           only update meta information, do not run
                        [default=False].
  --work-dir=WORK_DIR   working directory. Will be created if it does not
                        exist [default=none].
  --input-validation    perform input validation before starting
                        [default=False].

  pipeline logging configuration:
    --pipeline-logfile=PIPELINE_LOGFILE
                        primary logging destination.[default=pipeline.log].
    --shell-logfile=SHELL_LOGFILE
                        filename for shell debugging information. If it is not
                        an absolute path, the output will be written into the
                        current working directory. If unset, no logging will
                        be output. [default=none].

  Script timing options:
    --timeit=TIMEIT_FILE
                        store timeing information in file [none].
    --timeit-name=TIMEIT_NAME
                        name in timing file for this class of jobs [all].
    --timeit-header     add header for timing information [none].

  Common options:
    --random-seed=RANDOM_SEED
                        random seed to initialize number generator with
                        [none].
    -v LOGLEVEL, --verbose=LOGLEVEL
                        loglevel [1]. The higher, the more output.
    --log-config-filename=LOG_CONFIG_FILENAME
                        Configuration file for logger [logging.yml].
    --tracing=TRACING   enable function tracing [none].
    -?                  output short help (command line options only.

  cluster options:
    --no-cluster, --local
                        do no use cluster - run locally [False].
    --cluster-priority=CLUSTER_PRIORITY
                        set job priority on cluster [none].
    --cluster-queue=CLUSTER_QUEUE
                        set cluster queue [none].
    --cluster-num-jobs=CLUSTER_NUM_JOBS
                        number of jobs to submit to the queue execute in
                        parallel [none].
    --cluster-parallel-environment=CLUSTER_PARALLEL_ENVIRONMENT
                        name of the parallel environment to use [none].
    --cluster-options=CLUSTER_OPTIONS
                        additional options for cluster jobs, passed on to
                        queuing system [none].
    --cluster-queue-manager=CLUSTER_QUEUE_MANAGER
                        cluster queuing system [sge].
    --cluster-memory-resource=CLUSTER_MEMORY_RESOURCE
                        resource name to allocate memory with [none].
    --cluster-memory-default=CLUSTER_MEMORY_DEFAULT
                        default amount of memory to allocate [unlimited].

  Input/output options:
    -I FILE, --stdin=FILE
                        file to read stdin from [default = stdin].
    -L FILE, --log=FILE
                        file with logging information [default = stdout].
    -E FILE, --error=FILE
                        file with error information [default = stderr].
    -S FILE, --stdout=FILE
                        file where output is to go [default = stdout].

Show what would be done
python -m pipeline_slam_3UIs --local show full -v5

--local runs this on the locally rather than on a cluster.
show full shows what would be done if the "full" pipeline were run.
-v5 sets the loglevel to be highly verbose

pipeline_slam_3UIs.py
full()
The full() function defines what will be done, it actually doesn't contain anything as the body of the function is simply pass. However, it does control what is run by virtue of using @ruffus.follows() decorator which takes a list of "tasks" (i.e. other functions or steps in the pipeline) that are to be run first.
The minimal reproducible example holds...
@ruffus.follows(
    name_sort_bams,
    featureCountsReadAssignments,
    VCF_from_no4sU_control,
    index_VCF,
    all_introns_counts_and_info,
    summarize_all_introns_counts,
)
def full():
    """
    Utility function to run the whole pipeline via:
    python pipeline_slam_3UIs.py make full -v5
    """
    pass

...so we will go through what each of these does in turn.
Configuration comes from the pipeline.yaml file that, based on the cgatcore.pipeline.get_parameters() function can be in one of three locations...

"%s/pipeline.yml" % os.path.splitext(__file__)[0] - looks at the current filename and looks within a directory of the same name for pipeline.yaml (DEFAULT - current location of pipeline.yaml)
"../pipeline.yaml" - looks for the file a directory level up.
"pipeline.yaml" - the current directory.

name_sort_bams()
@ruffus.follows(ruffus.mkdir("name_sorted_bams"))
@ruffus.transform(
    "input_bams.dir/*.bam",
    ruffus.regex("input_bams.dir/(.+).bam"),
    r"name_sorted_bams/\1.sorted.bam",
)
def name_sort_bams(infile, outfile):
    """
    Takes `.bam` file and sorts it in name order, therefore reads from the same pair appear sequentially
    """
    statement = "samtools sort -n %(infile)s -o %(outfile)s"
    P.run(statement, job_memory="8G")

This function doesn't itself sort the .bam files.
1
The @ruffus.follows(ruffus.mkdir("name_sorted_bams")) decorator makes the named directory.
2
A system call to samtools to run the file operation sort by "read name"...
 ❱ samtools sort
Usage: samtools sort [options...] [in.bam]
Options:
  -l INT     Set compression level, from 0 (uncompressed) to 9 (best)
  -u         Output uncompressed data (equivalent to -l 0)
  -m INT     Set maximum memory per thread; suffix K/M/G recognized [768M]
  -M         Use minimiser for clustering unaligned/unplaced reads
  -R         Do not use reverse strand (only compatible with -M)
  -K INT     Kmer size to use for minimiser [20]
  -I FILE    Order minimisers by their position in FILE FASTA
  -w INT     Window size for minimiser indexing via -I ref.fa [100]
  -H         Squash homopolymers when computing minimiser
  -n         Sort by read name (natural): cannot be used with samtools index
  -N         Sort by read name (ASCII): cannot be used with samtools index
  -t TAG     Sort by value of TAG. Uses position as secondary index (or read name if -n is set)
  -o FILE    Write final output to FILE rather than standard output
  -T PREFIX  Write temporary files to PREFIX.nnnn.bam
      --no-PG
               Do not add a PG line
      --template-coordinate
               Sort by template-coordinate
      --input-fmt-option OPT[=VAL]
               Specify a single input file format option in the form
               of OPTION or OPTION=VALUE
  -O, --output-fmt FORMAT[,OPT[=VAL]]...
               Specify output format (SAM, BAM, CRAM)
      --output-fmt-option OPT[=VAL]
               Specify a single output file format option in the form
               of OPTION or OPTION=VALUE
      --reference FILE
               Reference sequence FASTA FILE [null]
  -@, --threads INT
               Number of additional threads to use [0]
      --write-index
               Automatically index the output files [off]
      --verbosity INT
               Set level of verbosity

However the @ruffus.transform() decorator is used to provide the input and output arguments to the name_sort_bams() function.
This itself takes positional arguments (no named arguments are used as written so it is inferred that these are...

input : "input_bams.dir/*.bam" - a directory where there are .bam file(s), Question - will this only ever be one file or are there lots? I'm guessing lots which is why the various wrappers are used :thinking:
filter : ruffus.regex("input_bams.dir/(.+).bam") - filters the list for files that are in the explicit directory input_bams.dir/ and end with .bam whatever the file is called is stored and used in the next step...
replace_inputs : r"name_sorted_bams/\1.sorted.bam" - Puts the output into the name_sorted_bams directory, each file uses the stem of the filename and appends the .sorted.bam suffix.

3
This list of input/output is then passed to cgatcore.pipeline.run() which assigns 8GB of RAM to the job. Not currently clear if each job (i.e. file) is sent individually as the only named arguments used in the call is job_memory= (which indicates that this is allocated on per thread but its not clear what the default job_threads is as its not explicitly included.
Comments

May be able to replace this using the pathlib library to glob the filenames and then build a dictionary of input (key) and output (value) which is then parallelised.
Need to find out how many jobs are used by default

featureCountsReadAssignments()
@ruffus.follows(ruffus.mkdir("read_assignments"))
@ruffus.transform(
    name_sort_bams,
    ruffus.regex("name_sorted_bams/(.+).sorted.bam"),
    r"read_assignments/\1/\1.sorted.assigned.bam",
)
def featureCountsReadAssignments(infile, outfile):
    """
    Takes `.bam` file and adds 2 tags to it:
        - XT tag tells us the transcript the reads map to
        - XS tag tells us which strand the gene is transcribed from
    """
    annotation = PARAMS["transcript_gtf"]
    outdir = os.path.dirname(outfile)
    rename_outfile = outfile.replace(".assigned.bam", "")
    statement = """featureCounts -a %(annotation)s
                                 -o %(outdir)s/counts.txt
                                 -T 2
                                 -R BAM
                                 %(infile)s &&
                    mv %(rename_outfile)s.bam.featureCounts.bam %(outfile)s"""
    P.run(statement, job_threads=2, job_memory="8G")

Again this function isn't actually doing the stated work, rather it is making a system call to featureCounts with various arguments supplied by the function (presumably as there are multiple files to be running this on).
1
Again a decorator (@ruffus.follows(ruffus.mkdir("read_assignments"))) is used to create the named directory.
2
A second @ruffus.transform() decorator is used to take the output from name_sort_bams (presumably a list of files returned by cgatcore.pipeline.run()????).
As with the previous step a regular expression is used to pick up the filenames from the previously created name_sorted_bams/(.+).sorted.bam and created a set of nested directories under the read_assignments/ directory which has been created. There is a directory for each file and within each the file stem is suffixed with .sorted.assigned.bam.
3
annotation = PARAMS["transcript_gtf"] is extracted from the PARAMS dictionary (loaded from wherever pipeline.yaml has been found. In this minimal example the value is files_for_regression_testing/test_wash1.gtf which I don't have a copy of (yet!)
4
outdir = os.path.dirname(outfile) is extracted from the filename.
5
The .assigned.bam which was appended to the filename in the @ruffus.transform() decorator is now removed to leave the rename_outfile
VCF_from_no4sU_control()
@ruffus.transform(
    "input_bams.dir/*no4sU*.bam",
    ruffus.regex("input_bams.dir/(.+)_no4sU_(.+).bam"),
    r"snp_vcf/\1.vcf.gz",
)
def VCF_from_no4sU_control(infile, outfile):
    """
    Takes the negative control `.bam` file (via the no4sU regex) and creates a SNP VCF file
    using Varscan. It looks for differences between the `.bam` file and the genome `.fasta`
    file to detect differences, where these are always present at the same position (above
    a treshold, see Varscan docs) it is classed as a SNP and included in output VCF file.
    """
    outfile_uncompressed = outfile.replace(".vcf.gz", ".vcf")
    genome_fasta = PARAMS["genome_fasta"]
    statement = """samtools mpileup -B -A -f %(genome_fasta)s %(infile)s |
                    varscan mpileup2snp --variants 1 --output-vcf 1 > %(outfile_uncompressed)s &&
                    bcftools view %(outfile_uncompressed)s -Oz -o %(outfile)s"""
    # removed sort -k1,1 -k2,2n %(outfile_uncompressed)s -o %(outfile_uncompressed)s &&
    # before bcftools view
    P.run(statement, job_threads=12, job_memory="1G")

index_VCF()
@ruffus.transform(VCF_from_no4sU_control, ruffus.suffix(".vcf.gz"), ".vcf.gz.tbi")
def index_VCF(infile, outfile):
    """
    Indexes the VCF file so it can be read with Pysam, generates the .vcf.gz.tbi file
    """
    statement = "tabix -p vcf %(infile)s"
    P.run(statement)

all_introns_counts_and_info()
@ruffus.follows(index_VCF, ruffus.mkdir("all_introns_counts_and_info"))
@ruffus.transform(
    featureCountsReadAssignments,
    ruffus.regex("read_assignments/(.+)/(.+)_(.+)_(.+)_(.+)_(.+).sorted.assigned.bam"),
    ruffus.add_inputs(r"snp_vcf/\2.vcf.gz"),
    r"all_introns_counts_and_info/\1.tsv",
)
def all_introns_counts_and_info(infiles, outfile):
    """
    Takes in the sorted and feature assigned `.bam` file from previous steps and passes them to a
    python script that uses pysam (python wrapper for htslib). This script iterates over the `.bam`
    file pair-by-pair (representing the sequencing insert), determines whether the read-pair shows
    evidence of intron splicing/retention and assigns these to specific events by referencing the
    `.gtf` and `.bed` files, and XT tag from featureCountsReadAssignments. Next, the script uses the
    XS tag from featureCountsReadAssignment to assign each read in the pair as the forward or reverse
    read, relative to the direction of transcription. Finally, it looks for T>C in FW read, A>G in RV
    read, checks these are not present in the SNP VCF file, and outputs metadata on each read-pair
    about it's event assignment, number of conversions, coverage etc.
    """
    bamfile, vcffile = infiles
    annotation = PARAMS["transcript_gtf"]
    utron_bed = PARAMS["all_introns_bed6"]
    script_path = (
        os.path.dirname(os.path.abspath(__file__))
        + "/pipeline_slam_3UIs/all_introns_counts_and_info.py"
    )

    statement = """python %(script_path)s -b %(bamfile)s
                                            -g %(annotation)s
                                            -o %(outfile)s
                                            -vcf %(vcffile)s
                                            -bed %(utron_bed)s -v5"""

    P.run(statement, job_memory="16G")

summarize_all_introns_counts()
@ruffus.follows(ruffus.mkdir("all_introns_counts_and_info/summarized"))
@ruffus.collate(
    all_introns_counts_and_info,
    ruffus.regex("(.+)/(.+)_.+(hr|4sU).+"),
    r"\1/summarized/\2_summarized.tsv",
)
def summarize_all_introns_counts(infiles, outfile):
    """
    R script to summarize the number of converted counts from each gene in each sample relative to
    unconverted counts, therefore we calcualte the "percent_converted" for each gene in each sample.
    """
    input_folder = os.path.dirname(infiles[0])
    day_regex = os.path.basename(outfile).split("_")[0]
    script_path = (
        os.path.dirname(os.path.abspath(__file__))
        + "/pipeline_slam_3UIs/summarize_counts.R"
    )
    statement = "Rscript %(script_path)s %(input_folder)s %(day_regex)s %(outfile)s"

    P.run(statement, job_memory="64G")

name_sort_bams()
@ruffus.follows(ruffus.mkdir("name_sorted_bams"))
@ruffus.transform(
    "input_bams.dir/*.bam",
    ruffus.regex("input_bams.dir/(.+).bam"),
    r"name_sorted_bams/\1.sorted.bam",
)
def name_sort_bams(infile, outfile):
    """
    Takes `.bam` file and sorts it in name order, therefore reads from the same pair appear sequentially
    """
    statement = "samtools sort -n %(infile)s -o %(outfile)s"
    P.run(statement, job_memory="8G")

This function doesn't itself sort the .bam files.
1
The @ruffus.follows(ruffus.mkdir("name_sorted_bams")) decorator makes the named directory.
2
A system call to samtools to run the file operation sort by "read name"...
 ❱ samtools sort
Usage: samtools sort [options...] [in.bam]
Options:
  -l INT     Set compression level, from 0 (uncompressed) to 9 (best)
  -u         Output uncompressed data (equivalent to -l 0)
  -m INT     Set maximum memory per thread; suffix K/M/G recognized [768M]
  -M         Use minimiser for clustering unaligned/unplaced reads
  -R         Do not use reverse strand (only compatible with -M)
  -K INT     Kmer size to use for minimiser [20]
  -I FILE    Order minimisers by their position in FILE FASTA
  -w INT     Window size for minimiser indexing via -I ref.fa [100]
  -H         Squash homopolymers when computing minimiser
  -n         Sort by read name (natural): cannot be used with samtools index
  -N         Sort by read name (ASCII): cannot be used with samtools index
  -t TAG     Sort by value of TAG. Uses position as secondary index (or read name if -n is set)
  -o FILE    Write final output to FILE rather than standard output
  -T PREFIX  Write temporary files to PREFIX.nnnn.bam
      --no-PG
               Do not add a PG line
      --template-coordinate
               Sort by template-coordinate
      --input-fmt-option OPT[=VAL]
               Specify a single input file format option in the form
               of OPTION or OPTION=VALUE
  -O, --output-fmt FORMAT[,OPT[=VAL]]...
               Specify output format (SAM, BAM, CRAM)
      --output-fmt-option OPT[=VAL]
               Specify a single output file format option in the form
               of OPTION or OPTION=VALUE
      --reference FILE
               Reference sequence FASTA FILE [null]
  -@, --threads INT
               Number of additional threads to use [0]
      --write-index
               Automatically index the output files [off]
      --verbosity INT
               Set level of verbosity

However the @ruffus.transform() decorator is used to provide the input and output arguments to the name_sort_bams() function.
This itself takes positional arguments (no named arguments are used as written so it is inferred that these are...

input : "input_bams.dir/*.bam" - a directory where there are .bam file(s), Question - will this only ever be one file or are there lots? I'm guessing lots which is why the various wrappers are used :thinking:
filter : ruffus.regex("input_bams.dir/(.+).bam") - filters the list for files that are in the explicit directory input_bams.dir/ and end with .bam whatever the file is called is stored and used in the next step...
replace_inputs : r"name_sorted_bams/\1.sorted.bam" - Puts the output into the name_sorted_bams directory, each file uses the stem of the filename and appends the .sorted.bam suffix.

3
This list of input/output is then passed to cgatcore.pipeline.run() which assigns 8GB of RAM to the job. Not currently clear if each job (i.e. file) is sent individually as the only named arguments used in the call is job_memory= (which indicates that this is allocated on per thread but its not clear what the default job_threads is as its not explicitly included.
Comments

May be able to replace this using the pathlib library to glob the filenames and then build a dictionary of input (key) and output (value) which is then parallelised.
Need to find out how many jobs are used by default

featureCountsReadAssignments()
@ruffus.follows(ruffus.mkdir("read_assignments"))
@ruffus.transform(
    name_sort_bams,
    ruffus.regex("name_sorted_bams/(.+).sorted.bam"),
    r"read_assignments/\1/\1.sorted.assigned.bam",
)
def featureCountsReadAssignments(infile, outfile):
    """
    Takes `.bam` file and adds 2 tags to it:
        - XT tag tells us the transcript the reads map to
        - XS tag tells us which strand the gene is transcribed from
    """
    annotation = PARAMS["transcript_gtf"]
    outdir = os.path.dirname(outfile)
    rename_outfile = outfile.replace(".assigned.bam", "")
    statement = """featureCounts -a %(annotation)s
                                 -o %(outdir)s/counts.txt
                                 -T 2
                                 -R BAM
                                 %(infile)s &&
                    mv %(rename_outfile)s.bam.featureCounts.bam %(outfile)s"""
    P.run(statement, job_threads=2, job_memory="8G")

Again this function isn't actually doing the stated work, rather it is making a system call to featureCounts with various arguments supplied by the function (presumably as there are multiple files to be running this on).
1
Again a decorator (@ruffus.follows(ruffus.mkdir("read_assignments"))) is used to create the named directory.
2
A second @ruffus.transform() decorator is used to take the output from name_sort_bams (presumably a list of files returned by cgatcore.pipeline.run()????).
As with the previous step a regular expression is used to pick up the filenames from the previously created name_sorted_bams/(.+).sorted.bam and created a set of nested directories under the read_assignments/ directory which has been created. There is a directory for each file and within each the file stem is suffixed with .sorted.assigned.bam.
3
annotation = PARAMS["transcript_gtf"] is extracted from the PARAMS dictionary (loaded from wherever pipeline.yaml has been found. In this minimal example the value is files_for_regression_testing/test_wash1.gtf which I don't have a copy of (yet!)
4
outdir = os.path.dirname(outfile) is extracted from the filename.
5
The .assigned.bam which was appended to the filename in the @ruffus.transform() decorator is now removed to leave the rename_outfile

VCF_from_no4sU_control()
@ruffus.transform(
    "input_bams.dir/*no4sU*.bam",
    ruffus.regex("input_bams.dir/(.+)_no4sU_(.+).bam"),
    r"snp_vcf/\1.vcf.gz",
)
def VCF_from_no4sU_control(infile, outfile):
    """
    Takes the negative control `.bam` file (via the no4sU regex) and creates a SNP VCF file
    using Varscan. It looks for differences between the `.bam` file and the genome `.fasta`
    file to detect differences, where these are always present at the same position (above
    a treshold, see Varscan docs) it is classed as a SNP and included in output VCF file.
    """
    outfile_uncompressed = outfile.replace(".vcf.gz", ".vcf")
    genome_fasta = PARAMS["genome_fasta"]
    statement = """samtools mpileup -B -A -f %(genome_fasta)s %(infile)s |
                    varscan mpileup2snp --variants 1 --output-vcf 1 > %(outfile_uncompressed)s &&
                    bcftools view %(outfile_uncompressed)s -Oz -o %(outfile)s"""
    # removed sort -k1,1 -k2,2n %(outfile_uncompressed)s -o %(outfile_uncompressed)s &&
    # before bcftools view
    P.run(statement, job_threads=12, job_memory="1G")

index_VCF()
@ruffus.transform(VCF_from_no4sU_control, ruffus.suffix(".vcf.gz"), ".vcf.gz.tbi")
def index_VCF(infile, outfile):
    """
    Indexes the VCF file so it can be read with Pysam, generates the .vcf.gz.tbi file
    """
    statement = "tabix -p vcf %(infile)s"
    P.run(statement)

pipeline_slam_3UIs/all_introns_counts_and_info.py
This is the meat of the work!

pipeline_slam_3UIs/summarize_counts.R
A simple R script
library(dplyr)
library(stringr)
library(tidyr)
library(tibble)
library(data.table)

args = commandArgs(trailingOnly = TRUE)

input_folder =  args[1]

input_regex = args[2]

outfile = args[3]

files = list.files(input_folder, pattern=input_regex, full.names = TRUE)

data_list = list()

for(file in files){
  file_name = paste(str_split(basename(file), "_")[[1]][1:2], collapse="_")

  df = fread(file) %>%
    group_by(Transcript_id,
             Chr,
             Strand,
             Start,
             End,
             Assignment,
             Conversions>=1) %>%
    summarise(counts = n()) %>%
    mutate(File = file_name)

  data_list[[length(data_list) + 1]] = df

  rm(df)
}

combined_df = rbindlist(data_list)

rm(data_list)

fwrite(combined_df, outfile, sep="\t")

Comment
I suspect most of this can easily be translated to Python, Pandas has group_by functions to do the counting.


added amforc

https://polkadot.subscan.io/extrinsic/15993949-2
https://kusama.subscan.io/extrinsic/18383220-2

Hello @tugytur, thank you for request!
This is how it will looks like in the app:

Spoiler!

@stepanLav thanks for the review pictures. I noticed a spacing mistake and updated the config


Good first issue

Documentation help

Hi,
I am interested in this one

if you can give me more details, I would also like to work on this

I would like to help you with your documentation.


BUG: Fix yeojohnson when transformed data has zero variance

Reference issue

Closes https://github.com/scipy/scipy/issues/10821
What does this implement/fix?

This PR updates yeojohnson_llf to handle cases where the variance is zero in the transformed space. Note that yeojohnson_llf returns _XMAX when the variance is too low, because the formula is -N/2 * log(variance). _neg_llf will consider those to be uninteresting by replacing the _XMAX with -_XMAX.
Additional information
The np.errstate(over='ignore') is used because _neg_llf will return _XMAX which causes an overflow in optimize.brent.

@thomasjpfan Just to check my understanding:

_neg_llf will consider those to be uninteresting by replacing the _XMAX with -_XMAX.

Theoretically, high values of llf = yeojohnson_llf(lmbda, data) are interesting, but for some values of lmbda explored during optimization, the variance of the transformed data is too low to calculate llf accurately*. We detect this case by looking for (essentially) infinite LLF, and we reject these values of lmbda.
* - Specifically, the llf is artificially inflated to +inf, but if all calculations had been done with infinite precision, the true llf would likely be worse - more negative - than llf is for other values of lmbda. So the optimal value of lmbda we get is essentially the same as we would get even if we had infinite precision.
If that's right, this LGTM!

Theoretically, high values of llf = yeojohnson_llf(lmbda, data) are interesting, but for some values of lmbda explored during optimization, the variance of the transformed data is too low to calculate llf accurately*

I am not sure. For the input in the original issue, the lmbda found on main leads to a llf of inf:
from scipy import stats
import numpy as np
x = np.array([3251637.22, 620695.44, 11642969.00, 2223468.22,
              85307500.00, 16494389.89, 917215.88, 11642969.00,
              2145773.87, 4962000.00, 620695.44, 651234.50,
              1907876.71, 4053297.88, 3251637.22, 3259103.08,
              9547969.00, 20631286.23, 12807072.08, 2383819.84,
              90114500.00, 17209575.46, 12852969.00, 2414609.99,
              2170368.23])

# lmbda found on main
lmbda_main = -4.275407328878737
stats.yeojohnson_llf(lmbda_main, x)
# inf

# lmbda found in this PR
lmbda_pr = -0.1283482789353538
stats.yeojohnson_llf(lmbda_pr, x)
-392.115

If we had infinite precision would lmbda_main result in a llf smaller than the llf with lmbda_pr?

One remaining question - why use _XMIN and _XMIN instead of 0 and np.inf?

Hm I think we can use np.inf. As for 0, I think it would work, using var == 0, but I tend to avoid float equality.

And how did you know to find these in ._constants

A grep for how np.finfo is used in the library.

If we had infinite precision would lmbda_main result in a llf smaller than the llf with lmbda_pr?

That's what I was checking. The answer is yes, as expected.

and the maximum is right there around lmbda_pr.

# don't judge : ) just trying to get the job done
import numpy as np
from scipy import stats
import matplotlib.pyplot as plt
from mpmath import *
mp.dps = 100

x = np.array([3251637.22, 620695.44, 11642969.00, 2223468.22,
              85307500.00, 16494389.89, 917215.88, 11642969.00,
              2145773.87, 4962000.00, 620695.44, 651234.50,
              1907876.71, 4053297.88, 3251637.22, 3259103.08,
              9547969.00, 20631286.23, 12807072.08, 2383819.84,
              90114500.00, 17209575.46, 12852969.00, 2414609.99,
              2170368.23])

def mp_yj_llf(lmbda, x):
    lmbda = mpf(lmbda)
    data = np.array([mpf(xi) for xi in x])
    trans = (np.power(data + 1, lmbda) - 1) / lmbda
    lmb = lmbda
    n_samples = len(x)
    loglike = -n_samples / 2 * log(np.var(trans))
    dp1 = abs(data) + 1
    loglike += (lmb - 1) * np.sum(np.sign(data) *
                                  np.array([log(xi) for xi in data]))
    return loglike

lmbdas = np.linspace(-5, 1, 1000)
np_llf = np.array([stats.yeojohnson_llf(lmbda, x) for lmbda in lmbdas])
mp_llf = np.array([mp_yj_llf(lmbda, x) for lmbda in lmbdas], dtype=float)

i = np_llf < 1e300
plt.plot(lmbdas[i], np_llf[i], '-', lmbdas, mp_llf, '--')
plt.ylim(-600, -300)
plt.legend(('NumPy/SciPy', 'mpmath/custom'))
plt.xlabel('lmbda')
plt.ylabel('YJ LLF')

Thanks @thomasjpfan!


Keep GitHub Actions up to date with GitHub's Dependabot

Fixes warning at the bottom right of
https://github.com/rossjrw/pr-preview-action/actions/runs/8246680558

https://docs.github.com/en/code-security/dependabot/working-with-dependabot/keeping-your-actions-up-to-date-with-dependabot
https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file#package-ecosystem

@rossjrw Your review, please.


Cannot reproduce test result

code updated to the latest. When tested on the image "motorcycle.png", no vision was shown. no errors or warnings were present except resource limitation.
https://github.com/kylesargent/ZeroNVS/assets/152853346/64eb4ba3-2941-4fc3-a631-f88451b6f621

I just re-ran the code and it seems to be working fine for me.
It seems you are rendering the model at iteration zero? If the training crashed due to resource limitations and rendered a video at step zero, obviously a NeRF which has not been trained at all would not reproduce the test results.

I ran the prediction with launch_inference.sh in test mode and  the model was  released by your pre-trained model here. Still can't get the test result

I have encountered the same problem. Have you solved it?

@maoqingsunny  Unfortunately I cannot assist you without significantly more details regarding the failure to reproduce the result.
Regarding my response earlier:
It seems you are rendering the model at iteration zero? If the training crashed due to resource limitations and rendered a video at step zero, obviously a NeRF which has not been trained at all would not reproduce the test results.
Does this apply to you?


add the entries attribute in xml config and fix the bug

add the entries attribute in xml config and fix the bug that attachDataSource() with null Object

Thank you for contributing. :)
I've implemented your proposed changes in this commit c2e4330a214fbe06e2d55ddab97cd7b53331453c.


Resolve ANN102

What does this PR do?
Resolve ANN102: missing-type-cls

Before submitting

[ ] Was this discussed/agreed via a Github issue? (no need for typos and docs improvements)
[ ] Did you read the contributor guideline, Pull Request section?
[ ] Did you make sure to update the docs?
[ ] Did you write any new necessary tests?

PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

Did you have fun?
Make sure you had fun coding 🙃

:books: Documentation preview :books:: https://torchmetrics--2149.org.readthedocs.build/en/2149/

I think I accidentally fixed the wrong linter error. I can close this PR if this isn't a style we want to enforce.

imo that would be fine but I think indexing the builtin type is not supported for every python version so we might want to consider something from the typing package instead

imo that would be fine but I think indexing the builtin type is not supported for every python version so we might want to consider something from the typing package instead

Yes, typing.Type should work here. type built-in for annotations was added in 3.9/3.10 I think

Unit tests for old failed RuntimeError: CUDA error: no kernel image is available for execution on the...

Unit tests for old failed RuntimeError: CUDA error: no kernel image is available for execution on the...

seems we have some flakes in the system... :/

@stancld do you still have a request for changes? :rabbit:


Constrained JSON sample needs ballerina.doc package import

The sample below [1] need the following import statements.
import ballerina.doc;
[1] https://ballerinalang.org/docs/by-example/constrained-json
Tried in version 0.93

Fixed with #3649


VM Service details page has wrong information tabs

Rancher Version: master 9/2
Steps to Reproduce:

Add a environment with VM checked and a packet host
Go to Stacks page and add a VM service
Click on the VM service to get to details page

Results: Has tabs for POrts, Containers, Labels and Links

Expected: Should have tabs for Disks, User Data, Health Check, Labels and Scheduling

Opps... I'm wrong


community-operators [R] keda (1.4.1 1.5.0 2.0.0 2.1.0 2.2.0 2.3.0)

Thanks submitting your Operator. Please check below list before you create your Pull Request.
Updates to existing Operators

[x] Did you create a ci.yaml file according to the update instructions?
[x] Is your new CSV pointing to the previous version with the replaces property if you chose replaces-mode via the updateGraph property in ci.yaml?
[x] Is your new CSV referenced in the appropriate channel defined in the package.yaml or annotations.yaml ?
[x] Have you tested an update to your Operator when deployed via OLM?
[x] Is your submission signed?

Your submission should not

[x] Modify more than one operator
[x] Modify an Operator you don't own
[x] Rename an operator - please remove and add with a different name instead
[x] Submit operators to both upstream-community-operators and community-operators at once
[x] Modify any files outside the above mentioned folders
[x] Contain more than one commit. Please squash your commits.

Operator Description must contain (in order)

[x] Description about the managed Application and where to find more information
[x] Features and capabilities of your Operator and how to use it
[x] Any manual steps about potential pre-requisites for using your Operator

Operator Metadata should contain

[x] Human readable name and 1-liner description about your Operator
[x] Valid category name1
[x] One of the pre-defined capability levels2
[x] Links to the maintainer, source code and documentation
[x] Example templates for all Custom Resource Definitions intended to be used
[x] A quadratic logo

Remember that you can preview your CSV here.
--
1 If you feel your Operator does not fit any of the pre-defined categories, file an issue against this repo and explain your need
2 For more information see here

@J0zi @mvalarh I have updated the older operator versions to not include a path to the test scorecard (it is not there). I hope it is ok.

/merge possible

@mvalarh this hasn't been automatically merged as well, even though I have updated the ci.yaml.

/merge possible


Conduit subnet release on deletion

Description
The conduit watcher in the IPAM is now deleting the conduit subnet when a conduit disappear from the conduit list sent by the NSP service.
Issue link
Without this fix https://github.com/Nordix/Meridio/pull/349, new problems could appear when deleting and recreating a conduit.
Checklist

Purpose

[x] Bug fix
[ ] New functionality
[ ] Documentation
[ ] Refactoring
[ ] CI

Test

[x] Unit test
[ ] E2E Test
[x] Tested manually

Introduce a breaking change

[ ] Yes (description required)
[x] No

Introduce changes in the Operator

[ ] Yes (description required)
[x] No

Looks good.


[Feature Request] Android Wear support

I would really love to be able to get OTP pin codes on my watch.
The way I am imagining it is this: scroll through the list of configured tokens on your watch until you get to the one you need, tap it, and you get the pin. the whole management of configured tokens would still be on the watch.
Can this be done?

Well, it can be done but the effort is a lot. And Wear OS 3.0 was released and there are quite some changes also.

IT actually works on Android Wear but UI is kind of clunky to use (Especially on a round display), and importing existing tokens is a pain.
So UI redesign for Wear would even make the current app pretty usable.

Yea, if you install the APK on wear os, it may just work. But it requires the app to sue the Wear OS API to be watch friendly. The most difficulty part is how to synchronize the Wear OS and phone app.

What I would like to see on my watch is just the list of all existing tokens, then scroll through it, tap one, and get the OTP pin.
Everything else (import/export/add new tokens/etc) should just stay on the phone or tablet...


Correction of a few bugs

The first couple of corrections were made for the code to match the problem description in the OSeMOSYS documentation. The last four corrections are pyomo syntax related issues.

Hi @ThomasHeggarty - many thanks for these contributions.  Could you clarify what the issues are that your pull request addresses?  Best practice is to create an issue (as you have done) with are detailed explanation of the problem and the effects of it. You can then link to it from your pull request by using the hash symbol e.g. #1

Hi @willu47 - As requested, I have added a description of each issue in #5

Perfect, super clear. Thanks so much! We’ll review and merge this in as soon as possible.


[#2108] do not download the same file again

Issue:
Download of a new signature file takes >15minutes so Exchange isn't able to download the files in our network. -> timeout
When Update-Engines.ps1 is run by a scheduled task every 15min it will re-download the current file and overwrite the existing. While the download is in progress the file cannot be used by Exchange and signature Update will fail.
Fix:
do not download the file again when it already exists and has the correct size
Resolved #2108

@microsoft-github-policy-service agree company="AOK-Bundesverband eGbR"

Pending test team to confirm we can merge this one.


Test command to iterate through all diffs in a review

Refs: https://github.com/microsoft/vscode-pull-request-github/issues/4089

[ ] anyOS @hediet
[ ] anyOS @cwebster-99

Complexity: 2
Create Issue

Install the pre-release version of GitHub Pull Requests and Issues
Open a folder that contains a GitHub repository
Check out a pull request (this feature only works on checked out PRs)
Use the command "Go to Next Diff in Pull Request" to view the changes in the PR. Verify that it works in a way that you'd expect.

@cwebster-99 can you share a link to the PR you're testing with? I've only been able to repro this occationally.

Figured out the problem, no further info needed.

Works now!


ChartsDemo-swift.h  file not found

I have faced problem which describe by ( ChartsDemo-swift.h file not found )  i want to focused that everything is working correctly only this file isn't imported .

actually it should be  "yourprojectname-swift.h" if you are using in your project.
also in build phase check "Embedded content contains swift code"s should be yes

Thanks @vishw3

sorry for late but still have the same error

@danielgindi @ahmadabudawod Please see http://stackoverflow.com/a/30648104/277370. When you use embedded framework, I had to do like the answer on SO. Using the #import <project>-Swift.h works well when you directly add the library files (Classes/) to your project (instead of just adding the .xcodeproj and add it to embed frameworks).
I think when you only add the xcodeproj, You must import it with #import "Charts/Charts-Swift.h"

@allaire  thumbs up. your solution helped me. I am not sure why, but the previous comments of importing #import -Swift.h did not solve the error.

@allaire Thumbs up again. Hey @danielgindi I think the same can be updated for newbies. I was stuck with the same, being newbie to objc-swift bridging. Thanks again @allaire , you saved the day.

@allaire
Xcode 8.1,iOS 10.1, I have this issue.
Thank you for your solution.

put it in .m file and leave a forward declaration in .h


Promise throws typechecking error on empty resolution value, requiring explicit undefined

Bug Report
🔎 Search Terms
Promise resolve optional parameter undefined type void allow error
🕗 Version & Regression Information

This changed between versions 4.0.5 and 4.1.5 and is still observed in the latest nightly available on the playground.
It is a breaking change because code that was perfectly valid before the change, and should have been perfectly valid after, was no longer allowed, but there was no corresponding change to the semver-breaking version number.

⏯ Playground Link
Playground link with relevant code
💻 Code

export class MyClass {
    typescriptExamplePromiseFail(jsonString: string) : Promise<any> {
        //Note: Using Promise<any | void> or Promise<void | any> as return type above doesn't help
        //Documentation at https://www.typescriptlang.org/docs/handbook/2/everyday-types.html#any
        //says 'any' should be used "whenever you don’t want a particular value to cause typechecking errors"
        //and that "'any' isn't type-checked."
        //but use of 'any' here doesn't prevent the typechecking error.
        return new Promise(function(resolve, reject) {
            try {
                const result = JSON.parse(jsonString); //might throw an error if the string doesn't have valid JSON
                resolve(result);
            } catch(err) {
                resolve(); //Expected 1 arguments, but got 0. Did you forget to include 'void' in your type argument to 'Promise'?
            }
        });
    }
    //It's OK to omit the argument for a synchronous method,
    //but if the method first fetches the JSON from an async source that's not really an option
    typescriptExampleSync(jsonString: string) : any {
        try {
            const result = JSON.parse(jsonString); //might throw an error if the string doesn't have valid JSON
            return result;
        } catch(err) {
            return;
        }
    }
    //Omitting the argument even works with the async keyword, but that's not always feasible & shouldn't be required
    async typescriptExampleAyncPass(jsonString: string) : Promise<any> {
        try {
            const result = JSON.parse(jsonString); //might throw an error if the string doesn't have valid JSON
            return result;
        } catch(err) {
            return;
        }
    }
    typescriptExamplePromisePass2(jsonString: string) : Promise<any> {
        return new Promise(function(resolve, reject) {
            try {
                const result = JSON.parse(jsonString); //might throw an error if the string doesn't have valid JSON
                resolve(result);
            } catch(err) {
                resolve(undefined); //must be explicit to avoid type error; not great w/ https://eslint.org/docs/rules/no-undefined
            }
        });
    }
}

🙁 Actual behavior
Though the return type of typescriptExamplePromiseFail is Promise<any> and the any resolution type is supposed to prevent typechecking failures, there is still a typechecking failure caused by the resolve() line in one possible execution path.  An explicit "undefined" is required by Typescript but prohibited by eslint for what appears to be good reason.
🙂 Expected behavior
No typechecking errors on a resolution value declared to be of any type and no requirement for an explicit undefined.
The typechecking error should occur if the resolution type is something narrower than any where an explicit undefined would also rightly fail.
Related
https://github.com/Microsoft/TypeScript/issues/8516
https://github.com/microsoft/TypeScript/issues/3356
https://github.com/microsoft/TypeScript/pull/22772 (“reverted for now”)
There are several issues around how resolve() was erroneously allowed for Promise<number> or other resolve types, but it should be allowed for Promise<any> according to documentation about any stopping typechecking errors.

ESLint disallows undefined because it can be shadowed by a local variable, but it's pretty unlikely that anyone would actually do that.  It was possible to completely overwrite it in ES3, but I doubt that's relevant unless you're targeting, say, IE 6.  In any case the rule appears to only disallow using the identifier undefined; you could write void 0 instead.

The point is that nothing should be required and using resolve() should not trigger a typechecking error with any as the resolution type. If that is not intended, the documentation about any should be updated to reflect that it doesn't stop typechecking errors as described.

It's not actually a type-checking error though - it's a "not enough parameters" error, the same reason this doesn't work:
declare function foo(x: any): void;
foo();  // expected 1 arguments, but got 0

Everything you've described here is the intended behavior.

Why does the first example fail typechecking while the other three pass?
The return/resolve value in the other cases is any and it's OK to omit the value in those cases, without an explicit undefined.  Why is it just required in the first case, if any should include nothing?
I suspect a Promise definition overload is needed.


image: add fallback className

Tested both cases with loadFallbackFirst set to true and false.

closes https://github.com/inveniosoftware/invenio-app-rdm/issues/1632
needed by https://github.com/inveniosoftware/invenio-app-rdm/pull/1677

As the classes change dinamically after loading the logo correctly there is a resize that can be appreciated if the connection is really slow, while assigning the new image in the src. It can be appreciated in the video in the second 2
https://user-images.githubusercontent.com/15194802/171155348-efb4cdaf-3b69-4361-879c-2a59841c6237.mp4

:+1:


Add copyrights

Signed-off-by: Jon Hawkes<EMAIL_ADDRESS>Next small step towards getting this closer to what's needed.

Fixed for service-a by https://github.com/eclipse/microprofile-service-mesh-service-a/issues/3

Fixes https://github.com/eclipse/microprofile-service-mesh-service-a/issues/3 for service-a


refactor(app): annotate types of legacy RPC API actions/selectors

overview
This PR is a follow up to #5785 that adds type annotations to a bunch of old actions + state in the app that deal with the RPC API. There will be another PR to make sure all other exports in the app are annotated, but this work specifically was relatively large in scope and required some source changes, so I split the PR out.
changelog

refactor(app): fully annotate types of legacy RPC API actions

Including the removal of 26 $FlowFixMe comments

Fix return tip failures during RPC calibration not making it into state

Bug was hidden by lack of unit test and type coverage

review requests

[ ] Code looks good
[ ] If possible, smoke test a protocol upload + calibrate + run flow

risk assessment
Low. The vast majority of this PR is type annotation changes. That being said, there are action creator and reducer changes in the actual source. Thankfully, unit test coverage of this code is relatively OK, and this PR adds a unit test for this only intentional logic change it makes

Do we see these new actions having a place in a post RPC world?

To be clear, these are not new actions. These are existing actions that the typechecker currently ignores. This PR allows the typechecker to see these actions; it does not change how they are being used.
They only exist to interact with the RPC API and will be removed along with RPC


Pass cost_attr and cost_budget from flaml.tune.run() to the search algo

Why are these changes needed?
This enables the user to specify in flaml.tune.run
Related issue number
Resolves https://github.com/microsoft/FLAML/issues/1166
Checks

[ ] I've used pre-commit to lint the changes in this PR (note the same in integrated in our CI checks).
[ ] I've included any doc changes needed for https://microsoft.github.io/FLAML/. See https://microsoft.github.io/FLAML/docs/Contribute#documentation to build and test documentation locally.
[ ] I've added tests (if relevant) corresponding to the changes introduced in this PR.
[ ] I've made sure all auto checks have passed.

The test fail seems to have nothing to do with the PR


Can you please re-run CI?

Comment:
Looks like the initial build failed because xgboost_ray wasn't in the index yet - can you re-run CI?

done


Update tools used in pre-commit testing.

This updates us to the latest versions of Black and Ruff, which results in some minor non-behavioral changes to the code.

Codecov Report
All modified lines are covered by tests :white_check_mark:
:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.

Files
Coverage Δ

apis/python/src/tiledbsoma/_collection.py
93.53% <100.00%> (ø)

apis/python/src/tiledbsoma/_experiment.py
100.00% <100.00%> (ø)

apis/python/src/tiledbsoma/_measurement.py
100.00% <100.00%> (ø)

...thon/src/tiledbsoma/io/_registration/signatures.py
94.30% <ø> (ø)

apis/python/src/tiledbsoma/io/ingest.py
87.43% <ø> (ø)

... and 76 files with indirect coverage changes
:loudspeaker: Thoughts on this report? Let us know!.


Updated downloadURL and versionNameTemplate to match new nwjs.io url

Fixed by https://github.com/mllrsohn/node-webkit-builder/commit/b86431a31fe97ba7b85e52308dbe399701aa6002 and https://github.com/mllrsohn/node-webkit-builder/commit/74e54720fa4b146de9ac245d68351d922c0e9de6 which is now live as version 1.0.6.
Thanks @CoreyClarkPhD


Model Architecture

It is really a great work!
From the title of the paper, I understand that you designed the architecture of the model and trained it to target single person pose estimation; however,  You still use a PAF detection pass as OpenPose which is really important in case of multi person pose estimation. So what will happen if we removed the PAF detection pass ?   Is this pass really important for better accuracy?

@AhmedEssamDakrory: Thank you a lot for the comment and the interesting question! In Appendix A of the published paper under "Skeleton estimation" in Table 7 you may observe effect of removing the detection pass performing skeleton estimation (i.e., estimation of PAFs) with EfficientPose I and EfficientPose II. By removing this pass, there is a significant loss in PCKh@50.


Adobe Animate Nested MovieClip "synched" mode issue

Hey folks, the Adobe Animate Export to Canvas has changed slightly over the last version.  It seems that MovieClips are wrapped in MovieClips and when you run these from Animate it all seems to work - but if you instantiate the outer clip with code, going to the labels are not working in the default "independent" mode.  If that is changed to "synch" in the exported code then it works.  If you need the full files I can send them. (Not my code - just passing the issue along).  Thanks!
(lib.BoyMC = function(mode,startPosition,loop) {
if (loop == null) { loop = false; }	this.initialize(mode,startPosition,loop,{sad:0,happy:29});

	// timeline functions:
	this.frame_0 = function() {
		this.stop();
	}
	this.frame_29 = function() {
		this.stop();
	}
	this.frame_59 = function() {
		this.___loopingOver___ = true;
	}

	// actions tween:
	this.timeline.addTween(cjs.Tween.get(this).call(this.frame_0).wait(29).call(this.frame_29).wait(30).call(this.frame_59).wait(1));

	// Layer_1_obj_

	/////this is original code that has been replaced////
	////////////////////////////////////////////////
	//this.Layer_1 = new lib.Boy_Layer_1();
	////////////////////////////////////////////////

	this.Layer_1 = new lib.Boy_Layer_1("synched"); // it all works now

	this.Layer_1.name = "Layer_1";
	this.Layer_1.parent = this;
	this.Layer_1.setTransform(200,199.5,1,1,0,0,0,200,199.5);
	this.Layer_1.depth = 0;
	this.Layer_1.isAttachedToCamera = 0
	this.Layer_1.isAttachedToMask = 0
	this.Layer_1.layerDepth = 0
	this.Layer_1.layerIndex = 0
	this.Layer_1.maskLayerName = 0

	this.timeline.addTween(cjs.Tween.get(this.Layer_1).wait(60));

}).prototype = p = new cjs.MovieClip();
p.nominalBounds = new cjs.Rectangle(0,0,400,400);

Then in our script we might have:
var boy = new lib.BoyMC();
stage.addChild(boy);
boy.gotoAndPlay("happy"); // does not work unless we make the change

Bump...  another comment... shammi  [3 minutes ago]
@Dan Zen hi i discovered another thing, there are child movieclips with stop action on the first frame the zim objects will not stop at the first frame for child movieclip.

The stops are affected as well.  Any thoughts?

Hey folks - any thoughts on this - thanks!


Provide a CONTRIBUTING.md

@awaltert  do we have a template for this?

@ISO50 I think we can adapt the guidelines provided by github.
What do you think?

See #66


⚠ Bofrost.be has degraded performance

In cbaf7be, Bofrost.be (https://www.bofrost.be) experienced degraded performance:

HTTP code: 200
Response time: 1056 ms

Resolved: Bofrost.be performance has improved in 948ffec after 9 minutes.


Rogue CSS class ".sr-only" breaks LESS compatability?

In File:
https://github.com/Eonasdan/bootstrap-datetimepicker/blob/master/src/less/bootstrap-datetimepicker-build.less
// Import bootstrap variables including default color palette and fonts
@import "../../node_modules/bootstrap/less/variables.less";

// Import datepicker component
@import "_bootstrap-datetimepicker.less";

.sr-only {
  position: absolute;
  width: 1px;
  height: 1px;
  margin: -1px;
  padding: 0;
  overflow: hidden;
  clip: rect(0,0,0,0);
  border: 0;
}

How important is the .sr-only? I need to include this less file without including the default bootstrap variables. Shouldn't the sr-only be inside of the file @import "_bootstrap-datetimepicker.less"; or have it's own file?

sr-only is a bootstrap mixin/class. It was added our build file to make some LESS compilers work if it couldn't find the mixin from bootstrap.

Oh OK, I guess that should be commented in? I didn't think to check the blame button, maybe it's my fault lol.

see also: #567

I added a comment to the less file


Fix >=0.12 interpolation

TF > 0.11 throws the following warning:
Warning: Interpolation-only expressions are deprecated

  on .terraform/modules/remote_state/bucket.tf line 143, in data "aws_iam_policy_document" "state_force_ssl":
 143:       "${aws_s3_bucket.state.arn}",

This change fixes that

@mattwillsher Thanks, LGTM!


Write a TypeScript program to find area of a rectangle

Description
Write a TypeScript program to find area of a rectangle
Area of rectangle = Length × Width
Length : 2
Width  : 3

Area : 6

How to contribute

Save the solution in program/FindAreaOfARectangle.ts file
Add FindAreaOfARectangle.ts file in find-area-of-a-rectangle folder

!assign


Missing dependence ' react-addons-update'

It seems that the package ' react-addons-update' is missiong from your package.json.
ERROR in ./~/richie/dist/Richie.js
Module not found: Error: Cannot resolve module 'react-addons-update' in /path/to/my/roject/node_modules/richie/dist
 @ ./~/richie/dist/Richie.js 1:224-254

Thanks for your package :)

Thanks for your report dragma.
Should be fixed. :)


bug(IconToggle): Element is not properly destroyed/unmounted

Bugs
Element is not properly destroyed/unmounted
What RMWC Version are you using?

1.5.4

What browser(s) and React Version is this bug affecting?

Chrome

What build system are you using?

Create React App

What are the steps to reproduce the bug?

Click on an icon toggle and immediatly remove the element from the DOM. In order to reproduce it just click on the recycle bin of any item in this codesandbox.

What is the expected behavior?

Properly destroy/unmount the element.

What is the actual behavior?

An exception is thrown when trying to access an unexisting element.

Any other information you believe would be useful?

Thank you so much for the repro, definitely helps. Will look into it.

@ricardobrandao that was a great bug and great find. Patched and deployed 1.5.5!

Hi James,
That was quick! 😄 Thank you so much for kindly solving this so fast.  I'll give it a try as soon as it hits npm.

I'm just confirming that this was fixed on 1.5.5 😃

Awesome, thanks for the validation @ricardobrandao


Make LineEdit secret character easier to change and enter

Fixes #81681.

Allow values longer than 1 character in the property, but trim characters after the first one.
Allow empty strings, this acts like if a space was used as a secret character, so that an error isn't printed when you erase all characters in the property.

About the implementation, we can add another field for displaying the char and so secret_character can only store the result of the above rule or vice versa. But that requires more change, so I decided to change the drawing part, calculate the right secret_character based on the rule above.

If extra characters are ignored, maybe we should have a configuration warning saying that.

By "configuration warning" I mean get_configuration_warning(). With your current implementation the warning will be spammed in the output.

You need to call update_configuration_warnings() in set_secret_character().

Thanks!


Review factor values to align with planned deployment

Factor/parameter values for modelled interventions are now out of date and do not align with expected deployment values.
These need to be reviewed and updated.

Note from Ken: Department has come up with requirements for deployment
All about enhanced corals, no consideration of SRM/MCB.


Increment the version number to re-enable publishing after

publish workflow fail.
TBR=arv@

LGTM


Adding control flow to avoid memcpy

Hi Soumith - sorry to add to your already busy schedule. Could you check if the following change addresses part of the speed issue? I should have run it on my machine but I have a K40, so the numbers may not be comparable.
The background story: for TensorFlow's session.run(target), it also automatically fetches the contents of the target from the TensorFlow backend into Python. If the targets are on GPU, an implicit MemcpyDtoH is called, and this may cause unnecessary delays. In practice, since the SGD will always be on the GPU, the fetching is not needed.
We realized that this side effect slipped off our notice because convolutional layers have very few parameters and the MemcpyDtoH latency got overlooked.
Again, thanks a lot for maintaining the benchmarking results!

Thanks. trying it now.

Your fixes improved results a little bit, but for the VGG_A to not go OOM, I had to show 2 Titan-X cards via CUDA_VISIBLE_DEVIES, it was going OOM if I showed only 1 Titan-X card. However, it was not using two cards for compute (I checked via nvidia-smi ). Updating the main post now. The raw logs are checked in. Thanks Yangqing


Adding status to MiddenCli to track projects

For the new MiddenCli, adding status to track projects will be super useful. Then in the insights researchers can actually see their performance.

I think this is a great idea. I'm a little weary about schema blot since we're "limping" along with markdown + front-matter instead of a JSON file when dealing with projects. However, this one seems pretty important. Being able to sort projects by status in the Project Catalog would be extremely useful too.


Translate concepts/configuration/manage-resources-containers.md in Korean

This is a Feature Request

What would you like to be added

Translate concepts/configuration/manage-resources-containers.md in Korean.
Why is this needed
There is no documents of concepts/configuration/manage-resources-containers.md which translated into Korean.
Comments

Page to update: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/
I'm translating this document now.
I'll submit this to dev-1.18-ko.3 branch after done for translating.

/language ko
/assign


URLs for gas fees estimates in Polygon changed

Hey! This morning the urls used to estimate gas fees in Polygon started to fail, according to the docs the urls changed and they dropped the old one:

old: https://gasstation-mainnet.matic.network/v2
new: https://gasstation.polygon.technology/v2

Thanks for the report @ramirotw 🙌🏻
Updates the urls via https://github.com/enzoferey/network-gas-price/pull/26. Just published version https://github.com/enzoferey/network-gas-price/releases/tag/v0.3.5 to include it 👍🏻

@ramirotw @smoove There was an breaking change (or bug dunno) on the Polygon station side and they sometimes return string values instead of numbers as stated in their docs. I have made the required changes to cover this anomaly via https://github.com/enzoferey/network-gas-price/releases/tag/v0.3.6.


Change warning message for move nft to own account

When move nft to own account, our app pop up warning message "feature come soon". We need update this message.

Move own account from move account list

This bug already fixed in IOS prod version 2.2.9(8)


Documentation: update references to external blogs on Hamilton

Is your feature request related to a problem? Please describe.
We need to update https://hamilton.dagworks.io/en/latest/get-started/learning-resources/#external-blogs.
Describe the solution you'd like
We need to provide updates for blogs on Hamilton.
Some that come to mind:

https://blog.dagworks.io/ <-- there a bunch here
https://pyright.blogspot.com/2024/07/dag-hamilton-workflow-for-toy-text.html
https://vyom-modi.medium.com/building-a-high-performance-rag-based-ai-with-qdrant-groq-langchain-and-dagworks-hamilton-fb1baa7415bc
https://blog.getwren.ai/how-do-we-rewrite-wren-ai-llm-service-to-support-1500-concurrent-users-online-9ba5c121afc3

Describe alternatives you've considered
N/A
Additional context
It would be great to semi-automate this. E.g. make updating this simpler to do, by running a command for example.

I'd like to take this up!

I'd like to take this up!

Great thank you. Assigned. Do you have an understanding of where and what to change?

If I've understood the issue correctly, we want to have the latest blogs in the external blogs section. Is that correct? If that's the case, then I was thinking if I could make use of a substack API(if it exists)/scrape https://substack.com/@dagworks/posts for the posts and then update the posts in external blog sections in the markdown file.

If I've understood the issue correctly, we want to have the latest blogs in the external blogs section. Is that correct? If that's the case, then I was thinking if I could make use of a substack API(if it exists)/scrape https://substack.com/@dagworks/posts for the posts and then update the posts in external blog sections in the markdown file.

yep pretty much. You'd need to filter to posts that are about hamilton. Otherwise there's a few other posts that are linked in the issue that should also be added.

@sikehish yes that's fine.
I think using a date cut off is fine. Since that's how we'd update/run it. Pull posts older than X and add to the docs please :)

Hi @skrawcz . I was able to fetch the relevant data fro blog.substack.io using headless web scraping. I'll start making the commits after the other PR #1194 gets through, to ensure that the commit history remains true to what the PR is.

HI @skrawcz . I've raised a PR #1208. Let me know if any changes need to be made.


Which version of numpy and netcdf4 are being used?

Netcdf4 version 1.7.1 has an issue on mac arm processors that I opened here.
Downgrading to netcdf4 version 1.6.5 resolved this. This also required me to downgrade numpy version 1.26.4.
What versions of numpy and netcdf4 were used in the development of this repo?
Posting this for visibility if other users run into this issue.

This has recently become as issue not only for mac OS, but linux as well.
On our linux version, we're running with netcdf4==1.7.0 and numpy==1.26.4.
We are currently preparing a big update to this repo to make the code compatible with keras 3.0.  This will have updated requirement files that reflect this.

In my case, this issue was caused by installing netcdf4 with PyPI. Installing netcdf4 with conda fixed this for me.
conda create --name tornado python=3.10 netcdf4=1.7.1


Implement raise () in libc

It is used for defining an asserting stub for the function raise
only, which is not used anywhere.
JerryScript-DCO-1.0-Signed-off-by: Akos Kiss<EMAIL_ADDRESS>

Because of issues that occured on ARM, had to shift the original focus of the PR:
The core functionality (i.e., the equivalent of kill (getpid (), sig);)
was already there in the implementation of abort (). Now, it got
factored out to raise () so that others (most importantly,
__aeabi_ldiv0) can call and link to it as well.
Also, removed LIBC_UNREACHABLE_STUB_FOR macro, as it is not used anymore.

LGTM

LGTM


fix project references, upgrade @types/graphql

this consolidates & simplifies the tsconfig project references tree and build steps, so that we have a proper cross-repository incremental build!
now if you run yarn tsc --watch, you'll get an incremental build of the entire repository, where only the changes you make incur a new build.
this will tide us over until we figure out how to use esbuild/vite together with references happily, or replace project references with another incremental build tool

[x] consolidate project references
[x] update @types/react, resolve with tsc
[ ] fix webpack issue with @types/react
[ ] add build:watch for yarn tsc --watch
[ ] add concurrently and yarn build:watch to start-graphiql

@thomasheyenbrock hey no worries! I wish I had more time outside of work to help review all of this! I might just apply for a graphql grant so I can keep up haha. These kinds of oversights are the kind of adaptations you need to make when the tooling isn't well documented or organized, like this repo!
I think we can leave it there for now, but move the vite build to build-bundles step, I just need to clean up this one webpack failure.
Also, what is the command/are the comands you're using for graphiql development?
Currently start-graphiql is supposed to be that, but I forgot that I tend to run yarn tsc --watch alongside it (all of this is half-documented :( which I should change ). Perhaps I can just add concurrently to start-graphiql?

Bahhh... so many errors when I try to rectify this... I apologise, this build tooling is such a house of cards.


[BUG] Throughput regression on perf run after 4/17.

Describe the bug
There has been a throughput regression using http_logs dataset after 4/17.

This regression was likely introduced with one of these commits.

This is likely the introduction of enabling segment replication backpressure by default with this commit.
After executing a local run of SO dataset I see a similar regression.
ec2-user@ip-172-31-19-240 ~]$ opensearch-benchmark compare --baseline 66647b6c-8cc8-4684-9a20-27c62874e8ba --contender 80dbabdc-c7c1-4f7a-b1e0-698e8a81a3ad

   ____                  _____                      __       ____                  __                         __
  / __ \____  ___  ____ / ___/___  ____ ___________/ /_     / __ )___  ____  _____/ /_  ____ ___  ____ ______/ /__
 / / / / __ \/ _ \/ __ \\__ \/ _ \/ __ `/ ___/ ___/ __ \   / __  / _ \/ __ \/ ___/ __ \/ __ `__ \/ __ `/ ___/ //_/
/ /_/ / /_/ /  __/ / / /__/ /  __/ /_/ / /  / /__/ / / /  / /_/ /  __/ / / / /__/ / / / / / / / / /_/ / /  / ,<
\____/ .___/\___/_/ /_/____/\___/\__,_/_/   \___/_/ /_/  /_____/\___/_/ /_/\___/_/ /_/_/ /_/ /_/\__,_/_/  /_/|_|
    /_/

Comparing baseline
  TestExecution ID: 66647b6c-8cc8-4684-9a20-27c62874e8ba
  TestExecution timestamp: 2023-04-24 01:18:48
  TestProcedure: append-no-conflicts
  ProvisionConfigInstance: external

with contender
  TestExecution ID: 80dbabdc-c7c1-4f7a-b1e0-698e8a81a3ad
  TestExecution timestamp: 2023-04-24 00:51:02
  TestProcedure: append-no-conflicts
  ProvisionConfigInstance: external

------------------------------------------------------
    _______             __   _____
   / ____(_)___  ____ _/ /  / ___/_________  ________
  / /_  / / __ \/ __ `/ /   \__ \/ ___/ __ \/ ___/ _ \
 / __/ / / / / / /_/ / /   ___/ / /__/ /_/ / /  /  __/
/_/   /_/_/ /_/\__,_/_/   /____/\___/\____/_/   \___/
------------------------------------------------------

|                                                        Metric |                     Task |    Baseline |   Contender |     Diff |   Unit |
|--------------------------------------------------------------:|-------------------------:|------------:|------------:|---------:|-------:|
|                    Cumulative indexing time of primary shards |                          |     143.836 |     142.999 | -0.83765 |    min |
|             Min cumulative indexing time across primary shard |                          |     2.32835 |     2.27668 | -0.05167 |    min |
|          Median cumulative indexing time across primary shard |                          |     2.40673 |     2.38605 | -0.02068 |    min |
|             Max cumulative indexing time across primary shard |                          |     2.45847 |      2.5085 |  0.05003 |    min |
|           Cumulative indexing throttle time of primary shards |                          |           0 |           0 |        0 |    min |
|    Min cumulative indexing throttle time across primary shard |                          |           0 |           0 |        0 |    min |
| Median cumulative indexing throttle time across primary shard |                          |           0 |           0 |        0 |    min |
|    Max cumulative indexing throttle time across primary shard |                          |           0 |           0 |        0 |    min |
|                       Cumulative merge time of primary shards |                          |     77.8617 |     75.8006 | -2.06112 |    min |
|                      Cumulative merge count of primary shards |                          |        1327 |        1325 |       -2 |        |
|                Min cumulative merge time across primary shard |                          |     1.08268 |     1.04545 | -0.03723 |    min |
|             Median cumulative merge time across primary shard |                          |     1.28234 |     1.22928 | -0.05307 |    min |
|                Max cumulative merge time across primary shard |                          |     1.64947 |      1.6319 | -0.01757 |    min |
|              Cumulative merge throttle time of primary shards |                          |     10.5335 |      9.5461 | -0.98743 |    min |
|       Min cumulative merge throttle time across primary shard |                          |    0.132583 |    0.125133 | -0.00745 |    min |
|    Median cumulative merge throttle time across primary shard |                          |     0.16245 |    0.153875 | -0.00857 |    min |
|       Max cumulative merge throttle time across primary shard |                          |      0.3117 |      0.3064 |  -0.0053 |    min |
|                     Cumulative refresh time of primary shards |                          |     57.5864 |     57.8795 |   0.2931 |    min |
|                    Cumulative refresh count of primary shards |                          |       11683 |       11651 |      -32 |        |
|              Min cumulative refresh time across primary shard |                          |      0.8724 |    0.852633 | -0.01977 |    min |
|           Median cumulative refresh time across primary shard |                          |    0.921975 |    0.902433 | -0.01954 |    min |
|              Max cumulative refresh time across primary shard |                          |     1.07292 |      1.1418 |  0.06888 |    min |
|                       Cumulative flush time of primary shards |                          |    0.773367 |    0.671017 | -0.10235 |    min |
|                      Cumulative flush count of primary shards |                          |          60 |          60 |        0 |        |
|                Min cumulative flush time across primary shard |                          |  0.00298333 |  0.00451667 |  0.00153 |    min |
|             Median cumulative flush time across primary shard |                          |   0.0112667 |  0.00998333 | -0.00128 |    min |
|                Max cumulative flush time across primary shard |                          |   0.0307667 |   0.0265833 | -0.00418 |    min |
|                                       Total Young Gen GC time |                          |     199.859 |     194.168 |   -5.691 |      s |
|                                      Total Young Gen GC count |                          |       23525 |       23101 |     -424 |        |
|                                         Total Old Gen GC time |                          |           0 |           0 |        0 |      s |
|                                        Total Old Gen GC count |                          |           0 |           0 |        0 |        |
|                                                    Store size |                          |     67.2936 |     67.0412 | -0.25246 |     GB |
|                                                 Translog size |                          | 6.14673e-06 | 6.14673e-06 |        0 |     GB |
|                                        Heap used for segments |                          |           0 |           0 |        0 |     MB |
|                                      Heap used for doc values |                          |           0 |           0 |        0 |     MB |
|                                           Heap used for terms |                          |           0 |           0 |        0 |     MB |
|                                           Heap used for norms |                          |           0 |           0 |        0 |     MB |
|                                          Heap used for points |                          |           0 |           0 |        0 |     MB |
|                                   Heap used for stored fields |                          |           0 |           0 |        0 |     MB |
|                                                 Segment count |                          |        1526 |        1419 |     -107 |        |
|                                                Min Throughput |             index-append |     26145.1 |     26052.6 |  -92.502 | docs/s |
|                                               Mean Throughput |             index-append |     28911.4 |     28165.7 | -745.719 | docs/s |
|                                             Median Throughput |             index-append |     28611.3 |     28027.6 | -583.639 | docs/s |
|                                                Max Throughput |             index-append |     32810.5 |       30220 | -2590.51 | docs/s |
|                                       50th percentile latency |             index-append |     1388.29 |     1389.87 |  1.57956 |     ms |
|                                       90th percentile latency |             index-append |     2148.72 |     2038.49 | -110.224 |     ms |
|                                       99th percentile latency |             index-append |     3188.14 |      3120.6 | -67.5362 |     ms |
|                                     99.9th percentile latency |             index-append |     3518.46 |     3693.98 |  175.521 |     ms |
|                                      100th percentile latency |             index-append |     3827.51 |     4199.15 |  371.642 |     ms |
|                                  50th percentile service time |             index-append |     1388.29 |     1389.87 |  1.57956 |     ms |
|                                  90th percentile service time |             index-append |     2148.72 |     2038.49 | -110.224 |     ms |
|                                  99th percentile service time |             index-append |     3188.14 |      3120.6 | -67.5362 |     ms |
|                                99.9th percentile service time |             index-append |     3518.46 |     3693.98 |  175.521 |     ms |
|                                 100th percentile service time |             index-append |     3827.51 |     4199.15 |  371.642 |     ms |
|                                                    error rate |             index-append |           0 |           0 |        0 |      % |
|                                                Min Throughput | wait-until-merges-finish |    0.102563 |   0.0737063 | -0.02886 |  ops/s |
|                                               Mean Throughput | wait-until-merges-finish |    0.102563 |   0.0737063 | -0.02886 |  ops/s |
|                                             Median Throughput | wait-until-merges-finish |    0.102563 |   0.0737063 | -0.02886 |  ops/s |
|                                                Max Throughput | wait-until-merges-finish |    0.102563 |   0.0737063 | -0.02886 |  ops/s |
|                                      100th percentile latency | wait-until-merges-finish |     9749.68 |     13566.9 |  3817.19 |     ms |
|                                 100th percentile service time | wait-until-merges-finish |     9749.68 |     13566.9 |  3817.19 |     ms |
|                                                    error rate | wait-until-merges-finish |           0 |           0 |        0 |      % |

-------------------------------
[INFO] SUCCESS (took 0 seconds)
-------------------------------

Suggest we revert this change and make this feature opt-in for clusters with indices of mixed replication strategy.  We can make it default when segment replication is enabled at a cluster level.

@Rishikesh1159 Can we use this issue to track and understand the root cause of the perf regression as well?

@bbarani this issue is reason for performance regression we have seen. @mch2 has looked into this issue of performance regression. We compared benchmark runs locally with this PR in included and reverting (excluding) the PR. We have seen an improvement after reverting the PR as posted in above comment. This PR (enables backpressure setting by default) is causing the regression in document replication, so we reverted it. We should see the improvement in upcoming nightly benchmark runs


libamplsolver: add darwin support

Description of changes
Add darwin support to libamplsolver, with debian-inspired patch to generate dylib.
Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[x] aarch64-darwin

[x] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
22.11 Release Notes (or backporting 22.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[ ] Fits CONTRIBUTING.md.

This push addresses @SuperSandro2000's comments, as well as unbreaking ipopt on Darwin by linking it by default to the library built by libamplsolver. Previously, attempting to build ipopt would produce this result:
error: builder for '/nix/store/idmzayi0f8z39p9s0db2nslwsnmfp44q-ipopt-3.14.5.drv' failed with exit code 2;
       last 10 log lines:
       >   CXXLD    libipoptamplinterface.la
       > ld: library not found for -lamplsolver

Thank you so much for your extra effort @gdinh 🚀


jenkinsx-quickstart-nuxeo-poc to 1.0.13

Promote jenkinsx-quickstart-nuxeo-poc to version 1.0.13

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.jenkins-x-bot seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.


Fix sonarcloud integration with Azure DevOps 97

Summary of change
Added analysis step in azure pipelines configuration
Related issue
Closes #97
Pull request checklist
Please keep this checklist in & ensure you have done the following:

[x] Read, understood and adhered to our contributing document.

[x] Ensure that you were first assigned to a relevant issue before creating this pull request
[x] Ensure code changes pass all tests

[x] Read, understood and adhered to our style guide. A lot of our code reviews are spent on ensuring compliance with our style guide, so it would save a lot of time if this was adhered to from the outset.

[x] Filled in the summary, context (if applicable) and related issue section. Replace the square brackets and its placeholder content with your contents. For an example, see any merged in pull request

[x] Created a branch that has a descriptive name (what your branch is for in a few words and includes the issue number at the end, e.g. test-reading-goal-123

[ ] Set this pull request to 'draft' if you are still working on it

[x] Resolved any merge conflicts

For any of the optional checkboxes (e.g. the screenshots one), still check it if it does not apply.
If in doubt, get in touch with us via our Slack workspace or by creating a new Q&A discussion on GitHub

Using GitHub actions


"install_requires" package to create a CyHy MongoDB connection

🐛 Summary
PE Reports and eventually the PE Mailer processes rely on including the package from mongo-db-from-config. The current implementation in. setup.py in branch ss-working-tests adds this requirement yet fails on install. Also attempted was the following taken from the cyhy-mailer repo:
       "mongo-db-from-config @ http://github.com/cisagov/mongo-db-from-config/tarball/develop#egg=mongo-db-from-config"
Question: Is there additional code or method that's need to be included, or alternatively is this now a step added to install instructions.
To reproduce
Steps to reproduce the behavior:

pe-reports % python3 setup.py install
Then this:
ModuleNotFoundError: No module named 'mongo_db_from_config'

Expected behavior
What did you expect to happen that didn't?
The mongo-db-from-config package installs on setup.

@dav3r I'm assuming you have seen this one once or twice.
Thanks

I'll take a look now.

@dav3r, I will revisit the  setup-env as it may be my issue. Will test following those steps and report back.

Also, in setup.py, I'm pretty sure you can safely replace:
"mongo-db-from-config @ https://github.com/cisagov/mongo-db-from-config/tarball/develop",

with
"mongo-db-from-config @ git+https://github.com/cisagov/mongo-db-from-config",

but @jsf9k or @mcdonnnj should probably double-check me on that.

@dav3r - why would you do that?

Also, in setup.py, I'm pretty sure you can safely replace:
"mongo-db-from-config @ https://github.com/cisagov/mongo-db-from-config/tarball/develop",

with
"mongo-db-from-config @ git+https://github.com/cisagov/mongo-db-from-config",

but @jsf9k or @mcdonnnj should probably double-check me on that.

Why would you do that?

I was able to reproduce @schmelz-ctr's issue from a fresh checkout with these commands:
$ pyenv virtualenv 3.9.7 pe-reports  # I also tried 3.7.9
$ pyenv local pe-reports
$ python3 setup.py install

That python3 setup.py install command is very old fashioned I think.  The install works fine with pip install -e . or pip install ., which I think is what you should be using anyway.

Also, in setup.py, I'm pretty sure you can safely replace:
"mongo-db-from-config @ https://github.com/cisagov/mongo-db-from-config/tarball/develop",

with
"mongo-db-from-config @ git+https://github.com/cisagov/mongo-db-from-config",

but @jsf9k or @mcdonnnj should probably double-check me on that.

Why would you do that?

Slightly shorter syntax and I think it's the same thang, but if I'm mistaken, please let me know.


Change filename of Swagger spec output

It appears that gcloud doesn't play nice with deploying filenames with dashes or underscores in them.

Current coverage is 78.99% (diff: 0.00%)

Merging #25 into master will not change coverage

@@             master        #25   diff @@
==========================================
  Files            34         34
  Lines          4514       4514
  Methods           0          0
  Messages          0          0
  Branches          0          0
==========================================
  Hits           3566       3566
  Misses          948        948
  Partials          0          0

Powered by Codecov. Last update eb92109...b768fe9


Use require to retrieve the app's config

This uses loader.js's require function to retrieve the app's environment config. This bypasses the importSync dependency checking that happens in Embroider builds which causes the build to fail since the dependency doesn't exist in the addon.
I included the commits of #42, #43 and #44 so that I could test it still works under all those new tests. I'll rebase after those are merged.

I'm seeing a build warning:

WARNING: The 'package.json' file for the addon at APP/node_modules/@fortawesome/ember-fontawesome/node_modules/ember-get-config
specifies an invalid, malformed or missing addon at relative path 'lib/baz'

Which looks to be caused by this change.


HCOres Config

    # Number of Materials returned from Smelting an Ore
    I:"Ore Production Count"=n

This part of the config has no effect on the game (latest,1.12)
Suggestion:

    # Number of Materials returned from Smelting an Ore **in a Kiln**
    I:"Ore Production Count"=n

Add an additional config for Kiln recipes. This would help smooth out the techtree and increase compatiblity between HCOres and "normal" Mods, since it allows you to eventually "move past" the reduced ore gen in an interesting and flavorful way.

there exists an additional config for kiln recipes but it's in a different spot

tweaks {
kilnsmelting {
# Number of Materials returned from Smelting an Ore in the Kiln
I:"Ore Production Count"=1
}
}

Forgot about that one. Maybe move it to HCOres ?

That config will stay with the kiln because it allows increasing the production rate of the kiln regardless of whether HCOres is enabled. So it can make the kiln give n ingots if HCOres is disabled.

I can see that, I dont think putting it with HCOres would be too confusing, considering you can tweak HCOres to basically have no effect/positive effect anyways.
Adopting the same syntax for Kiln  (one config to set type (Ingot/Nugget) and one config to set amount) sounds like an improvement, no matter where it is though.

It's not whether it's confusing, it's the fact that I literally cannot move it to HCOre due to how the code works. I also don't intend for there to be that many configs.

The suggestion was not the point of this issue:

# Number of Materials returned from Smelting an Ore
I:"Ore Production Count"=n

This part of the config has no effect on the game (latest,1.12)

Sorry about the nagging in that case, didnt see any indication of a commit in the closing statement.
Keep up the great work


Global constant defined by config.define should not replace variable name

Describe the bug
When config Vite:
{
  define: {
    __DEV__: JSON.stringify(process.env.NODE_ENV === 'development')
  }
}

Cause the following code replacement problems
https://github.com/BerkeleyTrue/warning/blob/122d08acff1b42ed421aa027c1f5fabe207f38d5/warning.js#L17
var __DEV__ = process.env.NODE_ENV !== 'production';

This line of code will be replaced with var false = false that Cause js to run an SyntaxError: Unexpected keyword 'false'.
Reproduction
When config Vite:
{
  define: {
    __DEV__: JSON.stringify(process.env.NODE_ENV === 'development')
  }
}

System Info
System:
  OS: macOS 10.15.7
  CPU: (8) x64 Intel(R) Core(TM) i5-8259U CPU @ 2.30GHz
  Memory: 1.65 GB / 16.00 GB
  Shell: 3.2.57 - /bin/bash
Binaries:
  Node: 14.18.0 - ~/.nvm/versions/node/v14.18.0/bin/node
  Yarn: 1.19.1 - /usr/local/bin/yarn
  npm: 6.14.15 - ~/.nvm/versions/node/v14.18.0/bin/npm
Browsers:
  Edge: 93.0.961.38
  Firefox: 87.0
  Safari: 15.0
npmPackages:
  @vitejs/plugin-vue: ^1.9.3 => 1.9.3
  vite: ^2.6.7 => 2.6.5

Used Package Manager
yarn
Logs
17: var __DEV__ = process.env.NODE_ENV !== 'production';
error during build:
SyntaxError: Unexpected keyword 'false' (15:4) in /Users/jony/projects/vue3-proj/node_modules/warning/warning.js
    at Parser.pp$5.raise (/Users/jony/projects/vue3-proj/node_modules/vite/node_modules/rollup/dist/shared/rollup.js:19495:13)
    at Parser.pp$4.checkUnreserved (/Users/jony/projects/vue3-proj/node_modules/vite/node_modules/rollup/dist/shared/rollup.js:19395:12)
    at Parser.pp$4.parseIdent (/Users/jony/projects/vue3-proj/node_modules/vite/node_modules/rollup/dist/shared/rollup.js:19431:10)
    at Parser.pp$2.parseBindingAtom (/Users/jony/projects/vue3-proj/node_modules/vite/node_modules/rollup/dist/shared/rollup.js:18057:15)
    at Parser.pp$1.parseVarId (/Users/jony/projects/vue3-proj/node_modules/vite/node_modules/rollup/dist/shared/rollup.js:17378:18)
    at Parser.pp$1.parseVar (/Users/jony/projects/vue3-proj/node_modules/vite/node_modules/rollup/dist/shared/rollup.js:17361:10)
    at Parser.pp$1.parseVarStatement (/Users/jony/projects/vue3-proj/node_modules/vite/node_modules/rollup/dist/shared/rollup.js:17227:8)
    at Parser.pp$1.parseStatement (/Users/jony/projects/vue3-proj/node_modules/vite/node_modules/rollup/dist/shared/rollup.js:16975:17)
    at Parser.pp$1.parseTopLevel (/Users/jony/projects/vue3-proj/node_modules/vite/node_modules/rollup/dist/shared/rollup.js:16877:21)
    at Parser.parse (/Users/jony/projects/vue3-proj/node_modules/vite/node_modules/rollup/dist/shared/rollup.js:16658:15)
error Command failed with exit code 1.

Validations

[X] Follow our Code of Conduct
[X] Read the Contributing Guidelines.
[X] Read the docs.
[X] Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
[X] Make sure this is a Vite issue and not a framework-specific issue. For example, if it's a Vue SFC related bug, it should likely be reported to https://github.com/vuejs/vue-next instead.
[X] Check that this is a concrete bug. For Q&A open a GitHub Discussion or join our Discord Chat Server.
[X] The provided reproduction is a minimal reproducible example of the bug.

Perhaps Vite could implement the same preventAssignment feature as @rollup/plugin-replace too. Currently, it uses its own plugin, which does simple replacements.
Related: #5256, #4271

Got this as well when using the notistack npm package.
Adding __DEV__  to the define config results in the error [commonjs] Unexpected keyword 'false' (731:4) in node_modules/notistack/dist/notistack.esm.js because the npm package includes a variable called DEV at
https://github.com/iamhosseindhv/notistack/blob/8c6d6bf599b5af1599a1849d7f32f59b451cec77/src/utils/warning.ts


Cannot edit address in checkout

Type: critical
Describe the bug
Address form "Save and continue" button disabled when editing an address
To Reproduce
Steps to reproduce the behavior:

Add products to cart and checkout
Enter a valid address, then "Save and continue"
"Change" the address
See error the "Save and continue" button disabled, even if you modify fields

Expected behavior
Save button should not be disabled
Screenshots

Desktop (please complete the following information):

OS: MacOS Mohajve
Browser: Chrome
Version: 71.0.3578.98 (Official Build) (64-bit)

Has there been any progress regarding this?

@mikemurray has this been resolved or discussed?

I could still observe this issue in rc11 / current dev channel.

Thanks @janus-reith .
@nnnnat can you take a look at this at some point?

@spencern I'll take a look at this today

I can still that error when adding an address as guest, then leave or reload the cart page and try editing it afterwards, or as a logged in user that already has an address.
In the second case, I can't edit the address from the beginning.

The error originates here in the component library: ShippingAddressCheckoutAction.js#L196
The first time the data is entered, these are the fields that get evaluated(passes as value) :
address1
address2
city
country
fullName
isCommercial
phone
postal
region
proto: Object
After saving and reloading, there are these additional values:
company: null,
isBillingDefault: false,
isShippingDefault: false,
__typename: "Address"
The check fails because of the company field being null
I guess there are several ways to handle this, the easiest would probably be to edit isFormFilled to also skip company, like it is already done with address2.
ShippingAddressCheckoutAction.js#L182
Apart from that it seems like the way the field company is used could be reconsired.

Created the accompanying ticket here: https://github.com/reactioncommerce/reaction-component-library/issues/416

IMHO a fix should be incorporated pretty fix, this is a problem every new user who starts from scratch with the whole current reaction-platform will run into.
During first checkout the user will notice that no shipping options are configured.
The user would go to the dashboard, create a shipping option, go back to the starterkit and wonder that nothing changed.
Usually, at least editing the entered shipping address would then refresh the shipping options, but as that won't work the user is stuck and needs to delete the stored session manually in order to proceed.

@kieckhafer @mikemurray @nnnnat @janus-reith
Seems like this is an issue that we need to resolve - do we have a sense of the effort level required to fix?
@janus-reith if you want to take a crack at this, we'd definitely look at a PR to resolve this. If not we'll try to prioritize as soon as we can

I already narrowed it down and sovled the issue here:
https://github.com/reactioncommerce/reaction-component-library/pull/417
Just need to pull that change and release an updated version of the component library and use that in the starterkit.
A more sophisticated solution that takes cases into account where company will be a usable and maybe required field will probably just become relevant if we have a way to set that behaviour somewhere, so this PR should do the job for now.

Will be closed in https://github.com/reactioncommerce/example-storefront/pull/588