tf.math.pow() is inaccurate with some small integer values presented by tf.float32 on GPU

Click to expand!
Issue Type
Bug
Source
binary
Tensorflow Version
2.9.2
Custom Code
Yes
OS Platform and Distribution
Colab
Mobile device
No response
Python version
No response
Bazel version
No response
GCC/Compiler version
No response
CUDA/cuDNN version
11.2.152, 8.1.1
GPU model and memory
No response
Current Behaviour?
tf.math.pow() loses accuracy when computing squares of small integer values represented by tf.float32 on GPU, such as 5.0^2.0,  7.0^2.0 and 13.0^2.0. What make this strange are:

- This is not happening on CPU
- The integer values are very small
- This is not happening on tf.float64

Standalone code to reproduce the issue
ints = tf.convert_to_tensor(np.arange(20, dtype=np.int32), dtype=tf.int32)
floats = tf.cast(ints, dtype=tf.float32)
doubles = tf.cast(ints, dtype=tf.float64)

with tf.device("/cpu:0"):
  print("cpu ints^2", tf.math.pow(ints, 2))
  print("cpu float32^2", tf.math.pow(floats, 2.))
  print("cpu float64^2", tf.math.pow(doubles, 2.))

with tf.device("/gpu:0"):
  print("gpu ints^2", tf.math.pow(ints, 2))
  print("INACCURATE gpu float32^2", tf.math.pow(floats, 2.))
  print("gpu float64^2", tf.math.pow(doubles, 2.))

Relevant log output
cpu ints^2 tf.Tensor(
[  0   1   4   9  16  25  36  49  64  81 100 121 144 169 196 225 256 289
 324 361], shape=(20,), dtype=int32)
cpu float32^2 tf.Tensor(
[  0.   1.   4.   9.  16.  25.  36.  49.  64.  81. 100. 121. 144. 169.
 196. 225. 256. 289. 324. 361.], shape=(20,), dtype=float32)
cpu float64^2 tf.Tensor(
[  0.   1.   4.   9.  16.  25.  36.  49.  64.  81. 100. 121. 144. 169.
 196. 225. 256. 289. 324. 361.], shape=(20,), dtype=float64)
gpu ints^2 tf.Tensor(
[  0   1   4   9  16  25  36  49  64  81 100 121 144 169 196 225 256 289
 324 361], shape=(20,), dtype=int32)
INACCURATE gpu float32^2 tf.Tensor(
[  0.         1.         4.         9.        16.        24.999998
  36.        48.999996  64.        81.        99.99999  121.
 144.       169.       196.       225.<PHONE_NUMBER>7
 324.       361.      ], shape=(20,), dtype=float32)
gpu float64^2 tf.Tensor(
[  0.   1.   4.   9.  16.  25.  36.  49.  64.  81. 100. 121. 144. 169.
 196. 225. 256. 289. 324. 361.], shape=(20,), dtype=float64)

I suppose that this is MLIR generated on GPU and it could be related to float precision in LLVM?
@cantonios What do you think?
/cc @frgossen

I am bit lost with all the things moving in tsl. Are we using this on CPU?
https://github.com/tensorflow/tensorflow/blob/d46cbbc6433e4e366e0bb45f8632cf84da4b1948/tensorflow/tsl/lib/math/math_util.h#L172-L180

Also https://docs.nvidia.com/cuda/cuda-compiler-driver-nvcc/index.html#options-for-steering-gpu-code-generation-use_fast_math

See https://github.com/tensorflow/tensorflow/issues/56998.

I can reproduce with this a raw CUDA program, so this is a CUDA issue:
#include <stdio.h>
#include <cuda_runtime.h>

__global__ void my_kernel(float x, float y, float* z) {
  *z = powf(x, y);
}

int main() {
  float* d_z;
  cudaMalloc(&d_z, sizeof(*d_z));
  my_kernel<<<1, 1>>>(5., 2., d_z);

  float h_z;
  cudaMemcpy(&h_z, d_z, sizeof(h_z), cudaMemcpyDeviceToHost);
  cudaDeviceSynchronize();
  printf("z: %f\n", h_z);  // prints 24.999998
}

@nluehr is it expected powf might be this inaccurate? If so I'll close this issue.

@reedwm Is that internally is implemented in CUDA as this gist?
https://colab.research.google.com/gist/bhack/5071ca357b0e860aa99f74185cc4ff1e/untitled195.ipynb

And the errors bounds are in this relative table:
https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#intrinsic-functions

@bhack. I'm not sure how it's internally implemented. But based on your error bound link, in Table 7 of this section, powf's error can be up to 10 ulps. The example in the original post has all outputs being off by only zero or one ulps, so the error is within the expected range. So I'll close this issue, since this is working as expected.

@bhack. I'm not sure how it's internally implemented

I was talking about same page:

In my Cuda Colab you can see it gives exactly the same result.

I see. I won't bother calculating the maximum ulp error based on the expression exp2f(y * __log2f(x)), but because the max ulp error in the original post's example is 1, this still falls within the expected ulp range.


Events websocket

Add support for the events websocket.
This was a little...special...in that we've mostly had this consistent API for working with a HTTP stateless api, and now we have what is essentially a socket client. I tried to document this as clearly as possible, but it's entirely possible that we'll have to iterate on this API a few times before it "feels right".

[Current coverage][cc-pull] is 92.86%

Merging [#132][cc-pull] into [master][cc-base-branch] will increase coverage by 0.40%

@@             master       #132   diff @@
==========================================
  Files            10         10
  Lines           424        448    +24
  Methods           0          0
  Messages          0          0
  Branches         51         54     +3
==========================================
+ Hits            392        416    +24
  Misses           20         20
  Partials         12         12

Powered by Codecov. Last updated by [9106309...2ea0f18][cc-compare]
[cc-base-branch]: https://codecov.io/gh/lxc/pylxd/branch/master?src=pr
[cc-compare]: https://codecov.io/gh/lxc/pylxd/compare/91063094a1f4aa38661b45cbe67358d226a5d2ea...2ea0f180b5a8c4ad921702080c4469bf49d94c77
[cc-pull]: https://codecov.io/gh/lxc/pylxd/pull/132?src=pr

+1


PR #17 - Change Image names WMS #8461

Per review, update the image names to NOT use underscore and only use lowercase

@andytael

PR #17

Issue resolved


Character Counter not working in rc.2, but does in 97.

No matter how I try, I cannot get the character counter to work in rc2.
I created this gist to show
https://gist.run/?id=7031cc832f07307adc99b3e626fcd83e
And I know that the new way in rc 2 is to use data-length='5' instead of length='5' which is the way it is done in .97

I did some more testing, and 98.7 is the last version it works in.

In the newest versions, we treat character counter like any other javascript component so you will have to initialize it in javascript.


Add handling of max_gen_len from mlc-llm chat_config

Plugged chat_config created during build of the model and started to use existed required parameters from this config instead of HF one
Get rid of the hardcoded context length, started to use max_gen_len parameter from chat_config
NOTE - this fix requires to rebase on mlc-llm PR1249 to end-to-end scenario work properly

We also need to update https://github.com/octoml/mlc-llm/blob/batch-serving/serve/mlc_serve/engine/staging_engine_worker.py#L296-L297.

We also need to update https://github.com/octoml/mlc-llm/blob/batch-serving/serve/mlc_serve/engine/staging_engine_worker.py#L296-L297.

My bad. Fixed
But need to evaluate the refactoring of the synced and staged engines. They have several duplicating parts

My bad. Fixed But need to evaluate the refactoring of the synced and staged engines. They have several duplicating parts

Yes, this is one of left-over work from https://github.com/octoml/mlc-llm/pull/48. I think we should keep the sync engine for debugging purpose, so it we should refactor. Welcome to do so if you are interested.


test: Don't use php package in TestExtraPackages

The Issue
Failures from packages.sury.org have broken our tests for a day or so, and this has happened before.
TestExtraPackages doesn't really need to test a PHP package, so at least for now it would be nice to move forward, so using a non-deb.sury.org package there.

https://github.com/oerdnj/deb.sury.org/issues/2046#issuecomment-2143131325

How This PR Solves The Issue
Use a non-deb.sury.org package for TestExtraPackages.

I'm going to pull this in the interest of getting green tests that we can build from, but looking forward to any input you may have on Monday @stasadev .


BACKLOG-23170 Prevent cut/delete/drag of categories

JIRA

https://jira.jahia.org/browse/BACKLOG-23170
Description
Prevent cut/delete/drag of categories

Also, I'm not sure how the categories root folder is created, but we can maybe make use of the new blockUiAction mixin on creation and block that generically in the categories app


MM-19005 - Adjusting suggestion list sizing

Summary
MM-19005 - Adjusting suggestion list sizing
This basically just reduces the size of the suggestion list to cater to the height for the remind plugin. Obviously a better solution would be to have the suggestion list render on the root level and then have check the amount of space available from the top/bottom of the screen and adjust its height based on that, but since we're not going too far with that now, this would do as a quick fix.

Ticket Link
https://mattermost.atlassian.net/browse/MM-19005

@asaadmahmood is there a ticket for the proper fixed as you described for future release?

@marianunez Here it is:
https://mattermost.atlassian.net/browse/MM-19799


Argento -> Alejandro

You missed reactivedrop_english.txt

Ok, thank you, I added the missing file.
But CI output looks like I made it worse, can you help?


Rewrite memory allocation to use heap

If I compile with debug flags, things are fine. However, any optimization flags cause segmentation faults.

Done- can optimize to my heart's content with GCC now.

Still getting segfaults- likely not due to stack/heap problems though.


Fix broken links to setup guides.

The links to the setup guides are broken. This fixes them.

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Ooops, someone else had just submitted this.


Added support for module-level behavior in the resource access policy

Description

Module-level behavior functionality has been added to the resource access policy which will allow an user to whitelist specific domain or wildcard domain for a specific module. With this, exceptions can be made to a module to make connection to a list of domains and wildcard domains.
Fixes #2
This is done by getting a stack trace (default of 10 lines), getting the list of file names from it and checking the file parameter as a regex in the file names string.
So a valid resource access policy would be :

const resourceAccessPolicy = {
    "outBoundRequest" : {
		"blockedDomains" : ["*.123.com", "stats.abc.com", "xyz.com", "abce.xyz", "*.abce.com"],
		"allowedDomains" : ["*.domdog.io", "abcd.xyz", "ptsv2.com", "*.abce.com",
			{
				"domains": [
					"abcd.xyz",
					"*.example.com",
				],
				"modules": [
					{
						"file": "\/node_modules\/axios\/"
					}
				]
			}
		]
	}
}

This will allow all files within the axios module to access abcd.xyz and *.example.com. Putting the IP address instead of domain name also works.
Note : If the domains are already mentioned in blockedDomains then it will be blocked without checking for module-level behavior and if it is mentioned in allowedDomains but not in blockedDomains, it will allow it without checking for module-level behavior.
If a specific file needs to be allowed it can be mentioned like : "file": "\/node_modules\/example_module\/file.js$".
The order of preference would be blockedDomains > allowedDomains(non module-level) > allowedDomains(module-level).

Fixes a typo at line 98 in attackMonitoring.js by replacing .lenght with .length
Fixes the checking of outBoundReqDomain in allowedDomains and blockedDomains at line 82 and 101 in attackMonitoring.js by replacing in with .includes as in checks if the index is valid in the array and not if the element is present in the array whereas .includes does the latter.
Example :

var arr = [5];

console.log('Is 5 is an item in the array: ', arr.includes(5)); // true
console.log('do we have the key 5: ', 5 in arr); // false
console.log('we have the key 0: ', 0 in arr); // true

Credit : https://stackoverflow.com/a/48499210
Type of Change

Bug fix
New feature

How has this been tested?
It has been tested with the following modules :

axios
mongodb
node-ssh
nodemailer

@p1xxxel Great job with this pull request. I have made a few comments, can you please make updates based on them.

@Lavakumar Updated.

Added support for appId with which appId can be specified like :
nodeSecurityShield.enableAttackMonitoring(appId, resourceAccessPolicy ,callbackFunction);

This will then set the document-uri to https://<appId> when a violation event occurs.
Fixes #7
However, with the specific example mentioned in the issue with appId as auth_gateway_10, sentry.io filters it like so :

It is due to the presence of the word auth.
Using something like email_gateway_10 works as intended :

Thanks @p1xxxel


installation issues

Hi,
I have a clean install of bcbio and everything went fine with installation and no errors. While I tried to run sample it gave the following errors
[ec2-user@isamples]$ /newvolume/bcbio/anaconda/bin/bcbio_nextgen.py ./sample.yaml -n 10
Running bcbio version: 1.2.3
global config: /newvolume/samples/bcbio_system.yaml
run info config: /newvolume/samples/sample.yaml
[2020-07-28T19:38Z] System YAML configuration: /newvolume/bcbio/galaxy/bcbio_system.yaml.
[2020-07-28T19:38Z] Locale set to C.utf8.
[2020-07-28T19:38Z] Resource requests: bwa, gatk, sambamba, samtools; memory: 3.00, 3.00, 3.00, 3.00; cores: 16, 16, 16, 16
[2020-07-28T19:38Z] Configuring 1 jobs to run, using 10 cores each with 30.1g of memory reserved for each job
[2020-07-28T19:38Z] Timing: organize samples
[2020-07-28T19:38Z] multiprocessing: organize_samples
[2020-07-28T19:38Z] Using input YAML configuration: /newvolume/samples/sample.yaml
[2020-07-28T19:38Z] Checking sample YAML configuration: /newvolume/samples/sample.yaml
[2020-07-28T19:38Z] Retreiving program versions from /newvolume/bcbio/manifest/python-packages.yaml.
[2020-07-28T19:38Z] Retreiving program versions from /newvolume/bcbio/manifest/r-packages.yaml.
[2020-07-28T19:38Z] Testing minimum versions of installed programs
Traceback (most recent call last):
  File "/newvolume/bcbio/anaconda/bin/bcbio_nextgen.py", line 245, in <module>
    main(**kwargs)
  File "/newvolume/bcbio/anaconda/bin/bcbio_nextgen.py", line 46, in main
    run_main(**kwargs)
  File "/newvolume/bcbio/anaconda/lib/python3.6/site-packages/bcbio/pipeline/main.py", line 50, in run_main
    fc_dir, run_info_yaml)
  File "/newvolume/bcbio/anaconda/lib/python3.6/site-packages/bcbio/pipeline/main.py", line 91, in _run_toplevel
    for xs in pipeline(config, run_info_yaml, parallel, dirs, samples):
  File "/newvolume/bcbio/anaconda/lib/python3.6/site-packages/bcbio/pipeline/main.py", line 128, in variant2pipeline
    [x[0]["description"] for x in samples]]])
  File "/newvolume/bcbio/anaconda/lib/python3.6/site-packages/bcbio/distributed/multi.py", line 28, in run_parallel
    return run_multicore(fn, items, config, parallel=parallel)
  File "/newvolume/bcbio/anaconda/lib/python3.6/site-packages/bcbio/distributed/multi.py", line 86, in run_multicore
    for data in joblib.Parallel(parallel["num_jobs"], batch_size=1, backend="multiprocessing")(joblib.delayed(fn)(*x) for x in items):
  File "/newvolume/bcbio/anaconda/lib/python3.6/site-packages/joblib/parallel.py", line 1029, in __call__
    if self.dispatch_one_batch(iterator):
  File "/newvolume/bcbio/anaconda/lib/python3.6/site-packages/joblib/parallel.py", line 847, in dispatch_one_batch
    self._dispatch(tasks)
  File "/newvolume/bcbio/anaconda/lib/python3.6/site-packages/joblib/parallel.py", line 765, in _dispatch
    job = self._backend.apply_async(batch, callback=cb)
  File "/newvolume/bcbio/anaconda/lib/python3.6/site-packages/joblib/_parallel_backends.py", line 208, in apply_async
    result = ImmediateResult(func)
  File "/newvolume/bcbio/anaconda/lib/python3.6/site-packages/joblib/_parallel_backends.py", line 572, in __init__
    self.results = batch()
  File "/newvolume/bcbio/anaconda/lib/python3.6/site-packages/joblib/parallel.py", line 253, in __call__
    for func, args, kwargs in self.items]
  File "/newvolume/bcbio/anaconda/lib/python3.6/site-packages/joblib/parallel.py", line 253, in <listcomp>
    for func, args, kwargs in self.items]
  File "/newvolume/bcbio/anaconda/lib/python3.6/site-packages/bcbio/utils.py", line 55, in wrapper
    return f(*args, **kwargs)
  File "/newvolume/bcbio/anaconda/lib/python3.6/site-packages/bcbio/distributed/multitasks.py", line 455, in organize_samples
    return run_info.organize(*args)
  File "/newvolume/bcbio/anaconda/lib/python3.6/site-packages/bcbio/pipeline/run_info.py", line 94, in organize
    out = _add_provenance(out, dirs, config, not is_cwl)
  File "/newvolume/bcbio/anaconda/lib/python3.6/site-packages/bcbio/pipeline/run_info.py", line 107, in _add_provenance
    versioncheck.testall(items)
  File "/newvolume/bcbio/anaconda/lib/python3.6/site-packages/bcbio/provenance/versioncheck.py", line 97, in testall
    "\n".join(msgs))
OSError: Program problems found. You can upgrade dependencies with:
bcbio_nextgen.py upgrade -u skip --tooldir=/usr/local

Installed version of samtools sort does not have support for multithreading (-@ option) required to support bwa piped alignment and BAM merging. Please upgrade to the latest version from http://samtools.sourceforge.net/
Then i tried to do
/newvolume/bcbio/anaconda/bin/bcbio_nextgen.py upgrade -u skip --tooldir=/newvolume/bcbio/tools/
Upgrading bcbio
Upgrading third party tools to latest versions
--2020-07-28 19:26:56--  https://github.com/chapmanb/cloudbiolinux/archive/master.tar.gz
Resolving github.com (github.com)... <IP_ADDRESS>
Connecting to github.com (github.com)|<IP_ADDRESS>|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://codeload.github.com/chapmanb/cloudbiolinux/tar.gz/master [following]
--2020-07-28 19:26:56--  https://codeload.github.com/chapmanb/cloudbiolinux/tar.gz/master
Resolving codeload.github.com (codeload.github.com)... <IP_ADDRESS>
Connecting to codeload.github.com (codeload.github.com)|<IP_ADDRESS>|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [application/x-gzip]
Saving to: ‘STDOUT’

     0K ........ ........ ........ ........ ........ ........ 16.8M
  3072K ........ ........ ........ .......                    15.2M=0.3s

2020-07-28 19:26:57 (16.1 MB/s) - written to stdout [5189923]

Reading packages from /newvolume/tmpbcbio-install/cloudbiolinux/contrib/flavor/ngs_pipeline_minimal/packages-conda.yaml
Checking for problematic or migrated packages in default environment
# Installing into conda environment default: age-metasv, arriba, bamtools=2.4.0, bamutil, bbmap, bcbio-prioritize, bcbio-variation, bcbio-variation-recall, bcftools, bedops, bedtools=2.27.1, bio-vcf, biobambam, bowtie, bowtie2, break-point-inspector, bwa, bwakit, cage, cancerit-allelecount, chipseq-greylist, cnvkit, coincbc, cramtools, cufflinks, cyvcf2, deeptools, delly, duphold, ensembl-vep=100.*, express, extract-sv-reads, fastp, fastqc>=0.11.8=1, fgbio, freebayes=<IP_ADDRESS>, gatk, gatk4, geneimpacts, genesplicer, gffcompare, goleft, grabix, gridss, gsort, gvcfgenotyper, h5py, hmftools-amber, hmftools-cobalt, hmftools-purple, hmmlearn, hts-nim-tools, htslib, impute2, kallisto>=0.43.1, kraken, ldc>=1.13.0, lofreq, macs2, maxentscan, mbuffer, minimap2, mintmap, mirdeep2=<IP_ADDRESS>, mirtop, moreutils, multiqc, multiqc-bcbio, ngs-disambiguate, novoalign, octopus>=0.5.1b, oncofuse, optitype>=1.3.4, parallel, pbgzip, peddy, perl-sanger-cgp-battenberg, picard, pindel, pizzly, pyloh, pysam>=0.14.0, pythonpy, qsignature, qualimap, rapmap, razers3=3.5.0, rtg-tools, sailfish, salmon, sambamba, samblaster, samtools, scalpel, seq2c<2016, seqbuster, seqcluster, seqtk, sickle-trim, simple_sv_annotation, singlecell-barcodes, snap-aligner=1.0dev.97, snpeff=4.3.1t, solvebio, spades, staden_io_lib, star=2.6.1d, stringtie, subread, survivor, tdrmapper, tophat-recondition, trim-galore, ucsc-bedgraphtobigwig, ucsc-bedtobigbed, ucsc-bigbedinfo, ucsc-bigbedsummary, ucsc-bigbedtobed, ucsc-bigwiginfo, ucsc-bigwigsummary, ucsc-bigwigtobedgraph, ucsc-bigwigtowig, ucsc-fatotwobit, ucsc-gtftogenepred, ucsc-liftover, ucsc-wigtobigwig, umis, vardict, vardict-java, variantbam, varscan, vcfanno, vcflib, verifybamid2, viennarna, vqsr_cnn, vt, wham, anaconda-client, awscli, bzip2, ncurses, nodejs, p7zip, readline, s3gof3r, xz, perl-app-cpanminus, perl-archive-extract, perl-archive-zip, perl-bio-db-sam, perl-cgi, perl-dbi, perl-encode-locale, perl-file-fetch, perl-file-sharedir, perl-file-sharedir-install, perl-ipc-system-simple, perl-lwp-protocol-https, perl-lwp-simple, perl-statistics-descriptive, perl-time-hires, perl-vcftools-vcf, bioconductor-annotate, bioconductor-apeglm, bioconductor-biocgenerics, bioconductor-biocinstaller, bioconductor-biocstyle, bioconductor-biostrings, bioconductor-biovizbase, bioconductor-bsgenome.hsapiens.ucsc.hg19, bioconductor-bsgenome.hsapiens.ucsc.hg38, bioconductor-bubbletree, bioconductor-cn.mops, bioconductor-copynumber, bioconductor-degreport, bioconductor-deseq2, bioconductor-dexseq, bioconductor-dnacopy, bioconductor-genomeinfodbdata, bioconductor-genomicranges, bioconductor-iranges, bioconductor-limma, bioconductor-rtracklayer, bioconductor-snpchip, bioconductor-titancna, bioconductor-vsn>=3.50.0, r-base, r-basejump=0.7.2, r-bcbiornaseq>=0.2.7, r-cghflasso, r-chbutils, r-devtools, r-dplyr, r-dt, r-ggdendro, r-ggplot2, r-ggrepel>=0.7, r-gplots, r-gsalib, r-knitr, r-pheatmap, r-plyr, r-pscbs, r-reshape, r-rmarkdown, r-rsqlite, r-sleuth, r-snow, r-stringi, r-viridis>=0.5, r-wasabi, r=3.5.1, xorg-libxt

# All requested packages already installed.

Collecting package metadata (current_repodata.json): ...working... done
Solving environment: ...working... done

# All requested packages already installed.

# Installing into conda environment dv: deepvariant

# All requested packages already installed.

Collecting package metadata (current_repodata.json): ...working... done
Solving environment: ...working... done

# All requested packages already installed.

# Installing into conda environment htslib1.10: mosdepth

# All requested packages already installed.

Collecting package metadata (current_repodata.json): ...working... done
Solving environment: ...working... done

# All requested packages already installed.

# Installing into conda environment python2: bismark, cpat, cutadapt=1.16, dkfz-bias-filter, gemini, gvcf-regions, hap.py, hisat2, htseq=0.9.1, lumpy-sv, manta, metasv, mirge, phylowgs, platypus-variant, sentieon, smcounter2, smoove, strelka, svtools, svtyper, theta2, tophat, vawk, vcf2db

# All requested packages already installed.

Collecting package metadata (current_repodata.json): ...working... done
Solving environment: ...working... done

# All requested packages already installed.

# Installing into conda environment python3: atropos, crossmap

# All requested packages already installed.

Collecting package metadata (current_repodata.json): ...working... done
Solving environment: ...working... done

# All requested packages already installed.

# Installing into conda environment r36: ataqv, bioconductor-purecn>=1.16.0

# All requested packages already installed.

Collecting package metadata (current_repodata.json): ...working... done
Solving environment: ...working... done

# All requested packages already installed.

# Installing into conda environment samtools0: ericscript

# All requested packages already installed.

Collecting package metadata (current_repodata.json): ...working... done
Solving environment: ...working... done

# All requested packages already installed.

Creating manifest of installed packages in /newvolume/bcbio/manifest
Third party tools upgrade complete.
Upgrade completed successfully.

but still getting the same error message.
I tried to do conda install samtools tools but it says everything is up to date

I think you probably have a samtools in your PATH that is superseding the one that bcbio installs. If you do which samtools, what comes up?

no it should not as this is new ecs optimized instance and has only bcbio installed on it and it does gives bcbio installed samtools
[ec2-user@ ~]$ which samtools
/newvolume/bcbio/anaconda/bin/samtools

I'm stumped. What does /newvolume/bcbio/anaconda/bin/samtools --version report?

hmm, i did that
(base) [ec2-user@ ~]$ which samtools
/newvolume/bcbio/anaconda/bin/samtools
(base) [ec2-user@i~]$ /newvolume/bcbio/anaconda/bin/samtools --version
/newvolume/bcbio/anaconda/bin/samtools: error while loading shared libraries: libcrypto.so.1.0.0: cannot open shared object file: No such file or directory
(base) [ec2-user@ ~]$ export LD_LIBRARY_PATH="/newvolume/bcbio/anaconda/lib"
(base) [ec2-user@i ~]$ /newvolume/bcbio/anaconda/bin/samtools --version
/newvolume/bcbio/anaconda/bin/samtools: error while loading shared libraries: libcrypto.so.1.0.0: cannot open shared object file: No such file or directory
Even after adding the libarary path it does not recognize it and i do see the file in the lib folder.

the library i have is (base) [ec2-user@ip-172-31-46-113 ~]$ ls -al /newvolume/bcbio/anaconda/lib/libcrypto.so.1.1
-rwxrwxr-x 1 ec2-user ec2-user 3278880 Jul 28 00:18 /newvolume/bcbio/anaconda/lib/libcrypto.so.1.1
i guess its expecting to have libcrypto.so.1.0.0:

Ok! I think bioconda is having some trouble with these libraries right now. https://github.com/bioconda/bioconda-recipes/issues/12100 has someone else with this issue.
Can you try:
conda install openssl=1.0 to roll back the shared library?

Hi,
Thank you. the above solution worked. now i have another road block
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/newvolume/bcbio/anaconda/bin/bcbio_nextgen.py", line 245, in
main(**kwargs)
File "/newvolume/bcbio/anaconda/bin/bcbio_nextgen.py", line 46, in main
run_main(**kwargs)
File "/newvolume/bcbio/anaconda/lib/python3.6/site-packages/bcbio/pipeline/main.py", line 50, in run_main
fc_dir, run_info_yaml)
File "/newvolume/bcbio/anaconda/lib/python3.6/site-packages/bcbio/pipeline/main.py", line 91, in _run_toplevel
for xs in pipeline(config, run_info_yaml, parallel, dirs, samples):
File "/newvolume/bcbio/anaconda/lib/python3.6/site-packages/bcbio/pipeline/main.py", line 154, in variant2pipeline
samples = genotype.parallel_variantcall_region(samples, run_parallel)
File "/newvolume/bcbio/anaconda/lib/python3.6/site-packages/bcbio/variation/genotype.py", line 208, in parallel_variantcall_region
"vrn_file", ["region", "sam_ref", "config"]))
File "/newvolume/bcbio/anaconda/lib/python3.6/site-packages/bcbio/distributed/split.py", line 35, in grouped_parallel_split_combine
final_output = parallel_fn(parallel_name, split_args)
File "/newvolume/bcbio/anaconda/lib/python3.6/site-packages/bcbio/distributed/multi.py", line 28, in run_parallel
return run_multicore(fn, items, config, parallel=parallel)
File "/newvolume/bcbio/anaconda/lib/python3.6/site-packages/bcbio/distributed/multi.py", line 86, in run_multicore
for data in joblib.Parallel(parallel["num_jobs"], batch_size=1, backend="multiprocessing")(joblib.delayed(fn)(*x) for x in items):
File "/newvolume/bcbio/anaconda/lib/python3.6/site-packages/joblib/parallel.py", line 1042, in call
self.retrieve()
File "/newvolume/bcbio/anaconda/lib/python3.6/site-packages/joblib/parallel.py", line 921, in retrieve
self._output.extend(job.get(timeout=self.timeout))
File "/newvolume/bcbio/anaconda/lib/python3.6/multiprocessing/pool.py", line 644, in get
raise self._value
NameError: name 'bcbio_bin' is not defined

I added this line to the above line in utils.py and it passed, but still have error from pool.py
bcbio_bin = get_bcbio_bin()

hi,
smoove is complaining and when i try to install this is what it says
conda install -c bioconda smoove
Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Solving environment: |
Found conflicts! Looking for incompatible packages.
This can take several minutes.  Press CTRL-C to abort.
failed
UnsatisfiableError: The following specifications were found
to be incompatible with the existing python installation in your environment:
Specifications:

smoove -> python[version='2.7.*|>=2.7,<2.8.0a0|<3']

Your python: python=3.6
If python is on the left-most side of the chain, that's the version you've asked for.
When python appears to the right, that indicates that the thing on the left is somehow
not available for the python version you are constrained to. Note that conda will not
change your python version to a different minor version unless you explicitly specify
that.

Hi @DiyaVaka ! What is the original smoove error? Why would you need to reinstall it?
Can you show what is in your path?
echo $PATH
Sergey

/newvolume/bcbio/anaconda/bin:/newvolume/bcbio/anaconda/condabin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/ec2-user/.local/bin:/home/ec2-user/bin

the original error with smoove was it could not find it and there is not one available

Hi!
Your installation seems to be broken. What was your installation command? A successful installation should create two bin directories:

/bcbio/anaconda/bin
/bcbio/tools/bin

You need to put both dirs in your PATH.
smoove goes into tools:
which smoove
/bcbio/tools/bin/smoove

Sergey

I see to be hitting a similar issue with 1.2.9 clean installed on a new EC2 instance running Ubuntu 20:
The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/koopa/opt/bcbio-nextgen/tools/bin/bcbio_nextgen.py", line 245, in <module>
    main(**kwargs)
  File "/opt/koopa/opt/bcbio-nextgen/tools/bin/bcbio_nextgen.py", line 46, in main
    run_main(**kwargs)
  File "/opt/koopa/app/bcbio-nextgen/1.2.9/install/anaconda/lib/python3.7/site-packages/bcbio/pipeline/main.py", line 50, in run_main
    fc_dir, run_info_yaml)
  File "/opt/koopa/app/bcbio-nextgen/1.2.9/install/anaconda/lib/python3.7/site-packages/bcbio/pipeline/main.py", line 91, in _run_toplevel
    for xs in pipeline(config, run_info_yaml, parallel, dirs, samples):
  File "/opt/koopa/app/bcbio-nextgen/1.2.9/install/anaconda/lib/python3.7/site-packages/bcbio/pipeline/main.py", line 242, in rnaseqpipeline
    samples = rnaseq_prep_samples(config, run_info_yaml, parallel, dirs, samples)
  File "/opt/koopa/app/bcbio-nextgen/1.2.9/install/anaconda/lib/python3.7/site-packages/bcbio/pipeline/main.py", line 467, in rnaseq_prep_samples
    [x[0]["description"] for x in samples]]])
  File "/opt/koopa/app/bcbio-nextgen/1.2.9/install/anaconda/lib/python3.7/site-packages/bcbio/distributed/multi.py", line 28, in run_parallel
    return run_multicore(fn, items, config, parallel=parallel)
  File "/opt/koopa/app/bcbio-nextgen/1.2.9/install/anaconda/lib/python3.7/site-packages/bcbio/distributed/multi.py", line 86, in run_multicore
    for data in joblib.Parallel(parallel["num_jobs"], batch_size=1, backend="multiprocessing")(joblib.delayed(fn)(*x) for x in items):
  File "/opt/koopa/app/bcbio-nextgen/1.2.9/install/anaconda/lib/python3.7/site-packages/joblib/parallel.py", line 1061, in __call__
    self.retrieve()
  File "/opt/koopa/app/bcbio-nextgen/1.2.9/install/anaconda/lib/python3.7/site-packages/joblib/parallel.py", line 940, in retrieve
    self._output.extend(job.get(timeout=self.timeout))
  File "/opt/koopa/app/bcbio-nextgen/1.2.9/install/anaconda/lib/python3.7/multiprocessing/pool.py", line 657, in get
    raise self._value
OSError: Program problems found. You can upgrade dependencies with:
bcbio_nextgen.py upgrade -u skip --tooldir=/usr/local

Installed version of samtools sort does not have support for multithreading (-@ option) required to support bwa piped alignment and BAM merging. Please upgrade to the latest version from http://samtools.sourceforge.net/

Looks like there's a problem with samtools dependency on libcrypto:
/opt/koopa/opt/bcbio-nextgen/install/anaconda/bin/samtools --version
# samtools: error while loading shared libraries: libcrypto.so.1.0.0: cannot open shared object file: No such file or directory

Here's the current list of environments:
/opt/koopa/opt/bcbio-nextgen/install/anaconda/condabin/conda env list

conda environments:

base                  *  /opt/koopa/app/bcbio-nextgen/1.2.9/install/anaconda
bwakit                   /opt/koopa/app/bcbio-nextgen/1.2.9/install/anaconda/envs/bwakit
dv                       /opt/koopa/app/bcbio-nextgen/1.2.9/install/anaconda/envs/dv
htslib1.10               /opt/koopa/app/bcbio-nextgen/1.2.9/install/anaconda/envs/htslib1.10
htslib1.11               /opt/koopa/app/bcbio-nextgen/1.2.9/install/anaconda/envs/htslib1.11
htslib1.12               /opt/koopa/app/bcbio-nextgen/1.2.9/install/anaconda/envs/htslib1.12
htslib1.12_py3.9         /opt/koopa/app/bcbio-nextgen/1.2.9/install/anaconda/envs/htslib1.12_py3.9
htslib1.9                /opt/koopa/app/bcbio-nextgen/1.2.9/install/anaconda/envs/htslib1.9
java                     /opt/koopa/app/bcbio-nextgen/1.2.9/install/anaconda/envs/java
python2                  /opt/koopa/app/bcbio-nextgen/1.2.9/install/anaconda/envs/python2
python3.6                /opt/koopa/app/bcbio-nextgen/1.2.9/install/anaconda/envs/python3.6
r35                      /opt/koopa/app/bcbio-nextgen/1.2.9/install/anaconda/envs/r35
rbcbiornaseq             /opt/koopa/app/bcbio-nextgen/1.2.9/install/anaconda/envs/rbcbiornaseq
samtools0                /opt/koopa/app/bcbio-nextgen/1.2.9/install/anaconda/envs/samtools0

@mjsteinbaugh
Which samtools version did you end up having in your main bcbio env? I've been seeing this problem recently as well: https://github.com/bcbio/bcbio-nextgen/issues/3557

@amizeranschi I'm working on debugging this! Will let you know

#3557


EventFormattingAgent seems to have inconsistent regex problems

Hello
I'm working on a formatting agent, and found I can't use the matchers as I would expect to be able to. I have two matchers, one works, the other does not:
working (string example: location": " Brainerd, MN, US):
       "path": "{{location}}",
      "regexp": "^(\\s+)(?<location>.*)",
      "to": "loc1"

not working (string example: #1234 Site Administrator, Network Solutions - ):
      "path": "{{title}}",
      "regexp": "^(?<id>%5CW%5Cd*)%5Cs(?<title>.*)%5CW%5CW%5CW(?<location>.*)",
      "to": "jobtitle"

The working example allows me to use {{loc1.location}}, and matches the string like I'd expect. The non-working example produces no output despite the regex matching. I've tried double-slashes in the non-working example as well (although this is not noted in the examples, it does appear to be required), and get the same result. I assume I'm doing something wrong, but I'm not sure if it's my shitty regex or some nuance in how I'm organizing things or what.

Why type the backslashes as ASCII hex code? Your example works fine if I use ^(?<id>\W\d*)\s(?<title>.*)\W\W\W(?<location>.*). (That's "regexp": "^(?<id>\\W\\d*)\\s(?<title>.*)\\W\\W\\W(?<location>.*)"
You only need to double backslash if you're in the JSON raw mode, I does that automatically if you type it in the regular mode.


 The wrapper image exists for 3 reasons:

it allows to add custom CA certs
it allows to embed HTTP proxy vars
it allows to download additional binaries (namely, the Aqua scanner) which are not freely distributed

For #620, but also for other reasons (simplicity, less error prone) it would be nice if we could get rid of wrapping images.
(3) Could be solved by switching from Aqua to Trivy, or, if that is not feasible short/mid-term, download the Aqua scanner during the pipeline run (and cache it for next runs)
(2) Could be solved by adding a config map with HTTP proxy env vars as keys and adding all keys of that config map to the task run. Interestingly, we do not need those env vars in our enterprise cluster at all anymore (though I am not certain how that is implemented right now)
(1) This is by far the trickiest. The problem is that a proper solution requires to run update-ca-trust, which requires root, which is not available in OpenShift. An alternative would be to try and configure things manually. The basis of any solution would be to store the cert in a secret, and then mount that secret into e.g. /etc/ssl/certs (this is e.g. described in https://paraspatidar.medium.com/add-self-signed-or-ca-root-certificate-in-kubernetes-pod-ca-root-certificate-store-cb7863cb3f87). That is already enough to make the cert available to all Go code (see https://go.dev/src/crypto/x509/root_linux.go), which is a big chunk of work. However, e.g. curl wouldn't pick up the cert, one would need to use curl --cacert /etc/ssl/certs/custom.pem. In a similar fashion, we would need to ensure that the build tasks (Python, Java, Node, ...) pick up the cert correctly and any other tools we use (buildah, cnes-report, skopeo, and so on ...) also find the right cert. This looks like a daunting task. Maybe there is also another, Kubernetes "native" way to do this. I have not found anything that fits entirely, but there is still development in that area, see e.g. https://github.com/kubernetes/enhancements/tree/master/keps/sig-auth/3257-trust-anchor-sets.

Thanks for the research and I also find this a good tradeoff. I like that it is less magic and of course that the need to have extra images would go away.
One would need to check for each language whether this works and document. For example in Python one could use https://superuser.com/a/1394302 so that one can do this at pip config time (optionally if certs are provided). I am not sure but would hope that the pip3 config set global.trusted-host "${NEXUS_HOST}" you mentioned could be removed once the certs are properly in place.


Use bitwise operation when sampling for resource leak detection.

Motivation:
Modulo operations are slow, we can use bitwise operation to detect if resource leak detection must be done while sampling.
Modifications:

Ensure the interval is a power of two
Use bitwise operation for sampling
Add benchmark.

Result:
Faster sampling.

@Scottmitch @nmittler PTAL

LGTM

@normanmaurer can you add the benchmark results before and after?

@nmittler will run again with both and add the benchmarks here.

@normanmaurer - you close by accident? or are you opening another PR?

Cherry-picked into 4.0 (577931e8bcf23d860e4fc20f6cf3c07ee354b83b) , 4.1 (a47685b24379f221a5260897fc72297328cd1be9) and master (04d14494911b8a91f91f991a9943bbe5fedfe7b4)

@Scottmitch yep.. now it can be closed :)

@normanmaurer did you run the benchmarks? Would be good to see the improvement.

@nmittler
Before:
Benchmark                            Mode  Cnt          Score          Error  Units
ResourceLeakDetectorBenchmark.open  thrpt   20  314563023.980 ± 21195477.017  ops/s

After:
ResourceLeakDetectorBenchmark.open  thrpt   20  331592614.560 ± 11564671.540  ops/s

:+1:


AU/Vista Possible Features

Waited for DJI FPV Hack from the very beginning, you finally did it! Thank you!
Unlocked adb in my Goggles and Vista, unfortunately i'm not a software developer so i hold breath and wait for interesting features.
I will try to help as much as I can, testing/debug, donate when the opportunity arises.
I can ask some interesting questions? (Maybe some questions will inspire the idea to implement interesting functions).

Theoretically is it possible to somehow hack AU/Vista and unlock features that are not available for Caddx Polar and other cameras?(adjust the image aspect ratio, scene, exposure, saturation, white balance, 120fps??);

"Theoretically Vistas should be able to support USB OTG", so whether it will be possible to connect the USB camera and broadcast video to Goggles? (Connect USB Web camera or IR camera like Seek Thermal... and switch between main/secondary cameras);

0600 added 50mbit mode, it is obvious that the speed limit is artificial, theoretically is it possible to somehow disable the upper speed threshold (50Mbit) to increase image quality? (810p/960p/1080p), or maybe it doesn't work the way we used to think? (More Mbit=Better);

Display the temperature of AU/Vista on screen?

Thank you!

Some of these features could possibly be emulated in software, but would require patching the video pipeline RTOS, which is very difficult to do compared to patching things in the Linux userland and would probably result in extra latency. Unfortunately all these settings happen on the 120fps camera ISP (ap1302) chip itself, so we can't just turn them on for other cameras.

Presuming we get OTG going then theoretically yes, but again the video pipeline resides in a separate RTOS outside the Linux userland which is significantly harder to reverse engineer/patch. It might also be possible to turn the video pipeline off in such a case and use the TCP/UDP stack available to userland, but it's unclear how much bandwidth that actually has available. I'd add that (with the proper patching of RTOS) it might also be possible to use 25mbit for the low latency camera and actually have a second 25mbit stream from USB going at all times for simultaneous streaming to an external decoder/screen in 50mbit wireless mode.

There's actually a secret DUML command to switch to an arbitary bitrate, but due to RTOS limitations latency takes a nosedive as soon as you go significantly above 50mbit. It's worth noting that 25mbit uses a 20mhz wide signal whereas 50mbit uses 40mhz. It's unclear if the hardware could perhaps push even more, but that'd require heavy modifications to the LTE derived baseband in the RTOS and DSPs. Another concern is latency when it comes to the h264 encoder/decoder, usually they'll have different performance characteristics at different bitrates/resolutions. With that said, the 120fps cameras all actually run at 1080p internally and the Goggles screens are 810p, so trading off resolution for effective bitrate might be an interesting proposition. However, again the video pipeline is separate and much harder to tinker with.

This is something for @bri3d's msp-osd. Probably can be done, I know he's already looking at what other metrics we could expose.

I have also a possible future for the airunit.
Maybe it´s posible to switch off the vtx compleat on startup of the quad and have switch on the glasses to switch it on or a switch in betaflight to eneably it. My Idea is if you have problems with gps you can get at first gps because the nois of the vtx is small or not availibe and the power it up to 25mw and after that if you disarm you quad you get full power what you have selected in the goggles.


Extract homepage copy from HTML

The filler text that's currently on the redesigned homepage is largely inlined directly into the HTML. It's preferable to extract the copy to a separate location. We might accomplish this using Hugo partials or localization functionality.
This way, finalized copy can be easily substituted into the design with minimal effort later on.

Closing because editing the homepage copy is not that challenging.


unread message notification

Subject: unread message notification
Description:
I have an unread message notification that I can't get to go away
Results:
once I've clicked on a message the notification should go away
Backend version: develop-2a85c9e0
Frontend version: develop-2a85c9e0
User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:98.0) Gecko/20100101 Firefox/98.0
Page: https://couchers.org/messages
User: salami (4679)

dupe https://github.com/Couchers-org/web-frontend/issues/304


Fix a typo breaking the link to lmath repository in readme

Fix a typo breaking the link to lmath repository in readme

Thanks for spotting that. Since the bot is flagging CLA, do you want to go ahead and do that? Or we can poke the fix in manually if you'd like to avoid the bother?

Thanks for spotting that. Since the bot is flagging CLA, do you want to go ahead and do that? Or we can poke the fix in manually if you'd like to avoid the bother?

It's done. 🙂

I just edited the title because I wrote Lmath instead of Imath

Thanks!


feat(workflow): add repo tagging and docker image tag and release wor…

Fixes #38  : All commits now generate tagged docker images. The images are pushed to ghcr, quay.io and dockerhub. To help with this, we're moving to fully automated semver and adding a version to every merged pull request on master.
For the docker builds and pushes:
You'll need to add variable to this repo's secrets.
QUAY_USERNAME
QUAY_PASSWORD (generate a token for this)
DOCKER_USERNAME
DOCKER_PASSWORD (generate a token for this)
For the semver:
How it works is that it automatically bumps bugfix versions on every commit, but certain keywords in commits (feat(...) and perf(...)) will trigger major and minor versions. For example this PR would raise the current version from 0.6.1 to 0.7.0 because it has feat(workflow) in it.
Before merging this you'll need to add the tag v0.6.1 to the current commit on the master branch. The next versions will be done automatically and we can do away with the version in the dockerfile.

Wow, thanks a lot for this and sorry for the delay. Tomorrow I'll take some time to test it!

I must thank you for the input, I worked on it and ended up with a different approach.

Some of the actions seem to be a c/p from the examples, I added some rationalisation to them
I prefer using buildah+podman to 100% adhere to OCI specs
Each pull request will trigger a helm linting and will build the image without pushing it
Only quay will be used to store the image.
I used the upstream git tag and bump action

I'll probably end up with a pull request later to see how to move this forward

Some of the actions seem to be a c/p from the examples, I added some rationalisation to them

Yup, both of them are. This is pretty bog-standard, no need to reinvent the wheel.

Each pull request will trigger a helm linting and will build the image without pushing it

Should this repo and k8s-mediaserver-helm be linked? For example remove all the helm components from here and have the operator deploy the chart being packaged in k8s-mediaserver-helm? We're duplicating the code and increasing complexity in maintaining it.

I'll probably end up with a pull request later to see how to move this forward

Usually we fix a PR and add more commits to make it fit the vision as it is being discussed in the thread.
I'll close this one out since it's a duplicate of #41


Fix InsertTasks query for Cassandra

What changed?

Fix the conditional update query of InsertTasks operation for Cassandra
Code cleanup

Why?
The conditional update query of InsertTasks should be a no-op. When inserting tasks, we do the conditional update to make sure the rangeID doesn't change and we don't want to update the metadata of tasklists.

How did you test it?
existing persistence test

Potential risks
There is no risk. The change only affect the LastUpdate metadata of tasklist, which is used for garbage collection. However, we only run the garbage collector of tasklists with database not supporting TTL.

Release notes

Documentation Changes

Pull Request Test Coverage Report for Build 0186b84c-99dc-4c61-8338-8615b83df672

9 of 9 (100.0%)  changed or added relevant lines in 3 files are covered.
93 unchanged lines in 11 files lost coverage.
Overall coverage increased (+0.08%) to 57.269%

Files with Coverage Reduction
New Missed Lines
%

common/membership/hashring.go
2
83.54%

common/persistence/historyManager.go
2
66.67%

common/task/weightedRoundRobinTaskScheduler.go
2
89.64%

service/matching/matcher.go
2
91.06%

common/cache/lru.go
3
90.73%

service/history/task/transfer_standby_task_executor.go
4
86.82%

common/task/fifoTaskScheduler.go
7
82.47%

common/persistence/serialization/parser.go
8
63.91%

common/persistence/serialization/thrift_decoder.go
8
57.14%

common/persistence/sql/workflowStateMaps.go
19
83.84%

Totals

Change from base Build 0186b663-3828-4fa7-95a9-b5b2060c7cc3:
0.08%

Covered Lines:
85255

Relevant Lines:
148868

💛  - Coveralls


[DEPRECATED] Use role instead. Only a single role can be passed to an IAM Instance Profile

There are warnings related to your configuration. If no errors occurred,
Terraform will continue despite these warnings. It is a good idea to resolve
these warnings in the near future.
Warnings:

aws_iam_instance_profile.aws-elasticbeanstalk-ec2-role: "roles": [DEPRECATED] Use role instead. Only a single role can be passed to an IAM Instance Profile

Thanks!
This deprecation was introduced at v0.9.3. In addition, role field was introduced at the same time.
If I replace roles to role now, errors will be occured at some environments where using Terraform v0.9.2 or below.
If roles field was removed (maybe next minor release v0.10?), I'll do replace.

Thanks!
Shall i close this issue or just keep it open until roles is removed in the next release?

Please keep this open. After I removed roles field, I'll close by myself.

@Laxman-SM you will need to remove the square brackets [] from around the value. For instance, you need to go from this
roles = ["${aws_iam_role.cluster_instance_role.name}"]
to this
role = "${aws_iam_role.cluster_instance_role.name}"
@dtan4 can this issue be closed yet?


Repair latest assignments

https://github.com/cmu-delphi/covidcast-indicators/issues/1685 revealed that there is at least one and probably loads of instances where the row labeled is_latest_issue in v3 was not, in fact, the latest issue. These faults were propagated to v4:
mysql> select value, issue, from_unixtime(value_updated_timestamp) from covid.signal_history_v where
source="jhu-csse" and `signal`="confirmed_incidence_num" and geo_type="state" and geo_value="ca"
and time_type="day" and time_value=20200403 and issue>20210000;
+-------+----------+----------------------------------------+
| value | issue    | from_unixtime(value_updated_timestamp) |
+-------+----------+----------------------------------------+
|  1227 | 20210126 | 2021-10-18 15:29:04                    |
|  1227 | 20210401 | 2021-10-22 21:44:36                    |
|  1235 | 20210414 | 2021-10-22 21:44:36                    |
|  1293 | 20210916 | 2021-09-16 18:26:24                    |
|  1293 | 20210917 | 2021-10-22 21:44:36                    |
|  1295 | 20210927 | 2021-10-22 21:44:36                    |
|  1324 | 20211030 | 2021-10-30 10:04:45                    |
+-------+----------+----------------------------------------+
7 rows in set (0.08 sec)

mysql> select value, issue, from_unixtime(value_updated_timestamp) from covid.signal_latest_v where
source="jhu-csse" and `signal`="confirmed_incidence_num" and geo_type="state" and geo_value="ca"
and time_type="day" and time_value=202004
03 and issue>20210000;
+-------+----------+----------------------------------------+
| value | issue    | from_unixtime(value_updated_timestamp) |
+-------+----------+----------------------------------------+
|  1295 | 20210927 | 2021-10-22 21:44:36                    |
+-------+----------+----------------------------------------+
1 row in set (0.01 sec)

We should develop, test, and deploy a repair routine which will fix these errors without perturbing the daily job schedule too much. We can start with the scripts in https://github.com/cmu-delphi/delphi-admin/tree/main/fix-epidata but the v4 ones should probably live in this repository as part of the acquisition system.

Fixed!


Replace SD driver w/ upgraded experimental version

Resolves #1105

Going to merge this anyway as the break is due to flakiness of the test and not from this change. Issue will resolve this #1211.


Simplified inject state, fixed jq filter for stateDataFilter section

Many thanks for submitting your Pull Request :heart:!
Please specify parts this PR updates:

[x] Specification
[x] Schema
[ ] Examples
[ ] Extensions
[ ] Roadmap
[ ] Use Cases
[ ] Community
[ ] TCK
[ ] Other

What this PR does / why we need it:
There are two changes

inject state simplified to remove attributes that do not bring value such as timeouts, errors and state data filter as intended behaviour of inject state is to inject static content so there is literally no way it can cause failures for just some instances and by that error handling and timeouts do not apply. Similar for stateDataFilter singe the data is static there is no need to apply filtering as data should be described without filter in the way it is expected to be injected.
fixed jq expression used in state data filter section

Special notes for reviewers:
Additional information (if needed):

+1 on removing timeouts.
not a big fan of removing its state data filter

@tsurdilo so what would be the use case for stateDataFilter? Since data is static, applying filter to it means not needed complexity- you should keep things simple and define data as it would be already after filter being applied.

@tsurdilo I had the same perspective than you, but @mswiderski does make a great point. I'd tend to agree that it's useless in that state, isn't it?

Yeah i think complexity mostly it could help:
For example if we have:
        {
        "vegetables": [
          {
            "name": "potato",
            "amount": 2
          },
          {
            "name": "broccoli",
            "amount": 0
          },
          {
            "name": "celery",
            "amount": 5
          }
        ]
      }

and our inject state adds a "needForMeal" prop to each vegetable item:
.{vegetables: .vegetables | map(. + {"needForMeal": true}) }
would make it simpler than
   {vegetables:  [ (.vegetables | map(. + {"needForMeal": true})  | .[] | select(.amount > 0) ) ] }

but also if you want to do some filtering regardless of the injected static data

Then why not do this:
 {
        "vegetables": [
          {
            "name": "potato",
            "amount": 2,
            "needForMeal": true
          },
          {
            "name": "broccoli",
            "amount": 0 ,
            "needForMeal": true
          },
          {
            "name": "celery",
            "amount": 5 ,
            "needForMeal": true
          }
        ]
      }

I mean that data is static so I don’t see a point in splitting that into two things.
Sorry but I really don’t see the use case that filter brings on static data that is embedded in workflow definition meaning won’t change between instances of that definition. Or am I missing something here?

oh ok, i forgot we removed that inject state "data" can no longer be an expression :(
in that case ok the only reason i could argue to keep state data filter is in case user wants to filter the workflow data output on the last state in the workflow definition.
If user wants to filter the workflow data to make whats returned is in a format and contains only data that they want as result, they would need to apply it in state data filter before the inject state (if inject is last wf state)

@cdavernas i think it was a mistake to make "data" in inject state a json object and no longer an expression. makes inject state kinda useless imo

@cdavernas i think it was a mistake to make "data" in inject state a json object and no longer an expression. makes inject state kinda useless imo

I wouldn't say that it makes it useless, rather specific in its responsibility. Executing expression to produce results that as well allow to do filtering - this is exactly what operations with expressions are for. Having inject doing the same wold be confusing in my opinion.
What I find useful about inject state is that is a great fit for fallback results. Let's say you invoke a rest call that return 404 so you can use inject to put in the data you would like to have for further processing. Or if you wait for an event with timeout, again better to have a data that clearly identifies the meaning rather than null etc.

If it's not too much of a problem, let's keep the state data filter for now and keep everything else as is in PR.
Removing the useless timeout for inject i think is awesome but think we can/should address the state data filter later if that's ok please.

Sure, will update the pr tomorrow morning

@tsurdilo updated to bring back stateDataFilter

@tsurdilo In Synapse, I use the inject state's data as an expression-based object, meaning I'll check all its properties recursively to determine whether or not I should evaluate them.


Develop branch support for OSX

There's a comment that mentioned OSX support for the develop branch, but it's from April.
There are bugs that crash the plugin and when the same export process is repeated in Windows, the Windows edition doesn't break Maya 2019.

Xcode 11 and OSX 10.15 have std::filesystem in the system library but it may cause crashes with Maya.

Thanks for the info! The cmake branch contains the latest tested version on MacOS, I'm still trying to test the develop branch on my Mac, but I have very little spare time right now.
The gulrak filesystem link seems very handy, will look into it.

I'm blocked on using Maya2glTF on the mac, every one of the production model we have causes the import to crash only on the Mac.
Add gulrak filesystem is straightforward, but with the writeJSON function I don't have enough information to debug.

So the cmake branch doesn't work for you?
Can you describe your problem in detail? What version of Maya are you using in your Mac? What version of MacOS?

Maya 2019 and Mac OS 10.14. Currently on windows, so I don't have the minor version.

Crashing in m_glAsset.writeJSON(&jsonWriter, &options); but I'm lost.

That call should use any filesystem related functions, so I guess it is most likely Xcode 10 that causes these crashes.
Does it crash with any GLTF file? So I can reproduce the issue

Here's a log but it's not very useful.
//current scene  = unDisclosed
//====================================================
//command history (most recent last):
//====================================================
//last tool: renderWindowSelectContextItem
//====================================================
//panel with focus: outlinerPanel1
//visible panels:
// modelPanel4 outlinerPanel1 shapePanel1
//====================================================
  4   maya2glTF.bundle                    0x00000001406f471f GLTF::Asset::writeJSON(void*, GLTF::Options*) + 18399
  5   maya2glTF.bundle                    0x00000001405f594c ExportableAsset::save() + 412
  6   maya2glTF.bundle                    0x000000014066da32 Exporter::exportScene(Arguments const&) + 50
  7   maya2glTF.bundle                    0x000000014066d5d6 Exporter::run(Autodesk::Maya::OpenMaya20180000::MArgList const&) + 358
  8   maya2glTF.bundle                    0x000000014066d464 Exporter::doIt(Autodesk::Maya::OpenMaya20180000::MArgList const&) + 36
  9   libOpenMaya.dylib                   0x00000001218c2530 libOpenMaya.dylib + 263472
  10  libCommandEngine.dylib              0x000000010403fd12 libCommandEngine.dylib + 109842
  11  libCommandEngine.dylib              0x000000010403e8f0 libCommandEngine.dylib + 104688
  12  libCommandEngine.dylib              0x00000001040908bf libCommandEngine.dylib + 440511
  13  libCommandEngine.dylib              0x000000010405f580 libCommandEngine.dylib + 238976
  14  libCommandEngine.dylib              0x000000010405f430 libCommandEngine.dylib + 238640
  15  libCommandEngine.dylib              0x000000010408e740 libCommandEngine.dylib + 431936
  16  libCommandEngine.dylib              0x0000000104047be6 libCommandEngine.dylib + 142310
  17  libExtensionLayer.dylib             0x0000000104c7a030 libExtensionLayer.dylib + 221232
  18  libCommandEngine.dylib              0x000000010403e8f0 libCommandEngine.dylib + 104688
  19  libCommandEngine.dylib              0x000000010405f580 libCommandEngine.dylib + 238976
  20  libCommandEngine.dylib              0x000000010409062f libCommandEngine.dylib + 439855
  21  libCommandEngine.dylib              0x000000010406bfa5 libCommandEngine.dylib + 290725
  22  libCommandEngine.dylib              0x000000010405f580 libCommandEngine.dylib + 238976
  23  libCommandEngine.dylib              0x0000000104090f66 libCommandEngine.dylib + 442214
  24  libCommandEngine.dylib              0x000000010405f580 libCommandEngine.dylib + 238976
  25  libCommandEngine.dylib              0x000000010405f430 libCommandEngine.dylib + 238640
  26  libCommandEngine.dylib              0x000000010408e740 libCommandEngine.dylib + 431936
  27  libCommandEngine.dylib              0x0000000104046f58 libCommandEngine.dylib + 139096
  28  libExtensionLayer.dylib             0x0000000104e3e067 libExtensionLayer.dylib + 2072679
  29  libExtensionLayer.dylib             0x0000000104da0797 libExtensionLayer.dylib + 1427351
  30  libExtensionLayer.dylib             0x0000000104c51867 libExtensionLayer.dylib + 55399
  31  libExtensionLayer.dylib             0x0000000104c519f0 libExtensionLayer.dylib + 55792
  32  libQt5Core.5.dylib                  0x000000010b8e0a76 libQt5Core.5.dylib + 2251382
  33  libQt5Widgets.5.dylib               0x000000010c0a7590 libQt5Widgets.5.dylib + 1050000
  34  libQt5Widgets.5.dylib               0x000000010c0a7425 libQt5Widgets.5.dylib + 1049637
  35  libQt5Widgets.5.dylib               0x000000010c0a8470 libQt5Widgets.5.dylib + 1053808
  36  libQt5Widgets.5.dylib               0x000000010bffaebc libQt5Widgets.5.dylib + 343740
  37  libQt5Widgets.5.dylib               0x000000010c0a81c0 libQt5Widgets.5.dylib + 1053120
  38  libQt5Widgets.5.dylib               0x000000010bfb91c6 libQt5Widgets.5.dylib + 74182
  39  libQt5Widgets.5.dylib               0x000000010bfbc5c3 libQt5Widgets.5.dylib + 87491
  40  libExtensionLayer.dylib             0x0000000104c5905a libExtensionLayer.dylib + 86106
  41  libQt5Core.5.dylib                  0x000000010b8ab4e4 libQt5Core.5.dylib + 2032868
  42  libQt5Widgets.5.dylib               0x000000010bfb9b7a libQt5Widgets.5.dylib + 76666
  43  libQt5Widgets.5.dylib               0x000000010c01cef2 libQt5Widgets.5.dylib + 483058

//====================================================
//Memory usage:
//    0.000 Mb	Free Memory
//    0.000 Mb	Free Swap
// 1055.070 Mb	Heap
//====================================================

It doesn't happen on these model, but I can't produce a distributable scene.
https://github.com/assimp/assimp-mdb/tree/master/model-db/fbx

I suspect it's related to the "Turtle renderer" node.

I'm unable to both rebase the mac code to develop and debug this problem. Are you able to help rebase the code?

What is it you want to rebase? Isn't the develop branch suitable for building the MacOS version?
After October 24th I will have more spare time to focus on Maya2glTF again, but right now I have to work hard to present a product on a consumer fair, thanks for your patience and help!

I tried a few minutes ago, the develop branch does not build on the mac.
Thanks for giving notice of availability though!
There should be a new stable release and it should have mac support, but I understand you're working to present a product.

After patching maya2gltf, it crashes constantly unless I give it a "-cm" option.

I don't have a Mac for testing, but I guess it crashes in ExportableResources.getImage since that function calls std filesystem.


[WIP] prep move to NET Standard 2

Connected to #258

[x] Upgraded to new project format
[x] Removed all versions and added support for .NET 4.7 only
[x] Command line build support via BullsEye
[x] Simplified AppVeyor configuration
[x] Removed Topics from namespaces and artifacts names
~move to .NET 4.7.2 only in light of supporting NET Standard 2~ Not needed to just go straight to NET Standard 2

Why support only 4.7? There are new framework features you would like to use in radical? Something related to performance that we can take advantage only if we compile against 4.7?

Probably that’s fine to go only with one target, old fx will stick to old radical version like we will do for some time


Question regarding the Docs and Enigma VB

First off, thanks for creating this package. I have a question regarding its usage.
I use electron and electron-packager to bundle my app.
The output directory contains a couple of dll files, locales and resources folders, .. and MyApp.exe which I can start the app with, which works fine.
I have managed to create an installer from this directory using Inno Setup. Everything works fine this way as well.
However, I'm trying to create a single portable executable now from that directory using Enigma Virtual Box. For that I'm using the enigmavirtualbox package and your generate-evb package. Unfortunately I get an error message when trying to open the final .exe: Cannot find function ordinal 345 in library COMCTL32.dll
var generateEvb = require('generate-evb');
var evb = path.resolve('./builds/windows/app.evb');
var exe = path.resolve('./builds/windows/MyAppBundled/MyApp.exe');
var result = path.resolve('./portable/MyApp.exe');
var input = path.resolve('./builds/windows/MyAppBundled');
generateEvb(evb, exe, result, input);

My questions is regarding the docs: inputExe (String) - the input executable file path. Enigma packs the files from path2pack into a copy of this exe
Does this mean that my path2pack directory must not contain the inputExe itself, which it does in my case? Is Enigma trying to copy my exe file into itself? How would I circumvent this?
This is probably not a problem with this package, but I thought you might have an idea! Thanks.

I think I answered my own question. I stop getting these weird errors when moving "MyApp.exe" out of the "MyAppBundled" directory before packing with Enigma. One problem remains, when moving the resulting single executable around it stops working, which might be due to absolute paths being used..

Hi @bendulum, glad you find it useful :)

Does this mean that my path2pack directory must not contain the inputExe itself, which it does in my case?

Yes, you need to make sure that the inputExe is not in the path2pack. The way to do that currently is to copy the entire folder some where else without the inputExe and use that as your path2pack. To make it easier in the future, I'll add support for a filter callback.
Another thing to consider is that you don't need an asar when using EVB. So instead of holding your app specific code in app.asar, you should hold it in an app folder.

One problem remains, when moving the resulting single executable around it stops working, which might be due to absolute paths being used..

That's odd. I just tried to create a packed Electron app myself and it works fine when moved. What happens when you try to run it? If you don't see an exception, try to run it from cmd and maybe it will print something to the console.

@etiktin Thank you for the quick reply and your feedback! I have "asar" packing already disabled in my electron-packager config, so that should be fine. As soon as I have time I will try again with moving the path2pack directory without the inputExe. I will report back thereafter!
PS: I just found this issue not sure if it could be related.

It works as you described, thanks. Also, moving the final executable around does not break it anymore.

@bendulum, I released a new version of generate-evb, with support for a filter a function. So inputExe can be in path2Pack and you can just filter it out.

Awesome, I will try this as soon as I get the chance!

@bendulum is this still working for you ? I cannot get EVB to work with electron 0.35.0
[30744:1118/135748:FATAL:scoped_handle.cc(184)] Check failed: 0U != (flags & 0x00000002) (0 vs. 0)

@matthiasg it works for me with Electron v0.35.1. I packed Electron's folder, and when I execute, it opens the default app as usual.
Things to check:

Make sure you don't pack inputExe in the path2pack (you can use a filter function to filter electron.exe out).
Don't pack you're app files with asar. It's redundant and might cause issues.
Make sure you're code doesn't try to write to your app folder.

i left the inputExe out (i dragged it in manually leaving the exe out), i will check whether an asar file was created.
i dont have an app loaded yet (its just an index.html plus mini-main to open it) and it doesnt want to write anywhere, but good to know.


In game GUI showing wrong deck

since i messed up im playing esper control and its showing my guildgate deck

Hi! Thank you for your message. Do you have this problem only with this deck or with all decks?

It was all the decks i played yesterday, for some reason its working now.

I just installed this today; when I play with a deck, I can only get the overlay to show the list of decks. How can I show the cards in the deck? Also, how often is the site updated with all of my recent events & matches? I want to be able to easily keep a running tally of all of my games.

The site is updating automatically. The overlay shows your and your cards in deck and how many copies remaining. Also it shows the cards that your opponent played during the match.

I understand what the program is supposed to do, what I don't understand is
how to get to that point from the decks screen. Simply clicking on it does
not seem to be the answer. I also don't understand how it works with
drafting and viewing the cards and my collection.
On Wed, Apr 3, 2019, 21:30 Aquaman88<EMAIL_ADDRESS>wrote:

The site is updating automatically. The overlay shows your and your cards
in deck and how many copies remaining. Also it shows the cards that your
opponent played during the match.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/Razviar/mtga-pro-tracker/issues/376#issuecomment-479715769,
or mute the thread
https://github.com/notifications/unsubscribe-auth/At59JDRRR1BITqb2xEwn6V4bgDLQ4Dqlks5vdVXGgaJpZM4cThQb
.

Just installed and played a few games with this app but no matter what deck i use it only displays the first deck in my list.

That's a similar issue to what I'm having. However, I can't even get the
deck list to show up in the app. It this an issue or is there something I'm
not doing properly?
On Wed, Apr 10, 2019, 16:50 broncapp<EMAIL_ADDRESS>wrote:

[image: mgtawrongdeck]
https://user-images.githubusercontent.com/49498405/55912394-92656100-5bb0-11e9-9344-63261944e63b.PNG
Just installed and played a few games with this app but no matter what
deck i use it only displays the first deck in my list.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/Razviar/mtga-pro-tracker/issues/376#issuecomment-481860333,
or mute the thread
https://github.com/notifications/unsubscribe-auth/At59JG2xvQePzaWGf6YG6VdqTVijJLqrks5vfk6zgaJpZM4cThQb
.

To get it to display click settings, then "Overlay" then check "stream mode". that should make the overlay stay in the foreground. Remember to save the settings then it should auto restart and appear as long as the game is open.

I have also been having this issue. I even reinstalled the program and still it only showed me the one deck.

You should try to check settings in your tracker. You can also try to switch on and switch off some functions.

I am having the same problem with the overlay, here is the issues:
1 - I am playing mono red and shows me my red+green deck in the overlay
2 - When is showing the other guys cards if it goes to the graveyard it counts as 2 cards, it should only count when it is played.

In Game Overlay does not work. I see the wrong deck. It always shows the same deck no matter which deck that I am using.
I deleted some old decks on the web site that had the same name, it helped for a while, then the issue returned.
This has been a constant problem for months.
I would like to contribute money to the developer but I can not since I am unable to use the inhame overlay feature.
Also, When I De-Select show timer, it still shows the timer.

Can you give us the screenshots of your problem and the link to your profile on mtgarena.pro?

I have deleted all of my decks on the .Pro site andafter playing one game
with each deck, I am now seeing the correct deck in the overlay
On Sun, Jul 7, 2019, 7:38 PM Aquaman88<EMAIL_ADDRESS>wrote:

Can you give us the screenshots of your problem and the link to your
profile on mtgarena.pro?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/Razviar/mtga-pro-tracker/issues/376?email_source=notifications&email_token=ALXQSCAH7CZVIULS463VFZTP6KEALA5CNFSM4HCOCQN2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZLWKGA#issuecomment-509043992,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ALXQSCB5LSJVTUEERAC3ISLP6KEALANCNFSM4HCOCQNQ
.


Broken code if a player has a red card and no pairing

https://github.com/cyanfish/heltour/blob/master/heltour/tournament/admin.py#L802
Getting into that situation is a bit of a mess, but it's annoying once you're in it. I'll try and fix it.

Also, it's my code. :sob:


Conversion issues with composite images

Skeletonise, AnalyseSkeleton, Thickness and TriplePointAngles crash if you try to run them with a composite binary image. This is because the input Dataset doesn't get converted into a byte
ImagePlus which these plugins need. Also the types of the output images needs to be changed from Dataset because they don't show as composite to the user.
How to produce a composite binary image

Open HeLa Cells image (File>Open Samples>HeLa Cells)
Binarise image (Process>Binary>Make Binary)

Fixed for TriplePointAngles in commit https://github.com/bonej-org/BoneJ2/commit/8138045de97ae8d2fd7e828170112c5a2aa5256d

Fixed for AnalyseSkeleton in commit https://github.com/bonej-org/BoneJ2/commit/cd5496ff235a0b9cfc10273934d792cf8468a14a

Fixed for Skeletonise in commit https://github.com/bonej-org/BoneJ2/commit/e936e72fcbdf216665ded8e7eb3fd3c904c42a4f

Fixed for Thickness in commit https://github.com/bonej-org/BoneJ2/commit/16666470f53ca50a34e503f16446cf69108f264f


CAMEL-19838 - Upgrade Debezium to version 2.0.33.Final

Description

Target

[x] I checked that the commit is targeting the correct branch (note that Camel 3 uses camel-3.x, whereas Camel 4 uses the main branch)

Tracking

[ ] If this is a large change, bug fix, or code improvement, I checked there is a JIRA issue filed for the change (usually before you start working on it).

Apache Camel coding standards and style

[x] I checked that each commit in the pull request has a meaningful subject line and body.

[x] I have run mvn clean install -DskipTests locally and I have committed all auto-generated changes

/component-tests camel-debezium/camel-debezium-common camel-debezium/camel-debezium-db2  camel-debezium/camel-debezium-mongodb  camel-debezium/camel-debezium-mysql  camel-debezium/camel-debezium-oracle  camel-debezium/camel-debezium-postgres  camel-debezium/camel-debezium-sqlserver


New subscribers receive both "Verify" and "Manage" subscription emails on signup

New subscribers should really only receive the verify as this takes them to the manage screen after anyway. This avoids spamming users with more emails than they probably want.

Resolved.


Database access results in php_network_getaddresses error

Bug description
Database access does not work resulting in the following error:
Illuminate\Database\QueryException with message 'SQLSTATE[HY000] [2002] php_network_getaddresses: getaddrinfo for mysql failed: Name or service not known (SQL: select * from `orders` where `orders`.`id` = 2 limit 1)'

To Reproduce
Run any code that requires access to the database. If the code does not require access to the database, it seems to work fine.
I can also confirm that my application has access to the database and running tinker locally with sail artisan tinker also works fine with the database.
The following is the code that I used:
//Tinker away!

use App\Models\Order;

Order::find(2);

Screenshots

Environment (please complete the following information):

IDE & Version: IntelliJ IDEA 2022.1.2 (Ultimate Edition)
PHP Version: 8.1.7
OS: Windows 10
Laravel Tinker plugin version: 2.3.4
Interpreter Type: Docker (using Laravel Sail)
Laravel Version: 8.83
Tinker version: 2.7.2

This usually means that the interpreter you are using not the one is you are developing with. Not a problem with tinker - just that where you run PHP cannot reach your DB from there.

Thanks for that, when I ran docker exec -it containername php artisan tinker, it worked fine. Is that not to be expected?

It's definitely the correct image and I've tried changing the path mappings to /var/www/html and PHP executable to the following to no avail:
whereis php # run in Laravel sail shell
php: /usr/bin/php8.1 /usr/bin/php /usr/lib/php /etc/php /usr/include/php /usr/share/php

Your help is much appreciated thanks...

I'm still 98% sure this is a thing in your setup that's not working. The plugin is not doing anything with your networking. Does a sail command do some extra linking whilst running as just a docker container isn't, maybe?
I could take a gander at it over a screenshare on discord if you want to. If you'd like, message me and we'll figure out a time. Roboroads#1337.

Thanks, request sent

Solution was to use docker-compose.yml and setting it as docker-compose exec:

I then faced the following issue which, I believe, can be solved by upgrading to the latest PHPStorm version: https://github.com/Roboroads/laravel-tinker/issues/14

Hello, I'm using OS Win11 with WSL2, in the current project I use Laradock with Laravel 7.
I tried to run the command as shown below, but the error is returned in the output.

CLI

Do I need to configure anything else in PhpStorm?

So that means the settings in your .env (or config/database.php ofc.) Are not valid for thst docker container. Make sure that the docker container has a way to connect to the database (make sure they are in the same network?)
I would suggest using docker compose, the setup is a little more user friendly.

I use Laradock in this project, the settings in the .env file are correct, even because I use migrations and access externally in DBMS normally.
That's why I thought it could have some additional configuration.
Where does Laravel Tinker get the credentials/host to access the database? from the .env file?

The plugin runs PHP like you configured. So in your case, Tinker runs as if you were to docker run --rm -it blankedout-workspace:latest php artisan tinker. You need to be sure to also add all the other settings to be able to connect to your DB container.
I just checked the Laradock docs, and it's using the following compose file for MySQL:
   mysql:
      # Redacted unuseful information
      ports:
        - "${MYSQL_PORT}:3306"
      networks:
        - backend

PHPStorn, using this method, spins up a new docker container. Your new container is probably not in this composes "backend" network - which you need to add to your interpreter settings. The reason your webcontainer is able to connect is because following the compose file, they are in the same network (as your migrations are working) and since port 3306 is exposed to yor hostcomputer, your DBMS is able to connect to localhost:3306.

Dont just use --network backend. Since it's a compose file the name is probablyt a little different. Check which network it is using docker network list.

In the latest PHPStorm there is no option for "docker-compose exec". When I run docker exec -it containername php artisan tinker it works fine and my interpreter settings are all correct

@quantumwebco
I, using the latest PHPStorm, have the option to choose.

You're not using compose btw, just raw docker. Make sure your newly spun up container can reach the mysql instance :)

@quantumwebco Your interpreter is "Docker", not "Docer compose".

(when creating an interpreter)
If you want to keep on using docker, the container needt all the settings it needs to reach the mysql container.

Aaah I see, thank you so much!! Just got it working. Might be worth mentioning that you need to choose "Docker for windows - default" as I trued with WSL first but that didn't work. It's my ignorance tho, I just haven't got my head round docker at all yet

@Roboroads I had the same issue with the database using docker. But when I changed php to docker-compose Tinker stopped work totally
Could you help, please?

Ok, I just wasted like a few hours on this too.
I have reasons to believe that phpstorm has problems with parsing the docker compose file which is version 3.
I asked chatgpt to convert the file to version 2.4 (just don't forget to back up).
Then I reconfigured the interpreter, server and testing frameworks in phpstorm. Then it started working...

I have reasons to believe that phpstorm has problems with parsing the docker compose file which is version 3. I asked chatgpt to convert the file to version 2.4 (just don't forget to back up).

That should not be the case.. Compose is just a way of running docker..


Final Backup Missing CSI Driver Tag

/kind bug
What happened?
Final backups don't contain the fsx.openzfs.csi.aws.com/cluster key denoting it was created from the CSI driver
What you expected to happen?
When SkipFinalBackup is set to false, the CSI driver should append its custom tag to the deletion parameter FinalBackupTags, similarly to how we do it during create.
How to reproduce it (as minimally and precisely as possible)?
Dynamically create a filesystem, and set SkipFinalBackupOnDeletion to false. Then delete the filesystem, and check the tags.

/remove-lifecycle stale /lifecycle frozen /good-first-issue

/remove-lifecycle stale
/lifecycle frozen
/good-first-issue


Testing and Documenting of Each Windows Box

As a user, I want my windows box installations to work easily and without issue.

Document: success of build, what had to be done to build, ease of build, time to build, any other comments. If build failed, document the output and exactly what you did.


Add FxA to Access Group invite help text

Before:

After:

Thanks to @nukeador for pointing us at the bug.

Closing this in favor of #2714 which limits the text to 99 chars to keep linting checks happy.


fix: update query code and error handling

Modifies the query code to be more optimized and handles query errors better, particularly for PACKAGE_UNAVAILABLE.
@W-12432030@

confirmed UT will fail without changes
 1) Package Install
       should continue polling publish status on certain query errors:
     SubscriberPackageVersionNotPublishedError: The subscriber package version is not fully available.


The error to predict the protein and RNA structure

When I use this code, the machine reports wrong, shown as below, did you know how to resolve this? Thank you very much.
Traceback (most recent call last):
File "/home/software/RoseTTAFold2NA/network/predict.py", line 345, in
pred.predict(inputs=args.inputs, out_prefix=args.prefix, ffdb=ffdb)
File "/home/software/RoseTTAFold2NA/network/predict.py", line 146, in predict
msa_i, ins_i = parse_a3m(a3m_i, unzip=False)
File "/home/software/RoseTTAFold2NA/network/parsers.py", line 221, in parse_a3m
msa = np.array([list(s) for s in msa], dtype='|S1').view(np.uint8)
ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (722,) + inhomogeneous part.

I‘ve solved this problem. That's caused by my own wrong .a3m files.


Redesign: Text box panel

Summary
Redesign for the Text box panel.

To-do

[ ] Fix Tests

User-facing changes

Before
After

Testing Instructions

QA

[ ] This is a non-user-facing change and requires no QA

This PR can be tested by following these steps:

Add a text
Verify the text box panel looks as in Figma. Test with None, Fill, Highlight and with locked/unlocked padding.

UAT

[ ] UAT should use the same steps as above.

This PR can be tested by following these steps:

Checklist

[x] This PR addresses an existing issue and I have linked this PR to it in ZenHub
[ ] I have tested this code to the best of my abilities
[ ] I have verified accessibility to the best of my abilities (docs)
[ ] I have verified i18n and l10n (translation, right-to-left layout) to the best of my abilities
[ ] This PR contains automated tests (unit, integration, and/or e2e) to verify the code works as intended (docs)
[ ] I have added documentation where necessary
[x] I have added a matching Type: XYZ label to the PR

Fixes #6474

...but otherwise this looks great!

That's actually a slight understatement. The new design looks awesome! Really lovely and smooth to look at...


docker-compose.yml should be for flare instead of blaze

The docker-compose.yml in this project should rather contain a configuration for running flare instead of a blaze test server. The documentation should refer to official blaze documentation on how to set up blaze.
Containing a docker-compose.yml and a Dockerfile for generally different purposes is just confusing.

The docker-compose.yml is used for development of FLARE, were one needs a Blaze server to be able to start FLARE. In which use case do you check out the FLARE repo? Development in FLARE or deployment of a system using FLARE?

I don't need a Blaze for running FLARE. Regardless of development or deployment. I can use whatever FHIR server I want to use. Of course setting up FLARE for development purposes might be a good choice, but as I mentioned putting a Dockerfile and a docker-compose.yml with generally different purposes in a repository is at least confusing.

Ok you are right, I'll fix that.


Remove reference to nf33

Will close #7

The data dir is for files with take a long time to download and process, it is the same for all users so we can save a lot of space and time by providing a shared public location for the files. If nf33 isn't the place then that is fine but we don't want everyone to be hammering the NASA servers and having to reprocess all this data when running on gadi, it will be wasteful and make a big barrier to getting started that people will give up on.
For local users I intend to package up the processed files to github releases, so at least some of the download will be reduced and the processing step skipped but focusing on ARE users on gadi for now.

I agree. I put the files in xp65.


kata-deploy for containerd does not use fully qualified v2 annotation

Description of problem
When kata-deploy is used with containerd and a /etc/containerd/config is created, it populates using the annotation [plugins.cri.containerd.runtimes.kata]   However, according to the v2 containerd config  the annotation should be [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.kata]    The first config does work for me on Clear Linux with containerd v1.3 but others have reported the longer annotation is needed for containerd v1.3 and that shorter annotation did not work for them.

Using the full path for the plugin makes sense to me. We could use help with a PR to update this!

You could detect the version from the TOML (v2 config files have "version = 2" on line 1). This is definitely a showstopper for using kata with such a system (seems to be the default in recent versions of containerd).

Thanks @dsyer - I don't suppose you have the time and inclination to make a PR with that change do you? :-)

There you go. It's only tested manually by me. Feel free to incorporate that change if you can.


Deployment

Deployment
User story
As a professional seeking employment
I want a portfolio that highlights my strongest work as well as the thought processes behind it
So that potential employers can view my information and make contact
Acceptance criteria
Given access to the repository
When I open the website link
Then I see a website that loads correctly with my name, a recent photo or avatar, and links to sections about me, my work, and how to contact me
And:

When one of the links in the navigation is clicked then the UI scrolls to the corresponding section
When viewing the section about my work then the section contains titled images of my applications
When presented with the my first application then that application's image is larger in size than the others
When images of the applications are clicked then I am taken to that deployed application
When the page is resized or viewed on various screens and devices then the layout is responsive and adapts to my viewport

Deployed.
Formatting and links checked.
All working fine.


[WIP] Rework the default format

Closes #74
See #74 for more details.
This PR tweaks the default format so it's more readable, parsable and sortable. It also adds a convenient level_style method for getting a styled log level that can be displayed. This is still a work in progress. We need some more details about the format stability in the docs.

The samples in the readme and docs of log output all need to be updated.

I've also thrown in something to fix #107, so that compiling with default-features = false will use standard library macros that do support the special test capturing.


v1.13 Backports 2023-07-26

[ ] #27002 (@michi-covalent)
[ ] #27014 (@qmonnet)

Once this PR is merged, you can update the PR labels via:
for pr in 27002 27014; do contrib/backporting/set-labels.py $pr done 1.13; done

or with
make add-labels BRANCH=v1.13 ISSUES=27002,27014

/test-backport-1.13


Implement in-place callbacks

[x] tests and linter show no problems (npm t)
[x] tests are added/updated for bug fixes and new features
[x] code is properly formatted (npm run fmt)
[ ] description of changes is added in CHANGELOG.md
[ ] update .d.ts typings

Landed in https://github.com/tshemsedinov/wasm-import/commit/c912333a1d72e5a8817c413523a6a1ed39b8db09 and https://github.com/tshemsedinov/wasm-import/commit/37633ffe3d4fcdb80213f38fc71f281dc801bb0b


Delivery tests for fork repos fail in main repo

Delivery tests for kubectl and helm fail after changes to push and pull images with a secret from fork-local repos like ghcr.io/cncf and ghcr.io/joshgav. I have an idea to let me run tests from a PR to nail down the issue and am working on it.

See #98 which demonstrates that the token associated with the workflow isn't able to pull the image even right after pushing it. Also see #97 for all kinds of troubleshooting commands I've tried.
Further troubleshooting by a repo admin is probably needed, it strangely looks like the workflow token can write but not read images here in the CNCF fork. If a repo admin wants to work through this together with me please notify me here and let's set up some time. In the meantime I'm going to move on to other scenarios - the test system does work in other forks!

This has been fixed by #102