x/crypto: argon2id hash mismatch with reference implementation

What version of Go are you using (go version)?

$ go version
go version go1.12.4 linux/amd64

Does this issue reproduce with the latest release?
I'm already using latest.
What operating system and processor architecture are you using (go env)?
Archlinux from 16.04.2019
CPU:
lscpu output
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
Address sizes:       39 bits physical, 48 bits virtual
CPU(s):              8
On-line CPU(s) list: 0-7
Thread(s) per core:  2
Core(s) per socket:  4
Socket(s):           1
NUMA node(s):        1
Vendor ID:           GenuineIntel
CPU family:          6
Model:               142
Model name:          Intel(R) Core(TM) i7-8550U CPU @ 1.80GHz
Stepping:            10
CPU MHz:             800.009
CPU max MHz:         4000.0000
CPU min MHz:         400.0000
BogoMIPS:            3985.00
Virtualization:      VT-x
L1d cache:           32K
L1i cache:           32K
L2 cache:            256K
L3 cache:            8192K
NUMA node0 CPU(s):   0-7
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf tsc_known_freq pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb invpcid_single pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid mpx rdseed adx smap clflushopt intel_pt xsaveopt xsavec xgetbv1 xsaves dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp flush_l1d

go env Output
$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/zmey/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/zmey/.gopath"
GOPROXY=""
GORACE=""
GOROOT="/usr/lib/go"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build851107517=/tmp/go-build -gno-record-gcc-switches"

What did you do?
I was using argon2 library (golang.org/x/crypto/argon2) and decid thed to check reproducibility of hash result with reference implemetation, which can be found at https://github.com/P-H-C/phc-winner-argon2 (https://godoc.org/golang.org/x/crypto/argon2 has a reference to it).
What did you expect to see?
Executing argon2() with same parameters for x/crypto lib and ref version I would expect at least same hash result and ideally similar execution time.
What did you see instead?
Instead i'v got totally irrelevant results. I'v written program and script for testing:
package main

import (
    "fmt"
    "golang.org/x/crypto/argon2"
    "time"
)

func main() {
    for iters := 2; iters <= 70000; iters *= 2 {
        tStart := time.Now()
        dkey := argon2.IDKey([]byte("SomeKey16CharLen"), []byte("somesalt"), uint32(iters), 2^16, 1, 8)
        tEnd := time.Now()
        fmt.Printf("iters: %d\tresult: %x\ttime: %v\n", iters, dkey, tEnd.Sub(tStart))
    }
}

#!/bin/bash

for ITERS in 2 4 8 16 32 64 128 256 512; do
	RESULT="$((time eval 'echo -n "SomeKey16CharLen" | phc/argon2 somesalt -id -t ${ITERS} -m 16 -p 1 -l 8') 2>&1)"
	HASH="$(grep Hash <<< "$RESULT" | awk '{print $2}')"
	TIME="$(grep real <<< "$RESULT" | awk '{print $2}')"
	echo -e "iters: ${ITERS}\tresult: ${HASH}\ttime: ${TIME}"
done

Here is results.
x/crypto/argon2:
iters: 2	result: 1a16f5cf58fb06fd	time: 139.74µs
iters: 4	result: e399b76071673961	time: 148.875µs
iters: 8	result: 2556103bad749067	time: 244.746µs
iters: 16	result: 9d2996926dabcf35	time: 439.948µs
iters: 32	result: d42584fbf7303d72	time: 914.983µs
iters: 64	result: e12ae1f5e7f6a6d2	time: 1.792167ms
iters: 128	result: 9d3505856901773f	time: 3.435604ms
iters: 256	result: 4029f33fc2b33910	time: 6.883196ms
iters: 512	result: aa6732b30a0bef04	time: 14.113788ms
iters: 1024	result: 04618924fc9a7dbf	time: 25.822188ms
iters: 2048	result: 662aa679f8737cb9	time: 47.643061ms
iters: 4096	result: cb5634a3bad2f2f2	time: 93.270525ms
iters: 8192	result: 021547747e42a912	time: 178.898982ms
iters: 16384	result: b4bdc42c8340cff5	time: 348.955178ms
iters: 32768	result: 6a053e39123c7f79	time: 683.547191ms
iters: 65536	result: 2f664e3585ebe007	time: 1.356076541s

reference binary:
iters: 2	result: 4d3ce6425c81d309	time: 0m0.193s
iters: 4	result: 7ba2b8fc42ca755d	time: 0m0.310s
iters: 8	result: de25c04e0123aba5	time: 0m0.562s
iters: 16	result: d017f15189665f13	time: 0m1.068s
iters: 32	result: 13bd536d9f5e017a	time: 0m2.048s
iters: 64	result: 5696809b6c932ce2	time: 0m3.972s
iters: 128	result: 557fd8e526c5ac81	time: 0m7.824s
iters: 256	result: 25bde0bff7f73888	time: 0m15.583s
iters: 512	result: 30585dbb577a2d31	time: 0m31.017s

So, am i dumb somewhere or it's just a bug in one of implementations?

You used 2^16 instead of 65536 for your test code for the memory usage .  The ^ operator here is actually a bitwise xor, not an exponent.

I have made the same mistake at least thrice :)

I have made the same mistake at least thrice :)


[DOC] Documentation for builtin ConfusionMatrix function.

please review if any changes are there to be done.

please review if any changes are there to be done.

I have also uploaded the output in private doc.

Hi @wizard-420 - Try to make sure that there are only commits related to ConfusionMatrix.
Tip: Google these keywords - how to rebase from command line, git rebase -i

Thank you, @wizard-420 - LGTM. 👍
🎉 🚀


The flutter_gallery app stutters during transitions.

In the Pesto and Shrine parts of the app, the transitions stutter when opening some of the items. This might be because of the larger numbers of items in the list, but it's just a hunch right now.

If you can grab a screenshot of the "performance overlay", that would help. Also, attaching a trace is very useful. Let me know if you need help with those. Thanks!

Here's a trace too:
dart-timeline-2016-6-15 (1).json.zip

Thanks!
I can confirm the stutters on Android M, Nexus 5X. First launch of Pesto has visible stutters.

In case it helps, here's a trace from a 5X, Android M, scoped to just the initial launch of the Pesto app from Gallery (built from flutter run --profile). This should more or less the same as what @dragostis attached above.
dart-timeline-2016-6-15.json.zip

https://github.com/flutter/flutter/issues/5045 is probably a more specific version of this. I think we have other more specific issues, all related to this general issue.
I think there are a few issues inside this overall issue. I'm inclined to close this and refer to real specific issues. Definitely appreciate you creating this issue and uploading the traces!


Wrap domain functions in a handler to ensure proper joining

When the first domain function raises an exception, we risk not joining the second domain. This is bad for reproducability.
Furthermore, unjoined underlying pthreads at exit can apparently cause crashes on Windows (not good either...).
We fix this by wrapping the function bodies in an exception handler and reraising an exception in the "root domain".
Thanks to @patricoferris for pointing this out :pray:

Rebased on main, hoping for more green lights...


ARROW-2301: [Python] Build source distribution inside the manylinux1 docker

@kou Once this is merged, I will upload the source distribution for 0.9.0.

Thanks!

@kou Sorry for the delay, I have uploaded them now.


Add new machine: NOAA Ursa (Hera's follow-on)

Is your feature request related to a problem? Please describe.
Hera will be replaced (time unknown to me) by Ursa (dual GPU/CPU machine). I believe it will be a NOAA Tier-1 platform. It looks like it has the following compiler(s)/mpi:
  intel-oneapi-compilers: intel-oneapi-compilers/2024.2.1

  intel-oneapi-mkl: intel-oneapi-mkl/2024.2.1

  intel-oneapi-mpi: intel-oneapi-mpi/2021.13.1

Currently, we have access to an 8-node pre-TDS via Niagara. It has no network connectivity.
Describe the solution you'd like
Need to add a site config for Ursa and install spack-stack there.

ufs-wm issue for porting to ursa: https://github.com/ufs-community/ufs-weather-model/issues/2435

This comment is copied from the JCSDA / spack-stack slack channel.

Installing spack-stack @ release/1.8.0 (aka develop) on an air-gapped system
Two hosts -- niagara (internet access) and ursa (air-gapped) -- share a filesystem /path/to/shared so that both hosts can see the shared space.
A set of site-specific files are created for ursa in /path/to/shared/spack-stack/configs/sites/tier1/ursa
On ursa, spack can be initialized and an env can be created, but with default settings, spack cannot concretize the env because clingo cannot be built -- spack is expecting internet access to properly bootstrap itself.
As noted in the latest spack-stack docs, a source mirror is necessary so that the air-gapped system has access to said mirror for subsequent installations.
Following are steps taken to create a local source mirror -- on ursa, per the suggestion to

Create an environment as usual, activate it and run the concretization step (spack concretize), but do not start the installation yet.

With the following setup:
clone spack-stack on niagara to the shared space -- niagara and ursa can see the cloned repo.

initialize spack

.  ./setup.sh

create an env

spack stack create env --compiler intel --dir /path/to/shared/envs --name unified-dev.ursa.intel --site ursa --template unified-dev

activate the env

spack env activate -p /path/to/shared/envs/unified-dev.ursa.intel

concretize the env

spack concretize 2>&1 | tee log.concretize

A first attempt to follow those steps in the latest spack-stack docs resulted in a failure to concretize
$ spack concretize 2>&1 | tee log.concretize
==> Installing gmake-4.4.1-uri5j7dm4txw4fqcdyngq66qlsnvzimp [1/21]
==> No binary for gmake-4.4.1-uri5j7dm4txw4fqcdyngq66qlsnvzimp found: installing from source
==> Error: FetchError: All fetchers failed for spack-stage-gmake-4.4.1-uri5j7dm4txw4fqcdyngq66qlsnvzimp
[...]
==> Error: cannot bootstrap the "clingo" Python module from spec "clingo-bootstrap@spack+python %gcc target=x86_64" due to the following failures:
github-actions-v0.5 raised RuntimeError: The binary index is empty
github-actions-v0.4 raised RuntimeError: The binary index is empty
spack-install raised InstallError: Terminating after first install failure: FetchError: All fetchers failed for spack-stage-gmake-4.4.1-uri5j7dm4txw4fqcdyngq66qlsnvzimp

as spack's github-actions for bootstrapping are enabled by default:
$ spack bootstrap list
Name: github-actions-v0.5 ENABLED

 Type: buildcache

 Info:
  url: https://mirror.spack.io/bootstrap/github-actions/v0.5
  homepage: https://github.com/spack/spack-bootstrap-mirrors
  releases: https://github.com/spack/spack-bootstrap-mirrors/releases

 Description:
  Buildcache generated from a public workflow using Github Actions.
  The sha256 checksum of binaries is checked before installation.

Name: github-actions-v0.4 ENABLED

 Type: buildcache

 Info:
  url: https://mirror.spack.io/bootstrap/github-actions/v0.4
  homepage: https://github.com/spack/spack-bootstrap-mirrors
  releases: https://github.com/spack/spack-bootstrap-mirrors/releases

 Description:
  Buildcache generated from a public workflow using Github Actions.
  The sha256 checksum of binaries is checked before installation.

Name: spack-install ENABLED

 Type: install

 Info:
  url: https://mirror.spack.io

 Description:
  Specs built from sources downloaded from the Spack public mirror.

Disabling the github-actions:
$ spack bootstrap disable github-actions-v0.5
==> "github-actions-v0.5" is now disabled and will not be used for bootstrapping
$ spack bootstrap disable github-actions-v0.4
==> "github-actions-v0.4" is now disabled and will not be used for bootstrapping

still resulted in a failure to concretize, perhaps (speculating here) because installing from the spack public mirror, i.e.
spack-install ENABLED
remains enabled.
Since the spack-stack repo is shared between niagara and ursa, the spack-install bootstrap method is left ENABLED, and a source bootstrap mirror is created from niagara:
Enable github-actions so that niagara can download bootstrap sources and create a mirror:
$ spack bootstrap enable github-actions-v0.5
$ spack bootstrap enable github-actions-v0.4

$ spack bootstrap mirror /path/to shared/bootstrap
==> Adding "clingo-bootstrap@spack+python %gcc target=x86_64" and dependencies to the mirror at /path/to/shared/bootstrap/bootstrap_cache
==> Warning: Error while fetching<EMAIL_ADDRESS> All fetchers failed for spack-stage-curl-8.7.1-gzgzlqxiqb23zoef5lnzkf74v5bvnz2b
==> Adding<EMAIL_ADDRESS>%gcc target=x86_64" and dependencies to the mirror at /path/to/shared/bootstrap/bootstrap_cache
==> Adding<EMAIL_ADDRESS>%gcc target=x86_64" and dependencies to the mirror at /path/to/shared/bootstrap/bootstrap_cache
==> Adding "gnuconfig" and dependencies to the mirror at /path/to/shared/bootstrap/bootstrap_cache

To register the mirror on the platform where it's supposed to be used, move "/path/to/shared/bootstrap" to its final location and run the following command(s):

 % spack bootstrap add --trust local-sources <final-path>/metadata/sources

NB: it's not clear to me if a failure to fetch<EMAIL_ADDRESS>will eventually be fatal at some point.
Note that the suggestion to move "/path/to/shared/bootstrap" to its final location isn't necessary, as it's already in a shared space for niagara and ursa.
Now on ursa:
$ spack bootstrap add --trust local-sources /path/to/shared/bootstrap/metadata/sources
==> New bootstrapping source "local-sources" added in the "user" configuration scope
==> "local-sources" is now enabled for bootstrapping

$ spack bootstrap list
Name: local-sources ENABLED

  Type: install

  Info:
    url: ../../bootstrap_cache

  Description:
    Mirror with software needed to bootstrap Spack
[...]

$ spack -b find
-- linux-centos7-x86_64 /<EMAIL_ADDRESS>----------------------------
<EMAIL_ADDRESS>clingo-bootstrap@spack <EMAIL_ADDRESS><EMAIL_ADDRESS>==> 4 installed packages

# disable github-actions
$ spack bootstrap disable github-actions-v0.5
==> "github-actions-v0.5" is now disabled and will not be used for bootstrapping
$ spack bootstrap disable github-actions-v0.4
==> "github-actions-v0.4" is now disabled and will not be used for bootstrapping

# verify that the only bootstrapping source available is "local-sources"
$ cat ~/.spack/bootstrap.yaml
bootstrap:
  trusted:
    github-actions-v0.5: false
    github-actions-v0.4: false
    local-sources: true
  sources:
  - name: local-sources
    metadata: /path/to/shared/bootstrap/metadata/sources

# also make sure that ~/.spack/linux/compilers.yaml DOES NOT EXIST

Next, create and activate the env, as before / noted above, and concretize:
$ spack stack create env [...]
$ spack env activate [...]
$ spack concretize 2>&1 | tee log.concretize
==> Warning: cannot detect libc from<EMAIL_ADDRESS>The compiler will not be used during concretization.
==> Error: concretization failed for the following reasons:

  1. No valid value for variant 'mpi' of package 'esmf'
  2. Cannot set the required compiler: esmf%intel

Poking around online and looking through this spack issue and looking at output of commands requested by spack developers:
$ /usr/bin/ldd --version
ldd (GNU libc) 2.34
Copyright (C) 2021 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Written by Roland McGrath and Ulrich Drepper.

$ /lib64/ld-linux-x86-64.so.2 --version
ld.so (GNU libc) stable release version 2.34.
Copyright (C) 2021 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.

so it appears that libc is available on the host. Also per suggestions in the issue, output from
$ spack python

for c in spack.compilers.all_compilers():
  print(c)
  print(c.compiler_verbose_output)
  print()

is voluminous, and can be made available if desired.
The compilers.yaml file for ursa:
$ cat configs/sites/tier1/hostB/compilers.yaml

compilers:
- compiler:
  spec<EMAIL_ADDRESS>  paths:
   cc: /apps/oneapi/compiler/2022.0.2/linux/bin/intel64/icx
   cxx: /apps/oneapi/compiler/2022.0.2/linux/bin/intel64/icpc
   f77: /apps/oneapi/compiler/2022.0.2/linux/bin/intel64/ifort
   fc: /apps/oneapi/compiler/2022.0.2/linux/bin/intel64/ifort
  flags: {}
  operating_system: rocky9
  modules:
  - intel-oneapi-compilers/2024.2.1
  environment:
   prepend_path:
    PATH: '/usr/bin'
    LD_LIBRARY_PATH: '/usr/lib:/usr/lib64'
    CPATH: '/usr/include/c++/11'
  extra_rpaths: []
- compiler:
  spec<EMAIL_ADDRESS>  paths:
   cc: /usr/bin/gcc
   cxx: /usr/bin/g++
   f77: /usr/bin/gfortran
   fc: /usr/bin/gfortran
  flags: {}
  operating_system: rocky9
  modules: []
  environment: {}
  extra_rpaths: []

There is not a system-provided module for GNU compilers.
Ideas?

Way too much for me to digest, but I will point out that your compiler config is wrong. Your paths (cc etc are pointing to 2022.0.2 but you are using @2024.2.1. Then, you use icx but icpc - why?
The convention (that works) is:

If using icc, icpc, ifort (or ifx, but stay away for now) then the compiler is called intel@ and the version is determined from running icc --version - it is NOT the oneapi version.
If using icx, icpx, ifort (or ifx, but stay away for now) then the compiler is called oneapi@ and the version is determined from running icx --version - which usually, but maybe not always, coincides with the oneapi version.

Thanks very much for pointing out the errors in compilers.yaml -- not sure how I crossed paths, as it were, but they are certainly incorrect.
And it's helpful to learn about the newer Intel compilers, first chance to use them.

niagara (internet access) and ursa (air-gapped) -- share a filesystem /path/to/shared so that both hosts can see the shared space.

clone spack-stack on niagara to the shared space -- niagara and ursa can see the cloned repo.

ursa provides the<EMAIL_ADDRESS>compiler suite, so add:

spack-stack/configs/sites/tier1/ursa/packages_oneapi.yaml

create source and binary bootstrap mirrors on niagara:

$ spack bootstrap mirror --binary-packages /path/to/shared/bootstrap
==> Adding "clingo-bootstrap@spack+python %gcc target=x86_64" and dependencies to the mirror at /path/to/shared/bootstrap/bootstrap_cache
==> Warning: Error while fetching<EMAIL_ADDRESS>  All fetchers failed for spack-stage-curl-8.7.1-gzgzlqxiqb23zoef5lnzkf74v5bvnz2b
==> Adding<EMAIL_ADDRESS>%gcc target=x86_64" and dependencies to the mirror at /path/to/shared/bootstrap/bootstrap_cache
==> Adding<EMAIL_ADDRESS>%gcc target=x86_64" and dependencies to the mirror at /path/to/shared/bootstrap/bootstrap_cache
==> Adding "gnuconfig" and dependencies to the mirror at /path/to/shared/bootstrap/bootstrap_cache
==> Adding binary packages from "https://github.com/spack/spack-bootstrap-mirrors/releases/download/v0.4/bootstrap-buildcache.tar.gz" to the mirror at /path/to/shared/bootstrap/bootstrap_cache

To register the mirror on the platform where it's supposed to be used, move "/path/to/shared/bootstrap" to its final location and run the following command(s):

  % spack bootstrap add --trust local-sources <final-path>/metadata/sources
  % spack bootstrap add --trust local-binaries <final-path>/metadata/binaries

NB: it's not clear to me if a failure to fetch<EMAIL_ADDRESS>will eventually be fatal at some point.
Note that the suggestion to move "/path/to/shared/bootstrap" to its final location isn't necessary, as it's already in a shared space for niagara and ursa.
Now on ursa:
# disable github-actions
$ spack bootstrap disable github-actions-v0.5
==> "github-actions-v0.5" is now disabled and will not be used for bootstrapping
$ spack bootstrap disable github-actions-v0.4
==> "github-actions-v0.4" is now disabled and will not be used for bootstrapping

$ spack bootstrap add --trust local-sources /path/to/shared/bootstrap/metadata/sources
==> New bootstrapping source "local-sources" added in the "user" configuration scope
==> "local-sources" is now enabled for bootstrapping

$ spack bootstrap add --trust local-binaries /path/to/shared/bootstrap/metadata/binaries
==> New bootstrapping source "local-binaries" added in the "user" configuration scope
==> "local-binaries" is now enabled for bootstrapping

$ spack -b find
-- linux-centos7-x86_64 /<EMAIL_ADDRESS>----------------------------
<EMAIL_ADDRESS>clingo-bootstrap@spack  <EMAIL_ADDRESS><EMAIL_ADDRESS>==> 4 installed packages

$ spack bootstrap list
Name: local-binaries ENABLED

  Type: buildcache

  Info:
    url: ../../bootstrap_cache
    homepage: https://github.com/spack/spack-bootstrap-mirrors
    releases: https://github.com/spack/spack-bootstrap-mirrors/releases
    tarball: https://github.com/spack/spack-bootstrap-mirrors/releases/download/v0.4/bootstrap-buildcache.tar.gz

  Description:
    Buildcache copied from a public tarball available on Github.The sha256 checksum of binaries is checked before installation.

Name: local-sources ENABLED

  Type: install

  Info:
    url: ../../bootstrap_cache

  Description:
    Mirror with software needed to bootstrap Spack

Name: spack-install ENABLED

  Type: install

  Info:
    url: https://mirror.spack.io

  Description:
    Specs built from sources downloaded from the Spack public mirror.

Name: github-actions-v0.5 DISABLED

Name: github-actions-v0.4 DISABLED

$ cat ~/.spack/bootstrap.yaml
bootstrap:
  trusted:
    github-actions-v0.5: false
    github-actions-v0.4: false
    local-sources: true
    local-binaries: true
  sources:
  - name: local-binaries
    metadata: /collab1/data/Richard.Grubin/contrib/spack-stack-rocky9/bootstrap/metadata/binaries
  - name: local-sources
    metadata: /collab1/data/Richard.Grubin/contrib/spack-stack-rocky9/bootstrap/metadata/sources

# also make sure that ~/.spack/linux/compilers.yaml DOES NOT EXIST

Create the unified-dev env:

initialize spack

.  ./setup.sh

create an env

spack stack create env --compiler oneapi --dir /path/to/shared/envs --name unified-dev.ursa.oneapi --site ursa --template unified-dev

activate the env

spack env activate -p /path/to/shared/envs/unified-dev.ursa.oneapi

concretize the env

spack concretize 2>&1 | tee log.concretize

Concretization is successful.

Following on with successful concretization:
spack mirrors for air-gapped systems

Steps 1, 2 are complete.
Step 3: git-lfs and python3 (v3.12.4) are available (via modules in a shared space, irrespective of internet connectivity). Compiler modules are different, and don't seem necessary, so not loaded.
Step 4: on niagara (connected host), in the same spack-stack repo as is available on ursa (air-gapped host), the air-gapped env is created and populated as described.
Step 5: no need to rysnc (or similar) the source mirror, as it lives on a filesystem shared by both hosts
Step 6: the source mirror is added and is listed at the top of output from spack mirror list
spack install begins but fails attempting to install libtiprc; a logfile is attached. It's noteworthy that<EMAIL_ADDRESS>with appropriate site config files appears to try to do "the right thing."
It is noted that ursa is likely not fully built out at this point, and that may be at play here. A help desk request has been submitted to install some utilities that spack-stack needs.
See: https://github.com/rickgrubin-noaa/spack-stack/tree/JCSDA/1297/configs/sites/tier1/ursa for details.
spack install logfile log.install.txt

Making progress thanks to RDHPCS folks installing some requested tools / software.
CRTM failed to build; it requires fix files:
[...]
  crtm:
    #require: '+fix'

and concretizes as such:
==> Concretized<EMAIL_ADDRESS> -   nemiyuu  crtm<EMAIL_ADDRESS>build_system=cmake build_type=Release generator=make arch=linux-rocky9-zen3
[...]
 -   vafxwrx      ^crtm-fix<EMAIL_ADDRESS>build_system=generic arch=linux-rocky9-zen3

and attempts to install as:
==> Installing crtm-fix-<IP_ADDRESS>_emc-vafxwrxlqqbzmcg4ngigso3hgul7xzs7 [29/343]
==> No binary for crtm-fix-<IP_ADDRESS>_emc-vafxwrxlqqbzmcg4ngigso3hgul7xzs7 found: installing from source
==> Error: FetchError: All fetchers failed for spack-stage-crtm-fix-<IP_ADDRESS>_emc-vafxwrxlqqbzmcg4ngigso3hgul7xzs7

When creating the air-gapped source mirror:
[...]
==> Adding package<EMAIL_ADDRESS>to mirror
==> Warning: Error while fetching<EMAIL_ADDRESS>  All fetchers failed for spack-stage-crtm-fix-<IP_ADDRESS>_emc-vafxwrxlqqbzmcg4ngigso3hgul7xzs7

What must I do so with respect to access for<EMAIL_ADDRESS>so that I can add it to the source mirror?

@rickgrubin-noaa I got same thing on Hera. I did also mirroring (from Hercules), and placed it here:
/contrib/spack-stack/mirror/
and changed site/mirrors.yaml file to:
(/contrib/spack-stack/spack-stack-1.8.0/envs/ue-intel-2021.5.0/site/mirrors.yaml)
mirrors:
  local-source:
    fetch:
      url: file:///contrib/spack-stack/mirror
      access_pair:
      - null
      - null
      access_token: null
      profile: null
      endpoint_url: null
    push:
      url: file:///contrib/spack-stack/mirror
      access_pair:
      - null
      - null
      access_token: null
      profile: null
      endpoint_url: null

It concretized like this:
(/contrib/spack-stack/spack-stack-1.8.0/envs/ue-intel-2021.5.0/log.concretize)
==> Concretized<EMAIL_ADDRESS> -   fvuseal  crtm<EMAIL_ADDRESS>build_system=cmake build_type=Release generator=make arch=linux-rocky8-haswell
 -   al2juhz      ^crtm-fix<EMAIL_ADDRESS>build_system=generic arch=linux-rocky8-haswell

and installed like this:
(/contrib/spack-stack/spack-stack-1.8.0/envs/ue-intel-2021.5.0/log.install)
==> Installing crtm-fix-<IP_ADDRESS>_emc-al2juhzkilqssvyg75jsni664uhcknls [22/291]
==> No binary for crtm-fix-<IP_ADDRESS>_emc-al2juhzkilqssvyg75jsni664uhcknls found: installing from source
==> Fetching file:///contrib/spack-stack/mirror/_source-cache/archive/6e/6e4005b780435c8e280d6bfa23808d8f12609dfd72f77717d046d4795cac0457.tgz
==> No patches needed for crtm-fix
==> crtm-fix: Executing phase: 'install'

After that crtm-fix-<IP_ADDRESS>_emc created needed fix files.

pnetcdf fails to install; quite confused:
==> Installing parallel-netcdf-1.12.3-oirtvs5yjc232ab7qajgvo5chycpmd7e [27/343]
==> No binary for parallel-netcdf-1.12.3-oirtvs5yjc232ab7qajgvo5chycpmd7e found: installing from source
==> Fetching file:///collab1/data/Richard.Grubin/git/EPIC/spack-stack/spack/var/spack/environments/air_gapped_mirror_env/mirror/_source-cache/archive/43/439e359d09bb93d0e58a6e3f928f39c2eae965b6c97f64e67cd42220d6034f77.tar.gz
==> No patches needed for parallel-netcdf
==> parallel-netcdf: Executing phase: 'autoreconf'
==> parallel-netcdf: Executing phase: 'configure'
[...]
configure: error: Directory '/apps/spack-2024-09/linux-rocky9-x86_64/oneapi-2024.2.1/intel-oneapi-mpi-2021.13.1-ss72gbndvat3oz22sa6lhmlbjkeabrn4/mpi/2021.13.1' specified in --with-mpi does not exist or is not a directory

and indeed that directory doesn't exist:
$ file /apps/spack-2024-09/linux-rocky9-x86_64/oneapi-2024.2.1/intel-oneapi-mpi-2021.13.1-ss72gbndvat3oz22sa6lhmlbjkeabrn4/mpi/2021.13.1
/apps/spack-2024-09/linux-rocky9-x86_64/oneapi-2024.2.1/intel-oneapi-mpi-2021.13.1-ss72gbndvat3oz22sa6lhmlbjkeabrn4/mpi/2021.13.1: cannot open `/apps/spack-2024-09/linux-rocky9-x86_64/oneapi-2024.2.1/intel-oneapi-mpi-2021.13.1-ss72gbndvat3oz22sa6lhmlbjkeabrn4/mpi/2021.13.1' (No such file or directory)

but deleting the trailing _.1_ from that path is a directory that exists:
$ file /apps/spack-2024-09/linux-rocky9-x86_64/oneapi-2024.2.1/intel-oneapi-mpi-2021.13.1-ss72gbndvat3oz22sa6lhmlbjkeabrn4/mpi/2021.13
/apps/spack-2024-09/linux-rocky9-x86_64/oneapi-2024.2.1/intel-oneapi-mpi-2021.13.1-ss72gbndvat3oz22sa6lhmlbjkeabrn4/mpi/2021.13: setgid, directory

packages_oneapi.yaml contains:
packages:
  all:
    compiler<EMAIL_ADDRESS>    providers:
      mpi<EMAIL_ADDRESS>      # Remove the next three lines to switch to intel-oneapi-mkl
      #blas:: [openblas]
      #fftw-api:: [fftw]
      #lapack:: [openblas]
  mpi:
    buildable: False
  intel-oneapi-mpi:
    externals:
    - spec: intel-oneapi-mpi<EMAIL_ADDRESS>      modules:
      - intel-oneapi-mpi/2021.13.1
      prefix: /apps/spack-2024-09/linux-rocky9-x86_64/oneapi-2024.2.1/intel-oneapi-mpi-2021.13.1-ss72gbndvat3oz22sa6lhmlbjkeabrn4

which I think is correct.
$ module show intel-oneapi-mpi/2021.13.1
-----------------------------------------------------------------------------------------------------------------------------------------------------------
   /apps/spack/modules/linux-rocky9-x86_64/Core/intel-oneapi-mpi/2021.13.1.lua:
-----------------------------------------------------------------------------------------------------------------------------------------------------------
whatis("Name : intel-oneapi-mpi")
whatis("Version : 2021.13.1")
[...]
family("mpi")
prepend_path("MODULEPATH","/apps/spack-2024-09/modules/linux-rocky9-x86_64/intel-oneapi-mpi/2021.13.1-aalgxls/Core")
setenv("LMOD_MPI_NAME","intel-oneapi-mpi")
setenv("LMOD_MPI_VERSION","2021.13.1-aalgxls")
[...]
setenv("I_MPI_ROOT","/apps/spack-2024-09/linux-rocky9-x86_64/gcc-11.4.1/intel-oneapi-mpi-2021.13.1-aalgxlsn6mj2wsdebjjv4fl4duddijzr/mpi/2021.13")

so I_MPI_ROOT is the directory "found" above, however spack is choosing to use the directory with the trailing _.1_,
.../mpi/2021.13.1
Am I configuring something incorrectly?

Update through 2024-10-04
OS

Rocky Linux 9.4 (Blue Onyx)

Compilers

<EMAIL_ADDRESS>(latest version)
icx
ifort

this is the last supported version of ifort
will also attempt to build a stack with ifx

icpx

MPI

<EMAIL_ADDRESS>
Currently waiting on a few RPMS (particularly related to X11) to be installed on the OS in order to satisfy packages that require x11-devel.

Thank you -- I've been using niagara and the shared disk location
(/collab1/data) when network access is necessary / helpful.
On Tue, Oct 8, 2024 at 7:04 AM RaghuReddy-NOAA @.***>
wrote:

@rickgrubin-noaa https://github.com/rickgrubin-noaa
Please NOTE: Even though the pre-TDS does not have external connectivity
it is possible to do downloads by setting the following two environment
variables to use the proxy:
export http_proxy=http://<IP_ADDRESS>:3128
export https_proxy=http://<IP_ADDRESS>:3128
Not all download may work but majority of them do with the above setting.
Anything that uses HTTP/HTTPS should work fine. I just wanted to let you
know!
We are still looking at the X11 issue above.
—
Reply to this email directly, view it on GitHub
https://github.com/JCSDA/spack-stack/issues/1297#issuecomment-2399791188,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/BI2WBDNU3E55NFU3O7EVR23Z2PJ5PAVCNFSM6AAAAABODYRJXKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGOJZG44TCMJYHA
.
You are receiving this because you were mentioned.Message ID:
@.***>

--
Rick Grubin
NOAA EPIC
Software Integration

@rickgrubin-noaa Please NOTE: Even though the pre-TDS does not have external connectivity it is possible to do downloads by setting the following two environment variables to use the proxy:
export http_proxy=http://<IP_ADDRESS>:3128
export https_proxy=http://<IP_ADDRESS>:3128

Not all download may work but majority of them do with the above setting. Anything that uses HTTP/HTTPS should work fine. I just wanted to let you know!
We are still looking at the X11 issue above.

thanks for looking into the X11 issue, @RaghuReddy-NOAA. it sounds like it would be helpful if this could be installed at the OS level to keep things consistent with other tier-1 machines and allow us to maintain spack-stack configurations on ursa as we do with the other platforms. please let us know if you will be able to install this into the os for ursa.

Currently blocked / do not have a workaround for the following:
A source mirror exists / is used, and a number of packages that do their work (primarily patch-ing) via the internet have been worked around; don't see or understand a way around this one at the moment:
configs/common/packages.yaml
[...]
py-cryptography:
  require: '+rust_bootstrap'

log.install
[...]
==> rust-bootstrap: Successfully installed rust-bootstrap-1.78.0-fcejx635hwjkgk7mbu56n6yfrjsjk74i
  Stage: 41.27s.  Install: 8.28s.  Post-install: 0.91s.  Total: 50.51s

[...]

==> Installing py-cryptography-38.0.1-fxoomsm4krjabklnvqycdlplevq7656x [284/341]
==> No binary for py-cryptography-38.0.1-fxoomsm4krjabklnvqycdlplevq7656x found: installing from source
[...]
running build_rust
 cargo rustc --lib --message-format=json-render-diagnostics --manifest-path src/rust/Cargo.toml --release -v --features pyo3/abi3-py36 pyo3/extension-module --crate-type cdylib --
   Updating crates.io index
 warning: spurious network error (3 tries remaining): [56] Failure when receiving data from the peer (CONNECT tunnel failed, response 403)
 warning: spurious network error (2 tries remaining): [56] Failure when receiving data from the peer (CONNECT tunnel failed, response 403)
 warning: spurious network error (1 tries remaining): [56] Failure when receiving data from the peer (CONNECT tunnel failed, response 403)
 error: failed to get `asn1` as a dependency of package `cryptography-rust v0.1.0 (/collab1/data/Richard.Grubin/git/EPIC/spack-stack/cache/build_stage/spack-stage-py-cryptography-38.0.1-fxoomsm4krjabklnvqycdlplevq7656x/spack-src/src/rust)`

 Caused by:
  download of config.json failed

 Caused by:
  failed to download from `https://index.crates.io/config.json`

 Caused by:
  [56] Failure when receiving data from the peer (CONNECT tunnel failed, response 403)
=============================DEBUG ASSISTANCE=============================
[...]
   Python: 3.11.7
   platform: Linux-5.14.0-427.24.1.el9_4.x86_64-x86_64-with-glibc2.34
   pip: 23.1.2
   setuptools: 63.4.3
   setuptools_rust: 1.6.0
   rustc: 1.78.0 (9b00956e5 2024-04-29)
   =============================DEBUG ASSISTANCE=============================

packages/py-cryptography/package.py
    [...]
    # To fix https://github.com/spack/spack/issues/29669
    # https://community.home-assistant.io/t/error-failed-building-wheel-for-cryptography/352020/14
    # We use CLI git instead of Cargo's internal git library
    # See reference: https://doc.rust-lang.org/cargo/reference/config.html#netgit-fetch-with-cli
    depends_on("git", type="build", when="@35:")

    def setup_build_environment(self, env):
        if self.spec.satisfies("@35:"):
            env.set("CARGO_NET_GIT_FETCH_WITH_CLI", "true")

@AlexanderRichert-NOAA had to work around the rust install problem on acorn if I remember correctly

@rickgrubin-noaa Please NOTE: Even though the pre-TDS does not have external connectivity it is possible to do downloads by setting the following two environment variables to use the proxy:
export http_proxy=http://<IP_ADDRESS>:3128
export https_proxy=http://<IP_ADDRESS>:3128

Not all download may work but majority of them do with the above setting. Anything that uses HTTP/HTTPS should work fine. I just wanted to let you know!

@RaghuReddy-NOAA could you provide an example whereby export-ing these variables allows some external connectivity?
The particular case that we are facing is that some spack-stack packages have build-time internet access requirements -- fetching something for a build, like a file directly from github, or a rust cargo file -- items that cannot necessarily be downloaded elsewhere, placed in a package's directory, and then modify package.py to look locally for the item(s).
Short of perhaps creating a container that matches ursa's OS / CPU at a minimum, and building various packages into a build cache which can then be downloaded to ursa, being air-gapped -- while understandable -- is a blocker at the moment.

[Richard.Grubin @ ursa] $ hostname
nfe91

[Richard.Grubin @ ursa] $ grep http .bashrc
export http_proxy=http://<IP_ADDRESS>:3128
export https_proxy=http://<IP_ADDRESS>:3128

[Richard.Grubin@nfe01 spack-stack]$ env | grep http
https_proxy=http://<IP_ADDRESS>:3128
http_proxy=http://<IP_ADDRESS>:3128

install log file:
/collab1/data/Richard.Grubin/contrib/spack-stack-rocky9/envs/unified-dev.ursa.oneapi/log.install
[...]
==> Installing py-cryptography-38.0.1-fxoomsm4krjabklnvqycdlplevq7656x [247/314]
==> No binary for py-cryptography-38.0.1-fxoomsm4krjabklnvqycdlplevq7656x found: installing from source
==> Using cached archive: /collab1/data/Richard.Grubin/git/EPIC/spack-stack/cache/source_cache/_source-cache/archive/1d/1db3d807a14931fa317f96435695d9ec386be7b84b618cc61cfa5d08b0ae33d7.tar.gz
==> No patches needed for py-cryptography
==> py-cryptography: Executing phase: 'install'
[...]
  running build_rust
  cargo rustc --lib --message-format=json-render-diagnostics --manifest-path src/rust/Cargo.toml --release -v --features pyo3/extension-module pyo3/abi3-py36 --crate-type cdylib --
      Updating crates.io index
  warning: spurious network error (3 tries remaining): [7] Couldn't connect to server (Failed to connect to index.crates.io port 443 after 446 ms: Couldn't connect to server)
  warning: spurious network error (2 tries remaining): [7] Couldn't connect to server (Failed to connect to index.crates.io port 443 after 71 ms: Couldn't connect to server)
  warning: spurious network error (1 tries remaining): [7] Couldn't connect to server (Failed to connect to index.crates.io port 443 after 70 ms: Couldn't connect to server)
  error: failed to get `asn1` as a dependency of package `cryptography-rust v0.1.0 (/collab1/data/Richard.Grubin/git/EPIC/spack-stack/cache/build_stage/spack-stage-py-cryptography-38.0.1-fxoomsm4krjabklnvqycdlplevq7656x/spack-src/src/rust)`

  Caused by:
    download of config.json failed

  Caused by:
    failed to download from `https://index.crates.io/config.json`

  Caused by:
    [7] Couldn't connect to server (Failed to connect to index.crates.io port 443 after 66 ms: Couldn't connect to server)

This failure occurs because:
spack/var/spack/repos/builtin/packages/py-cryptography/package.py
[...]
    # To fix https://github.com/spack/spack/issues/29669
    # https://community.home-assistant.io/t/error-failed-building-wheel-for-cryptography/352020/14
    # We use CLI git instead of Cargo's internal git library
    # See reference: https://doc.rust-lang.org/cargo/reference/config.html#netgit-fetch-with-cli
    depends_on("git", type="build", when="@35:")

    def setup_build_environment(self, env):
        if self.spec.satisfies("@35:"):
            env.set("CARGO_NET_GIT_FETCH_WITH_CLI", "true")

That is, per The Cargo Book:
CARGO_NET_GIT_FETCH_WITH_CLI — Enables the use of the git executable to fetch
and that use of the "git executable to fetch" fails because ursa is air-gapped and the http/https proxies do not seem to make a difference.
It may be that the proxies will work if I manipulate some settings in ~/.gitconfig, or perhaps by creating a ~/.cargo/config with a proper entry, or perhaps by creating a ssh agent for cargo build. That said, because this particular package requires access to an external source at build-time, it won't build without some other sort of intervention.

@RaghuReddy-NOAA
Any thoughts on how to move this forward?
Manipulations of ~/.gitconfig | ~/.cargo/config | ssh agent for cargo build have not been successful -- same error message with respect to connectivity.

@rickgrubin-noaa I am really sorry, I thought I had added a note this issue, looks like I forgot to hit the final "Comment" button...
In any case, at this point I think it will be best to wait until the actual Ursa-TDS (which will be called Oso) will be available shortly. Of course unlike this trial system (which doen's have external network connectivity) Oso will have the same network connectivity as Hera.
Thank you for looking into this and identifying some of the issues we are running into.  I will let you know when the Oso is ready for testing, which will not be before mid Nov.

@RatkoVasic-NOAA Do we have to push this out to spack-stack-1.10.0 (or a post-install for 1.9.0) given the timeline announced earlier today for Ursa?

@RatkoVasic-NOAA Do we have to push this out to spack-stack-1.10.0 (or a post-install for 1.9.0) given the timeline announced earlier today for Ursa?

We can push out to 1.10.0, but as soon as we get machine we will install 1.9.0 (and maybe even 1.6.0+addons if UFS-WM and SRW are still using that spack--stack).

I had add a note to another issue and just wanted to mention it here too, the Ursa pre-TDS (also known as nfe91) is now available for trying out software installations.  Previously you had to use a proxy to access outside sites, that should no longer be necessary. Just to be clear, there still are firewall rules, and if something is accessible from Hera/Niagara it should be accessible from nfe91.  Please let us know if you run into any issues!


chore(artwork lists): promote "v2" to default implementation

The type of this PR is: Chore
Follow-up to #12938 and #12951
This PR solves ONYX-5
Description
This final bit of cleanup…

Promotes all "v2" nomenclature to default naming
Drops the obsoleted version of ArtworkActionsSaveButton (the heart that appears under the image lightbox on the artwork page)

This is a dreary one to review, feel free to skim the commits! I swear this is the last of the Artwork Lists cleanup 😇

I haven't seen any problems on staging, gonna merge and keep this one moving. (Which closes out this cleanup mini-epic ✨ )


Call initial setupDeclarativeReflexes() on document.readyState === 'complete'

Type of PR
Bug Fix
Description
This improves the reliability when using Importmaps, so that we can be sure that the DOM is setup properly, so the setupDeclarativeReflexes() function can be called in the right order.
Additionally we export the setupDeclarativeReflexes() so users can call this function manually if needed, i.e. if they have a specific load order/setup.
Why should this be added
Improves the reliability when using StimulusReflex with Importmaps.
Thanks to @matheau for reporting this and providing their work-around.
Checklist

[x] My code follows the style guidelines of this project
[x] Checks (StandardRB & Prettier-Standard) are passing
[x] This is not a documentation update

Thank you for following up with good research!


Setup crowdin for internationalization

It would be nice to use a tool like crowdin to manage the translations.

Happy to help if you use Weblate.
Crowdin is a non-starter.

done, @comradekingu closing this issue now, feel free to open a new issue

Just to be clear. The guy downvoting this is a Crowdin Maintainer,
that pretends Crowdin is open source. (It isn't.)


Apps: Allow computed vars to appear in variable coverage table

Variable coverage table in ranked abundance and alpha diversity vizs does not add the computed variables. Note the computed variables are returned in the complete cases table response data, but this info doesn't make it onto the screen.

Possibly related to VEuPathDB/plot.data#164

Does the backend provide this data, and the frontend is simply not rendering it? Regardless, I just want to mention that we want to approach this in a generic way.

Good call. The backend does provide the information (see screenshot: feature mbio, GEMS, ranked abundance, no subset), but i'm guessing there is a check or filter somewhere on the front end to use only those vars in the table that match one of the input variables? I haven't looked into it deeply yet, that's just a guess.

Note that in the response, there is one row for each box in the boxplot. This is a consequence of using a collection-type variable on the x axis.

Please refer to what I temporarily did for alphadiv's variable coverage table before, but it is not a generic way to do
https://github.com/VEuPathDB/web-eda/pull/1018
I forgot a detailed info myself but there are several issues for abundance case, e.g., multiple values in completeCases, no specific info to search for X and Y variable details, etc. I may take a look at it later to remind myself :)

I want to point you both (@asizemore and @moontrip) to https://github.com/VEuPathDB/web-eda/issues/580. Ultimately, I envision a component (currently referred to as StandardVisualization) that would handle this sort of thing. It would help to think about how one would make a generic component that will do what we're discussing here.
I'm working on a prototype of StandardVisualization right now. Hopefully we can look at it soon, and think more about this issue.

Thank you both!
@moontrip thank you for the reminder - I will remind myself what we discussed and think about how we might want to change the response. You brought up some good points and you're right i haven't addressed them yet.
@dmfalke very cool! Really looking forward to the prototype!

@dpbisme assigning to you for qa, but @danicahelb and @SheenaTomko would be curious to know how intuitive you find the table as well. There's a little time to make adjustments before release.

I think the table makes sense. Is it worth adding the method to the name of the X-axis variable in the ranked abundance app? For instance in this image, say that the X-axis variable is "Relative abundance of phylum data; Method median"? In a scatter plot add it to the Overlay variable name? I was playing around with methods and noticed that different methods returned different numbers of categories (on the X-axis in box plots, overlay in scatter plots) and thought having the method there too might help make it more obvious why

This is done. Moving to complete


Enhance FluentAccordionItem with Flexible HeadingContent Parameter

Pull Request
📖 Description

I've updated the FluentAccordionItem component to enhance its flexibility and usability. Previously, attempting to control the heading slot resulted in a RZ2012 error. To address this issue, I've made the following changes:

Removed the EditorRequiredAttribute to avoid the RZ2012 error.
Introduced a new RenderFragment? parameter, HeadingContent, providing an alternative way to specify the heading content. This addition allows users to define custom heading content more flexibly.

🎫 Issues

👩💻 Reviewer Notes

📑 Test Plan

✅ Checklist
General

[x] I have added tests for my changes.
[x] I have tested my changes.
[x] I have updated the project documentation to reflect my changes.
[x] I have read the CONTRIBUTING documentation and followed the standards for this project.

Component-specific

[ ] I have added a new component
[x] I have added Unit Tests for my new compontent
[x] I have modified an existing component
[x] I have validate Unit Tests for an existing component

⏭ Next Steps

Thank you for this useful improvement.
Can you fix the unit tests that no longer work?
And add a test to validate the use of HeadingContent?

Thank you for this useful improvement. Can you fix the unit tests that no longer work? And add a test to validate the use of HeadingContent?

Of course, I'll give a try. 😊

Hi,
I've just pushed some commits related to unit testing. As this is my initial attempt at writing unit tests (bUnit), there might be areas for improvement. I would greatly appreciate your feedback and suggestions to enhance the quality of these tests.
Thank you🚀

I'm looking at the tests now. So far, looking good!
Can you please rename the HeaderContent name to HeaderTemplate. That is more consistent with how we do it in other places in the library. Also, add in the XML comments only one should be used and which one takes precedence if someone does use both.

Sorry, due to the time difference, I went out for dinner, and it took some time. Apologies.😭

No worries!! Hope you had a nice dinner. Thanks for making the initial changes. Getting it aligned to our naming is only a small change.


Update Composer dependencies (2024-06-20-00-13)

Visual regression test passed!
View the visual regression test report


learning content from the learning pathway Microsoft is not available for our users

Good morning,
We have made available the Microsoft

t Learning Pathway site but when users try to view a pathway, they get this error message
Here the error message : the Microsoft Learning path has a configuration problem, ask your administrator for help
Microsoft learning pathway vesion  : <IP_ADDRESS>
release date : July 2022

Hi @Ang2022,
Thank you for reporting this issue here. This error is generally caused by a few things.

The installation wasn't done with an administrative account so the tenant app property didn't get set properly.
The site was renamed and the tenant app property wasn't changed to go with it.
Permissions to the config list have been changed.

Can you please validate these and let me know if you are still seeing the error message?
Additionally, can you provide some more information about whether this issue is happening for all users or guest/external users. If you attach the browser console logs, that would be helpful for us to figure out more details about the error. Thanks!

Hello V-Pringunasek,
thank you for your answer, concerning the causes that you describes
Error.docx

The installation was done by an administrative account tenant
No, the sharepoint site has not been renamed and neither the tenant
Concerning your last point : we don't know what is this Config list and where is it ? and what we have to do about ? can you help us with more information

Only the administrative account tenant ( the account that we used to create the site can watch the pathways )
We tested to pass some internal users as site admin and after as visitor and as editor and we have always the same error (all people have access to the site but not at the pathways)
Thank you so much for your help ( we expend a lot of time triging to fixe this issue and we don't know what more to do.

Have you tried the DiagnosisPreCheck first?
https://github.com/pnp/custom-learning-office-365/blob/main/installation/DiagnosisPreCheck.md
Please follow the instructions there.

Hi @Ang2022,
Would like to know whether you are able to run the DiagnosisPreCheck ?
Thanks

Closed due to inactivity. If this is still an issue please create a new issue and reference this one.

Hi @dcashpeterson
Sorry for my late answer,
We have always the same error after the diagnostic and we don't know how to fix it.
Please find here the screenshoot.

Could you help us ? thank you in advance.

@Ang2022 from the errors you can see that the user in question doesn't have access to the configuration lists. So either the config lists don't exist in the site collection that the app thinks they should synergies365 or the current user doesn't have access to those lists. All users need read/write access to the config lists and we break inheritance when those lists are created.

Hello @dcashpeterson thank you for your answer.
We dont know where are those configuration lists
could you tell me where we can give access to the configuration list to users ?
Thank you

The webpart is looking for the config lists in the Synergies365 site collection.

I've encountered the exact same error message as Ang2022.  I'm accessing the learning pathway via Airforce DAF365 Hub.  Will you be correcting the Microsoft 365 learning pathway soon.. i'm trying to find some old MS Teams New User training.  Thank you!

@ralphberenics I'm not sure what the issue you are experiencing. As you can see from the responses above the issue that @Ang2022 reported is that when the webpart loads it can't find the lists either based on permissions or that the lists are not there. Based on what you are showing me I can't assume that this is the same issue. That is a generic error. Are you able to see errors in the console? We can't really assist based on what you are showing in your screenshot. It also looks like your tenant might be experiencing a service outage as well.

Hello @ralphberenics I'm looking in the site collection but I dont know what to do ? where I have to click to fix the problem.
you said : The webpart is looking for the config lists in the Synergies365 site collection...in this case, where I have to click and what I have to do ? sorry I'm lost.
Im looking in the collection site where I shared with you my screenshot and the next steps are ?

@Ang2022 The configuration lists are there CustomAssets, CustomConfig, and CustomPlaylists. CustomAssets and CustomPlaylists inherit permissions from the site. CustomConfig needs to have Members - Edit, Owners - Full Control, Visitors - Contribute. Permissions for the site should be like this Members - Edit, Owners - Full, Visitors - Read.
It looks like the lists exist in the site collection you are showing in the previous screen shot. What site collection is that?

Hello @dcashpeterson THANK YOU SO MUCH !!
We finnally undertood the problem and finally that is fixed :) our users can use the site.
thanks, thanks :)

hi @dcashpeterson and @Ang2022 ,
I am from Ecuador, I have the learning pathway installed but it is generating this error when accessing, it worked normally, suddenly this error appears, can you help me please,
they do not load the playlists and do not allow the site administrator to enter
Please can you help me!!

@jeancapi If you could please open another issue, follow the template and provide all the necessary information that would be great. That is a generic warning/error so you should run through the process in the issue template to give us more information so we can better assist.

thank you very much, i just opened it https://github.com/pnp/custom-learning-office-365/issues/694


Similar projects page to be shown while adding a new project.

Scored similarity between two projects on basic string matching.
UI needs further discussion and improvements. I've checked and this source should work without breaking any existing functionality.
Comments are welcome!
Screenshot when some relevant projects are found :

The score calculation is fast but blunt. Much can be improved in future.
When there are no similar projects, skipping the similar projects page and showing a toast in the approve project page:

This is great! If @DevMattM  likes it too he can merge away.

@PFWhite  can you approve your requested changes?

@DevMattM Committed changes as suggested. Kindly review.

Looks good. I will follow up with Kevin or Chris for the right text on the page, but this is a great first step.


Custom Body Type or response_serializer for incrementally streaming large amounts of data back to the client

Question
I'm using Beast 1.81 with Clang 13.0.0 to present a distributed data management API over HTTP.
I've been reading a lot of the documentation and it's not clear to me what the recommended solution is for streaming large amounts of data back to the client. The pages that look like they are close to what I want are:

https://www.boost.org/doc/libs/1_81_0/libs/beast/doc/html/beast/using_http/serializer_stream_operations.html
https://www.boost.org/doc/libs/1_81_0/libs/beast/doc/html/beast/using_http/custom_body_types.html

At the moment, my application presents something similar to POSIX read. That is, it forces the client to make multiple read requests to the HTTP server. This isn't ideal, but is good enough for a first pass (I'm still getting familiar with Beast).
You can view the code in question here:

https://github.com/irods/irods_client_http_api/blob/38d67aa5ab798952450dc4429c20d82586a375e8/src/endpoints/data_objects/impl.cpp#L417-L420

The application has a background thread pool for long running operations. I want to use that to incrementally stream the data back.
Q. Does a custom body type make sense here? Or would a response_serializer be better?
Q. Is transfer-encoding: chunked automatically supported by custom body types?
Q. Are there examples which demonstrate how to do this asynchronously?
BTW, thanks for the awesome library.

You can utilize a http::buffer_body to send the response part by part. This example might help you: https://www.boost.org/doc/libs/1_84_0/libs/beast/doc/html/beast/more_examples/send_child_process_output.html.
Alternatively, you can create a custom body type that reads from a stream into a fixed-size buffer and returns the corresponding buffers until it reaches the end of the stream.

Thanks. I'll give that a try.

The send child process output example worked.
Perhaps adding an async example that demonstrates the same thing would be helpful to others. Mostly to show how to manage the lifetime of the response_serializer across async calls.


check MSE first, then fallback to native HLS

closes #39
Changes logic to the following

check for MSE support first (Hls.isSupported())
then fallback to check for native HLS support

Unfortunately, the logic we had before of checking native HLS support first is leading to a bug in Android.
For some reason, on Android, when setting this.el.src = "....m3u8" it throws a CODE4 error. Note that this only happens with videojs. If we strip out videojs all together and set src="....m3u8" on a <video> element then the bug doesn't happen. So it has something to do with the videojs stack throwing this error specifically on Android and I couldn't get to the bottom of it.
Note that on Safari setting  this.el.src = "....m3u8" w/ videojs did not throw the error, so there's really something pernicious going on.
Change in underlying behavior:

Android: it was broken ❌  now it works ✅
Safari (iOS) / Chrome (iOS) / any iOS: no change (was previously using native HLS, and still is)
Safari (Desktop): change: was previously using native HLS, now is using MSE
Chrome (Desktop): no change (was previously using MSE, and now still is)
Firefox (Desktop): no change (was previously using MSE, and now still is)

The only actual change is in Safari Desktop and I think we can live with that.

thank you!


Layer control places all pmtiles layers a single layer control

Thanks for an amazing package, this has been really useful to me!
One small issue I have encountered is that while we can define multiple layers in a single pmtiles style block,
 style={
        "layers": [
            {
                "source-layer": "landuse",
                "type": "fill",
                "paint": {"fill-color": "steelblue"},
            },
            {
                "source-layer": "roads",
                "type": "line",
                "paint": {"line-color": "black"},
            },
        ],
    },

when we use the layer control tool to add them to the toggle menu, the two are grouped together as a single PMTilesVector layer -- we can't toggle off roads individually, say.  I know we can hack around this by making two completely separate PMTiles layers, but that seems at odds with the syntax above and also creates issues with the tooltip only applying to the most recently added layer.
(aside, though the examples don't show this use case, I was pleasantly surprised to realize we can color features by a data column as well, as described in https://maplibre.org/maplibre-gl-js/docs/examples/data-driven-lines/).
Thanks again for an amazing tool here.

Thanks for the suggestion. I'll look into this and see if this is possible!

I looked into this and as of now it will take a significant effort to have individual maplibre layers be toggleble. I have some ideas how to get it to work but best not to expect anything concrete anytime soon.


Filter participations without submissions + add submission count to participations

Checklist

[x] I tested all changes and all related features with different users (student, tutor, instructor, admin) on the test server https://artemistest.ase.in.tum.de.
[x] Server: I implemented the changes with a good performance and prevented too many database calls
[x] Server: I documented the Java code using JavaDoc style.
[x] Client: I documented the TypeScript code using JSDoc style.
[x] Client: I added multiple screenshots/screencasts of my UI changes
[x] Client: I translated all the newly inserted strings into German and English

Motivation and Context
At the moment, it is possible to filter participations to find those with failed submissions (without result). With this PR, participations can also be filtered to find those with no submissions. Additionally, a new column "Submissions" shows how many submissions a participation has (as it is already the case on the exercise scores page).
Description

Move transient property submissionCount from Result to Participation (submissionCount on Result is only used in one occurence and there the participation is also available. Also conceptually, it makes more sense that a Participation knows this information instead of a Result.)
Add  new query to the StudentParticipationRepository that counts the submissions per participation for an exercise and add a new service method that turns it into a mapping from participation id to submission count
Use this mapping in the REST controller for /exercise/{exerciseId}/participations and set the transient property submissionCount accordingly
In frontend, add the new filter radio button for participations without submissions and add a new column "Submissions" that shows the submission count and links to the overview of these submissions
For consistency, also link the submission count on the Exercise Scores page to the submission overview page
Small design improvement: Fix unnecessarily wide columns in participations table that would require horizontal scrolling even for medium-large screen sizes

Steps for Testing

Log in to Artemis
Navigate to Course Administration
Click on Exercises for a course
Click on Participations for an exercise
Check that new filter option "Show No Submissions" below headline exists and works as one would expect
Check that new column "Submissions" exists in table, that the value is correct, that sorting by it works and that it is linked to view these submissions

Screenshots

2 small code improvements, then we can merge
sorry for the inconsistent feedback, aber Ober schlägt Unter ;-)

I worked in the two points.


scip-go: 0.1.21 -> 0.1.22

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://github.com/sourcegraph/scip-go/releases.
meta.description for scip-go is: SCIP (SCIP Code Intelligence Protocol) indexer for Golang
meta.homepage for scip-go is: https://github.com/sourcegraph/scip-go/tree/v0.1.22
Updates performed

Golang update

To inspect upstream changes

Release on GitHub

Compare changes on GitHub

Impact
Checks done

built on NixOS
The tests defined in passthru.tests, if any, passed
found 0.1.22 with grep in /nix/store/gcng1fdxxsm1a199c7sxy3qbqcmb818m-scip-go-0.1.22
found 0.1.22 in filename of file in /nix/store/gcng1fdxxsm1a199c7sxy3qbqcmb818m-scip-go-0.1.22

Rebuild report (if merged into master) (click to expand)

2 total rebuild path(s)

2 package rebuild(s)

First fifty rebuilds by attrpath

scip-go

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/gcng1fdxxsm1a199c7sxy3qbqcmb818m-scip-go-0.1.22 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A scip-go https://github.com/r-ryantm/nixpkgs/archive/3f275c75b831cb0c4ed0e6f863361adb4df83658.tar.gz

Or:
nix build github:r-ryantm/nixpkgs/3f275c75b831cb0c4ed0e6f863361adb4df83658#scip-go

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/gcng1fdxxsm1a199c7sxy3qbqcmb818m-scip-go-0.1.22
ls -la /nix/store/gcng1fdxxsm1a199c7sxy3qbqcmb818m-scip-go-0.1.22/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
nixpkgs-review result
Generated using nixpkgs-review.
Command: nixpkgs-review

x86_64-linux

:white_check_mark: 1 package built:

scip-go

Maintainer pings
cc @arikgrahl for testing.

[!TIP]
As a maintainer, if your package is located under pkgs/by-name/*, you can comment @NixOS/nixpkgs-merge-bot merge to automatically merge this update using the nixpkgs-merge-bot.

Add a :+1: reaction to pull requests you find important.

@NixOS/nixpkgs-merge-bot merge

@NixOS/nixpkgs-merge-bot merge


Clean up uriutils, make it consistent, support py3, add tests

Included in this change:

Clean up the uriutils API, make it consistent across python2 and 3.
Support python3 in general.
Add tests for python2 and 3 for uriutils and basic API requests.

thanks!


fix: e2e cases error

Signed-off-by: liuying<EMAIL_ADDRESS>
What type of PR is this?
/kind bug

What this PR does / why we need it:
Which issue(s) this PR fixes:

Fixes #
Special notes for reviewers:

Does this PR introduced a user-facing change?

Additional documentation, usage docs, etc.:

Codecov Report

Merging #323 (3cbc12f) into release-1.3 (c159999) will not change coverage.
The diff coverage is n/a.

@@             Coverage Diff              @@
##           release-1.3     #323   +/-   ##
============================================
  Coverage        47.35%   47.35%
============================================
  Files                7        7
  Lines              606      606
  Branches           171      171
============================================
  Hits               287      287
  Misses             246      246
  Partials            73       73


Use test_random() in hashtable mt test for better reproducibility

Although due to to the multithreading, it won't be perfect, it might help.

what issue is this addressing?

what issue is this addressing?

I am trying to reproduce this failure:
https://github.com/openssl/openssl/actions/runs/11292741961/job/31409362706

This pull request is ready to merge

Merged to the master and 3.4 branches. Thank you for the reviews.


Safari显示连接中

Describe the bug
家里两台移动设备，iPad Air 5和iPhone 13，都一样的现象
网络环境，正常wifi链接，连接透明网关负责科学上网
现象：点击录音按键，一直显示“等待中”或“连接中”，无其他错误
Steps to reproduce

Open SpeechGPT on Safari
Fill in token for OpenAI & Azure
Click "Record"

Screenshots or additional context

same here, looks like mobile browsers are not supported

Please ensure you have enabled the 2 APIs below in Safsari's Experimental WebKit Features setting. In my case, the problem was solved after enabling these 2 APIs.

AudioSessionAPI
AudioSessionfullAPI

You may want to look at: How to Enable Experimental Features in Safari Mac and iPhone? – WebNots

This may also be due to network timeout and will be fixed in the future

How to Enable Experimental Features in Safari Mac and iPhone? – WebNots

Thanks for the pointer, but I can't find these 2 features on my iPhone(iOS 16.2). I also checked it on Mac Safari, no these AudioSession API options.

This may also be due to network timeout and will be fixed in the future

This could be the case. I can saw HTTP request to Azure is redirected through my gateway but nothing comes back, the connection just stalls there.

I run into the same problem when I want to use speecgpt on the iPhone for Edge

遇到同样的问题

I can use it with my Mac (No VPN), but I cannot use it with Safari or Chrome on my phone (The same Wifi).

The function "chat" and "Synthesis" is working.
The function "Recognition" is not working. And it is always showing "connecting".

我发现解决方案了。可以使用手机浏览器如 夸克浏览器。或者电脑上使用chrome 浏览器。

出现问题的原因是，浏览器无法调取 语音服务，而代码中没有针对各种可能出现无法访问语音服务时的提醒。所以需要一个可以自动弹出，设置打开麦克风权限的浏览器。比如 Chrome 浏览器，或者手机上的 夸克浏览器。


Query around n_samples argument

Hi, I am performing code generations using the following command
accelerate launch  main.py --model bigcode/santacoder --tasks humaneval --max_length_generation 256 \
--temperature 0.8 --top_p 0.95 --do_sample True --generation_only --n_samples 100 --batch_size 32 \
--output_generations generations/santacoder_temperature_0.8_top_p_0.95_task_humaneval.json \
--save_generations --allow_code_execution --trust_remote_code

I am expecting the number of candidate generations per task to be around 100. However, on inspecting the generations/santacoder_temperature_0.8_top_p_0.95_task_humaneval.json file I see that there are 96 generations per task.
Is there something I am missing? Thanks

Hi, can you share the number of GPUs you did the execution on and if you got any warnings during the process?

We are using 1 A100 80GB GOU for the inference. The accelerate config used is specified below
$ accelerate config
In which compute environment are you running? ([0] This machine, [1] AWS (Amazon SageMaker)): 0
Which type of machine are you using? ([0] No distributed training, [1] multi-CPU, [2] multi-GPU, [3] TPU [4] MPS): 2
How many different machines will you use (use more than 1 for multi-node training)? [1]: 1
Do you want to use DeepSpeed? [yes/NO]: NO
Do you want to use FullyShardedDataParallel? [yes/NO]: NO
How many GPU(s) should be used for distributed training? [1]:1
What GPU(s) (by id) should be used for training on this machine as a comma-seperated list? [all]:all
Do you wish to use FP16 or BF16 (mixed precision)? [NO/fp16/bf16]: bf16

This is the log message when I run the command
$ accelerate launch  main.py --model bigcode/santacoder --tasks humaneval --max_length_generation 256 \
--temperature 0.8 --top_p 0.95 --do_sample True --generation_only --n_samples 100 --batch_size 32 \
--output_generations new_generations/santacoder_temperature_0.8_top_p_0.95_task_humaneval.json \
--save_generations --allow_code_execution --trust_remote_code

The following values were not passed to `accelerate launch` and had defaults used instead:
        `--dynamo_backend` was set to a value of `'no'`
To avoid this warning pass in values for each of the problematic parameters or run `accelerate config`.

Selected Tasks: ['humaneval']
Loading the model and tokenizer
generation mode only
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  6.09it/s]
Generating solutions for task 164
number of problems for this task is 164

100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 492/492 [25:02<00:00,  3.05s/it]
generations were saved

Thanks, I will go ahead with n_samples = 100 and batch size = 100.

Hi, this setting worked for Santacoder model. I am performing similar experiment with Codegen-2B model. I am unable to use batch_size of 100 gives me CUDA OOM error. If I need to get 100 samples generated, what should be the value for batch_size and n_samples?

Hi, as I said nsamples just needs to be proportional to batch size. For example for nsamples 100, you can use a batch size of 2, 5, 10, 20, 25…


Fix issue with extraction of negative integers

Hi, this fixes issue 3, by adding signed=True to int.from_bytes in the integer extractors, and provides some test cases to cover negative integers.

Thank you for this!


Add POSTGRES_INIT_START to control initialization

This allows users to disable the initial postgres run which creates the
default database and user.  This allows users to run scripts which
cannot have postgres running, e.g., pg_basebackup restore.

Easier to review this when ignoring whitespace due to indentation change.

Closing old issue


fix (app): dark mode bug

This pull request is intended to solve issue #78
Closes #78

@selemondev
Thanks for PR
After testing shadcn-vue in fresh Nuxt project found out it's about tailwind.config['content']
You should include tailwind.css file in tailwind content

nuxt.config

with @nuxtjs/tailwindcss you can extend or override tailwind content,
Also, it has predefined paths for important Nuxt directories like layouts, pages, composables, plugins, utils, ...
// https://nuxt.com/docs/api/configuration/nuxt-config
export default defineNuxtConfig({
  modules: ['@nuxtjs/tailwindcss', '@vueuse/nuxt'],
  devtools: { enabled: true },
  components: [
    {
      path: '~/components/ui',
      // this is required else Nuxt will autoImport `.ts` file
      extensions: ['.vue'],
      // prefix for your components, eg: UiButton
      prefix: 'Ui'
    },
  ],
  tailwindcss: {
    config: {
      content(content) {
        return [
          ...content, // https://github.com/nuxt-modules/tailwindcss/blob/main/src/resolvers.ts#L25C1-L35C3
          './assets/css/tailwind.css',
        ]
      },
    },
  }
})

https://github.com/radix-vue/shadcn-vue/assets/17789047/8e6be759-f639-43fd-8de6-ba7646b54416

vite

Same as Nuxt but you have to extend and include tailwind.css file in your tailwind.config content

This should be documented I'll create PR for Vite and Nuxt sections

@sadeghbarati My pull request solves the Vite-Vue issue described by @ezekel on issue #119 , whereby the dark mode doesn't work until you place a word.

I know but we should follow the same structure as shadcn-ui


Miscellaneous HD44780 auxdisplay driver enhancements

This PR contains some hd44780 driver fixes and enhancements. Highlights:

riddance of excessive delays while waiting for some hd44780 command to finish (the driver now conforms to the hd44780 manual),
implementation of busy-flag polling mechanism (available only if RW pin is specified in DT), therefore allowing for faster completion of hd44780 commands as seen from MCU perspective.

@kuliga It looks like there are still some comments that needs to be applied? Please let us know when this is ready to be re-reviewed. Thanks!


fusedoc-4499-m minor fix to tmplt and redhat image update

This is the same update as was done in PR 192
minor typo fix in kamelet template
updated to ui-bundle.zip to change camel image to red hat images

2nd commit includes red hat branding changes to config.toml file


Partial support modules disappearing when row or section is duplicate.

Problem Description
I have built some simple modules with partial support and come across two problems.
The first one it is that if I duplicate the row or section where the modules are, the modules are not reloaded so I need to make a change on any module field to be able to see it again.
The second it is that if I have more that one of those modules on one column and then I duplicate the row or section. Then the modules are missing in the duplicate row or section.
Screenshot and/or gif
https://www.dropbox.com/s/rij8t4c0dxdjbjf/Modules missing on duplicate row or section.mp4?dl=0

QA Checklist
Reporter

[ ] Helpful title used

Helpful title format: Scope(s) :: One Sentence To Describe The Issue
Example: VB :: Floating Toolbar :: Jumps To Top Of Page

[ ] Detailed description

Description should usually be 150+ characters.

[ ] Steps to reproduce included
[ ] Support Ticket URL included
[x] Image and/or gif

Developer

[ ] Steps to reproduce proven solved
[ ] Test included and coverage acceptable and all Tests passing?
[ ] PR description has layman's description of problem and solution and "Areas Affected" listed/explained?
[ ] PR branch includes latest commits from master branch (at the time of checking)?
[ ] Estimated points on issue
[ ] Proper code documentation and conventions?
[ ] Proper RTL appearance
[ ] i18n used if applicable
[ ] Ajax/Form used? Nonce used? Perms checked? All user input sanitized and validated? Intentional unsanitized/unescaped functions used?
[ ] Database saved/updated? Data sanitized and validated?
[ ] Variable output properly escaped and validated?

Peer Review

[ ] PR branch includes latest commits from master branch (at the time of checking)?
[ ] Steps to reproduce proven solved
[ ] Affected areas are tested and ok
[ ] Proper code documentation and conventions
[ ] Proper RTL appearance
[ ] i18n used if applicable
[ ] Test(s) included and coverage acceptable
[ ] Ajax/Form used? Nonce used? Perms checked? All user input sanitized and validated? Intentional unsanitized/unescaped functions used?
[ ] Database saved/updated? Data sanitized and validated?
[ ] Variable output properly escaped and validated?

Quality Assurance

[ ] All tests passed on CircleCI.
[ ] Steps to reproduce passed, the fix works.
[ ] No errors in console or output
[ ] Affected areas are tested and ok
[ ] The fix does not change a customer's website in any surprising or unintended way

[ ] If Divi's design is adjusted, settings are migrated gracefully and invisibly

Design Review

[ ] UI is to spec
[ ] UX is to spec
[ ] Design is to spec
[ ] No surprising visual changes to a customer's website will be experienced after updating

[ ] Any/all visual changes are migrated gracefully

Dev Review

[ ] Test(s) included and coverage acceptable
[ ] Ajax/Form used? Nonce used? Perms checked? All user input sanitized and validated? Intentional unsanitized/unescaped functions used?
[ ] Database saved/updated? Data sanitized and validated?
[ ] Variable output properly escaped and validated?
[ ] i18n used if applicable
[ ] Implementation acceptable, DRY, Scaleable, future safe
[ ] Best practices used, cleverness and grokage, ES6 use, etc.

CSS also not working on partial support because default wrapper div is not creating itself.


You to say that is a perfect library!

You to say that is a perfect library!

Did you get any issue with this library? Please share.


add a cataloger for binaries built with rust-audit

Fixes https://github.com/anchore/syft/issues/1108
https://github.com/Shnatsel/rust-audit is a tool for embedding Rust dependency information into binaries for subsequent scanning/detection.
This PR adds support for detecting dependencies in binaries created with rust-audit. Per https://github.com/anchore/syft/issues/1108 it's not enabled by default. (A motivation for syft integration is to get greater experience with rust-audit with a view to adding it to Rust itself).
The new cataloger is similar to the golang binary one - I modularized some of the golang binary cataloger code. Detection/parsing of the dependency information is done via https://github.com/microsoft/go-rustaudit, a go library for extracting Rust audit dependency information.

Thank you so much for the PR @tofay! I'll take a look this morning and see if we can get this into the next syft release.

PR looks good from a functionality perspective -
I just had a question when running it against binaries in the wild.
Is this the constraint that this cataloger will only work for static assets built with https://github.com/Shnatsel/rust-audit and with auditable_build::collect_dependency_list(); in a build script?
The integration test you added passed, but when I rant his against other binaries I found no packages being detected.
If this is the case can we rename the file binary_cataloger ==> audit_binary_cataloger just for keep it clear.
Thanks for updating the docs as well that points to this condition so we're not just advertising we can do static analysis for any binary!
I also noticed Getting some real-world experience with this before committing to a stable data format as a part of their roadmap for moving this into cargo.
Are there any competing projects or other contenders that we might have to flip to in the future?

cc @wagoodman for a 2nd review here

Thanks for looking at this. I appreciate it's not as straightforward as "add support for widely used language ecosystem X" type PRs :)

Is this the constraint that this cataloger will only work for static assets built with https://github.com/Shnatsel/rust-audit and with auditable_build::collect_dependency_list(); in a build script?

Correct. Would it help set expectations for this cataloger if the cataloger name was prefixed with experimental-? I can change the file name as you suggest.

I also noticed Getting some real-world experience with this before committing to a stable data format as a part of their roadmap for moving this into cargo. Are there any competing projects or other contenders that we might have to flip to in the future?

There aren't other competing projects that I'm aware of. In terms of data formats - I'm looking into language-agnostic options to embed dependency information into binaries. e.g https://github.com/ossf/wg-vulnerability-disclosures/issues/76 and https://github.com/spdx/spdx-spec/issues/739

Cool! No need for the experimental name change unless @wagoodman has opinions there.
I'm happy with its current state since it's not in the default path and is something we can toggle/experiment with. Approved and thanks again so much for the contribution.


Blank screen on Nexus 5 devices

This could be a duplication of #37

From the slack conversation:

The above image seems to confirm that this is indeed a duplication of #37

This issue has been confirmed on at least 3 different Nexus 5. There is one reported Nexus 5 where the issue is not happening. Trying to get the chrome versions.
The browsers that seem to be failing so far are Chrome and the reported versions are 58.0.3029 and 57.0.2987.132.
The browser where seems to be working is also Chrome, running version 58.0.3029.83

This might be related: https://github.com/ember-cli/ember-cli/issues/5040

When viewing the HTML source of the page (in the repo does not appear), we find two cases of the integrity attribute being used:
    <script src="/assets/vendor-35aea396574469edfba68e595608c708.js" integrity="sha256-Z8Q3cTAE9fMvOdI+mFuWQlnf+DKfHqGo/tTwuO0dxCA= sha512-4i7Vtkvgykgag59sLoIzYyqzi7xfl9hGCzirotVtg6VuyYbV1BCF1QHKRkzHHA8eWYwIS2kmXihlUmrJ0yjY4Q==" ></script>
    <script src="/assets/vlc-tech-hub-62c894c7b9bd0eec8f0f81fbfd5fc5f2.js" integrity="sha256-2x0+kxCOHDegTJft9k8MpdyoWw94OEGS4jdg//ny4t0= sha512-Y76iUhjP3I7NE3rxpOpXrxTqHk4E366188PVrhEY9y/xQSRNW7ksPwXYaJp7lK2aGujC5DA1SOGnGSEAzlngqA==" ></script>

It's an issue with some versions of Chrome and some settings apparently
https://github.com/facebookincubator/create-react-app/issues/1231#issuecomment-266233702
It seems we could force to serve vlctechhub.org through https

Solved serving website in https


how can i set a default position for an element?

how can i set a default position for an element?

You can set left and top in CSS for the element. See demo on CodePen

when I set left and top, the position data of element starts from 0, 0..
Also I have another problem.
could you check this http://onungibiol.com/ ?
when I move the box which inside of the image frame, the box goes to top of the window. and I move again, it comes back to frame. it's strange..


Wrong text on "Standardize Times with the HTML5 datetime Attribute"

The sidebar text on "Standardize Times with the HTML5 datetime Attribute" says "The datetime attribute is an inline element that can wrap a date or time on a page."
https://www.freecodecamp.org/learn/responsive-web-design/applied-accessibility/standardize-times-with-the-html5-datetime-attribute
The attribute cannot be an inline element. Change it to: "The time element is an inline element that can wrap a date or time on a page. " or "The time is an inline element that can wrap a date or time on a page.".

Hey @markosankovic good spot there I'll sort this 👍


[BUG] JSQLParser Version 4.6 : SQL Server : parsing takes too long

Failing SQL Feature:
It takes too long to parse a query with multiple nested subqueries and/or joins with subqueries since 4.6
Can confirm, that 4.5 doesn't have this issue.
SQL Example:
SELECT (SELECT '1.0' AS '@version', (SELECT 'Test' AS 'name', (SELECT (SELECT DISTINCT 51.64315 AS '@lat', 14.31709 AS '@lon' FOR XML PATH('trkpt'), TYPE) FOR XML PATH(''), ROOT('trkseg'), TYPE) FOR XML PATH('trk'), TYPE) FOR XML PATH('gpx'), TYPE) FOR XML PATH('')
Originally posted here https://github.com/dbeaver/dbeaver/issues/20060
Software Information:

JSqlParser 4.6
MS SQL Server, but probably not only (we have an issue about PostgreSQL too, but without example)

Greetings!
I will look into this and return with a solution soonest.

@E1izabeth:
A first brief analysis reveals 2 challenges:

This particular clause FOR XML PATH(...) is not understood/supported by JSQLParser. Please see the latest supported syntax here.

Despite that, of course, the Parser should not freeze, but time-out gracefully.
This happens at least with the latest Version JSQLParser 4.7 SNAPSHOT, you can try it here online

As you can see, parsing times out after 6 seconds (which is configurable as explained here.
I will appreciate, if your team confirms from your end first, that latest Snapshot would time-out correctly with any given time-out period. (Otherwise I would need to help you getting the call implemented correctly).
In parallel, I would extend the grammar for support the clause.

Thank you. I'll write you back, when I try snapshot.

Confirm that 4.7 SNAPSHOT  doesn't have described issue with timeout on parse error. Thanks for your time!
Should change the title of the issue then as the main problem has been already solved?

Thank you for the feedback!
Please leave it open so I can add support for the FOR ... clause (which seems to be Microsoft specific.
The 4.7 Snapshot will yield in Version 5.0 which is a major breaking release and you may want to run some tests before adoption. I will send your team a migration guide soonest.
In the meantime, a JSQLParser 4.6.1 may be needed with a back-port of this particular freezing fix. I will see what I can do about that.

Greetings.
I have implemented the T_SQL FOR ... clause and you can test it here.

Seems like it can be parsed now. Thank you!


Use subprocess.check_call(...) because if the child process fails, it breaks docker build

If the docker build process continues despite a build error, the resulting docker image is broken.
When used in a pipeline, it's not obvious the problem is due to a model compilation error (often).

ping @carlos-salinas


Fix code sample syntax error

Fix code sample error on this article https://developers.google.com/web/updates/2017/04/headless-chrome

@jpmedley found another syntax error


How to draw the Recall-IoU curve?

I have tried to draw the Precision-Recall curve in pascal_voc.py like below:
    pl.plot(rec, prec, lw=lw, label = cls)
    pl.xlabel('Recall')
    pl.ylabel('Precision')
    plt.grid(True)
    pl.ylim([0.0, 1.05])
    pl.xlim([0.0, 1.0])
    pl.title('Precision-Recall')
    pl.legend(loc="upper right")
    plt.show()

This is based on this part of the code which has the variables rec, prec, and ap.
How can I draw the Recall-IoU curve and Recall-NumberOfProposals curve like below? Where can I get the IoU  and Number of proposals values?

thanks

What the location of your code added in pascal_voc.py? In this "for" loop or others？

BTW，whether you used the scikit-learn for these code？

where can i find the IOU value as is shown in the figure @hadi-ghnd

I am using the function below to calculate the recall-iou curve.
I based on: kitti_proposal_eval
function [overlap, recall, AR] = compute_average_recall(iou)
    all_overlaps= sort(iou(:), 'ascend');
    num_pos = numel(all_overlaps);
    dx = 0.001;

    overlap = 0:dx:1;
    overlap(end) = 1;
    recall = zeros(length(overlap), 1);
    for i = 1:length(overlap)
      recall(i) = sum(all_overlaps >= overlap(i)) / num_pos;
    end

    good_recall = recall(overlap > 0.5);
    AR = 2 * dx * trapz(good_recall);
end

% example
[overlaps, recall, ar] = compute_average_recall(iou);
figure;
plot(overlaps, recall, 'LineWidth', 2);
title(sprintf('AR %.4f', ar), 'FontSize', 16);


MTR enables concurrent detection; PTR_LOOKUP is True does not affect …

MTR enables concurrent detection;
PTR_LOOKUP is True does not affect the detection of each hop;
repairs the panic of nil point reference generated when RING_BUFFER_SIZE = 0;

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Put AriaNG e to separate repository

Motivation
Hi Alex.
Your enhanced AriaNG version is awesome. If you could bring us this version in separate repository it will be super useful. I want to use this in my docker image so would be super convenient to have simple github repository with release (i.e. zip file) as single archive. Maybe it helps in development process also. Please consider that. Thanks

It's our core proprietary right now. Maybe I will public the repo in the future. ( Users > 1M ？😀)


Forums supported?

Hi there,
are forums threads supported?
I always get the Error: BOT DOES NOT HAVE PERMISSION TO READ MESSAGE HISTORY!!!
The bot does have permission to read the history.
If i try on an other channel with threads, it does work.
Thanks for the cli.

@Jopo-JP If you join the Discord @ https://discord.gg/6Z6FJZVaDV and see the General channel, I recently sent some executables for the last commit where I think I fixed/supported threads & forums properly. I haven't pushed a release version since I haven't had confirmation from someone else that it's performing as expected.


Reverted the use of the target Parcelable's classloader for the generated classes' classloader

works to fix problems seen in #196

Speaking with @ansman, I think this need a little more research.  We changed this recently (#181) to handle android.net.Uri.


Timezone option not working

So I am trying to adopt node-cron, but I don't think the timezone setting is working, or maybe I am doing something wrong. I am based in Copehagen, Denmark i.e. timezone Europe/Copenhagen.
Consider the following code, to reproduce:
import * as cron from 'node-cron';

const getCurrentHour = new Date().getHours();
const oneMinuteInTheFuture = (new Date().getMinutes() === 60) ? 0 : new Date().getMinutes() + 1;

const pattern = `${oneMinuteInTheFuture} ${getCurrentHour} * * *`;
console.log(pattern);

const b = cron.schedule(pattern, () => {
        console.log('Timezone running');
    }, {
        timezone: 'Europe/Copenhagen',
        scheduled: false,
    });

const a = cron.schedule(pattern, () => {
    console.log('no-timezone running!');
}, {scheduled: false});

const start = () => {
    b.start();
    a.start();
};

start();

If you run this, and wait one minute, the output is:
no-timezone running!

And not as expected output:
Timezone running
no-timezone running!

Is this because the timezone option, only works as I expect if the computer is running UTC time or what is the explanation?

Hello @dauledk
By default, node-cron uses the system time. So in the fist example, node-cron has used the Europe/Copenhagen. In the second example, node-cron has used your machine timezone.

Hello @dauledk I've closed the issue for now, if you have some problem let me know.

Hi, node-cron is not working for a few of the specific timezones. Do we have any solution to run cron for all the timezones
?

This is the code of my cronjob, this is my time zone "Asia/Ho_Chi_Minh", i expected it to match to my machine time but it doesn't, in fact it is later than 8 hours. Am i doing anything wrong or it does not support Viet Name timezone?


HAVE_OPENSSL_STRDUP broken

OPENSSL_strdup is a macro around CRYPTO_strdup, so AC_SEARCH_LIBS
in configure.ac can't find it.  In any case, the code can't simply revert to the
standard library's strdup, as the OpenSSL libraries may/do add debugging
information around memory allocations and deallocations... the fallback code
must use OpenSSL allocation functions instead.  I suggest seeing if the version
of OpenSSL that doesn't have OPENSSL_strdup has CRYPTO_strdup and
switch all those OPENARC_malloc, etc. to use CRYPTO_... instead.

My fix is in https://github.com/trusteddomainproject/OpenARC/pull/96

#96 was merged last month.


Fix typo and add commas.

Coverage remained the same at 36.741% when pulling 62abfdfe4fb243ea15b02aa589dace6429169519 on lwm:fix-typo-add-commas into bf1fd3088d6ad2473f37b89d2ae6d1d19554319d on akrennmair:master.

Thank you!


[CP] bump linter to fix crashes / records false positives

Commit(s) to merge
92b963ec83b2b9a3ca40d08a67382d5a1644a37d
Target
beta
Prepared changelist for beta/stable
https://dart-review.googlesource.com/c/sdk/+/298741
Issue Description
Fixes issues w/ patterns/records:

crashes in use_build_context_synchronously
false positives in non_constant_identifier_names
false positives in no_leading_underscores_for_local_identifiers
false positives in constant_identifier_names
false positives in prefer_final_locals

What is the fix
See linked fixes.
This bumps to the version of the linter currently in Google3.
We elected to do this rather than a series of targeted cherry picks because:

we moved to Dart 3.0 since the linter in stable making simple cherry picks hard and
an aggregate branch on the linter that collected the required changes would be complex

Since this linter is currently in Google3 and analytics have not shown any new crashes and we haven't received any new bug reports we feel this is safer than the alternative.
Why cherry-pick
To fix crashes and fix recommended lint rule over-reporting for 3.0 code using patterns.
Risk
low
Issue link(s)
https://github.com/dart-lang/linter/pull/4279, https://github.com/dart-lang/linter/issues/4293, https://github.com/dart-lang/linter/issues/4295, https://github.com/dart-lang/linter/issues/4297, https://github.com/dart-lang/linter/issues/4299
Extra Info
Full list of changes since version in stable:
e36813b3 fix non_constant_identifier_names pattern field over-reporting (https://github.com/dart-lang/linter/issues/4300)
70509465 fix constant_identifier_names pattern field over-reporting (https://github.com/dart-lang/linter/issues/4298)
ae3501be fix no_leading_underscores_for_local_identifiers pattern field over-reporting (v4296)
17dcb460 prefer_final_in_for_each_test for final records (https://github.com/dart-lang/linter/issues/4294)
f38c72ed tests for prefer_final_in_for_each pattern support (https://github.com/dart-lang/linter/issues/4291)
a7facd7d tidy up formal parameter inspection (https://github.com/dart-lang/linter/issues/4289)
0c18a657 unnecessary_final support for ForEachPartsWithPatterns (https://github.com/dart-lang/linter/issues/4288)
2212d95e fix prefer_final_locals false positive in foreach loops (https://github.com/dart-lang/linter/issues/4287)
6646b99d fix Dart3 unsafe mixing in (https://github.com/dart-lang/linter/issues/4285)
e48f2863 Move use_key_in_widget_constructors tests to be reflective (https://github.com/dart-lang/linter/issues/4283)
82e7147c Fix unrelated_type_equality_checks when comparing enums (https://github.com/dart-lang/linter/issues/4282)
d8ccf815 fix pr labeler yaqml indent (https://github.com/dart-lang/linter/issues/4051)
ca2a9fe0 Lint about comparing booleans to boolean literals (https://github.com/dart-lang/linter/issues/3973)
4c2af043 Fix false negative in prefer_contains (https://github.com/dart-lang/linter/issues/2830)
efa4c979 Update releasing.md with more pre-release steps (https://github.com/dart-lang/linter/issues/3901)
a8c903cb bump cli_util dep (https://github.com/dart-lang/linter/issues/4206)
500c2cec New rule: Use matching super parameter names (https://github.com/dart-lang/linter/issues/4263)
e22fdaee [use_build_context_synchronously] Fix for unprotected 'last' invocation. (https://github.com/dart-lang/linter/issues/4279)
f9edf665 bump gh pages build to dev SDK (https://github.com/dart-lang/linter/issues/4275)
e944fc70 hasLeadingUnderscore extension cleanup (https://github.com/dart-lang/linter/issues/4274)
676c7117 inverted token type extension (https://github.com/dart-lang/linter/issues/4272)
686e85da => Dart 3.0! (https://github.com/dart-lang/linter/issues/4267)
74fa5ff0 Add comments to use_build_context_synchronously tests with fuller explanations (https://github.com/dart-lang/linter/issues/4271)
a2155130 prefer_const_constructors_in_immutables: simplify visitConstructorDeclaration override (https://github.com/dart-lang/linter/issues/4254)
eb0ca501 Account for records in type relatedness checks (https://github.com/dart-lang/linter/issues/4266)
0b4787f7 stop pana baseline checking (https://github.com/dart-lang/linter/issues/4269)
d9c05ab5 Move more use_build_context_synchronously tests (https://github.com/dart-lang/linter/issues/4264)
fffb74c3 Move several if-concerned use_build_context_synchronously tests (https://github.com/dart-lang/linter/issues/4255)
Note that the unrelated changes were deemed low-risk.

SGTM

@pq in the case that this ends up causing issues, would there be any concerns with simply rolling back the revision?

in the case that this ends up causing issues, would there be any concerns with simply rolling back the revision?

Nope. I think that would be the right thing to do.  If there are issues we can just go back to what's in beta currently and tough it out until we can put together something for the next stable release.

@jacob314 and @vsmenon could you take a look at this cherry-pick request?

I think another commit in the list is also a good reason to cherry-pick:

eb0ca501 Account for records in type relatedness checks (https://github.com/dart-lang/linter/pull/4266)

@athomas @leafpetersen - thoughts?  We're getting close enough to the release - the bar is high.  Are we better off holding this for the next dot release?

@pq can you summarize how common these issue are?  Is this fixing something that many people are likely to run into, or is this more of a "now and then" sort of thing?  We have a patch release going out ~1 week after stable, is it reasonable to push this out or is this something that's going to significantly impact the experience of a significant number of users when they try things out?

@scheglov: can you speak to the frequency of the crash you fixed? (It's in a flutter recommended lint so could potentially affect a lot of 3P folks.)
A few of the false positives are in recommended lints and will trigger with records in particular though those are easier to (temporarily) ignore.

High enough that I noticed it looking through crash reports.
I don't know enough to make good query, but probably using the existing crash UI this.
Currently it shows 751 crashes from 85 users that used beta.

One more note: we definitely saw the crashes drop after the fix made it to the Dart SDK. Crashes in general are reduced as well (we did not create any worse problem with the fix :D )

@pq and @srawlins  assuming the failures here are expected we can merge this one.

Yeah. Those failures are unrelated.