rocmPackages.* 5.7.1→ 6.0.2

Description of changes
CC https://github.com/NixOS/nixpkgs/issues/197885
This adds ROCm 6.0.2 alongside ROCm 5.7.
This PR

copies the version 5 folder and creates a new one for version 6

version 5 is still available as rocmPackages_5

use nix-update and some manual updates to get the package sources up to date
fixes the resulting build issues, including a few contributions from https://github.com/NixOS/nixpkgs/pull/289187

Remaining issues

migraphxdoes not build (https://github.com/ROCm/AMDMIGraphX/issues/2892)
~look at output sizes (since there seems to be a 3 GB limit if you want caching from cache.nixos.org)~
look at https://github.com/NixOS/nixpkgs/issues/276846 and checked if that issue is still present.
mark ROCm 5.7 as deprecated
~look through the packages marked as broken and remove them if they became irrelevant~
~redo file copying to preserve history (see https://stackoverflow.com/a/46484848)~
~add myself as a maintainer~
´composable_kernel´ profiler does not build, so it is skipped
still uses GCC 12 to build LLVM

Related work
I was not able to take the changes from https://github.com/Madouura/nixpkgs/tree/dev/rocm-rework,
because that branch has a large amount of changes which I felt would have taken me a long time review and understand, especially since I did not know how solid those changes are already.
Closes https://github.com/NixOS/nixpkgs/issues/280927
Things done

Built on platform(s)

[X] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

Result of nixpkgs-review run on x86_64-linux 1

14 packages marked as broken and skipped:

rocmPackages.llvm.flang
rocmPackages.llvm.flang.doc
rocmPackages.llvm.flang.info
rocmPackages.llvm.flang.man
rocmPackages.llvm.libclc
rocmPackages.rdc
rocmPackages.rdc.doc
rocmPackages_5.llvm.flang
rocmPackages_5.llvm.flang.doc
rocmPackages_5.llvm.flang.info
rocmPackages_5.llvm.flang.man
rocmPackages_5.llvm.libclc
rocmPackages_5.rdc
rocmPackages_5.rdc.doc

13 packages failed to build:

magma (magma-hip ,magma_2_7_2)
magma_2_6_2
opensyclWithRocm
rocmPackages.composable_kernel
rocmPackages.migraphx
rocmPackages.miopen (rocmPackages.miopen-hip)
rocmPackages.miopen-opencl
rocmPackages.rccl
rocmPackages.rocmlir
rocmPackages.rocmlir-rock
rocmPackages.rocmlir.external
rocmPackages.rocprofiler
rocmPackages_5.rocprofiler

175 packages built:

blender-hip
rocmPackages.clang-ocl
rocmPackages.clr
rocmPackages.clr.icd
rocmPackages.half
rocmPackages.hip-common
rocmPackages.hipblas
rocmPackages.hipcc
rocmPackages.hipcub
rocmPackages.hipfft
rocmPackages.hipfort
rocmPackages.hipify
rocmPackages.hiprand
rocmPackages.hipsolver
rocmPackages.hipsparse
rocmPackages.hsa-amd-aqlprofile-bin
rocmPackages.llvm.bintools
rocmPackages.llvm.clang
rocmPackages.llvm.clang-tools-extra
rocmPackages.llvm.clang-tools-extra.doc
rocmPackages.llvm.clang-tools-extra.info
rocmPackages.llvm.clang-tools-extra.man
rocmPackages.llvm.clang-unwrapped
rocmPackages.llvm.clang-unwrapped.doc
rocmPackages.llvm.clang-unwrapped.info
rocmPackages.llvm.clang-unwrapped.man
rocmPackages.llvm.compiler-rt
rocmPackages.llvm.libc
rocmPackages.llvm.libc.doc
rocmPackages.llvm.libcxx
rocmPackages.llvm.libcxx.doc
rocmPackages.llvm.libcxxabi
rocmPackages.llvm.libunwind
rocmPackages.llvm.libunwind.doc
rocmPackages.llvm.lld
rocmPackages.llvm.lld.doc
rocmPackages.llvm.lldb
rocmPackages.llvm.lldb.doc
rocmPackages.llvm.lldb.info
rocmPackages.llvm.lldb.man
rocmPackages.llvm.llvm
rocmPackages.llvm.llvm.doc
rocmPackages.llvm.llvm.info
rocmPackages.llvm.llvm.man
rocmPackages.llvm.mlir
rocmPackages.llvm.openmp
rocmPackages.llvm.openmp.doc
rocmPackages.llvm.openmp.info
rocmPackages.llvm.openmp.man
rocmPackages.llvm.polly
rocmPackages.llvm.polly.doc
rocmPackages.llvm.polly.info
rocmPackages.llvm.polly.man
rocmPackages.llvm.pstl
rocmPackages.llvm.rocmClangStdenv
rocmPackages.rocalution
rocmPackages.rocblas
rocmPackages.rocdbgapi
rocmPackages.rocdbgapi.doc
rocmPackages.rocfft
rocmPackages.rocgdb
rocmPackages.rocm-cmake
rocmPackages.rocm-comgr
rocmPackages.rocm-core
rocmPackages.rocm-device-libs
rocmPackages.rocm-docs-core
rocmPackages.rocm-docs-core.dist
rocmPackages.rocm-runtime
rocmPackages.rocm-smi
rocmPackages.rocm-thunk
rocmPackages.rocminfo
rocmPackages.rocprim
rocmPackages.rocr-debug-agent
rocmPackages.rocsolver
rocmPackages.rocsparse
rocmPackages.rocthrust
rocmPackages.roctracer
rocmPackages.rocwmma
rocmPackages.rpp (rocmPackages.rpp-hip)
rocmPackages.rpp-cpu
rocmPackages.rpp-opencl
rocmPackages.tensile
rocmPackages.tensile.dist
rocmPackages_5.clang-ocl
rocmPackages_5.clr
rocmPackages_5.clr.icd
rocmPackages_5.composable_kernel
rocmPackages_5.half
rocmPackages_5.hip-common
rocmPackages_5.hipblas
rocmPackages_5.hipcc
rocmPackages_5.hipcub
rocmPackages_5.hipfft
rocmPackages_5.hipfort
rocmPackages_5.hipify
rocmPackages_5.hiprand
rocmPackages_5.hipsolver
rocmPackages_5.hipsparse
rocmPackages_5.hsa-amd-aqlprofile-bin
rocmPackages_5.llvm.bintools
rocmPackages_5.llvm.clang
rocmPackages_5.llvm.clang-tools-extra
rocmPackages_5.llvm.clang-tools-extra.doc
rocmPackages_5.llvm.clang-tools-extra.info
rocmPackages_5.llvm.clang-tools-extra.man
rocmPackages_5.llvm.clang-unwrapped
rocmPackages_5.llvm.clang-unwrapped.doc
rocmPackages_5.llvm.clang-unwrapped.info
rocmPackages_5.llvm.clang-unwrapped.man
rocmPackages_5.llvm.compiler-rt
rocmPackages_5.llvm.libc
rocmPackages_5.llvm.libc.doc
rocmPackages_5.llvm.libcxx
rocmPackages_5.llvm.libcxx.doc
rocmPackages_5.llvm.libcxxabi
rocmPackages_5.llvm.libunwind
rocmPackages_5.llvm.libunwind.doc
rocmPackages_5.llvm.lld
rocmPackages_5.llvm.lld.doc
rocmPackages_5.llvm.lldb
rocmPackages_5.llvm.lldb.doc
rocmPackages_5.llvm.lldb.info
rocmPackages_5.llvm.lldb.man
rocmPackages_5.llvm.llvm
rocmPackages_5.llvm.llvm.doc
rocmPackages_5.llvm.llvm.info
rocmPackages_5.llvm.llvm.man
rocmPackages_5.llvm.mlir
rocmPackages_5.llvm.openmp
rocmPackages_5.llvm.openmp.doc
rocmPackages_5.llvm.openmp.info
rocmPackages_5.llvm.openmp.man
rocmPackages_5.llvm.polly
rocmPackages_5.llvm.polly.doc
rocmPackages_5.llvm.polly.info
rocmPackages_5.llvm.polly.man
rocmPackages_5.llvm.pstl
rocmPackages_5.llvm.rocmClangStdenv
rocmPackages_5.migraphx
rocmPackages_5.miopen (rocmPackages_5.miopen-hip)
rocmPackages_5.miopen-opencl
rocmPackages_5.miopengemm
rocmPackages_5.miopengemm.doc
rocmPackages_5.rccl
rocmPackages_5.rocalution
rocmPackages_5.rocblas
rocmPackages_5.rocdbgapi
rocmPackages_5.rocdbgapi.doc
rocmPackages_5.rocfft
rocmPackages_5.rocgdb
rocmPackages_5.rocm-cmake
rocmPackages_5.rocm-comgr
rocmPackages_5.rocm-core
rocmPackages_5.rocm-device-libs
rocmPackages_5.rocm-docs-core
rocmPackages_5.rocm-docs-core.dist
rocmPackages_5.rocm-runtime
rocmPackages_5.rocm-smi
rocmPackages_5.rocm-thunk
rocmPackages_5.rocminfo
rocmPackages_5.rocmlir
rocmPackages_5.rocmlir-rock
rocmPackages_5.rocmlir.external
rocmPackages_5.rocprim
rocmPackages_5.rocr-debug-agent
rocmPackages_5.rocsolver
rocmPackages_5.rocsparse
rocmPackages_5.rocthrust
rocmPackages_5.roctracer
rocmPackages_5.rocwmma
rocmPackages_5.rpp (rocmPackages_5.rpp-hip)
rocmPackages_5.rpp-cpu
rocmPackages_5.rpp-opencl
rocmPackages_5.tensile
rocmPackages_5.tensile.dist

Screenshot illustrating what failed and crazy long build times.

I added a few more fixes and the composable_kernel, miopenand rocmlir builds succeed now if is set gpuTarget to target only my RX 7600 XT (gfx1102) with a bunch of code along these lines:
--- a/pkgs/development/rocm-modules/6/default.nix
+++ b/pkgs/development/rocm-modules/6/default.nix
@@ -16,6 +16,7 @@
 let
   rocmUpdateScript = callPackage ./update.nix { };
 in rec {
+  gpuTargets = [ "gfx1102" ];
   ## ROCm ##
   llvm = recurseIntoAttrs (callPackage ./llvm/default.nix { inherit rocmUpdateScript rocm-device-libs rocm-runtime rocm-thunk clr; });

@@ -73,7 +74,7 @@ in rec {

   # Broken, too many errors
   rdc = callPackage ./rdc {
-    inherit rocmUpdateScript rocm-smi rocm-runtime;
+    inherit rocmUpdateScript rocm-smi rocm-runtime gpuTargets;
     stdenv = gcc12Stdenv;
     # stdenv = llvm.rocmClangStdenv;
   };
+[... and so on ...]

I can see that composable_kernel still has some issue (besides taking a loong time to build) and fails if I build for all the gpuTargets.

Hi! I just noticed that we've been working on the same thing. I hope you don't mind, but I went ahead  and cherry-picked your changes into my branch: https://github.com/ScatteredRay/nixpkgs/tree/rocm6 https://github.com/NixOS/nixpkgs/pull/289187

Yeah. It seems to be building now with stdenv gcc.. want to get ot building with the same the rocm-llvm stdenv, but running into a few conflicts. I'm not certain, but I think it's basically running into this issue: #192459  but with the rocm-llvm

Yeah. Rocprofiler seems to be building now with stdenv gcc.. want to get ot building with the same the rocm-llvm stdenv, but running into a few conflicts. I'm not certain, but I think it's basically running into this issue: #192459 but with the rocm-llvm

I don't know anything about that unfortunately.
Is there some actual benefit to building that with rocm-llvm?

Is there some actual benefit to building that with rocm-llvm?

I don't know specifically. I think some project need it to link against the hip code correctly.
And then from there, there were some problem with conflicting libraries from linking with separate stdenvs with separate libraries. So I think in general it is a bit easier to link with stdenvs that have fewer differences.

Results of building this against current master (fae73881baad4963709ef0f24e87eb1342a6ef0a):
Result of nixpkgs-review pr 287846 run on x86_64-linux 1

14 packages marked as broken and skipped:

rocmPackages.llvm.flang
rocmPackages.llvm.flang.doc
rocmPackages.llvm.flang.info
rocmPackages.llvm.flang.man
rocmPackages.llvm.libclc
rocmPackages.rdc
rocmPackages.rdc.doc
rocmPackages_6.llvm.flang
rocmPackages_6.llvm.flang.doc
rocmPackages_6.llvm.flang.info
rocmPackages_6.llvm.flang.man
rocmPackages_6.llvm.libclc
rocmPackages_6.rdc
rocmPackages_6.rdc.doc

7 packages failed to build:

magma (magma-hip ,magma_2_7_2)
magma_2_6_2
opensyclWithRocm
rocmPackages.migraphx (rocmPackages_6.migraphx)
rocmPackages.miopen-opencl (rocmPackages_6.miopen-opencl)
rocmPackages.rocmlir (rocmPackages_6.rocmlir)
rocmPackages.rocmlir.external (rocmPackages_6.rocmlir.external)

88 packages built:

blender-hip
rocmPackages.clang-ocl (rocmPackages_6.clang-ocl)
rocmPackages.clr (rocmPackages_6.clr)
rocmPackages.clr.icd (rocmPackages_6.clr.icd)
rocmPackages.composable_kernel (rocmPackages_6.composable_kernel)
rocmPackages.half (rocmPackages_6.half)
rocmPackages.hip-common (rocmPackages_6.hip-common)
rocmPackages.hipblas (rocmPackages_6.hipblas)
rocmPackages.hipcc (rocmPackages_6.hipcc)
rocmPackages.hipcub (rocmPackages_6.hipcub)
rocmPackages.hipfft (rocmPackages_6.hipfft)
rocmPackages.hipfort (rocmPackages_6.hipfort)
rocmPackages.hipify (rocmPackages_6.hipify)
rocmPackages.hiprand (rocmPackages.rocrand ,rocmPackages_6.hiprand ,rocmPackages_6.rocrand)
rocmPackages.hipsolver (rocmPackages_6.hipsolver)
rocmPackages.hipsparse (rocmPackages_6.hipsparse)
rocmPackages.hsa-amd-aqlprofile-bin (rocmPackages_6.hsa-amd-aqlprofile-bin)
rocmPackages.llvm.bintools (rocmPackages_6.llvm.bintools)
rocmPackages.llvm.clang (rocmPackages_6.llvm.clang)
rocmPackages.llvm.clang-tools-extra (rocmPackages_6.llvm.clang-tools-extra)
rocmPackages.llvm.clang-tools-extra.doc (rocmPackages_6.llvm.clang-tools-extra.doc)
rocmPackages.llvm.clang-tools-extra.info (rocmPackages_6.llvm.clang-tools-extra.info)
rocmPackages.llvm.clang-tools-extra.man (rocmPackages_6.llvm.clang-tools-extra.man)
rocmPackages.llvm.clang-unwrapped (rocmPackages_6.llvm.clang-unwrapped)
rocmPackages.llvm.clang-unwrapped.doc (rocmPackages_6.llvm.clang-unwrapped.doc)
rocmPackages.llvm.clang-unwrapped.info (rocmPackages_6.llvm.clang-unwrapped.info)
rocmPackages.llvm.clang-unwrapped.man (rocmPackages_6.llvm.clang-unwrapped.man)
rocmPackages.llvm.compiler-rt (rocmPackages_6.llvm.compiler-rt)
rocmPackages.llvm.libc (rocmPackages_6.llvm.libc)
rocmPackages.llvm.libc.doc (rocmPackages_6.llvm.libc.doc)
rocmPackages.llvm.libcxx (rocmPackages_6.llvm.libcxx)
rocmPackages.llvm.libcxx.doc (rocmPackages_6.llvm.libcxx.doc)
rocmPackages.llvm.libcxxabi (rocmPackages_6.llvm.libcxxabi)
rocmPackages.llvm.libunwind (rocmPackages_6.llvm.libunwind)
rocmPackages.llvm.libunwind.doc (rocmPackages_6.llvm.libunwind.doc)
rocmPackages.llvm.lld (rocmPackages_6.llvm.lld)
rocmPackages.llvm.lld.doc (rocmPackages_6.llvm.lld.doc)
rocmPackages.llvm.lldb (rocmPackages_6.llvm.lldb)
rocmPackages.llvm.lldb.doc (rocmPackages_6.llvm.lldb.doc)
rocmPackages.llvm.lldb.info (rocmPackages_6.llvm.lldb.info)
rocmPackages.llvm.lldb.man (rocmPackages_6.llvm.lldb.man)
rocmPackages.llvm.llvm (rocmPackages_6.llvm.llvm)
rocmPackages.llvm.llvm.doc (rocmPackages_6.llvm.llvm.doc)
rocmPackages.llvm.llvm.info (rocmPackages_6.llvm.llvm.info)
rocmPackages.llvm.llvm.man (rocmPackages_6.llvm.llvm.man)
rocmPackages.llvm.mlir (rocmPackages_6.llvm.mlir)
rocmPackages.llvm.openmp (rocmPackages_6.llvm.openmp)
rocmPackages.llvm.openmp.doc (rocmPackages_6.llvm.openmp.doc)
rocmPackages.llvm.openmp.info (rocmPackages_6.llvm.openmp.info)
rocmPackages.llvm.openmp.man (rocmPackages_6.llvm.openmp.man)
rocmPackages.llvm.polly (rocmPackages_6.llvm.polly)
rocmPackages.llvm.polly.doc (rocmPackages_6.llvm.polly.doc)
rocmPackages.llvm.polly.info (rocmPackages_6.llvm.polly.info)
rocmPackages.llvm.polly.man (rocmPackages_6.llvm.polly.man)
rocmPackages.llvm.pstl (rocmPackages_6.llvm.pstl)
rocmPackages.llvm.rocmClangStdenv (rocmPackages_6.llvm.rocmClangStdenv)
rocmPackages.miopen (rocmPackages.miopen-hip ,rocmPackages_6.miopen ,rocmPackages_6.miopen-hip)
rocmPackages.rccl (rocmPackages_6.rccl)
rocmPackages.rocalution (rocmPackages_6.rocalution)
rocmPackages.rocblas (rocmPackages_6.rocblas)
rocmPackages.rocdbgapi (rocmPackages_6.rocdbgapi)
rocmPackages.rocdbgapi.doc (rocmPackages_6.rocdbgapi.doc)
rocmPackages.rocfft (rocmPackages_6.rocfft)
rocmPackages.rocgdb (rocmPackages_6.rocgdb)
rocmPackages.rocm-cmake (rocmPackages_6.rocm-cmake)
rocmPackages.rocm-comgr (rocmPackages_6.rocm-comgr)
rocmPackages.rocm-core (rocmPackages_6.rocm-core)
rocmPackages.rocm-device-libs (rocmPackages_6.rocm-device-libs)
rocmPackages.rocm-docs-core (rocmPackages_6.rocm-docs-core)
rocmPackages.rocm-docs-core.dist (rocmPackages_6.rocm-docs-core.dist)
rocmPackages.rocm-runtime (rocmPackages_6.rocm-runtime)
rocmPackages.rocm-smi (rocmPackages_6.rocm-smi)
rocmPackages.rocm-thunk (rocmPackages_6.rocm-thunk)
rocmPackages.rocminfo (rocmPackages_6.rocminfo)
rocmPackages.rocmlir-rock (rocmPackages_6.rocmlir-rock)
rocmPackages.rocprim (rocmPackages_6.rocprim)
rocmPackages.rocprofiler (rocmPackages_6.rocprofiler)
rocmPackages.rocr-debug-agent (rocmPackages_6.rocr-debug-agent)
rocmPackages.rocsolver (rocmPackages_6.rocsolver)
rocmPackages.rocsparse (rocmPackages_6.rocsparse)
rocmPackages.rocthrust (rocmPackages_6.rocthrust)
rocmPackages.roctracer (rocmPackages_6.roctracer)
rocmPackages.rocwmma (rocmPackages_6.rocwmma)
rocmPackages.rpp (rocmPackages.rpp-hip ,rocmPackages_6.rpp ,rocmPackages_6.rpp-hip)
rocmPackages.rpp-cpu (rocmPackages_6.rpp-cpu)
rocmPackages.rpp-opencl (rocmPackages_6.rpp-opencl)
rocmPackages.tensile (rocmPackages_6.tensile)
rocmPackages.tensile.dist (rocmPackages_6.tensile.dist)

When building this branch by itself llvm.clang-tools-extra and rocprofiler also failed.
The former running until it was patching interpreter paths for the info output and no errors indicated and the latter having its tests time out. i did the build with the extra failures in the cloud, while the build against master was done locally.
The rocmlir build failing would be a regression if it also fails with gpuTargets = [ "gfx1102" ];.

opensyclWithRocm is broken due to the old version of ROCm. https://github.com/AdaptiveCpp/AdaptiveCpp/commit/3952b468c9da89edad9dff953cdcab0a3c3bf78c. My plan is to keep building this against ROCm 5.7.
[ 90%] Building CXX object src/runtime/CMakeFiles/rt-backend-hip.dir/hip/hip_hardware_manager.cpp.o
[ 92%] Building CXX object src/runtime/CMakeFiles/rt-backend-hip.dir/hip/hip_backend.cpp.o
[ 93%] Building CXX object src/runtime/CMakeFiles/rt-backend-hip.dir/hip/hip_code_object.cpp.o
/build/source/src/runtime/hip/hip_allocator.cpp:144:35: error: no member named 'memoryType' in 'hipPointerAttribute_t'
  out.is_optimized_host = attribs.memoryType == hipMemoryTypeHost;
                          ~~~~~~~ ^
1 error generated.
make[2]: *** [src/runtime/CMakeFiles/rt-backend-hip.dir/build.make:132: src/runtime/CMakeFiles/rt-backend-hip.dir/hip/hip_allocator.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....

magma probably fails for a similar reason, and will be handled similarly.
[257/3332] Building CXX object CMakeFiles/magma.dir/src/zpotf2_vbatched.cpp.o
[258/3332] Building CXX object CMakeFiles/magma.dir/interface_hip/blas_z_v2.cpp.o
FAILED: CMakeFiles/magma.dir/interface_hip/blas_z_v2.cpp.o
/nix/store/4ic7l20hx0508q6gks795hp65npllmik-clr-6.0.2/bin/hipcc -DUSE_PROF_API=1 -D__HIP_PLATFORM_AMD__=1 -Dmagma_EXPORTS -I/build/magma-2.7.2/build/include -I/build/magma-2.7.2/include -I/build/magma-2.7.2/control -I/build/magma-2.7.2/magmablas_hip -I/build/magma-2.7.2/sparse_hip/include -I/build/magma-2.7.2/sparse_hip/control -I/build/magma-2.7.2/testing -std=c++11 -DADD_ -fopenmp=libomp -Wall -Wno-unused-function -O3 -DNDEBUG -fPIC -MD -MT CMakeFiles/magma.dir/interface_hip/blas_z_v2.cpp.o -MF CMakeFiles/magma.dir/interface_hip/blas_z_v2.cpp.o.d -o CMakeFiles/magma.dir/interface_hip/blas_z_v2.cpp.o -c /build/magma-2.7.2/interface_hip/blas_z_v2.cpp
/build/magma-2.7.2/interface_hip/blas_z_v2.cpp:1853:9: error: no matching function for call to 'hipblasZtrmm'
        hipblasZtrmm(
        ^~~~~~~~~~~~
/nix/store/kw5m73874x1bzkamdws0w41z70xpilrz-hipblas-6.0.2/include/hipblas/hipblas.h:17918:32: note: candidate function not viable: requires 14 arguments, but 12 were provided
HIPBLAS_EXPORT hipblasStatus_t hipblasZtrmm(hipblasHandle_t             handle,
                               ^
1 error generated when compiling for gfx906.
[259/3332] Building CXX object CMakeFiles/magma.dir/src/zpotrf_panel_vbatched.cpp.o
[260/3332] Building CXX object CMakeFiles/magma.dir/interface_hip/interface.cpp.o
FAILED: CMakeFiles/magma.dir/interface_hip/interface.cpp.o
/nix/store/4ic7l20hx0508q6gks795hp65npllmik-clr-6.0.2/bin/hipcc -DUSE_PROF_API=1 -D__HIP_PLATFORM_AMD__=1 -Dmagma_EXPORTS -I/build/magma-2.7.2/build/include -I/build/magma-2.7.2/include -I/build/magma-2.7.2/control -I/build/magma-2.7.2/magmablas_hip -I/build/magma-2.7.2/sparse_hip/include -I/build/magma-2.7.2/sparse_hip/control -I/build/magma-2.7.2/testing -std=c++11 -DADD_ -fopenmp=libomp -Wall -Wno-unused-function -O3 -DNDEBUG -fPIC -MD -MT CMakeFiles/magma.dir/interface_hip/interface.cpp.o -MF CMakeFiles/magma.dir/interface_hip/interface.cpp.o.d -o CMakeFiles/magma.dir/interface_hip/interface.cpp.o -c /build/magma-2.7.2/interface_hip/interface.cpp
/build/magma-2.7.2/interface_hip/interface.cpp:216:65: error: no member named 'gcnArch' in 'hipDeviceProp_tR0600'
                    g_magma_devices[dev].cuda_arch       = prop.gcnArch;
                                                           ~~~~ ^
/build/magma-2.7.2/interface_hip/interface.cpp:467:22: error: no member named 'gcnArch' in 'hipDeviceProp_tR0600'
                prop.gcnArch );
                ~~~~ ^
/build/magma-2.7.2/interface_hip/interface.cpp:531:30: error: no member named 'memoryType' in 'hipPointerAttribute_t'
                return (attr.memoryType == hipMemoryTypeDevice);
                        ~~~~ ^
3 errors generated when compiling for gfx906.

The build for migraphx still fails with the following error:
[ 89%] Building CXX object src/targets/gpu/CMakeFiles/migraphx_gpu.dir/reverse.cpp.o
[ 89%] Building CXX object src/targets/gpu/CMakeFiles/migraphx_gpu.dir/rnn_variable_seq_lens.cpp.o
In file included from /build/source/src/targets/gpu/prefuse_ops.cpp:31:
/build/source/src/targets/gpu/include/migraphx/gpu/ck.hpp:33:10: fatal error: 'ck/host/device_gemm_multiple_d.hpp' file not found
#include "ck/host/device_gemm_multiple_d.hpp"
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
make[2]: *** [src/targets/gpu/CMakeFiles/migraphx_gpu.dir/build.make:524: src/targets/gpu/CMakeFiles/migraphx_gpu.dir/prefuse_ops.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....
make[1]: *** [CMakeFiles/Makefile2:13502: src/targets/gpu/CMakeFiles/migraphx_gpu.dir/all] Error 2
make: *** [Makefile:156: all] Error 2

the only similar file I can find is https://github.com/ROCm/composable_kernel/blob/9ce18b045d6ffa6cfa29134229b422c07984ffb7/include/ck/tensor_operation/gpu/device/device_gemm_multiple_d.hpp, but that does seems like an intentional difference since that file lives in tensor_operation/gpu/device and not host.
I am not sure how to resolve this situation.

I would like to mark ROCm 5.7 as deprecated, but using lib.warn for that makes ofborg-eval unhappy:
This PR does not cleanly list package outputs after merging.
--- a/pkgs/top-level/all-packages.nix
+++ b/pkgs/top-level/all-packages.nix
@@ -7757,7 +7757,7 @@ with pkgs;
   rar2fs = callPackage ../tools/filesystems/rar2fs { };

   rocmPackages = rocmPackages_6;
-  rocmPackages_5 = recurseIntoAttrs (callPackage ../development/rocm-modules/5 { });
+  rocmPackages_5 = lib.warn "ROCm 5.7 is deprecated in nixpkgs, please upgrade to ROCm 6.+" recurseIntoAttrs (callPackage ../development/rocm-modules/5 { });
   rocmPackages_6 = recurseIntoAttrs (callPackage ../development/rocm-modules/6 { });

   rune = callPackage ../development/interpreters/rune { };

It looks like all the packages marked as broken are still maintained upstream. I did not look into fixing them yet.
I also looked at the sizes of all of the output derivations. They did increase a bit, but none of the outputs really blew up in size significantly, and none seem to be close to the 3 GB size limit.

Since I don't have any ideas about how to get 'migraphx' to build, I have opened an issue about the build error upstream: https://github.com/ROCm/AMDMIGraphX/issues/2892
I am marking this as ready for review to get some feedback.
Maybe someone else has any ideas, maybe the solution for now is to just mark that package as broken and get this merged.

That PR (https://github.com/ROCm/composable_kernel/pull/1134) is not in the ROCm release tagged for 6.0.2 yet, and simply adding the correct include files to the migraphx build is not enough to make it build.
I'll see what they say in the issue that I opened, but it looks like marking migraphx as broken until there are tagged releases that work might be sensible.

Running nixpkgs-review, currently stuck on rocblas, seems to hang "forever" on 4 warnings generated when compiling for gfx942. I plan on letting it sit for some hours to see if it is doing something that is not showing but from CPU/RAM usage it doesn't look like it...
Other than that 46/54 packages are built so it looks optimistic.

If I remove gfx942 I instead get this error:
    codeObjectFiles += buildSourceCodeObjectFiles(CxxCompiler, kernelFiles, outputPath)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/build/source/tensile/lib/python3.11/site-packages/Tensile/TensileCreateLibrary.py", line 331, in buildSourceCodeObjectFiles
    coFiles = Common.ParallelMap(buildSourceCodeObjectFile, args, "Compiling source kernels")
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/build/source/tensile/lib/python3.11/site-packages/Tensile/Parallel.py", line 87, in ParallelMap
    rv = Parallel(n_jobs=threadCount)(delayed(pcall)(function, a, params) for a, params in pargs)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/r7g5xhf75h5basz58jmblkbj02gynmy8-python3.11-joblib-1.3.2/lib/python3.11/site-packages/joblib/parallel.py", line 1952, in __call__
    return output if self.return_generator else list(output)
                                                ^^^^^^^^^^^^
  File "/nix/store/r7g5xhf75h5basz58jmblkbj02gynmy8-python3.11-joblib-1.3.2/lib/python3.11/site-packages/joblib/parallel.py", line 1595, in _get_outputs
    yield from self._retrieve()
  File "/nix/store/r7g5xhf75h5basz58jmblkbj02gynmy8-python3.11-joblib-1.3.2/lib/python3.11/site-packages/joblib/parallel.py", line 1699, in _retrieve
    return [func(*args, **kwargs)
            ^^^^^^^^^^^^^^^^^^^^^
  File "/build/source/tensile/lib/python3.11/site-packages/Tensile/Parallel.py", line 53, in pcallWithGlobalParamsMultiArg
    return f(*args)
           ^^^^^^^^
  File "/build/source/tensile/lib/python3.11/site-packages/Tensile/TensileCreateLibrary.py", line 325, in buildSourceCodeObjectFile
    shutil.copyfile(src, dst)
  File "/nix/store/3v2ch16fkl50i85n05h5ckss8pxx6836-python3-3.11.8/lib/python3.11/shutil.py", line 258, in copyfile
    with open(dst, 'wb') as fdst:
         ^^^^^^^^^^^^^^^
PermissionError: [Errno 13] Permission denied: '/build/source/build/Tensile/library/TensileLibrary_Type_4xi8I_HPA_Contraction_l_Alik_Bljk_Cijk_Dijk_CU104_gfx90a-xnack-.hsaco'
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/build/source/tensile/lib/python3.11/site-packages/Tensile/bin/TensileCreateLibrary", line 43, in <module>
    TensileCreateLibrary()
  File "/build/source/tensile/lib/python3.11/site-packages/Tensile/TensileCreateLibrary.py", line 1362, in TensileCreateLibrary
    codeObjectFiles = writeSolutionsAndKernels(outputPath, CxxCompiler, None, solutions,
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/build/source/tensile/lib/python3.11/site-packages/Tensile/TensileCreateLibrary.py", line 607, in writeSolutionsAndKernels
    codeObjectFiles += buildSourceCodeObjectFiles(CxxCompiler, kernelFiles, outputPath)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/build/source/tensile/lib/python3.11/site-packages/Tensile/TensileCreateLibrary.py", line 331, in buildSourceCodeObjectFiles
    coFiles = Common.ParallelMap(buildSourceCodeObjectFile, args, "Compiling source kernels")
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/build/source/tensile/lib/python3.11/site-packages/Tensile/Parallel.py", line 87, in ParallelMap
    rv = Parallel(n_jobs=threadCount)(delayed(pcall)(function, a, params) for a, params in pargs)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/r7g5xhf75h5basz58jmblkbj02gynmy8-python3.11-joblib-1.3.2/lib/python3.11/site-packages/joblib/parallel.py", line 1952, in __call__
    return output if self.return_generator else list(output)
                                                ^^^^^^^^^^^^
  File "/nix/store/r7g5xhf75h5basz58jmblkbj02gynmy8-python3.11-joblib-1.3.2/lib/python3.11/site-packages/joblib/parallel.py", line 1595, in _get_outputs
    yield from self._retrieve()
  File "/nix/store/r7g5xhf75h5basz58jmblkbj02gynmy8-python3.11-joblib-1.3.2/lib/python3.11/site-packages/joblib/parallel.py", line 1699, in _retrieve
    self._raise_error_fast()
  File "/nix/store/r7g5xhf75h5basz58jmblkbj02gynmy8-python3.11-joblib-1.3.2/lib/python3.11/site-packages/joblib/parallel.py", line 1734, in _raise_error_fast
    error_job.get_result(self.timeout)
  File "/nix/store/r7g5xhf75h5basz58jmblkbj02gynmy8-python3.11-joblib-1.3.2/lib/python3.11/site-packages/joblib/parallel.py", line 736, in get_result
    return self._return_or_raise()
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/r7g5xhf75h5basz58jmblkbj02gynmy8-python3.11-joblib-1.3.2/lib/python3.11/site-packages/joblib/parallel.py", line 754, in _return_or_raise
    raise self._result
PermissionError: [Errno 13] Permission denied: '/build/source/build/Tensile/library/TensileLibrary_Type_4xi8I_HPA_Contraction_l_Alik_Bljk_Cijk_Dijk_CU104_gfx90a-xnack-.hsaco'
make[2]: *** [library/src/CMakeFiles/TENSILE_LIBRARY_TARGET.dir/build.make:2114: Tensile/library/TensileLibrary_Type_HS_HPA_Contraction_l_Ailk_Bljk_Cijk_Dijk_gfx906.dat] Error 1
make[1]: *** [CMakeFiles/Makefile2:230: library/src/CMakeFiles/TENSILE_LIBRARY_TARGET.dir/all] Error 2
make: *** [Makefile:156: all] Error 2

Let me know if there's anything I can do to debug this. I have a machine with 64 cores and 256GB memory so building is not a big issue

Result of nixpkgs-review pr 287846 run on x86_64-linux 1

16 packages marked as broken and skipped:

rocmPackages.llvm.flang
rocmPackages.llvm.flang.doc
rocmPackages.llvm.flang.info
rocmPackages.llvm.flang.man
rocmPackages.llvm.libclc
rocmPackages.migraphx
rocmPackages.rdc
rocmPackages.rdc.doc
rocmPackages_6.llvm.flang
rocmPackages_6.llvm.flang.doc
rocmPackages_6.llvm.flang.info
rocmPackages_6.llvm.flang.man
rocmPackages_6.llvm.libclc
rocmPackages_6.migraphx
rocmPackages_6.rdc
rocmPackages_6.rdc.doc

90 packages built:

blender-hip
rocmPackages.clang-ocl (rocmPackages_6.clang-ocl)
rocmPackages.clr (rocmPackages_6.clr)
rocmPackages.clr.icd (rocmPackages_6.clr.icd)
rocmPackages.composable_kernel (rocmPackages_6.composable_kernel)
rocmPackages.half (rocmPackages_6.half)
rocmPackages.hip-common (rocmPackages_6.hip-common)
rocmPackages.hipblas (rocmPackages_6.hipblas)
rocmPackages.hipcc (rocmPackages_6.hipcc)
rocmPackages.hipcub (rocmPackages_6.hipcub)
rocmPackages.hipfft (rocmPackages_6.hipfft)
rocmPackages.hipfort (rocmPackages_6.hipfort)
rocmPackages.hipify (rocmPackages_6.hipify)
rocmPackages.hiprand (rocmPackages.rocrand ,rocmPackages_6.hiprand ,rocmPackages_6.rocrand)
rocmPackages.hipsolver (rocmPackages_6.hipsolver)
rocmPackages.hipsparse (rocmPackages_6.hipsparse)
rocmPackages.hsa-amd-aqlprofile-bin (rocmPackages_6.hsa-amd-aqlprofile-bin)
rocmPackages.llvm.bintools (rocmPackages_6.llvm.bintools)
rocmPackages.llvm.clang (rocmPackages_6.llvm.clang)
rocmPackages.llvm.clang-tools-extra (rocmPackages_6.llvm.clang-tools-extra)
rocmPackages.llvm.clang-tools-extra.doc (rocmPackages_6.llvm.clang-tools-extra.doc)
rocmPackages.llvm.clang-tools-extra.info (rocmPackages_6.llvm.clang-tools-extra.info)
rocmPackages.llvm.clang-tools-extra.man (rocmPackages_6.llvm.clang-tools-extra.man)
rocmPackages.llvm.clang-unwrapped (rocmPackages_6.llvm.clang-unwrapped)
rocmPackages.llvm.clang-unwrapped.doc (rocmPackages_6.llvm.clang-unwrapped.doc)
rocmPackages.llvm.clang-unwrapped.info (rocmPackages_6.llvm.clang-unwrapped.info)
rocmPackages.llvm.clang-unwrapped.man (rocmPackages_6.llvm.clang-unwrapped.man)
rocmPackages.llvm.compiler-rt (rocmPackages_6.llvm.compiler-rt)
rocmPackages.llvm.libc (rocmPackages_6.llvm.libc)
rocmPackages.llvm.libc.doc (rocmPackages_6.llvm.libc.doc)
rocmPackages.llvm.libcxx (rocmPackages_6.llvm.libcxx)
rocmPackages.llvm.libcxx.doc (rocmPackages_6.llvm.libcxx.doc)
rocmPackages.llvm.libcxxabi (rocmPackages_6.llvm.libcxxabi)
rocmPackages.llvm.libunwind (rocmPackages_6.llvm.libunwind)
rocmPackages.llvm.libunwind.doc (rocmPackages_6.llvm.libunwind.doc)
rocmPackages.llvm.lld (rocmPackages_6.llvm.lld)
rocmPackages.llvm.lld.doc (rocmPackages_6.llvm.lld.doc)
rocmPackages.llvm.lldb (rocmPackages_6.llvm.lldb)
rocmPackages.llvm.lldb.doc (rocmPackages_6.llvm.lldb.doc)
rocmPackages.llvm.lldb.info (rocmPackages_6.llvm.lldb.info)
rocmPackages.llvm.lldb.man (rocmPackages_6.llvm.lldb.man)
rocmPackages.llvm.llvm (rocmPackages_6.llvm.llvm)
rocmPackages.llvm.llvm.doc (rocmPackages_6.llvm.llvm.doc)
rocmPackages.llvm.llvm.info (rocmPackages_6.llvm.llvm.info)
rocmPackages.llvm.llvm.man (rocmPackages_6.llvm.llvm.man)
rocmPackages.llvm.mlir (rocmPackages_6.llvm.mlir)
rocmPackages.llvm.openmp (rocmPackages_6.llvm.openmp)
rocmPackages.llvm.openmp.doc (rocmPackages_6.llvm.openmp.doc)
rocmPackages.llvm.openmp.info (rocmPackages_6.llvm.openmp.info)
rocmPackages.llvm.openmp.man (rocmPackages_6.llvm.openmp.man)
rocmPackages.llvm.polly (rocmPackages_6.llvm.polly)
rocmPackages.llvm.polly.doc (rocmPackages_6.llvm.polly.doc)
rocmPackages.llvm.polly.info (rocmPackages_6.llvm.polly.info)
rocmPackages.llvm.polly.man (rocmPackages_6.llvm.polly.man)
rocmPackages.llvm.pstl (rocmPackages_6.llvm.pstl)
rocmPackages.llvm.rocmClangStdenv (rocmPackages_6.llvm.rocmClangStdenv)
rocmPackages.miopen (rocmPackages.miopen-hip ,rocmPackages_6.miopen ,rocmPackages_6.miopen-hip)
rocmPackages.rccl (rocmPackages_6.rccl)
rocmPackages.rocalution (rocmPackages_6.rocalution)
rocmPackages.rocblas (rocmPackages_6.rocblas)
rocmPackages.rocdbgapi (rocmPackages_6.rocdbgapi)
rocmPackages.rocdbgapi.doc (rocmPackages_6.rocdbgapi.doc)
rocmPackages.rocfft (rocmPackages_6.rocfft)
rocmPackages.rocgdb (rocmPackages_6.rocgdb)
rocmPackages.rocm-cmake (rocmPackages_6.rocm-cmake)
rocmPackages.rocm-comgr (rocmPackages_6.rocm-comgr)
rocmPackages.rocm-core (rocmPackages_6.rocm-core)
rocmPackages.rocm-device-libs (rocmPackages_6.rocm-device-libs)
rocmPackages.rocm-docs-core (rocmPackages_6.rocm-docs-core)
rocmPackages.rocm-docs-core.dist (rocmPackages_6.rocm-docs-core.dist)
rocmPackages.rocm-runtime (rocmPackages_6.rocm-runtime)
rocmPackages.rocm-smi (rocmPackages_6.rocm-smi)
rocmPackages.rocm-thunk (rocmPackages_6.rocm-thunk)
rocmPackages.rocminfo (rocmPackages_6.rocminfo)
rocmPackages.rocmlir (rocmPackages_6.rocmlir)
rocmPackages.rocmlir-rock (rocmPackages_6.rocmlir-rock)
rocmPackages.rocmlir.external (rocmPackages_6.rocmlir.external)
rocmPackages.rocprim (rocmPackages_6.rocprim)
rocmPackages.rocprofiler (rocmPackages_6.rocprofiler)
rocmPackages.rocr-debug-agent (rocmPackages_6.rocr-debug-agent)
rocmPackages.rocsolver (rocmPackages_6.rocsolver)
rocmPackages.rocsparse (rocmPackages_6.rocsparse)
rocmPackages.rocthrust (rocmPackages_6.rocthrust)
rocmPackages.roctracer (rocmPackages_6.roctracer)
rocmPackages.rocwmma (rocmPackages_6.rocwmma)
rocmPackages.rpp (rocmPackages.rpp-hip ,rocmPackages_6.rpp ,rocmPackages_6.rpp-hip)
rocmPackages.rpp-cpu (rocmPackages_6.rpp-cpu)
rocmPackages.rpp-opencl (rocmPackages_6.rpp-opencl)
rocmPackages.tensile (rocmPackages_6.tensile)
rocmPackages.tensile.dist (rocmPackages_6.tensile.dist)

I'm running this PR + a patch to ollama and it works very well.
What's left before we can land this?

I'm running this PR + a patch to ollama and it works very well.
What's left before we can land this?

If there is some good way to mark ROCm 5.7 as deprecated I would still like to do that: https://github.com/NixOS/nixpkgs/pull/287846#issuecomment-1975496324. If not, i guess we have to skip that.
I would like to add myself to the ROCm team in maintainers/team-list.nix.
I assume that migraphx not building is acceptable and
https://github.com/NixOS/nixpkgs/issues/276846 can be looked at later.

In terms of review I have pinged Tungsten842 again and I guess also pinging the maintainers for review would be a good idea.

I think this in terms of content this PR is ready to get merged now.
I just want to allow a bit more time for reviews and for people to give input on those remaining issues.

I'm currently running nixpkgs-review, but from what I can tell the changes looks good. So if everything builds I would be happy to merge this.

Technically some reviews happened that left comments and not approvals.
I think I addressed those comments reasonably, but I don't know if I should still wait for those reviewers approve or leave further comments.
@ulrikstrid, if you say those reviewers left their comments and we don't have to go through the motions of waiting for their approval I'm fine with you merging this now. I would give them some more time, because I don't have that confidence, but I get that there is also a cost to dragging things out unnecessarily.

These are the results of my run (of just the commit, not merging the PR into master).
The build of rocprofiler for both 5.7 and 6.0 seems a bit flaky. With ´nixpkgs-review´ it fails quite often for me.
 There are no other open review comments, besides @Tungsten842's.
Using my own judgement about that issue and the discussion around it, I think we are at the point where we should merge.
I think most remaining issues listed in the top comment are worse than still using GCC 12, and we are also not waiting for those to be fixed either.

Lets ship it then!

Great job :tada:

torchWithRocm still depends on miopen-opencl so it now fails with 'miopen-opencl' has been deprecated.

Also rocblas times out on hydra: https://hydra.nixos.org/build/254017164

And composable_kernel exceeds the output limit: https://hydra.nixos.org/build/254015643

Also rocblas times out on hydra: https://hydra.nixos.org/build/254017164

I don't understand why rocblas times out, but other derivations that take much longer to build, like composable_kernel, do not.
rocsovleris the only ROCm package that I could find that has an explicit timeout set:
https://github.com/mschwaig/nixpkgs/blob/9a4f48bb251a2275293a85611a503b46bbdcf9cb/pkgs/development/rocm-modules/6/rocsolver/default.nix#L88-L98
How should we address this issue?

And composable_kernel exceeds the output limit: https://hydra.nixos.org/build/254015643

Why is composable_kernel too large?
$ du --max-depth 1 -h /nix/store/sfybgqj64vhgp386fd3xw0fkjx0wvzwf-composable_kernel-6.0.2
3.7M	/nix/store/sfybgqj64vhgp386fd3xw0fkjx0wvzwf-composable_kernel-6.0.2/include
30K	/nix/store/sfybgqj64vhgp386fd3xw0fkjx0wvzwf-composable_kernel-6.0.2/share
681M	/nix/store/sfybgqj64vhgp386fd3xw0fkjx0wvzwf-composable_kernel-6.0.2/lib
684M	/nix/store/sfybgqj64vhgp386fd3xw0fkjx0wvzwf-composable_kernel-6.0.2

That's weil below the 3 GB limit, and I though I would be surprised if symlinked contents counted I also cannot find any:
$ find /nix/store/sfybgqj64vhgp386fd3xw0fkjx0wvzwf-composable_kernel-6.0.2 -type l -ls
$
``

The NAR-size reported by nix-tree is orders of magnitude larger (3.69 GiB instead of 684 MB):
┌────────────────────────────┬────────────────────────────┬────────────────────────────┐
│                            │composable_kernel-6 3.69 GiB│                            │
│                            │                            │                            │
└────────────────────────────┴────────────────────────────┴────────────────────────────┘
/nix/store/sfybgqj64vhgp386fd3xw0fkjx0wvzwf-composable_kernel-6.0.2
NAR Size: 3.69 GiB | Closure Size: 3.69 GiB | Added Size: 3.69 GiB
Immediate Parents: -

Turns out I should have used the --apparent-size flag with du to get the actual size of the uncompressed data on ZFS, so the composable_kernel output is (probably) really too large for the cache.

Turns out the issue is one huge .a file, which can actually be compressed quite nicely.
zstd ck6/lib/libdevice_operations.a -o libdevice_operations.a
ck6/lib/libdevice_operations.a :  4.94%   (  3.69 GiB =>    186 MiB, libdevice_operations.a)

And migraphx and miopen, the only downstrem dependencies that I found, do not retain a runtime dependency on libdevice_operations.a and are also nowhere near that offensively large.
So I am actually tempted to compress that file in the output of the original build derivation, and then have another derivation based on runCommandLocal to decompress it.
Is that the kind of evil sorcery that would be allowed in nixpkgs?


Regression nag on bug that's not a regression

The new is_regression flag might make this obsolete, but...this comment is an example of a nag where the reporter wrote "This issue is not a regression" in the description.

Should be more precise now.


Tests for formatPercentage in ffe-formatters fail under certain conditions

Two test cases of ffe-formatter fail on certain yet to be discovered conditions on two machines in my possession (Linux + Mac). It seems Number.toLocaleString prefixes negative numbers with either an hyphen minus (char code 45) or a minus sign (char code 8722), whereas the later causes the test to break.
https://github.com/SpareBank1/designsystem/blob/209c8f5d7d7e6d1d19d308b779fe35b75338016c/packages/ffe-formatters/src/formatPercentage.spec.js#L17
https://github.com/SpareBank1/designsystem/blob/3e3cff94ade0f398dc24f22c0df75a9d34d62c69/packages/ffe-formatters/src/formatPercentage.spec.js#L33-L35
https://github.com/SpareBank1/designsystem/blob/209c8f5d7d7e6d1d19d308b779fe35b75338016c/packages/ffe-formatters/src/formatPercentage.js#L12-L15

What locale do you have set on those machines?

en-US, but should it matter as toLocaleString gets called with nb-NO?

are you running the same version of node on both machines? In that case, which version are you running?

v9.6.1 and v8.9.1. I've had this problem before, but chose to ignore it as it was on my private machine and it seemed to work everywhere else.

Could it be due to differences in the way node implements Date.prototype.toLocaleString?


libgit2 fixed upstream

What does this PR try to resolve?
This "FIXME" points to https://github.com/libgit2/libgit2/issues/2514 witch has bean fixed for... 8 years. So maybe it is no longer needed.
How should we test and review this PR?
By adding panic messages I was able to demonstrate that there were tests hitting this code path. Then I removed the "FIXME" and had the test still pass.

shrug tests pass
@bors r+

:pushpin: Commit 205fd1a765fe967e6d6d732303b126934ef74744 has been approved by epage
It is now in the queue for this repository.

:hourglass: Testing commit 205fd1a765fe967e6d6d732303b126934ef74744 with merge 7541cc7afdc2107f1802b64e0b28f9a5758ea193...

:sunny: Test successful - checks-actions
Approved by: epage
Pushing 7541cc7afdc2107f1802b64e0b28f9a5758ea193 to master...


Add info in getting started docs to use matcher

Description
I wasn't able to use smock matchers in my tests. I wasn't able to resolve my issue reading the doc, but then I noticed that in the "Basic Usage" section of the README.md file there is this line chai.use(smock.matchers).
IMO it's a good idea to include this in the official doc as well.

we usually don't release just docs or typos, please leave it open so it can be included on next release

Closed in favor of #186 , ty for the contribution!


uv pip compile --resolution lowest-direct treats transitive dependencies of file dependencies as direct.

MWE:
Lead with the bit that's interesting! After a bit of head-scratching it turned out to be quite simple:
# one/pyproject.toml
[project]
name = "one"
version = "0.0.1"
dependencies = ["two @ file://${PROJECT_ROOT}/../two"]

# two/pyproject.toml
[project]
name = "two"
version = "0.0.1"
dependencies = ["typing_extensions>=4.5.0"]

then
one❯ uv pip compile pyproject.toml --resolution lowest-direct
Resolved 2 packages in 86ms
# This file was autogenerated by uv via the following command:
#    uv pip compile pyproject.toml --resolution lowest-direct
two @ file://${PROJECT_ROOT}/../two
    # via one (pyproject.toml)
typing-extensions==4.5.0
    # via two

Expected behaviour: to get typing-extensions==4.12.2 (at time of writing).
Versions: I've reproduced this with uv at HEAD, 0.2.10, 0.2.4, 0.1.34.
Other variations:
The same bug also occurs in all of the following scenarios:

When expressed on the command line:

one❯ printf 'two @ file://${PROJECT_ROOT}/../two' | uv pip compile - --resolution lowest-direct

When expressed editably (side note, this one is our actual use-case).

one❯ printf '-e file://${PROJECT_ROOT}/../two' | uv pip compile - --resolution lowest-direct

When not using PROJECT_ROOT:

one❯ printf '-e file:///absolute/path/to/two' | uv pip compile - --resolution lowest-direct

(FWIW we found this problem as the resulting requirements.txt files go on to be used as constraints in one of our later build steps.)

I also encountered this same behavior yesterday, specifically in the context of "3.", and while surprised I personally found it useful, so wasn't sure if it was intentional or not.
If I have a local package I am passing into the requirement, uv can only see one version of it, so it made sense to me it would look for the lowest direct dependencies of that package, and was a better test for my environment.
But it is surprising, and not the behavior I would expect from the description.

Yeah this is intentional -- we consider anything local to be first-party, i.e., conceptually part of your local workspace.

Is this a real problem or undesirable behavior in your use-case, or just something that needs to be better-documented? (I'm not fully convinced that we should change it but open-minded of course.)

At least for us it's undesirable behaviour! What's going on is that we have several libraries that depend on each other. Most of the time we express this as a git dependency. But during local development, if making a cross-cutting change, then we switch that into an editable dependency. So
(a) ideally such dependencies would behave the same as normal PyPI dependencies, as they're still conceptually separate libraries;
(b) if I've grokked this correctly then I think the nature of our dependency resolution actually changes depending on whether we run uv compile whilst the editable dependency is in place or not!

Yeah, if you run with --resolution lowest-direct, then what's considered "direct" will change if you switch from using a Git dependency to a local dependency, because one is considered direct and the other is not.
I mean, the weird thing is, given:
# one/pyproject.toml
[project]
name = "one"
version = "0.0.1"
dependencies = ["typing-extensions"]

And then uv pip install ./one, typing-extensions isn't technically a direct dependency either -- it's a transitive dependency, the direct dependency is one @ file:///path/to/tone. But intuitively it should be treated as a direct dependency, right?  Similarly, given uv pip install ./one vs. uv pip install "one @ git+https://..., we treat typing-extensions as direct in the first case but not in the second case, so switching the source does again change what's considered direct. Separate from whether this should be applied transitively as it is now, do you think both of those decisions are correct?

But intuitively it should be treated as a direct dependency, right?

I don't think so -- at least subjectively by own intuition! :D
Like you say, these are transitive dependencies, I don't have a mental model that differs from that.

Similarly, given uv pip install ./one vs. uv pip install "one @ git+https://..., we treat typing-extensions as direct in the first case but not in the second case, so switching the source does again change what's considered direct.

Likewise, this doesn't fit my intution either. I don't expect where a dependency is located to affect anything whatsoever. (Beyond the possible need to make a network request.)

I would say that yesterday I arrived at the same mental model as @charliermarsh.
The local projects are projects that I own, so while these local project direct dependencies are transitive of the requirements I feed to uv, they are still the direct dependencies of my projects, which is more useful to my use case at least.
CC @potiuk, is airflow's CI use of lowest-direct referring to local files? A change in behavior here could break your setup.


Add read-all default permissions

Fixes 4/5 high issues in scorecard

It's infuriatingly hard to understand from https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#permissions what the default permissions are to understand whether this is actually changing anything or not :-(

If you specify the access for any of these scopes, all of those that are not specified are set to none.

I think the default permissions are just access to everything, only once you set a specific permission, do the others get disabled.


Add missing @throws JsonException in json_encode/decode()

Please see #634 and https://youtrack.jetbrains.com/issue/WI-45642
Отправлено с iPhone

7 окт. 2019 г., в 23:00, Benjamin Morel<EMAIL_ADDRESS>написал(а):
You can view, comment on, or merge this pull request online at:
https://github.com/JetBrains/phpstorm-stubs/pull/667
Commit Summary
Add missing @throws JsonException in json_encode/decode()
File Changes
M json/json.php (2)
Patch Links:
https://github.com/JetBrains/phpstorm-stubs/pull/667.patch
https://github.com/JetBrains/phpstorm-stubs/pull/667.diff
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

@b1rdex Thanks for the pointer! Closing this one.


Logistic Regression Student Admission notebook (C++).

Logistic Regression Student Admission notebook (C++).

Link to test - http://lab.mlpack.org/v2/gh/zoq/examples/student-lr-cpp?urlpath=lab%2Ftree%2Fstudent_admission_regression_with_logistic_regression%2Fstudent-admission-logistic-regression-cpp.ipynb

This is another awesome example with some nice visualizations! For this one, I'm not sure if it makes sense to try and convert it to raw C++, but I imagine it too can be converted to other languages pretty easily. But again, maybe not worth the effort for this PR, so we could open an issue for it too. 👍

Ideally we could convert the notebook into all other bindings include a raw C++ version, without the plot it's not as insightful as with the plot, but still a useable example. But I agree let's open a separate PR for this.

This notebook is ready from my side.

Alright - http://lab.mlpack.org/v2/gh/zoq/examples/student-lr-cpp?urlpath=lab%2Ftree%2Fstudent_admission_regression_with_logistic_regression%2Fstudent-admission-logistic-regression-cpp.ipynb works again.

Thanks everyone for the review.


Not all charactors allowed from XML spec as nameChar are not supported in Ballerina grammer

Description:
All NameChar, NameStartChar characters supported in W3C XML specification are not supported in ballerina grammar.
https://www.w3.org/TR/xml/#NT-NameStartChar
From XML Spec:

[4]  NameStartChar  ::= ":" | [A-Z] | "_" | [a-z] | [#xC0-#xD6] | [#xD8-#xF6] | [#xF8-#x2FF] |
    [#x370-#x37D] | [#x37F-#x1FFF] | [#x200C-#x200D] | [#x2070-#x218F] |  [#x2C00-#x2FEF] |
    [#x3001-#xD7FF] | [#xF900-#xFDCF] | [#xFDF0-#xFFFD] |  [#x10000-#xEFFFF]
[4a]  NameChar  ::=  NameStartChar | "-" | "." | [0-9] | #xB7 | [#x0300-#x036F] | [#x203F-#x2040]

From ballerina lexer:
fragment
NameChar
    :   NameStartChar
    |   '-' | '_' | '.' | DIGIT
    |   '\u00B7'
    |   '\u0300'..'\u036F'
    |   '\u203F'..'\u2040'
    ;

fragment
NameStartChar
    :   [a-zA-Z]
    |   '\u2070'..'\u218F'
    |   '\u2C00'..'\u2FEF'
    |   '\u3001'..'\uD7FF'
    |   '\uF900'..'\uFDCF'
    |   '\uFDF0'..'\uFFFD'
    ;

@rdhananjaya xml has been modified to start with a '_' by  https://github.com/ballerina-platform/ballerina-lang/pull/14164

@SupunS We need to support this in new parser.

This is supported in the new parser


XSS for img tag

Describe the issue
Hi everyone,
XSS is not fixed for all cases can you please consider this one for all modules <img src=y onerror="alert(1)">.
For now we can use the sanitize-html package to fix this one.
Procedure version
4.11.0
Reproduction link
No response

Fixed in 4.11.3


[WIP] Nested params

Ref #15

[x] Implement new nested parameters logic.
[x] Change integration specs with using new logic.
[ ] Add new documentation page about parameters.
[ ] Сheck for correctness all docs snippets.

let's rename ByBlockFilter and ByKeyFilter to just BlockFilter and KeyFilter respectively.

I agree :+1:

And for AllowFilter and DenyFilter, maybe, it would be better to use something like PassFilter and NullFilter.

I think NullFilter not very good  class name, because it's sounds like EmptyFilter or WithoutFilter(

I think NullFilter not very good class name, because it's sounds like EmptyFilter or WithoutFilter(

What about using Proxy instead of Filter? Then NullProxy makes sense

What about using Proxy instead of Filter? Then NullProxy makes sense

Hm, I like it :+1:

@palkan It looks like we're done. Can you look again?

@palkan done :+1: Feel free to merge to master)


its too good!

Its really good!

Thank you for bringing this to my attention, I will work on making it worse. :)

Marking this as completed because the script is now broken. Will reopen this issue when I get it working again. Glad to know your issue was fixed though! :) @yesman2000s

Will reopen this issue when I get it working again.
Been working a while i just forgor 💀


Skip compiler activation (on Linux) for 4.0.x

Checklist

[x] Used a personal fork of the feedstock to propose changes
[x] Bumped the build number (if the version is unchanged)
[ ] Reset the build number to 0 (if the version changed)
[x] Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
[x] Ensured the license file is being packaged.

Fixes https://github.com/conda-forge/ctng-compiler-activation-feedstock/issues/74

Backports PR ( https://github.com/conda-forge/r-base-feedstock/pull/213 ) to 4.0.x.

Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

@conda-forge-admin, please re-render

Thanks Devon! 🙏


Please invite me to the GitHub Community Organization

Name
Steven Roy
Discord Username (if applicable)
steven_roy
Additional Context
As a web & blockchain full stack developer, I am wondering about your organization.

It's great having you contribute to this project
If you would like to continue contributing to open source and would like to do it with an awesome inclusive community, you should join our Discord Server and our GitHub Organisation - we help and encourage each other to contribute to open source little and often 🤓 . Any questions let us know.

Invitation sent for the GitHub Organisation. Welcome to the community 🎉#
Don't forget after accepting to make it public so it appears on your GitHub profile for everyone else to see, you can do this by finding your name in the GitHub organisation list and change the dropdown to public https://github.com/orgs/jaidevstudio/people
Like this

Tips for practising:
Remember contributing to open source is not just about code, its about collaboration, communication and adding value
I hope that helps

Please check your email or organisation page 🙏

Duplicate issue


docs: mention image optimization in the intro

I sent against the v2 branch so I don't forget to merge it in there from master later. It doesn't really matter which it goes to as it will get deployed eventually either way

Are you implying that you don't want to merge master into version-2?

I'm not sure if we'll need to. We should if any PRs are merged there, but I don't know if any will be besides deprecation warnings. I think we should merge deprecation warnings to master at the last second and avoid merging those to version-2 because they'll all cause merge conflicts and won't be needed

Sounds fine to me. Still, I'll leave this one targeting version-2 since that's the way I started it, so that it doesn't cause issues changing the branch


new GO term for mol func - kleisin

Sir:
I suggest to include the term "kleisin" into the Molecular
Function category.
Definition: a factor that promotes the formation of ring-
like structure by other proteins.
Database refs: see Table 1 in Schleiffer et al (2003).
GO hierarchy:
cohesin
condensin
Reference:
Schleiffer A, Kaitna S, Maurer-Stroh S, Glotzer M,
Nasmyth K, Eisenhaber F. Related Articles, Links
Kleisins: a superfamily of bacterial and eukaryotic SMC
protein partners.
Mol Cell. 2003 Mar; 11(3): 571-5.
PMID: 12667442 [PubMed - indexed for MEDLINE]
Best wishes,
Mike Primig
Reported by: primig
Original Ticket: "geneontology/ontology-requests/1486":https://sourceforge.net/p/geneontology/ontology-requests/1486

thanks www.movied.org


Data_Files.set_File_Data - allows creation of files with any extension

Related to #19 and #20 , at the moment there is no limitations on the type of files that can be saved.
According with the current design, the only file paths that should be supported are .json files
Here is the test that proves the issue
  it 'Issue 23 - Data_Files.set_File_Data - allows creation of files with any extension', ->
    using new Data_Files(), ->
      create_File = (extension)=>
        file_Name     = 10.random_String() + extension
        file_Contents = 10.random_String()
        file_Path     = @.data_Path .path_Combine(file_Name)

        @.set_File_Data file_Name, file_Contents               # PAYLOAD: create file

        file_Path.assert_File_Exists()                         #   confirm file exists
                 .file_Delete().assert_Is_True()               #   delete temp file

      create_File '.json'                                      # these are the ones that should work

      create_File '.json5'                                     # these are the ones that should NOT work
      create_File '.coffee'
      create_File '.js'
      create_File '.exe'
      create_File '.html'
      create_File '.css'
      create_File '...'

This is now fixed due to the fact that the fix for #19 removed the ability to create new files
Here is the regression test
  it 'Issue 23 - Data_Files.set_File_Data - allows creation of files with any extension', ->
    using new Data_Files(), ->
      create_File = (extension)=>
        file_Name     = 10.random_String() + extension
        file_Contents = 10.random_String()
        file_Path     = @.data_Path.path_Combine(file_Name)

        @.set_File_Data file_Name, file_Contents               # PAYLOAD: create file

        file_Path.assert_File_Not_Exists()                     # confirm file doesn't exists

      create_File '.json'                                      # confirm that none work now
      create_File '.json5'
      create_File '.coffee'
      create_File '.js'
      create_File '.exe'
      create_File '.html'
      create_File '.css'
      create_File '...'


Fix preview page for promotions

Fix for previewing a promotion page

Resolves: [MWPW-129761]
Test URLs:

Before: https://stage--cc--adobecom.hlx.page/creativecloud/promotions/hub/ccx-design-links?martech=off
After: https://stage--cc--ruchika4.hlx.page/creativecloud/promotions/hub/ccx-design-links?martech=off

Codecov Report

Merging #28 (fddf303) into stage (4eb24ba) will not change coverage.
The diff coverage is 100.00%.

@@            Coverage Diff            @@
##             stage       #28   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files            2         2
  Lines           14        14
=========================================
  Hits            14        14

Impacted Files
Coverage Δ

creativecloud/features/article-promotion.js
100.00% <100.00%> (ø)

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


Support for Redis

Please add support for managing Redis databases.

We already support Redis in Enterprise Edition.
We don't plan to move it to CE version in near future (as well as other NoSQL or non-JDBC based drivers).

Well thats a show stopper. Is there any recomendations for tools that do support it in freedom versions of the software?

@serge-rider CE version support Redis is urgently needed.


Instructions to use the demo in a local Kubernetes cluster

The developer use case to deploy and use on Kubernetes on their local machines is real and should be officially supported.
Instructions should recommend an easy-to-deploy Kubernetes environment (likely use kind for this). As well as Helm instructions, we should also include the Kubernetes manifests required to deploy the application (#545 )

We now have Kubernetes manifests generated from the Helm chart. The demo can be installed to a Kubernetes cluster using
kubectl apply -f kubernetes/opentelemetry-demo.yaml

Next is to document this on the docs page.
Note: Helm requirements is actually 3.9+ not 3.0+... update this at the same time.

This is documented now: https://opentelemetry.io/docs/demo/kubernetes-deployment/#install-using-kubectl


Map: create leader and follower pawns based on the party

Currently the characters on the LocalMap are hard-coded, they're not related to the party. Each pawn should represent a playable character, the leader being the first character in the party, and the others being followers.
The code should also provide a function to clean up and rebuild the pawns when the party order changes (to control a different character)

Also... Spawnposition for players should look up to party members  formation in order to spawn properly (there are two position2D nodes right now)

This task is about the map. Let's open another issue for the combat arena


Make Typeable a superclass of Has.

It might be necessary to support #13.

Actually, Has was fine as it is, but the Typeable constraints were required on Phased.

Added in d3756835cf5a18e1a84d88a3488c1876759131f9.


podman: unknown diff.hide value

$ dive localhost/foobar --source podman
Using config file: /home/bk/.config/dive/podman.yaml
Image Source: podman://localhost/foobar
Fetching image... (this can take a while for large images)
Analyzing image...
Building cache...
unknown diff.hide value: changed

$ echo $?
1

$ $ podman image ls -a
REPOSITORY                      TAG      IMAGE ID       CREATED          SIZE
localhost/foobar                latest   f74515516328   11 minutes ago   276 MB
<none>                          <none>   7433a359c00e   11 minutes ago   276 MB
<none>                          <none>   78c3758c6fe7   11 minutes ago   276 MB
<none>                          <none>   14487ac36d5d   11 minutes ago   276 MB
<none>                          <none>   0383632b6d58   12 minutes ago   246 MB
<none>                          <none>   b186616c1a82   13 minutes ago   200 MB

Anything I can do?

I encounter the same error here using version dive 0.9.2
➜ dive nginx:latest
Using config file: /Users/mrivet/.dive.yaml
open ~/dive.log: no such file or directory
Image Source: docker://nginx:latest
Fetching image... (this can take a while for large images)
Analyzing image...
Building cache...
Failed to write to log, invalid argument
unknown diff.hide value: changed


WIP: NEUBURG-27: Use cityConfig API

This pull request belongs to an issue on our bugtracker.
You can find it there by looking for an issue with the key which is mentioned in the title of this pull request.
It starts with the keyword WEBAPP.

Some tests are still failing and I can't figure out why exactly. Any ideas?

For the cors you can install a addon https://addons.mozilla.org/en-US/firefox/addon/cors-everywhere or https://chrome.google.com/webstore/detail/allow-control-allow-origi/nlfbmbojpeacfghkpbjhddihlkkiljbi
We already set it up for api.wohnen.integreat-app.de - the addon is only necessary for your local environment.

For the cors you can install a addon https://addons.mozilla.org/en-US/firefox/addon/cors-everywhere or https://chrome.google.com/webstore/detail/allow-control-allow-origi/nlfbmbojpeacfghkpbjhddihlkkiljbi
We already set it up for api.wohnen.integreat-app.de - the addon is only necessary for your local environment.

Does it work for you with this browser extension? I already have it installed and it still didn't work until I enabled cors in the backend.

Yes, it did work, you also have to activate it once it's installed, did you do that as well? 🤔

I'm pretty sure I activated it :D
Maybe I configured something else incorrectly.

Ok, yeah idk^^ for me the add-on is green when I activate it in the toolbar (of chrome) - maybe I didn't try Firefox yet

Just reinstalled it, now it works :)

Another way would be to proxy the server locally. More work to do initially but works probably better :)


Change all instances of unified_search to search

This commit changes all instances of unified_search to search when accessing rummager. This provides consistency between the internal and external APIs in terms of naming.
As a side effect of upgrading the version of gds-api-adapters, this commit also changes ci_reporter to ci_reporter_minitest since version 2 of this gem has been split for different test frameworks.
Trello: https://trello.com/c/cj8UX2jX

LGTM 👍


pipeline shell (psql) default port doesn't works when postgresql is installed

When postgresql client/common is installed on the system, or more specifically the libpq is installed system-wide then pipeline try to connect to port 5432 or use the default postgresql .s.PGSQL.5432 unix socket.
The thing is : the default port/socket/everything are hardcoded in libpq.
It also try to use my ~/.pgpass and all stuff from http://www.postgresql.org/docs/9.4/static/libpq-envars.html
I guess a choice need to be done somehow :

use all defaults from postgresql
or change everything (including default env var, default file, etc)

pipeline --help say :

-p, --port=PORT          database server port (default: "6543")

but running pipeline :

pipeline: could not connect to server: Aucun fichier ou dossier de ce type
Is the server running locally and accepting
connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?

and pipeline -h <IP_ADDRESS> :

pipeline: could not connect to server: Connexion refusée
Is the server running on host "<IP_ADDRESS>" and accepting
TCP/IP connections on port 5432?

pipeline -h<IP_ADDRESS> -p6543 -d pipeline :

pipeline (9.4.4)
Type "help" for help.
pipeline=#

The problem here is that pipeline symlinks to /usr/bin/psql upon installation. I think the correct thing to do here is just to link /usr/bin/pipeline to <install directory>/bin/psql instead of the system-wide binary.

hummm nope. not in my case. i used --prefix to install it to a development directory and it used the pipeline version of psql :

ker2x@lily:~/build/bin$ pipeline --help
pipeline is the PipelineDB interactive terminal.
Usage:
[...]
Connection options:
-h, --host=HOSTNAME      database server host or socket directory (default: "local socket")
-p, --port=PORT          database server port (default: "6543")
-U, --username=USERNAME  database user name (default: "ker2x")
-w, --no-password        never prompt for password
-W, --password           force password prompt (should happen automatically)
For more information, type "?" (for internal commands) or "\help" (for SQL
commands) from within psql, or consult the psql section in the PostgreSQL
documentation.
Report bugs to<EMAIL_ADDRESS>

You're definitely right @ker2x. And we'd rather maintain seamless compatibility so don't want to have our own special clients. I think the best thing to do here is to just use PostgreSQL's defaults. We originally chose 6543 specifically to not conflict with an existing installation, but this is a conflict anyways :)
It's probably more likely that a user will have a system-wide libpq installed than it is that they'll have an existing PostgreSQL server running on the same hardware as PipelineDB. If that were the case, I believe any port conflicts that would cause would be more obvious to that user anyways.
So adding to @ker2x 's suggestions, I see three options:

Use all PostgreSQL defaults

Use our own client with its own environment vars

Keep our own defaults and force clients to explicitly override them:
psql -p 6543 ...

@usmanm do you have any thoughts on this?

Well, if you really want to keep 6543 and make it behave as expected (6543 as default, as said in the --help) you can easily add :

if (!options->port)
options->port = DEF_PGPORT_STR;

at the end of :

static void
parse_psql_options(int argc, char *argv[], struct adhoc_opts * options)

to override libpq default.

I'm going to side with 1 (2 is a no go, and 3 are not ideal either). But does changing defaults also mean that we change the default database name etc? I think I like @ker2x's idea of just overriding libpq in the code.

Yep, I like that approach too.

yay! \o/

Forget about my patch, it sux. I don't see how it can be done without patching libpq.

We're just going to use standard defaults.


Missing rados API components: function rados_ioctx_get_id

The rados API implements function(s) below but go-ceph does not:

rados_ioctx_get_id

This issue has been filed to track implementing this missing
functionality in the appropriate package within go-ceph.
Please refer to the Ceph C API for details about the function(s)
in question.
NOTE: This issue was generated by a script.
If the content is innacurate or wording seems a bit odd
please excuse the script and blame the creator @phlogistonjohn

Picking this one.


Add a --host option for serve and preview

This PR adds a --host [string] command line option to the serve and preview commands that maps to Vite's --host / host: configuration parameter.
Today, the host can be configured in vite.config.ts, but cannot be configured per-execution. Adding a --host command line option allows the host to be configured per-execution.
Use case
This parameter is useful to me when running playwright visual tests in a docker container.
When listening on <IP_ADDRESS> on the host (the default), the docker container cannot access Ladle. To let the container access Ladle, either Ladle must listen on an address in docker0 or on <IP_ADDRESS>. There are problems with both:

If Ladle is configured to listen on docker0, then it fails to bind when the docker0 interface is unavailable, which occurs when docker is down or not installed.
If Ladle is configured to listen on <IP_ADDRESS>, then it is by default accessible to external hosts unless a firewall is configured to block incoming connections.

This PR enables a setup where Ladle by default listens on <IP_ADDRESS>, but can be configured to listen on <IP_ADDRESS> (an address on docker0) when running playwright visual tests in docker.
For additional context into my use case, I run the following commands to run visual tests locally and in CI:
$ docker build --tag playwright-test:latest - < playwright.Dockerfile
$ pnpm start-server-and-test \
    'pnpm ladle build && pnpm ladle preview --host <IP_ADDRESS> --port 40855' \
    'http://<IP_ADDRESS>:40855' \
    'docker run --mount type=bind,source="$(realpath .)",target=/usr/app --network host --add-host host.docker.internal:host-gateway playwright-test:latest'

Nice, thanks!


[AGE-Viewer-Go-Desktop] Update Instruction for setting development environment and build from source.

@aked21 please merge, discussed with Sarthak


feat(spanner): add numeric samples

Wait for https://github.com/googleapis/google-cloud-ruby/pull/7631.

The CI test fails due to a random error not related to this change:
  1) Error:
Google Cloud Firestore API samples - Distributed Counter#test_0002_get_document:
Google::Cloud::NotFoundError: 5:No document to update: projects/ruby-firestore-ci/databases/(default)/documents/shards/5. debug_error_string:{"created":"@1600233739.544228349","description":"Error received from peer ipv4:<IP_ADDRESS>:443","file":"src/core/lib/surface/call.cc","file_line":1062,"grpc_message":"No document to update: projects/ruby-firestore-ci/databases/(default)/documents/shards/5","grpc_status":5}
    /usr/local/bundle/gems/google-cloud-firestore-v1-0.2.0/lib/google/cloud/firestore/v1/firestore/client.rb:818:in `rescue in commit'
    /usr/local/bundle/gems/google-cloud-firestore-v1-0.2.0/lib/google/cloud/firestore/v1/firestore/client.rb:784:in `commit'
    /usr/local/bundle/gems/google-cloud-firestore-2.1.0/lib/google/cloud/firestore/service.rb:131:in `commit'
    /usr/local/bundle/gems/google-cloud-firestore-2.1.0/lib/google/cloud/firestore/batch.rb:403:in `commit'
    /usr/local/bundle/gems/google-cloud-firestore-2.1.0/lib/google/cloud/firestore/client.rb:581:in `batch'
    /usr/local/bundle/gems/google-cloud-firestore-2.1.0/lib/google/cloud/firestore/document_reference.rb:404:in `update'
    /tmpfs/src/github/ruby-docs-samples/firestore/distributed_counters.rb:50:in `increment_counter'
    /tmpfs/src/github/ruby-docs-samples/firestore/acceptance/distributed_counters_test.rb:38:in `block (3 levels) in <top (required)>'

@TheRoyalTnetennba if you could help us merge in this one, that would be great. Merging is currently blocked on an unrelated firestore test failure.


Can you add a licence to your project?

Hello 👋
First, thank you very much for providing your code, I'm myself comptemplating moving everything to Obsidian 🙏
Without a licence your code follows the traditional copyright law and prevent most usage of said code.

Unless you include a license that specifies otherwise, nobody else can copy, distribute, or modify your work without being at risk of take-downs, shake-downs, or litigation. Once the work has other contributors (each a copyright holder), “nobody” starts including you.

Can you clarify what licence your code fall under?
If you don't know which to choose, it seems that the most prevalent one those days is the MIT one, but that your decision only 🙂

I deliberately didn't choose a license - didn't think anyone would (re)-use this.
I'll throw in an MIT license, as I do with most other projects.


cannot use with distributed pytorch

the id "id name i gave" already exists by one process so rest all workers stop.

Hi,
If you are using multiple gpus, you should modify the parameter --nproc_per_node in the training scripts. For example, for training with 2 gpus:
CUDA_VISIBLE_DEVICES=0,1 python -m torch.distributed.launch --nproc_per_node=2 --master_port 12345 train.py --config path/to/config.yml

Meanwhile, please make sure the entry id defined in the config yaml file is unique for each experiment. We create a unique folder named ${model_name}/${id}, so if the id is duplicated (given that ${model_name} is not changed), the program cannot create the logging folder. If that is the case, you should either delete the previous logging folder with the same id, or use another id for creating a new logging folder.


Add a few csx genes, fix incorrect cas5/6

There were a few csx genes that I missed from reference 3, which are added here. In addition, reference 2 nominally coined "cas5" and "cas6", however, they were referring to what are now known as cas9 and cas2, respectively. This commit fixes those incorrect categorizations.

I confirmed the cas5/cas9 and cas6/cas2 conflation in reference 2 by looking up those gene accessions (str0657 and str0659) and BLASTing them.


Setting taskbar thumbnail image

Hey all,
When something is being played on media player for example, when one hovers the mouse over media player's icon on the taskbar, he can see what is being played and not the entire window of media player (including everything), the only things that show are the cover art of the music, the title and the buttons to be clicked (previous, play/pause, next).
When one uses the setThumbarButtons() from the atom/electron library, it only adds the buttons under the image of the entire window.
How can we set the image to show when we hover the mouse over the taskbar's application icon ? (and do something like media player).
Thank you for your time,
Best Regards,
Hussein Hammoud.

Refs https://msdn.microsoft.com/en-us/library/windows/desktop/dd391701(v=vs.85).aspx.


Implement nfts api calls (CoinGecko)

CoinGecko now supports endpoints for retrieving floor_price for nfts
https://api.coingecko.com/api/v3/nfts/list

Does this work for chains other than mainnet? Otherwise OpenSea covers this right?

Yes, it supports the other chains too, have to check which ones, but go init( CoinGecko platform) for rpc server.

What might be an issue is that it don't have an endpoint to retrieve batch of tickers for nft tokens, might have throttling of requests. Need to check it

Ok. Lower priority?

Nice, if you get it working can you create an Android issue with the endpoints?


Sync Mode

A mode where you can sync a directory on the phone with a directory on the computer, copying only files not present and updating files that have changed.

I 2nd this suggestion 👍

Thanks for the suggestion. Right now I am trying to rebuild the native MTP module of this app as it is not compatible with certain devices! Once that is completed I shall definitely look into creating a new sync feature.
If any of you is interested in contributing towards the development of the new MTP kernel of this app, which is basically written is C++, it'd be great help as I am unable to squeeze in some time for the same.

Hello ganeshrvel. Unfortunately my coding skills end with Java Script and a bit of php.
Rebuilding the native MTP module is of much higher importance because openMTP 2.0 is not working with my Samsung Galaxy A8 ROM, either (but Android File Transfer and Commander One MTP are). Even uninstalling ATF and reboot did not help.

Rather (or I guess in addition to) than adding syncing to the app itself, could you expose a mount point for the android device, so that we just open openMTP and then use rsync or whatever existing synching app we want to take care of the rest?

@gjw : https://www.reddit.com/r/apple/comments/48yb2b/guide_mounting_android_device_under_os_x_using/ (or if you like YouTube: https://www.youtube.com/watch?v=_jzMiyz4Zt4 )
(please don't expect an Orange when you pick a Banana .. ;) )

@ganeshrvel : while we're in here, two suggestions:

please make the key combo for Quit the regular Mac cmd + q. Now it's cmd + shift - q, which is not usual (and is reserved for account log-out on some if not most Macs)
please replace the 'Root' on the bottom left of both panes with the Mac default / for hard disk base level. 'root' is reserved for super user / system admin

:)

@ganeshrvel @leebinder
Here a HTC 10 user 'Android File Transfer' (that i don't want to use) connects, but openMTP doesn't connect. Same computer, system and cable.
And yes, cmd + shift + q on Mac brings a message to logout the system.
Anyway thanks for the app, it looks beautiful and i wanna try! ;-))

I have been trying to build a new MTP kernel which can detect a wider range of devices. I have created a separate repo for the same here: https://github.com/ganeshrvel/mtp-nodejs-port-native-addon/tree/to-async
Due to my limited understanding in C++ I am unable to solve the final parts of solving this issue.
If anyone can help me solve this issue which is mentioned here: https://stackoverflow.com/questions/56347264/nodejs-libuv-non-blocking-event-callback I might n be able to release a new beta version pretty soon.

@bartatgithub @LeeBinder I was going through the keymaps and I don't see "cmd + shift + q" being referenced anywhere? Could you tell me where have been asked to use "cmd + shift + q " keyboard shortcut?

I have created a new GitHub thread for further discussions and updates.

@bartatgithub @LeeBinder I was going through the keymaps and I don't see "cmd + shift + q" being referenced anywhere? Could you tell me where have been asked to use "cmd + shift + q
" keyboard shortcut?

@LeeBinder thank you for the screenshot!
I have made a new release could you check and tell me whether your devices are getting recognised or not?

@ganeshrvel as I wrote my screenshot already is from your newest 2.1 release and no, my Samsung is still not recognized.

Thanks!
openMTP 2.1.0 works with a HTC 10 (Android 7.1), hopefully on other phones too.
Be sure you use a charge + data USB cable.
Keymaps are still "cmd + shift + q", same as Logout, and work only that way, see screenshot:

One thing with the interface:
When you double click a folder to open, the checkbox go's also in on/off state. So often I have to click four times to get in a folder. Copy works.
Will test more during the day's that come.

@bartatgithub that's wonderful! Happy to know that the app works on HTC smartphones now.
Regarding the "double click" on a folder. You can just double click on any item to open not 4 times.
I will fix the quit menu issue in the next release.

Ok, tested it and it was my impression because when you double click a folder name the checkbox go's also in on/off state. Gives a weird interaction impression, that's all, so not a big problem.
Could you separate the interaction, click, on the checkbox and the interaction, click, on the folder, when you have extra time? Thanks!

@bartatgithub I have fixed the Quit Keyboard shortcuts in the beta release: https://github.com/ganeshrvel/openmtp/releases/download/v2.2.0-beta.1/OpenMTP-2.2.0-beta.1.dmg along with some amazing performance improvement.

Quit Keyboard shortcuts works, thanks!

Quit Keyboard shortcuts works now on MacOS, thanks!

https://github.com/ganeshrvel/openmtp/releases/download/v2.2.2/OpenMTP-2.2.2.dmg
Better - but why the deviation from standard ⌘Q for quit - why ⌥⌘Q? What's your reasoning for not simply use ⌘Q as any other Mac app?
Also, it still says 'Root > Users > ..' vs. (correct) '/ > Users > ..'. root' is referred to as something else - the root user.

Thanks for confirming,
⌥⌘Q is used instead of the standard  ⌘Q because ⌘1 keymap is used to toggle the tabs. There are chances that users might accidentally press ⌘Q while trying to press ⌘1

⌥⌘Q: really? How many users don't take off their gloves before hitting the keyboard .. ;)?
Request: can you please make this shortcut configurable as in shortcut prefs?
Root vs. /: simply add space characters around the / and have the clickable path be " / ". Three characters is enough for people even with paws, parkinsons, or panic syndrome .. ;)

Strange in the beta release it's really ⌘Q and works also in that way?
@LeeBinder: Download and install that on, you will see.

@bartatgithub : neither in beta 1 nor in beta 2 it's ⌘Q ...

I see it right now on my screen and it works.
MacOS High Sierra
OpenMTP 2.2.0-beta.1

@bartatgithub : not here. Ganesh must have released TWO different 2.2.0-beta.1 then:

same in 2.2.0-beta.1 and 2.2.0 final..

That's weird?
Do you have downloaded this one: https://github.com/ganeshrvel/openmtp/releases/download/v2.2.0-beta.1/OpenMTP-2.2.0-beta.1.dmg

@bartatgithub please try: https://github.com/ganeshrvel/openmtp/releases/download/v2.2.2/OpenMTP-2.2.2.dmg

I will download OpenMTP-2.2.0-beta.2.dmg and see what happens.
Do you have deleted the previous version or replaced it?

@ganeshrvel: Ok, i will first try that one.

@bartatgithub

Do you have deleted the previous version or replaced it?

not sure why this is happening. Please try 2.2.2

Yes, ⌘Q and works also on the keyboard!
OpenMTP 2.2.2

@LeeBinder: Try to first delete the previous version and install OpenMTP 2.2.2, maybe that will work?

@LeeBinder: Try to first delete the previous version and install OpenMTP 2.2.2, maybe that will work?
That works for me.

removed all OpenMTP related files with App Uninstaller then double-checked with iTrash that indeed all settings/ prefs/ caches etc. are gone. Result is the same with 2.2.2 and below.
Some Voodoo with OpenMTP which only surfaces on your system as it seems.
I'm running High Sierra 10.13.6.

I have done some more tests and this are the weird results:
iMac — MacOS High Sierra 10.13.6
OpenMTP 2.2.0-beta.1 and OpenMTP 2.2.2
---> ⌘Q and works also on the keyboard!
MacBook Pro — MacOS El Capitan 10.11.4
OpenMTP 2.2.0-beta.1 and OpenMTP 2.2.2
---> ⌥⌘Q and works only with ⌥⌘Q on the keyboard?!
Will test later on a MacBook Pro — MacOS High Sierra 10.13.6.

In the meantime, for syncing, there's an ADB solution, by Google.
https://github.com/google/adb-sync

@hacker1024 interesting, thanks for the link. Command line only, from what I see. Do you know if there is a GUI frontend?

@ganeshrvel I'd also love to see a sync feature where a source (left pane) could be used to sync with a destination (right pane). Ideally, it would offer:

Merge
Mirror (left-to-right or right-to-left)

Meanwhile, I have moved away from macOS and into the Linux world, so unsubscribe here. ;-)


Implemented upload session APIs (multi-part uploads)

Changes

implemented upload session APIs (multi-part uploads)
implemented auto-handled upload session method UploadSession() with test
renamed internal download() to content() ("Content-upload endpoints" and "Content-download endpoints")
added internal decodeContent() helper
fixed PermanentlyDelete endpoint path (not sure if it was incorrect on purpose?)

Notes

Methods should probably use value parameters, as pointer parameters allow property changes after the call has been made. Also, it's good practise to use values where possible. However, I thought it was more important to follow the current method layout.
As an alternate API to reduce the size of files.go, the raw upload session methods (start/append/finish) could be moved to files_upload_session.go and then would become Files.UploadSession.Start(), .Append(), .Finish() - though we'd have to rename UploadSession() to something else.
As an additional feature, we could add a PerformRetries bool so if any chunk failed, it would be retransmitted. This would require a ChunkSize buffer though to hold data in case of failure and maybe the Dropbox API doesn't fail that often so it might not be worth it...

# cd .; git clone https://github.com/davecgh/go-spew /home/runner/workspace/src/github.com/davecgh/go-spew
Cloning into '/home/runner/workspace/src/github.com/davecgh/go-spew'...
error: RPC failed; curl 56 GnuTLS recv error (-9): A TLS packet with unexpected length was received.

Strange failure, maybe manually retry?

Ah yes, just remembered https://github.com/tj/go-dropbox/issues/2. So maybe the higher-level UploadSession() should go in go-dropy?

niceee, yea as much as possible I think the fancier stuff should be in go-dropy

Striped out high-level UploadSession() method, moved to go-dropy, see jpillora/go-dropy@5eee26d - Will PR it over there once this lands.

Adding new test with manually managed session upload...

Semaphore seems to be a bit buggy, https://semaphoreci.com/tj/go-dropbox/branches/pull-request-18/builds/4 also failed strangely

I'll sync with upstream again once #17 tests are fixed


Integrated terminal/navigation fails to parse file paths with line/column information formatted as .

Does this issue occur when all extensions are disabled?: Yes

VS Code Version: 1.69.1
OS Version: Ubuntu 22.04

Some tools use the formatting <file path>:<line>.<character> when printing diagnostics. If you click these strings in the integrated terminal, the navigation fails.
I've seen similar failures when parsing numbers that are supposed to be integers as floats before, but I'm not sure if that's what's going on here.
Steps to Reproduce:

Create a file named file.txt

echo "Hello, world!" > file.txt

Run echo file.txt:1.7
Click on the echoed string
Notice the URI is opened in the command/navigation palette but the line is parsed as "1.7" and not "line 1, column 7."

Thanks for creating this issue! It looks like you may be using an old version of VS Code, the latest stable release is 1.70.0. Please try upgrading to the latest version and checking whether this issue remains.
Happy Coding!

This is an awfully weird format, I'm curious what tools do this. Tbh I think they should change it instead of us supporting what looks like a decimal number at the end.

lbstanza formats as line.column. Since lines/columns are never decimal numbers I feel VS Code should fix its numeric parsing to handle this edge case.

Right now VS Code fails silently and just drops the full string in the navigation palette. Even if you choose not to support this for niche tools some alternatives make more sense :

Optimistically parse the file:line:column info to jump to 0:0 if the line/columns can't be parsed
Display an error message somewhere that the operation failed

Because right now it looks like VS Code is behaving weird, not the tools reporting weird file locators, and the behavior is quite annoying.


custom input ref

Is your feature request related to a problem? Please describe.
It seems to me, that there is no way to get input ref at this moment.
Describe the solution you'd like
It will be useful, if we can get input ref simple like:
<ReactDatePicker ref={ref}/>

hey @AlexSav94 , i've implemented a custom input using forwardRef:
const CustomInput = forwardRef((inputProps, ref) => {
        const { value, onChange } = inputProps
        return (
            <div>
                <StyledInput
                    value={value}
                    onChange={onChange}
                    placeholder='Pick a date'
                    ref={ref}
                />
            </div>
        )
    })

And then pop that into the DatePicker:
<ReactDatePicker
      customInput={<CustomInput />}
/>

Hope this helps!

I ran into this issue as well because I pass a ref into my DateInput component from the parent. Here's the pseudo code for my component that made this work:
const DateInput = forwardRef((props, ref) => {
  const CustomInput = useMemo(() => props => <Input {...props} ref={ref} />, []) // <Input /> is my custom input component

  return (
    <DatePicker customInput={<CustomInput />} />
  )
})

Notice that I'm ignoring the ref that is passed to CustomInput and just forwarding the ref I care about. I'm not sure if this has negative affects on the DatePicker, though, because I'm managing the open/closed state with the open prop. It works perfectly this way, and the ref is properly passed.
If you notice issues, you can try replacing the CustomInput with this:
// ref is from the parent
// customRef comes from DatePicker
// inputRef is passed to the custom input element and updates both the other refs when it changes
const CustomInput = useMemo(() => forwardRef((props, customRef) => {
    const inputRef = useRef()

    useEffect(() => {
      if (inputRef.current) {
        if (typeof customRef === 'function') {
          customRef(inputRef.current)
        }

        if (customRef.hasOwnProperty('current')) {
          customRef.current = inputRef.current
        }

        if (typeof ref === 'function') {
          ref(inputRef.current)
        }

        if (ref.hasOwnProperty('current')) {
          ref.current = inputRef.current
        }
      }
    }, [inputRef.current])

    return <Input {...props} ref={inputRef} />
  }), [])

It's really verbose, but it did appear to work in my basic test. Good luck!


Typo in Loggi-n601-k42.vrp

It looks like the below file Loggi-n601-k42.vrp has NAME field as Loggi-n600-k42 different from the file name.
https://github.com/laser-ufpb/CVRPController/blob/2ed0d6229b08e4a45b5f9a89447343a5985ef6cd/InstancesExplicitDistances/Loggi-n601-k42.vrp#L1
The controller creates outputs based on NAME field. Because of that, the files were sorted slightly differently when we populated PI values in google sheet. // A very minor thing though, just thought of sharing.
Thanks,
Rajesh.

It was fixed. Thanks, @mrprajesh


2.2 Preview 3 - DbHealthCheck supported scenario?

We have a MicroService SaaS application running in azure using a Azure SQL Elastic pool for its database. Each customer has their own database because of compliance requirements.
The connection string is set during dB context creation, based on the customers tenant Id.
We want to create a health check page for the customer, but we only want to run the health check for a specific tenant Id. Tags will help us with this but will the new dbcontext health check support this scenario?
It would be awesome if you could create your own “connection string factory” and override a default one.

/cc @divega @ajcvickers
How does your code construct a DbContext today to accomplish this?

We have created a custom IDbContextOptionsBuilder that gets the connectionstring from Azure KeyVault.
Our DbContext then uses it in its override OnConfigure method.
It’s a lot of code just to set the connection string and dbcontext pooling needs to be disabled.
I have always felt that the “one db per tenant” scenario isn’t fully supported today and I’m hoping it will get some more love in later releases :)

Sorry, perhaps I wasn't clear - what does you your code look like that obtains a DbContext - for instance how does the infrastructure you described know which tenant it belongs to?

One thought that I had here - do you really need the health checks system for what you want to accomplish? Does it make more sense for you to retrieve a DbContext as you normally do an call CanConnectToDatabase? If your goals are limited to giving users a way to test database connectivity for their tenant, this is a really lightweight way to do that.
Otherwise, it sounds like you could make this work but there are a few thing you will need to know:

I assume since you mentioned the HttpContext that you're using IHttpContextAccessor to get the currently executing request to find the tenant. If this is not the case then I need more information.
The health checks service creates its own DI scope and resolved/runs checks in there. It does not share scoped services with the current request.

Assuming you're using the middleware, if your tenant info is assigned to the HttpContext before the health checks middleware runs this should all just work. If you're calling the health checks service yourself, the same requirements apply - if you're inside a request, then the http context should be picked up and used when your DbContext is created.
I'd also encourage you to think about if there are cases where you want to run health checks for a request that doesn't have a tenant associated. You might want two middleware - one that runs for the app as a whole, and one that runs with per-tenant information.

Thank you for your elaborated answer.
Yes we are using IHttpContextAccessor to get the AccessToken containing the claims for the user.
As you mentioned, we actually have two use cases.

A customer wants to see the health for their system. That includes their azure database, their azure storage account, some (but not all) of the micro services and perhaps a custom ftp server address where they are receiving files.
We, as a SaaS provider, wants to see the health of all services and databases.

I see our health check model as a organisation chart where our “Maintenace” Micro Service, who is in the top,  starts the checks by calling “check health” for each of our Micro Services. Each MS checks the health of the services that it is using, and soo on. (Our system is using composit services)
Are Health Checks designed for these kind of scenarios or do you think a custom implementation would be better?

Sorry I lost track of this for a few days.
I think health checks are totally fine for this. We provide the IHealthChecksService so that you can call it in ways that make sense for you and show the results any way you want.  I expect that some of your scenarios will require writing some custom IHealthCheck implementations.
Since you will be writing code on both sides (eg: both calling IHealthChecksService and implementing checks) you might want to short circuit some of that by just using the abstractions and types yourself. I would still recommend trying to make this work with the built in implementation of the service first. We provide some niceties around logging, timing, cancellation, etc.

If I were in your position I would model this in the following way:

to check the health of a tenant register the appropriate checks - set up the system in such a way that they automatically detect the correct settings for the tenant based on the IHttpContextAccessor - it sounds like you've already got this figured out for the DbContext. Apply a tag to these checks so they are easy to group when calling the service.

for your overall service monitoring, I'd suggest writing your own checks that loop through the configurations and call the appropriate touch points. You could choose to model this as a composite or not.

I hope it doesn't seem like I'm asking a lot by suggesting that you author some of your own checks... My recommendation is that you think of them as simple glue code that adapts some underlying API to the health check service.  For an example of this, take a look at the DbContext health check - it's basically all glue code and configuration - all of the real logic in a method that EF added for us.

I think I've answered most of the questions here so far, and we can get more concrete if you have some examples or a simple mock up (repo) of what this might look like. Basically if you have some code, I'm happy to take a look and make suggestions.

Thanks for contacting us. We believe that the question you've raised have been answered. If you still feel a need to continue the discussion, feel free to reopen it and add your comments.


Accept pem as binary data.

As an alternative to supplying the pem path with APNS.pem =,
accept the pem binary data with APNS.pem_data =.

Closing this PR as it has been open for 18 months with no response by the maintainer.


CI improvement

A new inspection was created.

The inspection completed: No new issues

[Current coverage][cc-pull] is 63.24% (diff: 100%)

Merging [#131][cc-pull] into [develop][cc-base-branch] will not change coverage

@@            develop       #131   diff @@
==========================================
  Files            23         23
  Lines           936        936
  Methods           0          0
  Messages          0          0
  Branches          0          0
==========================================
  Hits            592        592
  Misses          344        344
  Partials          0          0

Powered by Codecov. Last update [d7cca8e...421e849][cc-compare]
[cc-base-branch]: https://codecov.io/gh/whtsky/WeRoBot/branch/develop?src=pr
[cc-compare]: https://codecov.io/gh/whtsky/WeRoBot/compare/d7cca8e2cf21c819a5c0e42c38450130d5fb7e53...421e8495e0c81aa30d844526b6ec93315e991994?src=pr
[cc-pull]: https://codecov.io/gh/whtsky/WeRoBot/pull/131?src=pr

A new inspection was created.

The inspection completed: No new issues