[LAPACK] build failure

Host Environment

OS: windows
Compiler: vcpkg-2021.05.12

To Reproduce
Steps to reproduce the behavior:
`./vcpkg install lapack:x64-windows
Failure logs
Please ensure you're using the latest portfiles with .\vcpkg update, then
submit an issue at https://github.com/Microsoft/vcpkg/issues including:
Package: lapack:x64-windows
Vcpkg version: 2021-05-05-9f849c4c43e50d1b16186ae76681c27b0c1be9d9
Additionally, attach any relevant sections from the log files above.
Additional context
This error occured when I tried to vcpkg ceres solver, then I remove all package and tried to only vcpkg lapack, it occured again.
this log I can't understand well, thank you for your help!

Please attach the failure log first.
Thanks.

We don't have enough information to solve this issue, unfortunately; if you have any more information to help us solve this, please reopen!

Hello @JackBoosY, I have a similar error and thought it might be better to comment instead of opening a whole new issue.
My config is the following:
package: lapack[core]:x64-windows -> 3#2
vcpkg version: 2021-11-02-af04ebf6274fd6f7a941bff4662b3955c64f6f42
vcpkg-tool version: 8cfe80209 2021-11-12 (16 hours ago)
Here are the logs
config-x64-windows-out.log:
[1/2] cmd /c "cd ../../x64-windows-dbg && "D:/sdk/vcpkg/downloads/tools/cmake-3.21.1-windows/cmake-3.21.1-windows-i386/bin/cmake.exe" "D:/sdk/vcpkg/ports/lapack" "-DCMAKE_PREFIX_PATH="D:/sdk/vcpkg/packages/lapack_x64-windows"" "-DBUILD_SHARED_LIBS=ON" "-DVCPKG_CHAINLOAD_TOOLCHAIN_FILE=D:/sdk/vcpkg/scripts/toolchains/windows.cmake" "-DVCPKG_TARGET_TRIPLET=x64-windows" "-DVCPKG_SET_CHARSET_FLAG=ON" "-DVCPKG_PLATFORM_TOOLSET=v143" "-DCMAKE_EXPORT_NO_PACKAGE_REGISTRY=ON" "-DCMAKE_FIND_PACKAGE_NO_PACKAGE_REGISTRY=ON" "-DCMAKE_FIND_PACKAGE_NO_SYSTEM_PACKAGE_REGISTRY=ON" "-DCMAKE_INSTALL_SYSTEM_RUNTIME_LIBS_SKIP=TRUE" "-DCMAKE_VERBOSE_MAKEFILE=ON" "-DVCPKG_APPLOCAL_DEPS=OFF" "-DCMAKE_TOOLCHAIN_FILE=D:/sdk/vcpkg/scripts/buildsystems/vcpkg.cmake" "-DCMAKE_ERROR_ON_ABSOLUTE_INSTALL_DESTINATION=ON" "-DVCPKG_CXX_FLAGS=" "-DVCPKG_CXX_FLAGS_RELEASE=" "-DVCPKG_CXX_FLAGS_DEBUG=" "-DVCPKG_C_FLAGS=" "-DVCPKG_C_FLAGS_RELEASE=" "-DVCPKG_C_FLAGS_DEBUG=" "-DVCPKG_CRT_LINKAGE=dynamic" "-DVCPKG_LINKER_FLAGS=" "-DVCPKG_LINKER_FLAGS_RELEASE=" "-DVCPKG_LINKER_FLAGS_DEBUG=" "-DVCPKG_TARGET_ARCHITECTURE=x64" "-DCMAKE_INSTALL_LIBDIR:STRING=lib" "-DCMAKE_INSTALL_BINDIR:STRING=bin" "-D_VCPKG_ROOT_DIR=D:/sdk/vcpkg" "-DZ_VCPKG_ROOT_DIR=D:/sdk/vcpkg" "-D_VCPKG_INSTALLED_DIR=D:/sdk/vcpkg/installed" "-DVCPKG_MANIFEST_INSTALL=OFF" "-Ax64" "-G" "Visual Studio 17 2022" "-DCMAKE_BUILD_TYPE=Debug" "-DCMAKE_INSTALL_PREFIX=D:/sdk/vcpkg/packages/lapack_x64-windows/debug""
FAILED: ../../x64-windows-dbg/CMakeCache.txt
cmd /c "cd ../../x64-windows-dbg && "D:/sdk/vcpkg/downloads/tools/cmake-3.21.1-windows/cmake-3.21.1-windows-i386/bin/cmake.exe" "D:/sdk/vcpkg/ports/lapack" "-DCMAKE_PREFIX_PATH="D:/sdk/vcpkg/packages/lapack_x64-windows"" "-DBUILD_SHARED_LIBS=ON" "-DVCPKG_CHAINLOAD_TOOLCHAIN_FILE=D:/sdk/vcpkg/scripts/toolchains/windows.cmake" "-DVCPKG_TARGET_TRIPLET=x64-windows" "-DVCPKG_SET_CHARSET_FLAG=ON" "-DVCPKG_PLATFORM_TOOLSET=v143" "-DCMAKE_EXPORT_NO_PACKAGE_REGISTRY=ON" "-DCMAKE_FIND_PACKAGE_NO_PACKAGE_REGISTRY=ON" "-DCMAKE_FIND_PACKAGE_NO_SYSTEM_PACKAGE_REGISTRY=ON" "-DCMAKE_INSTALL_SYSTEM_RUNTIME_LIBS_SKIP=TRUE" "-DCMAKE_VERBOSE_MAKEFILE=ON" "-DVCPKG_APPLOCAL_DEPS=OFF" "-DCMAKE_TOOLCHAIN_FILE=D:/sdk/vcpkg/scripts/buildsystems/vcpkg.cmake" "-DCMAKE_ERROR_ON_ABSOLUTE_INSTALL_DESTINATION=ON" "-DVCPKG_CXX_FLAGS=" "-DVCPKG_CXX_FLAGS_RELEASE=" "-DVCPKG_CXX_FLAGS_DEBUG=" "-DVCPKG_C_FLAGS=" "-DVCPKG_C_FLAGS_RELEASE=" "-DVCPKG_C_FLAGS_DEBUG=" "-DVCPKG_CRT_LINKAGE=dynamic" "-DVCPKG_LINKER_FLAGS=" "-DVCPKG_LINKER_FLAGS_RELEASE=" "-DVCPKG_LINKER_FLAGS_DEBUG=" "-DVCPKG_TARGET_ARCHITECTURE=x64" "-DCMAKE_INSTALL_LIBDIR:STRING=lib" "-DCMAKE_INSTALL_BINDIR:STRING=bin" "-D_VCPKG_ROOT_DIR=D:/sdk/vcpkg" "-DZ_VCPKG_ROOT_DIR=D:/sdk/vcpkg" "-D_VCPKG_INSTALLED_DIR=D:/sdk/vcpkg/installed" "-DVCPKG_MANIFEST_INSTALL=OFF" "-Ax64" "-G" "Visual Studio 17 2022" "-DCMAKE_BUILD_TYPE=Debug" "-DCMAKE_INSTALL_PREFIX=D:/sdk/vcpkg/packages/lapack_x64-windows/debug""
-- The C compiler identification is MSVC 19.30.30705.0
-- The CXX compiler identification is MSVC 19.30.30705.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.30.30705/bin/Hostx64/x64/cl.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.30.30705/bin/Hostx64/x64/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Using VCPKG FindLAPACK from package 'lapack-reference'
-- Looking for sgemm_
-- Looking for sgemm_ - not found
-- Looking for pthread.h
-- Looking for pthread.h - not found
-- Found Threads: TRUE
-- Looking for sgemm_
-- Looking for sgemm_ - found
-- Found BLAS: D:/sdk/vcpkg/installed/x64-windows/debug/lib/openblas.lib
-- Looking for cheev_
-- Looking for cheev_ - not found
-- Looking for cheev_
-- Looking for cheev_ - not found
CMake Error at D:/sdk/vcpkg/installed/x64-windows/share/lapack/FindLAPACK.cmake:522 (message):
  A required library with LAPACK API not found.  Please specify library
  location.
Call Stack (most recent call first):
  D:/sdk/vcpkg/installed/x64-windows/share/lapack/vcpkg-cmake-wrapper.cmake:9 (_find_package)
  D:/sdk/vcpkg/scripts/buildsystems/vcpkg.cmake:742 (include)
  CMakeLists.txt:4 (find_package)

-- Configuring incomplete, errors occurred!
See also "D:/sdk/vcpkg/buildtrees/lapack/x64-windows-dbg/CMakeFiles/CMakeOutput.log".
See also "D:/sdk/vcpkg/buildtrees/lapack/x64-windows-dbg/CMakeFiles/CMakeError.log".
[2/2] cmd /c "cd .. && "D:/sdk/vcpkg/downloads/tools/cmake-3.21.1-windows/cmake-3.21.1-windows-i386/bin/cmake.exe" "D:/sdk/vcpkg/ports/lapack" "-DCMAKE_PREFIX_PATH="D:/sdk/vcpkg/packages/lapack_x64-windows"" "-DBUILD_SHARED_LIBS=ON" "-DVCPKG_CHAINLOAD_TOOLCHAIN_FILE=D:/sdk/vcpkg/scripts/toolchains/windows.cmake" "-DVCPKG_TARGET_TRIPLET=x64-windows" "-DVCPKG_SET_CHARSET_FLAG=ON" "-DVCPKG_PLATFORM_TOOLSET=v143" "-DCMAKE_EXPORT_NO_PACKAGE_REGISTRY=ON" "-DCMAKE_FIND_PACKAGE_NO_PACKAGE_REGISTRY=ON" "-DCMAKE_FIND_PACKAGE_NO_SYSTEM_PACKAGE_REGISTRY=ON" "-DCMAKE_INSTALL_SYSTEM_RUNTIME_LIBS_SKIP=TRUE" "-DCMAKE_VERBOSE_MAKEFILE=ON" "-DVCPKG_APPLOCAL_DEPS=OFF" "-DCMAKE_TOOLCHAIN_FILE=D:/sdk/vcpkg/scripts/buildsystems/vcpkg.cmake" "-DCMAKE_ERROR_ON_ABSOLUTE_INSTALL_DESTINATION=ON" "-DVCPKG_CXX_FLAGS=" "-DVCPKG_CXX_FLAGS_RELEASE=" "-DVCPKG_CXX_FLAGS_DEBUG=" "-DVCPKG_C_FLAGS=" "-DVCPKG_C_FLAGS_RELEASE=" "-DVCPKG_C_FLAGS_DEBUG=" "-DVCPKG_CRT_LINKAGE=dynamic" "-DVCPKG_LINKER_FLAGS=" "-DVCPKG_LINKER_FLAGS_RELEASE=" "-DVCPKG_LINKER_FLAGS_DEBUG=" "-DVCPKG_TARGET_ARCHITECTURE=x64" "-DCMAKE_INSTALL_LIBDIR:STRING=lib" "-DCMAKE_INSTALL_BINDIR:STRING=bin" "-D_VCPKG_ROOT_DIR=D:/sdk/vcpkg" "-DZ_VCPKG_ROOT_DIR=D:/sdk/vcpkg" "-D_VCPKG_INSTALLED_DIR=D:/sdk/vcpkg/installed" "-DVCPKG_MANIFEST_INSTALL=OFF" "-Ax64" "-G" "Visual Studio 17 2022" "-DCMAKE_BUILD_TYPE=Release" "-DCMAKE_INSTALL_PREFIX=D:/sdk/vcpkg/packages/lapack_x64-windows""
FAILED: ../CMakeCache.txt
cmd /c "cd .. && "D:/sdk/vcpkg/downloads/tools/cmake-3.21.1-windows/cmake-3.21.1-windows-i386/bin/cmake.exe" "D:/sdk/vcpkg/ports/lapack" "-DCMAKE_PREFIX_PATH="D:/sdk/vcpkg/packages/lapack_x64-windows"" "-DBUILD_SHARED_LIBS=ON" "-DVCPKG_CHAINLOAD_TOOLCHAIN_FILE=D:/sdk/vcpkg/scripts/toolchains/windows.cmake" "-DVCPKG_TARGET_TRIPLET=x64-windows" "-DVCPKG_SET_CHARSET_FLAG=ON" "-DVCPKG_PLATFORM_TOOLSET=v143" "-DCMAKE_EXPORT_NO_PACKAGE_REGISTRY=ON" "-DCMAKE_FIND_PACKAGE_NO_PACKAGE_REGISTRY=ON" "-DCMAKE_FIND_PACKAGE_NO_SYSTEM_PACKAGE_REGISTRY=ON" "-DCMAKE_INSTALL_SYSTEM_RUNTIME_LIBS_SKIP=TRUE" "-DCMAKE_VERBOSE_MAKEFILE=ON" "-DVCPKG_APPLOCAL_DEPS=OFF" "-DCMAKE_TOOLCHAIN_FILE=D:/sdk/vcpkg/scripts/buildsystems/vcpkg.cmake" "-DCMAKE_ERROR_ON_ABSOLUTE_INSTALL_DESTINATION=ON" "-DVCPKG_CXX_FLAGS=" "-DVCPKG_CXX_FLAGS_RELEASE=" "-DVCPKG_CXX_FLAGS_DEBUG=" "-DVCPKG_C_FLAGS=" "-DVCPKG_C_FLAGS_RELEASE=" "-DVCPKG_C_FLAGS_DEBUG=" "-DVCPKG_CRT_LINKAGE=dynamic" "-DVCPKG_LINKER_FLAGS=" "-DVCPKG_LINKER_FLAGS_RELEASE=" "-DVCPKG_LINKER_FLAGS_DEBUG=" "-DVCPKG_TARGET_ARCHITECTURE=x64" "-DCMAKE_INSTALL_LIBDIR:STRING=lib" "-DCMAKE_INSTALL_BINDIR:STRING=bin" "-D_VCPKG_ROOT_DIR=D:/sdk/vcpkg" "-DZ_VCPKG_ROOT_DIR=D:/sdk/vcpkg" "-D_VCPKG_INSTALLED_DIR=D:/sdk/vcpkg/installed" "-DVCPKG_MANIFEST_INSTALL=OFF" "-Ax64" "-G" "Visual Studio 17 2022" "-DCMAKE_BUILD_TYPE=Release" "-DCMAKE_INSTALL_PREFIX=D:/sdk/vcpkg/packages/lapack_x64-windows""
-- The C compiler identification is MSVC 19.30.30705.0
-- The CXX compiler identification is MSVC 19.30.30705.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.30.30705/bin/Hostx64/x64/cl.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.30.30705/bin/Hostx64/x64/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Using VCPKG FindLAPACK from package 'lapack-reference'
-- Looking for sgemm_
-- Looking for sgemm_ - not found
-- Looking for pthread.h
-- Looking for pthread.h - not found
-- Found Threads: TRUE
-- Looking for sgemm_
-- Looking for sgemm_ - found
-- Found BLAS: D:/sdk/vcpkg/installed/x64-windows/lib/openblas.lib
-- Looking for cheev_
-- Looking for cheev_ - not found
-- Looking for cheev_
-- Looking for cheev_ - not found
CMake Error at D:/sdk/vcpkg/installed/x64-windows/share/lapack/FindLAPACK.cmake:522 (message):
  A required library with LAPACK API not found.  Please specify library
  location.
Call Stack (most recent call first):
  D:/sdk/vcpkg/installed/x64-windows/share/lapack/vcpkg-cmake-wrapper.cmake:9 (_find_package)
  D:/sdk/vcpkg/scripts/buildsystems/vcpkg.cmake:742 (include)
  CMakeLists.txt:4 (find_package)

-- Configuring incomplete, errors occurred!
See also "D:/sdk/vcpkg/buildtrees/lapack/x64-windows-rel/CMakeFiles/CMakeOutput.log".
See also "D:/sdk/vcpkg/buildtrees/lapack/x64-windows-rel/CMakeFiles/CMakeError.log".
ninja: build stopped: subcommand failed.

CmakeError.log:
[1/2] cmd /c "cd ../../x64-windows-dbg && "D:/sdk/vcpkg/downloads/tools/cmake-3.21.1-windows/cmake-3.21.1-windows-i386/bin/cmake.exe" "D:/sdk/vcpkg/ports/lapack" "-DCMAKE_PREFIX_PATH="D:/sdk/vcpkg/packages/lapack_x64-windows"" "-DBUILD_SHARED_LIBS=ON" "-DVCPKG_CHAINLOAD_TOOLCHAIN_FILE=D:/sdk/vcpkg/scripts/toolchains/windows.cmake" "-DVCPKG_TARGET_TRIPLET=x64-windows" "-DVCPKG_SET_CHARSET_FLAG=ON" "-DVCPKG_PLATFORM_TOOLSET=v143" "-DCMAKE_EXPORT_NO_PACKAGE_REGISTRY=ON" "-DCMAKE_FIND_PACKAGE_NO_PACKAGE_REGISTRY=ON" "-DCMAKE_FIND_PACKAGE_NO_SYSTEM_PACKAGE_REGISTRY=ON" "-DCMAKE_INSTALL_SYSTEM_RUNTIME_LIBS_SKIP=TRUE" "-DCMAKE_VERBOSE_MAKEFILE=ON" "-DVCPKG_APPLOCAL_DEPS=OFF" "-DCMAKE_TOOLCHAIN_FILE=D:/sdk/vcpkg/scripts/buildsystems/vcpkg.cmake" "-DCMAKE_ERROR_ON_ABSOLUTE_INSTALL_DESTINATION=ON" "-DVCPKG_CXX_FLAGS=" "-DVCPKG_CXX_FLAGS_RELEASE=" "-DVCPKG_CXX_FLAGS_DEBUG=" "-DVCPKG_C_FLAGS=" "-DVCPKG_C_FLAGS_RELEASE=" "-DVCPKG_C_FLAGS_DEBUG=" "-DVCPKG_CRT_LINKAGE=dynamic" "-DVCPKG_LINKER_FLAGS=" "-DVCPKG_LINKER_FLAGS_RELEASE=" "-DVCPKG_LINKER_FLAGS_DEBUG=" "-DVCPKG_TARGET_ARCHITECTURE=x64" "-DCMAKE_INSTALL_LIBDIR:STRING=lib" "-DCMAKE_INSTALL_BINDIR:STRING=bin" "-D_VCPKG_ROOT_DIR=D:/sdk/vcpkg" "-DZ_VCPKG_ROOT_DIR=D:/sdk/vcpkg" "-D_VCPKG_INSTALLED_DIR=D:/sdk/vcpkg/installed" "-DVCPKG_MANIFEST_INSTALL=OFF" "-Ax64" "-G" "Visual Studio 17 2022" "-DCMAKE_BUILD_TYPE=Debug" "-DCMAKE_INSTALL_PREFIX=D:/sdk/vcpkg/packages/lapack_x64-windows/debug""
FAILED: ../../x64-windows-dbg/CMakeCache.txt
cmd /c "cd ../../x64-windows-dbg && "D:/sdk/vcpkg/downloads/tools/cmake-3.21.1-windows/cmake-3.21.1-windows-i386/bin/cmake.exe" "D:/sdk/vcpkg/ports/lapack" "-DCMAKE_PREFIX_PATH="D:/sdk/vcpkg/packages/lapack_x64-windows"" "-DBUILD_SHARED_LIBS=ON" "-DVCPKG_CHAINLOAD_TOOLCHAIN_FILE=D:/sdk/vcpkg/scripts/toolchains/windows.cmake" "-DVCPKG_TARGET_TRIPLET=x64-windows" "-DVCPKG_SET_CHARSET_FLAG=ON" "-DVCPKG_PLATFORM_TOOLSET=v143" "-DCMAKE_EXPORT_NO_PACKAGE_REGISTRY=ON" "-DCMAKE_FIND_PACKAGE_NO_PACKAGE_REGISTRY=ON" "-DCMAKE_FIND_PACKAGE_NO_SYSTEM_PACKAGE_REGISTRY=ON" "-DCMAKE_INSTALL_SYSTEM_RUNTIME_LIBS_SKIP=TRUE" "-DCMAKE_VERBOSE_MAKEFILE=ON" "-DVCPKG_APPLOCAL_DEPS=OFF" "-DCMAKE_TOOLCHAIN_FILE=D:/sdk/vcpkg/scripts/buildsystems/vcpkg.cmake" "-DCMAKE_ERROR_ON_ABSOLUTE_INSTALL_DESTINATION=ON" "-DVCPKG_CXX_FLAGS=" "-DVCPKG_CXX_FLAGS_RELEASE=" "-DVCPKG_CXX_FLAGS_DEBUG=" "-DVCPKG_C_FLAGS=" "-DVCPKG_C_FLAGS_RELEASE=" "-DVCPKG_C_FLAGS_DEBUG=" "-DVCPKG_CRT_LINKAGE=dynamic" "-DVCPKG_LINKER_FLAGS=" "-DVCPKG_LINKER_FLAGS_RELEASE=" "-DVCPKG_LINKER_FLAGS_DEBUG=" "-DVCPKG_TARGET_ARCHITECTURE=x64" "-DCMAKE_INSTALL_LIBDIR:STRING=lib" "-DCMAKE_INSTALL_BINDIR:STRING=bin" "-D_VCPKG_ROOT_DIR=D:/sdk/vcpkg" "-DZ_VCPKG_ROOT_DIR=D:/sdk/vcpkg" "-D_VCPKG_INSTALLED_DIR=D:/sdk/vcpkg/installed" "-DVCPKG_MANIFEST_INSTALL=OFF" "-Ax64" "-G" "Visual Studio 17 2022" "-DCMAKE_BUILD_TYPE=Debug" "-DCMAKE_INSTALL_PREFIX=D:/sdk/vcpkg/packages/lapack_x64-windows/debug""
-- The C compiler identification is MSVC 19.30.30705.0
-- The CXX compiler identification is MSVC 19.30.30705.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.30.30705/bin/Hostx64/x64/cl.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.30.30705/bin/Hostx64/x64/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Using VCPKG FindLAPACK from package 'lapack-reference'
-- Looking for sgemm_
-- Looking for sgemm_ - not found
-- Looking for pthread.h
-- Looking for pthread.h - not found
-- Found Threads: TRUE
-- Looking for sgemm_
-- Looking for sgemm_ - found
-- Found BLAS: D:/sdk/vcpkg/installed/x64-windows/debug/lib/openblas.lib
-- Looking for cheev_
-- Looking for cheev_ - not found
-- Looking for cheev_
-- Looking for cheev_ - not found
CMake Error at D:/sdk/vcpkg/installed/x64-windows/share/lapack/FindLAPACK.cmake:522 (message):
  A required library with LAPACK API not found.  Please specify library
  location.
Call Stack (most recent call first):
  D:/sdk/vcpkg/installed/x64-windows/share/lapack/vcpkg-cmake-wrapper.cmake:9 (_find_package)
  D:/sdk/vcpkg/scripts/buildsystems/vcpkg.cmake:742 (include)
  CMakeLists.txt:4 (find_package)

-- Configuring incomplete, errors occurred!
See also "D:/sdk/vcpkg/buildtrees/lapack/x64-windows-dbg/CMakeFiles/CMakeOutput.log".
See also "D:/sdk/vcpkg/buildtrees/lapack/x64-windows-dbg/CMakeFiles/CMakeError.log".
[2/2] cmd /c "cd .. && "D:/sdk/vcpkg/downloads/tools/cmake-3.21.1-windows/cmake-3.21.1-windows-i386/bin/cmake.exe" "D:/sdk/vcpkg/ports/lapack" "-DCMAKE_PREFIX_PATH="D:/sdk/vcpkg/packages/lapack_x64-windows"" "-DBUILD_SHARED_LIBS=ON" "-DVCPKG_CHAINLOAD_TOOLCHAIN_FILE=D:/sdk/vcpkg/scripts/toolchains/windows.cmake" "-DVCPKG_TARGET_TRIPLET=x64-windows" "-DVCPKG_SET_CHARSET_FLAG=ON" "-DVCPKG_PLATFORM_TOOLSET=v143" "-DCMAKE_EXPORT_NO_PACKAGE_REGISTRY=ON" "-DCMAKE_FIND_PACKAGE_NO_PACKAGE_REGISTRY=ON" "-DCMAKE_FIND_PACKAGE_NO_SYSTEM_PACKAGE_REGISTRY=ON" "-DCMAKE_INSTALL_SYSTEM_RUNTIME_LIBS_SKIP=TRUE" "-DCMAKE_VERBOSE_MAKEFILE=ON" "-DVCPKG_APPLOCAL_DEPS=OFF" "-DCMAKE_TOOLCHAIN_FILE=D:/sdk/vcpkg/scripts/buildsystems/vcpkg.cmake" "-DCMAKE_ERROR_ON_ABSOLUTE_INSTALL_DESTINATION=ON" "-DVCPKG_CXX_FLAGS=" "-DVCPKG_CXX_FLAGS_RELEASE=" "-DVCPKG_CXX_FLAGS_DEBUG=" "-DVCPKG_C_FLAGS=" "-DVCPKG_C_FLAGS_RELEASE=" "-DVCPKG_C_FLAGS_DEBUG=" "-DVCPKG_CRT_LINKAGE=dynamic" "-DVCPKG_LINKER_FLAGS=" "-DVCPKG_LINKER_FLAGS_RELEASE=" "-DVCPKG_LINKER_FLAGS_DEBUG=" "-DVCPKG_TARGET_ARCHITECTURE=x64" "-DCMAKE_INSTALL_LIBDIR:STRING=lib" "-DCMAKE_INSTALL_BINDIR:STRING=bin" "-D_VCPKG_ROOT_DIR=D:/sdk/vcpkg" "-DZ_VCPKG_ROOT_DIR=D:/sdk/vcpkg" "-D_VCPKG_INSTALLED_DIR=D:/sdk/vcpkg/installed" "-DVCPKG_MANIFEST_INSTALL=OFF" "-Ax64" "-G" "Visual Studio 17 2022" "-DCMAKE_BUILD_TYPE=Release" "-DCMAKE_INSTALL_PREFIX=D:/sdk/vcpkg/packages/lapack_x64-windows""
FAILED: ../CMakeCache.txt
cmd /c "cd .. && "D:/sdk/vcpkg/downloads/tools/cmake-3.21.1-windows/cmake-3.21.1-windows-i386/bin/cmake.exe" "D:/sdk/vcpkg/ports/lapack" "-DCMAKE_PREFIX_PATH="D:/sdk/vcpkg/packages/lapack_x64-windows"" "-DBUILD_SHARED_LIBS=ON" "-DVCPKG_CHAINLOAD_TOOLCHAIN_FILE=D:/sdk/vcpkg/scripts/toolchains/windows.cmake" "-DVCPKG_TARGET_TRIPLET=x64-windows" "-DVCPKG_SET_CHARSET_FLAG=ON" "-DVCPKG_PLATFORM_TOOLSET=v143" "-DCMAKE_EXPORT_NO_PACKAGE_REGISTRY=ON" "-DCMAKE_FIND_PACKAGE_NO_PACKAGE_REGISTRY=ON" "-DCMAKE_FIND_PACKAGE_NO_SYSTEM_PACKAGE_REGISTRY=ON" "-DCMAKE_INSTALL_SYSTEM_RUNTIME_LIBS_SKIP=TRUE" "-DCMAKE_VERBOSE_MAKEFILE=ON" "-DVCPKG_APPLOCAL_DEPS=OFF" "-DCMAKE_TOOLCHAIN_FILE=D:/sdk/vcpkg/scripts/buildsystems/vcpkg.cmake" "-DCMAKE_ERROR_ON_ABSOLUTE_INSTALL_DESTINATION=ON" "-DVCPKG_CXX_FLAGS=" "-DVCPKG_CXX_FLAGS_RELEASE=" "-DVCPKG_CXX_FLAGS_DEBUG=" "-DVCPKG_C_FLAGS=" "-DVCPKG_C_FLAGS_RELEASE=" "-DVCPKG_C_FLAGS_DEBUG=" "-DVCPKG_CRT_LINKAGE=dynamic" "-DVCPKG_LINKER_FLAGS=" "-DVCPKG_LINKER_FLAGS_RELEASE=" "-DVCPKG_LINKER_FLAGS_DEBUG=" "-DVCPKG_TARGET_ARCHITECTURE=x64" "-DCMAKE_INSTALL_LIBDIR:STRING=lib" "-DCMAKE_INSTALL_BINDIR:STRING=bin" "-D_VCPKG_ROOT_DIR=D:/sdk/vcpkg" "-DZ_VCPKG_ROOT_DIR=D:/sdk/vcpkg" "-D_VCPKG_INSTALLED_DIR=D:/sdk/vcpkg/installed" "-DVCPKG_MANIFEST_INSTALL=OFF" "-Ax64" "-G" "Visual Studio 17 2022" "-DCMAKE_BUILD_TYPE=Release" "-DCMAKE_INSTALL_PREFIX=D:/sdk/vcpkg/packages/lapack_x64-windows""
-- The C compiler identification is MSVC 19.30.30705.0
-- The CXX compiler identification is MSVC 19.30.30705.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.30.30705/bin/Hostx64/x64/cl.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.30.30705/bin/Hostx64/x64/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Using VCPKG FindLAPACK from package 'lapack-reference'
-- Looking for sgemm_
-- Looking for sgemm_ - not found
-- Looking for pthread.h
-- Looking for pthread.h - not found
-- Found Threads: TRUE
-- Looking for sgemm_
-- Looking for sgemm_ - found
-- Found BLAS: D:/sdk/vcpkg/installed/x64-windows/lib/openblas.lib
-- Looking for cheev_
-- Looking for cheev_ - not found
-- Looking for cheev_
-- Looking for cheev_ - not found
CMake Error at D:/sdk/vcpkg/installed/x64-windows/share/lapack/FindLAPACK.cmake:522 (message):
  A required library with LAPACK API not found.  Please specify library
  location.
Call Stack (most recent call first):
  D:/sdk/vcpkg/installed/x64-windows/share/lapack/vcpkg-cmake-wrapper.cmake:9 (_find_package)
  D:/sdk/vcpkg/scripts/buildsystems/vcpkg.cmake:742 (include)
  CMakeLists.txt:4 (find_package)

-- Configuring incomplete, errors occurred!
See also "D:/sdk/vcpkg/buildtrees/lapack/x64-windows-rel/CMakeFiles/CMakeOutput.log".
See also "D:/sdk/vcpkg/buildtrees/lapack/x64-windows-rel/CMakeFiles/CMakeError.log".
ninja: build stopped: subcommand failed.

CmakeOutput.log:
The system is: Windows - 10.0.19041 - AMD64
Compiling the C compiler identification source file "CMakeCCompilerId.c" succeeded.
Compiler:
Build flags: ;/nologo;/DWIN32;/D_WINDOWS;/W3;/utf-8;/MP;
Id flags:

The output was:
0
Microsoft (R) Build Engine version 17.0.0+c9eb9dd64 for .NET Framework
Copyright (C) Microsoft Corporation. All rights reserved.

Build started 13/11/2021 16:47:31.
Project "D:\sdk\vcpkg\buildtrees\lapack\x64-windows-rel\CMakeFiles\3.21.1\CompilerIdC\CompilerIdC.vcxproj" on node 1 (default targets).
PrepareForBuild:
  Creating directory "Debug\".
  Creating directory "Debug\CompilerIdC.tlog\".
InitializeBuildStatus:
  Creating "Debug\CompilerIdC.tlog\unsuccessfulbuild" because "AlwaysCreate" was specified.
ClCompile:
  C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.30.30705\bin\HostX64\x64\CL.exe /c /nologo /W0 /WX- /diagnostics:column /Od /D _MBCS /Gm- /EHsc /RTC1 /MDd /GS /fp:precise /Zc:wchar_t /Zc:forScope /Zc:inline /Fo"Debug\\" /Fd"Debug\vc143.pdb" /external:W0 /Gd /TC /FC /errorReport:queue CMakeCCompilerId.c
  CMakeCCompilerId.c
Link:
  C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.30.30705\bin\HostX64\x64\link.exe /ERRORREPORT:QUEUE /OUT:".\CompilerIdC.exe" /INCREMENTAL:NO /NOLOGO kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib uuid.lib odbc32.lib odbccp32.lib /MANIFEST /MANIFESTUAC:"level='asInvoker' uiAccess='false'" /manifest:embed /PDB:".\CompilerIdC.pdb" /SUBSYSTEM:CONSOLE /TLBID:1 /DYNAMICBASE /NXCOMPAT /IMPLIB:".\CompilerIdC.lib" /MACHINE:X64 Debug\CMakeCCompilerId.obj
  CompilerIdC.vcxproj -> D:\sdk\vcpkg\buildtrees\lapack\x64-windows-rel\CMakeFiles\3.21.1\CompilerIdC\CompilerIdC.exe
PostBuildEvent:
  for %%i in (cl.exe) do @echo CMAKE_C_COMPILER=%%~$PATH:i
  :VCEnd
  CMAKE_C_COMPILER=C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.30.30705\bin\Hostx64\x64\cl.exe
FinalizeBuildStatus:
  Deleting file "Debug\CompilerIdC.tlog\unsuccessfulbuild".
  Touching "Debug\CompilerIdC.tlog\CompilerIdC.lastbuildstate".
Done Building Project "D:\sdk\vcpkg\buildtrees\lapack\x64-windows-rel\CMakeFiles\3.21.1\CompilerIdC\CompilerIdC.vcxproj" (default targets).

Build succeeded.
    0 Warning(s)
    0 Error(s)

Time Elapsed 00:00:01.68

Compilation of the C compiler identification source "CMakeCCompilerId.c" produced "CompilerIdC.exe"

Compilation of the C compiler identification source "CMakeCCompilerId.c" produced "CompilerIdC.vcxproj"

The C compiler identification is MSVC, found in "D:/sdk/vcpkg/buildtrees/lapack/x64-windows-rel/CMakeFiles/3.21.1/CompilerIdC/CompilerIdC.exe"

Compiling the CXX compiler identification source file "CMakeCXXCompilerId.cpp" succeeded.
Compiler:
Build flags: ;/nologo;/DWIN32;/D_WINDOWS;/W3;/utf-8;/GR;/EHsc;/MP;
Id flags:

The output was:
0
Microsoft (R) Build Engine version 17.0.0+c9eb9dd64 for .NET Framework
Copyright (C) Microsoft Corporation. All rights reserved.

Build started 13/11/2021 16:47:34.
Project "D:\sdk\vcpkg\buildtrees\lapack\x64-windows-rel\CMakeFiles\3.21.1\CompilerIdCXX\CompilerIdCXX.vcxproj" on node 1 (default targets).
PrepareForBuild:
  Creating directory "Debug\".
  Creating directory "Debug\CompilerIdCXX.tlog\".
InitializeBuildStatus:
  Creating "Debug\CompilerIdCXX.tlog\unsuccessfulbuild" because "AlwaysCreate" was specified.
ClCompile:
  C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.30.30705\bin\HostX64\x64\CL.exe /c /nologo /W0 /WX- /diagnostics:column /Od /D _MBCS /Gm- /EHsc /RTC1 /MDd /GS /fp:precise /Zc:wchar_t /Zc:forScope /Zc:inline /Fo"Debug\\" /Fd"Debug\vc143.pdb" /external:W0 /Gd /TP /FC /errorReport:queue CMakeCXXCompilerId.cpp
  CMakeCXXCompilerId.cpp
Link:
  C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.30.30705\bin\HostX64\x64\link.exe /ERRORREPORT:QUEUE /OUT:".\CompilerIdCXX.exe" /INCREMENTAL:NO /NOLOGO kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib uuid.lib odbc32.lib odbccp32.lib /MANIFEST /MANIFESTUAC:"level='asInvoker' uiAccess='false'" /manifest:embed /PDB:".\CompilerIdCXX.pdb" /SUBSYSTEM:CONSOLE /TLBID:1 /DYNAMICBASE /NXCOMPAT /IMPLIB:".\CompilerIdCXX.lib" /MACHINE:X64 Debug\CMakeCXXCompilerId.obj
  CompilerIdCXX.vcxproj -> D:\sdk\vcpkg\buildtrees\lapack\x64-windows-rel\CMakeFiles\3.21.1\CompilerIdCXX\CompilerIdCXX.exe
PostBuildEvent:
  for %%i in (cl.exe) do @echo CMAKE_CXX_COMPILER=%%~$PATH:i
  :VCEnd
  CMAKE_CXX_COMPILER=C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.30.30705\bin\Hostx64\x64\cl.exe
FinalizeBuildStatus:
  Deleting file "Debug\CompilerIdCXX.tlog\unsuccessfulbuild".
  Touching "Debug\CompilerIdCXX.tlog\CompilerIdCXX.lastbuildstate".
Done Building Project "D:\sdk\vcpkg\buildtrees\lapack\x64-windows-rel\CMakeFiles\3.21.1\CompilerIdCXX\CompilerIdCXX.vcxproj" (default targets).

Build succeeded.
    0 Warning(s)
    0 Error(s)

Time Elapsed 00:00:00.98

Compilation of the CXX compiler identification source "CMakeCXXCompilerId.cpp" produced "CompilerIdCXX.exe"

Compilation of the CXX compiler identification source "CMakeCXXCompilerId.cpp" produced "CompilerIdCXX.vcxproj"

The CXX compiler identification is MSVC, found in "D:/sdk/vcpkg/buildtrees/lapack/x64-windows-rel/CMakeFiles/3.21.1/CompilerIdCXX/CompilerIdCXX.exe"

Detecting C compiler ABI info compiled with the following output:
Change Dir: D:/sdk/vcpkg/buildtrees/lapack/x64-windows-rel/CMakeFiles/CMakeTmp

Run Build Command(s):C:/Program Files/Microsoft Visual Studio/2022/Community/MSBuild/Current/Bin/amd64/MSBuild.exe cmTC_974b9.vcxproj /p:Configuration=Debug /p:Platform=x64 /p:VisualStudioVersion=17.0 /v:m && Microsoft (R) Build Engine version 17.0.0+c9eb9dd64 for .NET Framework

Copyright (C) Microsoft Corporation. All rights reserved.

  CMakeCCompilerABI.c

  cmTC_974b9.vcxproj -> D:\sdk\vcpkg\buildtrees\lapack\x64-windows-rel\CMakeFiles\CMakeTmp\Debug\cmTC_974b9.exe

Detecting CXX compiler ABI info compiled with the following output:
Change Dir: D:/sdk/vcpkg/buildtrees/lapack/x64-windows-rel/CMakeFiles/CMakeTmp

Run Build Command(s):C:/Program Files/Microsoft Visual Studio/2022/Community/MSBuild/Current/Bin/amd64/MSBuild.exe cmTC_626b4.vcxproj /p:Configuration=Debug /p:Platform=x64 /p:VisualStudioVersion=17.0 /v:m && Microsoft (R) Build Engine version 17.0.0+c9eb9dd64 for .NET Framework

Copyright (C) Microsoft Corporation. All rights reserved.

  CMakeCXXCompilerABI.cpp

  cmTC_626b4.vcxproj -> D:\sdk\vcpkg\buildtrees\lapack\x64-windows-rel\CMakeFiles\CMakeTmp\Debug\cmTC_626b4.exe

Determining if the function sgemm_ exists passed with the following output:
Change Dir: D:/sdk/vcpkg/buildtrees/lapack/x64-windows-rel/CMakeFiles/CMakeTmp

Run Build Command(s):C:/Program Files/Microsoft Visual Studio/2022/Community/MSBuild/Current/Bin/amd64/MSBuild.exe cmTC_519cc.vcxproj /p:Configuration=Debug /p:Platform=x64 /p:VisualStudioVersion=17.0 /v:m && Microsoft (R) Build Engine version 17.0.0+c9eb9dd64 for .NET Framework

Copyright (C) Microsoft Corporation. All rights reserved.

  CheckFunctionExists.c

  cmTC_519cc.vcxproj -> D:\sdk\vcpkg\buildtrees\lapack\x64-windows-rel\CMakeFiles\CMakeTmp\Debug\cmTC_519cc.exe

@lukadriel7  Confirmed.
Since lapack-reference uses gfortran to build itself, the generated static library suffix is .a on Windows, not .lib, which caused call FindLapack.cmake to fail.
After fix that, we also should fix blas issues by https://github.com/microsoft/vcpkg/pull/19608.

Under windows, you can temporarily use clapack instead of lapack-reference to solve this problem.
Just modify vcpkg.json under lapack
{
  "name": "lapack",
  "version-string": "3",
  "port-version": 2,
  "description": "Metapackage for packages which provide LAPACK",
  "dependencies": [
    {
      "name": "clapack",
      "platform": "windows"
    },
    {
      "name": "lapack-reference",
      "platform": "!windows"
    }
  ]
}

好像不可以

This issue should be fixed by https://github.com/microsoft/vcpkg/pull/21479.

This problem affects too many ports. I think it may need to be repaired as soon as possible.

Sorry to ask here. I have same problem with 2024.7.10 vcpkg:
      -- Found BLAS: /tmp/tmpfzymumk5/build/vcpkg_installed/x64-linux/lib/libopenblas.a
      -- Using VCPKG FindLAPACK from package 'lapack-reference'
      -- The C compiler identification is GNU 11.4.0
      -- Detecting C compiler ABI info
      -- Detecting C compiler ABI info - done
      -- Check for working C compiler: /usr/bin/gcc - skipped
      -- Detecting C compile features
      -- Detecting C compile features - done
      -- Performing Test CMAKE_HAVE_LIBC_PTHREAD
      -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
      -- Found Threads: TRUE
      -- Looking for cheev_
      -- Looking for cheev_ - not found
      -- Looking for cheev_
      -- Looking for cheev_ - not found
      -- Looking for cheev_
      -- Looking for cheev_ - not found
      -- Looking for cheev_
      -- Looking for cheev_ - not found
      CMake Error at /tmp/tmpfzymumk5/build/vcpkg_installed/x64-linux/share/lapack/FindLAPACK.cmake:522 (message):
        A required library with LAPACK API not found.  Please specify library
        location.

If I build my project directly, everything is fine. But when I build it with scikit-build-core, the error arises. There is no other configuration, just add submodule to my python project and then:
[build-system]
requires = ["scikit-build-core", "pybind11", "numpy"]
build-backend = "scikit_build_core.build"

[tool.scikit-build]
logging.level = "INFO"
cmake.verbose = "True"
cmake.source-dir = "molcpp"
cmake.args = ["--preset=python"]

Could please help with it, or give me some debug suggestion?

@dg0yt Any suggestions here?

related problem #41048
issue_body.md


Fix array_sort fuzzer bug #2795

Thanks to @mbasmanova for discovering fix.
Bug is because we use the decodedvector base , which will potentially cause us to skip nulls. Validated that the test case fails with decodedVector.

Can you please add a brief description of the bug to the commit message

Can you please add a brief description of the bug in the commit message to provide more context as to what part of the function went wrong and how skipping nulls manifests into a tangible bug?


(fix) Peer dependencies issue related to React v18

This PR fixed the issue occurring in some environments while installing this package.
22:53:01.146 | npm ERR! Could not resolve dependency:
22:53:01.146 | npm ERR! peer<EMAIL_ADDRESS>\|\| ^17" from<EMAIL_ADDRESS>22:53:01.146 | npm ERR! node_modules/react-alert
22:53:01.146 | npm ERR<EMAIL_ADDRESS>from the root project

Any chance this could be merged?

@schiehll we're held up on this to upgrade our app, can we merge? 🙏

@DavidChouinard until this PR is merged you can try out it's fork with the React 18 support - https://www.npmjs.com/package/@blaumaus/react-alert

Fantastic thank you. @schiehll would still be great to have this merged to have an official supported distribution

@schiehll merge it pls

@schiehll when is the merge?

https://www.npmjs.com/package/@blaumaus/react-alert - this is not working and has the same issue as the original Node Package.

poke

Any reason this hasn't been merged yet?

maybe @schiehll is no longer with us


Build failure: python310Packages.debugpy

Steps To Reproduce
Steps to reproduce the behavior:

Run nix-shell -p python310Packages.debugpy

Build log
Running phase: installPhase
Executing pypaInstallPhase
*** Error compiling '/nix/store/g1ia17mcp79fzqq8iamm07x5z1aqlrzy-python3.10-debugpy-1.8.0/lib/python3.10/site-packages/debugpy/_vendored/pydevd/tests_python/resources/_debugger_case_scoped_stepping_target_async.py'...
  File "/nix/store/g1ia17mcp79fzqq8iamm07x5z1aqlrzy-python3.10-debugpy-1.8.0/lib/python3.10/site-packages/debugpy/_vendored/pydevd/tests_python/resources/_debugger_case_scoped_stepping_target_async.py", line 5
    await asyncio.sleep(.01)
    ^^^^^^^^^^^^^^^^^^^^^^^^
SyntaxError: 'await' outside function

rootdir: /build/source
configfile: pytest.ini
testpaths: tests
plugins: timeout-2.2.0, xdist-3.5.0
8 workers [1216 items] m
........................................................................ [  5%]
........................................................................ [ 11%]
........................................................................ [ 17%]

Then it gets stuck.
Additional context
This started happening a week ago, and seems like it could be related to this:
https://github.com/microsoft/debugpy/issues/1490
Notify maintainers
@kira-bruneau
@SuperSandro2000
Metadata
Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.
[user@system:~]$ nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-linux"`
 - host os: `Linux 6.6.9-arch1-1, Arch Linux, noversion, rolling`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.19.1

Also reported in
https://github.com/nix-community/home-manager/issues/4977
Add a :+1: reaction to issues you find important.

You are trying with python 3.10 but the issue you linked seems to be specific to 3.12. Also currently the main supported python version is 3.11.

Can't reproduce this on nixos-unstable with python310Packages.debugpy, python311Packages.debugpy is cached and python312Packages.debugpy is marked broken because pbr isn't compatible with 3.12.

Is there a workaround to always pick the python package with the version currently supported by nixpkgs?

Yes, you can use python3Packages.

Thanks for your help!

I think I can reproduce this with
nix build --rebuild --no-link --print-out-paths \
    "github:NixOS/nixpkgs/9df3e30ce24fd28c7b3e2de0d986769db5d6225d#python311Packages.debugpy"

Here are the full build logs.
Can we reopen this issue, or should I open a new one?

Removing pytestFlagsArray = ["--timeout=0"]; reveals that a whole bunch of tests are timing out:
test_args
test_run
test_frame_eval
test_input
test_log_dir_env
test_redirect_output
test_stdin_not_patched
test_success_exitcodes
test_systemexit
test_with_no_output

Adding them to disabledTests fixes the build, however I have no idea why they are timing out, so blindly disabling them might not be the best idea.


Standardize event data for enabled/disabled collateral event

Read our Contributor Guidelines and README.
Parse data from event which fetched on-chain to humman-readable format, for example, now we have user = data[0] and we want to have user = data.user

In method parse_collateral_enabled_disabled_event

Event data structure enabled collateral - event data

In method process_collateral_disabled_event

Event data structure disabled collateral - event data

Create Serializers model using pydantic in serializers.py
In method process_collateral_enabled_event and
process_collateral_disabled_event substitute fetching data by index Example: user= data[0] to data.user using serializer class

I would like to take this task, I love 🍍

Mind if I try this one?

Hey! I'm Gerson. 👋
Member of Dojo Coding. I have contributed before to other Python projects like:

giza-agents

Can I work on this issue?
Check my OnlyDust profile, I've contributed to many projects: Profile

Plan to Solve the Issue:

Study the codebase to understand the current structure and functionality.
Understand the problem that needs to be solved by thoroughly reading the issue details.
Leverage all provided resources within the issue to get a better grasp of the task at hand.
If I encounter roadblocks, I will ask questions ASAP in the Telegram group to move forward effectively.
I will open a PR and wait for ur feedback

The file structure seems to have changed

Can I work on this? I would like to take this task, I love 🍍
ETA 2days

I'm a python dev, a new comer here. I want to take up this issue and solve it.

The file structure seems to have changed and there are errors in this implementation

Can you text me in telegram please

I'm a python dev, a new comer here. I want to take up this issue and solve it.

Please, read guideline first

Can I be assigned to this?

Is it okay if I tackle this?
kindly assign :)

'I would like to take this task, I love 🍍'
ETA = 2hrs
I am a skilled python developer and i have worked on a similar task

I'd like to handle this task.

@djeck1432 thanks for assigning, working right away.

@aniruddhaaps Sorry, you were unassigned, didn't manage with this task.

Can I work on this please?

@djeck1432 i will like to work on this

Available for assignment? I like pineapple

@djeck1432 , Can i take this task? I love 🍍

Mind if I try this one?


Pivot column value in relation manager table missing after allowDuplicates()

Package
filament/filament
Package Version
3.2.122
Laravel Version
11.30.0
Livewire Version
3.5.12
PHP Version
8.2.18
Problem description
relation manager table pivot column value missing after add allowDuplicates() for table in relation manager
Expected behavior
relation manager table should show pivot column value normally
Steps to reproduce

create a many to many relation for 2 models
add any additional pivot column for the pivot table
add pivot columns to both models using method withPivot()
create relation manager for 1 of the resource
add pivot column to the relation manager table
add a new record in relation manager with pivot column value
pivot column should show value
add allowDuplicates() for relation manager table
now the pivot column won't show the value anymore

Reproduction repository (issue will be closed if this is not valid)
https://github.com/huiyang/example-app
Relevant log output
No response

Donate 💰 to fund this issue

You can donate funding to this issue. We receive the money once the issue is completed & confirmed by you.
100% of the funding will be distributed between the Filament core team to run all aspects of the project.
Thank you in advance for helping us make maintenance sustainable!

currently i add these to my relation manager to solve my issue

    protected function hydratePivotRelationForTableRecords(Collection | Paginator | CursorPaginator $records): Collection | Paginator | CursorPaginator
    {
        $table = $this->getTable();
        $relationship = $table->getRelationship();

        if ($table->getRelationship() instanceof BelongsToMany /*&& ! $table->allowsDuplicates()*/) {
            invade($relationship)->hydratePivotRelation($records->all());
        }

        return $records;
    }

and
return $table
            ->allowDuplicates()
            ->modifyQueryUsing(function($query, $table) {
                /** @var BelongsToMany $relationship */
                $relationship = $table->getRelationship();

                $columns = [
                    $query->getModel()->getTable() . '.*',
                    $relationship->getTable() . '.*',         // reversed the select sequence, moved this line to later
                ];

                // if (! $this->allowsDuplicates()) {
                    $columns = [
                        ...invade($relationship)->aliasedPivotColumns(),
                        ...$columns,
                    ];
                // }

                $query->select($columns);
            })


RBD: OOMKills occurs when secret metadata encryption type is used with multiple PVC create request.

I tested secret based encryption with 3.7.1 i dont see any crash with below limits
Limits:
      cpu:     500m
      memory:  256Mi
    Requests:
      cpu:     250m
      memory:  256Mi

[🎩]mrajanna@fedora rbd $]kubectl get pvc
NAME      STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS      AGE
claim0    Bound    pvc-81cb47af-dfbd-4360-b444-7636e8a2c359   1Gi        RWO            rook-ceph-block   4m48s
claim1    Bound    pvc-c1c20239-db18-4c56-b9da-8745b0046428   1Gi        RWO            rook-ceph-block   4m47s
claim10   Bound    pvc-7d6e66d8-ceea-4e0c-9775-64aa84b1548b   1Gi        RWO            rook-ceph-block   4m46s
claim2    Bound    pvc-3a61744c-2d0e-46c1-9d8c-3b0f5f49574c   1Gi        RWO            rook-ceph-block   4m47s
claim3    Bound    pvc-aa2613fb-4db8-4254-801b-8d9d72e83979   1Gi        RWO            rook-ceph-block   4m47s
claim4    Bound    pvc-00fbe104-1809-4a11-8c39-9e3ceee9d5c9   1Gi        RWO            rook-ceph-block   4m47s
claim5    Bound    pvc-7ad36255-755b-4bdd-a88c-4bbf695e8b69   1Gi        RWO            rook-ceph-block   4m47s
claim6    Bound    pvc-bcd48a37-3d8d-47ce-b780-511020690397   1Gi        RWO            rook-ceph-block   4m47s
claim7    Bound    pvc-1fac5dcf-1668-489a-8799-16630b74e971   1Gi        RWO            rook-ceph-block   4m47s
claim8    Bound    pvc-949006fd-5b47-4e9c-acc1-3808894245f8   1Gi        RWO            rook-ceph-block   4m46s
claim9    Bound    pvc-05ab1ba5-4c8f-41a5-ab09-c042b6089b23   1Gi        RWO            rook-ceph-block   4m46s

but when i tested with metadata type encryption i can see the crash, this confirms we have a memory leak?
Originally posted by @Madhu-1 in https://github.com/ceph/ceph-csi/issues/3402#issuecomment-1278691078

based on testing with secrets metadata and 128Mi memory limit with https://github.com/ceph/ceph-csi/pull/3422 fix
Seems to crash after this log continuously
https://github.com/ceph/ceph-csi/blob/07e9dede2cb10ceaa055b9b2486f688edf9008d6/internal/rbd/rbd_journal.go#L345
Only important thing that occurs after this logline is rbdvol.Destroy(), I think this is causing OOMKill
Lifting up memory limit will lead to all PVCs going to bound state and can be succesfully mounted to a pod.
I1104 10:08:34.816009       1 cephcsi.go:192] Driver version: canary and Git version: e91b849ffcccafc8f374855e2537c38eaf2baeac
I1104 10:08:34.816188       1 cephcsi.go:210] Initial PID limit is set to 14167
I1104 10:08:34.816233       1 cephcsi.go:216] Reconfigured PID limit to -1 (max)
I1104 10:08:34.816405       1 cephcsi.go:241] Starting driver type: rbd with name: rook-ceph.rbd.csi.ceph.com
I1104 10:08:34.816555       1 driver.go:94] Enabling controller service capability: CREATE_DELETE_VOLUME
I1104 10:08:34.816562       1 driver.go:94] Enabling controller service capability: CREATE_DELETE_SNAPSHOT
I1104 10:08:34.816565       1 driver.go:94] Enabling controller service capability: CLONE_VOLUME
I1104 10:08:34.816567       1 driver.go:94] Enabling controller service capability: EXPAND_VOLUME
I1104 10:08:34.816570       1 driver.go:107] Enabling volume access mode: SINGLE_NODE_WRITER
I1104 10:08:34.816577       1 driver.go:107] Enabling volume access mode: MULTI_NODE_MULTI_WRITER
I1104 10:08:34.816580       1 driver.go:107] Enabling volume access mode: SINGLE_NODE_SINGLE_WRITER
I1104 10:08:34.816582       1 driver.go:107] Enabling volume access mode: SINGLE_NODE_MULTI_WRITER
W1104 10:08:34.816589       1 driver.go:162] replication service running on controller server is deprecated and replaced by CSI-Addons, see https://github.com/ceph/ceph-csi/issues/3314 for more details
I1104 10:08:34.816675       1 server.go:126] Listening for connections on address: &net.UnixAddr{Name:"//csi/csi-provisioner.sock", Net:"unix"}
I1104 10:08:34.816679       1 server.go:114] listening for CSI-Addons requests on address: &net.UnixAddr{Name:"/csi/csi-addons.sock", Net:"unix"}
I1104 10:08:35.450896       1 utils.go:195] ID: 1 GRPC call: /csi.v1.Identity/Probe
I1104 10:08:35.451802       1 utils.go:206] ID: 1 GRPC request: {}
I1104 10:08:35.451921       1 utils.go:212] ID: 1 GRPC response: {}
I1104 10:08:35.452453       1 utils.go:195] ID: 2 GRPC call: /csi.v1.Identity/GetPluginInfo
I1104 10:08:35.452476       1 utils.go:206] ID: 2 GRPC request: {}
I1104 10:08:35.452484       1 identityserver-default.go:39] ID: 2 Using default GetPluginInfo
I1104 10:08:35.452546       1 utils.go:212] ID: 2 GRPC response: {"name":"rook-ceph.rbd.csi.ceph.com","vendor_version":"canary"}
I1104 10:08:35.452814       1 utils.go:195] ID: 3 GRPC call: /csi.v1.Identity/GetPluginCapabilities
I1104 10:08:35.452844       1 utils.go:206] ID: 3 GRPC request: {}
I1104 10:08:35.453035       1 utils.go:212] ID: 3 GRPC response: {"capabilities":[{"Type":{"Service":{"type":1}}},{"Type":{"VolumeExpansion":{"type":1}}},{"Type":{"Service":{"type":2}}}]}
I1104 10:08:35.453540       1 utils.go:195] ID: 4 GRPC call: /csi.v1.Controller/ControllerGetCapabilities
I1104 10:08:35.453571       1 utils.go:206] ID: 4 GRPC request: {}
I1104 10:08:35.453579       1 controllerserver-default.go:72] ID: 4 Using default ControllerGetCapabilities
I1104 10:08:35.453675       1 utils.go:212] ID: 4 GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":7}}},{"Type":{"Rpc":{"type":9}}}]}
I1104 10:10:57.438805       1 utils.go:195] ID: 5 Req-ID: pvc-3416c8ae-6eb7-484c-8824-272c5cea7bac GRPC call: /csi.v1.Controller/CreateVolume
I1104 10:10:57.439115       1 utils.go:195] ID: 6 Req-ID: pvc-21bab1d6-2e7e-4ec5-a6f1-ab5504487ee4 GRPC call: /csi.v1.Controller/CreateVolume
I1104 10:10:57.440471       1 utils.go:206] ID: 5 Req-ID: pvc-3416c8ae-6eb7-484c-8824-272c5cea7bac GRPC request: {"capacity_range":{"required_bytes":1073741824},"name":"pvc-3416c8ae-6eb7-484c-8824-272c5cea7bac","parameters":{"clusterID":"rook-ceph","csi.storage.k8s.io/pv/name":"pvc-3416c8ae-6eb7-484c-8824-272c5cea7bac","csi.storage.k8s.io/pvc/name":"rbd-pvc-2","csi.storage.k8s.io/pvc/namespace":"rook-ceph","encrypted":"true","encryptionKMSID":"user-secrets-metadata-test","imageFeatures":"layering","imageFormat":"2","pool":"replicapool"},"secrets":"***stripped***","volume_capabilities":[{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":1}}]}
I1104 10:10:57.441397       1 rbd_util.go:1279] ID: 5 Req-ID: pvc-3416c8ae-6eb7-484c-8824-272c5cea7bac setting disableInUseChecks: false image features: [layering] mounter: rbd
I1104 10:10:57.445250       1 utils.go:206] ID: 6 Req-ID: pvc-21bab1d6-2e7e-4ec5-a6f1-ab5504487ee4 GRPC request: {"capacity_range":{"required_bytes":1073741824},"name":"pvc-21bab1d6-2e7e-4ec5-a6f1-ab5504487ee4","parameters":{"clusterID":"rook-ceph","csi.storage.k8s.io/pv/name":"pvc-21bab1d6-2e7e-4ec5-a6f1-ab5504487ee4","csi.storage.k8s.io/pvc/name":"rbd-pvc-1","csi.storage.k8s.io/pvc/namespace":"rook-ceph","encrypted":"true","encryptionKMSID":"user-secrets-metadata-test","imageFeatures":"layering","imageFormat":"2","pool":"replicapool"},"secrets":"***stripped***","volume_capabilities":[{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":1}}]}
I1104 10:10:57.448371       1 utils.go:195] ID: 7 Req-ID: pvc-48c2c85c-a826-4755-9b9e-0e75df34c1f2 GRPC call: /csi.v1.Controller/CreateVolume
I1104 10:10:57.511231       1 rbd_util.go:1279] ID: 6 Req-ID: pvc-21bab1d6-2e7e-4ec5-a6f1-ab5504487ee4 setting disableInUseChecks: false image features: [layering] mounter: rbd
I1104 10:10:57.511792       1 utils.go:206] ID: 7 Req-ID: pvc-48c2c85c-a826-4755-9b9e-0e75df34c1f2 GRPC request: {"capacity_range":{"required_bytes":1073741824},"name":"pvc-48c2c85c-a826-4755-9b9e-0e75df34c1f2","parameters":{"clusterID":"rook-ceph","csi.storage.k8s.io/pv/name":"pvc-48c2c85c-a826-4755-9b9e-0e75df34c1f2","csi.storage.k8s.io/pvc/name":"rbd-pvc-5","csi.storage.k8s.io/pvc/namespace":"rook-ceph","encrypted":"true","encryptionKMSID":"user-secrets-metadata-test","imageFeatures":"layering","imageFormat":"2","pool":"replicapool"},"secrets":"***stripped***","volume_capabilities":[{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":1}}]}
I1104 10:10:57.512114       1 rbd_util.go:1279] ID: 7 Req-ID: pvc-48c2c85c-a826-4755-9b9e-0e75df34c1f2 setting disableInUseChecks: false image features: [layering] mounter: rbd
I1104 10:10:57.450004       1 utils.go:195] ID: 8 Req-ID: pvc-e76394a0-409e-46b4-b1cd-b8eb676c54f0 GRPC call: /csi.v1.Controller/CreateVolume
I1104 10:10:57.513912       1 utils.go:206] ID: 8 Req-ID: pvc-e76394a0-409e-46b4-b1cd-b8eb676c54f0 GRPC request: {"capacity_range":{"required_bytes":1073741824},"name":"pvc-e76394a0-409e-46b4-b1cd-b8eb676c54f0","parameters":{"clusterID":"rook-ceph","csi.storage.k8s.io/pv/name":"pvc-e76394a0-409e-46b4-b1cd-b8eb676c54f0","csi.storage.k8s.io/pvc/name":"rbd-pvc-4","csi.storage.k8s.io/pvc/namespace":"rook-ceph","encrypted":"true","encryptionKMSID":"user-secrets-metadata-test","imageFeatures":"layering","imageFormat":"2","pool":"replicapool"},"secrets":"***stripped***","volume_capabilities":[{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":1}}]}
I1104 10:10:57.450170       1 utils.go:195] ID: 9 Req-ID: pvc-179792d9-c723-4106-b3d1-b0532f1f96f1 GRPC call: /csi.v1.Controller/CreateVolume
I1104 10:10:57.514431       1 utils.go:206] ID: 9 Req-ID: pvc-179792d9-c723-4106-b3d1-b0532f1f96f1 GRPC request: {"capacity_range":{"required_bytes":1073741824},"name":"pvc-179792d9-c723-4106-b3d1-b0532f1f96f1","parameters":{"clusterID":"rook-ceph","csi.storage.k8s.io/pv/name":"pvc-179792d9-c723-4106-b3d1-b0532f1f96f1","csi.storage.k8s.io/pvc/name":"rbd-pvc-3","csi.storage.k8s.io/pvc/namespace":"rook-ceph","encrypted":"true","encryptionKMSID":"user-secrets-metadata-test","imageFeatures":"layering","imageFormat":"2","pool":"replicapool"},"secrets":"***stripped***","volume_capabilities":[{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":1}}]}
I1104 10:10:57.514731       1 rbd_util.go:1279] ID: 9 Req-ID: pvc-179792d9-c723-4106-b3d1-b0532f1f96f1 setting disableInUseChecks: false image features: [layering] mounter: rbd
I1104 10:10:57.515382       1 rbd_util.go:1279] ID: 8 Req-ID: pvc-e76394a0-409e-46b4-b1cd-b8eb676c54f0 setting disableInUseChecks: false image features: [layering] mounter: rbd
I1104 10:10:58.512480       1 omap.go:88] ID: 7 Req-ID: pvc-48c2c85c-a826-4755-9b9e-0e75df34c1f2 got omap values: (pool="replicapool", namespace="", name="csi.volumes.default"): map[csi.volume.pvc-48c2c85c-a826-4755-9b9e-0e75df34c1f2:a6f79f2d-e921-49e0-ab3d-d3f8a99ec978]
I1104 10:10:58.515450       1 omap.go:88] ID: 9 Req-ID: pvc-179792d9-c723-4106-b3d1-b0532f1f96f1 got omap values: (pool="replicapool", namespace="", name="csi.volumes.default"): map[csi.volume.pvc-179792d9-c723-4106-b3d1-b0532f1f96f1:84db196d-ab12-4b53-a3e1-36e9e9dc4429]
I1104 10:10:58.515609       1 omap.go:88] ID: 7 Req-ID: pvc-48c2c85c-a826-4755-9b9e-0e75df34c1f2 got omap values: (pool="replicapool", namespace="", name="csi.volume.a6f79f2d-e921-49e0-ab3d-d3f8a99ec978"): map[csi.imagename:csi-vol-a6f79f2d-e921-49e0-ab3d-d3f8a99ec978 csi.volname:pvc-48c2c85c-a826-4755-9b9e-0e75df34c1f2 csi.volume.encryptKMS:user-secrets-metadata-test csi.volume.owner:rook-ceph]
I1104 10:10:58.515776       1 omap.go:88] ID: 5 Req-ID: pvc-3416c8ae-6eb7-484c-8824-272c5cea7bac got omap values: (pool="replicapool", namespace="", name="csi.volumes.default"): map[csi.volume.pvc-3416c8ae-6eb7-484c-8824-272c5cea7bac:28966dbc-5810-43b7-8381-a5f12a6a12fd]
I1104 10:10:58.517980       1 omap.go:88] ID: 9 Req-ID: pvc-179792d9-c723-4106-b3d1-b0532f1f96f1 got omap values: (pool="replicapool", namespace="", name="csi.volume.84db196d-ab12-4b53-a3e1-36e9e9dc4429"): map[csi.imagename:csi-vol-84db196d-ab12-4b53-a3e1-36e9e9dc4429 csi.volname:pvc-179792d9-c723-4106-b3d1-b0532f1f96f1 csi.volume.encryptKMS:user-secrets-metadata-test csi.volume.owner:rook-ceph]
I1104 10:10:58.519165       1 omap.go:88] ID: 5 Req-ID: pvc-3416c8ae-6eb7-484c-8824-272c5cea7bac got omap values: (pool="replicapool", namespace="", name="csi.volume.28966dbc-5810-43b7-8381-a5f12a6a12fd"): map[csi.imagename:csi-vol-28966dbc-5810-43b7-8381-a5f12a6a12fd csi.volname:pvc-3416c8ae-6eb7-484c-8824-272c5cea7bac csi.volume.encryptKMS:user-secrets-metadata-test csi.volume.owner:rook-ceph]
I1104 10:10:58.521710       1 omap.go:88] ID: 6 Req-ID: pvc-21bab1d6-2e7e-4ec5-a6f1-ab5504487ee4 got omap values: (pool="replicapool", namespace="", name="csi.volumes.default"): map[csi.volume.pvc-21bab1d6-2e7e-4ec5-a6f1-ab5504487ee4:790e02db-a332-4e10-8db6-25debf9dcd5c]
I1104 10:10:58.523374       1 omap.go:88] ID: 6 Req-ID: pvc-21bab1d6-2e7e-4ec5-a6f1-ab5504487ee4 got omap values: (pool="replicapool", namespace="", name="csi.volume.790e02db-a332-4e10-8db6-25debf9dcd5c"): map[csi.imagename:csi-vol-790e02db-a332-4e10-8db6-25debf9dcd5c csi.volname:pvc-21bab1d6-2e7e-4ec5-a6f1-ab5504487ee4 csi.volume.encryptKMS:user-secrets-metadata-test csi.volume.owner:rook-ceph]
I1104 10:10:58.611162       1 omap.go:88] ID: 8 Req-ID: pvc-e76394a0-409e-46b4-b1cd-b8eb676c54f0 got omap values: (pool="replicapool", namespace="", name="csi.volumes.default"): map[csi.volume.pvc-e76394a0-409e-46b4-b1cd-b8eb676c54f0:23f9efa6-adfb-49b0-be4c-b8bc8acb5658]
I1104 10:10:58.611729       1 omap.go:88] ID: 8 Req-ID: pvc-e76394a0-409e-46b4-b1cd-b8eb676c54f0 got omap values: (pool="replicapool", namespace="", name="csi.volume.23f9efa6-adfb-49b0-be4c-b8bc8acb5658"): map[csi.imagename:csi-vol-23f9efa6-adfb-49b0-be4c-b8bc8acb5658 csi.volname:pvc-e76394a0-409e-46b4-b1cd-b8eb676c54f0 csi.volume.encryptKMS:user-secrets-metadata-test csi.volume.owner:rook-ceph]
I1104 10:10:58.737545       1 omap.go:158] ID: 7 Req-ID: pvc-48c2c85c-a826-4755-9b9e-0e75df34c1f2 set omap keys (pool="replicapool", namespace="", name="csi.volume.a6f79f2d-e921-49e0-ab3d-d3f8a99ec978"): map[csi.imageid:11a6728409fa])
I1104 10:10:58.737545       1 omap.go:158] ID: 5 Req-ID: pvc-3416c8ae-6eb7-484c-8824-272c5cea7bac set omap keys (pool="replicapool", namespace="", name="csi.volume.28966dbc-5810-43b7-8381-a5f12a6a12fd"): map[csi.imageid:11a6a8672781])
I1104 10:10:58.737611       1 rbd_journal.go:337] ID: 7 Req-ID: pvc-48c2c85c-a826-4755-9b9e-0e75df34c1f2 found existing volume (0001-0009-rook-ceph-0000000000000001-a6f79f2d-e921-49e0-ab3d-d3f8a99ec978) with image name (csi-vol-a6f79f2d-e921-49e0-ab3d-d3f8a99ec978) for request (pvc-48c2c85c-a826-4755-9b9e-0e75df34c1f2)
I1104 10:10:58.737611       1 rbd_journal.go:337] ID: 5 Req-ID: pvc-3416c8ae-6eb7-484c-8824-272c5cea7bac found existing volume (0001-0009-rook-ceph-0000000000000001-28966dbc-5810-43b7-8381-a5f12a6a12fd) with image name (csi-vol-28966dbc-5810-43b7-8381-a5f12a6a12fd) for request (pvc-3416c8ae-6eb7-484c-8824-272c5cea7bac)
I1104 10:10:58.811046       1 omap.go:158] ID: 9 Req-ID: pvc-179792d9-c723-4106-b3d1-b0532f1f96f1 set omap keys (pool="replicapool", namespace="", name="csi.volume.84db196d-ab12-4b53-a3e1-36e9e9dc4429"): map[csi.imageid:11a6b1278881])
I1104 10:10:58.811481       1 rbd_journal.go:337] ID: 9 Req-ID: pvc-179792d9-c723-4106-b3d1-b0532f1f96f1 found existing volume (0001-0009-rook-ceph-0000000000000001-84db196d-ab12-4b53-a3e1-36e9e9dc4429) with image name (csi-vol-84db196d-ab12-4b53-a3e1-36e9e9dc4429) for request (pvc-179792d9-c723-4106-b3d1-b0532f1f96f1)
I1104 10:10:58.910697       1 omap.go:158] ID: 6 Req-ID: pvc-21bab1d6-2e7e-4ec5-a6f1-ab5504487ee4 set omap keys (pool="replicapool", namespace="", name="csi.volume.790e02db-a332-4e10-8db6-25debf9dcd5c"): map[csi.imageid:11a64b6bbbc6])
I1104 10:10:58.910804       1 rbd_journal.go:337] ID: 6 Req-ID: pvc-21bab1d6-2e7e-4ec5-a6f1-ab5504487ee4 found existing volume (0001-0009-rook-ceph-0000000000000001-790e02db-a332-4e10-8db6-25debf9dcd5c) with image name (csi-vol-790e02db-a332-4e10-8db6-25debf9dcd5c) for request (pvc-21bab1d6-2e7e-4ec5-a6f1-ab5504487ee4)
I1104 10:10:59.010592       1 omap.go:158] ID: 8 Req-ID: pvc-e76394a0-409e-46b4-b1cd-b8eb676c54f0 set omap keys (pool="replicapool", namespace="", name="csi.volume.23f9efa6-adfb-49b0-be4c-b8bc8acb5658"): map[csi.imageid:11a6e8058d08])
I1104 10:10:59.010793       1 rbd_journal.go:337] ID: 8 Req-ID: pvc-e76394a0-409e-46b4-b1cd-b8eb676c54f0 found existing volume (0001-0009-rook-ceph-0000000000000001-23f9efa6-adfb-49b0-be4c-b8bc8acb5658) with image name (csi-vol-23f9efa6-adfb-49b0-be4c-b8bc8acb5658) for request (pvc-e76394a0-409e-46b4-b1cd-b8eb676c54f0)

Edit

Only important thing that occurs after this logline is rbdvol.Destroy(), I think this is causing OOMKill

My fix will call setupencryption() again in repairvol step. This is causing looping OOMKill.

OOMKill is due to this function
https://github.com/ceph/ceph-csi/blob/c65a4e1d8a869a0aa2c174259873b70f560d0716/internal/kms/secretskms.go#L274
<EMAIL_ADDRESS>This issue is isolated to secrets-metadata kms alone.

/follow

scrypt.Key

Interesting!
There have been some reports  in the go crypto package issue tracker about scrypt memory:
https://github.com/golang/go/issues/20000
https://github.com/golang/go/issues/40687
https://github.com/golang/go/issues/7168
Is this perhaps not a leak, but the GC not kicking in soon enough?

I have been looking into this a bit. Sharing my findings here hoping that they will be useful.

Confirmed that scrypt.Key is responsible for allocating most of the memory.
Why does it need so much? This line along is responsible for 8MB if I'm not mistaken:
https://github.com/ceph/ceph-csi/blob/c65a4e1d8a869a0aa2c174259873b70f560d0716/vendor/golang.org/x/crypto/scrypt/scrypt.go#L204
With the parameters we pass we get 32×32768×8 = 8388608 or 8 MB. When serving multiple requests at the same time, this quickly adds up!
I have check the memory volumes also, but they don't play a significant part in this from what I can tell. I initially suspected that we may be writing something large there.
Is GC slow? I'm not sure. If there are parallel requests I don't think there is any reasonable chance for it really. But I saw something strange actually: The memory usage drops suddenly about 4-5 minutes after creating the PVCs. Any clue what could be causing that?
Perhaps related to 4, there is an issue (https://github.com/kubernetes/kubernetes/issues/43916) about active page cache counted as "used memory". Not sure if it has anything to do with this though.

I'm not sure how to interpret these really, but here is the memory.stat of the container right before it was OOM killed (memory limit set to 128 MiB):
cache 1486848
rss 106082304
rss_huge 69206016
shmem 0
mapped_file 811008
dirty 0
writeback 0
swap 0
pgpgin 24156
pgpgout 11364
pgfault 24915
pgmajfault 0
inactive_anon 106618880
active_anon 0
inactive_file 868352
active_file 405504
unevictable 0
hierarchical_memory_limit 134217728
hierarchical_memsw_limit 134217728
total_cache 1486848
total_rss 106082304
total_rss_huge 69206016
total_shmem 0
total_mapped_file 811008
total_dirty 0
total_writeback 0
total_swap 0
total_pgpgin 24156
total_pgpgout 11364
total_pgfault 24915
total_pgmajfault 0
total_inactive_anon 106618880
total_active_anon 0
total_inactive_file 868352
total_active_file 405504
total_unevictable 0

And this is after it restarted:
cache 5136384
rss 23924736
rss_huge 0
shmem 0
mapped_file 2973696
dirty 0
writeback 0
swap 0
pgpgin 9471
pgpgout 2426
pgfault 10395
pgmajfault 33
inactive_anon 23916544
active_anon 0
inactive_file 4026368
active_file 942080
unevictable 0
hierarchical_memory_limit 134217728
hierarchical_memsw_limit 134217728
total_cache 5136384
total_rss 23924736
total_rss_huge 0
total_shmem 0
total_mapped_file 2973696
total_dirty 0
total_writeback 0
total_swap 0
total_pgpgin 9471
total_pgpgout 2426
total_pgfault 10395
total_pgmajfault 33
total_inactive_anon 23916544
total_active_anon 0
total_inactive_file 4026368
total_active_file 942080
total_unevictable 0

One thing that caught my eye is the rss_huge.
Anything else I could check here to help?

Some more experiments also hints that the issue is the parallel scrypt.Key calls. With a memory limit of 256MiB, I can create 100 PVCs without issue as long as I do it one at a time with 1 sec sleep between. On the other hand, if I create 5-10 PVCs in one go I immediately get OOM.
Any ideas for how to solve this? I'm not familiar with the code so unsure where to start looking. :slightly_frowning_face:

Is anyone planning to work on this?
I was looking at it briefly last week, but this code is new to me and not immediately obvious how I would solve the issue. Then I saw that it is marked as highest priority and thought I might as well wait until someone that knows the code takes a look :smile:

@lentzi90 please feel free to take it

Is anyone planning to work on this? I was looking at it briefly last week, but this code is new to me and not immediately obvious how I would solve the issue. Then I saw that it is marked as highest priority and thought I might as well wait until someone that knows the code takes a look smile

Currently no one is working on this, you can take this up.
As @nixpanic suggested we can surround the call to scrypt.Key with a semaphore to avoid this issue.

I picked this up today. Added a semaphore as suggested, proceeded to test it. Everything was looking good. With 256Mi limit I had no issues creating 100 PVCs. Then I went on to check that the issue still existed when using the upstream canary image and... had difficulty reproducing the issue :open_mouth:
It seems to me that it was actually solved already. I checked the commit log and found this one that could explain it: https://github.com/ceph/ceph-csi/commit/36fe1450f1b8cf4597c07d1cb796b25419e7fb2a
From the release notes:

ringhash: impose cap on max_ring_size to reduce possibility of OOMs (https://github.com/grpc/grpc-go/pull/5801)

Can someone else confirm?

On Fri, Jan 20, 2023 at 05:44:06AM -0800, Lennart Jern wrote:

I picked this up today. Added a semaphore as suggested, proceeded to test it. Everything was looking good. With 256Mi limit I had no issues creating 100 PVCs. Then I went on to check that the issue still existed when using the upstream canary image and... had difficulty reproducing the issue :open_mouth:
It seems to me that it was actually solved already. I checked the commit log and found this one that could explain it: https://github.com/ceph/ceph-csi/commit/36fe1450f1b8cf4597c07d1cb796b25419e7fb2a
From the release notes:

ringhash: impose cap on max_ring_size to reduce possibility of OOMs (https://github.com/grpc/grpc-go/pull/5801)

Can someone else confirm?

Thanks for looking into this :)
Good find! I have no idea if that is related... You could test your
build with @.*** and see if that makes a
difference.
As creating the encryption keys is also very CPU intensive, having a
semaphore that limits the amount of concurrent requests would still be a
nice thing to have.

As creating the encryption keys is also very CPU intensive, having a semaphore that limits the amount of concurrent requests would still be a nice thing to have.

I would say this should be a separate issue and perhaps not best solved with a semaphore. What about using CPU limits? OOM was a big issue since the container would not recover from it and just go OOM again. But I didn't see any issues with CPU usage in my tests (all could easily be tested on a laptop, it stayed responsive throughout), so unless there is some concrete example of how and why a semaphore is needed for that, I would avoid it.
It would be nice to get confirmation from someone else on the OOM issue though before closing this.

I would say this should be a separate issue and perhaps not best solved with a semaphore. What about using CPU limits?

I had hard times in figuring out a proper CPU limit and finally solved the OOM issue making my pod activities heavily CPU bounded, which helped with OOMs but slowed all the other operations.
A CPU throttling mechanism restricted to the most CPU intensive activities would be really appreciated...

I picked this up today. Added a semaphore as suggested, proceeded to test it. Everything was looking good. With 256Mi limit I had no issues creating 100 PVCs. Then I went on to check that the issue still existed when using the upstream canary image and... had difficulty reproducing the issue open_mouth
It seems to me that it was actually solved already. I checked the commit log and found this one that could explain it: 36fe145 From the release notes:

ringhash: impose cap on max_ring_size to reduce possibility of OOMs (grpc/grpc-go#5801)

Although this change may improve it, I think the idea to limit the number of concurrent requests is still valid. Creating 100 PVS  simultaneously will spawn 100 cryptsetup processes, and the pod memory utilization may reach the limit.

@paolo-depa what version are you using? The original issue was reported for v3.7.1. Now v3.8.0 is out and from my tests before I would expect it to be fixed there.

Just did some tests creating 100 pods using the v.3.8.0:

Limiting cpu to 150m and mem to 1Gb, I saw 2 OOMkill events - one of those making the container respawn - but still had 100% of pods active within 40 mins
Raising cpu limit to 1 and leaving mem untouched,  I saw 30 OOMkill events - one of those making the container respawn - ending up in 87 working / 13 non working pods in 15 minutes

Adding a snippet from the message log of one of those events:
2023-02-23T16:48:06.349522+01:00 germ39 kernel: [ 5733.862631] cryptsetup invoked oom-killer: gfp_mask=0xcc0(GFP_KERNEL), order=0, oom_score_adj=-997 2023-02-23T16:48:06.349544+01:00 germ39 kernel: [ 5733.862636] CPU: 1 PID: 27451 Comm: cryptsetup Kdump: loaded Not tainted 5.3.18-150300.59.93-default #1 SLE15-SP3 2023-02-23T16:48:06.349545+01:00 germ39 kernel: [ 5733.862636] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 0.0.0 02/06/2015 2023-02-23T16:48:06.349546+01:00 germ39 kernel: [ 5733.862637] Call Trace: 2023-02-23T16:48:06.349548+01:00 germ39 kernel: [ 5733.862646]  dump_stack+0x66/0x8b 2023-02-23T16:48:06.349549+01:00 germ39 kernel: [ 5733.862650]  dump_header+0x4a/0x210 2023-02-23T16:48:06.349549+01:00 germ39 kernel: [ 5733.862652]  oom_kill_process+0xec/0x130 2023-02-23T16:48:06.349570+01:00 germ39 kernel: [ 5733.862654]  out_of_memory+0x105/0x510 2023-02-23T16:48:06.349571+01:00 germ39 kernel: [ 5733.862657]  mem_cgroup_out_of_memory+0xb9/0xd0 2023-02-23T16:48:06.349572+01:00 germ39 kernel: [ 5733.862659]  try_charge+0x7c4/0x800 2023-02-23T16:48:06.349573+01:00 germ39 kernel: [ 5733.862662]  ? __alloc_pages_nodemask+0x15d/0x320 2023-02-23T16:48:06.349573+01:00 germ39 kernel: [ 5733.862663]  mem_cgroup_try_charge+0x70/0x190 2023-02-23T16:48:06.349574+01:00 germ39 kernel: [ 5733.862665]  mem_cgroup_try_charge_delay+0x1c/0x40 2023-02-23T16:48:06.349575+01:00 germ39 kernel: [ 5733.862668]  __handle_mm_fault+0x96d/0x1260 2023-02-23T16:48:06.349576+01:00 germ39 kernel: [ 5733.862670]  handle_mm_fault+0xc4/0x210 2023-02-23T16:48:06.349577+01:00 germ39 kernel: [ 5733.862671]  __get_user_pages+0x1f7/0x780 2023-02-23T16:48:06.349578+01:00 germ39 kernel: [ 5733.862673]  populate_vma_page_range+0x6d/0x80 2023-02-23T16:48:06.349578+01:00 germ39 kernel: [ 5733.862674]  __mm_populate+0xa9/0x150 2023-02-23T16:48:06.349579+01:00 germ39 kernel: [ 5733.862676]  vm_mmap_pgoff+0x113/0x130 2023-02-23T16:48:06.349579+01:00 germ39 kernel: [ 5733.862678]  ksys_mmap_pgoff+0xfb/0x220 2023-02-23T16:48:06.349580+01:00 germ39 kernel: [ 5733.862681]  do_syscall_64+0x5b/0x1e0 2023-02-23T16:48:06.349581+01:00 germ39 kernel: [ 5733.862684]  entry_SYSCALL_64_after_hwframe+0x61/0xc6 2023-02-23T16:48:06.349582+01:00 germ39 kernel: [ 5733.862687] RIP: 0033:0x7efd51456957 2023-02-23T16:48:06.349583+01:00 germ39 kernel: [ 5733.862689] Code: 54 41 89 d4 55 48 89 fd 53 4c 89 cb 48 85 ff 74 52 49 89 d9 45 89 f8 45 89 f2 44 89 e2 4c 89 ee 48 89 ef b8 09 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 79 5b 5d 41 5c 41 5d 41 5e 41 5f c3 66 0f 1f 2023-02-23T16:48:06.349583+01:00 germ39 kernel: [ 5733.862689] RSP: 002b:00007fffa47e7958 EFLAGS: 00000246 ORIG_RAX:<PHONE_NUMBER>000009 2023-02-23T16:48:06.349584+01:00 germ39 kernel: [ 5733.862691] RAX: ffffffffffffffda RBX:<PHONE_NUMBER>000000 RCX: 00007efd51456957 2023-02-23T16:48:06.349586+01:00 germ39 kernel: [ 5733.862691] RDX:<PHONE_NUMBER>000003 RSI:<PHONE_NUMBER>7c1000 RDI:<PHONE_NUMBER>000000 2023-02-23T16:48:06.349587+01:00 germ39 kernel: [ 5733.862692] RBP:<PHONE_NUMBER>000000 R08: 00000000ffffffff R09:<PHONE_NUMBER>000000 2023-02-23T16:48:06.349587+01:00 germ39 kernel: [ 5733.862692] R10:<PHONE_NUMBER>000022 R11:<PHONE_NUMBER>000246 R12:<PHONE_NUMBER>000003 2023-02-23T16:48:06.349588+01:00 germ39 kernel: [ 5733.862693] R13:<PHONE_NUMBER>7c1000 R14:<PHONE_NUMBER>000022 R15: 00000000ffffffff 2023-02-23T16:48:06.349589+01:00 germ39 kernel: [ 5733.862695] memory: usage 1048576kB, limit 1048576kB, failcnt<PHONE_NUMBER>-02-23T16:48:06.349590+01:00 germ39 kernel: [ 5733.862695] memory+swap: usage 0kB, limit<PHONE_NUMBER>740988kB, failcnt 0 2023-02-23T16:48:06.349591+01:00 germ39 kernel: [ 5733.862696] kmem: usage 20064kB, limit<PHONE_NUMBER>740988kB, failcnt 0 2023-02-23T16:48:06.349592+01:00 germ39 kernel: [ 5733.862696] Memory cgroup stats for /kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd17563cd_9da0_4d86_a89a_a4cb0f8070b3.slice/cri-containerd-f5ec1120df909b8125ebf2a5c2172e38fa978122c021c4e82afaa77cf5595f99.scope: 2023-02-23T16:48:06.349593+01:00 germ39 kernel: [ 5733.862708] anon<PHONE_NUMBER> 2023-02-23T16:48:06.349594+01:00 germ39 kernel: [ 5733.862708] file<PHONE_NUMBER>-02-23T16:48:06.349594+01:00 germ39 kernel: [ 5733.862708] kernel_stack 1253376 2023-02-23T16:48:06.349595+01:00 germ39 kernel: [ 5733.862708] slab 11677696 2023-02-23T16:48:06.349595+01:00 germ39 kernel: [ 5733.862708] sock 0 2023-02-23T16:48:06.349596+01:00 germ39 kernel: [ 5733.862708] shmem 0 2023-02-23T16:48:06.349597+01:00 germ39 kernel: [ 5733.862708] file_mapped 0 2023-02-23T16:48:06.349597+01:00 germ39 kernel: [ 5733.862708] file_dirty<PHONE_NUMBER>-02-23T16:48:06.349598+01:00 germ39 kernel: [ 5733.862708] file_writeback<PHONE_NUMBER>-02-23T16:48:06.349599+01:00 germ39 kernel: [ 5733.862708] anon_thp 557842432 2023-02-23T16:48:06.349599+01:00 germ39 kernel: [ 5733.862708] inactive_anon 49152 2023-02-23T16:48:06.349600+01:00 germ39 kernel: [ 5733.862708] active_anon<PHONE_NUMBER>-02-23T16:48:06.349601+01:00 germ39 kernel: [ 5733.862708] inactive_file<PHONE_NUMBER>-02-23T16:48:06.349601+01:00 germ39 kernel: [ 5733.862708] active_file<PHONE_NUMBER>-02-23T16:48:06.349602+01:00 germ39 kernel: [ 5733.862708] unevictable<PHONE_NUMBER> 2023-02-23T16:48:06.349603+01:00 germ39 kernel: [ 5733.862708] slab_reclaimable 6782976 2023-02-23T16:48:06.349603+01:00 germ39 kernel: [ 5733.862708] slab_unreclaimable 4894720 2023-02-23T16:48:06.349604+01:00 germ39 kernel: [ 5733.862708] pgfault 1986534 2023-02-23T16:48:06.349604+01:00 germ39 kernel: [ 5733.862708] pgmajfault 56991 2023-02-23T16:48:06.349605+01:00 germ39 kernel: [ 5733.862708] workingset_refault 1815 2023-02-23T16:48:06.349605+01:00 germ39 kernel: [ 5733.862708] workingset_activate 924 2023-02-23T16:48:06.349606+01:00 germ39 kernel: [ 5733.862708] workingset_nodereclaim 0 2023-02-23T16:48:06.349607+01:00 germ39 kernel: [ 5733.862708] pgrefill 1219243 2023-02-23T16:48:06.349607+01:00 germ39 kernel: [ 5733.862708] pgscan 2018920 2023-02-23T16:48:06.349608+01:00 germ39 kernel: [ 5733.862708] pgsteal<PHONE_NUMBER>-02-23T16:48:06.349609+01:00 germ39 kernel: [ 5733.862708] pgactivate<PHONE_NUMBER>-02-23T16:48:06.349610+01:00 germ39 kernel: [ 5733.862708] pgdeactivate 1219243 2023-02-23T16:48:06.349611+01:00 germ39 kernel: [ 5733.862708] pglazyfree 0 2023-02-23T16:48:06.349611+01:00 germ39 kernel: [ 5733.862708] Tasks state (memory values in pages): 2023-02-23T16:48:06.349612+01:00 germ39 kernel: [ 5733.862709] [  pid  ]   uid  tgid total_vm      rss pgtables_bytes swapents oom_score_adj name 2023-02-23T16:48:06.349613+01:00 germ39 kernel: [ 5733.862712] [   3868]     0  3868  1228537    15026  3878912   400476          -997 cephcsi 2023-02-23T16:48:06.349613+01:00 germ39 kernel: [ 5733.862714] [  23073]     0 23073    52104    24466   339968        0          -997 cryptsetup 2023-02-23T16:48:06.349614+01:00 germ39 kernel: [ 5733.862715] [  23854]     0 23854    53748    26083   352256        0          -997 cryptsetup 2023-02-23T16:48:06.349614+01:00 germ39 kernel: [ 5733.862716] [  25520]     0 25520    53768    26129   344064        0          -997 cryptsetup 2023-02-23T16:48:06.349615+01:00 germ39 kernel: [ 5733.862717] [  25521]     0 25521    45548    17914   286720        0          -997 cryptsetup 2023-02-23T16:48:06.349616+01:00 germ39 kernel: [ 5733.862719] [  26646]     0 26646    42008    14337   253952        0          -997 cryptsetup 2023-02-23T16:48:06.349617+01:00 germ39 kernel: [ 5733.862720] [  27034]     0 27034    39031    11379   233472        0          -997 cryptsetup 2023-02-23T16:48:06.349617+01:00 germ39 kernel: [ 5733.862721] [  27451]     0 27451   180188   138685  1253376        0          -997 cryptsetup 2023-02-23T16:48:06.349618+01:00 germ39 kernel: [ 5733.862722] [  28073]     0 28073    38785    11166   229376        0          -997 cryptsetup 2023-02-23T16:48:06.349619+01:00 germ39 kernel: [ 5733.862724] [  30712]     0 30712    38785    11175   237568        0          -997 cryptsetup 2023-02-23T16:48:06.349619+01:00 germ39 kernel: [ 5733.862725] [  31927]     0 31927    25860     2780   212992      501          -997 rbd 2023-02-23T16:48:06.349620+01:00 germ39 kernel: [ 5733.862727] [  32409]     0 32409     2230      262    65536       37          -997 blkid 2023-02-23T16:48:06.349620+01:00 germ39 kernel: [ 5733.862728] oom-kill:constraint=CONSTRAINT_MEMCG,nodemask=(null),cpuset=cri-containerd-f5ec1120df909b8125ebf2a5c2172e38fa978122c021c4e82afaa77cf5595f99.scope,mems_allowed=0,oom_memcg=/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd17563cd_9da0_4d86_a89a_a4cb0f8070b3.slice/cri-containerd-f5ec1120df909b8125ebf2a5c2172e38fa978122c021c4e82afaa77cf5595f99.scope,task_memcg=/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd17563cd_9da0_4d86_a89a_a4cb0f8070b3.slice/cri-containerd-f5ec1120df909b8125ebf2a5c2172e38fa978122c021c4e82afaa77cf5595f99.scope,task=blkid,pid=32409,uid=0 2023-02-23T16:48:06.349621+01:00 germ39 kernel: [ 5733.862736] Memory cgroup out of memory: Killed process 32409 (blkid) total-vm:8920kB, anon-rss:0kB, file-rss:1048kB, shmem-rss:0kB

This OOMKill happens in the csi-rbdplugin nodeplugin pod right ?
the nodeplugin calls cryptsetup.
This issue was originally to address OOMKills in provisioner pod during volume creation. That seems to be resolved now.
I'd expect the CO in this case kublet or the one responsible for issuing nodepublish/stage calls to have some kind of
limit for simultaneous csi calls.
Similar to the one for csi-provsioner https://github.com/kubernetes-csi/external-provisioner/blob/cd81ed5d31835d1aabad74db869c1165df9e3666/cmd/csi-provisioner/csi-provisioner.go#L81

I pushed the patch I was working on and created a draft PR: https://github.com/ceph/ceph-csi/pull/3693
Feel free to take it over, use it for testing or what you want. As @Rakshith-R mentioned, I think the original issue is already resolved so it probably doesn't make sense to merge this PR, but maybe it can help as inspiration for other issues.

Running some tests having applied the PR:  #3693 on top of the v3.8.0, results show high resilience even to the OOMkills occurring in the rbdplugin:  having released any memory and cpu limitationson the pods, I could create flawlessly 100 PVCs.
+1 to have it merged!


Cannot Inject HttpClient in ServerSide Mode

Is it possible to inject the IUrlHelper and HttpClient in the ServerSide Rendering Mode of Blazor cuz everytime I try to do so I get this Error
Error: System.InvalidOperationException: Unable to resolve service for type 'System.Net.Http.HttpClient' while attempting to activate 'Test.App.Services.WeatherForecastService'.
   at Microsoft.JSInterop.DotNetDispatcher.InvokeSynchronously(String assemblyName, String methodIdentifier, Object targetInstance, String argsJson)
   at Microsoft.JSInterop.DotNetDispatcher.BeginInvoke(String callId, String assemblyName, String methodIdentifier, Int64 dotNetObjectId, String argsJson)

   at Microsoft.JSInterop.DotNetDispatcher.InvokeSynchronously(String assemblyName, String methodIdentifier, Object targetInstance, String argsJson)
   at Microsoft.JSInterop.DotNetDispatcher.BeginInvoke(String callId, String assemblyName, String methodIdentifier, Int64 dotNetObjectId, String argsJson)
    at endInvokeDotNetFromJS (http://localhost:5000/_framework/blazor.server.js:23:48556)
    at http://localhost:5000/_framework/blazor.server.js:23:48125
    at new Promise (<anonymous>)
    at e.beginInvokeJSFromDotNet (http://localhost:5000/_framework/blazor.server.js:23:48094)
    at http://localhost:5000/_framework/blazor.server.js:16:11652
    at Array.forEach (<anonymous>)
    at e.invokeClientMethod (http://localhost:5000/_framework/blazor.server.js:16:11623)
    at e.processIncomingData (http://localhost:5000/_framework/blazor.server.js:16:10229)
    at e.connection.onreceive (http://localhost:5000/_framework/blazor.server.js:16:7298)
    at WebSocket.i.onmessage (http://localhost:5000/_framework/blazor.server.js:16:20346)

if it's not possible could you provide a method as to how we cn make http calls and get an Insatance of the UrlHelper when using server side rendering

@alfkonee With server-side Blazor you can just create a new HttpClient and call whatever HTTP endpoint you'd like, just like you would from an MVC or Razor Pages app. If you want to manage HttpClient instances using DI, you can use the new IHttpClientFactory support in ASP.NET Core.
You don't typically need the IUriHelper on the server for setting HTTP calls because you are already running on the server, so making HTTP calls back to yourself is typically unnecessary.
I hope this helps!


Create maven.yml

Changes:

Add maven.yml file to run CI workflow on changes incoming to main branch.
The workflow automatically builds the project and runs tests using Maven.

This workflow should only be failing now because there is no pom.xml file in the repository, or other maven project files. Once PR #3 is merged, this should be resolved.


Auto-export stops working, "Error: translation failed"

Support log ID: D3UP2CF5-euc
Exporter used: Better BibLaTex and Better CSL JSON
BBT 5.2.48
Zotero <IP_ADDRESS>
Over the last couple of days, I have recurrently observed that BBT stops auto-exporting my library to both Better CSL JSON or Better BibLaTex a while after setting up the auto-export, giving “Error: Translation failed” in the Preferences BBT panel.
I do not get any error messages: The progress bar for BBT export shows normally (somewhat slow), but eventually fills up and disappears (“Status: done” in Preferences). However, the export bib- or json-file remains unchanged. (I noticed this because new entries added in Zotero were not available to the Citations plugin I use in Obsidian, which relies on the BBT export.)
When I remove and re-install BBT and set up the auto-export anew, it will again work for a while but then stop working as described above. In the meantime, I will have added new entries to Zotero and manually changed citekeys. Some of these are exported as expected, I cannot pin down when export begins to fail. I personally cannot relate export failure to any specific use case.
Any help or pointers would be greatly appreciated – as is your committed work on this great plugin!
PS: Unsure about #7 and #8 of the support request requirements – my export problem does not relate to a specific reference and I do not have a problem with wrong export outputs.

Ah, I see. Just tried again several times - now I do not get an error message, yet the log is not being saved. Upon clicking "Save copy of log", I am prompted to choose file name and location, but after confirming this, nothing happens, no file being saved.

1421 doesn't fix the auto-export error (I still don't know what causes it) but it should make local-log-saving work. It's always worked for me, but this simplifies the error submission, so there's a better chance it works for you.
When you say "the log exceeded memory", how did you know this was the cause? Did you get an error message to that effect?

Thanks! Will try it with that build next time around.
Concerning the error: yes, I got an error message (system pop-up) to that effect. I don't remember the exact wording, though.

1423 will clip the log in an attempt to prevent the out of memory problem for long logs.

Good precaution :) I can confirm that with 1421 I was able to locally save a log as expected - just waiting for the export error to recur in typical fashion now, will upload that log then.

So the "translation failed" error just recurred, this time without disappearing upon manual triggering. I just tried to locally save the log (which worked on 1421 before), but no file is being saved after I confirmed file name and location. Notably, all four tabs in the log submission dialogue are empty:

However, Zotero debug logging has been activated and indicates "501289 lines logged".
Any idea what could be the issue here? Should I just try to submit a log online anyway?

Yes please.

Good point :D Maybe I should revert to 1421 in case it's in issue with the clipping mechanism?

Meanwhile I had to restart Zotero, now it worked - here is a log after two cycles of manually triggered export on "translation failed", after which the error disappeared. Had to unpack it since .tar attachments are apparently not accepted here.
debug.txt

You can install https://github.com/retorquere/zotero-debug-log alongside BBT -- it is an independent log submitter which will work if BBTs doesn't.

This just gets me a bit more logging. I still don't see errors, but there's a warning while loading the CSL cache, so I want to see if that's related to the problem somehow.

Ok, will install 1435 and get back with a log when recurring!

Right now, the BBT log submission doesn't work again (same as here). I have installed your independent log submitter add-on, but, uh ... where do I find it?

In the Help menu.

I see. As it happens, this time around it worked via BBT log submission - file attached.
debug.txt

If it worked via log submission, didn't you get a log ID?

Oh yes, sorry, that's 7VKX2FLZ-euc. I unpacked the txt-file to allow upload in the post.

No problem, just curious.

🤖 this is your friendly neighborhood build bot announcing test build <IP_ADDRESS>7 ("not all items have creators")

Should I try that build?

Yes please. The previous build introduced an error of its own.

Just in case that helps: here would be a log after the error under build 1437 (ID: 75LM8GGD-euc).
debug.txt

Just in case that helps: here would be a log after the error under build 1437 (ID: 75LM8GGD-euc).
debug.txt

That has the out-of-memory error. How large is the export you are creating, approximately?

1454 should be better but may actually solve the problem. I haven't been able to reproduce locally yet.

How large is the export you are creating, approximately? And how many items does it concern?
 That means a total of 12,117 items (housekeeping due...), resulting in a ~10MB export file.

I will try 1462 and see how it goes. Thank you very much so far!

 That means a total of 12,117 items (housekeeping due...), resulting in a ~10MB export file.

I have one test in my test suite that is a little over 24k items which exports to a 11M CSL-JSON file. That shouldn't be the problem.

I'm sorry to say that the error actually still does occur. Again, no reason to hurry from my side of things, but for when you have time to have another look here's the log. (Very attentive to switch to zip, BTW.)
Also, might testing export on a smaller collection be a good idea?

I'm afraid the smaller export will just not exhibit the problem. Unfortunately, I see no sign of an error in E7BRSTHJ-euc. But I was logging an inordinate amount of stuff from an older build, and because of that, the part of the log that had the error might have been clipped off. A new build is underway.

Thanks for your efforts!
Just to add another variation to the cabinet of curiosities: just now I've encountered the error in 1487 - BBT log submission showed empty panels again, so I tried the standalone "Send to file.io" option and repeatedly got this curious error (non-)message:

Anyway, will try to get a proper error log from 1487 in due time and upload that then.

Not very informative, no. I've pushed 2.0.8 of the file.io extension whuch attempts to show more, but that the message is empty is unexpected.

Thanks for your efforts!

And thank you for sticking this out.
I can reintroduce a feature where the log is always saved locally to a configurable dir. That might help.

Updated the file.io extension to 2.0.8. This time around, the BBT log submitter unfortunately was empty again, but the error message from the separate extension was now more informative:

I will try to obtain another log sooner after a restart of Zotero. Currently, I usually just have Zotero running on the side all along, which naturally leads to a lot of logging when enabled from the start.

Encountered "out of memory" again. Just to give it a try, I saved the log from Help > Debug Output Logging > View Output > File > Save, in case anything useful might be hidden in that jungle of information:
Debug Output.zip

If you hit the OOM, can you still go into the preferences and clear the cache? If so, can you see if that solves it (even if only temporarily)?
A build will drop here soon that will have a hidden BBT preference named caching. If you set that to false and restart, BBT will do no caching. Maybe the in-memory cache is the problem for you.
I don't think we'll need more logs for now. The problems seems to consistently be an OOM error, so we'll just have to find a way to consume less memory, I suppose. How much memory does your system have? Can you see if any of these tools allow you to see whether Zotero is steadily claiming more memory? If so, how about without BBT?

I have installed 1490 and deactivated caching, will see how that goes.

If you hit the OOM, can you still go into the preferences and clear the cache?

I feel a bit blind ... whereabout would I do that?

How much memory does your system have?

16 GB, of which currently 6.4 GB are in use, running Zotero, Firefox (978 MB), and Obsidian (455 MB). Google drive sync as a background process claims another 219 MB.

whether Zotero is steadily claiming more memory? If so, how about without BBT?

Memory use right now is compares as follows:
Zotero with BBT completely disabled: 484.7 MB
Zotero with BBT 1490 and caching disabled: 557.6 MB
With BBT enabled, I indeed just observed memory used by Zotero climbing up quickly while the BBT export status bar was active, peaking at ~ 2,200 MB.
Given that there's still quite some RAM usage not readily allocated to on-going processes, I'm also running a full antivirus scan right now, just in case.

I have installed 1490 and deactivated caching, will see how that goes.

If you hit the OOM, can you still go into the preferences and clear the cache?

I feel a bit blind ... whereabout would I do that?

Export - Misc - Reset cache

How much memory does your system have?

16 GB, of which currently 6.4 GB are in use, running Zotero, Firefox (978 MB), and Obsidian (455 MB). Google drive sync as a background process claims another 219 MB.

16 should be plenty. It's what I have, and my Linux system has 8.

Memory use right now is compares as follows:
Zotero with BBT completely disabled: 484.7 MB
Zotero with BBT 1490 and caching disabled: 557.6 MB
With BBT enabled, I indeed just observed memory used by Zotero climbing up quickly while the BBT export status bar was active, peaking at ~ 2,200 MB.

For an export that ends up being 11MB, the caches collectively use 1.5G? That is way too much. I would expect the caches in your case to take ~15MB + ~15MB per output combination (unique set of export options you picked and preferences you have set for that export). If you set up auto-exports and not change preferences, and not change any of the auto-export settings, that should be some 30MB total.
When the system is at 2GB+, can you close Zotero, and look at how big the cache files are? They live in the directory where your Zotero database lives, in subdirectory better-bibtex. They are only saved when Zotero is shut down cleanly.

Given that there's still quite some RAM usage not readily allocated to on-going processes, I'm also running a full antivirus scan right now, just in case.

I wouldn't expect that to be the scenario. If the memory is consumed by Zotero, and specifically when BBT is active, it seems pretty clear who the culprit is.

For an export that ends up being 11MB, the caches collectively use 1.5G?

Not quite sure if I'm following. The figures I gave (484 MB; 557 MB; 2,200 MB) were for alternative Zotero set-ups, nothing adding up. Obviously, though, memory usage for Zotero with BBT and BBT caching enabled is abnormal.

When the system is at 2GB+, can you close Zotero, and look at how big the cache files are?

This time, memory usage by Zotero peaked at just short of 3 GB. The directory you specified contains:

Those sizes should be reflective of the actual memory use, as they're just dumps of the in-memory structures. This is a real problem then.
In 1491, the error report dialog will show stats on your cache. Are those in line with what you see on disk here (with caching enabled of course)?

I enabled caching again to have a look - this is what the BBT log submission dialogue says:

These are the contents of the better-bibtex directory at the same time:

These are the contents of the cache directory after closing Zotero:

All of these sizes are in line with expectations. They don't add up to the discrepancy you are seeing.

1492 adds a memory breakdown, in the off chance that it shows something useful. After the window pops up, you'll have to cancel the error report, because the memory window remains inactive otherwise.

BBT has a memory leak. You're noticing it because Zotero on windows is capped to use ~3G, but memory use grows steadily when BBT is active on other platforms too.
The good news: I can measure this now, and my test suite offers a somewhat stable measurement of it. That means if I can make it go away here, it will likely be fixed for you too, and in the meantime you can change your setup to not be bothered by it (too often). Disabling the cache or workers does not seem to have real impact, so you can keep those on.
The bad news: memory leaks are hard to trace down, so this will take time to address.

Sorry for not getting back to you earlier - the last couple of days were quite busy and I lost sight of this, unfortunately.
Great to hear you could identify / reproduce the problem now - and sorry to hear it turns out to be so bothersome... I very much appreciate your work on this, obviously please take your time.
Is there any more information from my side that would help you at the moment?

No, I'm going up have to figure this out by myself.

Alright, good luck then! 👍

My juris-M stop working when trying to export library into Better CSL JSON/Better CSL YAML.
Export to bibtex is just fine.
And I found that in debug archive, so I guess maybe there is some Item type is not defined in better-bibtex, which cuased the error:
(3)(+0000001): {better-bibtex :error:} +1 worker translation failed, error: Error: Unexpected Zotero Item type "standard"

I tried to export without such items, then it works.
Related bug report id: FEUZYISV-apse

@chenyhsu you have a different problem, can you open a separate issue?

@chenyhsu you have a different problem, can you open a separate issue?

Opened issue #2212 . Thank you for your quick feedback.


Error including bytemd dependency

Describe the bug
Someone  on Discord reported that they can't get bytemd to work. It looks to me like vite-plugin-svelte is not detecting that bytemd's lodash.debounce dependency needs to be prebundled. There are also issues reported in the bytemd repo and on Discord where the error occurs on other dependencies
A couple things I noticed that are off about the packaging:

bytemd uses .esm.js and .cjs.js extensions instead of .mjs and .cjs
lodash.debounce has no main or any other entry point specified in the package.json

Even when I manually edit those I still get the same error.
Reproduction
https://github.com/benmccann/sveltekit-bytemd-bug-reproduction
Logs
8:07:24 PM [vite] Error when evaluating SSR module /node_modules/lodash.debounce/index.js?v=dc88d06f:
ReferenceError: module is not defined
    at /node_modules/lodash.debounce/index.js?v=dc88d06f:377:1

System Info
reproduction provided

Severity
The top two bugs in the bytemd repo are about how the project doesn't work with SvelteKit: https://github.com/bytedance/bytemd/issues/96 and https://github.com/bytedance/bytemd/issues/139

My bad. I assumed this was a pre-bundling bug and only fixed that, looks like the error is coming from loading the module in SSR 🤔

TL;DR: ssr.noExternal on a dep considers the dep to be source code (Everything is ESM including its transitive deps, like pre-bundling)

Wow this seems like a huge deal than I expected. Through some testing, I realized the issue is that ssrLoadModule is called on lodash.debounce, but that doesn't work because ssrLoadModule works for ESM code only. Ideally Vite expects this to never happen in the first place.
For our case, it happens because ssrLoadModule is called by ssrImport, which happens when you do import debounce from "lodash.debounce" in your code. And in this case, bytemd's is doing so.
However, this isn't an issue in most cases, as long as lodash.debounce is externalized, which Vite does so by default. When it is externalized, Vite's import analysis will keep the import as is, so you get this in Vite's complex SSR transformation:
const debounce = __vite_ssr_import__("lodash.debounce")

If not externalized, you get:
const debounce = __vite_ssr_import__("/node_modules/lodash.debounce/index.js")
// notice this is the path of the error message of the issue

The different path is important as it gets resolved differently by ssrImport.
In Svelte's case, this is not externalized because bytemd is Svelte library, and it's noExternal by default, which means any deps below it is not externalized, including lodash.debounce.
A solution is to externalize lodash.debounce explicitly, but bytemd has a lot more CJS-only deps that needs to be externalized. vite-plugin-svelte could use the same deps reinclusion logic from prebundling for here as well to automate this, but I think for dev only. And I'm not sure how safe it'll work in pnpm.
Or maybe adjustments can be made in Vite to externalized transitive deps by default.

replacing the unmaintained lodash.debounce with https://github.com/angus-c/just#just-debounce-it might be a quickfix for bytemd

There are a lot more deps there that has issues, like lodash.throttle, unified, and the remark plugins IIRC.

For lodash.throttle there is just-throttle too. but remark plugins don't always have an alternative that works with esm too i guess.
Looks like bytemd is actively working on their package structure, adding an exports field (see history from today) https://github.com/bytedance/bytemd/commits/main/packages/bytemd/package.json

I don't understand why we're only hitting this with this library. I thought there are lots of Svelte libraries with CJS deps that work.

I put a console.log statement at the end of resolveSSRExternal and found:
externals: ["@sveltejs/adapter-auto","@sveltejs/kit","svelte","@bytemd/plugin-gfm","bytemd > codemirror-ssr","bytemd > hast-util-sanitize","bytemd > lodash.debounce","bytemd > lodash.throttle","bytemd > rehype-raw","bytemd > rehype-sanitize","bytemd > rehype-stringify","bytemd > remark-parse","bytemd > remark-rehype","bytemd > unified","bytemd > unist-util-visit","bytemd > vfile","bytemd > word-count"]

But @bluwy said it wasn't being externalized, so I think perhaps something is going wrong in the Vite code in between where I was looking and where he was looking

I don't understand why we're only hitting this with this library. I thought there are lots of Svelte libraries with CJS deps that work.

Me too. This looks like something we would have spotted long ago, maybe I'm missing something that's only an issue with bytemd specifically, but it looks to me that it's a general issue 🤔

I put a console.log statement at the end of resolveSSRExternal and found:

I tried the same and got [   '@sveltejs/adapter-auto',   '@sveltejs/kit',   'svelte',   '@bytemd/plugin-gfm' ] though. And I don't think the > syntax works with SSR externals. It would be a bug if there's a way to reproduce that.
I'll try to reproduce this in vite-plugin-svelte again tonight and see if it's a general issue not specific to bytemd

I tried the same and got [ '@sveltejs/adapter-auto', '@sveltejs/kit', 'svelte', '@bytemd/plugin-gfm' ] though. And I don't think the > syntax works with SSR externals. It would be a bug if there's a way to reproduce that.

Oh, I think that's because I accidentally had checked in experimental.prebundleSvelteLibraries: true. I just pushed a change to remove that (https://github.com/benmccann/sveltekit-bytemd-bug-reproduction/commit/9d661d77094d1eb386b18c82fe7da09cf83c2e14). Hopefully you'll see the same as I do now

Ah yeah with experimental.prebundleSvelteLibraries: false I can see the same list now.

I don't think the > syntax works with SSR externals. It would be a bug if there's a way to reproduce that.

This seems to be the issue. It works (well the app is broken for other reasons, but reasons that make more sense) if I strip out the prefix:
    externals = externals.map(s => {
      const sep = ' > ';
      const i = s.lastIndexOf(sep);
      if (i < 0) {
        return s;
      }
      return s.substring(i + sep.length);
    });

Any idea where the code that inserts this might live or what a good solution to it might be?

I haven't got time to check this deeper today 😅 I think it's Vite that re-using the strings from optimizeDeps.include to ssr.external, hence the > there, but I can't seem to find where. If we can find it then I think it makes sense for Vite to support > in ssr.external too. Otherwise we can make the change in vite-plugin-svelte too by returning the correct externals here (may need some code to figure out the array, can be referenced here)
Re the code block in the comment, the syntax allows no spaces beside the > too, so ideally we substring from > onwards and trim it, which should be enough.

I think it's Vite that re-using the strings from optimizeDeps.include to ssr.external, hence the > there, but I can't seem to find where.

Yes, I think that's right. It looks like it's being passed as input to resolveSSRExternal. For example: https://github.com/vitejs/vite/blob/c703a3348adeaad9dc92d805a381866917f2a03b/packages/vite/src/node/server/index.ts#L371

I think it makes sense for Vite to support > in ssr.external too

Yeah, I agree that sounds like a good fix. Do you want to take a stab at that since you implemented it for optimizeDeps? I'm not sure I'd know how. The other option would be to strip everything before the > as I did above. If we want to go with that option I could clean it up and send a PR

Ah thanks for finding the source. I think a simple fix like the code you shown, and have it applied in the linked code would be enough. I’ve worked a bit on the fix in Vite today but having some hurdles making the tests. Feel free to make the patch too if you’d like, I haven’t put a lot of time into the fix yet.

Ok, I sent a PR here: https://github.com/vitejs/vite/pull/7154

This should be fixed in vite-plugin-svelte 1.0.0-next.40 and Vite 2.9.0-beta.2