Tensorflow lite build causing multiple undefined reference errors after running make

Issue type
Build/Install
Have you reproduced the bug with TensorFlow Nightly?
No
Source
source
TensorFlow version
tf 2.16.2
Custom code
Yes
OS platform and distribution
Linux Ubuntu 22.04
Mobile device
No response
Python version
3.12.4
Bazel version
6.5.0
GCC/compiler version
11.4.0
CUDA/cuDNN version
No response
GPU model and memory
No response
Current behavior?
Upon running make in the build directory after running cmake I get multiple undefined reference errors which I cannot understand why they occur. I am using all standard code from the minimal example except my CMakeLists.txt which is
cmake_minimum_required(VERSION 3.17)
project(TFLiteCheck)

set(CMAKE_CXX_STANDARD 14)

# include has 2 subdirectories: tensorflow and flatbuffers
INCLUDE_DIRECTORIES(${CMAKE_CURRENT_SOURCE_DIR}/../tflite-dist/include/)

# lib has 1 file: libtensorflowlite.so
ADD_LIBRARY(tensorflowlite SHARED IMPORTED)
set_property(TARGET tensorflowlite PROPERTY IMPORTED_LOCATION ${CMAKE_CURRENT_SOURCE_DIR}/../tflite-dist/libs/linux_x64/libtensorflowlite.so)

add_executable(TFLiteCheck main.cpp)
target_link_libraries(TFLiteCheck tensorflowlite)

cmake runs perfectly fine and the tensorflow.so file was also created within the bazel-bin directory without any issue. I have spent days trying to solve it so it is sad to see it doesn't work even after coming so close.
My main.cpp is also the exact same as the minimal example
Standalone code to reproduce the issue
#include <cstdio>

#include "tensorflow/lite/core/interpreter_builder.h"
#include "tensorflow/lite/interpreter.h"
#include "tensorflow/lite/kernels/register.h"
#include "tensorflow/lite/model_builder.h"
#include "tensorflow/lite/optional_debug_tools.h"

// This is an example that is minimal to read a model
// from disk and perform inference. There is no data being loaded
// that is up to you to add as a user.
//
// NOTE: Do not add any dependencies to this that cannot be built with
// the minimal makefile. This example must remain trivial to build with
// the minimal build tool.
//
// Usage: minimal <tflite model>

#define TFLITE_MINIMAL_CHECK(x)                              \
  if (!(x)) {                                                \
    fprintf(stderr, "Error at %s:%d\n", __FILE__, __LINE__); \
    exit(1);                                                 \
  }

int main(int argc, char* argv[]) {
  if (argc != 2) {
    fprintf(stderr, "minimal <tflite model>\n");
    return 1;
  }
  const char* filename = argv[1];

  // Load model
  std::unique_ptr<tflite::FlatBufferModel> model =
      tflite::FlatBufferModel::BuildFromFile(filename);
  TFLITE_MINIMAL_CHECK(model != nullptr);

  // Build the interpreter with the InterpreterBuilder.
  // Note: all Interpreters should be built with the InterpreterBuilder,
  // which allocates memory for the Interpreter and does various set up
  // tasks so that the Interpreter can read the provided model.
  tflite::ops::builtin::BuiltinOpResolver resolver;
  tflite::InterpreterBuilder builder(*model, resolver);
  std::unique_ptr<tflite::Interpreter> interpreter;
  builder(&interpreter);
  TFLITE_MINIMAL_CHECK(interpreter != nullptr);

  // Allocate tensor buffers.
  TFLITE_MINIMAL_CHECK(interpreter->AllocateTensors() == kTfLiteOk);
  printf("=== Pre-invoke Interpreter State ===\n");
  tflite::PrintInterpreterState(interpreter.get());

  // Fill input buffers
  // TODO(user): Insert code to fill input tensors.
  // Note: The buffer of the input tensor with index `i` of type T can
  // be accessed with `T* input = interpreter->typed_input_tensor<T>(i);`

  // Run inference
  TFLITE_MINIMAL_CHECK(interpreter->Invoke() == kTfLiteOk);
  printf("\n\n=== Post-invoke Interpreter State ===\n");
  tflite::PrintInterpreterState(interpreter.get());

  // Read output buffers
  // TODO(user): Insert getting data out code.
  // Note: The buffer of the output tensor with index `i` of type T can
  // be accessed with `T* output = interpreter->typed_output_tensor<T>(i);`

  return 0;
}

Relevant log output
(base) robocon2@robocon2:~/tflite_source_final/tflite/04_Linux_Installation/build$ make
[ 50%] Building CXX object CMakeFiles/TFLiteCheck.dir/main.cpp.o
[100%] Linking CXX executable TFLiteCheck
/usr/bin/ld: CMakeFiles/TFLiteCheck.dir/main.cpp.o: in function `main':
main.cpp:(.text+0x91): undefined reference to `tflite::FlatBufferModel::BuildFromFile(char const*, tflite::ErrorReporter*)'
/usr/bin/ld: main.cpp:(.text+0x117): undefined reference to `tflite::InterpreterBuilder::InterpreterBuilder(tflite::FlatBufferModel const&, tflite::OpResolver const&)'
/usr/bin/ld: main.cpp:(.text+0x13b): undefined reference to `tflite::InterpreterBuilder::operator()(std::unique_ptr<tflite::Interpreter, std::default_delete<tflite::Interpreter> >*)'
/usr/bin/ld: main.cpp:(.text+0x19e): undefined reference to `tflite::Interpreter::AllocateTensors()'
/usr/bin/ld: main.cpp:(.text+0x1fe): undefined reference to `tflite::PrintInterpreterState(tflite::Interpreter*)'
/usr/bin/ld: main.cpp:(.text+0x215): undefined reference to `tflite::Interpreter::Invoke()'
/usr/bin/ld: main.cpp:(.text+0x275): undefined reference to `tflite::PrintInterpreterState(tflite::Interpreter*)'
/usr/bin/ld: main.cpp:(.text+0x298): undefined reference to `tflite::InterpreterBuilder::~InterpreterBuilder()'
/usr/bin/ld: main.cpp:(.text+0x2ee): undefined reference to `tflite::InterpreterBuilder::~InterpreterBuilder()'
/usr/bin/ld: CMakeFiles/TFLiteCheck.dir/main.cpp.o: in function `std::default_delete<tflite::FlatBufferModel>::operator()(tflite::FlatBufferModel*) const':
main.cpp:(.text._ZNKSt14default_deleteIN6tflite15FlatBufferModelEEclEPS1_[_ZNKSt14default_deleteIN6tflite15FlatBufferModelEEclEPS1_]+0x22): undefined reference to `tflite::FlatBufferModel::~FlatBufferModel()'
/usr/bin/ld: CMakeFiles/TFLiteCheck.dir/main.cpp.o: in function `std::default_delete<tflite::Interpreter>::operator()(tflite::Interpreter*) const':
main.cpp:(.text._ZNKSt14default_deleteIN6tflite11InterpreterEEclEPS1_[_ZNKSt14default_deleteIN6tflite11InterpreterEEclEPS1_]+0x22): undefined reference to `tflite::Interpreter::~Interpreter()'
collect2: error: ld returned 1 exit status
make[2]: *** [CMakeFiles/TFLiteCheck.dir/build.make:98: TFLiteCheck] Error 1
make[1]: *** [CMakeFiles/Makefile2:83: CMakeFiles/TFLiteCheck.dir/all] Error 2
make: *** [Makefile:91: all] Error 2

I am facing exact same issue
OS : Ubuntu 22.04
built the libtensorflowlite.so from source with tf (v2.12.0) bazel (5.3.0) [tested configuration https://www.tensorflow.org/install/source#linux]
and then I followed the following project structure
├── CMakeLists.txt
├── include
│   ├── flatbuffers
│   └── tensorflow
├── libs
│   └── libtensorflowlite.so
├── models
│   └── face_detection_front.tflite
└── src
    └── main.cc

CMakeLists.txt
cmake_minimum_required(VERSION 3.10)
set(CMAKE_CXX_STANDARD 17)
project(tflite-project)

# Add the include directory
include_directories(${CMAKE_CURRENT_SOURCE_DIR}/include)

add_library(tflite SHARED IMPORTED)
set_target_properties(tflite PROPERTIES
  IMPORTED_LOCATION "${CMAKE_CURRENT_SOURCE_DIR}/libs/libtensorflowlite.so")
  #INTERFACE_INCLUDE_DIRECTORIES "${CMAKE_CURRENT_SOURCE_DIR}/include")

# Add your executable
add_executable(project src/main.cc)

# Link against TensorFlow Lite
target_link_libraries(project tflite)

And I am facing exact same errors as @AD-lite24 ...

Hi @ashish-2005 my issue was resolved, use the official CMakeLists.txt provided
set(TENSORFLOW_SOURCE_DIR "" CACHE PATH
  "Directory that contains the TensorFlow project"
)
if(NOT TENSORFLOW_SOURCE_DIR)
  get_filename_component(TENSORFLOW_SOURCE_DIR
    "${CMAKE_CURRENT_LIST_DIR}/../tensorflow-2.16.2/"
    ABSOLUTE
  )
endif()

add_subdirectory(
  "${TENSORFLOW_SOURCE_DIR}/tensorflow/lite"
  "${CMAKE_CURRENT_BINARY_DIR}/tensorflow-lite"
  EXCLUDE_FROM_ALL
)

Add all the other parts as required

Hi @AD-lite24, Thanks for the heads up. :rocket:
I also followed, Build with CMake which make a STATIC library within the /build folder and uses it, and now its working :smile: .
Turns out its hard to build the library from source once and use it in every project

source
Note: This generates a static library libtensorflow-lite.a in the current directory but the library isn't self-contained since all the transitive dependencies are not included. To use the library properly, you need to create a CMake project. Please refer the "Create a CMake project which uses TensorFlow Lite" section.

I think for every new project we have to make a CMake project with sub-directories of source library. And build/make that freshly.

You can just run sudo make install and install it your system libraries

You can just run sudo make install and install it your system libraries

Yup i read about that while searching for a solution here to build and install directly in the system...Are you aware of the pros and cons of this approach ? :thinking:

I am not very familiar with the dependency isolation systems in c++ tbh so I am not sure. But it shouldn't be a problem as such, you can always adjust your cmake if you wanted to install a different version of tensorflow for a certain problem


new command group: az marketplace terms

As suggested by @jiasli, it would be great to have a new command group az marketplace terms or just az terms.  #10896 added az vm image terms.
new commands

az marketplace terms list
az marketplace terms show --publisher {publisher} --offer {offer} --plan {plan}
az marketplace terms accept --publisher {publisher} --offer {offer} --plan {plan}
az marketplace terms cancel --publisher {publisher} --offer {offer} --plan {plan}

An alternative, would be to name it sign instead of  accept, but accept was used for az vm image terms as well as PowerShell Set-AzureRmMarketplaceTerms -Accept and Set-AzMarketplaceTerms -Accept.
The Marketplace Agreements REST API exposes two resource types.
providers/Microsoft.MarketplaceOrdering/offerTypes/virtualmachine/publishers/
The az vm image terms use these.
Create
PUT https://management.azure.com/subscriptions/{subscriptionId}/providers/Microsoft.MarketplaceOrdering/offerTypes/virtualmachine/publishers/{publisherId}/offers/{offerId}/plans/{planId}/agreements/current?api-version=2015-06-01
Get
GET https://management.azure.com/subscriptions/{subscriptionId}/providers/Microsoft.MarketplaceOrdering/offerTypes/virtualmachine/publishers/{publisherId}/offers/{offerId}/plans/{planId}/agreements/current?api-version=2015-06-01
providers/Microsoft.MarketplaceOrdering/agreements/
The az marketplace terms would use all of these.
 For example:
az rest --method get --uri /subscriptions/${subscriptionId}/providers/Microsoft.MarketplaceOrdering/agreements/arista-networks/offers/veos-router/plans/eos-4_22_1fx?api-version=2015-06-01

Gets the active agreements for:

thanks @ctaggart , will discuss detail with you offline.

@ctaggart Great proposal. We need study the impact when update command name.

@qwordy, I do not recommend changing the existing command group. I think you should leave az vm image terms as is. az marketplace terms would be a new command  group.

OK. Got it. They are different APIs.

Unfortunately, PowerShell is using Get-AzMarketplaceTerms with the Microsoft.MarketplaceOrdering/offerTypes/virtualmachine/publishers endpoint and not the Microsoft.MarketplaceOrdering/agreements endpoint. May be agreements should be used instead of terms.  I met  the team that manages this endpoint today. We should run the design by them. I'm curious if some agreements could be handled at the tenant level in the future.

Hi Guys
Any update on this? This is a highly anticipated cli command.
The alternative az cli to sign a agreement failed.
You must accept the legal terms through the Azure portal


Build following file lock wait results in malformed task run

Migrating from https://github.com/rust-lang/rust/issues/35658.
Steps to reproduce

Run emacs? (It makes and deletes autosave files in the same directories as working file buffers, which may help trigger it.)

flycheck-rust may be a contributor.

Iterate over a file and cargo test immediately after saving. Success if you see "waiting for file lock" in build output.
Observe that build process can't actually get the crate referenced properly (as below).

Workaround

cargo test again without touching any files.
Observe usual passing tests.

Error log
An example from a project I'm working on:
$ cargo test
    Blocking waiting for file lock on build directory
   Compiling wd40 v0.1.0 (file:///Users/talex/git/starfighter/wd40)
    Finished debug [unoptimized + debuginfo] target(s) in 14.0 secs
     Running target/debug/deps/wd40-f757f6198b80f6ed

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured

   Doc-tests wd40

running 11 tests
test account::orders_0 ... FAILED
test game::start_level_0 ... FAILED
test account::place_order_0 ... FAILED
test account::stock_orders_0 ... FAILED
test heartbeat_0 ... FAILED
test stock::order_cancel_0 ... FAILED
test stock::last_quote_0 ... FAILED
test stock::order_status_0 ... FAILED
test venue::heartbeat_0 ... FAILED
test venue::stocks_0 ... FAILED
test stock::orderbook_0 ... FAILED

failures:

---- account::orders_0 stdout ----
	error: extern location for wd40 does not exist: /Users/talex/git/starfighter/wd40/target/debug/deps/libwd40.rlib
<anon>:1:1: 1:19 error: can't find crate for `wd40` [E0463]
<anon>:1 extern crate wd40;
         ^~~~~~~~~~~~~~~~~~
error: aborting due to 2 previous errors
thread 'account::orders_0' panicked at 'Box<Any>', ../src/librustc_errors/lib.rs:674
note: Run with `RUST_BACKTRACE=1` for a backtrace.
thread 'account::orders_0' panicked at 'couldn't compile the test', ../src/librustdoc/test.rs:287

---- game::start_level_0 stdout ----
	error: extern location for wd40 does not exist: /Users/talex/git/starfighter/wd40/target/debug/deps/libwd40.rlib
<anon>:1:1: 1:19 error: can't find crate for `wd40` [E0463]
<anon>:1 extern crate wd40;
         ^~~~~~~~~~~~~~~~~~
error: aborting due to 2 previous errors
thread 'game::start_level_0' panicked at 'Box<Any>', ../src/librustc_errors/lib.rs:674
thread 'game::start_level_0' panicked at 'couldn't compile the test', ../src/librustdoc/test.rs:287

TRUNCATED

Meta

Rust: rustc 1.12.0-nightly (28ce3e8a5 2016-08-01)
OS: Latest OSX

I believe the underlying issue here is that if you run cargo test twice it may try to compile things both times, and we don't hold the lock while running tests. This in turn means that one process may stomp over the results needed by the other while tests are running.
We can probably solve this by just holding the lock on the target directories until the Compilation structure itself returned is dropped.


mblaze: 0.3.2 -> 0.4

Semi-automatic update generated by https://github.com/ryantm/nixpkgs-update tools. This update was made based on information from https://repology.org/metapackage/mblaze/versions.
meta.description for mblaze is: '"Unix utilities to deal with Maildir"'.
Compare changes on GitHub

Checks done (click to expand)

built on NixOS
Warning: no invocation of /nix/store/55gvk6xsfymscn70admmw5wdyn1lc6fg-mblaze-0.4/bin/mnext had a zero exit code or showed the expected version
Warning: no invocation of /nix/store/55gvk6xsfymscn70admmw5wdyn1lc6fg-mblaze-0.4/bin/mprev had a zero exit code or showed the expected version
Warning: no invocation of /nix/store/55gvk6xsfymscn70admmw5wdyn1lc6fg-mblaze-0.4/bin/mbnc had a zero exit code or showed the expected version
Warning: no invocation of /nix/store/55gvk6xsfymscn70admmw5wdyn1lc6fg-mblaze-0.4/bin/mfwd had a zero exit code or showed the expected version
Warning: no invocation of /nix/store/55gvk6xsfymscn70admmw5wdyn1lc6fg-mblaze-0.4/bin/mrep had a zero exit code or showed the expected version
/nix/store/55gvk6xsfymscn70admmw5wdyn1lc6fg-mblaze-0.4/bin/mrefile passed the binary check.
/nix/store/55gvk6xsfymscn70admmw5wdyn1lc6fg-mblaze-0.4/bin/maddr passed the binary check.
Warning: no invocation of /nix/store/55gvk6xsfymscn70admmw5wdyn1lc6fg-mblaze-0.4/bin/magrep had a zero exit code or showed the expected version
/nix/store/55gvk6xsfymscn70admmw5wdyn1lc6fg-mblaze-0.4/bin/mdate passed the binary check.
/nix/store/55gvk6xsfymscn70admmw5wdyn1lc6fg-mblaze-0.4/bin/mdeliver passed the binary check.
/nix/store/55gvk6xsfymscn70admmw5wdyn1lc6fg-mblaze-0.4/bin/mdirs passed the binary check.
/nix/store/55gvk6xsfymscn70admmw5wdyn1lc6fg-mblaze-0.4/bin/mexport passed the binary check.
/nix/store/55gvk6xsfymscn70admmw5wdyn1lc6fg-mblaze-0.4/bin/mflag passed the binary check.
/nix/store/55gvk6xsfymscn70admmw5wdyn1lc6fg-mblaze-0.4/bin/mflow passed the binary check.
Warning: no invocation of /nix/store/55gvk6xsfymscn70admmw5wdyn1lc6fg-mblaze-0.4/bin/mgenmid had a zero exit code or showed the expected version
Warning: no invocation of /nix/store/55gvk6xsfymscn70admmw5wdyn1lc6fg-mblaze-0.4/bin/mhdr had a zero exit code or showed the expected version
Warning: no invocation of /nix/store/55gvk6xsfymscn70admmw5wdyn1lc6fg-mblaze-0.4/bin/minc had a zero exit code or showed the expected version
/nix/store/55gvk6xsfymscn70admmw5wdyn1lc6fg-mblaze-0.4/bin/mlist passed the binary check.
Warning: no invocation of /nix/store/55gvk6xsfymscn70admmw5wdyn1lc6fg-mblaze-0.4/bin/mmime had a zero exit code or showed the expected version
/nix/store/55gvk6xsfymscn70admmw5wdyn1lc6fg-mblaze-0.4/bin/mpick passed the binary check.
/nix/store/55gvk6xsfymscn70admmw5wdyn1lc6fg-mblaze-0.4/bin/mscan passed the binary check.
/nix/store/55gvk6xsfymscn70admmw5wdyn1lc6fg-mblaze-0.4/bin/msed passed the binary check.
Warning: no invocation of /nix/store/55gvk6xsfymscn70admmw5wdyn1lc6fg-mblaze-0.4/bin/mseq had a zero exit code or showed the expected version
/nix/store/55gvk6xsfymscn70admmw5wdyn1lc6fg-mblaze-0.4/bin/mshow passed the binary check.
/nix/store/55gvk6xsfymscn70admmw5wdyn1lc6fg-mblaze-0.4/bin/msort passed the binary check.
/nix/store/55gvk6xsfymscn70admmw5wdyn1lc6fg-mblaze-0.4/bin/mthread passed the binary check.
/nix/store/55gvk6xsfymscn70admmw5wdyn1lc6fg-mblaze-0.4/bin/mcolor passed the binary check.
Warning: no invocation of /nix/store/55gvk6xsfymscn70admmw5wdyn1lc6fg-mblaze-0.4/bin/mcom had a zero exit code or showed the expected version
Warning: no invocation of /nix/store/55gvk6xsfymscn70admmw5wdyn1lc6fg-mblaze-0.4/bin/mless had a zero exit code or showed the expected version
/nix/store/55gvk6xsfymscn70admmw5wdyn1lc6fg-mblaze-0.4/bin/mmkdir passed the binary check.
/nix/store/55gvk6xsfymscn70admmw5wdyn1lc6fg-mblaze-0.4/bin/mquote passed the binary check.
/nix/store/55gvk6xsfymscn70admmw5wdyn1lc6fg-mblaze-0.4/bin/museragent passed the binary check.
19 of 32 passed binary check by having a zero exit code.
0 of 32 passed binary check by having the new version present in output.
found 0.4 with grep in /nix/store/55gvk6xsfymscn70admmw5wdyn1lc6fg-mblaze-0.4
directory tree listing: https://gist.github.com/dbcda53f1a5b826fb9d603500b0df9b5
du listing: https://gist.github.com/597ae6fff02928f12887200fab40f236

Test this build with Cachix (click to expand)

One time setup in nixpkgs Git checkout:
cachix use r-ryantm
git remote add r-ryantm https://github.com/r-ryantm/nixpkgs.git

Test this build:
git fetch r-ryantm && git checkout cf9faeef3aa25964fc2e332ff23cd2a13dad7843 && nix-shell --pure -I nixpkgs=. -p mblaze

cc @ajgrf for testing.

Success on x86_64-linux (full log)
Attempted: mblaze
Partial log (click to expand)
stripping (with command strip and flags -S) in /nix/store/55gvk6xsfymscn70admmw5wdyn1lc6fg-mblaze-0.4/bin
patching script interpreter paths in /nix/store/55gvk6xsfymscn70admmw5wdyn1lc6fg-mblaze-0.4
/nix/store/55gvk6xsfymscn70admmw5wdyn1lc6fg-mblaze-0.4/bin/museragent: interpreter directive changed from "/bin/sh" to "/nix/store/czx8vkrb9jdgjyz8qfksh10vrnqa723l-bash-4.4-p23/bin/sh"
/nix/store/55gvk6xsfymscn70admmw5wdyn1lc6fg-mblaze-0.4/bin/mmkdir: interpreter directive changed from "/bin/sh" to "/nix/store/czx8vkrb9jdgjyz8qfksh10vrnqa723l-bash-4.4-p23/bin/sh"
/nix/store/55gvk6xsfymscn70admmw5wdyn1lc6fg-mblaze-0.4/bin/mquote: interpreter directive changed from "/bin/sh" to "/nix/store/czx8vkrb9jdgjyz8qfksh10vrnqa723l-bash-4.4-p23/bin/sh"
/nix/store/55gvk6xsfymscn70admmw5wdyn1lc6fg-mblaze-0.4/bin/mless: interpreter directive changed from "/bin/sh" to "/nix/store/czx8vkrb9jdgjyz8qfksh10vrnqa723l-bash-4.4-p23/bin/sh"
/nix/store/55gvk6xsfymscn70admmw5wdyn1lc6fg-mblaze-0.4/bin/mcom: interpreter directive changed from "/bin/sh" to "/nix/store/czx8vkrb9jdgjyz8qfksh10vrnqa723l-bash-4.4-p23/bin/sh"
/nix/store/55gvk6xsfymscn70admmw5wdyn1lc6fg-mblaze-0.4/bin/mcolor: interpreter directive changed from "/usr/bin/awk -f" to "/nix/store/zdadpaj5z8k9ifkgsc16yvfjn4wdv0r2-gawk-4.2.1/bin/awk -f"
checking for references to /build in /nix/store/55gvk6xsfymscn70admmw5wdyn1lc6fg-mblaze-0.4...
/nix/store/55gvk6xsfymscn70admmw5wdyn1lc6fg-mblaze-0.4

Success on aarch64-linux (full log)
Attempted: mblaze
Partial log (click to expand)
stripping (with command strip and flags -S) in /nix/store/xf7r0b9d5xrfa4iyiifjn3z730f6x7wa-mblaze-0.4/bin
patching script interpreter paths in /nix/store/xf7r0b9d5xrfa4iyiifjn3z730f6x7wa-mblaze-0.4
/nix/store/xf7r0b9d5xrfa4iyiifjn3z730f6x7wa-mblaze-0.4/bin/museragent: interpreter directive changed from "/bin/sh" to "/nix/store/fqm2x6kiay1q4vg7pqp4wp17bdijlyc3-bash-4.4-p23/bin/sh"
/nix/store/xf7r0b9d5xrfa4iyiifjn3z730f6x7wa-mblaze-0.4/bin/mquote: interpreter directive changed from "/bin/sh" to "/nix/store/fqm2x6kiay1q4vg7pqp4wp17bdijlyc3-bash-4.4-p23/bin/sh"
/nix/store/xf7r0b9d5xrfa4iyiifjn3z730f6x7wa-mblaze-0.4/bin/mmkdir: interpreter directive changed from "/bin/sh" to "/nix/store/fqm2x6kiay1q4vg7pqp4wp17bdijlyc3-bash-4.4-p23/bin/sh"
/nix/store/xf7r0b9d5xrfa4iyiifjn3z730f6x7wa-mblaze-0.4/bin/mless: interpreter directive changed from "/bin/sh" to "/nix/store/fqm2x6kiay1q4vg7pqp4wp17bdijlyc3-bash-4.4-p23/bin/sh"
/nix/store/xf7r0b9d5xrfa4iyiifjn3z730f6x7wa-mblaze-0.4/bin/mcom: interpreter directive changed from "/bin/sh" to "/nix/store/fqm2x6kiay1q4vg7pqp4wp17bdijlyc3-bash-4.4-p23/bin/sh"
/nix/store/xf7r0b9d5xrfa4iyiifjn3z730f6x7wa-mblaze-0.4/bin/mcolor: interpreter directive changed from "/usr/bin/awk -f" to "/nix/store/vz2ppga20x31xmcp4n3i3mbhmrqv1y2k-gawk-4.2.1/bin/awk -f"
checking for references to /build in /nix/store/xf7r0b9d5xrfa4iyiifjn3z730f6x7wa-mblaze-0.4...
/nix/store/xf7r0b9d5xrfa4iyiifjn3z730f6x7wa-mblaze-0.4

Success on x86_64-darwin (full log)
Attempted: mblaze
Partial log (click to expand)
strip is /nix/store/vvbmqsh5ikrxkiwj71dwfbzcwf4bfav8-cctools-binutils-darwin/bin/strip
stripping (with command strip and flags -S) in /nix/store/iqfwq8caikh6igwh4ra74qizi42jcl6d-mblaze-0.4/bin
patching script interpreter paths in /nix/store/iqfwq8caikh6igwh4ra74qizi42jcl6d-mblaze-0.4
/nix/store/iqfwq8caikh6igwh4ra74qizi42jcl6d-mblaze-0.4/bin/mless: interpreter directive changed from "/bin/sh" to "/nix/store/dmgydn3x7ayd7rpfdqw8nrw0z93w549d-bash-4.4-p23/bin/sh"
/nix/store/iqfwq8caikh6igwh4ra74qizi42jcl6d-mblaze-0.4/bin/mcom: interpreter directive changed from "/bin/sh" to "/nix/store/dmgydn3x7ayd7rpfdqw8nrw0z93w549d-bash-4.4-p23/bin/sh"
/nix/store/iqfwq8caikh6igwh4ra74qizi42jcl6d-mblaze-0.4/bin/mmkdir: interpreter directive changed from "/bin/sh" to "/nix/store/dmgydn3x7ayd7rpfdqw8nrw0z93w549d-bash-4.4-p23/bin/sh"
/nix/store/iqfwq8caikh6igwh4ra74qizi42jcl6d-mblaze-0.4/bin/mquote: interpreter directive changed from "/bin/sh" to "/nix/store/dmgydn3x7ayd7rpfdqw8nrw0z93w549d-bash-4.4-p23/bin/sh"
/nix/store/iqfwq8caikh6igwh4ra74qizi42jcl6d-mblaze-0.4/bin/mcolor: interpreter directive changed from "/usr/bin/awk -f" to "/nix/store/bbz18g9g4n8cnppi64irlzshcyqzjc3v-gawk-4.2.1/bin/awk -f"
/nix/store/iqfwq8caikh6igwh4ra74qizi42jcl6d-mblaze-0.4/bin/museragent: interpreter directive changed from "/bin/sh" to "/nix/store/dmgydn3x7ayd7rpfdqw8nrw0z93w549d-bash-4.4-p23/bin/sh"
/nix/store/iqfwq8caikh6igwh4ra74qizi42jcl6d-mblaze-0.4


MH-13662, Update LTI Information

The default LTI page is displaying some basic information about LTI in
Opencast: The current log-in status, LTI connection information, and
links to possible LTI tools. The most commonly used series LTI tool is,
unfortunately, missing from this list.
This patch adds additional information and makes a few simple style
adjustments to make the page look a bit nicer.
Work sponsored by SWITCH

'grats on PR 1000 :)


ACARS  api add response

For example at the prefille stage there is an error when the airplane is not at the specified airport would be nice to have a response so we can alert the user about the error

@vangelisb Is this still open?

@vangelisb bump


Publish DEB for Ubuntu 22.04

As per #262.

[x] Publish DEB package for Ubuntu 22.04 on  packages.microsoft.com  - @d3r3kk
[ ] Update documentation - @brunoborges

Closed as of July-2023-PSU release.


Proposal: Rewrite the entire package. Design doc discussion

Having been using this package for a few years now, I've encountered a number of issues that lead me to want to redesign this package entirely: burn it down and start over, copying only the fundamental parts of the code, and not worrying about backwards compatibility.
Some specific issues I've experienced:

Too much magic. Recently I spent a day debugging an problem where a .zip file couldn't be extracted every time with archiver. Sometimes it would work, sometimes it wouldn't. I eventually discovered that it's because archiver determines which extractor to use based on the extension and the file header while iterating through a map of formats (which is not ordered). If the Zip format came first, it matched by extension but failed to extract; if the TarGz format came first, it matched by file header (because it was actually a tar.gz file), and extraction succeeded.

Weak API. Apparently I was able to accidentally create a .tar.gz file with the Zip archiver, because the name I built for the file was not attached to which archiver format I was using. I can do archiver.TarGz.Make("file.zip") without errors, which is bad. Here's the code that led to my bug in the first place (notice the missing . in "zip"):

var a archiver.Archiver = archiver.TarGz
if filepath.Ext(outputFile) == "zip" {
	a = archiver.Zip
}

^ Bad package design.

Not enough customizability. Namely: compression level; whether to include a top-level named folder vs. just its contents), similar to how rsync works based on presence of a trailing slash; and whether to overwrite existing files when outputting.

Lack of native streaming capabilities. Recently there were From a library perspective, I should be able to stream in a zip file and spit out individual files, or stream in individual files (or a list of filenames?) and spit out a single zip file.

There is no true cross-platform native solution to zip-slip (yet). I had to disable the "security feature" that prevented me extracting a perfectly safe archive. Even "SecureJoin" solutions don't cut it (read the linked thread, and its linked threads). For now, these "mitigations" only get in the way.

Not enough power to inspect archives or cherry-pick files. It would be helpful to be able to work with archives' contents without performing an extraction, such as getting listings, or filtering which files are extracted, etc.

General solutions:

When possible (almost always), match only by file header and ignore file extension. If the file contents are not (yet) available, then use extension but only after a warning or explicit opt-in.

Be verbose in the error messages; if doing any magic, report it or make the magic explicitly opt-in either with a configuration parameter or a special high-level API that is documented as being magical, which wraps the underlying, concrete, explicit functions.

Couple the file extension to the archiver. For example: don't allow the Zip archiver to make a .tar.gz file. For example, the buggy code above could have been avoided with something more like this: archiver.Make(outputFile, files...) which uses the extension of outputFile to force a format that matches.

Expand the API so that an archiver is created for a specific format before being used, rather than having hard-coded globals like archiver.Zip like we do now. This will allow more customization too. Imagine zipArch := archiver.Zip{CompressionLevel: 10} or something similar.

Be explicit about our threat model, which is being adjusted, to state that the files are expected to be trusted, i.e. don't download files you don't trust. Maybe it is possible to inspect a file before extracting it to know whether it could be malicious (e.g. look for zip-slip patterns in file names), but I am not sure about that yet.

Moar interfaces. We have one, Archiver, but we might need more, to accommodate an expanded design with more features. Small interfaces are the best.

Rename the package to archive.

This issue is to track the discussion about the new design; work will hopefully begin soon, as I can find the time.

Thinking we should do the following to focus our efforts in the re-design and channel the communities help towards this?

Update README on this enhancement and create a new project to track related issues that might be better handled in this re-design.
Make the current version maintenance (bug fixes) only and update the README.
Close all PR related older than 6 months..mostly they are around fixing zip slip and other conditions

@johnarok Yes, that all sounds good -- let's draw attention to it and triage old issues that will be handled by the redesign. As soon as I get a chance to get https://relicabackup.com off the ground, I want to turn my attention (at least on weekends) to this, since we use this package extensively when doing releases.

I spent some hours on a detailed API design today that addresses all my concerns, with lots of room for growth as well. Everything here is a very rough draft, subject to change and input, and I'm probably overlooking many implementation/technical difficulties that could cause this design draft to change...
I started by pouring over the open and closed issues and PRs and assembled a wishlist of functionalities:

[x] Make an archive FILE (on disk) by giving it a list of source files

Archiver interface

[x] Extract an entire archive FILE (on disk) to a given directory on disk

Unarchiver interface

[x] Make an archive STREAM (io.Writer) by adding files one at a time (io.Reader + file info)

Writer interface

[x] Extract an archive STREAM (io.Reader) by reading files one at a time (io.Writer + file info)

Reader interface

[x] Walk the contents of an archive and get info about the files

Walker interface

[x] Extract a single file from an archive, possibly during a walk

Extractor or Walker+File interfaces

[x] Get info about a single file from an archive without extracting it

Walker interface (caller of Walk can skip undesired files)

[x] Compress a single file, or byte stream

Compress and CompressFile interfaces

[x] Decompress a single file, or byte stream

Decompress and DecompressFile interfaces

I believe all these requirements are met by the following type definitions:
type Archiver interface {
	Archive(sources []string, destination string) error
}

type Unarchiver interface {
	Unarchive(source, destination string) error
}

type Writer interface {
	Create(out io.Writer) error
	Write(f File) error
	Close() error
}

type Reader interface {
	Open(in io.Reader) error
	Read() (File, error)
	Close() error
}

type Extractor interface {
	Extract(targets []string, source, destination string) error
}

type Walker interface {
	Walk(archive string, walkFn WalkFunc) error
}

type WalkFunc func(f File, err error) error

type File struct {
	// The exact fields we choose could be whatever;
	// these inspired by os.FileInfo
	Name    string
	Size    int64
	Mode    os.FileMode
	ModTime time.Time
	IsDir   bool

	// The original header info; depends on
	// type of archive -- could be nil, too.
	Header interface{}

	// Allow the file contents to be read as an io.Reader.
	io.Reader
}

type FileCompressor interface {
	CompressFile(source, destination string) error
}

type FileDecompressor interface {
	DecompressFile(source, destination string) error
}

type Compressor interface {
	Compress(in io.Reader, out io.Writer) error
}

type Decompressor interface {
	Decompress(in io.Reader, out io.Writer) error
}

Then a Zip archive implementation would start with a struct definition like this:
type Zip struct {
	CompressionLevel     int
	OverwriteExisting    bool
	SelectiveCompression bool
	FollowSymlinks       bool
	ExtractToFolder      bool

	w io.Writer // explained later
	r io.Reader // explained later
}

It could then implement the methods that this archive format is reasonably capable of. Not all formats need to implement all interfaces, that's the beauty of this design. And for the formats that support it, this design also has full support for dealing with streams efficiently.
Reader and Writer allow you to treat the archive kind of like a stream, but it's a little awkward since it's more like a bufio type: you read a file, or write a file (technically just its header under the hood) in a discrete fashion, and then the contents of it can be read or written in the usual io.Reader/io.Writer style. The only other thing is how to keep the state for Reading/Writing. Imagine this (some of this is kind of made-up, just roll with it):
z := &archive.Zip{
	CompressionLevel: 10,
	SelectiveCompression: true,
}

err := z.Create(w)
if err != nil {
	return err
}
defer f.Close()

for _, f := range openFiles {
	err := z.Write(archive.File{
		Name: f.Name(),
		// ...
		Reader: f,
	})
	if err != nil {
		return err
	}
}

We should only allow a Zip instance to either Open or Create (read or write), since otherwise Close is ambiguous. Unless we want to allow closing both. I dunno. There's no performance benefits to reusing a Zip (or other format), so maybe we should prevent reuse to prevent confusion.
Other than that, I'm quite happy with this design so far -- assuming it works -- and will get started implementing it soon.
Feedback welcome


Cannot compile on Windows due to wepoll

Recent upgrades to async-std seems to be using wepoll which requires a build step with C++.  There doesn't seem to be a pre-built image.  For machines without the necessary build tools for C/C++, it fails to build a crate.
Is there anywhere I can download a pre-built image of wepoll, or avoid it altogether?

I ended up installing Mingw64 on Windows and wepoll builds correctly.  However, shouldn't there be a pre-built binary distributed as convenience?


Allowlist verification - Enhancement #16

First draft at implementing Enhancement #16

Add support for new JSON allowlist spec
If the first non-whitespace character is a "{" then we treat it as
JSON. Else we convert the old plain text format into the same structured
data before passing it along
Allow checksum support for the allowlist
Allow GPG signature verification for the allowlist
Allow pulling the allowlist and the signature from a URL

Outstanding items:

CLI parameter validation so things can only be combined in ways that make sense and requirements are validated.
Better error handling
Minor quibbles with the spec in the enhancement which we'll need to shake out

Cursory look, shows this is looking good to me. Do we need to import a public key into a keyring for the verification part, I guess this might be a documentation thing?

Cursory look, shows this is looking good to me. Do we need to import a public key into a keyring for the verification part, I guess this might be a documentation thing?

Do we need to import a public key into a keyring for the verification part, I guess this might be a documentation thing?

No, it's setup that you provide the public key to keylime_tenant on the CLI.

Do we need to import a public key into a keyring for the verification part, I guess this might be a documentation thing?

No, it's setup that you provide the public key to keylime_tenant on the CLI.

btw, this is still a WIP because I need to write tests for it. I keep trying to start that and it keeps being pushed off by other things. Will make a priority this week.

btw, this is still a WIP because I need to write tests for it. I keep trying to start that and it keeps being pushed off by other things. Will make a priority this week.

Tried to run theverifier, and got an error due to missing python module gnupg. Needs to be added to requirements.

Maybe I am missing something, but wasn't it supposed to be backward compatible?
/usr/local/bin/keylime_tenant -c update -t <IP_ADDRESS> -v <IP_ADDRESS> -u 96A1B6FA-77B4-5F02-B3E2-0AEDE0DE617B --cert /var/lib/keylime/ca/96A1B6FA-77B4-5F02-B3E2-0AEDE0DE617B --mb_refstate /tmp/mb_refstate_96A1B6FA-77B4-5F02-B3E2-0AEDE0DE617B.txt --allowlist /tmp/whitelist_96A1B6FA-77B4-5F02-B3E2-0AEDE0DE617B.txt --exclude /ima_exclude.txt --tpm_policy '{"23":"0000000000000000000000000000000000000000000000000000000000000000"}'
Using config file /etc/keylime.conf
2021-03-11 17:40:08.435 - gnupg - INFO - Log opened: Thu Mar 11 17:40:08 2021 UTC
importing ['keylime-ibmextras.elchecking'], __package__='keylime.elchecking'
2021-03-11 17:40:08.498 - keylime.tpm - INFO - TPM2-TOOLS Version: 5.0
2021-03-11 17:40:08.502 - keylime.tenant - INFO - Setting up client TLS in /var/lib/keylime/cv_ca
2021-03-11 17:40:08.502 - keylime.tenant - INFO - TPM PCR Mask from policy is 0x800000
2021-03-11 17:40:08.502 - keylime.tenant - INFO - TPM PCR Mask from policy is 0x808000
2021-03-11 17:40:08.536 - keylime.ima - DEBUG - Loading allowlist from /tmp/whitelist_96A1B6FA-77B4-5F02-B3E2-0AEDE0DE617B.txt
2021-03-11 17:40:08.577 - keylime.ima - DEBUG - Loaded allowlist from /tmp/whitelist_96A1B6FA-77B4-5F02-B3E2-0AEDE0DE617B.txt with checksum 5a99b724c558ad1191e1dd0748161b71e919fd5e5fc36c646692657e4d7d1713
2021-03-11 17:40:08.577 - keylime.ima - DEBUG - Converting legacy allowlist format to JSON
Traceback (most recent call last):
  File "/usr/local/bin/keylime_tenant", line 11, in <module>
    load_entry_point('keylime==0.0.0', 'console_scripts', 'keylime_tenant')()
  File "/usr/local/lib/python3.8/site-packages/keylime-0.0.0-py3.8.egg/keylime/cmd/tenant.py", line 15, in main
    tenant.main()
  File "/usr/local/lib/python3.8/site-packages/keylime-0.0.0-py3.8.egg/keylime/tenant.py", line 1080, in main
    mytenant.init_add(vars(args))
  File "/usr/local/lib/python3.8/site-packages/keylime-0.0.0-py3.8.egg/keylime/tenant.py", line 223, in init_add
    al_data = ima.read_allowlist(args["allowlist"], args["allowlist_checksum"], args["allowlist_sig"], args["allowlist_sig_key"])
  File "/usr/local/lib/python3.8/site-packages/keylime-0.0.0-py3.8.egg/keylime/ima.py", line 469, in read_allowlist
    (checksum_hash, path) = line.split(None, 1)
ValueError: not enough values to unpack (expected 2, got 0)

where the contents of allowlist are (just the beginning)
sha256:4309217e9709def327c3409407f4d52907aeba46ef1b4f46529fd095f2ef9fb2  boot_aggregate

46286ac09df5d09d9b588f950837bb5e1a870404  /KL/keylime/.github/ISSUE_TEMPLATE.md
121d5c7fd8504219a2ba8ed70af80c9c54d26246  /KL/keylime/.github/workflows/greetings.yml
ae279cd3e82bcc74ab7076cf5e1ff4a0ca33884e  /KL/keylime/.github/workflows/semantic-release.yml
e8c1c8e7b55e987961eb5655ee31f1c729861f50  /KL/keylime/.github/workflows/keylime-bot.yml
5b14b8d133383cdfefd7cca751e32fe9a0550a71  /KL/keylime/.github/workflows/test.yml
f5df290c7c3fecd9ca445e22e38d540c8be2e7df  /KL/keylime/.github/stale.yml
1f621973dbe438207eb64c3efb2a77f2bccf3f8f  /KL/keylime/ROADMAP.md
06a388b72b39ca1ef558af373fc09f4adaef0525  /KL/keylime/CONTRIBUTORS.md
197226ca0a74ae6087de34bcc4d5d47f9f480825  /KL/keylime/archive/script_create_quote.py
22bef900d367a21b9eae8be860bd4c215ff3c0dc  /KL/keylime/archive/script_vtpm_register.sh
e50bb8210fb315826bd221a4b3166b58b10d80be  /KL/keylime/archive/script_old_quote.py
974708bd962f52b82da32b00bf69c342582d8e32  /KL/keylime/archive/script_check_quote.py
6451c6e45eda09f36516bbfe7d8061b33fe05da1  /KL/keylime/archive/ima/allowlist.txt
8e4ee61b9916e691a1c1c29a6e2406ef687b2efe  /KL/keylime/archive/ima/ascii_runtime_measurements_ima
b6d69bbc83f786fd14ad905b0674a63c682cfd35  /KL/keylime/archive/ima/exclude.txt
8f27cc51961b5facfd73da1b4512e7e2ac8ed616  /KL/keylime/archive/ima/ascii_runtime_measurements
8b7e3da906b0806df5115a036a2cd2c0acc664a0  /KL/keylime/archive/script_check_quote_raw.py
c1d3531f0b72eea3e575c09ee6876e9e70d4bb5e  /KL/keylime/archive/moc-new.bu.edu-pubek.pem
45fe2b7b94fc0bfc740f98dddcc2616fedabf917  /KL/keylime/archive/llsrc-vtpm-host0_pubek.pem
5043744ef4ef84c25da70394b9dc791501ded301  /KL/keylime/archive/README.md
...

@maugustosilva it is supposed to be backwards compatible so it appears you found a bug :)

Tried to run theverifier, and got an error due to missing python module gnupg. Needs to be added to requirements.

@mpeters just to reiterate this point... I think the requirements.txt should list gnupg and not python-gnupg (certainly did not work for me on my local tests, I had to do pip3 install gnupg by hand

Tried to run theverifier, and got an error due to missing python module gnupg. Needs to be added to requirements.

@mpeters just to reiterate this point... I think the requirements.txt should list gnupg and not python-gnupg (certainly did not work for me on my local tests, I had to do pip3 install gnupg by hand

It's actually really confusing because gnupg and python-gnupg are 2 separate packages, one forked from the other and they are not backwards compatible in subtle ways. And to add to the problem, you use both of them by saying import gnupg so depending on which one(s) are installed, you get something else. So it should be python-gnupg in requirements, but gnupg in import.
@maugustosilva  It's working for me and for CI, so I'm not sure why it's not working for you but we should track that down.

@maugustosilva it is supposed to be backwards compatible so it appears you found a bug :)

So my parser that converted the old format to a new format didn't handle blank lines well, this is now fixed in the latest version with the tests adjusted to cover that condition.

@maugustosilva it is supposed to be backwards compatible so it appears you found a bug :)

So my parser that converted the old format to a new format didn't handle blank lines well, this is now fixed in the latest version with the tests adjusted to cover that condition.

I can confirm it is fully working for me now. LGTM


To-do App in React

In this branch, I have created To-do App in which :

User can add to-do.
User can delete to-do.
User can login.
User can edit to-do after login.
User can visit the profile page after login.
To-dos are stored in local storage.
User data is stored in local storage.

Hello @batoolfatima2135 ,
You did a great job so far👏🏾👏🏾
There are some issues you still need to work on to go to the next project, but you are almost there
To highlight 💪🏾

Gitflow was correctly used.
The page contains the add and remove todo
The code is DRY
👍🏾Good Job

STATUS: CHANGES REQUESTED♻
Please check under the review and make all required changes
Optional suggestions
_Every comment with the [OPTIONAL] prefix is not crucial enough to stop the approval of this PR. However, I strongly recommend you take them into account as they can make your code better:
Cheers, and Happy coding!👏🏾👏🏾👏🏾💻
Feel free to leave any questions or comment in the PR thread if something is not 100% clear.
Please ping me @BenM4k when you comment so I can receive the notification.


Dejkstra newAllShortest: runtime.MemStats.Sys leak

What are you trying to do?
My app needs to rebuild dejkstra allShortest by updated edges data during its lifetime.
What did you do?
After a few rebuilds, I noticed that the app consumes memory more and more. Code inspection leads nothing... It creates a new instance of WeightedDirectedGraph and calculates DijkstraAllPaths, there's no link to old data.
I wrote a simple example to check if this clean version will leak.
What did you expect to happen?
The app consumes the same amount of memory after graph re-init (approximately, using same dataset).
What actually happened?
After each graph re-init, we got increasing memory consumption and it's not freed.

=> loading graph: 1 time
Alloc = 7076 MiB        TotalAlloc = 7087 MiB   Sys = 7404 MiB  HeapInuse = 7081 MiB        HeapSys = 7103 MiB  HeapIdle = 22 MiB   HeapReleased = 22 MiB       StackInuse = 0  StackSys = 0    NumGC = 6
=> loading graph: 2 time
Alloc = 7076 MiB        TotalAlloc = 14163 MiB  Sys = 12060 MiB HeapInuse = 7082  MiB       HeapSys = 11583 MiB HeapIdle = 4501 MiB HeapReleased = 265 MiB      StackInuse = 0  StackSys = 0    NumGC = 7
=> loading graph: 3 time
Alloc = 7076 MiB        TotalAlloc = 21240 MiB  Sys = 12069 MiB HeapInuse = 7083 MiB        HeapSys = 11583 MiB HeapIdle = 4499 MiB HeapReleased = 458 MiB      StackInuse = 0  StackSys = 0    NumGC = 9
..
=> loading graph: 6 time
Alloc = 7076 MiB        TotalAlloc = 42469 MiB  Sys = 15505 MiB HeapInuse = 7083        HeapSys = 14911 HeapIdle = 7828 HeapReleased = 3573     StackInuse = 0  StackSys = 0    NumGC = 17

runtime.MemStats.Sys is increasing
What version of Go and Gonum are you using?
Go version: v1.13.4
gonum version: v0.6.0
Does this issue reproduce with the current master?
Yes.
Code example
import (
	"bufio"
	"fmt"
	"io/ioutil"
	"math"
	"os"
	"runtime"

	graphPath "gonum.org/v1/gonum/graph/path"
	"gonum.org/v1/gonum/graph/simple"
)

func main() {
	var file, err = os.Open("edges_dump.json")
	if err != nil {
		panic(err)
	}

	var b []byte
	if b, err = ioutil.ReadAll(file); err != nil {
		panic(err)
	}

	var edges model.Edges
	if err = edges.Unmarshal(b); err != nil {
		panic(err)
	}

	log("JSON parsed! %d edges found.", len(edges))

	i := 1

	input := bufio.NewScanner(os.Stdin)
	for {
		log("Press Enter to start loading")
		input.Scan()

		log("  => loading graph: %d time", i)

		printMemUsage()

		dg := simple.NewWeightedDirectedGraph(0, math.Inf(1))

		// add edges to graph (for same edges use less weight)
		for _, e := range edges {
			if e.StartPointID == e.EndPointID {
				continue
			}

			var weight float64 = 99999 // no distance - no trip
			if e.RouteMapWeight > 0 {
				weight = float64(e.RouteMapWeight)
			}

			w, ok := dg.Weight(e.StartPointID, e.EndPointID)
			if ok && w <= weight {
				continue
			}

			weightedEdge := dg.NewWeightedEdge(simple.Node(e.StartPointID), simple.Node(e.EndPointID), weight)
			dg.SetWeightedEdge(weightedEdge)
		}

		log("  => DG created!")
		printMemUsage()

		allShortest := graphPath.DijkstraAllPaths(dg)

		log("  => all shortest created!")
		printMemUsage()

		path, weight, unique := allShortest.Between(2, 3)

		// just to escape 'unused variables' error
		log("  => path: %v, weight: %f, unique: %t\n", path, weight, unique)

		i++
	}
}

func log(s string, args ...interface{}) {
	fmt.Printf(s+"\n", args...)
}

func printMemUsage() {
	var m runtime.MemStats
	runtime.ReadMemStats(&m)
	// https://golang.org/pkg/runtime/#MemStats
	fmt.Printf("==================================\n")
	fmt.Printf("Alloc = %v MiB", bToMb(m.Alloc))
	fmt.Printf("\tTotalAlloc = %v MiB", bToMb(m.TotalAlloc))
	fmt.Printf("\tSys = %v MiB", bToMb(m.Sys))
	fmt.Printf("\tHeapInuse = %v MiB", bToMb(m.HeapInuse))
	fmt.Printf("\tHeapSys = %v MiB", bToMb(m.HeapSys))
	fmt.Printf("\tHeapIdle = %v MiB", bToMb(m.HeapIdle))
	fmt.Printf("\tHeapReleased = %v MiB", bToMb(m.HeapReleased))
	fmt.Printf("\tStackInuse = %v MiB", bToMb(m.StackInuse))
	fmt.Printf("\tStackSys = %v MiB", bToMb(m.StackSys))
	fmt.Printf("\tNumGC = %v\n", m.NumGC)
	fmt.Printf("==================================\n")
}

func bToMb(b uint64) uint64 {
	return b / 1024 / 1024
}

Does the value of MemStats.HeapSys eventually settle at a value? In my quick test with a graph generated by gonum.org/v1/gonum/graph/graphs/gen it increases but in the end stays constant after 4-5 iterations. The number of iterations seems to increase for larger graphs, so in your case it may take longer.
I don't know enough about Go runtime, how it allocates memory and whether it is affected by things like memory fragmentation but to me this increase (which in the end levels off) looks like the way the Go memory allocator works (for allocation patterns that your use-case produces) and not like an issue.

I've just noticed that you mention "leak" in the issue title. This is not a leak. Notice that Alloc is constant and what the documentation says about HeapSys:
// HeapSys estimates the largest size the heap has had.

Can you post a self-contained reproducer? The code above depends on your edge file.

@kortschak I've created a small repo https://github.com/savalin/exapmle where I committed the full code and edges dump.
Is it enough?

btw it seems similar https://blog.cloudflare.com/recycling-memory-buffers-in-go/
Maybe it could help somehow.

Yeah, this is not a leak, it's just a consequence of the way the GC works. There's really nothing we can do about this in graph (we go to lengths to avoid allocation in floats, mat, blas and lapack and also to a degree in stat), but graph is much more difficult because of the dynamic nature of the constructs and that users have access to the values that get allocated by the packages (this means sync.Pool is not appropriate). Note that I do make fairly substantial efforts to reduce allocations in graph, it's just that this is much harder.
If this is causing you a problem, it would be best to ask at go-nuts or file an issue against the language (probably in that order).

Thank you for investigation.
Currently, we found the only one workaround: downgrade golang to v1.11.*
Tried the following things:

compilation using go 1.12.13, 1.13.4, master (go1.4-bootstrap-20170531)
gonum forking, implementation Flush() methods for WeightedDirectedGraph/AllShortest internal slices and maps force delete (and it helped for the case when we don't have two graphs in memory at same time)
manual runtime.GC() / debug.FreeOSMemory() calls
GODEBUG=madvdontneed=1

It looks like this is being handled at golang/go, so closing this. Please comment here again if we need to do more.


Cross Over Review

This Week

[x] Create a Backend API to store sorting times
[x] Create a Frontend Wireframe that sorts
[x] Start on the Fibonacci sequence page

Justins Work

Created a working leaderboard that is currently working with frontend using localhost
link

Finn's Work

Create Most of the javascript logic behind the sorting, and also started on fib visualization
js
bars

Theo's Work

Created Most Styling and helped with the creation of the sorting wireframe
wireframe

What's left

We have already mostly integrated our features with local host so the only things left are figuring how to deploy on was (broken rn), finishing up fib, polishing the SASS of the website

I like that everyone is contributing to the project and project seems to be going along well - progress was made. I'm not fully sure though if everyone has Java contributions and it may have been nice to show key commits instead of just links.


The gauge is not centered if too small

Seems that the alignment for the gauge stops working for smaller size, as on the filter shop...

So not related to the size, but to box-sizing: border-box;


societal_poi_municipality_intersection

When getting data from Oracle I frequently see the error unknown type nulltype - setting type to varchar 255
0.13.2
https://openmetadata.slack.com/archives/C02B6955S4S/p1675847523160079

for Oracle 19


Import Key Vault Certificate

See https://github.com/dotnet/aspnetcore/issues/29891

@marcelWCH
In Step 10 of the workshop, the instruction to add a certificate is not for your Database. Af the end of the section "Publishing to a new App Service", you can refer to the following statement:
If we publish the app at this point, it will return a server error and fail to start. This is because we first need to configure a signing key for IdentityServer. During development, we used a development key (see BlazingPizza.Server/appsettings.Development.json), but in production we need to configure an actual certificate for issuing tokens. We'll do that using Azure Key Vault.
As you can see, The application makes use of Identity Server as an Identity Provider for your application. While developing lcoally it uses local development certificate to encryot the identity token. But when you deploy to your app service - you need a certificate uploaded to your app service for the token to be encrypted.
With respect to you not able to add a certificate to your app service:  The portal is already giving you a warning that "SSL bindings can be used in Basic plans and higher". Looks like you have your pricing tier set to F1 (Free) or D1 (Shared) tier. You need to be in minimum BASIC tier or above BASIC tier to add a TLS/SSL certificate.
Hope this clarifies your question.


Updating steps for module 1; adding instructions for environment set-up

Hey @jpechane, can you check this out?
It updates the module 1 to Strimzi 0.7.0. It also provides an Ansible set-up to prepare some box you can SSH into.

I'm going to rebase and merge this; the issue with metrics will have to wait for another time. Thanks a lot for your help on that one so far, @scholzj. Might come back to you with more details later on.


Add Windows support

Created the Windows project using:
flutter create --platforms=windows .

Demo:

I'm not sure how to review this - is it all generated?

The first two commits are generated by flutter create --platforms=windows ., the last commit contains manual edits: https://github.com/google/generative-ai-dart/pull/59/commits/27e1c8ccc34dee72267a55860e24ebf02285da2c


Modify generate.sh to take a dynamic number of GPUs as input

Issue
Data generation requires exactly 8 GPUs to be present. This doesn't make the code run properly on machines with less than 8 GPUs (for instance, I am using 4 H100s)
Changes
Changed generate.sh so that the user can specify the number of GPUs as input, with a minimum number of 1. The default is still set to 8 GPUs and the number will change only if the --gpu parameter is specified

@angelahzyuan could you take a look at this when you have time? Would love to merge it if it looks good to you. Thanks!


recordException should support chained exceptions

What are you trying to achieve?
Many languages support a concept of a chained exception. For instance, JavaScript has Error.cause. Java also has Throwable.getCause(). When present, is just as important (and often more so), than the wrapping exception. Currently recordException() loses this information.
It'd be valuable to clarify whether the relevant languages should support the cause and how.

Just ran into this problem as well and ended up here. Has there been any further discussion?
I don't really have anything to contribute other than expressing support for tackling this...would chaining span events somehow be an option?
Thanks!


6/5 Publish

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs-ref-conceptual/install-azure-cli.md
:white_check_mark:Succeeded
View (azure-cli-latest) View (azure-cli-2017-03-09-profile)

docs-ref-conceptual/release-notes-azure-cli.md
:white_check_mark:Succeeded
View (azure-cli-latest) View (azure-cli-2017-03-09-profile)

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


cmd/gobgp: don't use timeout context for requests

timeout context is for only connecting. We should refactor the code to
avoid global context variant.
Signed-off-by: FUJITA Tomonori<EMAIL_ADDRESS>

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@0eb3dce). Click here to learn what that means.
The diff coverage is 0%.

@@            Coverage Diff            @@
##             master      #85   +/-   ##
=========================================
  Coverage          ?   34.61%
=========================================
  Files             ?       59
  Lines             ?    27953
  Branches          ?        0
=========================================
  Hits              ?     9675
  Misses            ?    17245
  Partials          ?     1033

Impacted Files
Coverage Δ

cmd/gobgp/root.go
0% <0%> (ø)

cmd/gobgp/common.go
14.58% <0%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0eb3dce...ee53d20. Read the comment docs.


[Enhancement] Actor Nametags

Ports over Actor Nametags from SoH https://github.com/HarbourMasters/Shipwright/pull/3083
Notable differences from SoH include:

Nametags render roughly 2x farther (this could be changed to any value really, but if its too far away you cant really read it anyways)
Individual nametags can set whether ZBuffer should be used (obscured by geometry)

Debug nametags from the actor viewer currently set this so that it is easier to see them. We can always make this a configurable option in the actor viewer.

Hooks are optimized to (un)registered when there exists nametags

https://github.com/user-attachments/assets/eb3fb847-9e86-4469-a1f6-e1e8a3f42f97

New Hooks:

OnPlayDrawWorldEnd - Fires after the world is done drawing, but before the interface. This allows enhancements to render things that only need to render while the world is visible (etc, collision viewer, nametags). This also is within the frame interpolation recording bounds.
OnActorDestroy - Called right before an actor is freed from the arena. When actors are removed via room transitions or during day/night transitions, they will not always call Actor_Kill, but Actor_Destroy will always be called in both cases. OnActorDestroy is a better hook to use over OnActorKill

I've also regrouped some things in the actor viewer and fixed a bug where having a selected actor could crash through room transitions or day/night transitions.
Additionally this added a new ShipUtils where I have currently added some helper methods for things I needed. If we feel this should just belong to BenPort, I can move them there too, but I was trying to avoid that becoming a catch all file.

Build Artifacts

2ship-linux.zip
2ship-windows.zip
2ship-mac.zip

Updates since last review:

Refactor a bit of the actor viewer to get rid of unused variables
Improved dynamic hook (un)registration for selected actor
Added more options for actor name tag display


Integrate install_yamls wrapper in libvirt_manager

This change leverage the install_yamls wrapper from within
libvirt_manager.
It requires some changes to the use_install_yamls role as well as to the
generate_make_tasks.py module:
use_install_yamls:

remove useless parameter
ensure we export edpm_env even if we don't provide global settings

generate_make_tasks:

allow providing specific parameters when we include a task file
allow overriding specific environment using the combine filter

This PR has:

[ ] Appropriate testing (molecule, python unit tests)
[ ] Appropriate documentation (README in the role, main README is up-to-date)

/recheck

/test all


Separate backend and cloud/third party documentation

As suggested in PR #2354 we should split up the current backend and hosted services documentation. E.g. keep backend docs in the current section and move everything relating to self-hosting or third party hosted services to its own section.
The purpose of this would mainly be to:

Make the section more readable by separating concerns.
Maybe to be able to make a note in the separate section for third party/cloud services that this information is contributed by other users, as the rustic core developers will not be able to make sure it's always up to date.

The thing is that some of the third party services, albeit using the same protocols as the supported backends in restic, often require some specific configuration item or key, and this is something restic core devs can't always keep an eye on. Hence, this section in the documentation will rely more on user contributions.
A related proposal was to move the user contributed ones to wiki posts in the forum. I would vote against that, because I think it's almost equally simple for users to contribute to the documentation on GitHub, so I'd prefer to not split up the documentation in two separate places. Better keep it consolidated in one place.
Keep in current section:

Local
SFTP
REST server
Amazon S3
OpenStack Swift
Backblaze B2

Move to separate section:

Wasabi

Undecided:

Minio Server
Other services via rclone
Microsoft Azure Blob Storage
Google Cloud Storage

IMHO users should be encouraged to use rclone for all backends except local and REST server. This would make it easier to phase out all the backends in restic in a long term (see #2411).

In one sense what you say is indeed right, it would lessen the burden on rustic.
Personally though I never use clone, because it involves downloading and maintaining another binary, and having to manage also another configuration (I wish rclone allowed running without an established configuration). I really don't like it, and as long as I can use rustic without rclone, I will do so. The only point I see in using rclone is if there's some service I need to connect to that restic can't handle. But I never had that need.
I'm hoping many others than just me also use restic without rclone.


Need a Yes/No editor prompt that returns True/False to template parameter

Is your feature request related to a problem? Please describe.
The Custom Learning PnP template includes a parameter to enable or disable telemetry. We would like to make this parameter visible when the template is installed.
Describe the solution you'd like
When provisioning the template, the user is presented with Yes or No radio buttons, which they can select in response to a text prompt.
Describe alternatives you've considered
We tried using the Boolean editor, in settings.json as follows:
      {
          "name":"Telemetry",
          "caption":"Improve learning pathways by sending usage telemetry to Microsoft?",
          "description":"Microsoft uses this data to help improve future Microsoft 365 learning pathways solutions. To learn more about Microsoft privacy policies see https://go.microsoft.com/fwlink/?LinkId=521839.",
          "editor": "Boolean"
      }

But it returned this error:
There was an unexpected issue. The model item passed into the dictionary is of type 'SharePointPnP.ProvisioningApp.WebApp.Models.TemplateParameterModel', but this dictionary requires a model item of type 'System.Boolean'.
Additional context
Bert suggested I tag @PaoloPia in my inquiry.

Hi,
Thanks for your feedback. I'm going to close this item because it has been stale for a long time. Thanks for your understanding.


Support instantiating ES6 classes

When exporting a ES6 class (constructor), natively supported unter node.js 4.x/5.x, upon instantiation you get the exception "TypeError: Class constructors cannot be invoked without 'new'", because of the indirect instantiation in utils.applyConstruct. This fixes it by directly invoked the constructor with new.

Cheers, nice PR :+1:


Replaced Music with EvW Soundtrack

Replaced the mod's music with the soundtrack from East vs. West. This
includes the main theme, which is actually Russian Battle 2 from East
vs. West, but the new version doesn't contain the sounds of someone
clicking through the HoI4 main menu and doesn't cut out 30 seconds
before the track ends.
Additionally, instead of having the mod's music replace the original
filenames, the original tracks are disabled through songs.txt. Songs.txt
also makes it so that certain tracks are only played under certain
circumstances (in a war, recently lost a war, democratic government,
non-democratic government, etc.).

DelnarErsike your pull request removes all music files, but you do not commit the updates nor do you add the songs.txt. Please make a commit with those.

Whoops, looks like those files got deleted for some reason, even though they were included in the commit, might have been BitTorrent Sync messing with things. Either way, I've commited them again and submitted a pull request.


Dynamic Shrinking Microbatches

Instead of crashing with CUDA out of memory, we dynamically scale the number of microbatches so we can fit the logical batchsize irrespective of the underlying hardware. Right now, we only decrease if we crash.
As of now, this is not set as default and we emit a warning if it is used. This causes errors if used with methods which change memory usage (as of now, this is just seq_length_warmup)
Related Design Doc
Test Log:

Resnet 56 on CIFAR with batchsize initially set to 8000

Epoch 0:   0%|          | 0/6 [00:02<?, ?it/s]
1 -> 2
Epoch 0:   0%|          | 0/6 [00:02<?, ?it/s]
2 -> 4
Epoch 0:  83%|████████▎ | 5/6 [00:08<00:01,  1.40s/it, loss/train=3.2449]

We are able to successfully increase grad_accum until it fits on GPU

Resnet 56 on CIFAR with batchsize initially set to 8000 + progressive resizing

Epoch 39: 100%|██████████| 6/6 [00:04<00:00,  1.25it/s, loss/train=1.4595]
Epoch 40, Batch 240 (val): 100%|██████████| 10/10 [00:00<00:00, 17.03it/s, accuracy/val=0.2500]
Epoch 40:   0%|          | 0/6 [00:01<?, ?it/s]
1 -> 2
Epoch 40: 100%|██████████| 6/6 [00:05<00:00,  1.10it/s, loss/train=1.5751]
Epoch 41, Batch 246 (val): 100%|██████████| 10/10 [00:00<00:00, 17.10it/s, accuracy/val=0.2736]

We are able to successfully run with 8000 batchsize until images start scaling up, at which point we increase grad_accum

GPT2_52m with default params

Epoch 0:   0%|          | 0/9000 [00:02<?, ?it/s]
1 -> 2
Epoch 0:   0%|          | 0/9000 [00:02<?, ?it/s]
2 -> 4
4 -> 8
Epoch 0:   0%|          | 0/9000 [00:02<?, ?it/s]
8 -> 16
Epoch 0:   0%|          | 1/9000 [00:03<7:39:21,  3.06s/it, loss/train=10.9392]
16 -> 32
Epoch 0:   0%|          | 2/9000 [00:04<4:50:48,  1.94s/it, loss/train=10.9292]
32 -> 64
Epoch 0:   0%|          | 3/9000 [00:04<3:19:16,  1.33s/it, loss/train=10.6871]
64 -> 128
Epoch 0:   0%|          | 4/9000 [00:05<2:20:50,  1.06it/s, loss/train=10.4537]
128 -> 256
Epoch 0:   0%|          | 11/9000 [01:38<35:01:09, 14.02s/it, loss/train=9.6154]

Multi-GPU GPT2 (on 2 GPUs)

Epoch 0:   0%|          | 0/9000 [00:03<?, ?it/s]
1 -> 2
Epoch 0:   0%|          | 0/9000 [00:03<?, ?it/s]
2 -> 4
Epoch 0:   0%|          | 0/9000 [00:04<?, ?it/s]
4 -> 8
Epoch 0:   0%|          | 1/9000 [00:05<13:06:39,  5.24s/it, loss/train=10.9348]
8 -> 16
Epoch 0:   0%|          | 2/9000 [00:06<7:29:42,  3.00s/it, loss/train=10.9314]
16 -> 32
Epoch 0:   0%|          | 35/9000 [03:28<15:42:19,  6.31s/it, loss/train=7.1741]

Also, could you add a check to raise an error if someone tries to use this when training on CPU?

I'm fine doing -1 to enable dynamic, but one issue is that you can't set a starting point for the search. For example, if I want to use it with a large GPT model, it's going to take 6-8 failures before it ramps up to a good grad accum size. Probably fine since its all at start and not too long...?

I'm fine doing -1 to enable dynamic, but one issue is that you can't set a starting point for the search. For example, if I want to use it with a large GPT model, it's going to take 6-8 failures before it ramps up to a good grad accum size. Probably fine since its all at start and not too long...?

Yeah, it's not perfect, but it's a lot more approachable from the user's perspective. I'm OK with not letting the user configure the starting point for now - we can always add it later.
My hope is that dynamic grad accum is going to become the de facto standard - that there will be essentially no reason not to use it. It will take a bit of time to validate that, but if so, we'll probably want to change the API around grad accum anyways.

Updated based on all the comments :) Will now start testing

Talked w Jamie a bit about deadlocks. TDLR: There is a slight risk of deadlock with multiple GPUs, but it's very low and seems very hard to mitigate
We currently hide DDP latency behind the computation of the backward pass. We deadlock if 1) one GPU finishes the forward and backward pass with no issues but 2) one GPU crashes midway through the backward pass after sending at least some of its gradients to the GPU which finished. In this case, the second GPU will wait on weights from the first one it already received before crashing.
I think this is probably low prob because (to my knowledge, could be wrong) pytorch allocates the memory it needs for graph for backward pass at start. This is also extra low prob if grad_accum > 1 since the graph is already populated at start of last pass, so theoretically, it shouldn’t need more allocation (but of course, theres always random stuff happening). This is probably fine as a transient error since it happens at the start of training (usually, unless ur doing something that increases in memory through training) and u can just restart
At least experimentally, I haven’t been able to trigger this. I ran GPT 2 a few times on 2 GPUs with only 24/4268 MiB free per nvidia-smi, and no issues (see screenshot at top of PR).
We could solve this by not hiding the latency, but this seems like a steep performance cost. I think the best course is to just slap a warning on when someone uses auto. Also, note that this is an issue if people manually set grad_accum as well

I think this is in a good shape and tests seem to be fine. I'd love some input on https://github.com/mosaicml/composer/pull/580, and then I'll merge that here and rerun all tests again

There's been a lot going on here, so to recap on current situation:
It currently works and is functional! We're throwing a warning for using it, and it's not compatible with certain (just one, sequence length warmup) algorithms.
Theoretically, we can also run into issues of deadlock if just 1 rank OOMs. Ravi and I discussed this last week, and there's some ideas on how to propagate errors across ranks. This feels like a more useful thing in general, so I'll be playing around with that and possibly generalizing to a decorator we might use more broadly.

OK great, I think we are close to getting this merged. I re-requested some reviews. Given the scope of this refactor of the trainer loop itself, we should also touch off some regression runs. Right before merge, let's run a full sweep of regressions.

@ravi-mosaicml rerequesting review! I've wrapped the train loop to sync if any rank fails and rerun. I propose throwing out any progress and restarting the batch if anything fails. In the scenario where only one rank fails, the rest of the ranks either

successfully complete and have to redo some work (which is fine since it only happens once) or
they throw an error (say one rank errors, so everyone else times out at a barrier because it never reaches a gradient sync), which is caught. At the end, we see another rank OOMed, so we retry training on all ranks after updating grad_accum

Ran some tests to confirm results are identical. With resnet 50 on imagenet 8 GPUs, we get basically the same curves (theyre overalpping) when comparing

dev vs. this branch + no auto grad_accum vs. this branch + auto grad_accum
same as above except including blurpool + label smoothing + mixup + channels last

Note that grad_accum is set to 2 since it was run on coda, so the auto grad_accum did actually have to activate and change grad_accum


Support HttpContext as SseRequestOptions

Hi.
I need to put HttpContext as SseRequestOptions, but the interface doesn't support it.
Can you allow to put httpContext as an option?
    this._sseClient
      .stream(
        'api/sse/connect',
        {
          keepAlive: true,
          reconnectionDelay: 1_000,
          responseType: 'event',
        },
        {
          context: new HttpContext().set(NGX_LOADING_BAR_IGNORED, true),
        } as any, // here!
        'GET'
      )
      .subscribe(event => {
        if (event.type === 'error') {
          const errorEvent = event as ErrorEvent;
          console.error(errorEvent.error, errorEvent.message);
        } else {
          const messageEvent = event as MessageEvent;
          console.info(
            `SSE request with type "${messageEvent.type}" and data "${messageEvent.data}"`
          );
        }
      });

Hey @JayAhn2 , thank you for your input. Unfortunately it's been a while since I am not working with Angular, but I will include your request I soon as I can. If you are comfortable, feel free to create a PR I will be glad to take a look.

Hey @JayAhn2 , I am happy to tell you that I just release the v3.0.0. This version has the official support for Angular >=v12 and also supports the context attribute on the HTTP request options.


Add flume-shared module to assembly xmls

Fixes the issue that the assembly plugin created the source package without the flume-shared project in it so maven compile failed on that.

+1 from me


Please add: Logo Design Wizard

https://chat.openai.com/g/g-OfG13AhZC-logo-design-wizard

Please raise a pull request for the same


Add taxClass field to ProductType type

Integrate tax classes with products, product types and shipping methods API.
Impact

[ ] New migrations
[ ] New/Updated API fields or mutations
[ ] Deprecated API fields or mutations
[ ] Removed API types, fields, or mutations
[ ] Documentation needs to be updated

Pull Request Checklist

[ ] Privileged queries and mutations are guarded by proper permission checks
[ ] Database queries are optimized and the number of queries is constant
[ ] Database migration files are up to date
[ ] The changes are tested
[ ] GraphQL schema and type definitions are up to date
[ ] Changes are mentioned in the changelog

Task linked: SALEOR-7192 Add API to query/update tax classes for products, product types and shipping methods.


Json extractor

Single values extracted from a JSON array are wrapped in an array. This makes it difficult to use the extracted value in a subsequent request.
This change is to unwrap extracted values if they are arrays of size 1.
Note: This changes existing behavior.

Described jmeter-standard approach was implemented


Video link not working

I think video link or YouTube video has some issues in the Heroku doc.
https://docs.appsmith.com/setup/heroku
@cokoghenun can you look into this?

@harshilp24 I'm not sure if it's a Gitbook issue but it works now


android crash if sharedPreferencesName not supplied

I'll submit a PR shortly.
Exception in native call
                                                             com.facebook.react.bridge.NoSuchKeyException: couldn't find key sharedPreferencesName in dynamic object
                                                                 at com.facebook.react.bridge.ReadableNativeMap.getString(Native Method)
                                                                 at br.com.classapp.RNSensitiveInfo.RNSensitiveInfoModule.getItem(RNSensitiveInfoModule.java:26)
                                                                 at java.lang.reflect.Method.invoke(Native Method)
                                                                 at com.facebook.react.bridge.BaseJavaModule$JavaMethod.invoke(BaseJavaModule.java:345)
                                                                 at com.facebook.react.cxxbridge.JavaModuleWrapper.invoke(JavaModuleWrapper.java:136)
                                                                 at com.facebook.react.bridge.queue.NativeRunnable.run(Native Method)

What state is this project in, is it still being maintained?  I tried to submit a PR but then noticed that there have been a bunch of unreleased commits since the npm version (4.0.0). My fix doesn't really apply to master.

Thank you for working on this issue. The problem with the unreleased commits was solved. Right now we are busy with other projects, but you're welcome to open a pull request.

@ricardofavero here you go https://github.com/classapp/react-native-sensitive-info/pull/22
I was a bit confused by the initKeyStore stuff that's appeared in there.  It doesn't seem to be used, but also broke my build - i couldn't even compile.  I've backed out a build file change that was causing the problem as a part of this PR, but there seems to be quite a bit of stuff in there still that isn't used (and maybe is partially reverted??).


feat: add RedHat 8 support

Have you read the Contributing Guidelines?
Changes description
Adding support for RedHat 8. Installation based on Posit docs
I had to do some changes in code related to test and CI. The main one is additional platform in molecule.yml configuration - I had to use Red Hat Universal Base Image

Task linked: CU-862j847m2 Add RedHat support for Posit/RStudio Ansible roles


update devcontainer to bookworm container using rust-nightly

supersedes #169

What about using idf-rust tags, which are generated in this workflow from this container file. The container file could probably be improved, and IIRC, you did some work on that a while ago? We could pick that work, finish it and leverage it here too

What about using idf-rust tags, which are generated in this workflow from this container file.

If i understand you correctly you want to (depending on the tag) just link to an existing build image from the repo? yeah we may should do this.
As you see i already only include the xtensa toolchain if riscv is not set in this. Also i removed espflash from the container and some libusb dep's that really only needed if one would attempt to forward the usb-connection.
I think its good to switch from base bookworm to the rust-lang nightly image. I also thinking about adding an unrelated thing, but useful for guys using the container to put the compiled binary into a different outdir with this option inside the config.toml
[build]
.....
out-dir = "bin" # copy build artifacts to bin/

[unstable]
unstable-options = true # for out-dir usage

the container don't build because the action itself sources the source /home/esp/export-esp.sh. I could adapt the action with a simple if / else, but i think i will look into the published idf-rust containers first

for now i created https://github.com/esp-rs/rust-build/issues/247 where we can coordinate potential improvements for all projects

I think its good to switch from base bookworm to the rust-lang nightly image. Your opinion here?

There is even a nightly-bookworm-slim which is 240 MB smaller, we could also evaluate it.
Regarding setting the output to bin I dont have an strong opinion, to be honest. Im used to locate the elf files under the target folder, but I might be a bit biased. Also, espflash/espfhash can be very handy here.

I think its good to switch from base bookworm to the rust-lang nightly image. Your opinion here?

There is even a nightly-bookworm-slim which is 240 MB smaller, we could also evaluate it.

this should be the same image just from docker-hub and not from ghcr ?

Regarding setting the output to bin I dont have an strong opinion, to be honest. Im used to locate the elf files under the target folder, but I might be a bit biased. Also, espflash/espfhash can be very handy here.

Does espflash correctly autofinds debug/release elf on no_std projects? for my projects i always need to give it the path to the elf

this should be the same image just from docker-hub and not from ghcr ?

Just checked and its also available in GitHub registry: https://github.com/rust-lang/docker-rust-nightly/pkgs/container/rust/144487152?tag=nightly-bookworm-slim

Does espflash correctly autofinds debug/release elf on no_std projects? for my projects i always need to give it the path to the elf

No, I meant it for the save-image subcomand


[Feature] EvalAlgorithmInterface.evaluate should accept a list of DataConfigs for consistency

Today EvalAlgorithmInterface.evaluate is typed to return List[EvalOutput] ("for dataset(s)", per the docstring), but its dataset_config argument only accepts Optional[DataConfig].
It seems like most concrete eval algorithms (like QAAccuracy here) either take the user's data_config for a single dataset, or take all the pre-defined DATASET_CONFIGS relevant to the evaluator's problem type.
...So the internal logic of evaluators is set up to support providing multiple datasets and returning multiple results already, but we seem to prevent users from calling evaluate() with multiple of their own datasets for no particular reason?

Your understanding is correct. Currently, evaluate can either be configured to use a single user-provided dataset (via data_config) or configured to use all of the "built-in" datasets. Your feature request certainly makes sense; there isn't a particularly compelling reason I can think of for why we shouldn't be able to evaluate multiple "custom" (i.e. user-provided) datasets.


Update frozen.py

Frozen error handler is triggered too quickly for HSE calcs. Proposed changing from 1 hr to 10 hrs

Thanks for fixing this! 🥳


Addressing flakes in listener Conflict Test

Adding a sleep between service pushes to make sure they get different
creationTimestamps.

Use the push_status endpoint rather than scraping pilot logs.

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: nmittler
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approver: baodongli
If they are not already assigned, you can assign the PR to them by writing /assign @baodongli in a comment when ready.
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

tests/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@nmittler: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-pilot-e2e-envoyv2-v1alpha3.sh
17881cd7ab49d78fd25f9712a198ffef7ff24317
link
/test istio-pilot-e2e-envoyv2-v1alpha3

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

It appears to still fail...

/lgtm

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: andraxylia, nmittler
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approver: baodongli
If they are not already assigned, you can assign the PR to them by writing /assign @baodongli in a comment when ready.
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

tests/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@nmittler: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-pilot-e2e-envoyv2-v1alpha3.sh
01a4b296c44744588cb05f54e2bdae94f51ad708
link
/test istio-pilot-e2e-envoyv2-v1alpha3

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@nmittler: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-pilot-e2e-envoyv2-v1alpha3.sh
01a4b296c44744588cb05f54e2bdae94f51ad708
link
/test istio-pilot-e2e-envoyv2-v1alpha3

prow/e2e-bookInfoTests-v1alpha3.sh
01a4b296c44744588cb05f54e2bdae94f51ad708
link
/test e2e-bookInfo-envoyv2-v1alpha3

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/test e2e-bookInfo-envoyv2-v1alpha3
/test istio-pilot-e2e-envoyv2-v1alpha3

@nmittler: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/e2e-bookInfoTests-v1alpha3.sh
01a4b296c44744588cb05f54e2bdae94f51ad708
link
/test e2e-bookInfo-envoyv2-v1alpha3

prow/istio-pilot-e2e-envoyv2-v1alpha3.sh
01a4b296c44744588cb05f54e2bdae94f51ad708
link
/test istio-pilot-e2e-envoyv2-v1alpha3

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/test istio-pilot-e2e-envoyv2-v1alpha3


Add support for quotation inside interpolation in attributes

Original (ruby) implementation counts curly braces occurrences with ruby code and ignores quotes occurrences unless they are balanced. I implemented the same approach only with recursive regular expressions.

:+1:

Thanks @gekola!


Add Xai chain details and banner to layout

PR-Codex overview
The focus of this PR is to enhance the chain page layout in the dashboard by adding badges, updating banner images, and improving button styling.
Detailed summary

Updated banner images for chains
Added badges for verified and sponsored chains
Enhanced button styling
Improved chain metadata handling

✨ Ask PR-Codex anything about this PR by commenting with /codex {your question}

[!WARNING]
This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

#2626  👈
#2541
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @jnsdls and the rest of your teammates on  Graphite

replaced by #2629


Add pixels for importing google passwords metrics

Task/Issue URL: https://app.asana.com/0/608920331025315/1208787586139232/f
Description
Steps to test this PR
Feature 1

[ ]
[ ]

UI changes

Before
After

!(Upload before screenshot)
(Upload after screenshot)

[!WARNING]
This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

#5284  👈 (View in Graphite)
#5098
#5097
#4601 : 1 other dependent PR (#4602 )
#5096
develop

This stack of pull requests is managed by Graphite. Learn more about stacking.

LGTM, everything works as expected.
Just a note, when reading in the latest scenario "Launch password import flow; verify you don't see any UI hints on which button to press", I was surprised because I didn't have hint in the previous scenarios. Let's review that tomorrow.

Isolated the problem and fixed in https://github.com/duckduckgo/Android/pull/5295. this branch has been restacked on top, meaning if you test again you should see the hints working now. good spot!


Error when trying to get deplo.io applications for a project

I tried to get the applications in a project with the following and received an error:
$ nctl get applications --project renuo-hello-world
nctl: error: applications.apps.nine.ch is forbidden: User<EMAIL_ADDRESS>cannot list resource "applications" in API group "apps.nine.ch" in the namespace "renuo-hello-world"

Thanks for reporting this, there's a bug with the permission binding in projects. Should be fixed early next week on our API, no changes should be needed in nctl.

@rnestler The permissions should be fixed, can you give this another try?

Yes that looks good, there are no apps in that project. I'll close this for now.


404 page for SSO and demo website

Acceptance criteria

[ ] 404 page is shown when user goes to a path that does not exist
[ ] designs match figma for mobile and desktop

Referance in figma https://www.figma.com/file/NuupSK3R2ddSzSuWk7gcee/Website-error-page?node-id=0%3A1&t=kpA63HOmOkylKoNV-1
needed SVGs and PNGs https://drive.google.com/drive/u/2/folders/1415y-CmxNzj8kgIxGdcGXtkZRMDCKQNJ


Update LICENSE

Change Gnosis to Safe Ecosystem Foundation
@gnosis/mobile-devs

Please do NOT update the year


ROCm: Support hipCUB

hipCUB can target either rocPRIM or CUB:
https://rocmdocs.amd.com/en/latest/ROCm_Libraries/ROCm_Libraries.html#hipcub
We might already have the build infrastructure added (https://github.com/cupy/cupy/pull/2666), not sure though.

A bug related to using hipCUB is being fixed in #3835.


Create the Select Component

Select Component

[ ] Option to set the default value
[ ] Only one value can be choose
[ ] A user can choose between one way or return flight type.
[ ] One option should be always selected

behavior:
After the user clicks on the InputField, modal with options will be open (animation from the bottom - can be avoided in the first implementation).

Wait until issue(create trip component) on universal will be done.https://github.com/kiwicom/universal-components/issues/92


xExchangeHelper: Rename common functions and fix common style issues

Pull Request (PR) description
Renames all common functions in xExchangeHelper.psm1 to use approved Verb-Noun format. Fixes a number of style issues in xExchangeHelper.psm1, as well as in calls to helper functions from other resources in the module. Removes duplicate helper functions in MSFT_xExchTransportService.
This Pull Request (PR) fixes the following issues
Fixes #306
Fixes #313
Fixes #314
Task list

[x] Added an entry under the Unreleased section of the change log in the CHANGELOG.md.
Entry should say what was changed, and how that affects users (if applicable).
[ ] Resource documentation added/updated in README.md.
[ ] Resource parameter descriptions added/updated in README.md, schema.mof
and comment-based help.
[x] Comment-based help added/updated.
[ ] Localization strings added/updated in all localization files as appropriate.
[ ] Examples appropriately added/updated.
[x] Unit tests added/updated. See DSC Resource Testing Guidelines.
[x] Integration tests added/updated (where possible). See DSC Resource Testing Guidelines.
[x] New/changed code adheres to DSC Resource Style Guidelines and Best Practices.

Test Results
All unit and integration tests ran and passed against Exchange 2016 CU10 running on Windows 2016.

This change is

Codecov Report

Merging #316 into dev will increase coverage by 0.01%.
The diff coverage is 1.3%.

@@           Coverage Diff           @@
##             dev   #316      +/-   ##
=======================================
+ Coverage   2.69%   2.7%   +0.01%
=======================================
  Files         38     38
  Lines       4454   4435      -19
=======================================
  Hits         120    120
+ Misses      4334   4315      -19

Impacted Files
Coverage Δ

...irtualDirectory/MSFT_xExchEcpVirtualDirectory.psm1
1.92% <0%> (ø)
:arrow_up:

...chJetstressCleanup/MSFT_xExchJetstressCleanup.psm1
1.83% <0%> (ø)
:arrow_up:

..._xExchExchangeServer/MSFT_xExchExchangeServer.psm1
1.61% <0%> (ø)
:arrow_up:

...chReceiveConnector/MSFT_xExchReceiveConnector.psm1
0.44% <0%> (ø)
:arrow_up:

...lboxDatabase/MSFT_xExchWaitForMailboxDatabase.psm1
2.94% <0%> (ø)
:arrow_up:

...MSFT_xExchImapSettings/MSFT_xExchImapSettings.psm1
2.77% <0%> (ø)
:arrow_up:

...ectory/MSFT_xExchAutodiscoverVirtualDirectory.psm1
1.85% <0%> (ø)
:arrow_up:

...ExchOutlookAnywhere/MSFT_xExchOutlookAnywhere.psm1
1.61% <0%> (ø)
:arrow_up:

..._xExchAutoMountPoint/MSFT_xExchAutoMountPoint.psm1
0.86% <0%> (ø)
:arrow_up:

...MalwareScanning/MSFT_xExchAntiMalwareScanning.psm1
2.85% <0%> (ø)
:arrow_up:

... and 27 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c22d6c9...d231bf2. Read the comment docs.

Have this on my todo, I have prioritized other issues and PR's before this. I will try to get to this as soon as I can. 🙂

@johlju : Changes made, tested, and submitted. Let me know if anything else needs to be done. Thanks!


[DNM] txn: Print diagnostic info for lock acquiring and releasing (base)

Signed-off-by: MyonKeminta<EMAIL_ADDRESS>
What is changed and how it works?

Issue Number: Close #xxx
What's Changed:
Add diagnostic info to the merge base of PR https://github.com/tikv/tikv/pull/12749 .
The PR is submitted for requesting CI to build a binary. DNM.

Related changes

PR to update pingcap/docs/pingcap/docs-cn:
Need to cherry-pick to the release branch

Check List
Tests
Side effects

Performance regression

Consumes more CPU
Consumes more MEM

Breaking backward compatibility

Release note
None

/release

/release

/release


[feature request] latex labels for piloting and ODE solvers

Wow, that looks awesome.
This library makes me want to try ocaml  instead of my current julia workflow because I miss ML/OCaml/haskell.
I have a feature request :) It looks like ploting part cannot generate latex style labels and titles. It would be nice to have.
and I have a question. would owl arrays work with Sundials/ML http://inria-parkas.github.io/sundialsml/ ? if not, maybe there it would be nice to have a wraper of ODE solver in owl from gsl or sundials :)

Hi @AlexanderKoshkarov, thanks for trying out Owl.

I have a feature request :) It looks like ploting part cannot generate latex style labels and titles. It would be nice to have.

The plotting function is built atop of Plplot. Plplot supports postscript driver but I am not sure whether it supports latex math format actually. @hcarty knows better than I do, so maybe he can answer :)

and I have a question. would owl arrays work with Sundials/ML http://inria-parkas.github.io/sundialsml/ ? if not, maybe there it would be nice to have a wraper of ODE solver in owl from gsl or sundials :)

At the moment, there is no ODE module in Owl. I may consider to add one module in future just for convenience, but I don't intend to compete with other specialised libs. Sundialsml seems a good option for ODE. I tried a bit a moment ago, to answer your question.
For Sundials.RealArray2.t, it is very easy since it is just float64 Bigarray.Array2.t  You simply use wrap and unwrap functions in Sundials.RealArray2 as below
let x = Sundials.RealArray2.make 5 5 1.;;
let y = Sundials.RealArray2.unwrap x;;

let z = Owl.Mat.tile y [|2;2|];;
let z = Sundials.RealArray2.wrap z;;

Note wrap and unwrap doesn't create extra copies so very efficient.
For Sundials.RealArray.t, it is interesting because it is also just Bigarray.Array1.t so Owl should be able to handle it. But unlike Sundials.RealArray2, there is not wrap and unwrap function in this module and Sundials.RealArray.t seems wrapped into abstract type. So in order to pass it into Owl, you need to convert using float arrays (if you don't want to use obj (black) magic). E.g.,
let x = Sundials.RealArray.make 5 1.;;
let y = Mat.of_array (Sundials.RealArray.to_array x) 1 5;;
...

I think currently the best way is using Owl for more complicated matrix operations then using Sundials for ODE, these two should be able to work together without any problem imo.

Plplot does not have any special latex labeling support to my knowledge.

Hello @ryanrhymes and @AlexanderKoshkarov. This does look like a cool library. A few details about arrays and nvectors in Sundials/ML:

As you say, Sundials.RealArray2.unwrap is efficient (=fst), but Sundials.RealArray2.wrap has a cost as it must generate a column pointer array (https://github.com/inria-parkas/sundialsml/blob/2.7.0-release/src/sundials/sundials_ml.c#L110).
Sundials.RealArray.t is just a synonym for a BigArray (https://github.com/inria-parkas/sundialsml/blob/2.7.0-release/src/sundials/sundials.mli#L97), so no conversions are needed.
Nvectors also include wrap and unwrap functions from and to BigArrays (e.g., https://github.com/inria-parkas/sundialsml/blob/2.7.0-release/src/nvectors/nvector_serial.mli#L39). Again, unwrap is almost for free and wrap is more expensive.
In any case, you can continue to access an array after wrapping it. The underlying data is shared.
Sundials/ML also interfaces the Dls (dense and banded) Sls (sparse) matrix data structures sometimes used within Sundials. Interoperability with these may be trickier.

It should definitely be possible to use Owl in callbacks from Sundials/ML. Any suggestions are welcome. I don't think we have a lot of users, so it should still be possible to improve the API if necessary.

Just to add a little further confirmation to what hcarty wrote about LaTeX strings, I noticed that the documentation for plmtex or and plmtex3, two label-writing functions, doesn't mention LaTeX despite what's suggested by their names.  Just to be sure I tried inserting some LaTeX into a string passed to plmtex` and found, as expected, that it was printed as is.  "tex" in the names is apparently just an abbreviation for "text".


Bot c#: How to restart the conversation with the last input from user

In my Bot I use prompt.text and in the ResumeAfter method I want to check if user said yes, no or something else, and if it's not yes or no to call again my Root dialog(which is Luis class) with the input from the user. Here is the code:
PromptDialog.Text(context, ResumeAfterAsync, $"Was this helpful?");
private async Task ResumeAfterAsync(IDialogContext context, IAwaitable result)
{
string msg = await result;
if(msg=="yes")
{
await context.PostAsync($"Ok, thanks.");
}
else if(msg=="no")
{
await context.PostAsync($"another");
}
else
{
context.Wait(MessageReceived);  //-Here I need to send msg and to start again the conversation
}

You might consider using IDialogStack.Forward to forward the message on to the LUIS dialog as a new child (instead of the existing root):
https://docs.botframework.com/en-us/technical-faq/#what-is-idialogstack-forward-in-the-c-builder-sdk
But what are you going to do if the user said "yes" or "no"?  You have to go somewhere next:
https://docs.botframework.com/en-us/technical-faq/#ensure-all-dialog-methods-end-with-a-plan-to-handle-the-next-message
and would that place be the initial root LUIS dialog?

If user said "yes" or "no", I'm calling different methods, It's working well.
I've changed the else statment with this:
context.Forward(new MainDialog(),ResumeCopleted,msg,System.Threading.CancellationToken.None);
MainDialog is my LuisDIalog
And now in ResumeCopleted what should I do??
I've tried
private Task ResumeCopleted(IDialogContext context, IAwaitable result)
{
throw new NotImplementedException();
}
but that's not the right implementation

The IAwaitable in ResumeCopleted carries the result (of type T) from the MainDialog, so that you can post a message to the user or take some other action before resuming the wait for the next user message.

And It doesn't start the new search with cancer.I need to take that input and start the conversation with that. If I type again cancer then the result will be returned, but that's not my point.

Ok, thanks It's working now

Hi msBot1111
Could you please share code snippet how did u pass the user input back to Luis dialog.
Even I want map user prompt to one of the entity in the luis and set it to a variable.

how to achieve the same in v4