Bidirectional cxxbridge Issues

Apologies in advance if this is the wrong place to ask this, but anyway:
I have tried for a bit to use Corrosion to build and share code between a C++
project and a Rust project, starting with a simple example such as reading
images on the C++ side and sending the results back.
The CMake file is pretty straightforward:
# Rust <-> C++ integration.
find_package(corrosion)

if (corrosion_FOUND)
  corrosion_import_crate(MANIFEST_PATH ${PROJECT_SOURCE_DIR}/ext/gfx-rs/Cargo.toml)
  corrosion_add_cxxbridge(imageio-cxx
    CRATE gfx
    MANIFEST_PATH ${PROJECT_SOURCE_DIR}/ext/gfx-rs
    FILES imageio_cxx.rs)
  corrosion_link_libraries(gfx ccl_common)
endif()

The C++ bridge looks something like this:
#include "gfx/src/imageio_cxx.rs.h"
#include "rust/cxx.h"

struct Rgba;
struct RsImage;

RsImage read_image(const rust::Str path);
void write_image(const rust::Str path, const RsImage &image);

and the Rust side of the same looks like this:
#[cxx::bridge]
pub mod ffi
{
    #[derive(Debug)]
    struct Rgba
    {
        r: f32,
        g: f32,
        b: f32,
        a: f32,
    }

    #[derive(Debug)]
    struct RsImage
    {
        width: usize,
        height: usize,
        raster: Vec<Rgba>,
    }
    unsafe extern "C++"
    {
        include!("gfx/external/imageio-bridge.h");
        fn read_image(path: &str) -> Result<RsImage>;
        fn write_image(path: &str, image: &RsImage) -> Result<()>;
    }
}

The build process works just fine, but any rust binary that attempts to use the
bridge triggers a linker error:
  = note: Undefined symbols for architecture arm64:
            "_cxxbridge1$read_image", referenced from:
                gfx::imageio_cxx::ffi::read_image::h5122f98c3963e4d5 in libgfx.rlib(gfx.gfx.58b6ab36-cgu.5.rcgu.o)
          ld: symbol(s) not found for architecture arm64
          clang: error: linker command failed with exit code 1 (use -v to see invocation)

Some of the examples I found in the repository suggested that I should construct
the bridge on the CMake side (e.g., the test/cxxbridge), so I tried to add the
following lines to the CMakeLists.txt:
  add_library(imageio-bridge ${PROJECT_SOURCE_DIR}/ext/gfx-rs/external/imageio-bridge.cpp)
  target_link_libraries(imageio-bridge PUBLIC imageio-cxx)
  corrosion_link_libraries(gfx imageio-cxx)

This however seems to trigger dependency graph cycles in CMake:
CMake Error: The inter-target dependency graph contains the following strongly connected component (cycle):
  "_cargo-build_gfx" of type UTILITY
    depends on "imageio-cxx" (strong)
  "cargo-build_gfx" of type UTILITY
    depends on "_cargo-build_gfx" (strong)
  "imageio-cxx" of type STATIC_LIBRARY
    depends on "cargo-build_gfx" (strong)
    depends on "_cargo-build_gfx" (strong)

So, I'm currently at a bit of loss, what is the recommended way for constructing
these kinds of bridges?
If nothing else, perhaps a slightly larger example with a bidirectional
cxxbridge is warranted?

I'm not too familiar with cxx, so take everything I say with a grain of salt.

This however seems to trigger dependency graph cycles in CMake:

Looking at your example, it seems this is because you do have a cycle in your headers. The rust bridge depends on the C++ header and the C++ header depends on the rust bridge.
Could you split your C++ header into two files, so that you have a clean split between what functions / types C++ exports and the generated header from the Rust side?

Looking at your example, it seems this is because you do have a cycle in your headers. The rust bridge depends on the > C++ header and the C++ header depends on the rust bridge.
Could you split your C++ header into two files, so that you have a clean split between what functions / types C++ > exports and the generated header from the Rust side?

Hmm. It is possible to remove the bridge-include in the header, e.g.:
#ifndef EXTERNAL_IMAGEIO_BRIDGE_H
#define EXTERNAL_IMAGEIO_BRIDGE_H

#include "rust/cxx.h"

struct Rgba;
struct RsImage;

RsImage read_image(const rust::Str path);
void write_image(const rust::Str path, const RsImage &image);

#endif /* EXTERNAL_IMAGEIO_BRIDGE_H */

But this didn't change anything though. Or am I misunderstanding your suggestion?

But this didn't change anything though. Or am I misunderstanding your suggestion?

Well, once you removed the cyclic depenency in the headers you would also have to adapt the CMake dependencies

corrosion_link_libraries(gfx imageio-cxx)

You probably want to remove this line - I would expect you would want to link the rust bindings into a C++ library and not into the rust crate.

Appologies for the slow replies, ended up being rather busy during the holidays, forgetting this thread. Anyways:

You probably want to remove this line - I would expect you would want to link the rust bindings into a C++ library and > not into the rust crate.

Removing that line does seem to work to fix the CMake dependency cycle, but it unfortunately comes back to the other problem of missing the read_image() symbols:
  = note: Undefined symbols for architecture arm64:
            "_cxxbridge1$read_image", referenced from:
                gfx::imageio_cxx::ffi::read_image::h5122f98c3963e4d5 in libgfx.rlib(gfx.gfx.58b6ab36-cgu.5.rcgu.o)
          ld: symbol(s) not found for architecture arm64
          clang: error: linker command failed with exit code 1 (use -v to see invocation)

I guess there's something missing on the Rust side then to make these symbols (that are compiled on the CMake side) available?

In which language was the final executable defined again? Rust or C++?

In this case you might be missing a corrosion_link_libraries(texture_test <cxx_lib_containing_read_image_definition>) to tell the linker to include the C++ library containing read_image into the executable.

Although, looking at this it looks a little weird, I don't really see why the C++ compiler is invoked here

It is quite common that the linker is not invoked directly, but via the compiler. Corrosion by default sets the linker to be the C++ compiler (if C++ is compiled) to ensure that the default C++ libraries are linked. On the most recent master there is a new option NO_LINKER_OVERRIDE to corrosion_import_crate() which prevents this, but then you would have to manually specify all default C++  libraries (via corrosion_link_libraries().

Ah, I just realized that this is probably where the circular dependency issue comes up. Could you checkout the bridge branch (PR https://github.com/corrosion-rs/corrosion/pull/293) and see if the circular dependency issue still exists (when using corrosion_link_libraries?
I'm not super familiar with cxx, so it could be that I'm also overlooking something.

Actually, it seems like limiting the dependencies to just the targets that need it seems to work, although one could argue that it is a bit of a workaround. That said, the build went a lot further this time, but fails when it tries to find the cxxbridge generated headers:
[  1%] Building CXX object CMakeFiles/imageio-cxx.dir/corrosion_generated/cxxbridge/imageio-cxx/src/imageio_cxx.cpp.o
/Library/Developer/CommandLineTools/usr/bin/c++ -DHAVE_LIBPNG=1 -DHAVE_OPENEXR=315 -DHAVE_ZLIB=1 -I/opt/homebrew/Cellar/openexr/3.1.5/include -I/opt/homebrew/Cellar/openexr/3.1.5/include/OpenEXR -I/opt/homebrew/Cellar/imath/3.1.6/include -I/opt/homebrew/Cellar/imath/3.1.6/include/Imath -I/opt/homebrew/Cellar/libpng/1.6.39/include/libpng16 -I/Users/gustaf/Documents/git/phd/ocl-spikes/build/corrosion_generated/cxxbridge/imageio-cxx/include -Wall -Wpedantic -Wno-deprecated -Werror -pthread -Wno-unknown-pragmas -Wno-unused-private-field -arch arm64 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX12.3.sdk -std=gnu++17 -MD -MT CMakeFiles/imageio-cxx.dir/corrosion_generated/cxxbridge/imageio-cxx/src/imageio_cxx.cpp.o -MF CMakeFiles/imageio-cxx.dir/corrosion_generated/cxxbridge/imageio-cxx/src/imageio_cxx.cpp.o.d -o CMakeFiles/imageio-cxx.dir/corrosion_generated/cxxbridge/imageio-cxx/src/imageio_cxx.cpp.o -c /Users/gustaf/Documents/git/phd/ocl-spikes/build/corrosion_generated/cxxbridge/imageio-cxx/src/imageio_cxx.cpp
/Users/gustaf/Documents/git/phd/ocl-spikes/build/corrosion_generated/cxxbridge/imageio-cxx/src/imageio_cxx.cpp:1:10: fatal error: 'gfx/external/imageio-bridge.h' file not found
#include "gfx/external/imageio-bridge.h"
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
make[2]: *** [CMakeFiles/imageio-cxx.dir/corrosion_generated/cxxbridge/imageio-cxx/src/imageio_cxx.cpp.o] Error 1
make[1]: *** [CMakeFiles/imageio-cxx.dir/all] Error 2
make: *** [all] Error 2

Which is interesting, and probably easy to fix: Does corrosion place these headers somewhere specific?

Ah, I just realized that this is probably where the circular dependency issue comes up. Could you checkout the bridge  branch (PR https://github.com/corrosion-rs/corrosion/pull/293) and see if the circular dependency issue still exists (when using corrosion_link_libraries? I'm not super familiar with cxx, so it could be that I'm also overlooking something.

Sure, I'll give that a branch a spin tomorrow when I got some more time!

Yes, the header is placed under ${CMAKE_CURRENT_BINARY_DIR}/corrosion_generated/cxxbridge/${cxx_target}/include and this path is added to the include directories of cxx_target  (imageio-cxx in your case).

Seems like using the line corrosion_link_libraries(gfx imageio-cxx) still gives a circular dependency even on the bridge branch, example output below:
CMake Error: The inter-target dependency graph contains the following strongly connected component (cycle):
  "_cargo-build_gfx" of type UTILITY
    depends on "imageio-cxx" (strong)
  "cargo-build_gfx" of type UTILITY
    depends on "_cargo-build_gfx" (strong)
  "imageio-cxx" of type STATIC_LIBRARY
    depends on "cargo-build_gfx" (strong)
At least one of these targets is not a STATIC_LIBRARY.  Cyclic dependencies are allowed only among static libraries.
CMake Generate step failed.  Build files cannot be regenerated correctly.

Yes, the header is placed under ${CMAKE_CURRENT_BINARY_DIR}/corrosion_generated/cxxbridge/${cxx_target}/include and this path is added to the include directories of cxx_target (imageio-cxx in your case).

Hmm, odd. My paths are slightly different:
gustaf@ip81-1 build % find corrosion_generated/cxxbridge/imageio-cxx/include
corrosion_generated/cxxbridge/imageio-cxx/include
corrosion_generated/cxxbridge/imageio-cxx/include/imageio-cxx
corrosion_generated/cxxbridge/imageio-cxx/include/imageio-cxx/imageio_cxx.h

There seems to be a second ${cxx_target} after the include. Is that intentional?
After hacking at it awhile, I the Rust side of the bridge to this:
#[cxx::bridge]
pub mod ffi
{
    #[derive(Debug)]
    struct Rgba
    {
        r: f32,
        g: f32,
        b: f32,
        a: f32,
    }

    #[derive(Debug)]
    struct RsImage
    {
        width: usize,
        height: usize,
        raster: Vec<Rgba>,
    }

    unsafe extern "C++"
    {
        include!("imageio-bridge.h");
        fn read_image(path: &str) -> Result<RsImage>;
        fn write_image(path: &str, image: &RsImage) -> Result<()>;
    }
}

And forced the include directory corrosion_generated/cxxbridge/imageio-cxx to be available in the cmake-file:
  corrosion_link_libraries(texture_test imageio-cxx)
  target_include_directories(imageio-cxx PUBLIC ${PROJECT_SOURCE_DIR}/ext/gfx-rs/external/)

This got me a tiny bit further. Does corrosion keep the cxxbridge header utilities somewhere as well? Since now I'm missing these headers:
[  2%] Building CXX object CMakeFiles/imageio-cxx.dir/corrosion_generated/cxxbridge/imageio-cxx/src/imageio_cxx.cpp.o
/Library/Developer/CommandLineTools/usr/bin/c++ -DHAVE_LIBPNG=1 -DHAVE_OPENEXR=315 -DHAVE_ZLIB=1 -I/opt/homebrew/Cellar/openexr/3.1.5/include -I/opt/homebrew/Cellar/openexr/3.1.5/include/OpenEXR -I/opt/homebrew/Cellar/imath/3.1.6/include -I/opt/homebrew/Cellar/imath/3.1.6/include/Imath -I/opt/homebrew/Cellar/libpng/1.6.39/include/libpng16 -I/Users/gustaf/Documents/git/phd/ocl-spikes/build/corrosion_generated/cxxbridge/imageio-cxx/include -I/Users/gustaf/Documents/git/phd/ocl-spikes/ext/gfx-rs/external -Wall -Wpedantic -Wno-deprecated -Werror -pthread -Wno-unknown-pragmas -Wno-unused-private-field -arch arm64 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX12.3.sdk -std=gnu++17 -MD -MT CMakeFiles/imageio-cxx.dir/corrosion_generated/cxxbridge/imageio-cxx/src/imageio_cxx.cpp.o -MF CMakeFiles/imageio-cxx.dir/corrosion_generated/cxxbridge/imageio-cxx/src/imageio_cxx.cpp.o.d -o CMakeFiles/imageio-cxx.dir/corrosion_generated/cxxbridge/imageio-cxx/src/imageio_cxx.cpp.o -c /Users/gustaf/Documents/git/phd/ocl-spikes/build/corrosion_generated/cxxbridge/imageio-cxx/src/imageio_cxx.cpp
In file included from /Users/gustaf/Documents/git/phd/ocl-spikes/build/corrosion_generated/cxxbridge/imageio-cxx/src/imageio_cxx.cpp:1:
/Users/gustaf/Documents/git/phd/ocl-spikes/ext/gfx-rs/external/imageio-bridge.h:4:10: fatal error: 'rust/cxx.h' file not found
#include "rust/cxx.h"
         ^~~~~~~~~~~~
1 error generated.
make[2]: *** [CMakeFiles/imageio-cxx.dir/corrosion_generated/cxxbridge/imageio-cxx/src/imageio_cxx.cpp.o] Error 1
make[1]: *** [CMakeFiles/imageio-cxx.dir/all] Error 2
make: *** [all] Error 2

Which I belive are the headers to the helper structs that cxxbridge uses

Seems like using the line corrosion_link_libraries(gfx imageio-cxx) still gives a circular dependency even on the bridge branch, example output below:

In this case you might be missing a corrosion_link_libraries(texture_test <cxx_lib_containing_read_image_definition>)

Did this one not work? I think you would want to link the imageio-cxx crate into your final executable, not an intermediate library.

There seems to be a second ${cxx_target} after the include. Is that intentional?

Yes, that seems to be intentional. I don't remember why though - I'd have to look into that. Probably simply to add a namespace to your includes, so you do `#include "cxx_target/blah.h" to make conflicts less likely.

This got me a tiny bit further. Does corrosion keep the cxxbridge header utilities somewhere as well? Since now I'm missing these headers:

AFAIK this extra utliity header does not exist when using the commandline cxxbridge tool. Instead the definitions will be inside your header in the cxx_target directory.

I've added a simple test based on what you wrote, and it seems to be compiling now.
You can have a look here: test/cxxbridge/cxxbridge_rust2cpp/CMakeLists.txt
One thing that I haven't figured out yet is how to get a non-absolute path working with the include!() macro in the cxx bridge, I'll investigate that if I have some more time.

Did this one not work? I think you would want to link the imageio-cxx crate into your final executable, not an intermediate library.

Oh, I think I misunderstood your previous comment: I figured you wanted me to try to use the line corrosion_link_libraries(gfx imageio-cxx) on the bridges branch to see if that (still) caused the same circular dependency. Using corrosion_link_libraries(test_textures imageio-cxx) works on both the bridges and the master/main branch.
That said however, I must say that this feels like a bit of a band-aid rather than a proper fix. At least in my case, the imageio-cxx is technically a dependency on the library and not of the individual (test) binaries. Although, I have no idea if that is feasible to fix. Perhaps it's possible to decouple the gfx (Rust) library from the build process? (The _cargo-build_gfx target I guess?)

Yes, that seems to be intentional. I don't remember why though - I'd have to look into that. Probably simply to add a namespace to your includes, so you do `#include "cxx_target/blah.h" to make conflicts less likely.

Makes perfect sense to me, just wanted to be sure :)

AFAIK this extra utliity header does not exist when using the commandline cxxbridge tool. Instead the definitions will be inside your header in the cxx_target directory.

Ah, I see, I must have missed that note in the cxxbridge documentation, and indeed changing that takes me a bit further still, but now I get a bunch of pretty concerning looking errors:
[  1%] Building CXX object CMakeFiles/imageio-cxx.dir/corrosion_generated/cxxbridge/imageio-cxx/src/imageio_cxx.cpp.o
/Users/gustaf/Documents/git/phd/ocl-spikes/build/corrosion_generated/cxxbridge/imageio-cxx/src/imageio_cxx.cpp:710:23: error: '$' in identifier [-Werror,-Wdollar-in-identifier-extension]
const char *cxxbridge1$exception(const char *, ::std::size_t);
                      ^
/Users/gustaf/Documents/git/phd/ocl-spikes/build/corrosion_generated/cxxbridge/imageio-cxx/src/imageio_cxx.cpp:740:32: error: '$' in identifier [-Werror,-Wdollar-in-identifier-extension]
::rust::repr::PtrLen cxxbridge1$read_image(::rust::Str path, ::RsImage *return$) noexcept {
                               ^
/Users/gustaf/Documents/git/phd/ocl-spikes/build/corrosion_generated/cxxbridge/imageio-cxx/src/imageio_cxx.cpp:740:79: error: '$' in identifier [-Werror,-Wdollar-in-identifier-extension]
::rust::repr::PtrLen cxxbridge1$read_image(::rust::Str path, ::RsImage *return$) noexcept {
                                                                              ^
/Users/gustaf/Documents/git/phd/ocl-spikes/build/corrosion_generated/cxxbridge/imageio-cxx/src/imageio_cxx.cpp:741:25: error: '$' in identifier [-Werror,-Wdollar-in-identifier-extension]
  ::RsImage (*read_image$)(::rust::Str) = ::read_image;
                        ^
/Users/gustaf/Documents/git/phd/ocl-spikes/build/corrosion_generated/cxxbridge/imageio-cxx/src/imageio_cxx.cpp:742:29: error: '$' in identifier [-Werror,-Wdollar-in-identifier-extension]
  ::rust::repr::PtrLen throw$;
                            ^
/Users/gustaf/Documents/git/phd/ocl-spikes/build/corrosion_generated/cxxbridge/imageio-cxx/src/imageio_cxx.cpp:745:20: error: '$' in identifier [-Werror,-Wdollar-in-identifier-extension]
        new (return$) ::RsImage(read_image$(path));
                   ^
/Users/gustaf/Documents/git/phd/ocl-spikes/build/corrosion_generated/cxxbridge/imageio-cxx/src/imageio_cxx.cpp:745:43: error: '$' in identifier [-Werror,-Wdollar-in-identifier-extension]
        new (return$) ::RsImage(read_image$(path));
                                          ^
/Users/gustaf/Documents/git/phd/ocl-spikes/build/corrosion_generated/cxxbridge/imageio-cxx/src/imageio_cxx.cpp:746:14: error: '$' in identifier [-Werror,-Wdollar-in-identifier-extension]
        throw$.ptr = nullptr;
             ^
/Users/gustaf/Documents/git/phd/ocl-spikes/build/corrosion_generated/cxxbridge/imageio-cxx/src/imageio_cxx.cpp:748:28: error: '$' in identifier [-Werror,-Wdollar-in-identifier-extension]
      [&](const char *catch$) noexcept {
                           ^
/Users/gustaf/Documents/git/phd/ocl-spikes/build/corrosion_generated/cxxbridge/imageio-cxx/src/imageio_cxx.cpp:749:14: error: '$' in identifier [-Werror,-Wdollar-in-identifier-extension]
        throw$.len = ::std::strlen(catch$);
             ^
/Users/gustaf/Documents/git/phd/ocl-spikes/build/corrosion_generated/cxxbridge/imageio-cxx/src/imageio_cxx.cpp:749:41: error: '$' in identifier [-Werror,-Wdollar-in-identifier-extension]
        throw$.len = ::std::strlen(catch$);
                                        ^
/Users/gustaf/Documents/git/phd/ocl-spikes/build/corrosion_generated/cxxbridge/imageio-cxx/src/imageio_cxx.cpp:750:14: error: '$' in identifier [-Werror,-Wdollar-in-identifier-extension]
        throw$.ptr = const_cast<char *>(::cxxbridge1$exception(catch$, throw$.len));
             ^
/Users/gustaf/Documents/git/phd/ocl-spikes/build/corrosion_generated/cxxbridge/imageio-cxx/src/imageio_cxx.cpp:750:53: error: '$' in identifier [-Werror,-Wdollar-in-identifier-extension]
        throw$.ptr = const_cast<char *>(::cxxbridge1$exception(catch$, throw$.len));
                                                    ^
/Users/gustaf/Documents/git/phd/ocl-spikes/build/corrosion_generated/cxxbridge/imageio-cxx/src/imageio_cxx.cpp:750:69: error: '$' in identifier [-Werror,-Wdollar-in-identifier-extension]
        throw$.ptr = const_cast<char *>(::cxxbridge1$exception(catch$, throw$.len));
                                                                    ^
/Users/gustaf/Documents/git/phd/ocl-spikes/build/corrosion_generated/cxxbridge/imageio-cxx/src/imageio_cxx.cpp:750:77: error: '$' in identifier [-Werror,-Wdollar-in-identifier-extension]
        throw$.ptr = const_cast<char *>(::cxxbridge1$exception(catch$, throw$.len));
                                                                            ^
/Users/gustaf/Documents/git/phd/ocl-spikes/build/corrosion_generated/cxxbridge/imageio-cxx/src/imageio_cxx.cpp:752:15: error: '$' in identifier [-Werror,-Wdollar-in-identifier-extension]
  return throw$;
              ^
/Users/gustaf/Documents/git/phd/ocl-spikes/build/corrosion_generated/cxxbridge/imageio-cxx/src/imageio_cxx.cpp:755:32: error: '$' in identifier [-Werror,-Wdollar-in-identifier-extension]
::rust::repr::PtrLen cxxbridge1$write_image(::rust::Str path, const ::RsImage &image) noexcept {
                               ^
/Users/gustaf/Documents/git/phd/ocl-spikes/build/corrosion_generated/cxxbridge/imageio-cxx/src/imageio_cxx.cpp:756:21: error: '$' in identifier [-Werror,-Wdollar-in-identifier-extension]
  void (*write_image$)(::rust::Str, const ::RsImage &) = ::write_image;
                    ^
/Users/gustaf/Documents/git/phd/ocl-spikes/build/corrosion_generated/cxxbridge/imageio-cxx/src/imageio_cxx.cpp:757:29: error: '$' in identifier [-Werror,-Wdollar-in-identifier-extension]
  ::rust::repr::PtrLen throw$;
                            ^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
20 errors generated.
make[2]: *** [CMakeFiles/imageio-cxx.dir/corrosion_generated/cxxbridge/imageio-cxx/src/imageio_cxx.cpp.o] Error 1
make[1]: *** [CMakeFiles/imageio-cxx.dir/all] Error 2
make: *** [all] Error 2

Have you seen any of these before?

One thing that I haven't figured out yet is how to get a non-absolute path working with the include!() macro in the cxx bridge, I'll investigate that if I have some more time.

That's interesting, I think I got that to work using the line:
  target_include_directories(imageio-cxx
    PUBLIC
    ${PROJECT_SOURCE_DIR}/ext/gfx-rs/external/
    ${PROJECT_BINARY_DIR}/corrosion_generated/cxxbridge/imageio-cxx/include)

(The include!() header file is located in ${PROJECT_SOURCE_DIR}/ext/gfx-rs/external/).

I've added a simple test based on what you wrote, and it seems to be compiling now.  You can have a look here: test/cxxbridge/cxxbridge_rust2cpp/CMakeLists.txt

This looks good to me at least, perhaps the include statement I mentioned above should be included? Or perhaps that should be done as a part of the corrosion_add_cxxbridge() cmake function?
On a separate note, I also noticed that the imageio-cxx target is not properly regenerated when imageio_cxx.rs is edited. Accordingly, I need to remove the old config (rm -rf *) and regenerate it anew for changes to come into effect. Is there perhaps some dependency missing here that prevents a proper update?

As I was unable to progress with the previous approach, I tried a different tack
that I had been thinking of for a while: Using features and environment
variables to propagate any necessary information. This way I was actually able
to get something running.
First, on the CMake side, I only import the crate and set the environment
variables as necessary:
# Rust <-> C++ integration.
find_package(corrosion)
if (corrosion_FOUND)
  set(CORROSION_VERBOSE_OUTPUT ON)
  corrosion_import_crate(
    MANIFEST_PATH ${PROJECT_SOURCE_DIR}/ext/gfx-rs/Cargo.toml
    FEATURES corrosion)

  # corrosion_link_libraries(gfx ccl_common)

  # Convert semicolons to colons to prevent CMake from converting it into lists.
  string(REPLACE ";" ":" link_libs "ccl_common;${OpenEXR_LIBRARIES}")
  string(REPLACE ";" ":" link_search "${PROJECT_BINARY_DIR}/src;${OpenEXR_LIBRARY_DIRS}")
  corrosion_set_env_vars(gfx
    CARGO_FEATURE_CORROSION_INCLUDES=${PROJECT_SOURCE_DIR}/src
    CARGO_FEATURE_CORROSION_LINK_LIBS=${link_libs}
    CARGO_FEATURE_CORROSION_LINK_SEARCH=${link_search})
endif()

Annoyingly, I also need to remove all semicolons or translate them to something
else (such as colons in this case) to be able to set multiple libraries in these
variables. Otherwise, it seems like CMake expands the "lists" to strings using
spaces, which causes all manner of havoc later on. Maybe there's a better way to do this?
Further, in the Rust build script I then extract this information from
the same environment variables:
    if let Ok(_s) = std::env::var("CARGO_FEATURE_CORROSION")
    {
        let includes = std::env::var("CARGO_FEATURE_CORROSION_INCLUDES").unwrap_or("".to_string());
        let link_libs = std::env::var("CARGO_FEATURE_CORROSION_LINK_LIBS").unwrap_or("".to_string());
        let link_search = std::env::var("CARGO_FEATURE_CORROSION_LINK_SEARCH").unwrap_or("".to_string());

        let pat = |sym|{sym == ';' || sym == ':' || sym == ' '};
        for lib in link_libs.split_terminator(pat)
        {
            println!("cargo:rustc-link-lib={}", lib);
        }
        for path in link_search.split_terminator(pat)
        {
            println!("cargo:rustc-link-search={}", path);
        }
        let ccl_includes: Vec<_> = includes.split_terminator(pat).collect();

        // Attempt to build the C++-bridge.
        cxx_build::bridge("src/imageio_cxx.rs")
            .file("external/imageio-bridge.cpp")
            .includes(&ccl_includes)
            .flag_if_supported("-std=c++17")
            .flag("-Wno-deprecated")
            .compile("libimageio-bridge");
        println!("cargo:rerun-if-changed=external/imageio-bridge.h");
        println!("cargo:rerun-if-changed=external/imageio-bridge.cpp");
    }

However, while this works, I'm a bit concerned with the scalability of this
solution since I have to manually specify all dependencies. I was hoping that I
would only need to specify ccl_common as a requirement using
corrosion_link_libraries(gfx ccl_common), and that works for symbols exclusive
to the ccl_common library, but not for any of the dependencies. I.e., in the
above example ccl_common depends on OpenEXR, but the libraries and search
paths are not included for it. Is that intentional or would that be considered a
bug?

Otherwise, it seems like CMake expands the "lists" to strings using spaces, which causes all manner of havoc later on.

Well, in CMake a list is just a semicolon delimited string, so you have to pay attention and quote the list variable basically each time you use it to prevent expansion. Possibly quoting the arguments to corrosion_set_env_vars would already be sufficient, but it's also possible that the COMMAND_EXPAND_LISTS argument of the custom command which calls cargo build would anyway expand the list without a way to prevent it. I don't quite remember the interaction here.
While working on #293 I did realize that the add_cxxbridge function currently only works properly when linking Rust into C/C++, but not (yet) the other way around in all cases. I think for your particular case it should not be that hard to fix though. Could you have a look at #293, especially at the newly added testcase and improve the C++ / rust code to something that corresponds somewhat to your failing project? I could then take another look with a bit more realistic of a test project.

Well, in CMake a list is just a semicolon delimited string, so you have to pay attention and quote the list variable basically each time you use it to prevent expansion. Possibly quoting the arguments to corrosion_set_env_vars would already be sufficient, but it's also possible that the COMMAND_EXPAND_LISTS argument of the custom command which calls cargo build would anyway expand the list without a way to prevent it. I don't quite remember the interaction here.

Hmm, that's probably another bug then, here's the behavior between quoting the variables vs not:
[ 22%] Built target cargo-prebuild_gfx
/Library/Developer/CommandLineTools/usr/bin/make  -f CMakeFiles/_cargo-build_gfx.dir/build.make CMakeFiles/_cargo-build_gfx.dir/depend
cd /Users/gustaf/Documents/git/phd/ocl-spikes/build && /opt/homebrew/Cellar/cmake/3.25.1/bin/cmake -E cmake_depends "Unix Makefiles" /Users/gustaf/Documents/git/phd/ocl-spikes /Users/gustaf/Documents/git/phd/ocl-spikes /Users/gustaf/Documents/git/phd/ocl-spikes/build /Users/gustaf/Documents/git/phd/ocl-spikes/build /Users/gustaf/Documents/git/phd/ocl-spikes/build/CMakeFiles/_cargo-build_gfx.dir/DependInfo.cmake --color=
/Library/Developer/CommandLineTools/usr/bin/make  -f CMakeFiles/_cargo-build_gfx.dir/build.make CMakeFiles/_cargo-build_gfx.dir/build
cd /Users/gustaf/Documents/git/phd/ocl-spikes/ext/gfx-rs && /opt/homebrew/Cellar/cmake/3.25.1/bin/cmake -E env CARGO_FEATURE_CORROSION_INCLUDES=/Users/gustaf/Documents/git/phd/ocl-spikes/src CARGO_FEATURE_CORROSION_LINK_LIBS=ccl_common OpenEXR-3_1 OpenEXRUtil-3_1 OpenEXRCore-3_1 Iex-3_1 IlmThread-3_1 Imath-3_1 CARGO_FEATURE_CORROSION_LINK_SEARCH=/Users/gustaf/Documents/git/phd/ocl-spikes/build/src /opt/homebrew/Cellar/openexr/3.1.5/lib /opt/homebrew/Cellar/imath/3.1.6/lib CARGO_TARGET_AARCH64_APPLE_DARWIN_LINKER=/Library/Developer/CommandLineTools/usr/bin/c++ CC_aarch64-apple-darwin=/Library/Developer/CommandLineTools/usr/bin/cc CXX_aarch64-apple-darwin=/Library/Developer/CommandLineTools/usr/bin/c++ SDKROOT=/Library/Developer/CommandLineTools/SDKs/MacOSX12.3.sdk LIBRARY_PATH=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/lib CORROSION_BUILD_DIR=/Users/gustaf/Documents/git/phd/ocl-spikes/build CARGO_BUILD_RUSTC=/Users/gustaf/.rustup/toolchains/nightly-aarch64-apple-darwin/bin/rustc /Users/gustaf/.rustup/toolchains/nightly-aarch64-apple-darwin/bin/cargo rustc --lib --target=aarch64-apple-darwin --features corrosion --package gfx --manifest-path /Users/gustaf/Documents/git/phd/ocl-spikes/ext/gfx-rs/Cargo.toml --target-dir /Users/gustaf/Documents/git/phd/ocl-spikes/build/./cargo/build -- -Cdefault-linker-libraries=yes -L/Users/gustaf/Documents/git/phd/ocl-spikes/build/src -lccl_common
No such file or directory
make[2]: *** [CMakeFiles/_cargo-build_gfx] Error 1
make[1]: *** [CMakeFiles/_cargo-build_gfx.dir/all] Error 2
make: *** [all] Error 2

Versus:
[ 22%] Built target cargo-prebuild_gfx
/Library/Developer/CommandLineTools/usr/bin/make  -f CMakeFiles/_cargo-build_gfx.dir/build.make CMakeFiles/_cargo-build_gfx.dir/depend
cd /Users/gustaf/Documents/git/phd/ocl-spikes/build && /opt/homebrew/Cellar/cmake/3.25.1/bin/cmake -E cmake_depends "Unix Makefiles" /Users/gustaf/Documents/git/phd/ocl-spikes /Users/gustaf/Documents/git/phd/ocl-spikes /Users/gustaf/Documents/git/phd/ocl-spikes/build /Users/gustaf/Documents/git/phd/ocl-spikes/build /Users/gustaf/Documents/git/phd/ocl-spikes/build/CMakeFiles/_cargo-build_gfx.dir/DependInfo.cmake --color=
/Library/Developer/CommandLineTools/usr/bin/make  -f CMakeFiles/_cargo-build_gfx.dir/build.make CMakeFiles/_cargo-build_gfx.dir/build
cd /Users/gustaf/Documents/git/phd/ocl-spikes/ext/gfx-rs && /opt/homebrew/Cellar/cmake/3.25.1/bin/cmake -E env CARGO_FEATURE_CORROSION_INCLUDES=\"/Users/gustaf/Documents/git/phd/ocl-spikes/src\" CARGO_FEATURE_CORROSION_LINK_LIBS=\"ccl_common OpenEXR-3_1 OpenEXRUtil-3_1 OpenEXRCore-3_1 Iex-3_1 IlmThread-3_1 Imath-3_1\" CARGO_FEATURE_CORROSION_LINK_SEARCH=\"/Users/gustaf/Documents/git/phd/ocl-spikes/build/src /opt/homebrew/Cellar/openexr/3.1.5/lib /opt/homebrew/Cellar/imath/3.1.6/lib\" CARGO_TARGET_AARCH64_APPLE_DARWIN_LINKER=/Library/Developer/CommandLineTools/usr/bin/c++ CC_aarch64-apple-darwin=/Library/Developer/CommandLineTools/usr/bin/cc CXX_aarch64-apple-darwin=/Library/Developer/CommandLineTools/usr/bin/c++ SDKROOT=/Library/Developer/CommandLineTools/SDKs/MacOSX12.3.sdk LIBRARY_PATH=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/lib CORROSION_BUILD_DIR=/Users/gustaf/Documents/git/phd/ocl-spikes/build CARGO_BUILD_RUSTC=/Users/gustaf/.rustup/toolchains/nightly-aarch64-apple-darwin/bin/rustc /Users/gustaf/.rustup/toolchains/nightly-aarch64-apple-darwin/bin/cargo rustc --lib --target=aarch64-apple-darwin --features corrosion --package gfx --manifest-path /Users/gustaf/Documents/git/phd/ocl-spikes/ext/gfx-rs/Cargo.toml --target-dir /Users/gustaf/Documents/git/phd/ocl-spikes/build/./cargo/build -- -Cdefault-linker-libraries=yes -L/Users/gustaf/Documents/git/phd/ocl-spikes/build/src -lccl_common
No such file or directory
make[2]: *** [CMakeFiles/_cargo-build_gfx] Error 1
make[1]: *** [CMakeFiles/_cargo-build_gfx.dir/all] Error 2
make: *** [all] Error 2

It seems like the quotes become "part" of the first and last element somehow, not sure how that works.
Here's the CMake changes from above, just in case:
  # Convert semicolons to colons to prevent CMake from converting it into lists.
  string(REPLACE ";" ";" link_libs "ccl_common;${OpenEXR_LIBRARIES}")
  string(REPLACE ";" ";" link_search "${PROJECT_BINARY_DIR}/src;${OpenEXR_LIBRARY_DIRS}")
  corrosion_set_env_vars(gfx
    CARGO_FEATURE_CORROSION_INCLUDES="${PROJECT_SOURCE_DIR}/src"
    CARGO_FEATURE_CORROSION_LINK_LIBS="${link_libs}"
    CARGO_FEATURE_CORROSION_LINK_SEARCH="${link_search}")

While working on https://github.com/corrosion-rs/corrosion/pull/293 I did realize that the add_cxxbridge function currently only works properly when linking Rust into C/C++, but not (yet) the other way around in all cases. I think for your particular case it should not be that hard to fix though. Could you have a look at https://github.com/corrosion-rs/corrosion/pull/293, especially at the newly added testcase and improve the C++ / rust code to something that corresponds somewhat to your failing project? I could then take another look with a bit more realistic of a test project.

Sure, I'll try to squeeze that in soon-ish

It seems like the quotes become "part" of the first and last element somehow, not sure how that works.

Yeah, you need to quote the whole thing, otherwise CMake will keep the quotes.

corrosion_set_env_vars(gfx
"CARGO_FEATURE_CORROSION_INCLUDES=${PROJECT_SOURCE_DIR}/src"
"CARGO_FEATURE_CORROSION_LINK_LIBS=${link_libs}"
"CARGO_FEATURE_CORROSION_LINK_SEARCH=${link_search}")

Hmm, strange, that didn't actually change anything:
cd /Users/gustaf/Documents/git/phd/ocl-spikes/ext/gfx-rs && /opt/homebrew/Cellar/cmake/3.25.1/bin/cmake -E env \
CARGO_FEATURE_CORROSION_INCLUDES=/Users/gustaf/Documents/git/phd/ocl-spikes/src \
CARGO_FEATURE_CORROSION_LINK_LIBS=ccl_common OpenEXR-3_1 OpenEXRUtil-3_1 OpenEXRCore-3_1 Iex-3_1 IlmThread-3_1 Imath-3_1 \
CARGO_FEATURE_CORROSION_LINK_SEARCH=/Users/gustaf/Documents/git/phd/ocl-spikes/build/src /opt/homebrew/Cellar/openexr/3.1.5/lib /opt/homebrew/Cellar/imath/3.1.6/lib

Just for completeness I also tried to escape-quote the variable values:
corrosion_set_env_vars(gfx
  "CARGO_FEATURE_CORROSION_INCLUDES=\"${PROJECT_SOURCE_DIR}/src\""
  "CARGO_FEATURE_CORROSION_LINK_LIBS=\"${link_libs}\""
  "CARGO_FEATURE_CORROSION_LINK_SEARCH=\"${link_search}\"")

and to use single quote the internal ones:
corrosion_set_env_vars(gfx
  "CARGO_FEATURE_CORROSION_INCLUDES='${PROJECT_SOURCE_DIR}/src'"
  "CARGO_FEATURE_CORROSION_LINK_LIBS='${link_libs}'"
  "CARGO_FEATURE_CORROSION_LINK_SEARCH='${link_search}'")

But this just gave the same result as before (inserting quotes in the first/last
element).
Either way, this is clearly a separate issue. Perhaps this should be lifted out
to a new ticket? Personally, my opinion on this is that list variables should
probably be passed in verbatim (somehow), and that the documentation for this
call should include a descriptive example, as this could very easily be a source
of confusion.

Hmm, strange, that didn't actually change anything:

In that case this is because of the COMMAND_EXPANDS_LIST setting also expanding this list. I'm afraid there is nothing that can really be done here though, since corrosion needs COMMAND_EXPANDS_LIST .

Perhaps this should be lifted out to a new ticket?

This can't be fixed - The user replacing the semicolon with another character is really the only solution then, AFAIU.
Anyway, I did get something close to your example working, but since there are circular dependencies, this doesn't work with every linker. You may want to consider if you really need a circular dependency between the rust and the C++ libraries. Otherwise, linking with lld does seem to work.


Withdrawal questions:

When I withdrew from L2 to L1 wallet, I don't know why the system automatically replaced the uppercase letters in my L1 address with lowercase letters, causing my L1 wallet to not receive the withdrawal
This is the L1 address of my ETH:
0x9586f4B3dC7C0C3bc2D667CfADCa640896160b91
This is my transaction details from L2 withdrawal to L1:
https://zkscan.io/explorer/transactions/sync-tx:1bee1aaac37208f1606684415a6e5ef6195af1b5735f59f04287af56db4265c0

Although I only lost a small amount of ETH, I think this is a BUG that cannot be ignored.

Hi @djs1215 !
Sorry for the late reply. The funds have arrived onto your wallet:
https://etherscan.io/address/0x9586f4b3dc7c0c3bc2d667cfadca640896160b91#internaltx
The casing does not matter in Ethereum. The mixed case is used to make sure the address is correct, but lowercase is just fine. If the problem persists it is likely that your wallet does not support receiving funds from smart contracts. You should contact the wallet's support.

Hi @djs1215 !
Sorry for the late reply. The funds have arrived onto your wallet:
https://etherscan.io/address/0x9586f4b3dc7c0c3bc2d667cfadca640896160b91#internaltx
The casing does not matter in Ethereum. The mixed case is used to make sure the address is correct, but lowercase is just fine. If the problem persists it is likely that your wallet does not support receiving funds from smart contracts. You should contact the wallet's support.

Oh,I see.Thank u so much!


ROX-12713: Push marketing telemetry data to Segment

Description

Added Segment integration;
Removed Amplitude integration;
Refactored;
Added three more fields to the /v1/metadata response to be used by the frontend:

Central ID, which is the central deployment UID;
Organization ID, which is to be set by fleet manager;
Segment Key, which is the Segment write key required for sending events.

Checklist

[x] Investigated and inspected CI test results
[ ] Unit test and regression tests added
[ ] Evaluated and added CHANGELOG entry if required
[x] Determined and documented upgrade steps
[ ] Documented user facing changes (create PR based on openshift/openshift-docs and merge into rhacs-docs)

If any of these don't apply, please comment below.
Testing Performed
https://app.segment.com/redhat-devtools/sources/acs_instances_backend_dev/overview
Metadata response example:
{"version":"3.72.x-608-g6d0b983dd1-dirty","buildFlavor":"development","releaseBuild":false,"licenseStatus":"VALID","centralId":"83150944-4169-4517-91e2-602aee63b18d","organizationId":"some:org:id"}

@0x656b694d , I was hoping to test this image in action, but it looks like the PR build failed.

I think in general there is test code missing. Do you plan to add unit tests later?

Yes. Having hard times to find what exactly to test the way it'd make sense.

@0x656b694d , I was hoping to test this image in action, but it looks like the PR build failed.

@vjwilson, I'm sorry about that. There were issues with style. There's a ready to use image now.

/test gke-nongroovy-e2e-tests

Superseded by #4070.


disable dataset viewer for 'nfaa' tag

See https://huggingface.slack.com/archives/C042H6N5JR0/p1714771691945989 (internal)
On the first backfill cron job after this PR is merged, all the datasets with the "nfaa" tag will be deleted from the dataset viewer's cache.

cc @allendorf @No-m4d for vis on this


Observation data possible?

Component is not showing latest observation data as a current weather condition. Seems to be using fmi forecast data as a current temperature/weather. Would it be possible to use fmi's observation data to show current conditions because observation data is more accurate for current weather conditions?

Hey, is this still valid?
I checked the current code and look like it uses weather data correctly. Closest location is fetch by coordinates using fmi's opendata interface.


fetchOne not linking

I have a model service layer with well defined custom classes.  When I search for an entry with entries().fetchAll(...), I get a well defined object list with linked subentries marshalled as expected.  However, if I use entries().fetchOne(id) I get a well defined object, but the linked subentries are LinkedTreeMaps and I get this error:
java.lang.ClassCastException: com.google.gson.internal.LinkedTreeMap cannot be cast to ...
thoughts?  Am I missing a setting to perform the linking?
EDIT: I'm using version 2.0.1
    <dependency>
      <groupId>com.contentful.java</groupId>
      <artifactId>java-sdk</artifactId>
      <version>2.0.1</version>
    </dependency>

Hi James,
This is a limitation of the underlying API. When fetching multiple entries (e.g. GET /spaces/{spaceId}/entries?content_type={contentTypeId}&include=2) the response is a collection object that contains linked resources under it's includes property. However, when fetching an individual entry (e.g. GET /spaces/{spaceId}/entries/{entryId}) the response is the entry itself, without any includes. This means that the Java library can't stitch together the object references without performing additional requests.
The simplest workaround is to use fetchAll(ImmutableMap.of("sys.id", id)).getItems().get(0) instead of fetchOne(id) in your model layer (note that ImmutableMap comes from Guava, I just used it to keep the example concise). Adding support for includes to individual resources may happen in a future version of the API.

In addition, links might not resolve properly due to other reasons (i.e. the linked resource was deleted), so in order to workaround the ClassCastException you could optionally use nullifyUnresolvedLinks() method of Client.Builder, which as the name suggests replaces any unresolved links with null.

Thanks. Does the objective c library have this same issue?  I don't recall running across it, but maybe I need to take another look

Apparently the objc SDK generates this query automatically in case you use it's equivalent of fetchOne(), see: https://github.com/contentful/contentful.objc/blob/master/Code/CDAClient.m#L215-L233
Perhaps we could make this the default behaviour for the Java SDK as well.

That sounds great. I will match that in my service layer wrapper as well.

Feel free to close the issue if desired

Current users might rely on testing the status code of the response or even the response body on failures using a RetrofitError instance. In case we modify the fetchOne() function as discussed above, it will result in completely different behaviour (e.g. when requesting a non-existing resource by ID then an empty array would be returned). This is a fairly simple workaround to implement if desired, I thing it is best to keep things as they are.

That is fine.  Here's what I'm doing
    private <T extends CMEntry> T fetch(String identifier) {
        Map<String, String> query = new HashMap<String, String>();
        query.put("sys.id", identifier);
        CDAArray results = client.entries().fetchAll(query);
        CDAResource entry = null;
        if(results != null && results.getItems().size() > 0) {
            entry = results.getItems().get(0);
        }

        //noinspection unchecked
        return (T) entry;
    }

And I'll be adding a boolean param to the fetch to indicate whether to use the includes in which case it will either run this code or the code you provide.

@jameshays thanks for sharing!


Update template contract

Summary
The template contract should be updated to use provwasm 2.0, and it should also use the latest optimizer 0.14.0.
Problem Definition
The optimizer isn't consistent across the Makefile and the cargo metadata scripts. They are not the same version so there may be times where optimization fails and it can be hard to debug. This failure is from older optimizer versions.
Proposal
Update provwasm in Makefile.
Update optimizer version in Makefile
Update optimizer version in Cargo.toml

For Admin Use

[ ] Not duplicate issue
[ ] Appropriate labels applied
[ ] Appropriate contributors tagged
[ ] Contributor assigned/self-assigned

closed by #129


Add Signers in ReadAppTxFlags

I feel like the refactor from this structural change should maybe extend to sendTx - also maybe to the SignAndPostTx function in light client - like maybe we shouldn't need to wrap the Tx before sending here. gtg now but maybe we can discuss these changes and group in with this PR

Looks great.  Code works the same, but yes, this seems more usable from your application.
Nice changes.  Thank you for the feedback.


XLS without rows?

I have an XLS structured as:
DIMENSIONS at 1295496
ROW at 1295516
LABELSST at 1295530
...
LABELSST at 1295530
LABELSST at 2745118
WINDOW2 at 2745140
EOF at 2745144
1637 at 2745173

It means no Indexand starting with Dimensions(and it is not a problem) but without rows, only shared strings.
Has anyone idea how to load such a file?
The Library handles only the first row, even if Excel shows thousands of rows :)
That's because moveToNextRecordNoIndexhalts founding EOF at 2745144.
To bypass the error you need to change readWorkSheetGlobalsas follows:
            XlsBiffRecord trec = rec; // <<< init to previously read record
            XlsBiffDimensions dims = null;

            do
            {
                if (trec.ID == BIFFRECORDTYPE.DIMENSIONS)
                {
                    dims = (XlsBiffDimensions)trec;
                    break;
                }
                trec = m_stream.Read(); // <<< first check, then read next record

            } while (trec != null && trec.ID != BIFFRECORDTYPE.ROW);

I'm wonder how read this binary Excel file...
For privacy reason I'm going to modify binary data and I'll upload a test case ASAP.
Thanks for helping me.

We've come across a variation of this. No ROW records at all, only LABELSST and NUMBER.
LABELSST
LABELSST```

Unfortunately I had no chance to find an efficient solution. You?

I've made a small attempt at implementing support for Excel files without ROW records. In your case, is it that there are no ROW records or only one when there are in fact more than one rows?

I have two files generated by the same "procedure" by IT supplier, I'll give it a try...
Thanks a lot!

Just tested: unfortunately it does not handle my files.
Still loading first ROW only...

Ok, a bit difficult without a sample to test with. :)

What is the status of this? @wizardgsz  How did you obtain the initial xls structure? Through ExcelDataReader or some other tool?

Yes, I used the Library to dump the data but also BiffView Tool
:)

The current code in develop doesn't use ROW records so I think we can close this issue.

@wizardgsz I'm closing, but please reopen if it still fails with latest. It would be really useful with an anonymized xls to test with.
For ease, the latest prealpha nugets are here: https://ci.appveyor.com/project/andersnm/exceldatareader/build/<IP_ADDRESS>/artifacts

Where can I find an example to read Excel using new interface? I need a DataSet to test if new implementation works fine or fail.

Do you have VS2017 to build the current develop branch? The "TestApp" is built with it and easy for random testing.
Otherwise, to use the Nugets above in an earlier VS, you'd have to configure your solution with a Nuget.config. See f.ex https://docs.microsoft.com/en-us/nuget/schema/nuget-config-file
The API and sample code for using ExcelDataReader is mostly unchanged. The biggest change so far is to change from using Excel; to using ExcelDataReader; in existing code.

Unfortunately no administrative user on corporate laptop (and I cannot install new software).

@wizardgsz Have you been able to test further with the 3.0 release to verify that this has been fixed?

Without a repro there is nothing to do here.


[Fix it] Domain webextensions.in is down/parked

Describe the bug
webextensions.in appears to be parked at a domain squatter.
To Reproduce
Steps to reproduce the behavior:

visit webextensions.in
see offer to "buy this domain"

Expected behavior
Valid domain used
Something else you want us to know
mozilla/extension-workshop links to this heavily
Can you park this under github.io instead?

Thanks on it @actualben

@actualben its hosted now on webext.eu as someone parked the domain as you mentioned. Thanks for filing the bug really appreciate it. 💯


Implement AdaGrad #127

@rxwei @saeta @eaplatanios As far as my understanding of Adagrad goes,  This implementation is missing a learning rate decay.  It should also have a precondition and the learning rate should get updated each loop.
One of the features of the Adagrad algorithm is the learning rate decay, since the authors argue that in usual Machine learning, learning rate remained constant.

Yes @Shashi456, there is no learning rate decay. I think embedding the learning rate decay strategy into the optimization logic is not good. There are many different ways to update the learning rate (see: https://arxiv.org/abs/1506.01186).

@Mistobaan I'll take a look. A variety of implementations i've looked across frameworks seem to have some kind of decay in place. Thanks for the paper link :P

Learning rate decay is not part of adagrad per se and is independent.
Adagrad simply adapts the learning rate by using the accumulated gradient
norms and that’s sufficient to obtain the guarantees shown in the paper as
far as I remember. There is many kinds of learning rate schedules you can
use on top of adagrad but I don’t believe they should be mixed up with the
optimizer implementations as they can be used with any optimizer.
On Mon, Jun 10, 2019 at 10:02 PM Pawan Sasanka Ammanamanchi <
<EMAIL_ADDRESS>wrote:

@rxwei https://github.com/rxwei @saeta https://github.com/saeta
@eaplatanios https://github.com/eaplatanios As far as my understanding
of Adagrad goes, This implementation is missing a learning rate decay. It
should also have a precondition and the learning rate should get updated
each loop.
One of the features of the Adagrad algorithm is the learning rate decay,
since the authors argue that in usual Machine learning, learning rate
remained constant.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/tensorflow/swift-apis/pull/192?email_source=notifications&email_token=AAJ4EXB4LPF2O7SUJXQOQXTPZ4WWRA5CNFSM4HWZH3YKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXL6E4I#issuecomment-500687473,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAJ4EXHCRJO2GJVG3HKN5WDPZ4WWRANCNFSM4HWZH3YA
.

Updates with all the comments. Added simple XOR test case.


TypeError

Hello
I get this error, can you help me?

TypeError: document.getElementById(...) is null

could you paste your code?

Also your browser and version, Thanks for your issue.

Hello
Thank you for return,

Dont have all button in the one page, if dont have all buttons, I get this error

TypeError: document.getElementById(...) is null

I have been using one button in Content.php page and working but if I have multiple button same ID in the a page then only a button is working.. Others buttons is not working

I use 4 same button (button ID=bekleyin) in Content.php but only one button is working
http://pastebin.com/SFWNZs7i

In one page, The ID is unique.
you have used jquery, so you can call alerty like this
$('#Bekleyin') .on('click', function(event) {
  alerty.toasts('Please wait..', {bgColor: '#00A651', fontColor: '#fff', time:5550, place: 'top'})
})

but first you shoud chage your ID different, such as Bekleyin01...
if toasts content is same you can call alerty with class selector

I changed codes:

$('#Bekleyin').on('click', function(event) { ...

Thanks!
BTW I ask one more question please
I want the show automatically without clicking any button. How do I show automatically when the page load?

first import alerty.js, then call it
// directly
alerty.toasts('Please wait..')
// or
window.onload = function(){
  alerty.toasts('Please wait..')
}

Yes, working now, you are great man!
Thank you for your time,
from Turkey

That's fine, thanks for your using.


I have get an error when I use  the following method "[AFHTTPSessionManager GET:parameters:progress:success:failure:]"

when i use [AFHTTPSessionManager GET:parameters:progress:success:failure:] , i get the following eorro:
"-[AFHTTPSessionManager GET:parameters:progress:success:failure:]: unrecognized selector sent to instance "
but , if i use [AFHTTPSessionManager GET: parameters: success:failure:] , it work

Have you tried completely cleaning your dev environment? These issues are typically related to a bad cache locally.

Thanks for your reply!

I fix the issue by remove the bulid finder  and  than   install  pods
sometime there will be  not found for  lAFNetwork
for this  you can choose  another simulators to fix it


es: use geo_point and related types for geometries

Is your feature request related to a problem? Please describe.
Currently the coordinates property of the locations field only accepts floats (or an array of those).
Describe the solution you'd like
It would be nice to take advantage of Elasticsearch's geo_point and geo_shape. Among others to facilitate bounding box searches.
Take into account possible restrictions that might need to be documented. For example:

Please note that string geo-points are ordered as lat,lon, while array geo-points are ordered as the reverse: lon,lat.
Originally, lat,lon was used for both array and string, but the array format was changed early on to conform to the format used by GeoJSON.

The idea is to create a Dumper that is able to distinguish the shape and dump it in the correct ES field (this would require changes to the mapping). This dumper should take care also of serializing it back (i.e. for searches).

Change the ES mappings to be able to index correctly the coordinates
Record.dumps()/Records.loads() should be idempotent. If needed a Elasticsearch dumper extension can be written see: https://github.com/inveniosoftware/invenio-rdm-records/blob/master/invenio_rdm_records/records/api.py#L40-L41

There are three approaches we could take to integrating GeoJSON, with each being a different dividing line between the GeoJSON schema and the InvenioRDM schema:

record['metadata']['locations'] can be a GeoJSON FeatureCollection
record['metadata']['locations'] can be an array of GeoJSON Features
record['metadata']['locations'] can be an array of objects each with a GeoJSON geometry as their geometry attribute

Features can have a null geometry, so we can still support only-text locations with all three.
In the first two cases, the InvenioRDM schema will still need to constrain the GeoJSON elements to handle identifiers, place and description, which I'd propose is done with GeoJSON foreign members instead of entries in the GeoJSON properties. This means we won't clash with anyone importing existing GeoJSON records who have their own properties, though does allow people to include arbitrary metadata within their GeoJSON, which may not be desirable.
FeatureCollections can't contain other FeatureCollections, so the first option is safe from arbitrary nesting.
To begin with we could enforce the type of all geometries to be Point, assuming we can easily migrate from geo_point to geo_shape in ES later. Or we could calculate the centroid ourselves using shapely, and index that separately as a geo_point.
I'd say the FeatureCollection approach is more elegant and has better reuse, but that the geometry-only approach is far more tightly constrained and loses nothing for our purposes. The Feature approach gains a bit of terseness, but is less elegant and is no more constrained than using FeatureCollections, so wins on neither metric.

External JSON
I guess the most future proof solution is to imitate a FeatureCollection using foreign members. Thus on the REST API we would serve something like:
{
    "locations": {
        "type": "FeatureCollection",
        "features": [{
            "type": "Feature",
            "geometry": {
                "type": "Point",
                "coordinates": [6.05, 46.23333]
            },
            "identifiers": {
                "geonames": "2661235",
                "tgn": "http://vocab.getty.edu/tgn/8703679"
            },
            "place": "CERN",
            "description": "Invenio birth place."
        }]
    }
}

This means we could later add the bound box bbox for instance or other foreign members to the feature collection.
Also using foreign member, could allow for user specified metadata in the properties field as mention.
Internal JSON
Internally, I would, however, remove the type key for FeatureColleciton and Feature so that internally we would store:
{
    "locations": {
        "features": [{
            "geometry": {
                "type": "Point",
                "coordinates": [6.05, 46.23333]
            },
            "identifiers": {
                "geonames": "2661235",
                "tgn": "http://vocab.getty.edu/tgn/8703679"
            },
            "place": "CERN",
            "description": "Invenio birth place."
        }]
    }
}

Both "type": "FeatureCollection",  and "type": "Feature" can easily be added via serializer as they are just constants and just increase the data size of the JSON document.
DataCite compatibility
I think the above structure is compatible with DataCite.
Initial implementation
As a first initial implementation, I would go with only support for "type": "Point" geometries, so that indexing becomes simple. Then later, once we know use cases a bit better, we can extend with other types.

Comments from GEO: https://geosecretariatgeneva-my.sharepoint.com/:w:/g/personal/ffranziskakis_geosecretariatgeneva_onmicrosoft_com/EbBAaAFJ7G5ElWL7RHdJZ6gBpGN9prlk94DRX8XBSkH1kg?e=cKuO8X

That seems reasonable to me :-)

Should we rename place on locations to title, for consistency with the rest of our schema?


[Issue 6761] Correct tokenSecretKey base64 inline description

Motivation
Fixes #6761
Modifications
Correct tokenSecretKey base64 inline description in documents (based master & version 2.5.1) and examples.

/pulsarbot run-failure-checks

@murong00 Would you please also fix the "data:base64," in test "TokenAuthWithSymmetricKeys.java"?

@jiazhai What do you mean? This pr has been included fix data:base64, in TokenAuthWithSymmetricKeys.java which is a base test.

@murong00 Can you also fix the documentation of 2.5.1?

@jiazhai What do you mean? This pr has been included fix data:base64, in TokenAuthWithSymmetricKeys.java which is a base test.

I found that change. thanks for the work.

@murong00 Can you also fix the documentation of 2.5.1?

Already fixed in this pr.

/pulsarbot run-failure-checks

/pulsarbot run-failure-checks


Preflight request gives 401 error with combination of MP-JWT and CORS Support

Environment Details

Helidon Version: 3.1.2
Helidon MP
JDK version: Java 17
OS: Windows
Docker version (if applicable): not using

Problem Description
I'm using MicroProfile JWT Auth and Helidon's CORS Support in my application, but I'm having trouble with a 401 error when making a Preflight request from a browser to an authenticated endpoint.
Specifically, let's say you have an app that uses MP-JWT and CORS support like this:

sample code

@LoginConfig(authMethod = "MP-JWT")
@ApplicationPath("api")
@ApplicationScoped
public class ApiGatewayApplication extends Application {
    @Override
    public Set<Class<?>> getClasses() {
        return Set.of(ApiGatewayResource.class);
    }
}
--------------
@Path("resources")
@ApplicationScoped
public class ApiGatewayResource {
    @GET
    @Produces(MediaType.APPLICATION_JSON)
    public List<ResourceDto> getAll() {
        return ...
    }
    ...
}

application.yaml

cors:
  enabled: true
  paths:
    - path-pattern: /{+}
      allow-headers: "*, Authorization"
      allow-origins: http://localhost:3000
      allow-credentials: true

After JWT authentication for this app, when the browser accesses GET /api/resources from a cross-domain, the browser asks the server for permission to use the Authorization header that sends a Bear token, so the browser calls OPTIONS /api/resources 's preflight request to the server, but the OPTIONS request results in an error in MP-JWT authentication, and cross-domain API calls cannot be made.
After checking, the cause of the error was that the MP-JWT was processed before the CORS Support request was processed.
OPTIONS /api/resources preflight request does not have Authorization header with Bear Token set, so you will get an authorization error before CORS processing.

I have a request
I want to use MP-JWT and CORS Support in combination.
In order to be able to use MP-JWT and CORS Support without problems, could you please do one of the following in Helidon?

Change to process CORS Support before processing MP-JWT
Add an exclusion setting function such as not performing authentication for the OPTIONS method as a Helidon original function of MP-JWT.

For reference, I confirmed that 1. works after modifying the priority locally as shown below (I didn't pass the detailed test..)
@Priority(Priorities.AUTHENTICATION - 100)
//@Priority(Priorities.HEADER_DECORATOR)
class CrossOriginFilter implements ContainerRequestFilter, ContainerResponseFilter {
    ...
}

What CrossOriginFilter is doing is certainly HEADER_DECORATOR, but due to the processing characteristics, I feel that it is appropriate to put it before AUTHENTICATION.
So, I would like to request 1. rather than 2.
I love Helidon so please!

@extact-io A quick follow-up question for you...
Have you tried adding a method similar to the following to your ApiGatewayResource:
@OPTIONS
public void optionsForResources() {}

I believe this would include the options method in your security scheme, thereby allowing the security check to pass so the CORS checking and preflight processing would follow.
Your suggestion about reversing the order in which the CORS and security filters run seems as if it would avoid the problem you found, but I need to think more about whether that's the best choice we have.

Hi @tjquinno
Thank you for looking at my app.
I immediately tried your suggestion, but unfortunately the result is still the same, I get a 401 error in authentication and the preflight request is not processed.
I understand that my ideas need to be thought through.
If there is another way to avoid this, please let me know.

@extact-io Thanks very much for trying.
I expect to make more progress on this today (US time).

hi @tjquinno
I've tried various methods other than the method of bypassing the authentication check that you taught me before, but there is no way to go well other than changing the priority of CrossOriginFilter.
@tjquinno any progress?
I hope that this problem will be resolved, so I would appreciate it if you could do something about it.


WIP: Stop discovery

Coverage decreased (-2.6%) to 74.569% when pulling ac38846c49468f49e39df0456ed0f5921bc60400 on feature/2-stop-discovery into 585353c14f08c689b874bd222b558fcbb2599e41 on master.

Coverage decreased (-2.6%) to 74.569% when pulling 5759acb3b9ee24b3e690c7829a444045843843b3 on feature/2-stop-discovery into 585353c14f08c689b874bd222b558fcbb2599e41 on master.

Coverage decreased (-2.7%) to 74.459% when pulling dbcc63a1c0b077a82ce50bd4a91b1aecdb23db30 on feature/2-stop-discovery into 585353c14f08c689b874bd222b558fcbb2599e41 on master.

Coverage decreased (-2.7%) to 74.459% when pulling 39f1689552cc7b131f60f9e06b1fa708ce378438 on feature/2-stop-discovery into 585353c14f08c689b874bd222b558fcbb2599e41 on master.


MNT change extend to minibatch

MNT change extend to minibatch to split columns of both X and y into batches instead of X only.

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 100.00%. Comparing base (d4306c4) to head (4e42fa3).
Report is 15 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##              main       #24    +/-   ##
==========================================
  Coverage   100.00%   100.00%
==========================================
  Files            3         6     +3
  Lines          110       400   +290
==========================================
+ Hits           110       400   +290

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


⚠ ClamAV has degraded performance

In f8f1ee8, ClamAV (https://spamassassin.apache.org/) experienced degraded performance:

HTTP code: 200
Response time: 82 ms

Resolved: ClamAV performance has improved in 6f95c94.


How I get table columns?

Hi, I want to get table columns in json or array from model declaration. That is possible?
Example:
User table:
user
-id
-name
-lastname
var object = bookshelf.Model.extend({
		tableName: "user"
	});

var fields = object.getFields(); // this function is only for example (this function not exist) result i wish ['id', 'name', 'lastname']

function like "getFields()" in the example, already exist? or someway to get the example result??
Thank you
PD. Sorry for my bad english.

Check out columnInfo over at Knex.


[BUG] <Shenyu gateway proxy websocket lost the user-defined closeStatus (code and reason).>

Is there an existing issue for this?

[X] I have searched the existing issues

Current Behavior
Use shenyu gateway to proxy websockets for backend services.
Encountered a problem when closing a connection using a specific status code (such as 4400). If connected directly, the custom code and reason will be returned to the client. But when using shenyu gateway proxy, both code and reason will be lost.
Expected Behavior

Steps To Reproduce
1.run: shenyu-example-spring-native-websocket
The key codes are as follows:
protected void handleTextMessage(final WebSocketSession session, final TextMessage message) throws Exception {
        // Get the message from the client.
        String payload = message.getPayload();
        Object token = session.getAttributes().get("token");
        LOG.info("server received {}, sent {}", token, payload);
        if (Objects.nonNull(payload) && Objects.equals(payload, "testCloseStatus")) {
            session.close(new CloseStatus(4400, "test:apache shenyu server close, return closeStatus"));
        }else{
            session.sendMessage(new TextMessage("apache shenyu server send to " + token + " message : -> " + payload));
        }
    }

run shenyu-gateway
send request: ws://<IP_ADDRESS>:8868/myWebSocket?token=123

Environment
all versions.

Debug logs
No response
Anything else?
No response

I will fix it.


Consolidate logic between std_vec and heap_based session implementations.

Specifically the update function, how it's implemented in HeapSessionManager is much more readable and clear, and we can likely consolidate the logic into a single place.
At the minimum update StdVecSessionManager to mimic the style.

I thought on this as well. I think we can go with HeapSessionManager. This has the same logic with some improvements. It uses the alloc crate, so it can be used in no_std and std environments.
I think a more contrast implementation would be having more static memory usage.


Issue with loading xxx_prompts.yaml file specified under "examples" section in the .yaml file passed as parameter in load_prompt()

System Info
Langchain version: 0.0.272
Python version: 3.10
Host System: Windows 11
I'm loading Few Shot Prompts from a fewshot_prompts.yaml file by using load_prompt() function. The fewshot_prompts.yaml file has a section with the title "examples:" to load the Few Shot Prompts from file example_prompts.yaml.  The files fewshot_prompts.yaml and example_prompts.yaml both are in the same directory. But the _load_examples() function is not able to locate/load the example_prompts.yaml. There is no way to specify the path to this file.
Due to the above issue, the loading of the example_prompts.yaml file fails.
Who can help?
No response
Information

[ ] The official example notebooks/scripts
[ ] My own modified scripts

Related Components

[ ] LLMs/Chat Models
[ ] Embedding Models
[X] Prompts / Prompt Templates / Prompt Selectors
[ ] Output Parsers
[ ] Document Loaders
[ ] Vector Stores / Retrievers
[ ] Memory
[ ] Agents / Agent Executors
[ ] Tools / Toolkits
[ ] Chains
[ ] Callbacks/Tracing
[ ] Async

Reproduction
The
_type: few_shot
input_variables:
["bot_response"]
prefix:
The following are excerpts from conversations with an AI assistant. Given an input text and a set of rules,the assistant strictly follows the rules and provides an "yes" or "no" answer.Here are some examples
example_prompt:
_type: prompt
input_variables:
["bot_response","answer"]
template:
"bot_response: {bot_response}\nanswer: {answer}"
examples:
example_prompts.yaml

Unable to find the file example_prompts.yaml
Expected behavior

Provide a way to specify the path to load the example_prompts.yaml file

@dosu‐bot,
Thank you for your detailed response.
I have a concern that if I provide absolute path for the example_prompts.yaml,
it will not work when deployed mainstream or when testing across different
Operating Systems.
On Sat, 9 Sept, 2023, 10:17 am dosu-beta[bot], @.***>
wrote:

🤖
Hello,
Thank you for reaching out and providing detailed information about your
issue.
From your description, it seems like you're trying to load a prompt from a
fewshot_prompts.yaml file, which in turn references an
example_prompts.yaml file under the "examples" section. However, the
_load_examples() function is unable to locate/load the
example_prompts.yaml file.
In LangChain version 0.0.272, the load_prompt() function does support
loading prompts from a file specified in the yaml file. The function
_load_prompt_from_file() is used to load the prompt from a file. It
checks the file extension and if it's a .json or .yaml file, it loads the
file content into a dictionary named config. This config dictionary is
then passed to the load_prompt_from_config() function which loads the
prompt based on the configuration.
Here is the relevant code:
def _load_prompt_from_file(file: Union[str, Path]) -> BasePromptTemplate:
"""Load prompt from file."""
# Convert file to a Path object.
if isinstance(file, str):
file_path = Path(file)
else:
file_path = file
# Load from either json or yaml.
if file_path.suffix == ".json":
with open(file_path) as f:
config = json.load(f)
elif file_path.suffix == ".yaml":
with open(file_path, "r") as f:
config = yaml.safe_load(f)
else:
raise ValueError(f"Got unsupported file type {file_path.suffix}")
# Load the prompt from the config now.
return load_prompt_from_config(config)
This function is then used in the load_prompt() function:
def load_prompt(path: Union[str, Path]) -> BasePromptTemplate:
"""Unified method for loading a prompt from LangChainHub or local fs."""
if hub_result := try_load_from_hub(
path, _load_prompt_from_file, "prompts", {"py", "json", "yaml"}
):
return hub_result
else:
return _load_prompt_from_file(path)
In the load_prompt() function, it first tries to load the prompt from
LangChainHub. If it fails, it then tries to load the prompt from a local
file using the _load_prompt_from_file() function.
However, it seems like the issue you're facing is related to the relative
path of the example_prompts.yaml file. The _load_examples() function
might not be able to locate the file if it's not in the same directory as
the script that's running the load_prompt() function.
To resolve this issue, you can try providing the absolute path to the
example_prompts.yaml file in the fewshot_prompts.yaml file. Here's how
you can do it:
examples:
C:/path/to/your/example_prompts.yaml
Replace "C:/path/to/your/example_prompts.yaml" with the actual path to
your example_prompts.yaml file.
Please try this solution and let me know if it resolves your issue. If
you're still facing the issue, please provide more details about your
directory structure and the exact error message you're getting.
Best,
Dosu
Sources

libs/langchain/langchain/prompts/loading.py
https://github.com/langchain-ai/langchain/blob/-/libs/langchain/langchain/prompts/loading.py
libs/langchain/tests/unit_tests/prompts/test_loading.py
https://github.com/langchain-ai/langchain/blob/-/libs/langchain/tests/unit_tests/prompts/test_loading.py

This response is meant to be useful and save you time. It is not meant
to be a precise solution, but rather a starting point for your own
research.
Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it
is irrelevant.
If you want to continue the conversation, start your reply with @dosu-bot
https://github.com/dosu-bot.
—
Reply to this email directly, view it on GitHub
https://github.com/langchain-ai/langchain/issues/10390#issuecomment-1712413713,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AMY7COAZICZPPNXYPLOYA7LXZPYEHANCNFSM6AAAAAA4RGAOPE
.
You are receiving this because you authored the thread.Message ID:
@.***>

@dosu‐bot,
I have 2 questions

Would it not be better if there was a way to specify the path to example
file in the YAML file?

is the below change part of upcoming langchain release?

On Sun, 10 Sept, 2023, 6:51 am dosu-beta[bot], @.***>
wrote:

🤖
Hello,
Thank you for reaching out and providing a detailed explanation of the
issue you're facing. It seems like the problem is related to the file path
of example_prompts.yaml. The _load_examples() function is trying to open
the file directly from the path provided, which might not work if the file
is not in the same directory as the script or if the path is not an
absolute path.
To make it compatible across different environments and operating systems,
you can use the os module to get the absolute path of the file. Here's
how you can modify the _load_examples() function:
import os
def _load_examples(config: dict) -> dict:
"""Load examples if necessary."""
if isinstance(config["examples"], list):
pass
elif isinstance(config["examples"], str):
# Get the absolute path of the file
absolute_file_path = os.path.abspath(config["examples"])
    with open(absolute_file_path) as f:
        if absolute_file_path.endswith(".json"):
            examples = json.load(f)
        elif absolute_file_path.endswith((".yaml", ".yml")):
            examples = yaml.safe_load(f)
        else:
            raise ValueError(
                "Invalid file format. Only json or yaml formats are supported."
            )
    config["examples"] = examples
else:
    raise ValueError("Invalid examples format. Only list or string are supported.")
return config

This modification ensures that the function can locate the file regardless
of the current working directory or the operating system.
Please note that this change should be made in the load.py file located
at libs/experimental/langchain_experimental/prompts/load.py or
libs/langchain/langchain/prompts/loading.py depending on your setup.
I hope this helps! If you have any other questions or run into any issues,
please don't hesitate to ask.
Best,
Dosu
Sources

libs/experimental/langchain_experimental/prompts/load.py
https://github.com/langchain-ai/langchain/blob/-/libs/experimental/langchain_experimental/prompts/load.py
libs/langchain/langchain/prompts/loading.py
https://github.com/langchain-ai/langchain/blob/-/libs/langchain/langchain/prompts/loading.py

This response is meant to be useful and save you time. It is not meant
to be a precise solution, but rather a starting point for your own
research.
Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it
is irrelevant.
If you want to continue the conversation, start your reply with @dosu-bot
https://github.com/dosu-bot.
—
Reply to this email directly, view it on GitHub
https://github.com/langchain-ai/langchain/issues/10390#issuecomment-1712678272,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AMY7COE7GUNETIKYCMTGDRTXZUIYBANCNFSM6AAAAAA4RGAOPE
.
You are receiving this because you authored the thread.Message ID:
@.***>

@dosu-bot,
Can you provide me an example for how to use the function get_validated_relative_path() ?
The following YAML file and example_prompts.yaml file both are located in the same directory. But the script is unable to find the example_prompts.yaml file.
_type: few_shot
input_variables:
["bot_response"]
prefix:
The following are excerpts from conversations with an AI assistant. Given an input text and a set of rules,the assistant strictly follows the rules and provides an "yes" or "no" answer.Here are some examples
example_prompt:
_type: prompt
input_variables:
["bot_response","answer"]
template:
"bot_response: {bot_response}\nanswer: {answer}"
examples:
example_prompts.yaml


Request: Make background colors different.

On Discord, they are a lot of colors even if they're not very different and Dracula theme makes them the same it is really hard to see a difference, e.x. embed messages from a bot has the same color as the background, so you would like to change it a little be it would be really nice, thanks.

I agree with this person, I like to put my servers in folders to keep them organized. The Dracula theme doesn't give the folders an unique background color, which just makes it look even more chaotic than without folders at all.

And codeblocks don't have any seperation from the background

Now this isn't a huge issue since they have a monospaced font but some seperation would be helpful

This is fixed with the new version


fixed need some margin on profile section #8210

Fixes Issue
Need some magin on profile section

Closes #8210
Changes proposed

Check List (Check all the applicable boxes)

[x] My code follows the code style of this project.
[ ] My change requires changes to the documentation.
[ ] I have updated the documentation accordingly.
[x] All new and existing tests passed.
[x] This PR does not contain plagiarized content.
[x] The title of my pull request is a short description of the requested changes.

Screenshots
Before :

After :

Note to reviewers

can someone review my pr

I have changed my-2 to mt-3 can you review the changes.


Curved Edges

Hi,
I was using sigma js before the upgrade, and when I would draw edges they would be curved. I recently learned of the new sigma js and decided to use it. But now when I draw my graphs all of the edges are straight. Is there a way to curve the edges?
Thanks!

Any reason I did this and still the edges are coming as straight? What does this do anyway? Curvemmeans just a random curve or follows some specific algorithm?


Wikipedia Feeds not displaying images + wrong formatting

Describe the bug
Hello, love the app and use it everyday but following three feeds do not display images in them as well as not formatted properly as in does not display any text
The feeds in question are

Wikipedia - Picture of the Day
Wikipedia - Featured Article
Wikipedia - On This Day

To Reproduce
Steps to reproduce the behavior:

Go to Adding a New feed
Add any or all of the above feeds
Wait for article to appear
See error

Screenshots

Device Information

Device: Pixel 4a 5g
OS: Android 14
App Version [e.g. 1.0.0]

Opened this issue on March 21 but due to my account being flagged it just became visible.

Thanks for reporting the issue. Apart from the images, did the formatting issue resolved in the newer versions of the app?

The "On This Day" Feed sometimes doesn't display anything after a line or so

The "Feature Article" Feed has been working fine but the image issue remains

"Picture of the Day" still shows the wrong formatting with all text alinged to the very right as displayed with the image in the original post

This all was tested with the latest nightly


Detect Boost.Python for older versions of Boost (<1.67)

Extends how Boost.Python is found so that it covers earlier Boost versions (<1.67).
This makes it possible to build OpenEXR with Boost 1.66 and Python 3 support.
Boost 1.66 is the specified version for VFX Platform CY2019.
This fixes #360

Cherry-picked and merged into master, thanks.


Housekeeping

this needs #34 and #33 (hapi updates) merged before, they are included in this optimistic branch

merged as b3ba865cec62bd976aa085c58f80ad17b4309f12 and 0c0c63e44a37ceb7b42b4511e5d284042dae9681 - thank you!

merged as b3ba865cec62bd976aa085c58f80ad17b4309f12 and 0c0c63e44a37ceb7b42b4511e5d284042dae9681 - thank you!


Fix upgrade script with the official elasticsearch docker images

The k8s statefulsets/deployments were changed to work with the official docker images  in #36 . The upgrade script should also use the same images.
Thanks!

Please test this

Ok to test

Please test this


Stop hidding classes

Classes all have at least one filed, (__x__count). There is no need to hide them.

I don't think we should remove the capability to hide classes. Classes might be hidden because of a user-specified business need, not just because they are not representable in GraphQL. We should allow the user to specify that some classes should be hidden, even if they are representable in GraphQL.

Coverage decreased (-0.1%) to 93.381% when pulling 010d741c888ac32b1833b4dcb001ad51014a694d on stop-hidding-classes into 5ece61fd611b6a11bc471853fd41a3f459ee50bf on master.


Release Notes tab always reload its content on tab focus

VSCode Version: Code 1.9.0 (27240e71ef390bf2d66307e677c2a333cebf75af, 2017-02-02T08:31:00.827Z)
OS Version: Windows_NT ia32 10.0.14393
Extensions:

Extension
Author
Version

githistory
donjayamanne
0.1.5

code-runner
formulahendry
0.6.7

beautify
HookyQR
0.7.3

vscode-hexdump
slevesque
1.1.4

gitblame
waderyan
1.3.0

Steps to Reproduce:

Help / Release Notes
Scroll in middle of the document
File / Preferences / Settings
switch back to opened Release Notes

Expected: previous opened document with our scroll position.
Actual: reloaded document with zero scroll positions.

https://github.com/Microsoft/vscode/issues/15214


/for/open-source/ page CSS isn't quite right

With a moderately wide screen, things get misaligned in the responsive CSS:

@timabbott Can't seem to reproduce this. On which device(s) / around how many pixels in width is this occurring?

Hmm, I'm having a hard time reproducing this now too.  Though, the Zulip logo's position doesn't stay fixed as one resizes in a kinda odd way, that makes me think something is broken.

This was made irrelevant through some other changes to this page by @brockwhittaker.


4.15 WebScene.saveAs throwing error

I am reporting an issue with

[ ] TypeScript definitions
[ x] another resource in this repository

Steps to reproduce:
//note: not from portalItem, just basic map options
const map = new WebScene({
    "basemap": "hybrid",
    "center": [-95, 38],
    "zoom": 4
  })

const view = new SceneView({
        map,
        container: element
      })

async function saveMapAs(portalItem){
    await map.updateFrom(view);
    await map.saveAs(portalItem)
}

//call first time -- succeeds in adding the item
saveMapAs({title: 'My Web Scene'})

//call second time -- fails to add item
saveMapAs({title: 'My Web Scene - Copy'})
/*result:
Error:
   details: undefined
   message: "Copying of embedded resources is currently not supported"
   name: "webscene:failed-to-copy-embedded-resources"
*/

The place where this is failing is here in the compiled 4.15 WebScene.js source:
c.prototype._verifySaveAs = function(a, b, c) {
           //canSaveAs is returning true -- which to me means it SHOULD save, but instead it is rejecting
            return this.canSaveAs(b) ? E.reject(H.createCopyError()) : this._verifySave(a, b, c, !0)
        }
        ;
        c.prototype.verifySaveAs = function(a) {
            var b = this._enableVerifyItemRelativeUrls({
                origin: "web-scene",
                messages: []
            })
              , c = this.write(null, b);
            return this._verifySaveAs(c, b, a)
        }
        ;
        c.prototype.canSaveAs = function(a) {
            a || (a = this._enableVerifyItemRelativeUrls({
                origin: "web-scene",
                messages: []
            }),
            this.write(null, a));
            return a.verifyItemRelativeUrls && 0 < a.verifyItemRelativeUrls.writtenUrls.length
        }

snippet of the object passed to canSaveAs in my example:
{...
verifyItemRelativeUrls:
    rootPath: "https://myportal/portal/sharing/rest/content/items/86e88152bc764032a5c0897d32c441c5",
    writtenUrls: [
        0: "https://services.arcgisonline.com/ArcGIS/rest/services/World_Imagery/MapServer"
        1: "https://services.arcgisonline.com/ArcGIS/rest/services/Reference/World_Boundaries_and_Places/MapServer"
]
}

sorry for the late reply, just saw this. Is it still a problem?
Are you trying to save to a portal? We've been trying to repro by saving to ArcGIS Online, but it seems to work:
https://codepen.io/nicolaraluk/pen/mdEqojd?editors=1000


Support code generator plugins with injection point

Protobuf custom code generator plugins can inject code at the given insertion point. The insertion point must refer to a file generated upstream by either the specific language generator, or another plugin.
In my particular scenario, I am trying to generate code with my custom plugin and insert it in the Java specific code generated by protoc. However, the order in which the arguments to protoc are specified, is crucial, i.e. the --java_out argument should precede my custom plugin's out.
For that reason, I propose to modify the usage of ArgLineBuilder in SourceCodeGenerator, to first specify the enabled languages and then the resolved plugins.

I was just having a think about this. Are there any cases do we think where the converse to this would be true (i.e. the user would want the plugin to execute after the code generation)?
I am wondering if it is worth having functionality to allow specifying whether the plugin runs first or last. I am not sure if this would be needed though as I have not ever really had to work with such constraints myself.

I don't think this would be the case. That would mean the language specific generators would try to insert code into the code generated by a particular plugin.
What you might consider, is to allow full control of the order in which the plugins are specified. Currently, the order of the same type of plugin (e.g. binaryMavenPlugin) is as the user specifies them in the pom. However, if someone wants to inject code in the output of a jvmMavenPlugin from a binaryMavenPlugin, according to the code in SourceCodeGenerator.discoverPlugins, this would not be possible. So maybe a ordering attribute could be added.

Think that is reasonable to be honest. I'll raise an issue for it. Thanks.

I think the particular situation described above could also be managed with different executions and maven build stages though.


Stay on same page when changing scopes

Description
Previously, redirectPathAfterChange of useContentScopeConfig had to be explicitly set to ensure that the Admin stays on the same page after changing scopes.
This was often forgotten, resulting in redirects to the default page (usually the dashboard page).
Now, as it is the preferred behavior, the Admin will stay on the same page per default.
The previous behavior can be restored by using the useContentScopeConfig hook:
import { useContentScopeConfig } from "@comet/cms-admin";

function MainMenuPage() {
    // We want to redirect to the dashboard page after changing the scope.
    useContentScopeConfig({ redirectPathAfterChange: "/dashboard" });
}

Example

[x] I have verified if my change requires an example

Changeset

[x] I have verified if my change requires a changeset

Related tasks and documents
https://vivid-planet.atlassian.net/browse/COM-1102

Was the old default chosen to avoid errors when the user doesn't have access to the current page in the new scope?

Was the old default chosen to avoid errors when the user doesn't have access to the current page in the new scope?

I don't think so. This behavior was added in Comet v3, before we had content scope checks.
However, I just tried what happens if I have access to the news in one scope, but not in another. With this solution, it would result in an error because I'd be automatically redirected to the news page. However, when I add useContentScopeConfig({ redirectPathAfterChange: "/structured-content/news" }) (because I want to stay on the same page), I encounter the same error. So we need to fix this anyway.

So we need to fix this anyway.

Created a task for this: https://vivid-planet.atlassian.net/browse/COM-1221


Can numpy objects be supported?

What happened:
I am using netcdf to save xarray objects containing a list of lists (or numpy array of objects) inside the metadata. However, since numpy arrays are not supported, I convert this to a list of lists. When saving, I run into a Deprecation Warning and then a TypeError.
I wonder if this is the best place to solve it?
What you expected to happen:
Ideally, h5netcdf realizes this is a list of lists with different lengths, and converts it to numpy object rather then numpy array.
MCVE Code Sample
Using mne-python and mne-connectivity
import numpy as np
from mne import EpochsArray, create_info
from mne_connectivity import spectral_connectivity_epochs

    rng = np.random.RandomState(0)
    n_epochs, n_chs, n_times, sfreq, f = 10, 2, 2000, 1000., 20.
    data = rng.randn(n_epochs, n_chs, n_times)
    sig = np.sin(2 * np.pi * f * np.arange(1000) / sfreq) * np.hanning(1000)
    data[:, :, 500:1500] += sig
    info = create_info(n_chs, sfreq, 'eeg')
    tmin = -1
    epochs = EpochsArray(data, info, tmin=tmin)

    conn = spectral_connectivity_epochs(epochs,
        fmin=(4, 8, 13, 30), fmax=(8, 13, 30, 45), faverage=True)
    conn.save('foo.nc')

we get the error message:
mne_connectivity/spectral/tests/test_spectral.py:569: in test_save
    conn.save('foo.nc')
mne_connectivity/base.py:828: in save
    self.xarray.to_netcdf(fname, mode='w',
.venv/lib/python3.9/site-packages/xarray/core/dataarray.py:2846: in to_netcdf
    return dataset.to_netcdf(*args, **kwargs)
.venv/lib/python3.9/site-packages/xarray/core/dataset.py:1900: in to_netcdf
    return to_netcdf(
.venv/lib/python3.9/site-packages/xarray/backends/api.py:1072: in to_netcdf
    dump_to_store(
.venv/lib/python3.9/site-packages/xarray/backends/api.py:1119: in dump_to_store
    store.store(variables, attrs, check_encoding, writer, unlimited_dims=unlimited_dims)
.venv/lib/python3.9/site-packages/xarray/backends/common.py:265: in store
    self.set_variables(
.venv/lib/python3.9/site-packages/xarray/backends/common.py:303: in set_variables
    target, source = self.prepare_variable(
.venv/lib/python3.9/site-packages/xarray/backends/h5netcdf_.py:339: in prepare_variable
    nc4_var.attrs[k] = v
.venv/lib/python3.9/site-packages/h5netcdf/attrs.py:76: in __setitem__
    dtype = np.asarray(value).dtype
E   numpy.VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray.

mne_connectivity/spectral/tests/test_spectral.py:570: in test_save
    conn.save('foo.nc')
mne_connectivity/base.py:828: in save
    self.xarray.to_netcdf(fname, mode='w',
.venv/lib/python3.9/site-packages/xarray/core/dataarray.py:2846: in to_netcdf
    return dataset.to_netcdf(*args, **kwargs)
.venv/lib/python3.9/site-packages/xarray/core/dataset.py:1900: in to_netcdf
    return to_netcdf(
.venv/lib/python3.9/site-packages/xarray/backends/api.py:1072: in to_netcdf
    dump_to_store(
.venv/lib/python3.9/site-packages/xarray/backends/api.py:1119: in dump_to_store
    store.store(variables, attrs, check_encoding, writer, unlimited_dims=unlimited_dims)
.venv/lib/python3.9/site-packages/xarray/backends/common.py:265: in store
    self.set_variables(
.venv/lib/python3.9/site-packages/xarray/backends/common.py:303: in set_variables
    target, source = self.prepare_variable(
.venv/lib/python3.9/site-packages/xarray/backends/h5netcdf_.py:339: in prepare_variable
    nc4_var.attrs[k] = v
.venv/lib/python3.9/site-packages/h5netcdf/attrs.py:79: in __setitem__
    self._h5attrs[key] = value
h5py/_objects.pyx:54: in h5py._objects.with_phil.wrapper
    ???
h5py/_objects.pyx:55: in h5py._objects.with_phil.wrapper
    ???
.venv/lib/python3.9/site-packages/h5py/_hl/attrs.py:103: in __setitem__
    self.create(name, data=value)
.venv/lib/python3.9/site-packages/h5py/_hl/attrs.py:180: in create
    htype = h5t.py_create(original_dtype, logical=True)
h5py/h5t.pyx:1663: in h5py.h5t.py_create
    ???
h5py/h5t.pyx:1687: in h5py.h5t.py_create
    ???
h5py/h5t.pyx:1747: in h5py.h5t.py_create
    ???
E   TypeError: Object dtype dtype('O') has no native HDF5 equivalent

Expected Output
Anything else we need to know?:
Version
Output of print(h5py.version.info, f"\nh5netcdf {h5netcdf.__version__}")
Summary of the h5py configuration
h5py    3.6.0
HDF5    1.12.1
Python  3.9.5 (default, May 18 2021, 12:31:01)
[Clang 10.0.0 ]
sys.platform    darwin
sys.maxsize<PHONE_NUMBER>854775807
numpy   1.22.2
cython (built with) 0.29.24
numpy (built against) 1.19.3
HDF5 (built against) 1.12.1
h5netcdf 999

@adam2392 Thanks for raising this. A pure h5netcdf MCVE would be nice nevertheless.
For NetCDF4 data model

Attributes are always scalar values or 1D arrays, which can be associated with either a variable or the file as a whole. Although there is no enforced limit, the user is expected to keep attributes small.

So this means that this would be a non-standard usage which h5netcdf could only support using invalid_netcdf keyword arg.
Also HDF5 speaks of Attributes as small datasets.

How small is "small" is not defined by the library and is up to the user’s interpretation.

As the request is not conforming to the NetCDF4 standard I'm inclined to keep the current behavior. You might go the suggested way (HDF5-link above) of adding the data as another variable with an attribute pointing to the it.

Xref: https://github.com/mne-tools/mne-connectivity/issues/87

After reading the discussion over at https://github.com/mne-tools/mne-connectivity/issues/87 I think I might be missing a bit of context.
@adam2392 In the mne-connectivity docs the output format is mentioned as NetCDF. What you request would finally create non-standard NetCDF files. Maybe I'm missing something. Could you shed some light?

It isn't possible to directly put arbitrary NumPy object arrays into HDF5 files, because these can contain arbitrary Python objects that may not be supported by HDF5.
If desired, arbitrary Python objects could be serialized into bytes with pickle. But this would not be appropriate for h5netcdf unless users need to explicitly opt-in each time they open a file with pickle support (e.g., by specifying something like allow_pickle=True), because deserializing pickle data allows arbitrary code execution.

I'm closing this as wont-fix. If there is more discussion needed, please reopen @adam2392.