Enable function to be created outside of StartHandler.

Function is useless outside of the lambda package as there is no way to create a function with a non-nil handler. I have need to, for example, change the error conditions from log.Fatal within the entrypoint, but not having access to a Function constructor stops me from writing a reasonable replacement in my application code.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Codecov Report

Merging #192 into master will not change coverage.
The diff coverage is 0%.

@@          Coverage Diff           @@
##           master    #192   +/-   ##
======================================
  Coverage    77.2%   77.2%
======================================
  Files          18      18
  Lines         636     636
======================================
  Hits          491     491
  Misses        104     104
  Partials       41      41

Impacted Files
Coverage Δ

lambda/function.go
60% <0%> (-2.5%)
:arrow_down:

lambda/entry.go
0% <0%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 43eb3b9...0c6cfb3. Read the comment docs.

I have need to, for example, change the error conditions from log.Fatal within the entry-point

Super curious about your use-case! I suppose your plan also includes and handle rpc.Register and rpc.Accept yourself?

Super curious about your use-case! I suppose your plan also includes calling rpc.Register and rpc.Accept yourself?

It's not all that interesting - we're using a custom logger for use with localstack on our development machines and the messages from log.Fatal don't come through in a way that's immediately inspectable for us. The version we have right now simply returns an error instead of killing the process, and that error is handled by our application entrypoint.
Although now that we've done this, we see that sometimes the bind call is trying to re-use the same port within the lambda container from a previous invocation of the function, so we may end up adding a small backoff to net.Listen.

we see that sometimes the bind call is trying to re-use the same port within the lambda container from a previous invocation of the function

eg: logs like this?

2019/03/19 18:56:09 listen tcp <IP_ADDRESS>:53604: bind: address already in use
2019/03/19 18:56:09 exit status 1

Are you currently seeing this behavior? We recently deployed a change in the service to mitigate a similar sounding issue we saw in our own internal tests.

It's not all that interesting - we're using a custom logger for use with localstack on our development machines and the messages from log.Fatal don't come through in a way that's immediately inspectable for us.

Gotcha. Out of curiosity, is this an OSS logging formatter like https://github.com/sirupsen/logrus, or is it something else? Is this something that could be solved by using https://golang.org/pkg/log/#SetOutput?

Yes, those logs. It turns out that it was bad behavior on the part of localstack plus the image lambci/lambda:go1.x. It's an issue between the workflow between these two parts. Localstack issues a docker exec command into binary wrapping the lambda handler in the lambda:go1.x image. This image will start a new process for each invocation and kill it once the lambda function completes. This makes the container start a new listener on the same hard-coded port for each invocation, where they clash under any amount of concurrency.
Rewriting the wrapper in that container solves our issue (and we're trying to push that fix upstream). I don't think this issue has anything to do with this SDK (although the visibility made possible by this change was nice).

Out of curiosity, is this an OSS logging formatter like https://github.com/sirupsen/logrus, or is it something else?

It's using gomol.

Is this something that could be solved by using https://golang.org/pkg/log/#SetOutput?

There are lots of solutions, but we were using a TCP/JSON output logger to stream logs in real time via socat (and could then grep through it via jq). We come form an environment where we were using an ELK stack when our stuff was hosted out-of-cloud.

Ah, guess I had missed the emphasis on the "local" bit. Thanks for bearing with the questions, we're always interested in the details of how our customers are developing applications.


android studio support plugin

I am using Android Studio 4.1.3, but I get this error when I run the Jetcaster project
This version of the Android Support plugin for IntelliJ IDEA (or Android Studio) cannot open this project, please retry with version 2020.3.1 or newer.

As per the README, you need to use the latest Android Studio canary.


Add reference to Pentester's Guide

This shouldn't land until after Metakitty is updated to handle a book reference correctly.

Fixes #2

Tried to assign this to Thao, but @tdoan-r7 hasn't accepted my collaboration invitation for this repo yet. :(

Dammit this PR is wrong.


Compile error with xsimd on.

Here is the traceback, unfortunately I cannot share a bigger reproducible due to proprietary code, but feel free to ask for more information.
The xtensor-dependent program compiles fine with xsimd off.
error: no viable conversion from returned value of type 'xsimd::batch_bool<float, xsimd::neon64>' to function return type 'simd_return_type<float>' (aka 'xsimd::batch<float>')

MCVE:
#include <xtensor/xtensor.hpp>

int main() {
    xt::xtensor<double, 2> a = {{1}};
    xt::xtensor<double, 2> b = {{1}};
    auto out = xt::eval(!(xt::isnan(a) | xt::isnan(b)));
}

Complete log with xsimd
```
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/habbasi/Code/xtensor-test/build
Consolidate compiler generated dependencies of target xtensor_test
[ 50%] Building CXX object CMakeFiles/xtensor_test.dir/main.cpp.o
/Users/habbasi/Code/xtensor-test/main.cpp:4:17: error: unknown type name 'float64_t'; did you mean 'float_t'?
    xt::xtensor a = {{1}};
                ^~~~~~~~~
                float_t
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/math.h:44:19: note: 'float_t' declared here
    typedef float float_t;
                  ^
/Users/habbasi/Code/xtensor-test/main.cpp:5:17: error: unknown type name 'float64_t'; did you mean 'float_t'?
    xt::xtensor b = {{1}};
                ^~~~~~~~~
                float_t
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/math.h:44:19: note: 'float_t' declared here
    typedef float float_t;
                  ^
2 errors generated.
make[2]: *** [CMakeFiles/xtensor_test.dir/main.cpp.o] Error 1
make[1]: *** [CMakeFiles/xtensor_test.dir/all] Error 2
make: *** [all] Error 2
❯ rm -rf build/
❯ CMAKE_EXPORT_COMPILE_COMMANDS=1 cmake . -B build/
-- The C compiler identification is Clang 13.0.1
-- The CXX compiler identification is Clang 13.0.1
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /Users/habbasi/mambaforge/envs/cpp/bin/arm64-apple-darwin20.0.0-clang - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /Users/habbasi/mambaforge/envs/cpp/bin/arm64-apple-darwin20.0.0-clang++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Performing Test arch_native_supported
-- Performing Test arch_native_supported - Failed
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/habbasi/Code/xtensor-test/build
❯ cmake --build build/
[ 50%] Building CXX object CMakeFiles/xtensor_test.dir/main.cpp.o
/Users/habbasi/Code/xtensor-test/main.cpp:4:17: error: unknown type name 'float64_t'; did you mean 'float_t'?
    xt::xtensor a = {{1}};
                ^~~~~~~~~
                float_t
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/math.h:44:19: note: 'float_t' declared here
    typedef float float_t;
                  ^
/Users/habbasi/Code/xtensor-test/main.cpp:5:17: error: unknown type name 'float64_t'; did you mean 'float_t'?
    xt::xtensor b = {{1}};
                ^~~~~~~~~
                float_t
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/math.h:44:19: note: 'float_t' declared here
    typedef float float_t;
                  ^
2 errors generated.
make[2]: *** [CMakeFiles/xtensor_test.dir/main.cpp.o] Error 1
make[1]: *** [CMakeFiles/xtensor_test.dir/all] Error 2
make: *** [all] Error 2
❯ cmake --build build/
Consolidate compiler generated dependencies of target xtensor_test
[ 50%] Building CXX object CMakeFiles/xtensor_test.dir/main.cpp.o
[100%] Linking CXX executable xtensor_test
[100%] Built target xtensor_test
❯ rm -rf build/
❯ CMAKE_EXPORT_COMPILE_COMMANDS=1 cmake . -B build/
-- The C compiler identification is Clang 13.0.1
-- The CXX compiler identification is Clang 13.0.1
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /Users/habbasi/mambaforge/envs/cpp/bin/arm64-apple-darwin20.0.0-clang - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /Users/habbasi/mambaforge/envs/cpp/bin/arm64-apple-darwin20.0.0-clang++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Performing Test arch_native_supported
-- Performing Test arch_native_supported - Failed
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/habbasi/Code/xtensor-test/build
❯ cmake --build build/
[ 50%] Building CXX object CMakeFiles/xtensor_test.dir/main.cpp.o
In file included from /Users/habbasi/Code/xtensor-test/main.cpp:1:
In file included from /Users/habbasi/mambaforge/envs/cpp/include/xtensor/xtensor.hpp:20:
In file included from /Users/habbasi/mambaforge/envs/cpp/include/xtensor/xcontainer.hpp:25:
In file included from /Users/habbasi/mambaforge/envs/cpp/include/xtensor/xmath.hpp:28:
In file included from /Users/habbasi/mambaforge/envs/cpp/include/xtensor/xmanipulation.hpp:13:
In file included from /Users/habbasi/mambaforge/envs/cpp/include/xtensor/xbuilder.hpp:30:
/Users/habbasi/mambaforge/envs/cpp/include/xtensor/xfunction.hpp:772:16: error: no viable conversion from returned value of type 'xsimd::batch_bool' to function return type 'simd_return_type' (aka 'xsimd::batch')
        return load_simd_impl(std::make_index_sequence(), i);
               ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/Users/habbasi/mambaforge/envs/cpp/include/xtensor/xfunction.hpp:830:23: note: in instantiation of function template specialization 'xt::xfunction>, 2, xt::layout_type::row_major> &>, xt::xfunction>, 2, xt::layout_type::row_major> &>>::load_simd' requested here
            .template load_simd(i))...);
                      ^
/Users/habbasi/mambaforge/envs/cpp/include/xtensor/xassign.hpp:686:67: note: in instantiation of function template specialization 'xt::xfunction>, 2, xt::layout_type::row_major> &>, xt::xfunction>, 2, xt::layout_type::row_major> &>>>::load_simd' requested here
            e1.template store_simd(i, e2.template load_simd(i));
                                                                  ^
/Users/habbasi/mambaforge/envs/cpp/include/xtensor/xassign.hpp:391:47: note: in instantiation of function template specialization 'xt::linear_assigner::run>, 2, xt::layout_type::row_major>, xt::xfunction>, 2, xt::layout_type::row_major> &>, xt::xfunction>, 2, xt::layout_type::row_major> &>>>>' requested here
                linear_assigner::run(de1, de2);
                                              ^
/Users/habbasi/mambaforge/envs/cpp/include/xtensor/xassign.hpp:414:20: note: in instantiation of function template specialization 'xt::xexpression_assigner_base::assign_data>, 2, xt::layout_type::row_major>, xt::xfunction>, 2, xt::layout_type::row_major> &>, xt::xfunction>, 2, xt::layout_type::row_major> &>>>>' requested here
        base_type::assign_data(e1, e2, trivial_broadcast);
                   ^
/Users/habbasi/mambaforge/envs/cpp/include/xtensor/xassign.hpp:205:40: note: in instantiation of function template specialization 'xt::xexpression_assigner::assign_xexpression>, 2, xt::layout_type::row_major>>, xt::xexpression>, 2, xt::layout_type::row_major> &>, xt::xfunction>, 2, xt::layout_type::row_major> &>>>>>' requested here
            xexpression_assigner::assign_xexpression(e1, e2);
                                       ^
/Users/habbasi/mambaforge/envs/cpp/include/xtensor/xsemantic.hpp:623:13: note: in instantiation of function template specialization 'xt::assign_xexpression>, 2, xt::layout_type::row_major>, xt::xfunction>, 2, xt::layout_type::row_major> &>, xt::xfunction>, 2, xt::layout_type::row_major> &>>>>' requested here
        xt::assign_xexpression(*this, e);
            ^
/Users/habbasi/mambaforge/envs/cpp/include/xtensor/xsemantic.hpp:489:37: note: in instantiation of function template specialization 'xt::xcontainer_semantic>, 2, xt::layout_type::row_major>>::assign_xexpression>, 2, xt::layout_type::row_major> &>, xt::xfunction>, 2, xt::layout_type::row_major> &>>>>' requested here
        return this->derived_cast().assign_xexpression(e);
                                    ^
/Users/habbasi/mambaforge/envs/cpp/include/xtensor/xtensor.hpp:555:24: note: in instantiation of function template specialization 'xt::xsemantic_base>, 2, xt::layout_type::row_major>>::assign>, 2, xt::layout_type::row_major> &>, xt::xfunction>, 2, xt::layout_type::row_major> &>>>>' requested here
        semantic_base::assign(e);
                       ^
/Users/habbasi/mambaforge/envs/cpp/include/xtensor/xeval.hpp:48:16: note: in instantiation of function template specialization 'xt::xtensor_container>, 2, xt::layout_type::row_major>::xtensor_container>, 2, xt::layout_type::row_major> &>, xt::xfunction>, 2, xt::layout_type::row_major> &>>>>' requested here
        return std::forward(t);
               ^
/Users/habbasi/Code/xtensor-test/main.cpp:6:20: note: in instantiation of function template specialization 'xt::eval>, 2, xt::layout_type::row_major> &>, xt::xfunction>, 2, xt::layout_type::row_major> &>>>>' requested here
    auto out = xt::eval(!(xt::isnan(a) | xt::isnan(b)));
                   ^
/Users/habbasi/mambaforge/envs/cpp/include/xsimd/config/../types/../types/xsimd_batch.hpp:46:11: note: candidate constructor (the implicit copy constructor) not viable: no known conversion from 'xsimd::batch_bool' to 'const xsimd::batch &' for 1st argument
    class batch : public types::simd_register
          ^
/Users/habbasi/mambaforge/envs/cpp/include/xsimd/config/../types/../types/xsimd_batch.hpp:46:11: note: candidate constructor (the implicit move constructor) not viable: no known conversion from 'xsimd::batch_bool' to 'xsimd::batch &&' for 1st argument
/Users/habbasi/mambaforge/envs/cpp/include/xsimd/config/../types/../types/xsimd_batch.hpp:58:9: note: candidate constructor not viable: no known conversion from 'xsimd::batch_bool' to 'double' for 1st argument
        batch(T val) noexcept;
        ^
/Users/habbasi/mambaforge/envs/cpp/include/xsimd/config/../types/../types/xsimd_batch.hpp:59:9: note: candidate constructor not viable: no known conversion from 'xsimd::batch_bool' to 'std::initializer_list' for 1st argument
        batch(std::initializer_list data) noexcept;
        ^
/Users/habbasi/mambaforge/envs/cpp/include/xsimd/config/../types/../types/xsimd_batch.hpp:61:9: note: candidate constructor not viable: no known conversion from 'xsimd::batch_bool' to 'xsimd::batch::register_type' (aka 'float64x2_t') for 1st argument
        batch(register_type reg) noexcept;
        ^
/Users/habbasi/mambaforge/envs/cpp/include/xsimd/config/../types/../types/xsimd_batch.hpp:60:18: note: explicit constructor is not a candidate
        explicit batch(batch_bool_type const& b) noexcept;
                 ^
/Users/habbasi/mambaforge/envs/cpp/include/xsimd/arch/../types/xsimd_neon_register.hpp:101:9: note: candidate function
        XSIMD_DECLARE_SIMD_REGISTER(unsigned long long int, neon, detail::neon_vector_type);
        ^
/Users/habbasi/mambaforge/envs/cpp/include/xsimd/config/../types/xsimd_register.hpp:41:9: note: expanded from macro 'XSIMD_DECLARE_SIMD_REGISTER'
        operator register_type() const noexcept { return data; }   \
        ^
In file included from /Users/habbasi/Code/xtensor-test/main.cpp:1:
In file included from /Users/habbasi/mambaforge/envs/cpp/include/xtensor/xtensor.hpp:20:
In file included from /Users/habbasi/mambaforge/envs/cpp/include/xtensor/xcontainer.hpp:25:
In file included from /Users/habbasi/mambaforge/envs/cpp/include/xtensor/xmath.hpp:28:
In file included from /Users/habbasi/mambaforge/envs/cpp/include/xtensor/xmanipulation.hpp:13:
In file included from /Users/habbasi/mambaforge/envs/cpp/include/xtensor/xbuilder.hpp:30:
/Users/habbasi/mambaforge/envs/cpp/include/xtensor/xfunction.hpp:1045:16: error: no viable conversion from returned value of type 'xsimd::batch_bool' to function return type 'simd_return_type' (aka 'xsimd::batch')
        return (p_f->m_f.simd_apply)(std::get(m_st). template step_simd()...);
               ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/Users/habbasi/mambaforge/envs/cpp/include/xtensor/xfunction.hpp:1052:16: note: in instantiation of function template specialization 'xt::xfunction_stepper>, 2, xt::layout_type::row_major> &>, xt::xfunction>, 2, xt::layout_type::row_major> &>>::step_simd_impl' requested here
        return step_simd_impl(std::make_index_sequence());
               ^
/Users/habbasi/mambaforge/envs/cpp/include/xtensor/xfunction.hpp:1045:66: note: in instantiation of function template specialization 'xt::xfunction_stepper>, 2, xt::layout_type::row_major> &>, xt::xfunction>, 2, xt::layout_type::row_major> &>>::step_simd' requested here
        return (p_f->m_f.simd_apply)(std::get(m_st). template step_simd()...);
                                                                 ^
/Users/habbasi/mambaforge/envs/cpp/include/xtensor/xfunction.hpp:1052:16: note: in instantiation of function template specialization 'xt::xfunction_stepper>, 2, xt::layout_type::row_major> &>, xt::xfunction>, 2, xt::layout_type::row_major> &>>>::step_simd_impl' requested here
        return step_simd_impl(std::make_index_sequence());
               ^
/Users/habbasi/mambaforge/envs/cpp/include/xtensor/xassign.hpp:972:61: note: in instantiation of function template specialization 'xt::xfunction_stepper>, 2, xt::layout_type::row_major> &>, xt::xfunction>, 2, xt::layout_type::row_major> &>>>::step_simd' requested here
                res_stepper.store_simd(fct_stepper.template step_simd());
                                                            ^
/Users/habbasi/mambaforge/envs/cpp/include/xtensor/xassign.hpp:400:57: note: in instantiation of function template specialization 'xt::strided_loop_assigner::run>, 2, xt::layout_type::row_major>, xt::xfunction>, 2, xt::layout_type::row_major> &>, xt::xfunction>, 2, xt::layout_type::row_major> &>>>>' requested here
            strided_loop_assigner::run(de1, de2);
                                                        ^
/Users/habbasi/mambaforge/envs/cpp/include/xtensor/xassign.hpp:414:20: note: (skipping 2 contexts in backtrace; use -ftemplate-backtrace-limit=0 to see all)
        base_type::assign_data(e1, e2, trivial_broadcast);
                   ^
/Users/habbasi/mambaforge/envs/cpp/include/xtensor/xsemantic.hpp:623:13: note: in instantiation of function template specialization 'xt::assign_xexpression>, 2, xt::layout_type::row_major>, xt::xfunction>, 2, xt::layout_type::row_major> &>, xt::xfunction>, 2, xt::layout_type::row_major> &>>>>' requested here
        xt::assign_xexpression(*this, e);
            ^
/Users/habbasi/mambaforge/envs/cpp/include/xtensor/xsemantic.hpp:489:37: note: in instantiation of function template specialization 'xt::xcontainer_semantic>, 2, xt::layout_type::row_major>>::assign_xexpression>, 2, xt::layout_type::row_major> &>, xt::xfunction>, 2, xt::layout_type::row_major> &>>>>' requested here
        return this->derived_cast().assign_xexpression(e);
                                    ^
/Users/habbasi/mambaforge/envs/cpp/include/xtensor/xtensor.hpp:555:24: note: in instantiation of function template specialization 'xt::xsemantic_base>, 2, xt::layout_type::row_major>>::assign>, 2, xt::layout_type::row_major> &>, xt::xfunction>, 2, xt::layout_type::row_major> &>>>>' requested here
        semantic_base::assign(e);
                       ^
/Users/habbasi/mambaforge/envs/cpp/include/xtensor/xeval.hpp:48:16: note: in instantiation of function template specialization 'xt::xtensor_container>, 2, xt::layout_type::row_major>::xtensor_container>, 2, xt::layout_type::row_major> &>, xt::xfunction>, 2, xt::layout_type::row_major> &>>>>' requested here
        return std::forward(t);
               ^
/Users/habbasi/Code/xtensor-test/main.cpp:6:20: note: in instantiation of function template specialization 'xt::eval>, 2, xt::layout_type::row_major> &>, xt::xfunction>, 2, xt::layout_type::row_major> &>>>>' requested here
    auto out = xt::eval(!(xt::isnan(a) | xt::isnan(b)));
                   ^
/Users/habbasi/mambaforge/envs/cpp/include/xsimd/config/../types/../types/xsimd_batch.hpp:46:11: note: candidate constructor (the implicit copy constructor) not viable: no known conversion from 'xsimd::batch_bool' to 'const xsimd::batch &' for 1st argument
    class batch : public types::simd_register
          ^
/Users/habbasi/mambaforge/envs/cpp/include/xsimd/config/../types/../types/xsimd_batch.hpp:46:11: note: candidate constructor (the implicit move constructor) not viable: no known conversion from 'xsimd::batch_bool' to 'xsimd::batch &&' for 1st argument
/Users/habbasi/mambaforge/envs/cpp/include/xsimd/config/../types/../types/xsimd_batch.hpp:58:9: note: candidate constructor not viable: no known conversion from 'xsimd::batch_bool' to 'double' for 1st argument
        batch(T val) noexcept;
        ^
/Users/habbasi/mambaforge/envs/cpp/include/xsimd/config/../types/../types/xsimd_batch.hpp:59:9: note: candidate constructor not viable: no known conversion from 'xsimd::batch_bool' to 'std::initializer_list' for 1st argument
        batch(std::initializer_list data) noexcept;
        ^
/Users/habbasi/mambaforge/envs/cpp/include/xsimd/config/../types/../types/xsimd_batch.hpp:61:9: note: candidate constructor not viable: no known conversion from 'xsimd::batch_bool' to 'xsimd::batch::register_type' (aka 'float64x2_t') for 1st argument
        batch(register_type reg) noexcept;
        ^
/Users/habbasi/mambaforge/envs/cpp/include/xsimd/config/../types/../types/xsimd_batch.hpp:60:18: note: explicit constructor is not a candidate
        explicit batch(batch_bool_type const& b) noexcept;
                 ^
/Users/habbasi/mambaforge/envs/cpp/include/xsimd/arch/../types/xsimd_neon_register.hpp:101:9: note: candidate function
        XSIMD_DECLARE_SIMD_REGISTER(unsigned long long int, neon, detail::neon_vector_type);
        ^
/Users/habbasi/mambaforge/envs/cpp/include/xsimd/config/../types/xsimd_register.hpp:41:9: note: expanded from macro 'XSIMD_DECLARE_SIMD_REGISTER'
        operator register_type() const noexcept { return data; }   \
        ^
2 errors generated.
make[2]: *** [CMakeFiles/xtensor_test.dir/main.cpp.o] Error 1
make[1]: *** [CMakeFiles/xtensor_test.dir/all] Error 2
make: *** [all] Error 2
```

Compiles fine without xsimd.

Godbolt link: https://godbolt.org/z/vj4fEGzqq

Wihch version of xtensor and xsimd do you use? With xtensor 0.24.2 (and master), and xsimd 8.1.0, everything builds and works fine.

Then I think the issue may be specific to ARM64/Apple Silicon. I'm on those exact versions, I'm uploading a project including an environment.yml and a ./reproduce.sh.
xtensor-test.zip

I have verified this isn't an issue on Linux/AMD64.

I think I'm seeing the same thing with https://github.com/facebookincubator/velox
FAILED: velox/type/CMakeFiles/velox_type.dir/Filter.cpp.o
ccache /usr/bin/c++ -DBOOST_ALL_NO_LIB -DBOOST_ATOMIC_DYN_LINK -DBOOST_CONTEXT_DYN_LINK -DBOOST_FILESYSTEM_DYN_LINK -DBOOST_PROGRAM_OPTIONS_DYN_LINK -DBOOST_REGEX_DYN_LINK -DBOOST_SYSTEM_DYN_LINK -DBOOST_THREAD_DYN_LINK -DFMT_LOCALE -DGFLAGS_IS_A_DLL=0 -I/home/ubuntu/velox/. -I/home/ubuntu/velox/velox/external/xxhash -I/home/ubuntu/velox/third_party/xsimd/include -isystem /home/ubuntu/velox/velox -isystem /home/ubuntu/velox/velox/external -isystem /home/ubuntu/velox/velox/external/duckdb -isystem /home/ubuntu/velox/velox/external/duckdb/tpch/dbgen/include -mcpu=native -std=c++17 -Wno-psabi -D USE_VELOX_COMMON_BASE -D HAS_UNCAUGHT_EXCEPTIONS -O3 -DNDEBUG -std=gnu++17 -MD -MT velox/type/CMakeFiles/velox_type.dir/Filter.cpp.o -MF velox/type/CMakeFiles/velox_type.dir/Filter.cpp.o.d -o velox/type/CMakeFiles/velox_type.dir/Filter.cpp.o -c /home/ubuntu/velox/velox/type/Filter.cpp
/home/ubuntu/velox/velox/type/Filter.cpp: In member function ‘virtual xsimd::batch_bool<long int> facebook::velox::common::BigintValuesUsingHashTable::testValues(xsimd::batch<long int>) const’:
/home/ubuntu/velox/velox/type/Filter.cpp:219:57: error: no matching function for call to ‘xsimd::batch<long unsigned int>::batch(xsimd::batch<long int>&)’
  219 |   xsimd::batch<int64_t> indices(xsimd::batch<uint64_t>(x) * M & sizeMask_);
      |                                                         ^
In file included from /home/ubuntu/velox/third_party/xsimd/include/xsimd/xsimd.hpp:61,
                 from /home/ubuntu/velox/./velox/common/base/SimdUtil.h:24,
                 from /home/ubuntu/velox/./velox/type/Filter.h:28,
                 from /home/ubuntu/velox/velox/type/Filter.cpp:25:
/home/ubuntu/velox/third_party/xsimd/include/xsimd/types/xsimd_batch.hpp:49:9: note: candidate: ‘template<class ... Ts> xsimd::batch<T, A>::batch(T, T, Ts ...) [with Ts = {Ts ...}; T = long unsigned int; A = xsimd::neon64]’
   49 |         batch(T val0, T val1, Ts... vals) noexcept;
      |         ^~~~~
/home/ubuntu/velox/third_party/xsimd/include/xsimd/types/xsimd_batch.hpp:49:9: note:   template argument deduction/substitution failed:
/home/ubuntu/velox/velox/type/Filter.cpp:219:57: note:   candidate expects at least 2 arguments, 1 provided
  219 |   xsimd::batch<int64_t> indices(xsimd::batch<uint64_t>(x) * M & sizeMask_);
      |                                                         ^
In file included from /home/ubuntu/velox/third_party/xsimd/include/xsimd/xsimd.hpp:61,
                 from /home/ubuntu/velox/./velox/common/base/SimdUtil.h:24,
                 from /home/ubuntu/velox/./velox/type/Filter.h:28,
                 from /home/ubuntu/velox/velox/type/Filter.cpp:25:
/home/ubuntu/velox/third_party/xsimd/include/xsimd/types/xsimd_batch.hpp:467:12: note: candidate: ‘xsimd::batch<T, A>::batch(xsimd::batch<T, A>::register_type) [with T = long unsigned int; A = xsimd::neon64; xsimd::batch<T, A>::register_type = std::conditional<false, __Int64x2_t, __Uint64x2_t>::type]’
  467 |     inline batch<T, A>::batch(register_type reg) noexcept
      |            ^~~~~~~~~~~
/home/ubuntu/velox/third_party/xsimd/include/xsimd/types/xsimd_batch.hpp:467:45: note:   no known conversion for argument 1 from ‘xsimd::batch<long int>’ to ‘xsimd::batch<long unsigned int>::register_type’ {aka ‘std::conditional<false, __Int64x2_t, __Uint64x2_t>::type’}
  467 |     inline batch<T, A>::batch(register_type reg) noexcept
      |                               ~~~~~~~~~~~~~~^~~
/home/ubuntu/velox/third_party/xsimd/include/xsimd/types/xsimd_batch.hpp:457:12: note: candidate: ‘xsimd::batch<T, A>::batch(const xsimd::batch_bool<T, A>&) [with T = long unsigned int; A = xsimd::neon64]’
  457 |     inline batch<T, A>::batch(batch_bool<T, A> const& b) noexcept
      |            ^~~~~~~~~~~
/home/ubuntu/velox/third_party/xsimd/include/xsimd/types/xsimd_batch.hpp:457:55: note:   no known conversion for argument 1 from ‘xsimd::batch<long int>’ to ‘const xsimd::batch_bool<long unsigned int, xsimd::neon64>&’
  457 |     inline batch<T, A>::batch(batch_bool<T, A> const& b) noexcept
      |                               ~~~~~~~~~~~~~~~~~~~~~~~~^
/home/ubuntu/velox/third_party/xsimd/include/xsimd/types/xsimd_batch.hpp:434:12: note: candidate: ‘xsimd::batch<T, A>::batch(T) [with T = long unsigned int; A = xsimd::neon64]’
  434 |     inline batch<T, A>::batch(T val) noexcept
      |            ^~~~~~~~~~~
/home/ubuntu/velox/third_party/xsimd/include/xsimd/types/xsimd_batch.hpp:434:33: note:   no known conversion for argument 1 from ‘xsimd::batch<long int>’ to ‘long unsigned int’
  434 |     inline batch<T, A>::batch(T val) noexcept
      |                               ~~^~~
In file included from /home/ubuntu/velox/third_party/xsimd/include/xsimd/xsimd.hpp:61,
                 from /home/ubuntu/velox/./velox/common/base/SimdUtil.h:24,
                 from /home/ubuntu/velox/./velox/type/Filter.h:28,
                 from /home/ubuntu/velox/velox/type/Filter.cpp:25:
/home/ubuntu/velox/third_party/xsimd/include/xsimd/types/xsimd_batch.hpp:46:9: note: candidate: ‘xsimd::batch<T, A>::batch() [with T = long unsigned int; A = xsimd::neon64]’
   46 |         batch() = default; ///< Create a batch initialized with undefined values.
      |         ^~~~~
/home/ubuntu/velox/third_party/xsimd/include/xsimd/types/xsimd_batch.hpp:46:9: note:   candidate expects 0 arguments, 1 provided
In file included from /home/ubuntu/velox/third_party/xsimd/include/xsimd/xsimd.hpp:61,
                 from /home/ubuntu/velox/./velox/common/base/SimdUtil.h:24,
                 from /home/ubuntu/velox/./velox/type/Filter.h:28,
                 from /home/ubuntu/velox/velox/type/Filter.cpp:25:
/home/ubuntu/velox/third_party/xsimd/include/xsimd/types/xsimd_batch.hpp:35:11: note: candidate: ‘constexpr xsimd::batch<long unsigned int>::batch(const xsimd::batch<long unsigned int>&)’
   35 |     class batch : public types::simd_register<T, A>
      |           ^~~~~
/home/ubuntu/velox/third_party/xsimd/include/xsimd/types/xsimd_batch.hpp:35:11: note:   no known conversion for argument 1 from ‘xsimd::batch<long int>’ to ‘const xsimd::batch<long unsigned int>&’
/home/ubuntu/velox/third_party/xsimd/include/xsimd/types/xsimd_batch.hpp:35:11: note: candidate: ‘constexpr xsimd::batch<long unsigned int>::batch(xsimd::batch<long unsigned int>&&)’
/home/ubuntu/velox/third_party/xsimd/include/xsimd/types/xsimd_batch.hpp:35:11: note:   no known conversion for argument 1 from ‘xsimd::batch<long int>’ to ‘xsimd::batch<long unsigned int>&&’

Solution: build with -flax-vector-conversions -fsigned-char
AVX uses the same register types for signed or unsigned types, e.g:

Would it be possible then to add these flags in the CMakeLists.txt? I'll try to find out over the weekend where and add them myself.

flax-vector-conversions should definitely be avoided as it allows hidden implicit conversions that should not occur. batch_cast should be used to fix this kind of issue.

-flax-vector-conversions is just a work-around.  The "right answer" is to use the appropriate vreinterpretq calls when performing operations like
    xsimd::batch<int64_t> x = {1, 2};
    // This fails to compile with ARM64 gcc 12.1
    // but it succeeds with x86-64 gcc 12.1
    xsimd::batch<uint64_t> y(x);

I started playing around in xsimd_neon_register.hpp to see if there was a way to do this at the register level, but it might have to happen in batch.  This is more a job for someone with xsimd experience, and I'm just a neon nerd.

@jlinford we have xsimd::bitwise_cast<T0>(xsimd::batch<T1,A>) to perform a reinterpret_cast between two different types. But when you just want to do a static_cast (which does not imply a vreinterpretq but ana ctual conversion). I checked and we don't have that particular cast, but we should, I'll add this.


download info failed

Using this program, everything works:
const GooglePlayAPI = require('@gonetone/google-play-api');
var gpAPI = new<EMAIL_ADDRESS>"33783f43efa...");

gpAPI.getGoogleToken(
   'oauth2_4/0AX4XfWitLKPqHVx072mfsTFrsGTv6u1HMFIRxHBh4tpGftkbJXc3GHVI7zV_t...',
   'token.txt'
).then(async (token) => {
   await gpAPI.googleAuth(token);
   const downloadInfo = await gpAPI.downloadInfo('com.github.android');
   console.log(downloadInfo);
});

but if I change the app, it fails:
Error: Get "air.ITVMobilePlayer" download info failed, may be the device does not support.
    at GooglePlayAPI.downloadInfo (D:\Desktop\etc\node_modules\@gonetone\google-play-api\src\GooglePlayAPI.js:386:11)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async D:\Desktop\etc\play.js:17:25

its a valid app:
https://play.google.com/store/apps/details?id=air.ITVMobilePlayer

Device does not support.
Please use a supported device to get a GSF ID and try again.


Dashboards V2

We are doing a major Dashboard revamp

Dashboard should have specified time windows. (This should show 3-4pm)

Querying should all be moved to GraphQL and should allow custom group by keys
https://github.com/Helicone/helicone/issues/295

https://github.com/Helicone/helicone/issues/715

p75 and p90
https://github.com/Helicone/helicone/issues/699


Updating touch-policy to "cached"&pin-policy to "once": no effect on existing slots

Environment

OS: macOS ventura
age-plugin-yubikey version: 0.3.2

What were you trying to do
I used age-plugin-yubikey --pin-policy once --touch-policy cached to reduce the number of times I have to provide my PIN and touch the key (especially for rekeying secrets with https://github.com/ryantm/agenix).
What happened
:;    age-plugin-yubikey --pin-policy once --touch-policy cached
✨ Let's get your YubiKey set up for age! ✨

This tool can create a new age identity in a free slot of your YubiKey.
It will generate an identity file that you can use with an age client,
along with the corresponding recipient. You can also do this directly
with:
    age-plugin-yubikey --generate

If you are already using a YubiKey with age, you can select an existing
slot to recreate its corresponding identity file and recipient.

When asked below to select an option, use the up/down arrow keys to
make your choice, or press [Esc] or [q] to quit.

🔑 Select a YubiKey: Yubico YubiKey FIDO+CCID (Serial: 6900832)
🕳  Select a slot for your age identity: Slot 1 (asf personal 4c, created: Sun, 05 Jun 2022 14:25:06 +0000)
Use existing identity in slot 1? yes

📝 File name to write this identity to: age-yubikey-identity-18fdcdd5.txt
File exists. Overwrite it? yes

✅ Done! This YubiKey identity is ready to go.

🔑 Here's the corresponding YubiKey recipient:
  age1yubikey1q0m3cprmex0d2thamkx9pwvhsqwcm2gel50rdxmeclvkgqtem9dxz4smpwv

Here are some example things you can do with it:

- Encrypt a file to this identity:
  $ cat foo.txt | rage -r age1yubikey1q0m3cprmex0d2thamkx9pwvhsqwcm2gel50rdxmeclvkgqtem9dxz4smpwv -o foo.txt.age

- Decrypt a file with this identity:
  $ cat foo.txt.age | rage -d -i age-yubikey-identity-18fdcdd5.txt > foo.txt

- Recreate the identity file:
  $ age-plugin-yubikey -i --serial 6900832 --slot 1 > age-yubikey-identity-18fdcdd5.txt

- Recreate the recipient:
  $ age-plugin-yubikey -l --serial 6900832 --slot 1

💭 Remember: everything breaks, have a backup plan for when this YubiKey does.

and then,
:;    cat age-yubikey-identity-18fdcdd5.txt
#       Serial: 6900832, Slot: 1
#         Name: asf personal 4c
#      Created: Sun, 05 Jun 2022 14:25:06 +0000
#   PIN policy: Once   (A PIN is required once per session, if set)
# Touch policy: Always (A physical touch is required for every decryption)
#    Recipient: age1yubikey1q0m3cprmex0d2thamkx9pwvhsqwcm2gel50rdxmeclvkgqtem9dxz4smpwv
AGE-PLUGIN-YUBIKEY-1VPXXJQYZRR7UM4GESUNXL
:;    rage -i identities/personal-4c.txt -d some-secret.age
Enter PIN for YubiKey with serial 6900832:
yay!
:;    rage -i identities/personal-4c.txt -d some-secret.age
Enter PIN for YubiKey with serial 6900832:
yay!

Note that each PIN prompt also requires a touch of the yubikey. I'm not sure what's going on here, but my hypothesis is that existing slots' settings can't be changed? Is that right?

There is no way in this plugin to alter the PIN or touch policies of an existing YubiKey slot. I don't recall if that is because it is simply not possible for YubiKeys, or only because the yubikey crate doesn't implement the capability; that would need to be investigated.
If you select an existing slot via the text UI, the existing identity in that slot gets its identity file recreated (equivalent to age-plugin-yubikey --identity). Selecting an empty slot is equivalent to age-plugin-yubikey --generate; the latter programmatic command can be used to overwrite a slot with a fresh identity.

You certainly can't change the policy to a laxer one, otherwise an attacker would just do that, and bypass the point of e.g. always having to touch the key to do an operation.

OK, I think then I'll add a laxer yubikey slot. I was surprised by this because gpg --card-edit does let you change this, IIRC - I guess there, in order to go from strict to laxer, you still have to authenticate once.

So, update: I generated a new age key in a second slot, with --pin-policy=once --touch-policy=cached and that seems to work... half-way: It doesn't require a touch on every decryption (only once every 15s, as advertised!), but still prompts for a PIN every time I run rage -d. Is that intentional?
I'm only running rage -d twice in a row, not touching the yubikey outside the initial touch; if I'm reading the "Agent" section of the readme right, this ought to only prompt once?

You're reading the "Agent" section for current main, not for 0.3.2. Until a few days ago, that section documented that "once" policies were treated like "always" policies, but we've managed to fix this (at least partially; there still seems to be some kind of cache eviction going on that we don't fully understand). Feel free to try installing current main to test and give feedback 🙂

You're reading the "Agent" section for current main, not for 0.3.2.

Ooooooops 😓
I did try with latest main just now, and same result: It prompts for the PIN again, despite the slot being configured as follows:
#       Serial: 6900832, Slot: 3
#         Name: age identity d96b79b2
#      Created: Thu, 05 Jan 2023 21:54:16 +0000
#   PIN policy: Once   (A PIN is required once per session, if set)
# Touch policy: Cached (A physical touch is required for decryption, and is cached for 15 seconds)
age1yubikey1qt6xfjt40g22etwzkeyzakhe23k9hjfgjef78nhzn8wwtqsj5qfl2zlrdnr

The touch cache works, just as it did on 0.3.2.

We've found and fixed several bugs in the initial implementation of the PIN cache, and at least one more needs to be fixed by making changes to the yubikey crate. I'm hoping to get enough fixes into the 0.4.0 release that it is reliable enough to leave documented, so I expect to be publishing an RC or two for people to test.

Ahhh, I just read through https://github.com/str4d/age-plugin-yubikey/commit/d9c5abaa15d00694a0127873aa39cf6be30d39ca#diff-f8cca32b0c315d712da3375dab4d6c52abd816618850f4c0ce5578d35efe7deaR79 and now I get why it keeps asking me the PIN every time: I am using a yubikey4c nano.
Does the 5 series fare better / do the PINs get cached there?

Ah yep, the YubiKey 4 problem is a recent discovery.

Does the 5 series fare better / do the PINs get cached there?

As far as I can tell, yes. We had some initial bugs as I mentioned above, but it seems to be working pretty well now (I haven't been receiving bug reports for YubiKey 5s, and IIRC those who reported previous bugs have said they are now fixed).


$joinEager does not using join at total data

when using $joinEager count doesnt use join in total (count) but when im using $joinRelation the count is using inner join. Is this bug? it always give me error when i tried filter from $eager column

This is the expected behavior. what are you trying to achieve and what errors do you get? please share detailed examples.

heres my table

and heres my model (kamus)
static get relationMappings() {
const tipeKamusModel = require("./tipe-kamus.model");
const subKamusModel = require("./sub-kamus.model");
const skpdModel = require("./skpd.model");
return {
  tipeKamus: {
    modelClass: tipeKamusModel,
    relation: Model.BelongsToOneRelation,
    join: {
      from: "kom.kamus.tipe_kamus_id",
      to: "kom.tipe_kamus.id"
    }
  },
  subKamus: {
    modelClass: subKamusModel,
    relation: Model.HasManyRelation,
    join: {
      from: "kom.kamus.id",
      to: "kom.sub_kamus.kamus_id"
    }
  },
  skpd: {
    modelClass: skpdModel,
    relation: Model.BelongsToOneRelation,
    join: {
      from: "kom.kamus.skpd_id",
      to: "ref.skpd.id"
    }
  }
};

}
my goal is to get data where kamus.skpd_id ilike 'something%' but include tipeKamus and skpd and kamus.skpd_id is null of course.
my current query is {
$eager : '[tipeKamus]',
$joinEager : '[tipeKamus,skpd],
'skpd.id' : {
$ilike : 'something%',
}
}
it giving me error because in count there isnt column skpd.id
well what a shame i got a solution.. now its working. im sorry
{
$eager: '[tipeKamus, skpd]',
$joinEager: '[tipeKamus,skpd]',
$or: [
{
skpd_id: {
$null: true,
},
},
{
skpd_id: {
$ilike: 123%,
},
},
],
}


how to make the font italic

How to make the font italic?

@Chinazhouwy That is controlled by themes. i.e. themes have the ability to select certain scopes and give the text color, boldness or italics. You can create a custom theme either by starting from scratch or from taking an existing theme and customizing it.
Further information in the docs page: https://code.visualstudio.com/docs/customization/themes#_creating-a-custom-theme


VM: Implement left-to-right evaluation order for instance method invocations

This is the VM specific subissue for #36744, which has the details.

Please don't start work on this until we have resolved any breaking change issues.

Closing: The language specification is being changed to specify the current behavior, so no implementation work is needed.


Capturing a simple application using glBlitFramebuffer() behaves strangely

Description
Capturing a simple application using glBlitFramebuffer() behaves strangely, specifically the texture viewer.
A capture of frame №0: has a backbuffer with a size of 1×1, only capturing the pixel at (0, 0), the renderbuffer with the gradient does get shown correctly though.
Captures of frames after: the backbuffer is the expected size, but if you look at the renderbuffer the gradient is stored in, it appears completely black, despite not having been glClear()ed.  The backbuffer is also completely black.
I’m not certain whether this is a bug, or me misunderstanding some aspect of framebuffers or renderbuffers; but visually, the program works as intended, showing the gradient square scaled 10×, so it seems like a bug.
Steps to reproduce
Compile the attached C program as cc $(pkg-config epoxy --cflags --libs) $(sdl2-config --cflags --libs) -o renderdoc-glblitframebuffer-mve main.c, and capture it (there is a .cap JSON included, only needs the path to the executable to be modified).  There are also captures of the first 8 frames attached.
renderdoc-glblitframebuffer-mve.zip
Environment

RenderDoc version: v1.21
Operating System: Void Linux
Graphics API: OpenGL
GPU: AMD Radeon RX 6600 XT
Driver: Mesa 22.1.7
glxinfo -t -l -s output: Attached

I don't have a linux machine running to test this immediately but I believe I can see the problem looking at the source.
The backbuffer being 1x1 in frame 0's capture is expected, because GL is a badly designed API and the backbuffer is not very well defined. RenderDoc treats the backbuffer as being resized only at the time a present happens to 'snapshot' the association between a window and a context, since otherwise the backbuffer could change size mid-capture which is not feasible or useful to support. Since frame 0 is everything up to the first present, there's nothing to snapshot so the backbuffer is empty.
The renderbuffers are black because RenderDoc does not save the contents of renderbuffers from previous frames for subsequent frames. This is because renderbuffers in general are a pointless legacy holdover from decades ago before render-to-texture existed, and now the only realistic use is for framebuffer attachments which explicitly never need to be sampled from (basically just depth textures when only depth testing is needed). It is possible to read from them as your sample proves, but this is awkward and not worth supporting in RenderDoc.
Tomorrow I can verify this is definitely accurate since as I say it's only from a surface reading of the small sample source.

That seems to be correct, if I change rbo_colour to a texture it works fine, e.g. with:
glGenTextures(1, &rbo_colour);
glBindTexture(GL_TEXTURE_2D, rbo_colour);
glTexImage2D(GL_TEXTURE_2D, 0, GL_SRGB8, 100, 100, 0, GL_RGB, GL_UNSIGNED_BYTE, NULL);
// …
glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D, rbo_colour, 0);

That’s unfortunate but understandable, from what I understand about renderbuffers (mostly just the Khronos OpenGL wiki’s page on them, https://www.khronos.org/opengl/wiki/Renderbuffer_Object), considering I don’t need to sample it from a shader in this case, it seems like the more “correct” & possibly slightly more performant(?) option; but I can just add an option --no-renderbuffers to my game that uses textures instead for debugging, since apart from that they’re interchangeable.  At some point I will probably add Vulkan too, but one of the things I would like to try to support is at least slightly older hardware; e.g. until recently my main PC was ~10 years old with a GTX 750 Ti (I think) with no Vulkan support (or well, the proprietary drivers did IIRC, but they stopped working for some reason), but otherwise worked fine.
Thank you for responding so quickly, too.

As far as I'm aware the distinction is not significant and drivers do not do anything drastic with it, if they do anything at all. Especially since the blit you are doing is a scaling blit which likely means it's going to run through a shader and be sampled anyway, there's not likely to be anything a driver can do with the fact that 'technically' these can't be sampled from (except when they can).
Renderbuffers work fine in RenderDoc if you're using them as temporary storage - only if the contents are generated within the frame you're capturing. Generating them once and holding onto them indefinitely won't work as you've found, but while a niche number of GL programs use renderbuffers I have never seen a program that uses them for anything but discardable rendertargets, usually depth or maybe MSAA before a resolve.
The display of No Resource is an inevitable result of again OpenGL's terrible API design. At the time the parameter is serialised, it's serialised as no binding because it's '0', and this is only later replaced and identified as referring to the current backbuffer (which depends on the current context at the time).
Closing this then since it seems to be working as expected, and there are no plans to implement renderbuffer preservation.


Fix error when input is number

If input data is number, then app crached with error:
Error: There is no such converter: undefined

Numbers should be processed as strings.

Thanks! Looks good! :sparkle:
Will merge it when I merge #9. :+1:


sectionExpanded listener

Hi,
I have an android app using the sectioned-recyclerview and I was wondering if there was something like a listener to check if the section is expanded or not? I have an arrow in my header xml which I want to point down when the section is expanded and up when the section is not exanded.

n.v.m. I found it in your sample. Didn't find it at start :P


Add audio to text

Add audio to text to all.yaml
also update the import sample script to accept REPO & branch as ENV

@Jdubrick updated readme


Searching for accounts via public key.

Hello,
I would like to be able to search the "safebox" to retrieve pascal account numbers which use the same public key.
Currently I only know the public key of a certain account, but I don't know the pascal account number, so I would like to be able to find it (the account number) via the public key.
This could also be considered a "security feature" to make sure that there are no other accounts with the same public key. So that the public key can be considered "unique".
Bye
Skybuck.

I wrote a little bit of code to try this out as follows (real public key replaced with 'SomePublicKey' for privacy purposes ;):
procedure TFRMWallet.Button1Click(Sender: TObject);
var
AccountNumber: Cardinal;
account : TAccount;
begin
for AccountNumber := 600000 to 800000 do
begin
account := FNode.Operations.SafeBoxTransaction.Account(AccountNumber);
	if TAccountComp.AccountPublicKeyExport(account.accountInfo.accountKey) = 'SomePublicKey' then
	begin
		ShowMessage('AccountNumber: ' + IntToStr(AccountNumber) + ' Found !');
	end;
end;

end;
This seems to run quite slow ! Not yet sure why, have not debugged into... Just like to point this out, that if this feature is to be implemented, something faster will have to be used ! :)

This new method works much faster, it first converts the public key to some kind of binary key.
This binary key is then searched, which works much much much much faster.
procedure TFRMWallet.Button2Click(Sender: TObject);
var
  vPublicKeyString : AnsiString;
  vAccountKey : TAccountKey;
  vErrors : AnsiString;

  vAccountNumber: Cardinal;
  vAccount : TAccount;

begin
  vPublicKeyString := '3GhhboqLaD34ereBtoezwAzTHE5GPM7bKHEuoc1covYaZmTdkW7YF5nYmDzftTQvfb4B5Pr3aFDh6MRXtm8KruuBSevJd4P7ZmLnHs';

  // first convert public key to account key
  if TAccountComp.AccountPublicKeyImport( vPublicKeyString, vAccountKey, vErrors ) then
  begin
    ShowMessage('Success');
    // now search account key among account keys.
    for vAccountNumber := 0 to 800000 do
    begin
      vAccount := FNode.Operations.SafeBoxTransaction.Account(vAccountNumber);

      if TAccountComp.EqualAccountKeys( vAccountKey, vAccount.accountInfo.accountKey ) then
      begin
        ShowMessage('AccountNumber: ' + IntToStr(vAccountNumber) + ' Found !');
      end;
    end;
  end;
end;

I don't know yet how to retrieve number of accounts yet so this code searches up to 800000, users of this code might need to change this to find theirs ! :)

(Fast code above integrated into searchValues and search select form and so forth in custom version).


Does this extension support Remote SSH mode (the MS addon)?

VS Code supoorts remote development via SSH tunnel and that is what I am currently using. Here is the description of the feature: https://code.visualstudio.com/docs/remote/ssh
The way I understand it, extensions should generally be unaware of this and think they are running on the remote machine.
With that in mind, I set up the launch config as if it was a local GDB launch:
        {
            "type": "gdb",
            "request": "launch",
            "name": "Debug bootcamp",
            "target": "../build/myexecutable",
            "cwd": "${workspaceRoot}",
            "valuesFormatting": "parseText"
        }

However, I get no results. When I start the debugging, the Debug console will show "Running executable" in orange text and nothing else will really happen. On the top side of the screen, I can see the Pause/Step/Stop controls, but pause does not really do anything.
So is this compatible with the remote SSH feature, or do I have to use GDB server? Obviously using GDB directly would be nicer as it does not require additional config of hostname and port.

Are there any other ways to debug this extension on SSH mode?

I have my vscode connected via SSH to remote machine.
I have Native Debug extension installed on remote.
My launch configuration is:

{
	"type": "gdb",
	"request": "launch",
	"name": "Launch UT",
	"target": "/path/to/project/build/unitTests",
	"cwd": "/path/to/project",
	"valuesFormatting": "parseText",
	"internalConsoleOptions": "openOnSessionStart",
	"stopAtEntry": true,
	"showDevDebugOutput": true,
	"printCalls": true
}

When I choose

Then nothing happens and on vscode's DEBUG CONSOLE I can see:
1-gdb-set target-async on
2-list-features
3-environment-directory "/path/to/project"
4-file-exec-and-symbols "/path/to/project/build/unitTests"
GDB -> App: {"outOfBandRecord":[{"isStream":false,"type":"notify","asyncClass":"thread-group-added","output":[["id","i1"]]}]}
GDB -> App: {"token":1,"outOfBandRecord":[],"resultRecords":{"resultClass":"done","results":[]}}
GDB -> App: {"token":2,"outOfBandRecord":[],"resultRecords":{"resultClass":"done","results":[["features",["frozen-varobjs","pending-breakpoints","thread-info","data-read-memory-bytes","breakpoint-notifications","ada-task-info","python"]]]}}
GDB -> App: {"token":3,"outOfBandRecord":[],"resultRecords":{"resultClass":"done","results":[["source-path","/path/to/project:$cdir:$cwd"]]}}
-gdb-exit

That's it. I don't know what is wrong 😔
When I open ssh to my remote a manualy run: gdb /path/to/project/build/unitTest then it loads all symbols correctly and I'm able to run the tests.

There's no crash visible in your output. You may want to recheck with "showDevDebugOutput": true, "printCalls": true, in launch.json, this way we may be able to catch that and return a useful message to the user.
... in any case: if gdb chrashes then this seems to be unrelated to this extension and you may want to close this issue.
To the question: "Are there any other ways to debug this extension on SSH mode?", which I read as "to debug with this extension on SSH remote": sure connecting with jeanp413.open-remote-ssh, then debug. In the case you want to do that with a WSL container, have a look at https://github.com/jeanp413/open-remote-ssh/issues/96.


How to train CADRL ,RGL , LSTM_RL in this env?

How are you @Shuijing725 ?
I have a single question same as a title in this page.
In your paper, CADRL, RGL, LSTM-RL(compare with DSRNN) had been trained in this env(DS-RNN env)?
If answer is yes, then how to training them in this env?

You can do this by placing our gym environment code into the official repos of CADRL, RGL, and LSTM_RL. And leave everything else as it is.

Thank you


Support user action to delete builds in database

fixes: #306

Signed-off-by: OscarQQ<EMAIL_ADDRESS>

@llxia

Throwing this in as food for thought, the first dialog is really a question of "removing from list" or "hiding", second dialog is asking if you want to "delete from DB"...  is the recommended action, if hiding, then delete from DB?  I am guessing so...  the unusual case would be if hiding, keep in DB...
2 dialogs is a lot of interruption, can it be done with 1?  2 button choices, Hide / Delete and then an "Are you sure you wish to delete the data from the database?"
Perhaps we can look at standard other applications or some UX best practices for how they word the choices in dialog boxes, so this is more predictable and less interruptive to users.

How about deleting the popconfirm instead of creating 2 action buttons? So there is only one dialog (first dialog seems useless to see, just want to let user double check)

If the user changes his mind he could click "X" on the top right of the dialog
Using two action buttons might seem weird when the user accidentally clicks on Add Row and wants to delete that row

I agree that we can reduce to have one dialog.
Title: Delete URL and build data
Text: Delete Build URL means this build will not be monitored by TRSS anymore. Delete build data means all build data related to this Build URL will be permanently deleted.
Three buttons:

Delete URL only
Delete URL and build data
Cancel

Cancel button should be the only one highlighted (type="primary") and focused.

Is there ever a real case where we want to hide the data from being presented in the UI but keep it in the DB, and if so, what is the process for clearing that data out of the DB at a later time?

Sorry, I just read Lan's comment above, the use case may be that we no longer want to monitor a build anymore, but want to keep the data.  I guess this could be when a build pipeline gets renamed and/or no longer exists (retired jdk versions, that we may still want to cross-reference when we are able to search DB contents across versions, which I do not think is yet supported).

No apologies needed. I do share your concerns, Shelley. You are right that we should be cautious/strict on the user actions that we provide. Otherwise, it will cause more maintenance later (someone has to clean/remove the data eventually).
By keeping one choice, `delete it from view and DB', it simplifies user scenario and our logic. In the case of pipeline rename or keeping a longer history, we can just keep the build URL in the Build Monitor List. The only cost is some CPU power to check for if there is any new build. Since TRSS gets either an invalid URL (rename case) or no new build (build no longer run), TRSS will skip them and proceed on the next URL in the Build Monitor List.

loading icon shows up when we are deleting build data

@llxia


Add tests for QdrantTranslator

This is just a simple suite of unit tests for #5567
Who can review?
VectorStores / Retrievers / Memory

@dev2049

@kacperlukawski is qdrant-client compatible with python 3.11?

@baskaryan It is, I'm not sure why this build failed, as I'm working on a different PR, and no issues there. Let me check that!

@kacperlukawski Hi , could you, please, resolve the merging issues? After that ping me and I push this PR for the review. Thanks!


feat(axe-core-4.3.2): Upgrade to use axe-core 4.3.2

Details
This PR upgrades the axe-sarif converter to use the latest version of axe-core: v4.3.2.
Motivation
Feature work, upgrading all products to use axe-core 4.3.2
Context
There is currently a bug in @axe-core/puppeteer that causes the standard w3citylights-axe-4.3.2.reporter-raw.json test resource to fail to generate. An issue has been filed with Deque here:
https://github.com/dequelabs/axe-core-npm/issues/323
That file has not been tested against in our past unit tests, so it shouldn't be a blocking issue but may be something we want to revisit once the issue has been resolved. The basic-axe-4.3.2.reporter-raw.json, which is referenced in unit tests did generate properly and passes the tests.
The chromedriver yarn resolution was added due to selenium's chromedriver being out of date and causing issues running the test resource generator. The fact that we are able to use a ">=" version should allow it to continue upgrading into the future without removing the resolution, making this resolution less likely than most to cause issues down the line.
Pull request checklist

[x] PR title respects Conventional Commits (starts with fix:, feat:, etc, and is suitable for user-facing release notes)
[x] PR contains no breaking changes, OR description of both PR and final merge commit starts with BREAKING CHANGE:
[n/a] (if applicable) Addresses issue: #0000
[x] Added relevant unit tests for your changes
[x] Ran yarn precheckin
[x] Verified code coverage for the changes made

:tada: This PR is included in version 2.8.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Issue loading Numpy array into pclpy.PointCloud object

It's not clear where the problem is, but it appears that the from_array method for PointCloud objects is not fully implemented. I haven't done any digging, but here is how to duplicate the problem:
testpoints = numpy.full((100,4),1.0).astype(np.float32)
testcloud = pcl.PointCloud.PointXYZI()
cloud.from_array(testcloud)

Using C++, this equivalent procedure would yield a fully populated pointcloud.

User error. The cloud.from_array method must be stored in a second object.


Initial Requirement 2 Lofi-Prototyping

Description: Prototyping based on Initial Requirement 2.
Goals: Create a Prototype based on the Requirement.
Success or Completion Criteria: Prototype Figma.
Allocated Time: 2 days
Start Date: 31/08/23
End Date: 01/09/23
Main Task :  Create lo-fi prototypes/wireframes based for the initial requirements

Completed.


[tiff] Add libdeflate support, add usage

Feature tiff[libdeflate] passed with following triplets:
x86-windows
x64-windows
x64-windows-static

Usage test passed on x64-windows.

Thanks for the new feature!


(graphcache) - Storage's onOnline event listener leaks Client

When using makeDefaultStorage there is a memory leak if clients are no longer in use. This is because the listener from onOnline is never removed.
This could also be dangerous because since old storage is still subscribed to the event listener it will have it's callbacks called. I haven't really looked into this.
Ideally there would be some sort of API for unsubscribing these listeners in addition to not having this leak for makeDefaultStorage. Which would make it easy for custom implementations of storage to not have to micro-manage the listeners that are required by storage.
If you create many re-create client many times you can see the memory footprint continue to climb. Using a production build of react (https://mookiies.github.io/onOnlineDemo/) memory footprint went from 1.7mb to 7.6mb after creating a client 150 times.
urql version & exchanges:
 "@urql/exchange-graphcache": "^4.3.0",
"urql": "^2.0.4",
Steps to reproduce

Create a client that uses makeDefaultStorage
Destroy client

Expected behavior

Client instances are garbage collected

Actual behavior

Client instances are kept around

Code Example
Based off the example of destroying client for cache invalidation here: https://formidable.com/open-source/urql/docs/advanced/authentication/#cache-invalidation-on-logout

Click here to expand
import React, { useState, useMemo } from "react";
import {
  createClient,
  dedupExchange,
  fetchExchange,
  Provider
} from "urql";

import { offlineExchange } from "@urql/exchange-graphcache";
import { makeDefaultStorage } from "@urql/exchange-graphcache/default-storage";
import { Todos } from "./components";

const storage = makeDefaultStorage({
  idbName: "graphcache-v3", // The name of the IndexedDB database
  maxAge: 7 // The maximum age of the persisted data in days
});

const cache = offlineExchange({
  storage
});

const clientConfig = {
  url: "https://0ufyz-4000.sse.codesandbox.io",
  exchanges: [
    dedupExchange,
    cache,
    fetchExchange
  ],
  requestPolicy: "cache-first"
};

const App = () => {
  const [isLoggedIn, setIsLoggedIn] = useState(false);
  const toggleLoggedIn = () => {
    setIsLoggedIn((previousValue) => !previousValue);
  };
  const client = useMemo(() => {
    if (!isLoggedIn) {
      return null;
    }

    console.log('createClient');
    return createClient(clientConfig);
  }, [isLoggedIn]);

  const app = <Provider value={client}>
     <Todos />
  </Provider>

  return (
      <main>
        <button onClick={toggleLoggedIn}>Toggle login</button>
        {isLoggedIn && app}
      </main>
  );
};

export default App;

Demo deployed here with the above code that can be profiled, but easier to run locally with unminified js
client: https://mookiies.github.io/onOnlineDemo/
server: https://0ufyz-4000.sse.codesandbox.io
(built from https://github.com/Mookiies/onOnlineDemo)

We did realise that, but overall the client doesn't have a lot of mechanisms to be exposed, since they're not commonly needed. Granted, we could dispose of the storage and allow this to be done manually. However, you're saying this gets severe when you create 150 clients, right? I find this to be a little of an unlikely scenario, so I'm wondering how relevant this actually is, given that it adds more user requirements.
I'm not super opposed to adding disposers to storage, but I am at the same time curious whether this is worth solving 😅 Our last conclusion was that it's unlikely to cause any issues for prolonged times/sessions.

I agree that the memory footprint side probably isn't the biggest deal as it's unlikely that a client will re-created that many times.
What I'm really more concerned about is the potential side effects that could arise from having a client that is no longer being used trying to execute mutations and potentially overriding cache items as a result.  Are there any mechanism that would stop and old client from trying to execute?

Oh, I see, i.e. in a scenario where it's discarded, comes back online, and reruns several mutations? Hm, we did consider that but we assumed that since the "dispatcher" of these mutations couldn't tell whether the client was online or offline, it also shouldn't matter too much whether those are eventually sent or not 🤔


Generating references without kerchunk

VirtualiZarr + zarr chunk manifests re-implement so much of kerchunk that the only part left is kerchunk's backends - the part that actually generates the byte ranges from a given legacy file. It's interesting to imagine whether we could make virtualizarr work without using kerchunk or fsspec at all.
https://github.com/TomNicholas/VirtualiZarr/issues/61#issuecomment-2047826810 discusses how the rust object-store crate might allow us to read actual bytes from zarr v3 stores with chunk manifests over S3, without using fsspec.
The other place we use fsspec (+ kerchunk) is to generate the references in the first place. But can we imagine alternative implementations for generating that byte range information?
Arguments for doing this without using kerchunk + fsspec are essentially:

increased reliability
clearer interfaces
not using an overly complex tool (i.e. fsspec, which can read from all sorts of systems) to read from just two places (local or S3)
possible performance increases during reference generation (though this is unlikely to be a major bottleneck)
separation of concerns - if we can find other libraries that generate the byte range information already for their own purposes (e.g. h5py using the ros3 driver, hidefix, or cog3pio), we might be able to avoid bearing that maintainance burden, which would be great.

From @sharkinsspatial:

I just had a quick look at using hidefix to support ChunkManifest generation without kerchunk. Digging in a bit more detail, IIUC hidefix is still using the core HDF5 library to generate an Index object which can then be used to bypass the HDF5 concurrency limitations.
As we are only iterating over the chunk offsets and not actually reading any data, this doesn’t provide us any advantage over h5py.  Given that, I’ll try to focus on just creating an h5py based ChunkManifest lib for the near term.

I took a quick, first look at Virtualizarr last night (amazing 🎊 , thank you for pushing this forward).  I can re-purpose/simplify the existing kerchunk HDF backend to directly support generating ChunkManifests .
A few questions for creating a PR for this.

kerchunk's SingleHdf5ToZarr will generate corresponding Zarr groups for HDF5 groups in a file.  I may be misunderstanding the open_virtual_dataset logic but it is not currently attempting to build Dataset containers representing nested HDF5 groups in a file correct?  If I am understanding this correctly, is this something we do want to support or should variables in nested HDF5 groups be flattened.
As this PR would only replace kerchunk use for ChunkManifest generation for HDF5 files, we would still require the existing logic for using KerchunkStoreRefs for other formats.  What would be the least intrusive way to incorporate this, some format specific branching logic directly in open_virtual_dataset?

I can re-purpose/simplify the existing kerchunk HDF backend to directly support generating ChunkManifests .

That would be awesome, thanks @sharkinsspatial ❤ . I think whilst we're doing this we should try hard here to improve test coverage and understanding of behaviour in nasty cases. I'm thinking about #38 in particular.

is this something we do want to support or should variables in nested HDF5 groups be flattened.

This is totally equivalent to the xarray.Dataset vs xarray.DataTree correspondence. We should actually add a open_virtual_datatree function which opens all the groups, and add a group kwarg to open_virtual_dataset. I'll make a new issue for groups now. (See also https://github.com/TomNicholas/VirtualiZarr/issues/11)

What would be the least intrusive way to incorporate this, some format specific branching logic directly in open_virtual_dataset?

I think what you're suggesting is the least instrusive way. The main thing is to keep code that actually depends on the kerchunk library isolated from the rest of the code base. I would perhaps make a new readers directory or something to distinguish it from the kerchunk.py wrapper of the kerchunk backends.


Make indentation and SQL formatting consistent

Indentation for SQL examples weren't consistent - I've made to use 4 spaces where it's not so that it's same everywhere.
SQL keywords, int, VARCHAR etc weren't consistent as at they're UPPERCASE at places but not everywhere - I've made it so.
Add new lines in some places for better clarity

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: kabirbaidhya sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.

@kabirbaidhya : Thanks for your contribution! The author(s) have been notified to review your proposed change.

@kabirbaidhya : Thanks for your contribution! The author(s) have been notified to review your proposed change.

@CarlRabeler, if you approve, should we move this to the private repo for staging, or are you confident it's ok as viewed here? Thanks.


Provide GPU support

Pytorch and Tensorflow have GPU trained models. Need to support this in the RunInference classes, and also make sure they get configured in Dataflow.
Imported from Jira BEAM-13986. Original Jira may contain additional context.
Reported by: yeandy.
Subtask of issue #21435

Subtask of https://github.com/apache/beam/issues/22117

@yeandy what pieces are missing to support this?

The main one should be https://github.com/apache/beam/pull/22795

@yeandy now that #22795 is done is there any additional work to be done here?

.close-issue

@yeandy please clarify the scope of this issue. Thanks!

Clarification on this issue:
PyTorch RunInference converts data and model to GPU by using the tensor.to(device) command to move a tensor to a device. We also do implicit GPU -> CPU conversion if GPU doesn't exist in the worker.

FYI, you can edit the description or title of the issues.

FYI, you can edit the description or title of the issues.

Probably not as a non-committer unfortunately


Edit / Delete Genders

I'd like to be able to delete "woman" from genders as this is an all-male dating site (including "couples").
That's the ideal.  Is there a fairly uncomplicated way to do that, or is it too complicated (i.e. "woman" is all through the database, or in code files that would get overwritten with any updates?
Thanks

Hello,
I had removed the "couple" option in files using Searchmonkey (cross-platform). There is also Recoll, for instance (but only for Unix/Linux).
I just had a difficulty with one file, but I think it was ok.
But I didn't test everything to be sure there was no problem.
And women can still contact women, and men still contact men. I would prefer it was not the case for a specific project.
Here you can normally find solutions:
https://github.com/pH7Software/pH7-Social-Dating-CMS/issues/359
https://github.com/pH7Software/pH7-Social-Dating-CMS/issues/358
I think I will try them after an "automatic" upgrade of the software when the new stable version comes out.

Hey @Outlines
This might also help you https://github.com/pH7Software/pH7-Social-Dating-CMS/wiki/Remove-the-"couple"-option
Instead of removing the "couple" option, remove the "woman" option. You will need to do the same for the "Looking for" field.

Thank you for the suggestions - that did the trick!  (will these get overwritten by any updates?)

Happy it helps!

Yes, it will if you want to update your website... that's why you always have to backup your website before. In that way, you can put back your changes to the new files once updated (or see the changes done for the next version in the git history).

BTW @Outlines, don't forget to Star the Github repo 😊

I will close this issue by now @Outlines
If you have any further problems, please open a new issue.
Have a great day!


Pin ipopt to x.x

Since the new ipopt minor version changed the soname

ipopt has run_exports so it should be ok, maybe it was added after the last build so bumping bonmin should be enough

@conda-forge-admin, please rerender

Hi! This is the friendly automated conda-forge-webservice.
I just wanted to let you know that I rerendered the recipe in conda-forge/coinbonmin-feedstock#4.


Reland: implement build bundle with assemble

Description
This is a reland of https://github.com/flutter/flutter/pull/37508. The depfile's were not correctly escaped for windows, which caused failures when rebuilding.
Additionally fixes a bug in the file cache cleanup, and ensures we use a package uri when building a snapshot in assemble.

I can't see if the devicelab failure is real or not, so I'll rollback for now


Configure automatically based on HW platform

Signed-off-by: Taewan Kim<EMAIL_ADDRESS>Description
For user convenience, when make create_context is executed,
the .config file is created automatically according to the HW platform and CONFIG_CONTAINER=y.
Type of change

[x] New feature (non-breaking change which adds functionality)
[x] Documentation update

How Has This Been Tested?
Run make create_context
Test Configuration:

Firmware version: Ubuntu 20.04, Raspberry Pi OS
Hardware: x86-64, ARM
Edge Orchestration Release: Coconut

Checklist:

[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[ ] I have commented my code, particularly in hard-to-understand areas
[x] I have made corresponding changes to the documentation
[x] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[x] New and existing unit tests pass locally with my changes

@tdrozdovsky will open a PR to replace this.


Query::get_multiple for accessing multiple entities simultaneously from the same query

What problem does this solve or what need does it fill?
When working with queries, you sometimes need to access the data on multiple components at once. This is particularly frustrating if you require mutable access, due to Rust's borrow checking rules.
What solution would you like?
Add the Query::get_multiple(entities: HashSet<Entity>) -> HashMap<Entity, Result<[Elided](https://docs.rs/bevy/0.5.0/bevy/ecs/system/struct.Query.html#method.get_mut)>> method and the associated get_multiple_mut. By passing in a HashSet of entities, we can ensure that we never have mutable access to the entities at the same time.
Also add get_component analogues for this pattern.
What alternative(s) have you considered?
iter_combinations solves the common case, but  other more bespoke access patterns certainly exist.
You can work around this by storing data, but that only works for Clone components.
@TheRawMeatball felt this should use a different API / impl:

it'd be more like a wrapper you put the query into
and then it'd pass out smart pointers wrappers of smart pointers

Additional context
#1470 may be easier to work around with this style of API.
It may make sense to use this implementation for iter_combinations in some way, but be mindful of perf as iter_combinations is likely to be used in tight loops for things like collision detection.

Unsurprisingly, a parallel API for direct world access is probably a good idea too.

https://github.com/rust-lang/rfcs/pull/3100 treads similar ground, creating a trait for this style of functionality in the standard library. We could migrate this functionality to use that trait later once it's stabilized.

Duplicate of #2042.

Just fyi, if you really really want access to multiple entries, then you might consider a cuckoo table.  Those could've some index method which finds the positions for the key, and then you access those positions directly as if they were a Vec.  Any write accesses could bump buckets contents of course, but only into a few other buckets, whic the index still locates.  I've long meant to write a cuckoo table crate, because there are a bunch of really cool optimizations, but not yet done so.

Really neat data structure! Would the main benefits be performance?

I'm unsure how performance works out, usually cuckoo filters wind up more memory efficent.  I think few people use them because all the optimizations look more subtle than hash tables.
Afaik they work fine with the same good hashers and hash tables, like SipHash or HashBrown, but they're more sensitive to bad hashers, and some devs love plugging in bad hasher for speed.


Release 0.3.17

@JuliaRegistrator register(branch=release)

Registration pull request created: JuliaRegistries/General/9848
After the above pull request is merged, it is recommended that a tag is created on this repository for the registered package version.
This will be done automatically if Julia TagBot is installed, or can be done manually through the github interface, or via:
git tag -a v0.3.17 -m "<description of version>" 7ecc89bfa5213adddd10ee1315395af2ee4fdc8f
git push origin v0.3.17


Crash/Force termination in distributed training

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): No
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): RHEL 7.9
TensorFlow installed from (source or binary): source
TensorFlow version (use command below): 2.5.0
Python version: 3.8.6
Bazel version (if compiling from source): 3.7.2
GCC/Compiler version (if compiling from source): 10.2.0
CUDA/cuDNN version: 11.1.1
GPU model and memory: A100

Describe the current behavior
When running the distributed training, i.e. with MultiWorkerMirroredStrategy, the application runs through and then crashes with "terminate called without an active exception"
Standalone code to reproduce the issue
Taken from #50790:
import tensorflow as tf
from mpi_cluster_resolver import MPIClusterResolver

resolver = MPIClusterResolver()
strategy = tf.distribute.MultiWorkerMirroredStrategy(cluster_resolver=resolver)

with strategy.scope():
    (x_train, y_train), _ = tf.keras.datasets.mnist.load_data()
    x_train = x_train / 255.0
    train_data = tf.data.Dataset.from_tensor_slices((x_train, y_train)).batch(128)

    model = tf.keras.models.Sequential([
        tf.keras.layers.Flatten(input_shape=(28, 28)),
        tf.keras.layers.Dense(128, activation='relu'),
        tf.keras.layers.Dense(10),
    ])

    model.compile(
        optimizer=tf.keras.optimizers.SGD(),
        loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
        metrics=['accuracy'])

is_master = resolver.task_id == 0
verbose = 2 if is_master else 0
model.fit(train_data.repeat(), epochs=1, steps_per_epoch=10, verbose=verbose)

With
mpi_cluster_resolver.py.txt
Other info / logs
I traced the crash to TF_DeleteGraph Python wrapper by following the stacktrace and inserting debug information:
tf.txt
As you can see the code is called during runtime shutdown and ultimately reaches pybind11s ~gil_scoped_release which calls PyEval_RestoreThread. This is documented as terminating when the runtime is finalized: https://docs.python.org/3/c-api/init.html#c.PyEval_RestoreThread
So to fix this either all threads created by TF must be collected and freed before python starts finalizing or the pybind11::gil_scoped_release must be removed from cleanup functions
See also https://github.com/pybind/pybind11/pull/2657

does this has a workaround？

Workaround/fix is https://github.com/tensorflow/tensorflow/pull/56634


[Discussion] tryAllParsing/default switch from loadDefinedFile

I put this as a PR to show that as of right now the tryAllParsing "feature" of an unknown extension is broken. There weren't any successful tests, so here are some that should work (but fail).
https://github.com/davidtheclark/cosmiconfig/blob/143ff724d2a791316f68c6298d0d43eda67ee823/src/loadDefinedFile.js#L57-L63
Initially I thought just swapping these functions would solve the issue, but instead require and require-from-string will return {} when presented valid yaml, such as yaml: true.
On the flip side, the yaml parser will return { 'module.exports = { js': 'true }' } when parsing a js file.
Additionally I do not think the try all parsing behavior is a good thing to do. It is highly error-prone, and more than likely there was a mistake made and an error should be thrown.
I propose to remove this catchall of unknown file types. Thoughts?
FYI, I think this is the last thing I'm tackling for https://github.com/davidtheclark/cosmiconfig/issues/109. I would have missed this if it weren't for flow because there aren't any successful tests.

More than likely if the user is getting here, there was a mistake made and there should be an error thrown.

@chrisblossom: I believe the typical use case is to have a config file without an extension that can be parsed as either YAML or JSON. I think that feature is worth keeping, right? If it's too hard to also fit in JS parsing, I think we could definitely drop that attempt — say that extensionless files can only be loaded if they are in JSON or YAML format. This is already how search behaves, I believe.

I am not sure about YAML (no real experience, all of the YAML configs I've seen personally have an extension), but I can see your point with JSON (examples being .babelrc and .eslintrc). Although I do not think the YAML parser should be responsible for parsing extensionless JSON, which it currently is.

I force updated this PR to make the requested changes. Should be ready to be merged unless more changes requested / not wanted. Please review.
For reference, original PR: https://github.com/chrisblossom/cosmiconfig/commit/3e0047f9e3d8be8523abfc6399c584051bb73591.

Although I do not think the YAML parser should be responsible for parsing extensionless JSON

Thanks — this forced me to remind myself of the details here ...
YAML parsers understand JSON because valid JSON is valid YAML — the spec says "every JSON file is also a valid YAML file". So I believe it's fine to use the YAML parser to parse extensionless rc files that happen to contain JSON.
Your understandable concern, though, is the reason we have the rcStrictJson option: for those who want to ensure that extensionless rc files for their program are only valid JSON, not JSON-plus-some-YAML (which is actually just YAML, I guess).
So if the user has set rcStrictJson: true, we should only use the JSON parser, not the YAML parser at all. While if they have not used that setting, we can just use the YAML parser.
That's how we're doing it in search when we loadRcFileWithoutExtensions: https://github.com/davidtheclark/cosmiconfig/blob/143ff724d2a791316f68c6298d0d43eda67ee823/src/loader.js#L83-L90. I guess we neglected load.
So I think that the change here to try JSON first before YAML isn't necessary, because it doesn't actually get us anything (e.g. more helpful error messages). If your JSON is not quite strict enough and the JSON parser errors, we're still going to try the YAML parser anyway, right?
Instead, what we should do is make load pay attention to the rcStrictJson option, as search does, and use parseJson if it's on. (We could include that here or take care of it in another commit.)
That's my understanding, at least. Make sense?

Definitely makes sense.

So I think that the change here to try JSON first before YAML isn't necessary, because it doesn't actually get us anything (e.g. more helpful error messages). If your JSON is not quite strict enough and the JSON parser errors, we're still going to try the YAML parser anyway, right?

https://github.com/davidtheclark/cosmiconfig/blob/143ff724d2a791316f68c6298d0d43eda67ee823/src/loadDefinedFile.js#L19-L20
I think parsing should be explicit and consistent. Currently rightfully so, JSON has its own parser. I would expect that the JSON parser be used for all JSON parsing, not sometimes the YAML parser depending on how it is loaded. In other words, .babelrc written in JSON and .babelrc.json should be parsed the exact same way.
Another reason why it should be consistent is for things like https://github.com/davidtheclark/cosmiconfig/issues/119 requesting JSON5 support. I do think this is a valid request, even babel uses it to parse their JSON configs. Maybe #109 should allow custom loadJson and loadYaml loaders too?
Thoughts?

So if the user has set rcStrictJson: true, we should only use the JSON parser, not the YAML parser at all. While if they have not used that setting, we can just use the YAML parser.

I can add this to the PR if wanted.

Thanks again, @chrisblossom. I'll merge this and we'll move on from here.

In other words, .babelrc written in JSON and .babelrc.json should be parsed the exact same way.

I don't know of a way to do this, unless you the user use the rcStrictJson option to disallow extensionless config files from being interpreted as YAML. Besides, the YAML parser's treatment of JSON has been perfectly consistent and reliable.

Maybe #109 should allow custom loadJson and loadYaml loaders too?

Maybe we should add an ability to add your own arbitrary files to the list that gets attempted in each directly — that way when someone in the future requests XML or INI or CSV or whatever support, it will already be available to them.


Use the selector atom-text-editor[mini] to select mini-editors.

In index.less: Use the selector atom-text-editor[mini] to select mini-editors.

This is a duplicate of #38 which has been fixed in v0.3.2.


Scala plugin doesn't provide subproject as dependency

Context
I'm attempting to use Gradle for a Java/Scala multi-project build. Unfortunately, it appears that when having project A depend on project B, the scalaCompile task for project A is not finding the compiled classes from project B.
In my gradle.build file below, the root project depends on macros.
https://github.com/carymrobbins/intellij-haskforce/blob/44ad20c9ebb221d9367bfe768a18b15ccb23fe1a/build.gradle#L99-L102
However, the compileScala task complains that it can't find the compiled classes from macros -
https://travis-ci.org/carymrobbins/intellij-haskforce#L287
Expected Behavior
The compileScala task should be able to find its dependencies' compiled classes.
Current Behavior
The compileScala task is not able to find its dependencies' compiled classes.
Steps to Reproduce (for bugs)
% git clone https://github.com/carymrobbins/intellij-haskforce
% cd intellij-haskforce
% ./gradlew assemble

Your Environment
This happens on Mac and Linux with Gradle 4.1.

It seems the culprit may be that assemble doesn't put the compiled Scala classes into the resulting jar. I've checked the jars that are assembled and they will only contain either sources compiled in src/main/java or src/main/resources.

Downgrading from Gradle 4.0.1 to 3.5 resolves the issue, so the problem lies with Gradle 4 compatibility.

This is an issue with the org.jetbrains.intellij plugin.  It changes the output directories for Scala compilation when using instrumentation: https://github.com/JetBrains/gradle-intellij-plugin/blob/master/src/main/groovy/org/jetbrains/intellij/IntelliJPlugin.groovy#L239
In Gradle 4.0+, there are multiple classes directories and the plugin doesn't take this into account.
That's the source of the deprecation warning.
You can get around this by either disabling instrumentation
intellij { instrumentCode = false }

Or forcing a single classes directory (this sacrifices build cache support)
def singleClassesDir = sourceSets.main.output.classesDir
sourceSets.main.output.classesDir = singleClassesDir

@big-guy Ah makes sense, thanks for tracking down the issue!


Multiple disk with capacity based scheduling

Current Longhorn replicas will use the directory at /var/lib/rancher/longhorn to store the data. We should support multiple user-specified locations on the node to use Longhorn.
We can support the same locations (e.g. in addition to the default location, user can add /opt/storage as the second location, and make sure the directory is available to use on all the nodes) for all the nodes as the first step if we don't need to store node-specific information (hopefully).
In the end, the user can update the node config and set desired disk (mounted to a node directory) to store Longhorn volume data. Longhorn can choose where to place the volume data depends on how much space the disk had left.

Phase 1, https://github.com/rancher/longhorn-manager/pull/85 merged.

Done.


chore: Fix typo "pipline" -> "pipeline"

Contributor checklist

[ ] Bug fixes include regression tests
[ ] Features include unit/acceptance tests

🚀 Thank you for your contribution! 🚀


Always set Content-Type header in RestJson1 responses

The protocol specifies it should be set by default to application/json
in HTTP responses, even when there are no output shape members, since in
that case an empty JSON payload {} is serialized. There are server
protocol tests asserting this behavior.
To implement this behavior, the RestJsonHttpBindingResolver class has
been introduced, which mostly delegates to the
HttpTraitHttpBindingResolver, but tweaks the response Content-Type
header.
Note this behavior is different than the client's, which should not
serialize a JSON payload if there are no input shape members to be
serialized to the body, and as such does not need to set the
Content-Type header.
This commit also deletes the ServerRestJson and ServerRestXml
classes, since they were exact copies of their client counterparts. The
latter was hosted in ServerRustXml.kt, which has been renamed to
ServerRestXml.kt.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

I know you don't like the precommit config, but https://github.com/awslabs/smithy-rs/runs/5065001814?check_suite_focus=true 😛


Clean up and optimize .gitignore

Complete the files generated by Jekyll and Bundler

Add files generated by OS

Thanks! @jekyllbot: merge +dev


it does not support background color  style for html tag.(html to pdf)

@iabhishek-oli  i will add this in next update
if you can work on this issue you can create PR

sorry looks like ios iOS-related issue.


[ui] Need to add one more permission/role for GKE

Current text in add cluster page of GKE:
Create a Service Account with a JSON private key and provide the JSON here. See Google Cloud docs for more info about creating a service account. Three IAM roles are required: computer/viewer, project/viewer, kubernetes-engine/admin and service-account/user, more info on roles can be found here.

One more permission is needed: compute.networks.list.

Really? Compute viewer should include listing everything.

That's based on comments from a customer. I haven't gotten a chance to test it. Will have to ask @sangeethah to help us out on verifying this.

Yes, required. And it is strange since compute.networks.list in the compute viewer and I see that all specified roles are added to service account.

It seems it is sporadic GCE issue, third attempt to create service account was successful.

Can't reproduce yet, it works when adding the given IAM roles.

@superseb Maybe only new users (compute engine is not enabled, k8s service not initialized) are affected. GCE UI is slow and back to 90 (wait several minutes, try a few minutes later and so on).

@sangeethah i don't believe tihs is an actual issue, but can you test based on the permissions provided in the UI and give feedback?

moved back to 2.1, since its to-test for reverification


Redirecting stderror to stdouput leads to enexpected behavior

Steps to reproduce
I am using openssl to check if all certificates in all of my systems got updated. It works on bash but in PS there are seems to be race conditions between stderr and stdoutput
write-output "q" | openssl s_client -showcerts -connect "microsoft.com:443" 2>&1 | openssl x509 -noout -dates -subject | grep After

Expected behavior
notAfter=Jun  5 20:48:18 2021 GMT

Actual behavior
really unpredictable errors can occurs, but kind of them really does not matter since they are all caused by PS messing with output order. Correct output of openssl s_client -showcerts -connect "microsoft.com:443" 2>&1 should be:
CONNECTED(00000005)
depth=2 C = IE, O = Baltimore, OU = CyberTrust, CN = Baltimore CyberTrust Root
verify return:1
depth=1 C = US, ST = Washington, L = Redmond, O = Microsoft Corporation, OU = Microsoft IT, CN = Microsoft IT TLS CA 1
verify return:1
depth=0 CN = microsoft.com
verify return:1
---
Certificate chain
{...}

but sometimes it is written as:
CONNECTED(00000005)
---
Certificate chain

or
depth=2 C = IE, O = Baltimore, OU = CyberTrust, CN = Baltimore CyberTrust Root
verify return:1
depth=1 C = US, ST = Washington, L = Redmond, O = Microsoft Corporation, OU = Microsoft IT, CN = Microsoft IT TLS CA 1
verify return:1
depth=0 CN = microsoft.com
verify return:1
CONNECTED(00000005)
---
Certificate chain

According to my tests removing 2>&1 fix the issue (PS correctly finds certificate date), but outputs stderror to console.
Environment data

Name                           Value
----                           -----
PSVersion                      6.2.3
PSEdition                      Core
GitCommitId                    6.2.3
OS                             Linux 5.0.0-27-generic #28~18.04.1-Ubuntu SMP Thu Aug 22 03:00:32 UTC 2019
Platform                       Unix
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           <IP_ADDRESS>
WSManStackVersion              3.0

This also works every time, so it seems that PS is redirecting really randomly instead of in order it receive data.
 write-output "q" | openssl s_client -showcerts -connect "microsoft.com:443" 2>$null | openssl x509 -noout -dates -subject | grep After

Thanks, @npodbielski; what you're reporting sounds like a duplicate of #5424 (which has been open for a long time).

yes. It looks like the same issue. I guess we can close this one.


doc: link to go1.5 release notes, devel/release.html

See https://github.com/golang/go/blob/master/doc/contrib.html#L34.

We also need to add the short release notes to devel/release.html.


Break MdxHierarchySet and MdxSet into two

Assign functions like union and cross_join accordingly.
With the new design, the code for a cross join would look like this:
from mdxpy import MdxHierarchySet, Member, MdxSet

set1 = MdxHierarchySet.members([Member.of("[Version].[Version].[Actual]")])
set2 = MdxHierarchySet.members([Member.of("[Year].[Year].[2021]"), Member.of("[Year].[Year].[2022]")])

set3 = MdxHierarchySet.members([Member.of("[Version].[Version].[Budget]")])
set4 = MdxHierarchySet.members([Member.of("[Year].[Year].[2023]")])

crossjoin1 = MdxSet.cross_joins([set1, set2])
crossjoin2 = MdxSet.cross_joins([set3, set4])

final = MdxSet.unions([crossjoin1, crossjoin2])
print(final.to_mdx())

All prior code would still work since MdxHierarchySet inherits all functions from MdxSet

Hi @MariusWirtz looks good 👍

Hi @MariusWirtz looks good 👍


Connect: network configuration to support StacksMocknet

With the recent @stacks/network version, a StacksMocknet property will be provided. The last network support PR doesn't include that yet:
import {
  StacksMainnet,
  StacksTestnet,
} from '@blockstack/stacks-transactions';

cc @friedger

This is a good idea.  This should not preclude letting the client apps setting the URLs to use for either of these networks.

If I understand the code correctly, it does not matter. A provided network will always be respected whether it is mainnet, testnet or mocknet. The code in the mentioned PR is misleading.

Ah, I saw this section and thought it was an update to connect. I think you're right @friedger, I'm closing this