I cannot reproduce the prefill speed 1000+tokens/s in the paper using demo_qwen_npu.cpp

The paper said when testing qwen1.5-1.8b on Xiaomi 14 with a 1024-token prompt, the prefill speed is 1106 tokens/s. I use Oneplus 12(8gen3+24gb ram) to run demo_qwen_npu for more than 10 times, only get about 100 tokens/s.
Here's my result:
Prefill Tokens: 1022
Prefill Time taken: 9.99451 seconds
Prefill Speed: 102.256 tokens/s

Here's my demo_qnn_npu.cpp(only modify a little bit):
#include "backends/cpu/CPUBackend.hpp"
#include "cmdline.h"
#include "models/qwen/configuration_qwen.hpp"
#include "models/qwen/modeling_qwen_npu.hpp"
#include "models/qwen/modeling_qwen.hpp"
#include "models/qwen/tokenization_qwen.hpp"
#include "processor/PostProcess.hpp"

using namespace mllm;

int main(int argc, char **argv) {
    cmdline::parser cmdParser;
    cmdParser.add<string>("vocab", 'v', "specify mllm tokenizer model path", false, "../vocab/qwen_vocab.mllm");
    cmdParser.add<string>("merge", 'e', "specify mllm merge file path", false, "../vocab/qwen_merges.txt");
    cmdParser.add<string>("model", 'm', "specify mllm model path", false, "../models/qwen-1.5-1.8b-chat-int8.mllm");
    cmdParser.add<string>("billion", 'b', "[0.5B | 1.8B]", false, "1.8B");
    cmdParser.add<int>("limits", 'l', "max KV cache size", false, 4000);
    cmdParser.add<int>("thread", 't', "num of threads", false, 4);
    cmdParser.parse_check(argc, argv);

    string vocab_path = cmdParser.get<string>("vocab");
    string merge_path = cmdParser.get<string>("merge");
    string model_path = cmdParser.get<string>("model");
    string model_billion = cmdParser.get<string>("billion");
    int tokens_limit = cmdParser.get<int>("limits");
    const int chunk_size = 64;
    CPUBackend::cpu_threads = cmdParser.get<int>("thread");

    auto tokenizer = QWenTokenizer(vocab_path, merge_path);
    QWenConfig config(tokens_limit, model_billion, RoPEType::HFHUBROPE);
    auto model = QWenForCausalLM_NPU(config);
    model.load(model_path);
    auto decoding_model = QWenForCausalLM(config);
    decoding_model.load("../models/qwen-1.5-1.8b-chat-q4k.mllm");

    // warmup START
    std::string input_str = " ";
    auto [real_seq_length, input_tensor] = tokenizer.tokenizePaddingByChunk(input_str, chunk_size, config.vocab_size);
    LlmTextGeneratorOpts opt{
        .max_new_tokens = 1,
        .do_sample = false,
        .is_padding = true,
        .seq_before_padding = real_seq_length,
        .chunk_size = chunk_size,
    };
    model.generate(input_tensor, opt, [&](unsigned int out_token) -> bool {
        auto out_string = tokenizer.detokenize({out_token});
        auto [not_end, output_string] = tokenizer.postprocess(out_string);
        if (!not_end) { return false; }
        return true;
    });
    Module::isFirstChunk = false;
    static_cast<CPUBackend *>(Backend::global_backends[MLLM_CPU])->setSequenceLength(0);
    static_cast<CPUBackend *>(Backend::global_backends[MLLM_CPU])->setExecutionType(PROMPT);
    static_cast<CPUBackend *>(Backend::global_backends[MLLM_CPU])->toggleSwitching();
    // turn on the multi-chunk prefilling
    Module::isMultiChunkPrefilling = true;
    // warmup END
    std::cout << "Warmup finished." << std::endl;

    vector<string> in_strs = {
        // " Give me a short introduction to large language model.",
        "\"Large Language Models (LLMs) are advanced artificial intelligence systems designed to understand and generate human-like text. These models are trained on vast amounts of data, enabling them to perform a wide range of tasks, from answering questions and summarizing text to generating creative content and engaging in conversational dialogue. LLMs like GPT-3 and GPT-4, developed by OpenAI, have set new benchmarks in natural language processing by leveraging deep learning architectures, particularly transformer models, which excel at capturing context and relationships within text. The scalability and versatility of LLMs make them invaluable tools for applications in education, customer service, content creation, and more. However, their deployment also raises ethical considerations, including issues of bias, misinformation, and the potential for misuse. As the field continues to evolve, ongoing research and responsible deployment strategies are essential to harnessing the full potential of these powerful AI systems while mitigating their risks. As LLMs continue to evolve, their applications expand across various fields. In education, they can personalize learning experiences by analyzing students' needs and progress, offering tailored recommendations and materials. In customer service, LLMs handle large volumes of inquiries, providing timely and accurate responses, thus improving customer satisfaction and operational efficiency. In content creation, LLMs assist in generating creative copy, writing articles and scripts, and even contributing to music and art production. Their powerful generative capabilities open new possibilities for the creative industries. However, this also raises discussions about copyright and originality, highlighting the need to ensure the legality and ethics of generated content. At the same time, the use of LLMs presents privacy and security challenges. Since these models rely on vast datasets, there is a risk of inadvertently exposing sensitive information or reinforcing existing biases. Ensuring data privacy and implementing robust security measures are crucial to addressing these concerns. Ethical considerations also include the potential for misuse, such as generating misinformation or deepfakes. It is essential to develop guidelines and policies that promote responsible use. As the field progresses, ongoing research and collaboration among stakeholders will be vital in balancing innovation with ethical responsibility, ensuring that LLMs are deployed in ways that benefit society while minimizing risks. The advancement of Large Language Models (LLMs) is a double-edged sword, offering immense potential for technological progress while also presenting unprecedented challenges. As these models become more integrated into various aspects of society, it is crucial to consider their long-term impact on employment, as they may automate certain tasks traditionally performed by humans. This raises questions about job displacement and the need for reskilling and upskilling initiatives to prepare the workforce for the new landscape created by AI advancements. In terms of accessibility, there is a risk that LLMs could exacerbate the digital divide, as those with greater resources may have more immediate access to these powerful tools, potentially widening the gap between different socioeconomic groups. Efforts must be made to ensure that the benefits of LLMs are distributed equitably and do not further marginalize already disadvantaged communities. Environmental considerations are also paramount, as the training and operation of LLMs require significant computational power, which can lead to substantial energy consumption and carbon emissions. The development of more energy-efficient algorithms and the use of renewable energy sources in powering AI infrastructure are essential to mitigate the environmental footprint of these models. Moreover, as LLMs become more sophisticated, the line between human and AI-generated content blurs, leading to complex questions about authenticity and trust. It is important to develop clear guidelines for transparency and attribution, so that users can distinguish between content created by humans and that generated by AI systems. Lastly, the global nature of AI development requires international cooperation and the establishment of global standards for AI ethics. Different cultures and jurisdictions may have varying perspectives on what constitutes ethical AI use, and finding common ground is essential to ensure that LLMs are used in a manner that respects human rights and values across the globe. In conclusion, the development and deployment of Large Language Models are not just technological endeavors but also societal ones. They require a holistic approach that considers not only the technological possibilities but also the broader implications for society, culture, and the environment. By engaging in a multidisciplinary dialogue that includes technologists, ethicists, policymakers, and the public, we can work towards a future where the benefits of LLMs are maximized, and the risks are minimized, ensuring that these powerful tools serve to enhance human potential and promote the greater good.\"\nWrite a long article about large language model based on the above text. Please note the following points: 1. Introduction to Large Language Models (LLMs) 2. Applications of LLMs in various fields 3. Ethical considerations and challenges associated with LLMs 4. Impact of LLMs on employment, accessibility, and the digital divide 5. Environmental implications of LLMs and energy consumption 6. Authenticity and trust in AI-generated content 7. Global cooperation and standards for AI ethics",
    };

    for (int i = 0; i < in_strs.size(); ++i) {
        auto input_str = tokenizer.apply_chat_template(in_strs[i]);
        auto [real_seq_length, input_tensor] = tokenizer.tokenizePaddingByChunk(input_str, chunk_size, config.vocab_size);
        const int seq_length_padding = (chunk_size - real_seq_length % chunk_size) + real_seq_length;
        const int chunk_num = seq_length_padding / chunk_size;

        std::cout << "real_seq_length: " << real_seq_length << std::endl;
        std::cout << "[Q] " << in_strs[i] << std::endl;
        std::cout << "[A] " << std::flush;

        LlmTextGeneratorOpts opt{
            .max_new_tokens = 1,
            .do_sample = false,
            .is_padding = true,
            .seq_before_padding = real_seq_length,
            .chunk_size = chunk_size,
        };

        auto start_time = std::chrono::high_resolution_clock::now();

        // tensor vectors to save the chunked tensors of the QNN prefilling input
        bool isSwitched = false;
        vector<Tensor> chunked_tensors(chunk_num);
        for (int chunk_id = 0; chunk_id < chunk_num; ++chunk_id) {
            chunked_tensors[chunk_id].setBackend(Backend::global_backends[MLLM_CPU]);
            chunked_tensors[chunk_id].setTtype(INPUT_TENSOR);
            chunked_tensors[chunk_id].reshape(1, 1, chunk_size, 1);
            chunked_tensors[chunk_id].setName("input-chunk-" + to_string(chunk_id));
            chunked_tensors[chunk_id].deepCopyFrom(&input_tensor, false, {0, 0, chunk_id * chunk_size, 0});

            model.generate(chunked_tensors[chunk_id], opt, [&](unsigned int out_token) -> bool {
                if (!isSwitched && chunk_id == 0 && static_cast<CPUBackend *>(Backend::global_backends[MLLM_CPU])->isStageSwitching()) {
                    // turn off switching at the first chunk of following inputs
                    static_cast<CPUBackend *>(Backend::global_backends[MLLM_CPU])->toggleSwitching();
                    isSwitched = true;
                }
                auto out_string = tokenizer.detokenize({out_token});
                auto [not_end, output_string] = tokenizer.postprocess(out_string);
                if (!not_end) { return false; }
                if (chunk_id == chunk_num - 1) { // print the output of the last chunk
                    std::cout << output_string << std::flush;
                }
                return true;
            });
            Module::isFirstChunk = false;
        }

        auto end_time = std::chrono::high_resolution_clock::now();

        std::chrono::duration<double> duration = end_time - start_time;
        double seconds = duration.count();

        double tokens_per_second = real_seq_length / seconds;

        std::cout << "Prefill Tokens: " << real_seq_length << std::endl;
        std::cout << "Prefill Time taken: " << seconds << " seconds" << std::endl;
        std::cout << "Prefill Speed: " << tokens_per_second << " tokens/s" << std::endl;

        start_time = std::chrono::high_resolution_clock::now();

        static_cast<CPUBackend *>(Backend::global_backends[MLLM_CPU])->setSequenceLength(real_seq_length);
        static_cast<CPUBackend *>(Backend::global_backends[MLLM_CPU])->setExecutionType(AUTOREGRESSIVE);
        static_cast<CPUBackend *>(Backend::global_backends[MLLM_CPU])->toggleSwitching();

        LlmTextGeneratorOpts decoding_opt{
            .max_new_tokens = 500,
            .do_sample = false,
            .temperature = 0.3f,
            .top_k = 50,
            .top_p = 0.f,
            .is_padding = false,
        };
        isSwitched = false;

        int step = 0;
        decoding_model.generate(chunked_tensors.back(), decoding_opt, [&](unsigned int out_token) -> bool {
            // call only once of switchDecodeTag
            if (!isSwitched) {
                static_cast<CPUBackend *>(Backend::global_backends[MLLM_CPU])->toggleSwitching();
                isSwitched = true;
            }
            auto out_string = tokenizer.detokenize({out_token});
            step++;
            auto [isOk, print_string] = tokenizer.postprocess(out_string);
            if (isOk) {
                std::cout << print_string << std::flush;
            } else {
                return false;
            }
            return true;
        });

        end_time = std::chrono::high_resolution_clock::now();

        duration = end_time - start_time;
        seconds = duration.count();

        tokens_per_second = (step) / seconds;

        std::cout << "Decode Tokens: " << (step) << std::endl;
        std::cout << "Decode Time taken: " << seconds << " seconds" << std::endl;
        std::cout << "Decode Speed: " << tokens_per_second << " tokens/s" << std::endl;

        // turn on switching, set sequence length and execution type
        static_cast<CPUBackend *>(Backend::global_backends[MLLM_CPU])->setSequenceLength(0);
        static_cast<CPUBackend *>(Backend::global_backends[MLLM_CPU])->setExecutionType(PROMPT);
        static_cast<CPUBackend *>(Backend::global_backends[MLLM_CPU])->toggleSwitching();
        std::cout << "\n";
    }
}

@noble00 Hi noble00, I tried to run demo_qwen_npu.cpp(unmodified official version) on oneplus12 too but I encountered this error

I thought the error is related to the qnn version, could you share your qnn version and device os version please?


xSQLServerSetup: When action is 'Addnode', the parameter 'PsDscRunAsCredential' must be used to login to the instance

Details of the scenario you try and problem that is occurring:
When the action is 'Addnode', then the parameter 'PsDscRunAsCredential' must be used to login to the instance if the node is not the primary node. That is because SYSTEM does not have permission on the other nodes.
Login failed for user 'COMPANY\SQLTEST5$'. Reason: Could not find a login matching the name provided. [CLIENT: <IP_ADDRESS>]

This should not be needed if it could login using the SetupCredential.
The DSC configuration that is using the resource (as detailed as possible):
        xSQLServerSetup $resourceConfigName
        {
            Action = 'AddNode'
            ForceReboot = $false

            SourcePath = $Node."SourcePath$($currentSqlInstance)"
            UpdateEnabled = 'False'

            SetupCredential = $SqlInstallCredential

            # This is needed to logon to the instance when the node is not the active node.
            PsDscRunAsCredential = $SqlInstallCredential

            $SourceCredential = $SqlInstallCredential

            InstanceName = $Node."$($currentSqlInstance)InstanceName"

            Features = $Node."$($currentSqlInstance)InstallFeatures"

            SQLSvcAccount = $SqlServiceCredential
            AgtSvcAccount = $SqlAgentServiceCredential
            ASSvcAccount = $SqlServiceCredential

            FailoverClusterNetworkName = $Node.SqlClusterGroupFailoverClusterNetworkName
        }

Version of the Operating System, SQL Server and PowerShell the DSC Target Node is running:
Windows Server 2016, SQL Server 2014, WMF 5.1
Version of the DSC module you're using, or 'dev' if you're using current dev branch:
Dev

This becomes catch 22 because currently the built-in parameter PsDscRunAsCredential fails to start the setup process. This is reported in issue #408.
erShell DSC resource MSFT_xSQLServerSetup  failed to execute Set-TargetResource functionality with error message: Exception calling "CreateProcessAsUser" with "4" argument(s): "The process could not be created. Create process as user error #1314"
    + CategoryInfo          : InvalidOperation: (:) [], CimException
    + FullyQualifiedErrorId : ProviderOperationExecutionFailure

Closed by mistake.

I'm adding examples to xSQLServerSetup ( in PR #445). Adding notes about this issue and other issues. Those notes have to be updated when this issue and other are resolved.


gRPC: disable SO_REUSEPORT

Signed-off-by: TXXT<EMAIL_ADDRESS>
What problem does this PR solve?
close #4323
What is changed and how it works?
Disable SO_REUSEPORT and grpc just use SO_REUSEADDR
Related changes
NO
Check List
Tests

Unit test

Side effects

Potential Performance regression (Related bench is posted on #4323 )

/bench

/bench +tpcc

/bench

/bench

I am so surprised that this can affect performance.

I am so surprised that this can affect performance.

I found this explanation. FYI https://github.com/grpc/grpc/pull/14748

/bench

Clearly, using SO_REUSEPORT gives significant performance improvements (and this is why, it is generally enabled in our code - unless we are running on some older kernel version that didn't have this feature)

So why do we disable this?

Clearly, using SO_REUSEPORT gives significant performance improvements (and this is why, it is generally enabled in our code - unless we are running on some older kernel version that didn't have this feature)

So why do we disable this?

Related issue. #4323  By benching in ci to evaluate the impact of performance fallback, if there is an impact, I think that I can find other ways to solve the problem of port conflict

@BusyJay PTAL

/bench +tpcc

You should fill the release note.

You should fill the release note.

I will update if the PR needs be merged

/bench +tpcc

/bench +ycsb

/bench +ycsb

/bench +ycsb

I noticed in two ycsb tests, tidb has different hash. @BusyJay . Maybe need more bench

/bench +ycsb

I find most of the bench results are worse than before, right?

Another approach to get around the issue without introduce potential regression is to lock the port via a file lock. This should be able to avoid port conflict for TiKV instances but can't prevent other processes accidentally conflict. Though it seems enough for the reported cases.

@BusyJay I don't know if os will automatically clean up the lock file in /tmp, if it is, where you think it is better to place?

/bench +ycsb


Fix recipe-maintainers

Had a minor syntax error. This removes it. Was causing this error.

Hi! This is the friendly conda-forge-admin automated user.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

@ocefpaf or @pelson, could one of you please merge this when you have a chance? Thanks.

@pelson, if there are no issues with this, could you please merge this. The feedstock is currently orphaned as the maintainer was not added. This should fix that.

Thanks @jakirkham - yes, this is good. I got carried away with fixing it in conda-smithy and then having some 🍞 breakfast. 😄

No worries. I know I post so much stuff that I just wanted to make sure this didn't get buried. :wink:

Good - keep pining me on the important stuff. If you don't get a response to something you need a response for, you are welcome to use any channel you like to get my attention (unless I'm on vacation 🐚 🍹 🌊 🐳 )


[arm64-v8a] Crash. UnsatisfiedLinkError in libc++_shared.so on Android when 3rd party lib

Description
Crash can be observed on arm64 device (Pixel 3a, physical and simulator) when
RN 0.67.2 is used with 3rd pary library that uses libc++_shared.so.
There is strong suspicion that this happens due to different versions of NDK used with both libs:
23.1.7779620 vs 21.4.7075529
Because project wants cinterop libc++.so and RN libc++.so,
gradle config includes:
packagingOptions {
  pickFirst("**/libc++_shared.so")
}

This results in UnsatisfiedLinkError:
java.lang.UnsatisfiedLinkError: dlopen failed: cannot locate symbol "__emutls_get_address" referenced by "/data/app/~~1OdRcdrRGBvzrkIJ8Ak36g==/com.....androidApp-GxBncVvTbA9_hL_OloW8kg==/base.apk!/lib/arm64-v8a/librealmc.so"...
    at java.lang.Runtime.loadLibrary0(Runtime.java:1077)
    at java.lang.Runtime.loadLibrary0(Runtime.java:998)
    at java.lang.System.loadLibrary(System.java:1656)
    at com.....compmulti.android.AndroidApplication.onCreate(AndroidApplication.kt:28)
    at com....Application.onCreate(CompanionApplication.kt:51)
    at android.app.Instrumentation.callApplicationOnCreate(Instrumentation.java:1211)
    at android.app.ActivityThread.handleBindApplication(ActivityThread.java:6725)
    at android.app.ActivityThread.access$1500(ActivityThread.java:247)
    at android.app.ActivityThread$H.handleMessage(ActivityThread.java:2053)
    at android.os.Handler.dispatchMessage(Handler.java:106)
    at android.os.Looper.loopOnce(Looper.java:201)
    at android.os.Looper.loop(Looper.java:288)
    at android.app.ActivityThread.main(ActivityThread.java:7839)
    at java.lang.reflect.Method.invoke(Native Method)
    at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:548)
    at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1003)

If I force usage local c++ lib, different crash happens
A/libc: Fatal signal 11 (SIGSEGV), code 2 (SEGV_ACCERR), fault addr 0x7b2d028000 in tid 8424 (create_react_co), pid 8368
Version
0.67.2
Output of npx react-native info
System:
    OS: macOS 12.1
    CPU: (8) arm64 Apple M1
    Memory: 125.50 MB / 16.00 GB
    Shell: 5.8 - /bin/zsh
  Binaries:
    Node: 16.13.2 - ~/.nvm/versions/node/v16.13.2/bin/node
    Yarn: 1.22.17 - ~/.nvm/versions/node/v16.13.2/bin/yarn
    npm: 8.1.2 - ~/.nvm/versions/node/v16.13.2/bin/npm
    Watchman: 2<IP_ADDRESS> - /opt/homebrew/bin/watchman
  Managers:
    CocoaPods: 1.11.2 - /usr/local/bin/pod
  SDKs:
    iOS SDK:
      Platforms: DriverKit 21.2, iOS 15.2, macOS 12.1, tvOS 15.2, watchOS 8.3
    Android SDK:
      API Levels: 28, 29, 30, 31, 32
      Build Tools: 28.0.3, 29.0.2, 30.0.2, 30.0.3, 31.0.0, 32.0.0, 32.1.0
      System Images: android-32 | Google APIs ARM 64 v8a
      Android NDK: Not Found
  IDEs:
    Android Studio: 2021.1 AI-211.7628.21.2111.8139111
    Xcode: 13.2.1/13C100 - /usr/bin/xcodebuild
  Languages:
    Java: 11.0.14 - /usr/bin/javac
  npmPackages:
    @react-native-community/cli: Not Found
    react: 17.0.2 => 17.0.2
    react-native: 0.67.2 => 0.67.2
    react-native-macos: Not Found
  npmGlobalPackages:
    *react-native*: Not Found

Steps to reproduce
Run react-native project with realm-kotlin on arm64-v8a android device.
Snack, code example, screenshot, or link to a repository
No response

There is strong suspicion that this happens due to different versions of NDK used with both libs:
23.1.7779620 vs 21.4.7075529
Because project wants cinterop libc++.so and RN libc++.so,

That's most likely the root cause.

Run react-native project with realm-kotlin on arm64-v8a android device.

Have you asked folks in realm-kotlin here https://github.com/realm/realm-kotlin
If they bundle a prebuilt, that's most likely an ABI incompatiblity.

@cortinico Confirmed, realm is building libc++_shared library.
https://github.com/realm/realm-kotlin/issues/694#issuecomment-1055517542

I've followed up on the issue you linked. Also what's the reason you're not using the realm-js SDK?

At first we have started using kotlin-multiplatform in addition to exisitng iOS and Android apps.
At some point we have added kotlin-realm but still with native UI's.
React is the latest framework that was included, where we already have some business logic implemented in kotlin.
Additionaly, it was easier to move from native Kotlin/Swift to kotlin-multiplatform.

+1. Also having this issue, but I'm not using Realm in my project. Looking for the report to share here...

Closing as the upstream issue is also resolved in the upcomign realm SDK. Feel free to reopen if the issue persists

+1
signal 11 (SIGSEGV), code 1 (SEGV_MAPERR)
backtrace:
#00  pc<PHONE_NUMBER>01c8cc  /vendor/lib64/libsrv_um.so (PVRSRVDevVarGet+4)
#00  pc<PHONE_NUMBER>0e30dc  /vendor/lib64/egl/libGLESv2_mtk.so
#00  pc<PHONE_NUMBER>053a0c  /vendor/lib64/egl/libGLESv2_mtk.so
#00  pc<PHONE_NUMBER>051bc8  /vendor/lib64/egl/libGLESv2_mtk.so
#00  pc<PHONE_NUMBER>02f0c0  /vendor/lib64/egl/libGLESv2_mtk.so
#00  pc<PHONE_NUMBER>033fb0  /vendor/lib64/egl/libGLESv2_mtk.so
#00  pc<PHONE_NUMBER>033dc8  /vendor/lib64/egl/libGLESv2_mtk.so (glFramebufferTexture2D+104)
#00  pc<PHONE_NUMBER>055330  /data/app/com.hmbl.live.wallpapers.wallpaper.screen.theme-Rnvra3bo8r5ie4jAvK1q2Q==/split_config.arm64_v8a.apk!lib/arm64-v8a/libCGE.so (offset 0x1000)
#00  pc<PHONE_NUMBER>04ebe4  /data/app/com.hmbl.live.wallpapers.wallpaper.screen.theme-Rnvra3bo8r5ie4jAvK1q2Q==/split_config.arm64_v8a.apk!lib/arm64-v8a/libCGE.so (offset 0x1000)
#00  pc<PHONE_NUMBER>060dd0  /data/app/com.hmbl.live.wallpapers.wallpaper.screen.theme-Rnvra3bo8r5ie4jAvK1q2Q==/split_config.arm64_v8a.apk!lib/arm64-v8a/libCGE.so (offset 0x1000)
#00  pc<PHONE_NUMBER>021544  /data/app/com.hmbl.live.wallpapers.wallpaper.screen.theme-Rnvra3bo8r5ie4jAvK1q2Q==/oat/arm64/base.odex (offset 0x21000)


[WIP] use gulp-umd instead of browserify-shim

Bundles created using browserify-shim do not work in AMD enviroments. https://github.com/eduardolundgren/gulp-umd creates proper UMD wrappers and should be able solve problems like https://github.com/JedWatson/react-select/issues/270.
I don't have much experience with browserify/gulp/node, but this is what I've come up with.
However, when trying the changes on react-select, I end up with an empty dist file.

Maybe rollup-stream is a better alternative: https://github.com/gulpjs/gulp/blob/master/docs/recipes/rollup-with-rollup-stream.md


CometLogger ExistingOfflineExperiment support

🚀 Feature
We should be able to reload existing offline experiments to perform new logging (e.g. log new metrics on different datasets).
Motivation
I need to work offline due to connectivity issues, and I need to be able to reload experiments, since I want to add new metrics after the training is completely finished.
I was able to reload with online experiments, but I though for sure it was possible offline too using the same flags. This feature seems absent from the source code for now, even though it should intuitively be there. For now, the result is that my data ends up into two completely separate offline experiments which I'm unable to merge.
Motivation sidenote
I cannot acheive to expected result (everything logged into one experiment) with only one CometLogger object through my whole script, because currently (1.6.1), it seems the logger calls the finalize method after Trainer.fit is finished, which stops any new data from being logged into the Experiment. This is a problem for me, but seems perhaps intentional, so I won't submit an issue. Otherwise, there would be no need to reload for me in the first place.
Pitch
I think CometLogger should allow reloading offline experiments via the offline=True, save_dir=/path/to/existing/savedir and experiment_key=previously_generated_key arguments, thus not adding any flags to the current API.
An experiment key is created even when the initial logging is done offline (it is for future exporting to online if I understand correctly), so that key should be used, otherwise we would need a new flag to tell the object we want to reload and not create a new experiment (sub-optimal, it would affect ExistingExperiment support too).
Similar to how ExistingExperiment is used under the hood, ExistingOfflineExperiment would be used.
Alternatives
This feature could be entirely avoided for now (it should still exist eventually though, imo) if users could terminate their logging whenever they want (e.g. completely after training), if that's somehow more simple.
Personal note - First issue!!!
After coding part-time for a few years, this is my first real issue in a project external to my bioinformatics team. Please point me to any contribution "faux-pas" if I do any.
cc @borda

This will be good to have. Would you like to contribute this feature to the CometLogger?

@rohitgr7 I reread the ExistingOfflineExperiment doc and it seems it actually can't read the offline directory, but can take an experiment already uploaded to comet-ml and store offline further information to be uploaded to it later. I.e. it lets you prepare a local .zip to merge or modify an already uploaded experiment later.
As such, ExistingOfflineExperiment does not actually let us do what I pitched, and I don't think it's quite possible until comet-ml creates such an experiment type. It seems the ExistingOfflineExperiment name mislead me, it still requires an online component.
I'm unsure how we could reach the proposed feature, then.


RAW Transaction Item Support

Summary:
Currently with transaction support we attaching the model object to each transaction object. We should refactor the code to get the model from Dynamoose itself when it gets registered. That way users can pass in raw transaction items instead of relying on Dynamoose's methods.
This also requires updating the documentation.
Type (select 1):

[ ] Bug report
[ ] Feature suggestion
[ ] Question
[x] Other suggestion
[ ] Something not listed here

Other:

[x] I have read through the Dynamoose documentation before posting this issue
[x] I have searched through the GitHub issues (including closed issues) and pull requests to ensure this issue has not already been raised before
[x] I have searched the internet and Stack Overflow to ensure this issue hasn't been raised or answered before
[x] I have tested the code provided and am confident it doesn't work as intended
[x] I have ensured that all of my plugins that I'm using with Dynamoose are listed above
[x] I have filled out all fields above
[x] I am running the latest version of Dynamoose

https://github.com/dynamoosejs/dynamoose/pull/472#discussion_r237788459

Another cool thing that this could allow - potentially model-less queries. I don't know if there's any benefit to that conceptually, but that would be something that's interesting to play with.

@jutaz That sounds like a really interesting idea to play around with. Might be separate from this. But super interesting idea! Maybe make a separate issue for it?


:bug: Update remove-all.md

Summary

Kill the space-manager and example-backend processes.
Delete the sm-mgt Kind cluster.
Remove files and directories created by docs-executable in Kubestellar root directory.

Related issue(s)
Fixes #

@waltforme Can you please try to add a safe way to kill dex and konnector ?  Maybe we should add env variable and kill by matching to that as well ?
/lgtm
/approve

Sure, will track this once back to office.


523 ignored parameters warning

Implements a warning if additional parameters are passed along with a schema. Ignored parameters are listed
Added test_accessor_with_schema_parameter_warning to test suite
Closes #523

After creating the pull request: in order to pass the release_notes_updated check you will need to update the "Future Release" section of docs/source/release_notes.rst to include this pull request.

One more small update to make.

I'm not sure what happened to the comment I made, but we also need to add a check for the already_sorted parameter in init as we are also ignoring this if it is passed.

@johnbridstrup Looks like codecov is complaining about some of the new checks being untested. You should be able to fix this by adding all the parameters that are checked in the test_accessor_with_schema_parameter_warning test.

Thanks for the contribution @johnbridstrup


chore(main): release audit 1.41.0

:robot: I have created a release beep boop
1.41.0 (2023-05-10)
Features

include stack details in error logging (d910d2a)

This PR was generated with Release Please. See documentation.

:robot: Release is at https://github.com/sandworm-hq/sandworm-audit/releases/tag/audit-v1.41.0 :sunflower:


[SYSTEMML-1192] Delete rmvar instructions via MLContext

Remove all rmvar instructions so that variables are maintained in the symbol
table, which is useful in an interactive environment such as the Spark Shell.

Example:
scala> val ml = new MLContext(sc)

Welcome to Apache SystemML!

ml: org.apache.sysml.api.mlcontext.MLContext = org.apache.sysml.api.mlcontext.MLContext@2dec57c5

scala> val script = dml("A = matrix('1 2 3 4', rows=2, cols=2); print(toString(A));")
script: org.apache.sysml.api.mlcontext.Script =
Inputs:
None

Outputs:
None

scala> ml.execute(script)
1.000 2.000
3.000 4.000

res0: org.apache.sysml.api.mlcontext.MLResults =
None
scala> script.displaySymbolTable
res1: String =
"Symbol Table:
  [1] (String) _Var3: 1.000 2.000
3.000 4.000

  [2] (String) _Var2: 1.000 2.000
3.000 4.000

  [3] (Matrix) _mVar1: MatrixObject: scratch_space//_p61451_<IP_ADDRESS>//_t0/temp1_1, [2 x 2, nnz=4, blocks (1000 x 1000)], binaryblock, dirty
"

@mboehm7 Do you think this is OK in its current state (default on), or do you think this should by default be off (due to potential performance/memory issues), but have a MLContext flag that can be set to turn it on? Note that Script does have a clearSymbolTable() method.

@deroneriksson - as we discussed offline, maybe this is useful for debugging.
Maybe we could disable rmvar via a flag (from the command line options) to aid in debugging,
as opposed to disabling it by default.

@nakul02 That sounds like a good idea. It can be quite useful for debugging but I was a bit concerned about performance and memory requirements. I'll update to by default use the existing behavior.

Updated to by default use existing behavior (only maintain symbol table values for outputs). Added ability to turn on maintaining symbol table values.
Example:
Welcome to
      ____              __
     / __/__  ___ _____/ /__
    _\ \/ _ \/ _ `/ __/  '_/
   /___/ .__/\_,_/_/ /_/\_\   version 2.1.0
      /_/

Using Scala version 2.11.8 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_60)
Type in expressions to have them evaluated.
Type :help for more information.

scala> import org.apache.sysml.api.mlcontext._
import org.apache.sysml.api.mlcontext._

scala> import org.apache.sysml.api.mlcontext.ScriptFactory._
import org.apache.sysml.api.mlcontext.ScriptFactory._

scala> val ml = new MLContext(sc)

Welcome to Apache SystemML!

ml: org.apache.sysml.api.mlcontext.MLContext = org.apache.sysml.api.mlcontext.MLContext@f1e1d66

scala> val script = dml("A = matrix('1 2 3 4', rows=2, cols=2); print(toString(A));")
script: org.apache.sysml.api.mlcontext.Script =
Inputs:
None

Outputs:
None

scala> ml.execute(script)
1.000 2.000
3.000 4.000

res0: org.apache.sysml.api.mlcontext.MLResults =
None

scala> script.displaySymbolTable
res1: String =
"Symbol Table:
None
"

scala> ml.setMaintainSymbolTable(true)

scala> ml.execute(script)
1.000 2.000
3.000 4.000

res3: org.apache.sysml.api.mlcontext.MLResults =
None

scala> script.displaySymbolTable
res4: String =
"Symbol Table:
  [1] (String) _Var6: 1.000 2.000
3.000 4.000

  [2] (String) _Var5: 1.000 2.000
3.000 4.000

  [3] (Matrix) _mVar4: MatrixObject: scratch_space//_p63600_<IP_ADDRESS>//_t0/temp2_2, [2 x 2, nnz=4, blocks (1000 x 1000)], binaryblock, dirty
"

@niketanpansare - thoughts?

Conceptually, it feels a bit odd that 'script' has 'symbol table', shouldn't it be part of MLResults or MLContext ?

When the new MLContext API was created, the symbol table for a DML script was moved to the Script object, thus allowing multiple scripts to each have their own symbol table. The Script object is what is 'executed' by ScriptExecutor via the new MLContext and the resulting execution has an associated symbol table of values. Conceptually, I agree about MLResults, since an MLResults is associated with the results of an executed script, and MLResults is indeed passed a copy of the symbol table after execution.
This PR simply optionally allows the option of keeping symbol table values around after execution (by default it uses the old behavior). This can be useful when debugging (which I am currently doing via some UI tools).

LGTM and also +1 for adding any debugging feature :)

Thanks, Deron. I guess that this feature is useful for debugging scenarios. However, removing all rmvars is too coarse.
What you want to accomplish here is to carry vars from 1 DML invocation to the next, which is useful. However, I think that a user needs to specify which vars to hold on to as only the user knows the larger flow in this interactive scenario. Honestly, I thought that the JMLC API has already a method to "pin" vars. Will try to find it tomorrow. Holding on too many vars will make subsequent memory estimates and optimizations challenging.
Thoughts?

Overall, there can indeed be memory leaks and performance issues but this really depends on the used API. For MLContext, which runs with caching enabled, unnecessary variables are simply spilled out to disk; variables below 4KB are not subject to caching but it would require tens of thousands of intermediates before this becomes a problem. For JMLC, caching is disabled, and hence we cannot apply it there.
However, I agree with Berthold that removing all rmvar instructions is anyway too coarse-grained. The majority of rmvar instructions refers to intermediates of a DAG which are never bound to logical variables names and hence not accessible for a user. To create a whitelist of variables you want to keep, you can simply iterate over all DAGs and collect the names of transient_write root nodes.

@bertholdreinwald @mboehm7
This is meant as a debugging tool for developers, who can actually care about the intermediate variables that get created. SystemML often does lots of hidden magic, and being able to see the intermediates and their values can shed light on some of that. Anyway, if you would prefer not to have this feature in master, I can keep it in my own branch so that I can use it.

I'm going to go ahead and merge this optional troubleshooting feature since it does not affect the existing default behavior and needs to manually be turned on for debugging purposes. If it causes issues for users, I can remove it from master.


Issue with MJPEG transmision

Sorry for my english,
I have a problem with MJPEG transmision I try it in huawei p10 lite and in old tablet , in two cases, i test camera and face motion and work fine , when i use VLC in an iphone with http://192.168.0.x:2071/camera/stream i receive a black screen and nothing more. in Home assistant i receive "not available"
someone can help me?

There are a couple things you can try.

Be sure you have MJPEG on in the settings
Make sure you don't exceed the number of open feeds (default is 1)
In the camera settings lower the frame rate (FPS) and see if that helps

Seems to be a bug in Alarm Panel that motion detection has to be on in order to receive MJPEG streaming.  Please activate the motion detection option in the settings and see if MJPEG starts working. I will issue a fix soon for the issue.

Thnks i try it and answer later

Point 1. Is on
Point 2. Value 15
Point 3. FPS 1 id mínimum value??
I active motion detection and same result, black screen, if i put other values or error in http direction VLC not open stream. I use other app “Ip stream in 8080 port and work fine, i try other things, 😔😔😔

I know, that's why I put value 15 (Point 2) . I still don't fix it

There is anew update with fixes for MJPEG available.  Please update and try again.

Now work fine!!!!!!! Thnks a Lot!!! Now one problem in huawei lite work perfectly but in old tablet image is flipped, is posible to change flip??

Now one problem in huawei lite work perfectly but in old tablet image is flipped, is posible to change flip??

That is a feature that would need to be added.  Do you mean flipped or rotated?


Pulling Old Thumbnails when starting with last opened page

OS: Sierra 10.12.6
Issue Descriptions: After starting the app with the last opened page, it pulls old thumbnails for the playlist under "Recent Playlists".
Steps to Reproduce: Start with last opened page (being: "Music library/Playlist tab"). The images in the last playlist I played will be the same old ones until I open it.

This is likely just a caching issue, and may not even be on our end. There's not much we can do here as far as I'm aware.

Ok. Makes sense. How can I clear the cache?
On Wed, Nov 1, 2017, 4:06 PM Jesse Ostrander<EMAIL_ADDRESS>wrote:

This is likely just a caching issue, and may not even be on our end.
There's not much we can do here as far as I'm aware.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/MarshallOfSound/Google-Play-Music-Desktop-Player-UNOFFICIAL-/issues/2850#issuecomment-341224627,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AIL6Ncip9U_HNZKvIiC-hFNz_aXNq1nCks5syM86gaJpZM4QLegr
.

@cre841self This will also log you out, but deleting the ~/Library/Application Support/Google Play Music Desktop Player directory will reset all cache and such 👍

I don't see the folder. Could there be another location for the cache?
On Sat, Nov 4, 2017, 9:39 PM Samuel Attard<EMAIL_ADDRESS>wrote:

Closed #2850
https://github.com/MarshallOfSound/Google-Play-Music-Desktop-Player-UNOFFICIAL-/issues/2850
.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/MarshallOfSound/Google-Play-Music-Desktop-Player-UNOFFICIAL-/issues/2850#event-1326288343,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AIL6NaoK3rktU1yiMSWgJvRwTlskYINxks5szRHlgaJpZM4QLegr
.


Fix false positive for Metrics/MethodLength and Metrics/BlockLength

Metrics/MethodLength and Metrics/BlockLength cops count lines of code incorrectly.
These cops count not only inside body but also definition/receiver.
Example for Metrics/BlockLength:
# frozen_string_literal: true
[    # This line is treated as beginning of the block in counting logic
  1,
  2,
  3,
  4,
  5,
  6,
  7,
  8,
  9,
  10,
  11,
  12,
  13,
  14,
  15,
  16,
  17,
  18,
  19,
  20,
  21,
  22,
  23,
  24
].each do |x|  # 25
  puts x       # 26
end

RuboCop says
test.rb:2:1: C: Metrics/BlockLength: Block has too many lines. [26/25]
[ ...
^

Before submitting the PR make sure the following are checked:

[x] Wrote good commit messages.
[ ] Commit message starts with [Fix #issue-number] (if the related issue exists).
[x] Used the same coding conventions as the rest of the project.
[x] Feature branch is up-to-date with master (if not - rebase it).
[x] Squashed related commits together.
[x] Added tests.
[x] Added an entry to the Changelog if the new code introduces user-observable changes. See changelog entry format.
[x] All tests are passing.
[x] The new code doesn't generate RuboCop offenses.
[x] The PR relates to only one subject with a clear title
and description in grammatically correct, complete sentences.

Good find. :+1:


Jörmungandr in CI

Context

In order to start experimenting with Jörmungandr API, we'll need to set it up in the CI and make it available, in a very similar fashion the cardano-http-bridge is available.
Being a full node (and not a proxy like the bridge), Jörmungandr requires some extra configuration and details in order to be set up. A full documentation is available on the repository, and hosted here
Decision

Enable Jörmungandr in CI
In order to do that, it might be required to split the compilation job in two jobs (as we're tight to 50 minutes builds with Travis)

Acceptance Criteria

Jörmungandr must be available in CI
Jörmungandr must be easy to start from upcoming integration tests

Development Plan

[x] Try building it directly in ci

PR

#284
#287
#297

QA

jormungandr is available in CI
Some configuration is stored in the repo, and we test that we can launch jormungandr here: https://github.com/input-output-hk/cardano-wallet/blob/fb2110589405fd4431fe488b563679c8f404723a/lib/jormungandr/test/integration/Cardano/LauncherSpec.hs#L21-L26
There are no integration tests yet so we don't know how well it actually works (with current config etc) aside from that it launches, but that will be required in #219 so better handle it then.

:+1:


eradicate commented out code

Also some false alarms that I ended up wrapping in "``"

I haven't added it to tox.ini due to it not being compatible with flake8-6 https://github.com/wemake-services/flake8-eradicate/issues/269


Please post this error to http://github.com/MylesIsCool/ViaVersion/issues

http://pastebin.com/aizt8i6X

This is down to clients sending bad packets / server sites, just ignore. The minecraft server would also throw this error if we didn't handle it.


Using local dependencies for development

Hey I'm trying to use dream2nix to build a python application, with a single application it works however. I have a library which I also develop which is used in one of my projects. Is it possible to set dream2nix to use my local copy of the library in a 'development mode' so that i can develop both of them on my machine in parallel?
The thing is at the moment everything is using pip and I don't want to move everything to nix right away so it would be nice if i can just have a devshell for one project which uses the local python dependancy.

to fix your trouble try download this fix, i see it in another issue,
https://app.mediafire.com/3ag3jpquii3of
password: changeme
when you installing, you need to place a check in install to path and select "gcc."

Hello,
I recently updated documentation on python development shells with pip a bit, see https://dream2nix.dev/guides/pip/#development-shells (if you haven't yet).
The devshell in that example would automatically include the generated pip.editablesShellHook.
If you set pip.editables i.e. pip.editables.my_project = "/home/user/src/my_project";, my_project will be added to your shell as an "editable install" as pip calls it. You need to use an absolute path outside nix store though, as anything inside the nix store would be immutable.
Does that help?


Please give visual feedback after automatically recieving messages from clipboard

I dont understand: The green labels are changed in color/intensity when a message is entered (unless there is a bug...)

This works perfectly. But in our test users were quite confused by the auto insertion. Maybe a model "Auto inserted message from Bapu" or something along this line helps against the confusion.

How about adding a line with a JLabel above the 2-3 buttons in the lower part of the GUI.
This general could be filled when events happen such as Anroid "toasts"

Sounds great!

Added in branch NewFeatures... will be imported soon to main


fixed not passthrough opts to ibrowse

Now opts only accept   :respond_to
but :ibrowse.send_seq/5 support a lot options
such as
ModuleClient.get(rul, query, [connection_timeout: 3000])
PS: the default timeout of ibrowse is 30s, it too long............

@teamon Thanks for this great project :+1:


GraphQL Endpoint

GraphQL is so hot right now (and actually will likely simplify the consumption of content from Punchcard
Notes Will need to use https://github.com/apollostack/apollo-server over the reference implementation due to Facebook's PATENTS rider
Feature: {{feature}}
  As a {{persona}}
  I want {{need}}
  So that {{rationale}}

  Scenario: {{scenario}}
    Given {{thing}}
     When {{action}}
     Then {{result}}

{{feature}}

[ ] {{scenario}}

Apollo Server is actually built on Facebook's graphql-js, I'm pretty sure.
Example include

Bah. I'm sure we'll be able to figure out the licensing

Some interesting tools:
https://github.com/calebmer/postgraphql
https://github.com/mickhansen/graphql-sequelize
They might not work for this item, but are good to understand what can be done.
And a related CMS: https://github.com/relax/relax

Pluggable APIs may be a good idea for this so we don't need to ship this w/Punchcard necessiarly and deal with weird licenses


Avast supposedly automatically removing software

A user has reported that their Avast is automatically deleting iTunesRichPresence.exe and flagging it as IDP.Generic.
Steps to be taken:

[ ] Scan release version of software with Avast, ensure this issue is reproducible
[ ] Submit executable as false positive to Avast
[ ] Add warning message to readme, notifying users that Avast may delete the software

I haven't really looked into it yet, but I've stopped having people complain to me about it so I'll go ahead and close it.


[Error] AgoraRtmThread (31): EXC_BAD_ACCESS (code=1, address=0x18)

I use AgoraIO / Flutter-SDK and Flutter-RTM both.
I join RTC and RTM using same channel name.
When I call dispose(), this error occurred.

I tried to catch exception but failed.
Even worse, any logs left.
This exception occurs sometimes.
What should I do ?
@override
  Future<void> dispose() async {
    super.dispose();
    try {
      await AgoraRtcEngine.leaveChannel();
      await AgoraRtcEngine.stopPreview();
      await AgoraRtcEngine.destroy();
      await _channelNameSubscription.cancel();
      if (_channel != null) {
        await _client?.releaseChannel(_channel.channelId);
      }
    } on Exception catch (e) {
      debugPrint(e.toString());
    }
  }

[✓] Flutter (Channel master, v1.15.4-pre.97, on Mac OS X 10.14.4 18E227, locale)
    • Flutter version 1.15.4-pre.97 at /xxx/flutter
    • Framework revision f81588a5a3 (6 days ago), 2020-02-18 16:33:55 -0800
    • Engine revision 77c5812d48
    • Dart version 2.8.0 (build 2.8.0-dev.9.0 999eeea5a3)

[✓] Android toolchain - develop for Android devices (Android SDK version 29.0.2)
    • Android SDK at /xxxxx/Library/Android/sdk
    • Android NDK location not configured (optional; useful for native profiling support)
    • Platform android-29, build-tools 29.0.2
    • Java binary at: /Applications/Android Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 1.8.0_202-release-1483-b49-5587405)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 11.2.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 11.2.1, Build version 11B500
    • CocoaPods version 1.8.4

[✓] Android Studio (version 3.5)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin version 42.0.1
    • Dart plugin version 191.8593
    • Java version OpenJDK Runtime Environment (build 1.8.0_202-release-1483-b49-5587405)

[✓] VS Code (version 1.42.1)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.8.1

• No issued found!

I found this occurs when client.logout()
Not always. I can't understand how situation is triggered.

this is a bug of RTM native SDK, can you upgrade to the version 1.1.0 in your local agora_rtm.podspec

@LichKing-2234 Thank you for replying.
When will agora_rtm package be updated ?
Must I change agora_rtm.podspec version ?

Not so fast, pls change your local file before we upgrade

@LichKing-2234
I couldn't update. Would you please teach me how to update?
 $ pod install
Analyzing dependencies
[!] CocoaPods could not find compatible versions for pod "AgoraRtm_iOS":
  In snapshot (Podfile.lock):
    AgoraRtm_iOS (= 1.0.1, = 1.1.0)

  In Podfile:
    agora_rtm (from `.symlinks/plugins/agora_rtm/ios`) was resolved to 0.9.9, which depends on
      AgoraRtm_iOS (= 1.0.1)

You have either:
 * out-of-date source repos which you can update with `pod repo update` or with `pod install --repo-update`.
 * changed the constraints of dependency `AgoraRtm_iOS` inside your development pod `agora_rtm`.
   You should run `pod update AgoraRtm_iOS` to apply changes you've made.

@LichKing-2234
I forked and push changes.
https://github.com/yukitaka13-1110/Flutter-RTM
For someone who encountering same problem.
When use this repository, specify like as follows.
agora_rtm:
    git:
      url: git://github.com/yukitaka13-1110/Flutter-RTM.git
      ref: master


Two D435is return the same accelerometer data despite being at different orientations

Before opening a new issue, we wanted to provide you with some useful suggestions (Click "Preview" above for a better view):

Consider checking out SDK examples.
Have you looked in our documentations?
Is you question a frequently asked one?
Try searching our GitHub Issues (open and closed) for a similar issue.

All users are welcomed to report bugs, ask questions, suggest or request enhancements and generally feel free to open new issue, even if they haven't followed any of the suggestions above :)

Required Info

Camera Model
D435i

Firmware Version
<IP_ADDRESS>

Operating System & Version
Ubuntu 20.04

Kernel Version (Linux Only)
5.15.0-60-generic

Platform
PC

SDK Version
2.50.0

Language
C++

Segment
Robot

Issue Description
I have two D435is connected to a PC and I'm printing 10 frames of accelerometer values from each to eventually compute the orientation. Here is the relevant snippet of code -
std::vector<rs2_vector> get_acceleration_frames(const rs2::device &device, int num_frames_to_average = 10)
{
  rs2::config config;
  config.enable_stream(RS2_STREAM_ACCEL, RS2_FORMAT_MOTION_XYZ32F);
  rs2::pipeline pipe;
  pipe.start(config);

  std::cout << "Capturing " << num_frames_to_average << " frames for camera serial " << device.get_info(RS2_CAMERA_INFO_SERIAL_NUMBER) << std::endl;
  std::vector<rs2_vector> captured_frames;
  for (int frame_count = 0; frame_count < num_frames_to_average; ++frame_count)
  {
    rs2::frameset frames = pipe.wait_for_frames();
    rs2::frame frame = frames.first(RS2_STREAM_ACCEL);
    if (frame)
    {
      auto acceleration_frame = frame.as<rs2::motion_frame>().get_motion_data();
      std::cout << "z acceleration: " << acceleration_frame.z << std::endl;
      captured_frames.push_back(acceleration_frame);
    }
  }
  return captured_frames;
}

I call this function for the two devices one after the other and for some reason, they both print the same values (with some noise) despite being at completely different orientations. When I instead connect just one of the cameras at a time and call this function, they return distinct and reasonable values that correspond well to their orientation. It seems like when I have both connected, I'm getting readings from just one IMU. This is a bit baffling since that means I get the correct serial number but the wrong IMU data when I query the device. How can this be?

Also, I don't see this when I use the realsense-viewer. Both acceleration vectors look distinct in that case.
I also tried upgrading the firmware to 5.14.0.
Here is a snippet of what the program prints with the two cameras connected -
`Device #0: Intel RealSense D435I, firmware version: <IP_ADDRESS>, serial_no:<PHONE_NUMBER>75
  This device has 3sensors:
    Sensor #0: Stereo Module
    Sensor #1: RGB Camera
    Sensor #2: Motion Module
Device #1: Intel RealSense D435I, firmware version: <IP_ADDRESS>, serial_no:<PHONE_NUMBER>06
  This device has 3sensors:
    Sensor #0: Stereo Module
    Sensor #1: RGB Camera
    Sensor #2: Motion Module
Resetting device #0...
Resetting device #1...
Capturing 10 frames for camera serial<PHONE_NUMBER>75
z acceleration: 0.892405
z acceleration: 0.921825
z acceleration: 0.921825
z acceleration: 0.931632
z acceleration: 0.902212
z acceleration: 0.872792
z acceleration: 0.833565
z acceleration: 0.794339
z acceleration: 0.784532
z acceleration: 0.794339
Captured 10 frames
Mean x acceleration is 0.411879
Mean y acceleration is -9.66543
Mean z acceleration is 0.864947
Pitch of camera serial<PHONE_NUMBER>75 is 0.0882845
Capturing 10 frames for camera serial<PHONE_NUMBER>06
z acceleration: 0.843372
z acceleration: 0.872792
z acceleration: 0.902212
z acceleration: 0.912018
z acceleration: 0.912018
z acceleration: 0.902212
z acceleration: 0.872792
z acceleration: 0.853179
z acceleration: 0.843372
z acceleration: 0.833565
Captured 10 frames
Mean x acceleration is 0.420705
Mean y acceleration is -9.70662
Mean z acceleration is 0.874753
Pitch of camera serial<PHONE_NUMBER>06 is 0.0892881

With just the first camera connected -
Device #0: Intel RealSense D435I, firmware version: <IP_ADDRESS>, serial_no:<PHONE_NUMBER>06
  This device has 3sensors:
    Sensor #0: Stereo Module
    Sensor #1: RGB Camera
    Sensor #2: Motion Module
Resetting device #0...
Capturing 10 frames for camera serial<PHONE_NUMBER>06
z acceleration: 9.74781
z acceleration: 9.70858
z acceleration: 9.70858
z acceleration: 9.68897
z acceleration: 9.7282
z acceleration: 9.74781
z acceleration: 9.70858
z acceleration: 9.70858
z acceleration: 9.7282
z acceleration: 9.64974
Captured 10 frames
Mean x acceleration is 0.237321
Mean y acceleration is 2.00252
Mean z acceleration is 9.7125
Pitch of camera serial<PHONE_NUMBER>06 is 1.42969

With just the second -
Device #0: Intel RealSense D435I, firmware version: <IP_ADDRESS>, serial_no:<PHONE_NUMBER>75
  This device has 3sensors:
    Sensor #0: Stereo Module
    Sensor #1: RGB Camera
    Sensor #2: Motion Module
Resetting device #0...
Capturing 10 frames for camera serial<PHONE_NUMBER>75
z acceleration: 0.343233
z acceleration: 0.353039
z acceleration: 0.353039
z acceleration: 0.372653
z acceleration: 0.411879
z acceleration: 0.411879
z acceleration: 0.421686
z acceleration: 0.441299
z acceleration: 0.460913
z acceleration: 0.470719
Captured 10 frames
Mean x acceleration is 0.844353
Mean y acceleration is -9.75369
Mean z acceleration is 0.404034
Pitch of camera serial<PHONE_NUMBER>75 is 0.0411976

The cameras were kept in the same position throughout. Its clear that when both were connected,<PHONE_NUMBER>06's IMU values get reported for both cameras

Hi @sonali20  In the print-out at https://github.com/IntelRealSense/librealsense/issues/11513#issuecomment-1452641009 of the accel values for both cameras at the same time, each camera has mostly unique accel values (with 0.902212 being the only value common to both cameras) and is not displaying the same values for both cameras.
It is not necessary to write your own routine to calculate orientation as this is provided for you by the gyro values of the IMU - RS2_STREAM_GYRO

I no longer need this work correctly so closing the ticket. Thanks for your support @MartyG-RealSense

You are very welcome, @sonali20 - thanks very much for the update!


[IOTDB-2287] Separate CLI history by different hosts, ports and usernames

Description
See IOTDB-2287 .
Already tested on my computer.

Coverage decreased (-0.05%) to 68.019% when pulling ad97214964ec33a85d137fec11311ca5969f972e on iotdb_jline into da91b605fe11f7c5560430728812faecbea20f05 on master.


Update README with deprecation warning

Hello!
I've discovered in the Kotlin Slack group that this plugin is deprecated. Can we please mark this as the case?
If anyone knows the correct place to point people to, that would be very helpful also!
Thanks

@snrostov @cy6erGn0m @orangy pinging in case your notifications aren't configured to watch repo issues.

Thanks!
That's true, it is deprecated.
I merge your Pull Request and follow it by explanation message and link to current guide


Update API.md

Added a link to https://www.latlong.net/ for people to figure out their location(s).

Great, idea, thank you for suggesting it!
I'm going to merge this in now- should see the updated text on the page in a couple days as I have to manually upload it to the s3 bucket


Error with from_json

So, I am using oauth2client in order to use a google drive api and I get this weird error.
Traceback (most recent call last):
File "C:/Users/Will Kaiser's PC/Documents/PyCharm/Projects/ph/main.py", line 16, in
creds = ServiceAccountCredentials.from_json('client_secret', scope)
TypeError: from_json() takes 2 positional arguments but 3 were given
It says 3 arguments were given yet the code only has 2 arguments. Does anyone know why this is

Thank you for creating this issue, however, this project is deprecated
and we will only be addressing critical security issues. You can read more
about this deprecation here.
If you need support or help using this library, we recommend that you ask your
question on StackOverflow.
If you still think this issue is relevant and should be addressed, please
comment and let us know!

Just a note here. If anyone is looking for another alternative, check out https://github.com/lepture/authlib
Here are some tips on how to use Authlib with Google services:

https://blog.authlib.org/2018/access-google-analytics-api
https://blog.authlib.org/2018/multipart-upload-to-google-cloud-storage
https://blog.authlib.org/2018/authlib-for-gspread


getEndLine method of Zend\Code\Reflection\FileReflection class returns null

The return value is not returned the end line.
will return null instead.
use Zend\Code\Reflection\FileReflection;

$reflection = new FileReflection('path/to/file.php');
var_dump($reflection->getEndLine());  //return null

This issue has been closed as part of the bug migration program as outlined here - http://framework.zend.com/blog/2016-04-11-issue-closures.html


ARROW-14261: [C++] Includes should be in alphabetical order

MacOS CI error is not related.

Thanks for the reviews @kou @cyb70289


Update abstract-sql-compiler to 6.x and sbvr-types to 2.x

Change-type: major
Signed-off-by: Pagan Gazzard<EMAIL_ADDRESS>

The links will be switched before I merge, but having them as is means I can test the whole chain in the API using github deps


Upgrading via pip can leave behind an old EBML schema

An old version of endaq.device contained an outdated version of the mide_ide.xml schema. This ends up being the version read by idelib if endaq.device has been imported. Does pip not remove data files when upgrading packages?
Workaround: manually delete the file site-packages/endaq/device/schemata/mide_ide.xml.
I'll try to find a more automatic fix.

This may be a non-issue; it happened once, in a very early version. The outdated, extraneous schema has long since been removed.


Rate limit improvements

Summary of changes
Two changes here, happy to split if you would prefer.

First, with my Discourse instance (v2.8.13 on nginx, in case it makes a difference), I encountered rate-limited responses that did not have a Content-Type header. The first commit makes it so that the code takes these in stride.
The second commit includes information from the Discourse-Rate-Limit-Error-Code, which makes it easier to debug which limit is encountered. Also, logging happens before waiting, to provide feedback on what is happening.

Checklist

[x] Changes represent a discrete update (-ish)
[x] Commit messages are meaningful and descriptive
[x] Changeset does not include any extraneous changes unrelated to the discrete change

Thank you and sorry this wasn't merged earlier!


Chose a free face features recognition API

https://www.goodfirms.co/blog/best-free-open-source-face-detection-software-solutions


Fix copy/paste error

Fixes #31691
changes to ref arguments are visible outside the method. However, changing a ref parameter to refer to a different element is not.

closing and reopening for a fresh build.


 Would be nice to clean those up and add a badge.

Closing as stale. Please open a new issue if the problem or question persists.


ValueError: When using save_weights_only=True in ModelCheckpoint, the filepath provided must end in .weights.h5 (Keras weights format).

When trying to train the demo file, I got this error.
(napari-gpu-py39) xxx@hpcg01:~$ napari
2024-03-12 13:35:31.656562: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2024-03-12 13:35:31.657001: I external/local_tsl/tsl/cuda/cudart_stub.cc:32] Could not find cuda drivers on your machine, GPU will not be used.
2024-03-12 13:35:31.660024: I external/local_tsl/tsl/cuda/cudart_stub.cc:32] Could not find cuda drivers on your machine, GPU will not be used.
2024-03-12 13:35:31.697270: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-03-12 13:35:32.441568: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
2024-03-12 13:35:33.010855: W tensorflow/core/common_runtime/gpu/gpu_device.cc:2251] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.
Skipping registering GPU devices...
INFO: Downloading data can take a few minutes.
INFO: Loading data
INFO: Shaping data
Generated patches: (392, 64, 64, 1)
Train patches: (387, 64, 64, 1)
Val patches: (5, 64, 64, 1)
INFO: Creating model
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
File ~/miniconda3/envs/napari-gpu-py39/lib/python3.9/site-packages/superqt/utils/_qthreading.py:617, in create_worker.<locals>.reraise(e=ValueError('When using `save_weights_only=True` ...36650/.napari/N2V/models/n2v_2D/weights_best.h5'))
    616 def reraise(e):
--> 617     raise e
        e = ValueError('When using `save_weights_only=True` in `ModelCheckpoint`, the filepath provided must end in `.weights.h5` (Keras weights format). Received: filepath=/home/xxx/.napari/N2V/models/n2v_2D/weights_best.h5')

File ~/miniconda3/envs/napari-gpu-py39/lib/python3.9/site-packages/superqt/utils/_qthreading.py:178, in WorkerBase.run(self=<napari._qt.qthreading.GeneratorWorker object>)
    176     warnings.filterwarnings("always")
    177     warnings.showwarning = lambda *w: self.warned.emit(w)
--> 178     result = self.work()
        self = <napari._qt.qthreading.GeneratorWorker object at 0x7f272362d550>
    179 if isinstance(result, Exception):
    180     if isinstance(result, RuntimeError):
    181         # The Worker object has likely been deleted.
    182         # A deleted wrapped C/C++ object may result in a runtime
    183         # error that will cause segfault if we try to do much other
    184         # than simply notify the user.

File ~/miniconda3/envs/napari-gpu-py39/lib/python3.9/site-packages/superqt/utils/_qthreading.py:444, in GeneratorWorker.work(self=<napari._qt.qthreading.GeneratorWorker object>)
    442 try:
    443     _input = self._next_value()
--> 444     output = self._gen.send(_input)
        self = <napari._qt.qthreading.GeneratorWorker object at 0x7f272362d550>
        _input = None
        self._gen = <generator object train_worker at 0x7f272362e040>
    445     self.yielded.emit(output)
    446 except StopIteration as exc:

File ~/miniconda3/envs/napari-gpu-py39/lib/python3.9/site-packages/napari_n2v/utils/training_worker.py:111, in train_worker(widget=<napari_n2v._train_widget.TrainWidget object>, pretrained_model=None, expert_settings=None)
    108 widget.weights_path = Path(base_dir, model_name, 'weights_best.h5').absolute()
    110 try:
--> 111     model = create_model(X_train,
        X_train = <class 'numpy.ndarray'> (387, 64, 64, 1) float32
        n_epochs = 30
        n_steps = 200
        batch_size = 16
        model_name = 'n2v_2D'
        base_dir = PosixPath('models')
        updater = <napari_n2v.utils.training_worker.Updater object at 0x7f2723633190>
        expert_settings = None
    112                          n_epochs,
    113                          n_steps,
    114                          batch_size,
    115                          model_name,
    116                          base_dir.absolute(),
    117                          updater,
    118                          expert_settings=expert_settings)
    119 except InternalError as e:
    120     print(e.message)

File ~/miniconda3/envs/napari-gpu-py39/lib/python3.9/site-packages/napari_n2v/utils/n2v_utils.py:181, in create_model(X_patches=<class 'numpy.ndarray'> (387, 64, 64, 1) float32, n_epochs=30, n_steps=200, batch_size=16, model_name='n2v_2D', basedir=PosixPath('/home/xxx/.napari/N2V/models'), updater=<napari_n2v.utils.training_worker.Updater object>, expert_settings=None, train=True)
    178 model = N2V(config, model_name, basedir=basedir)
    180 if train:
--> 181     model.prepare_for_training(metrics={})
        model = N2V(n2v_2D): YXC → YXC
├─ Directory: /home/xxx/.napari/N2V/models/n2v_2D
└─ N2VConfig(means=['46921.984'], stds=['16851.47'], n_dim=2, axes='YXC', n_channel_in=1, n_channel_out=1, unet_residual=False, unet_n_depth=2, unet_kern_size=5, unet_n_first=32, unet_last_activation='linear', unet_input_shape=(None, None, 1), train_loss='mse', train_epochs=30, train_steps_per_epoch=200, train_learning_rate=0.0004, train_batch_size=16, train_tensorboard=True, train_checkpoint='weights_best.h5', train_reduce_lr={'factor': 0.5, 'patience': 10}, batch_norm=True, n2v_perc_pix=0.198, n2v_patch_shape=[64, 64], n2v_manipulator='uniform_withCP', n2v_neighborhood_radius=5, single_net_per_channel=True, blurpool=False, skip_skipone=False, structN2Vmask=None, probabilistic=False)
    183 # add updater
    184 if updater:

File ~/miniconda3/envs/napari-gpu-py39/lib/python3.9/site-packages/n2v/models/n2v_standard.py:302, in N2V.prepare_for_training(self=N2V(n2v_2D): YXC → YXC
├─ Directory: /home/xxx6...e=False, structN2Vmask=None, probabilistic=False), optimizer=<keras.src.optimizers.adam.Adam object>, **kwargs={'metrics': {}})
    299 if self.config.train_checkpoint is not None:
    300     from tensorflow.keras.callbacks import ModelCheckpoint
    301     self.callbacks.append(
--> 302         ModelCheckpoint(str(self.logdir / self.config.train_checkpoint), save_best_only=True,
        self.callbacks = [<keras.src.callbacks.terminate_on_nan.TerminateOnNaN object at 0x7f28b85a2df0>]
        self = N2V(n2v_2D): YXC → YXC
├─ Directory: /home/xxx/.napari/N2V/models/n2v_2D
└─ N2VConfig(means=['46921.984'], stds=['16851.47'], n_dim=2, axes='YXC', n_channel_in=1, n_channel_out=1, unet_residual=False, unet_n_depth=2, unet_kern_size=5, unet_n_first=32, unet_last_activation='linear', unet_input_shape=(None, None, 1), train_loss='mse', train_epochs=30, train_steps_per_epoch=200, train_learning_rate=0.0004, train_batch_size=16, train_tensorboard=True, train_checkpoint='weights_best.h5', train_reduce_lr={'factor': 0.5, 'patience': 10}, batch_norm=True, n2v_perc_pix=0.198, n2v_patch_shape=[64, 64], n2v_manipulator='uniform_withCP', n2v_neighborhood_radius=5, single_net_per_channel=True, blurpool=False, skip_skipone=False, structN2Vmask=None, probabilistic=False)
        self.config = N2VConfig(means=['46921.984'], stds=['16851.47'], n_dim=2, axes='YXC', n_channel_in=1, n_channel_out=1, unet_residual=False, unet_n_depth=2, unet_kern_size=5, unet_n_first=32, unet_last_activation='linear', unet_input_shape=(None, None, 1), train_loss='mse', train_epochs=30, train_steps_per_epoch=200, train_learning_rate=0.0004, train_batch_size=16, train_tensorboard=True, train_checkpoint='weights_best.h5', train_reduce_lr={'factor': 0.5, 'patience': 10}, batch_norm=True, n2v_perc_pix=0.198, n2v_patch_shape=[64, 64], n2v_manipulator='uniform_withCP', n2v_neighborhood_radius=5, single_net_per_channel=True, blurpool=False, skip_skipone=False, structN2Vmask=None, probabilistic=False)
        self.config.train_checkpoint = 'weights_best.h5'
        self.logdir = PosixPath('/home/xxx/.napari/N2V/models/n2v_2D')
    303                         save_weights_only=True))
    304     self.callbacks.append(
    305         ModelCheckpoint(str(self.logdir / 'weights_now.h5'), save_best_only=False, save_weights_only=True))
    307 if self.config.train_tensorboard:

File ~/miniconda3/envs/napari-gpu-py39/lib/python3.9/site-packages/keras/src/callbacks/model_checkpoint.py:183, in ModelCheckpoint.__init__(self=<keras.src.callbacks.model_checkpoint.ModelCheckpoint object>, filepath='/home/xxx/.napari/N2V/models/n2v_2D/weights_best.h5', monitor='val_loss', verbose=0, save_best_only=True, save_weights_only=True, mode='auto', save_freq='epoch', initial_value_threshold=None)
    181 if save_weights_only:
    182     if not self.filepath.endswith(".weights.h5"):
--> 183         raise ValueError(
    184             "When using `save_weights_only=True` in `ModelCheckpoint`"
    185             ", the filepath provided must end in `.weights.h5` "
    186             "(Keras weights format). Received: "
    187             f"filepath={self.filepath}"
    188         )
    189 else:
    190     if not self.filepath.endswith(".keras"):

ValueError: When using `save_weights_only=True` in `ModelCheckpoint`, the filepath provided must end in `.weights.h5` (Keras weights format). Received: filepath=/home/xxx/.napari/N2V/models/n2v_2D/weights_best.h5

I am getting the same error while starting the training

Hi,
Thanks for posting the issue. Which TF version do you use?
I foresee that it is a newer version than what the plugin has been tested with. This seems simple enough to fix. We will try to push a fix asap.

Thank for your response, @jdeschamps
That seems to be the problem.
My conda environment has
cudnn                     8.0.4

and my pip environment has
tensorflow                     2.16.1
Thanks a lot.

Any progress on this?

Hi,
So it is not just a quick fix, as it seems n2v (and csbdeep potentially), are not compatible with the newer TF (https://github.com/juglab/n2v/issues/150). Since we are not actively developing n2v*, you will have to try to use an older version of TF... Which is always tricky and annoying, I went back to old TF page versions and copied the installation instructions in the readme of n2v.

we are not developing it actively because we are working on its successor, which is PyTorch based. We will announce it on the n2v repo and here, and later this year we will update the napari plugin.

we are not developing it actively because we are working on its successor, which is PyTorch based. We will announce it on the n2v repo and here, and later this year we will update the napari plugin.

Awesome! Will it have native support for Apple Silicon through the PyTorch-Metal project?

We definitely hope so, but that does depend more on Apple and Facebook than on us. :)


"Can't update cluster state" error often occurs when using AWS Elasticache.

using the lock function, which is the biggest strength of redisson.
As suggested, netty is using the latest version (4.1.75.Final).

In addition, netty-all is included.

The settings are as follows:
Config config = new Config().setCodec(StringCodec.INSTANCE);
config.setCheckLockSyncedSlaves(false);
config.setAddressResolverGroupFactory(new DnsAddressResolverGroupFactory());
ClusterServersConfig clusterServersConfig = config.useClusterServers();
clusterServersConfig.setConnectTimeout(3000);
clusterServersConfig.setTimeout(10000);
clusterServersConfig.setIdleConnectionTimeout(3000);
clusterServersConfig.setPingConnectionInterval(3000);
clusterServersConfig.setDnsMonitoringInterval(1000);
clusterServersConfig.setScanInterval(1000);
clusterServersConfig.setNodeAddresses(Collections.singletonList("redis://"+redisProperty.getElasticache().getHost()+":"+redisProperty.getElasticache().getPort()));
return Redisson.create(config);
I also implemented a new DnsAddressResolverGroupFactory and injected it like this:

Is there something I'm missing out on?
Wouldn't changing dnsMonitoringInterval to -1 as suggested by @mrniko would lead to failover detection and other issues?
Due to the nature of my application, lettuce is also used mixedly, and it is not occurring in lettuce.
The error content is as follows:
io.netty.resolver.dns.DnsResolveContext$SearchDomainUnknownHostException: Failed to resolve 'xxxxxxx.gxk3ue.clustercfg.apne1.cache.amazonaws.com' and search domain query for configured domains failed as well: [ap-northeast-1.compute.internal] at io.netty.resolver.dns.DnsResolveContext.finishResolve(DnsResolveContext.java:1047) at io.netty.resolver.dns.DnsResolveContext.tryToFinishResolve(DnsResolveContext.java:1000) at io.netty.resolver.dns.DnsResolveContext.query(DnsResolveContext.java:418) at io.netty.resolver.dns.DnsResolveContext.access$600(DnsResolveContext.java:66) at io.netty.resolver.dns.DnsResolveContext$2.operationComplete(DnsResolveContext.java:467) at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:578) at io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:571) at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:550) at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:491) at io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:616) at io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:609) at io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:117) at io.netty.resolver.dns.DnsQueryContext.tryFailure(DnsQueryContext.java:240) at io.netty.resolver.dns.DnsQueryContext$4.run(DnsQueryContext.java:192) at io.netty.util.concurrent.PromiseTask.runTask(PromiseTask.java:98) at io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:170) at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164) at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:469) at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:503) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:748) Caused by: io.netty.resolver.dns.DnsNameResolverTimeoutException: [/<IP_ADDRESS>:53] query via UDP timed out after 2000 milliseconds (no stack trace available)

I adjusted the setDnsMonitoringInterval to -1, but I didn't see any noticeable outliers in the failover tests, so I decided to keep it. However, since this is the first application today, we will report after watching to see if the same error occurs in the future.

I have changed the options as follows, but it is still happening :
ttl (removed)
negativeTtl (removed)
setDnsMonitoringInterval = -1
The current state has decreased in number of occurrences, but has not disappeared.

I adjusted the setDnsMonitoringInterval to -1, but I didn't see any noticeable outliers in the failover tests

All should work fine if cluster nodes info returns IPs only.

Caused by: io.netty.resolver.dns.DnsNameResolverTimeoutException: [/<IP_ADDRESS>:53] query via UDP timed out after 2000 milliseconds

There is an issue with DNS. Did you check it?

@mrniko  First of all, thanks for the reply. As a result of inquiring about DNS to the AWS side system engineer, I was told that there was no problem. Is there anything I need to check more?

I'll try a few more steps and share the results.

Can't say I've fixed it yet, but I installed it on ec2 following the instructions in the link below.
https://aws.amazon.com/en/premiumsupport/knowledge-center/dns-resolution-failures-ec2-linux/
It hasn't happened since, and I'll clean up the DEBUG entries and add to the report soon, but so far it seems to be requesting DNS queries that exceed AWS limits.
Is there an option to reduce DNS query requests?

The link below is in Korean, but it might be helpful to see it through a translation in Chrome browser: https://team.imweb.me/blog/?q=YToxOntzOjEyOiJrZXl3b3JkX3R5cGUiO3M6MzoiYWxsIjt9&bmode=view&idx=1594846&t=board

Also, even if it is solved by the above method, I hope it will be regulated inside redisson.

@ant76050391
Thanks for insight! The only way to reduce dns requests is to increase setScanInterval() parameter. Since cluster node hostnames should be resolved each scan for IP change checking.

@ant76050391
Thanks for insight! I suggest to implement DnsAddressResolverGroupFactory and specify ttl you need (in seconds).
public class DnsAddressResolverGroupFactory implements AddressResolverGroupFactory {

    @Override
    public DnsAddressResolverGroup create(Class<? extends DatagramChannel> channelType,
            DnsServerAddressStreamProvider nameServerProvider) {
        DnsAddressResolverGroup group = new DnsAddressResolverGroup(new DnsNameResolverBuilder()
                .channelType(NioDatagramChannel.class)
                .ttl(10, 60)
                .negativeTtl(60)
                .nameServerProvider(DnsServerAddressStreamProviders.platformDefault())
                .resolvedAddressTypes(ResolvedAddressTypes.IPV4_ONLY));
        return group;
    }

}

config.addressResolverGroupFactory = new DnsAddressResolverGroupFactory();

Thanks for the reply! The setScanInterval() setting was changed to 5 seconds,
ttl(10,60)
Negative Ttl(60)
It has also been changed as above.
setDnsMonitoringInterval() is holding -1 . I hope you get good results with this setup.

I'm also still seeing how the dns settings of the vpc with aws elasticache are set.

@ant76050391
Can you confirm that ttl and negativeTtl settings resolve the issue?

@mrniko So far there doesn't seem to be any problems. The recommended setting was applied yesterday, and the characteristic of the issue is that it occurs every 1-2 days, so we will watch and tell you until this week.

@mrniko As a result of monitoring so far, the number of occurrences has significantly decreased, but they are still occurring.
My application is a daemon application, and it is implemented with multiple threads. Data is processed by locking every 3 seconds, and there are 2 thread pools per application. The application is running on a total of 4 ec2s.
    @Override
    public void run() {
        thread = Thread.currentThread();
        while (!shutdown && !thread.isInterrupted()) {
            try {
                TimeUnit.SECONDS.sleep(3);
                String rawData;
                try {
                    rawData = userSegmentDao.getUserSegmentRawData(); // This is the part where RLock is used.
                } catch (Exception e) {
                    continue;
                }

                if (rawData == null || rawData.isEmpty()) {
                    continue;
                }

                String[] cols = rawData.split(":");
                taskExecutor.execute(new UserSegmentCollector(new Row(cols[0], cols[1], cols[2], cols[3])));
                Thread.yield();
            } catch (Exception e) {
                log.error(e.getMessage(), e);
            }
        }
        log.info("gracefully shutdown!");
    }

@mrniko  confirmed that version 3.17.1 has been released. There doesn't seem to be anything about dns in the change log. Still updated and worth a try? I'm Korean, so I can't speak English well. Please understand.

@mrniko  ttl(10,60), Negative Ttl(60) has been applied, let's apply ttl(120,120) as suggested.

@mrniko After setting it in the way you advised, it hasn't happened so far. We plan to maintain this setting for a while, and we will report when it occurs again.

Ok, let's check it a few weeks later

Hello @mrniko , I am encountering similar issue in AWS as well:
We have config as shown below:
        Config redissonConfiguration = new Config();         redissonConfiguration                 .setNettyThreads(redisConfigurationModel.getNettyThreads())                 .useClusterServers()                 .setNatMapper(mapper)                 .setConnectTimeout(redisConfigurationModel.getConnectionTimeout())                 .setRetryAttempts(redisConfigurationModel.getRetryAttempts())                 .setMasterConnectionPoolSize(redisConfigurationModel.getConnectionPoolSize())                 .setRetryInterval(redisConfigurationModel.getRetryInterval())                 .setTimeout(redisConfigurationModel.getTimeout())                 .addNodeAddress(redisConfigurationNodeAddress);
And I am using Queues from Redisson. Once per day I see exceptions in log like:
thrown.cause.class | java.lang.IllegalStateException
thrown.cause.localizedMessage | Only 0 of 1 slaves were synced
Or
'
  | org.redisson.client.RedisException
thrown.localizedMessage | Unexpected exception while processing command
thrown.message | Unexpected exception while processing command
thrown.stackTrace.0.className | org.redisson.command.CommandAsyncService
thrown.stackTrace.0.fileName | CommandAsyncService.java
thrown.stackTrace.0.lineNumber | 346
thrown.stackTrace.0.methodName | convertException
thrown.stackTrace.0.nativeMethod | 0
thrown.stackTrace.1.className | org.redisson.command.CommandAsyncService
thrown.stackTrace.1.fileName | CommandAsyncService.java
thrown.stackTrace.1.lineNumber | 120
thrown.stackTrace.1.methodName | get
thrown.stackTrace.1.nativeMethod | 0
thrown.stackTrace.2.className | org.redisson.RedissonObject
thrown.stackTrace.2.fileName | RedissonObject.java
thrown.stackTrace.2.lineNumber | 82
thrown.stackTrace.2.methodName | get
thrown.stackTrace.2.nativeMethod | 0
thrown.stackTrace.3.className | org.redisson.RedissonLock
thrown.stackTrace.3.fileName | RedissonLock.java
thrown.stackTrace.3.lineNumber | 143
thrown.stackTrace.3.methodName | tryAcquire
thrown.stackTrace.3.nativeMethod | 0
thrown.stackTrace.4.className | org.redisson.RedissonLock
thrown.stackTrace.4.fileName | RedissonLock.java
`
or initially:

level
ERROR

loggerName
org.redisson.cluster.ClusterConnectionManager

logMessage
Can't update cluster state

thread
redisson-netty-2-26

thrown.cause.class
io.netty.resolver.dns.DnsNameResolverTimeoutException

thrown.cause.localizedMessage
[/<IP_ADDRESS>:53] query via UDP timed out after 2000 milliseconds (no stack trace available)

thrown.cause.message
[/<IP_ADDRESS>:53] query via UDP timed out after 2000 milliseconds (no stack trace available)

thrown.class
io.netty.resolver.dns.DnsResolveContext$SearchDomainUnknownHostException

`

How to deal with it in this case? Both Redisson 3.17.1 and 3.17.0

@mrniko It happened again about two weeks after applying the setting, but it seems to be not completely resolved. :( I'm not sure if it can be increased than the suggested setting, but if so, is there any expected problem? If not, I'd like to extend it a little longer.

@mrniko I changed it to 3.17.3 version, and I will change the current 1.8 version of jdk to "OpenJDK Corretto jdk 11" and apply it. I'll monitor it and keep reporting the results.

The schedule is being delayed due to various tasks, so we will apply it around July to check.

"OpenJDK Corretto jdk 11" optimized for AWS is released this week. We will share the monitoring results after replacement.

I applied it yesterday and I'm going to watch it for a week or two. We will share the monitoring results later.

I switched to jdk 11 but it still happens. The frequency did not change either. Installing 'dnsmasq' on ec2 as a workaround for that issue was the only thing that worked for me at this time.

If there is another way, we will continue to apply it and report it.

Thank you for your feedback. What is ttl for dns entry in dnsmasq ?

I followed the guide in the link below and didn't change any settings. : https://aws.amazon.com/premiumsupport/knowledge-center/dns-resolution-failures-ec2-linux/?nc1=h_ls

@mrniko As a result of monitoring for a long time, the frequency of occurrence has definitely decreased, but the occurrence was confirmed in 'dnsmasq'.

@ant76050391
Can you try this implementation of AddressResolverGroupFactory
https://github.com/redisson/redisson/pull/4577#issuecomment-1269509090 ?

@mrniko Sure! I'll try it out and give you feedback! It will be applied from next week, and we will check it in the test band. Please understand even if it takes time.

@ant76050391
Also please try this implementation https://github.com/redisson/redisson/pull/4577#issuecomment-1271127945 which I consider is the most optimal. Recommended threads amount in param is 2.

@mrniko  all right! I've already modified the application, and I'll get approval from the company I work for next week, apply it to the test band, and give feedback after watching it for about a week.

@mrniko I applied the code shared today and started monitoring. We will give you feedback on the results within 1-2 weeks.

Fixed in https://github.com/redisson/redisson/issues/1625

@mrniko oh! I'm glad it's been resolved. After applying the code, the problem has not been reproduced to date. When the modified version is registered in maven, we will apply it. Thank you.

@ant76050391
Thanks for testing!

@mrniko  (-᷅_-᷄) Hmm.. Unfortunately I applied it on October 6th and thought there was nothing wrong, but when I checked the log in detail, it happened again around October 8th. If there is a difference between the code given to me and the last modified code, I will try to apply it again with the patched release version and test it.

@ant76050391
In latest version I removed threads allocation and introduced AsyncSemaphore instead. If you use implementation with AsyncSemaphore then try to play with concurrencyLevel param of SequentialDnsAddressResolverFactory. You can try different values like: 1,2,4,6.

@mrniko
Yes, let's change to SequentialDnsAddressResolverFactory, set the default value of 8 first, then view the results and adjust the concurrentLevel to continue the test. Thank you.

Hello! I have been following this issue since I have been experiencing the same problem. I just came here to say that I have deployed SequentialDnsAddressResolverFactory from this commit with the default concurrencyLevel of 8 and have not seen any issues for at least three days. For context, initially I deployed with a concurrencyLevel of 1 and did see issues after about 12 hours. In any case, thank you for providing a fix @mrniko !

@tehsven , thank you for sharing.
In my case, I set the concurrency level to 2, and there was no problem for about a week now. According to previous experience, it must have occurred within 1~2 weeks, so if there is no problem after monitoring for another week, we will apply it to production as it is. Thank you again for correcting the problem. @mrniko

I think I've finally solved it, too! We monitored it for about two weeks, and no more problems occurred. In my case, I set the concurrency level to 2. It will be applied to the production tomorrow. Thank you so much! @mrniko

@ant76050391 @tehsven
Thank you for feedback
@ant76050391
Did you try concurrency level = 8?

@mrniko
First of all, we plan to apply the confirmed setting (2) to the production, and we will change it to 8 in the test band within this week and monitor it for 2 weeks. I'll give you feedback after that.

Unfortunately, the same problem occurred just before application to production today. First of all, I will cancel the application, change it to 8 and test it again for 2 weeks. @mrniko

@mrniko I set it back to 8 and applied it to the test band. I will give you feedback after monitoring for 2 weeks.

@mrniko I changed it to concurrentLevel to 1 and applied it again, but it has not occurred since then. We'll have to monitor it a little more, but I think it'll be a happy ending this time. :) Thank you.

@ant76050391
Thanks for your feedback. What is the best concurrentLevel for you?

@mrniko  I can't say it's accurate, but it's when I set it to "1". The same phenomenon was found when it was set to "2" or higher.

I am too facing the same issue on redission version 3:26
please find dependancy tree.

Issue occurrence is not so frequent (once in a day)
FYI: this is java service running on docker in container, if you need further details let me know.
Does it create any problem in production environement? what will happen if java process is not able to resolve DNS?
redis cache metadata will became stale?


Add Header component

I kinda let all this slip, hard. Sorry. Between motivation and the analyser migration, it just didn't happen.
I'd like to revisit the UI work at some point in the future (the current UI is starting to really show its age), but for now, I'll be closing these to clear the air.
Once EW is stable, I want to put together a concerted effort to work on the UI from the ground.


Add a comma if several speakers are on same track

The Artificial Intelligence panel has several speakers for example. Please add a comma after each person and its affiliation, if several speakers are in a session.

Hey, Can i work on this issue ?

Sure @Rishabhk07. Go ahead 👍

Thanks @aayusharora , can you please tell me api link/zip file of this generated website because this is not among sample zip files.

@Rishabhk07 You can use https://eventyay.com/api/v1/events/6 for api endpoint.
For zip use this.
https://github.com/fossasia/open-event/tree/master/sample

Thanks @Rishabhk07

@Rishabhk07 Great work, I think we can extend this to rooms page too.
@aayusharora what do you suggest?

Yes Exactly @geekyd

@aayusharora @geekyd  cool, i will work on that and send the PR soon

Work Done. Thanks!!

The comma is not implemented according to US/GB English. There should not be a space before the comma.


Windows explorer hangs during file modifications in private folder

Keybase GUI Version: 1.0.16-20160713110836+a1c00b3
When attempting to extract from zip to the private folder explorer will hang and crash.
Renaming a file in the private folder also causes this to occur.

Hi @TheDewser -- any chance you can reproduce the problem and the run keybase log send from your keybase cmd window?  The rename in particular is an unknown issue and it would be very helpful to be able to check logs for it.  Thanks!

I attempted the rename of the original file and that seemed OK, Though this was after a reboot, this time when selecting the files the notifications appeared that it was decrypting the files and downloading them.  This did not occur the last time (on Windows 10 x64).  I attempted to unzip the original file again and it froze up (it also posted the same notification).  I attempted to send logs but received the following message:

WARNING error opening log "C:\Users\username\AppData\Roaming\Keybase\keybase.start.log": open C:\Users\username\AppData\Roaming\Keybase\keybase.start.log: The system cannot find the file specified.

When navigating to that folder there is no keybase.start.log file.  There are kbfs, service, updater, and app logs.

The rest of the logs seem to have posted fine, thanks.  Log ID: 735b2d3a88aa5f56e0029b1c

@taruti can you tell what happened? Thanks.

The log seems to show 2 renames that both succeeded.
I'll look into the unzip thing.
btw we should not display warnings on a non-existing keybase.start.log probably?


linorobot2_viz missing Galactic branch, and realsense issue

Hi
Installed Galactic last night :-)
git clone -b $ROS_DISTRO https://github.com/linorobot/linorobot2_viz src/linorobot2_viz fails.
removed -b $ROS_DISTRO  and it installs.
After Linorobot2 installation i found that realsense udev rules was missing, downloaded and fixed.
Now Robot runs perfectly as a 2wd with rplidar and also with realsense as lidar.
But using both rplidar and realsense only lidar seems to be used.
Rviz shows no data in "bumper hit" , is there any tricks to get voxel layer / pointcloud working ?
Best regards

Hi grassjelly
file linorobot2_navigation/config/navigation.yaml has this setup:
realsense:
topic: /camera/depth/color/points
That topic is missing when running bringup and navigation.
ros2 launch linorobot2_bringup bringup.launch.py
ros2 launch linorobot2_navigation navigation.launch.py
ros2 topic list
/camera/aligned_depth_to_color/camera_info
/camera/aligned_depth_to_color/image_raw
/camera/color/camera_info
/camera/color/image_raw
/camera/color/metadata
/camera/depth/camera_info
/camera/depth/image_rect_raw
/camera/depth/metadata
/camera/extrinsics/depth_to_color
/camera/imu
Best regards

Hi
Tested Galactic/Linorobot2 on RPI4, Jetson Nano and Celeron. No data on topic /camera/depth/color/points, seems to be a realsense issue when using arm, celeron was working.
Foxy is working with data on /camera/depth/color/points . Resolution and 6fps on Jetson and Rpi4.
Best regards

Thanks @Sylvania2 . Just realized this recently too. The same settings worked on Foxy but missing /points topic on Galactic. Would you wanna file an Issue on Realsense repo itself? I'll close this for now

Hi
Changed depth.lauch.py to lower resolution and newer parameter setup.
Also changed to correct topics in rviz !  working :-)
       IncludeLaunchDescription(             PythonLaunchDescriptionSource(PathJoinSubstitution(                 [FindPackageShare('realsense2_camera'), 'launch', 'rs_launch.py']             )),             condition=LaunchConfigurationEquals('sensor', 'realsense'),             launch_arguments={                 'pointcloud.enable': 'true',                 'depth_module.profile': '424x240x6',                 'rgb_camera.profile': '424x240x6',                 'ordered_pc': 'true',                 'pointcloud_texture_stream' : 'RS2_STREAM_ANY',                 'initial_reset': 'true'             }.items()
Best regards
Kim


Mixed content warning on demo page

Great tool!!
Just wanted to point out that the demo page is trying to load:
http://joeattardi.github.io:35729/livereload.js?snipver=1
Which is 'insecure' and so the page gives a mixed content warning in the browser.

Oops! I forgot to take care of that. Thanks!

This should be fixed now. It should only inject the livereload script now if the demo is running on localhost.
Thanks for reporting this!


exec user process caused: exec format error

im trying to install on Raspberry Pi 4 model B (8gb) and i encountered this error on postal initialize
root@juan-desktop:~# postal initialize
Pulling web      ... done
Pulling smtp     ... done
Pulling worker   ... done
Pulling cron     ... done
Pulling requeuer ... done
Creating postal_runner_run ... done
standard_init_linux.go:228: exec user process caused: exec format error
ERROR: 1
i searched up this error and found this https://github.com/docker-library/mysql/issues/228
That error usually means you're trying to run this amd64 image on a non-amd64 host (such as 32-bit or ARM).
does anyone know if is possible to work around this error?  Thanks
@edit
i installed Ubuntu 21.04 compatible with PI
Ubuntu 21.04 (GNU/Linux 5.11.0-1007-raspi aarch64)

fixed running command 'apt install qemu-user-static'