OV5642 blurr images with movement

Hi, I am too evaluating the OV5642 and have the same blurry images with movement. I have the version with 8Mbyte FIFO. My test program is as follows.
I capture multiple images until the fifo is full.
Then I copy the images to SD card.
When I look at the images I see a few blurry images when there is movement.
From my observation since the images are stored directly to the FIFO it seems that this is an issue with the arducamchip and OV5642.
Has anyone managed to get this fixed some how?

@wdouglas1999
Hi,
Would you please attach us some pictures for the  phenomenon?
Let us know if you need more help.
Best regards,
ArduCAM support team.

Hi thank you for the reply. Attached are my pictures. Images come straight from the FIFO. Resolution is 640x480. When there is movement it is blur. Any way you can help me to solve this issue?

@wdouglas1999
Hi,
Have you used our ArduCAM_Mini_5MP_Plus_short_movie_clip demo?
Do you changed the #define   rate     0x0a ?
Let us know if you need more help.
Regards,
ArduCAM support team.

Hi, no I have not used this demo. Will the demo work at 640 x 480?

@wdouglas1999
Hi,
Yes,but you should change the resolution in the AVI header.
Let us know if you need more help.
Regards,
ArduCAM support team.

Do I change the resolution with this line?
myCAM.OV5642_set_JPEG_size(OV5642_320x240) ?
To   myCAM.OV5642_set_JPEG_size(OV5642_640x480)
Is this all that needs to be changed?
Thx

@wdouglas1999
Hi,
You should not only change  myCAM.OV5642_set_JPEG_size(OV5642_320x240) ?
To myCAM.OV5642_set_JPEG_size(OV5642_640x480) but also change the AVI header in the
const int avi_header[AVIOFFSET] {....}.

Regards,
ArduCAM support team.

@wdouglas1999
Hi，
sorry for the first configure,
Please refer to this below;
const int avi_header[AVIOFFSET] PROGMEM ={
0x52, 0x49, 0x46, 0x46, 0xD8, 0x01, 0x0E, 0x00, 0x41, 0x56, 0x49, 0x20, 0x4C, 0x49, 0x53, 0x54,
0xD0, 0x00, 0x00, 0x00, 0x68, 0x64, 0x72, 0x6C, 0x61, 0x76, 0x69, 0x68, 0x38, 0x00, 0x00, 0x00,
0xA0, 0x86, 0x01, 0x00, 0x80, 0x66, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x10, 0x00, 0x00, 0x00,
0x64, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x80, 0x02, 0x00, 0x00, 0xe0, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x4C, 0x49, 0x53, 0x54, 0x84, 0x00, 0x00, 0x00,
0x73, 0x74, 0x72, 0x6C, 0x73, 0x74, 0x72, 0x68, 0x30, 0x00, 0x00, 0x00, 0x76, 0x69, 0x64, 0x73,
0x4D, 0x4A, 0x50, 0x47, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x01, 0x00, 0x00, 0x00, rate, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x0A, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x73, 0x74, 0x72, 0x66,
0x28, 0x00, 0x00, 0x00, 0x28, 0x00, 0x00, 0x00, 0x80, 0x02, 0x00, 0x00, 0xe0, 0x01, 0x00, 0x00,
0x01, 0x00, 0x18, 0x00, 0x4D, 0x4A, 0x50, 0x47, 0x00, 0x84, 0x03, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x4C, 0x49, 0x53, 0x54,
0x10, 0x00, 0x00, 0x00, 0x6F, 0x64, 0x6D, 0x6C, 0x64, 0x6D, 0x6C, 0x68, 0x04, 0x00, 0x00, 0x00,
0x64, 0x00, 0x00, 0x00, 0x4C, 0x49, 0x53, 0x54, 0x00, 0x01, 0x0E, 0x00, 0x6D, 0x6F, 0x76, 0x69,
};
Let us know if you need more help.
Regards,
ArduCAM support team.

Hi! Thanks again for all your help. The example works fine at 640 x 480, however there is still a ghosting effect and the images are not clear when there is movement. As the images are loaded directly from the sensor to the FIFO by the Aducam chip I am not sure how I will be able to resolve this issue. Is this the normal performance of the camera ? Can anything else be done?

@wdouglas1999
Hi,
It is related to the frame rate of cameras,What about 320x240 resolution?
The big resolution will make our 5MP 's frame low.
Let us know if you need more help.
Regards,
ArduCAM support team.

Hi, the 320 x 240 is better, but no 100%. This module is expensive for only this capability. Why is frame rate low if the module has the RAM to buffer all the images from the sensor? Then the aducamchip is too slow?
It is strange because the frame rate looks fast enough but the image has ghosting. I've worked with the VC 0706 chip and when the frame rate is low the movement is jerky but now ghosting effect. I was hoping your camera would perform better since it has all that RAM onboard. Do you think it could be the image sensors exposure settings or do you think it is purely a frame rate issue? Anyway, the product is still great but just not quite there yet. Thx

@wdouglas1999
Hi,
We just give you a demo and we don't configure the register in detail.
It is related to the register configure.
You should configure relevant register to increase the frame rate which
mat take you some time.
Let us know if you need more help.
Regards,
ArduCAM support team.

Hi, ok thank you. I will play around and see if I can make progress.

@wdouglas1999
Hi,
The performance of the processor is also a factor to influence the frame rate.
Regards,
ArduCAM support team.

Hi, thank you for the feedback. After reading the application note on the OV5642 it seems that they do mention that the rolling shutter of the sensor typically causes this type of issues (smearing effect) if the frame rate is not high enough. So it may be that nothing can be done to resolve this issue from a user point of view.

Hi, one more question. Can arducam chip work with "frame exposure mode" because frame exposure mode is different to rolling exposure mode.
Do you have a command for this maybe?
Thank you again

@wdouglas1999
Hi,
We are optimizing and developing more functions for the OV5642 camera.
Let us know if you need more help.
Regards,
ArduCAM support team.


Block edge 4.4.27

4.4.27 needs to be disabled:
5 clusters currently failing (15%),  9 gone (26%),and 18 successful (53%), out of 34 who attempted the update over 7d
Currently failing classification:
* Untriaged: 1
* Version retarget: 1
* volumesnapshots v1alpha1: 4

4.4.27 can be enabled again:
2 clusters currently failing ( 6%),  9 gone (27%),and 20 successful (61%), out of 33 who attempted the update over 7d
Currently failing classification:
* Version retarget: 1
* volumesnapshots v1alpha1: 2


Consider adding functional tests for key creation

We could use the same framework that package upload, login, and user creation functional tests use.

we have very low coverage for UI tests. When we decide to change this, we can handle this as well.


Integrate HERE SDK 3.13 and update min iOS version to 11.0

Codecov Report

Merging #142 into master will decrease coverage by 0.01%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master     #142      +/-   ##
==========================================
- Coverage   87.55%   87.53%   -0.02%
==========================================
  Files          89       59      -30
  Lines        5931     3683    -2248
==========================================
- Hits         5193     3224    -1969
+ Misses        738      459     -279

Flag
Coverage Δ

#ios
87.53% <ø> (-0.02%)
:arrow_down:

Impacted Files
Coverage Δ

MSDKUI_Demo/IconButton.swift

MSDKUI_Demo/OptionsViewController.swift

MSDKUI_Demo/ManeuversOverviewViewController.swift

MSDKUI_Demo/GuidanceDashboardViewController.swift

MSDKUI_Demo/ExtensionArray.swift

MSDKUI_Demo/RouteViewController.swift

MSDKUI_Demo/ExtensionUITableViewCell.swift

MSDKUI_Demo/ExtensionBundle.swift

MSDKUI_Demo/ExtensionNMAMapMarker.swift

...UI_Demo/GuidanceDashboardTableViewDataSource.swift

... and 19 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3762fe4...6c0e9b3. Read the comment docs.

Some tests still needs to be updated.


Add tapable as a direct dependency

Fixes #139

Pull Request Test Coverage Report for Build 642

1 of 1 (100.0%)  changed or added relevant line in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 92.308%

Totals

Change from base Build 630:
0.0%

Covered Lines:
85

Relevant Lines:
86

💛  - Coveralls


Be a bit more specific about what deck_type we accept to avoid long queries.

Fixes #5004.

Travis tests have failed
Hey @bakert,
Please read the following log in order to understand the failure reason.
It'll be awesome if you fix what's wrong and commit the changes.


I dont

I dont know what this is and I refuse to read the documentation.
Have you ever had a dream?

hello brother thank you for the criticism and suggestions, see how to use
it through the video https://www.youtube.com/watch?v=vmDmQvQ00D4.
Pada tanggal Sel, 25 Jan 2022 pukul 23.04 catboy6969 <
@.***> menulis:

I dont know what this is and I refuse to read the documentation.

—
Reply to this email directly, view it on GitHub
https://github.com/ExsoKamabay/kmy-beautify/issues/1, or unsubscribe
https://github.com/notifications/unsubscribe-auth/AKXWPC5VRFQLPKGJSVT2AYLUX23XNANCNFSM5MYQB5XA
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>


Have jsconfig.json AND have each file treated separately

According to https://code.visualstudio.com/docs/languages/jsconfig

Visual Studio Code's JavaScript support can run in two different modes:
File Scope - no jsconfig.json: In this mode, JavaScript files opened in Visual Studio Code are treated as independent units. As long as a file a.js doesn't reference a file b.ts explicitly (either using import or CommonJS modules), there is no common project context between the two files.
Explicit Project - with jsconfig.json: A JavaScript project is defined via a jsconfig.json file. The presence of such a file in a directory indicates that the directory is the root of a JavaScript project. The file itself can optionally list the files belonging to the project, the files to be excluded from the project, as well as compiler options (see below).

In my context, I have a number of Javascript files. Each can use the same APIs, will be run with the same Javascript version etc. However each file is independent, and I want Visual Studio code to treat them that way
Which it does, if I have no jsconfig.json. But I want one of those to control the options used (e.g. strict checking). But then the files are considered together, and I get errors about conflicting definitions etc between the files
So is there some way I can have both a jsconfig.json, and have Visual Studio Code treat the files as independent units?

This feature is powered by the TypeScript project. Please file a feature request upstream
Potentially also related to https://github.com/microsoft/TypeScript/issues/14279

Thanks. Wasn't obvious to me I should have requested it elsewere. The linked existing feature request covers what I want.


fix new writer in logx

NewWriter 方法用于自定义 logx 的 Writer，例如：
NewWriter method is used to customize logx's Writer, e.g.:
logx.SetWriter(logx.NewWriter(w))

但当前的 logx.NewWriter 使用 log.New 和 logx.newLogWriter 对传入的自定义 writer 进行了包装；
实际调用时，调用路径为，log.Print -> log.output -> writer.Write；
这会导致一系列问题，因为 log 库中使用了 sync.Pool 产生 *[]byte 用于拼接字符串；
如果自定义 writer 是同步写入，则可能不会出现问题；
如果自定义 writer 是异步写入，如使用 chan 来实现并发安全，这时实际传入 writer.Write 的 []byte 可能具有同一地址，造成写入重复内容；
并且 logx.newLogWriter 函数还忽略了自定义 writer 的 Close 方法；
通过修改 logx.NewWriter 方法，直接传入 io.WriteCloser，可以避免这些问题；
在实际测试中也是符合预期的；
When it is actually called, the path is log.Print -> log.output -> writer.Write;
This will lead to some problems, the std log library uses sync.Pool to generate *[]byte for string splicing;
If the custom writer writes synchronously, there may not be a problem;
But if the custom writer writes asynchronously, e.g., using chan for concurrency safety, the []byte that is actually passed to writer.Write may have the same address, resulting in duplicate content being written;
And logx.newLogWriter function also ignores the Close method of the custom writer;
These problems can be avoided by modifying the logx.NewWriter method and passing in io.WriteCloser directly;
It also meets the expectation in the actual test;

Thanks for your contribution!
But this is a breaking change. And please add unit tests.

Add NewAsyncWriter function to avoid affecting previous code.
Add unit test for NewAsyncWriter.

Merge this PR later. Let's figure out if it's possible to fix the problem with the same method NewWriter.
Would you please give a simple example to reproduce the problem?

NewWriter has two main issues.

NewWriter uses log.New to wrap the original writer, and use log.Print to write data, log package use a sync.Pool to get *[]byte when write data. If the original writer does not write the data immediately, but processes it asynchronously, this can result in duplicate ([]byte reuse) or no write ([]byte reset) of the data actually written.
Ignores the Close() method of raw writer, If raw writer has asynchronous processing, the cleanup logic in the Close() method will not be executed properly, which may result in data loss.

Simply replacing NewAyncWriter with NewWriter for the unit test in this commit will reproduce the problem and the test will fail.

2024年8月27日 22:19，Kevin Wan @.***> 写道：
Merge this PR later. Let's figure out if it's possible to fix the problem with the same method NewWriter.
Would you please give a simple example to reproduce the problem?
—
Reply to this email directly, view it on GitHub https://github.com/zeromicro/go-zero/pull/4311#issuecomment-2312707818, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAJTM3GB677YQ62CZRWJSHLZTSDHHAVCNFSM6AAAAABMH5YMHKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMJSG4YDOOBRHA.
You are receiving this because you authored the thread.


Share training code and data preparation code of DualFocus

Tanks for the great work! Please release the code of DualFocus

We are cleaning the code, The training code will be released soon

@panzhang0212  when will the training code be released? thanks


Tooltip reappear when switching tabs

Hello, have issue with tooltip.
Using Nuxt 2.15.8 and floating-vue 1.0.0-beta.19.
We have modal component with dark translucent background that is hidden with v-if, also we have button with v-tooltip text, when you click on button - modal appears and tooltip disappear. BUT, if you switch to another tab and switch back - tooltip appears again, but now over modal. If needed - I can provide video.

I'm having a similar issue.
Did you solve it?


Failed to access http://createjs.com/ in China

When accessing http://createjs.com/ after a long time loading. Could you please take a look? Will be pleasure if you need my help to troubleshoot.

@jonlimx we currently use a few scripts from CDN's that may be blocked in China, if you can let us know which ones you can see or not, we might be-able find suitable alternatives.
Currently we use:
https://ajax.googleapis.com/ajax/libs/jquery/2.0.3/jquery.min.js
http://use.typekit.net/cek3iwo.js
https://code.createjs.com/createjs-2015.05.21.min.js

@wdamien Thanks for you response. I can't assess the first one(the googleapis), but able to access the rests. I missed something in the first post - actually,  when accessing http://createjs.com/, I got a blank page after a long time loading.
I would like to send you a screenshot, but seems like we can't attach an image here.

We can not visit google's any content, so we have to use proxy to visit the website which use  google api and fonts. I am from china too -_-  @jonlimx  You can use GoAgent when you want access to these sites.

@liuyi Thanks for your reminder. GoAgent doesn't always work, cuz it could be blocked by gov some day.

Good to have verification that its the Google apis causing the issue. Can you try these other cdn links?  Its an easy change to use another one that will work everywhere.
https://cdnjs.cloudflare.com/ajax/libs/jquery/2.1.4/jquery.min.js
http://cdn.jsdelivr.net/jquery/2.1.4/jquery.min.js

@jonlimx  http://libs.useso.com/ This link would be help you.

@wdamien Thanks for your follow up.These two CDN work in China.

@jonlimx createjs.com is updated to use cloudflare instead of the google apis, hopefully the site will work in China now.

@wdamien I enjoyed a one week trip in the past week, sorry for the late response. It works well now! Thanks very much!


Problems trying to use github repo in Windows 10.

I'm using Windows 10 Home, and I am trying to use NodeOS for Windows 10 from the github zip download. I unzipped it and now I'm trying to use it, first I did "npm install", their were a lot of red errors, but I'm pretty sure it might be okay (idk, it made a log file), but when I do "npm run build", it throws: " 'scripts' is not recognized as an internal or external command".

I put "node" before it says "scripts/build" in the build key of scripts in the package.json. But I get a new error:
#!/usr/bin/env bash
^
SyntaxError: Invalid or unexpected token

You have to run the scripts under Linux, bash scripts don't work in PowerShell and Window's cmdline.

You could just use a VM.

Btw I ended up just using Ubunto WSL.


feat: update Rebecca Miller's talk name for January 16 event

PR Checklist

[ ] Addresses an existing open issue: fixes #000
[ ] That issue was marked as status: accepting prs
[x] Steps in CONTRIBUTING.md were taken

Overview
Updates the name of Rebecca Miller's (my) talk at the January 16, 2025 TS meetup, to its final name.
🤘

Btw @rebecca-millerr do you prefer your name to be displayed on our website as Rebecca or Becca? Josh mentioned both so I want to make sure I got it right :)

Btw @rebecca-millerr do you prefer your name to be displayed on our website as Rebecca or Becca? Josh mentioned both so I want to make sure I got it right :)

Either works! I left it as Rebecca so it's consistent with my LinkedIn, but will likely introduce myself as Becca on the day of. I'm happy with either though in general 😄

Noted! Welp, if you change your mind, you know how to update the website now :)

@all-contributors please add @rebecca-millerr for code.

🤖 Beep boop! This comment was added automatically by all-contributors-auto-action.
Not all contributions can be detected from Git & GitHub alone. Please comment any missing contribution types this bot missed.
...and of course, thank you for contributing! 💙


Compiler error execvp: No such file or directory (Errno)

sh-4.2# crystal build --release server.cr
execvp: No such file or directory (Errno)
  from ???
  from ???
  from ???
...
Error: you've found a bug in the Crystal compiler. Please open an issue, including source code that will allow us to reproduce the bug: https://github.com/crystal-lang/crystal/issues
sh-4.2# crystal -v
Crystal 0.26.1 [391785249] (2018-08-27)

LLVM: 4.0.0
Default target: x86_64-unknown-linux-gnu
sh-4.2# uname -a
Linux 64809d7bf557 4.9.125-linuxkit #1 SMP Fri Sep 7 08:20:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

Dockerfile to reproduce
FROM amazonlinux:latest

RUN curl https://dist.crystal-lang.org/rpm/setup.sh | bash
RUN yum install -y crystal-0.26.1 git gcc gcc-c++

ENV PORTS=""
WORKDIR /app

COPY shard.* ./
COPY server.cr ./
RUN shards install
RUN crystal build --release server.cr

EXPOSE ${PORTS}

CMD server

Simple websocket kemal server
require "kemal"
require "random/secure"

Kemal.config.port = 8080

ws "/" do |socket|
  socket_id = Random::Secure.hex(32)
  puts "[#{socket_id}}] connected!"
  socket.send "connected!"
  socket.on_message do |message|
    puts "[#{socket_id}}] message: #{message}"
    socket.send "echo: #{message}"
  end

  # Remove clients from the list when it’s closed
  socket.on_close do
    puts "[#{socket_id}}] closed"
  end
end

Kemal.run

This seems to be an issue with PATH, either it's malconfigured or a required executable is missing. Probably similar to #7082.
If you prefix the crystal build command with strace -f it should print the arguments to execvp.
But I think we should really improve the error message for execvp to include the executable name. Users shouldn't be required to run strace just to fine out which file can't be executed.

Thanks I will try the strace -f

[pid    17] lseek(12, 51912232, SEEK_SET) = 51912232
[pid    17] read(12, "\1\21\0\20\6U\6\3\10\33\10%\10\23\5\0\0\0'\1\0\0\2\0\211\0\0\0\1\1\373\16"..., 8192) = 8192
[pid    17] lseek(12, 51603959, SEEK_SET) = 51603959
[pid    17] lseek(12, 0, SEEK_CUR)      = 51603959
[pid    17] read(12, "\1\304\205C\0\20E'\0crt/x86_64/crtn.s\0/home"..., 8192) = 8192
[pid    17] lseek(12, 0, SEEK_CUR)      = 51612151
[pid    17] lseek(12, 0, SEEK_CUR)      = 51612151
[pid    17] lseek(12, 66038848, SEEK_SET) = 66038848
[pid    17] close(12)                   = 0
[pid    17] write(9, "  from ???\n", 11  from ???
) = 11
[pid    17] write(9, "  from ???\n", 11  from ???
) = 11
[pid    17] write(9, "  from ???\n", 11  from ???
) = 11
[pid    17] write(9, "  from ???\n", 11  from ???
) = 11

I think my issue is like this one https://github.com/crystal-lang/crystal/issues/6464

Confirmed. It can be reproduced by removing which:
$ crystal eval ''
$ rm /usr/bin/which /bin/which
$ crystal eval ''
execvp (which "pkg-config"): No such file or directory: No such file or directory (Errno)

(don't try this at home)
This is the faulty line:
https://github.com/crystal-lang/crystal/blob/3c3647d9edfb8b1d004bddb5fbf06568bfb16d2e/src/compiler/crystal/codegen/link.cr#L119
I suppose we can simply replace the call to which with Process.find_executable("pkg-config").

Or, just run pkg-config and check the errno if it fails?


Implement internal_bit

Closes #30.

Seeing the usage of ceil_pow2 in the original ACL, I think the signature should be fn(u32) -> u32.

LGTM.  Since three noodles depends on this modules, I think this PR should be merged ASAP.

$ cargo +nightly bench --bench bench
   Compiling a v0.1.0 (/home/ryo/src/local/hage/a)
    Finished bench [optimized] target(s) in 0.50s
     Running target/release/deps/bench-13a9f15c9072cc25

running 2 tests
test naive              ... bench:   2,115,928 ns/iter (+/- 117,993)
test with_leading_zeros ... bench:     104,699 ns/iter (+/- 8,538)

test result: ok. 0 passed; 0 failed; 0 ignored; 2 measured; 0 filtered out

@qryxip Just for curiosity, could you tell me which two function you compared?  The original one (while loop + comparison) and the Rust implementation (count leading zeros)?

This. (I'm not used to bench)
#![feature(test)]

extern crate test;

use test::Bencher;

#[bench]
fn with_leading_zeros(bencher: &mut Bencher) {
    let values = values();
    bencher.iter(move || {
        values
            .iter()
            .map(|v| 32 - v.saturating_sub(1).leading_zeros())
            .fold(0, u32::wrapping_add)
    });
}

#[bench]
fn naive(bencher: &mut Bencher) {
    let values = values();
    bencher.iter(move || {
        values
            .iter()
            .map(|&v| {
                let mut x = 0;
                while (1 << x) < v {
                    x += 1;
                }
                x
            })
            .fold(0, u32::wrapping_add)
    });
}

fn values() -> Vec<u32> {
    std::fs::read_to_string("./values.txt")
        .unwrap()
        .split_ascii_whitespace()
        .map(|s| s.parse().unwrap())
        .collect()
}

import random
s = ''
for _ in range(100000):
    s += str(random.randint(0, 1 << 30))
    s += '\n'
with open('./values.txt', 'w') as file:
    file.write(s)

$ cargo +nightly bench --bench bench
   Compiling a v0.1.0 (/home/ryo/src/local/hage/a)
    Finished bench [optimized] target(s) in 0.45s
     Running target/release/deps/bench-13a9f15c9072cc25

running 2 tests
test naive              ... bench:   2,144,545 ns/iter (+/- 99,698)
test with_leading_zeros ... bench:     144,275 ns/iter (+/- 11,680)

test result: ok. 0 passed; 0 failed; 0 ignored; 2 measured; 0 filtered out

#[bench]
fn with_leading_zeros(bencher: &mut Bencher) {
    #[inline(never)]
    fn run(v: u32) -> u32 {
        32 - v.saturating_sub(1).leading_zeros()
    }

    let values = values();
    bencher.iter(move || values.iter().copied().map(run).fold(0, u32::wrapping_add));
}

#[bench]
fn naive(bencher: &mut Bencher) {
    #[inline(never)]
    fn run(v: u32) -> u32 {
        let mut x = 0;
        while (1 << x) < v {
            x += 1;
        }
        x
    }

    let values = values();
    bencher.iter(move || values.iter().copied().map(run).fold(0, u32::wrapping_add));
}

Can somebody merge this so that others can implement (lazy)segtree ?


no python modules build on armbian \  Rock PiC

Hi.
After cmake and make goes without an issue make install produces the following  output (listed below, this is a second run).
As it can be seen there are no python modules whats or ever installed.
The system is Debian 10.7 aarch64 running on Rock Pi 4C board.
All prerequisites are installed incl Python 2,7 and Python 3.7 and respective dev packages.
Please advice how to debug python module compilation.
-- Install configuration: ""
-- Up-to-date: /usr/local/include/mraa.h
-- Up-to-date: /usr/local/include/mraa.hpp
-- Up-to-date: /usr/local/include/mraa
-- Up-to-date: /usr/local/include/mraa/types.h
-- Up-to-date: /usr/local/include/mraa/common.h
-- Up-to-date: /usr/local/include/mraa/types.hpp
-- Up-to-date: /usr/local/include/mraa/pwm.hpp
-- Up-to-date: /usr/local/include/mraa/gpio.hpp
-- Up-to-date: /usr/local/include/mraa/led.h
-- Up-to-date: /usr/local/include/mraa/firmata.h
-- Up-to-date: /usr/local/include/mraa/iio_kernel_headers.h
-- Up-to-date: /usr/local/include/mraa/spi.hpp
-- Up-to-date: /usr/local/include/mraa/spi.h
-- Up-to-date: /usr/local/include/mraa/gpio.h
-- Up-to-date: /usr/local/include/mraa/uart_ow.hpp
-- Up-to-date: /usr/local/include/mraa/i2c.hpp
-- Up-to-date: /usr/local/include/mraa/iio.hpp
-- Up-to-date: /usr/local/include/mraa/uart.hpp
-- Up-to-date: /usr/local/include/mraa/i2c.h
-- Up-to-date: /usr/local/include/mraa/pwm.h
-- Up-to-date: /usr/local/include/mraa/uart_ow.h
-- Up-to-date: /usr/local/include/mraa/initio.hpp
-- Up-to-date: /usr/local/include/mraa/aio.h
-- Up-to-date: /usr/local/include/mraa/led.hpp
-- Up-to-date: /usr/local/include/mraa/initio.h
-- Up-to-date: /usr/local/include/mraa/iio.h
-- Up-to-date: /usr/local/include/mraa/uart.h
-- Up-to-date: /usr/local/include/mraa/common.hpp
-- Up-to-date: /usr/local/include/mraa/aio.hpp
-- Up-to-date: /usr/local/lib/pkgconfig/mraa.pc
-- Up-to-date: /usr/local/lib/libmraa.so.2.2.0
-- Up-to-date: /usr/local/lib/libmraa.so.2
-- Up-to-date: /usr/local/lib/libmraa.so
-- Up-to-date: /usr/local/share/mraa/examples
-- Up-to-date: /usr/local/share/mraa/examples/c
-- Up-to-date: /usr/local/share/mraa/examples/c/uart_advanced.c
-- Up-to-date: /usr/local/share/mraa/examples/c/aio.c
-- Up-to-date: /usr/local/share/mraa/examples/c/i2c_mpu6050.c
-- Up-to-date: /usr/local/share/mraa/examples/c/initio.c
-- Up-to-date: /usr/local/share/mraa/examples/c/uart.c
-- Up-to-date: /usr/local/share/mraa/examples/c/gpio_multiple_rw.c
-- Up-to-date: /usr/local/share/mraa/examples/c/spi.c
-- Up-to-date: /usr/local/share/mraa/examples/c/led.c
-- Up-to-date: /usr/local/share/mraa/examples/c/hellomraa.c
-- Up-to-date: /usr/local/share/mraa/examples/c/gpio.c
-- Up-to-date: /usr/local/share/mraa/examples/c/i2c_hmc5883l.c
-- Up-to-date: /usr/local/share/mraa/examples/c/CMakeLists.txt
-- Up-to-date: /usr/local/share/mraa/examples/c/pwm.c
-- Up-to-date: /usr/local/share/mraa/examples/c/gpio_advanced.c
-- Up-to-date: /usr/local/share/mraa/examples/c/iio.c
-- Up-to-date: /usr/local/share/mraa/examples/c/uart_ow.c
-- Up-to-date: /usr/local/share/mraa/examples/javascript
-- Up-to-date: /usr/local/share/mraa/examples/javascript/GPIO_DigitalRead.js
-- Up-to-date: /usr/local/share/mraa/examples/javascript/bmp85.js
-- Up-to-date: /usr/local/share/mraa/examples/javascript/AioA0.js
-- Up-to-date: /usr/local/share/mraa/examples/javascript/Blink-IO.js
-- Up-to-date: /usr/local/share/mraa/examples/javascript/isr.js
-- Up-to-date: /usr/local/share/mraa/examples/javascript/uart.js
-- Up-to-date: /usr/local/share/mraa/examples/javascript/initio.js
-- Up-to-date: /usr/local/share/mraa/examples/javascript/GPIO_DigitalWrite.js
-- Up-to-date: /usr/local/share/mraa/examples/javascript/spi.js
-- Up-to-date: /usr/local/share/mraa/examples/javascript/rgblcd.js
-- Up-to-date: /usr/local/share/mraa/examples/javascript/firmata.js
-- Up-to-date: /usr/local/share/mraa/examples/javascript/gpio-tool.js
-- Up-to-date: /usr/local/share/mraa/examples/CMakeLists.txt
-- Up-to-date: /usr/local/share/mraa/examples/c++
-- Up-to-date: /usr/local/share/mraa/examples/c++/gpio_advanced.cpp
-- Up-to-date: /usr/local/share/mraa/examples/c++/aio.cpp
-- Up-to-date: /usr/local/share/mraa/examples/c++/gpio.cpp
-- Up-to-date: /usr/local/share/mraa/examples/c++/spi.cpp
-- Up-to-date: /usr/local/share/mraa/examples/c++/uart.cpp
-- Up-to-date: /usr/local/share/mraa/examples/c++/i2c.cpp
-- Up-to-date: /usr/local/share/mraa/examples/c++/led.cpp
-- Up-to-date: /usr/local/share/mraa/examples/c++/pwm.cpp
-- Up-to-date: /usr/local/share/mraa/examples/c++/CMakeLists.txt
-- Up-to-date: /usr/local/share/mraa/examples/c++/uart_ow.cpp
-- Up-to-date: /usr/local/share/mraa/examples/c++/iio.cpp
-- Up-to-date: /usr/local/share/mraa/examples/python
-- Up-to-date: /usr/local/share/mraa/examples/python/firmata.py
-- Up-to-date: /usr/local/share/mraa/examples/python/uart.py
-- Up-to-date: /usr/local/share/mraa/examples/python/gpio.py
-- Up-to-date: /usr/local/share/mraa/examples/python/spi_loopback.py
-- Up-to-date: /usr/local/share/mraa/examples/python/gpio_advanced.py
-- Up-to-date: /usr/local/share/mraa/examples/python/spi.py
-- Up-to-date: /usr/local/share/mraa/examples/python/pwm.py
-- Up-to-date: /usr/local/share/mraa/examples/python/i2c_rgb.py
-- Up-to-date: /usr/local/share/mraa/examples/python/uart_sender.py
-- Up-to-date: /usr/local/share/mraa/examples/python/led.py
-- Up-to-date: /usr/local/share/mraa/examples/python/i2c_bmp85.py
-- Up-to-date: /usr/local/share/mraa/examples/python/aio.py
-- Up-to-date: /usr/local/share/mraa/examples/python/uart_receiver.py
-- Up-to-date: /usr/local/share/mraa/examples/java
-- Up-to-date: /usr/local/share/mraa/examples/java/Isr.java
-- Up-to-date: /usr/local/share/mraa/examples/java/SpiMAX7219.java
-- Up-to-date: /usr/local/share/mraa/examples/java/BlinkIO.java
-- Up-to-date: /usr/local/share/mraa/examples/java/AioA0.java
-- Up-to-date: /usr/local/share/mraa/examples/java/Example.java
-- Up-to-date: /usr/local/share/mraa/examples/java/SpiMCP4261.java
-- Up-to-date: /usr/local/share/mraa/examples/java/FTDITest.java
-- Up-to-date: /usr/local/share/mraa/examples/java/UartExample.java
-- Up-to-date: /usr/local/share/mraa/examples/java/I2cCompass.java
-- Up-to-date: /usr/local/share/mraa/examples/java/CyclePwm3.java
-- Up-to-date: /usr/local/share/mraa/examples/java/GpioRead6.java
-- Up-to-date: /usr/local/share/mraa/examples/java/GpioMmapped.java
-- Up-to-date: /usr/local/share/mraa/examples/java/BlinkOnboard.java
-- Up-to-date: /usr/local/share/mraa/examples/java/Bmp85.java
-- Up-to-date: /usr/local/share/mraa/examples/java/HelloEdison.java
-- Up-to-date: /usr/local/share/mraa/examples/samples.mapping.txt
-- Up-to-date: /usr/local/share/mraa/examples/platform
-- Up-to-date: /usr/local/share/mraa/examples/platform/up2-leds.cpp
-- Up-to-date: /usr/local/share/mraa/examples/platform/turbotjson.json
-- Up-to-date: /usr/local/share/mraa/examples/platform/gpio_edison.c
-- Up-to-date: /usr/local/share/mraa/examples/platform/initio.c
-- Up-to-date: /usr/local/share/mraa/examples/platform/i2c_firmata.c
-- Up-to-date: /usr/local/share/mraa/examples/platform/initio.py
-- Up-to-date: /usr/local/share/mraa/examples/platform/CMakeLists.txt
-- Up-to-date: /usr/local/share/mraa/examples/platform/firmata_curie_imu.c
-- Up-to-date: /usr/local/share/mraa/examples/platform/Up2Leds.java
-- Up-to-date: /usr/local/share/mraa/examples/platform/intel-aero.json
-- Up-to-date: /usr/local/share/mraa/examples/platform/initio.cpp
-- Up-to-date: /usr/local/bin/mraa-gpio
-- Up-to-date: /usr/local/bin/mraa-i2c
-- Up-to-date: /usr/local/bin/mraa-uart

Ok
Seems the same as #1019
committed cmake result
-- INFO - libmraa Version v2.2.0
-- INFO - cmake Version 3.13.4
-- INFO - Target arch is aarch64
-- Found PythonInterp: /usr/bin/python2.7 (found suitable version "2.7.16", minimum required is "2.7")
-- Found PythonLibs: /usr/lib/aarch64-linux-gnu/libpython2.7.so (found suitable exact version "2.7.16")
-- Found PythonInterp: /usr/bin/python3 (found suitable version "3.7.3", minimum required is "3")
-- Found PythonLibs: /usr/lib/aarch64-linux-gnu/libpython3.7m.so (found suitable exact version "3.7.3")
-- INFO - Adding onewire backend support
-- INFO - Adding support for platform ALL
-- INFO - Adding support for all platforms
-- Found PkgConfig: /usr/bin/pkg-config (found version "0.29")
INFO - Adding ARM platforms
-- Found SWIG: /usr/bin/swig3.0 (found version "3.0.12")
-- PYTHON3 attempting to build!
CMake Warning (dev) at /usr/share/cmake-3.13/Modules/UseSWIG.cmake:564 (message):
Policy CMP0078 is not set.  Run "cmake --help-policy CMP0078" for policy
details.  Use the cmake_policy command to set the policy and suppress this
warning.
Call Stack (most recent call first):
src/python/python3/CMakeLists.txt:9 (swig_add_library)
This warning is for project developers.  Use -Wno-dev to suppress it.
-- Could NOT find GTest (missing: GTEST_LIBRARY GTEST_INCLUDE_DIR GTEST_MAIN_LIBRARY)
-- Install Google Test to enable additional unit testing
-- Configuring done
-- Generating done
-- Build files have been written to: /usr/local/src/mraa/build

Ok
Seems the same as #1019
committed cmake result
-- INFO - libmraa Version v2.2.0
-- INFO - cmake Version 3.13.4
-- INFO - Target arch is aarch64
-- Found PythonInterp: /usr/bin/python2.7 (found suitable version "2.7.16", minimum required is "2.7")
-- Found PythonLibs: /usr/lib/aarch64-linux-gnu/libpython2.7.so (found suitable exact version "2.7.16")
-- Found PythonInterp: /usr/bin/python3 (found suitable version "3.7.3", minimum required is "3")
-- Found PythonLibs: /usr/lib/aarch64-linux-gnu/libpython3.7m.so (found suitable exact version "3.7.3")
-- INFO - Adding onewire backend support
-- INFO - Adding support for platform ALL
-- INFO - Adding support for all platforms
-- Found PkgConfig: /usr/bin/pkg-config (found version "0.29")
INFO - Adding ARM platforms
-- Found SWIG: /usr/bin/swig3.0 (found version "3.0.12")
-- PYTHON3 attempting to build!
CMake Warning (dev) at /usr/share/cmake-3.13/Modules/UseSWIG.cmake:564 (message):
Policy CMP0078 is not set.  Run "cmake --help-policy CMP0078" for policy
details.  Use the cmake_policy command to set the policy and suppress this
warning.
Call Stack (most recent call first):
src/python/python3/CMakeLists.txt:9 (swig_add_library)
This warning is for project developers.  Use -Wno-dev to suppress it.
-- Could NOT find GTest (missing: GTEST_LIBRARY GTEST_INCLUDE_DIR GTEST_MAIN_LIBRARY)
-- Install Google Test to enable additional unit testing
-- Configuring done
-- Generating done
-- Build files have been written to: /usr/local/src/mraa/build

OK, I pulled the python-mraa_2.0.0-2_arm64.deb
and manually grabbed _mraa.aarch64-linux-gnu.so and  mraa.py files and placed them to /usr/lib/python2.7/dist-packages/ on my Rock Pi4.
Now I can import mraa in my python with no issue what so ever.

OK, I pulled the python-mraa_2.0.0-2_arm64.deb
and manually grabbed _mraa.aarch64-linux-gnu.so and  mraa.py files and placed them to /usr/lib/python2.7/dist-packages/ on my Rock Pi4.
Now I can import mraa in my python with no issue what so ever.


Add a sample code of custom random password generator for the Ask password documentation.

When you try to create a user with Ask password option, Identity Server will create a random password for that user using the extension class org.wso2.carbon.user.mgt.common.DefaultPasswordGenerator, until the user configures a new password. So when you enable a Password Policy [1] it might give an error if the generated random password violates the policy that you have defined in your configurations.
To overcome that issue you can write a custom random password generator which does not violate your password policy. I have attached sample code (wso2-sample-custom-password-generator.zip) of a custom password generator for your reference. And do the following steps to use your custom password generator for ask password option.

Build the custom password generator source and get the com.wso2.password.policy-1.0.0.jar file.
Copy the jar file into <IS_HOME>/repository/components/lib directory.
Start the server.
Then you need to configure your custom password generator class form the management console in order to use it instead of DefaultPasswordGenerator. Please do the following steps.

Click Resident under Identity Providers found in the Main tab of the IS management console.
Expand the Account Management Policies tab.
Expand the User Onboarding tab and define your custom password generator class in Temporary password generation extension class section. (ex: com.wso2.password.policy.CustomPasswordGenerator)

Then you should be able to create a new user with ask password option without any issue.
Add the above sample for Ask password documentation [2].
[1] https://docs.wso2.com/display/IS541/Password+Patterns
[2] https://docs.wso2.com/display/IS541/Creating+Users+using+the+Ask+Password+Option

it looks like this was already added here

it looks like this was already added here

possibly superseded by #4600

This issue is being closed due to extended inactivity. Please feel free to reopen it if further attention is needed. Thank you for helping us keep the issue list relevant and focused!


Inconsistent prediction results after dumping and loading lightgbm.LGBMClassifier via pickle

Basically, title.

train model LGBMClassifier
get predictions for test set
dump model via pickle (I also tried joblib - had same issue)
load model
get different predictions for test set

Both models have same parameters and metrics in properties.
I'm using python 3.7.3 and lightgbm 2.2.3

@slam3085 Please post the whole snippet. Also, attach the data you use, if it's not sensitive. Or can you reproduce this issue with random data?
We do have a test where predictions are the same:
https://github.com/microsoft/LightGBM/blob/be206a96e7be553b0918805f6641e91285ce89d1/tests/python_package_test/test_sklearn.py#L155-L178

Hi!
I can't share the data and the snippet looks like this:
try:
    with open(path, "rb") as f:
        mdl = pickle.load(f)
    print(f'lightgbm_fitted_model_{today} loaded')
except:
    print(f'Unable to open lightgbm_fitted_model_{today}; fitting...')
    X_actual_train, X_eval, y_actual_train, y_eval = train_test_split(X_train, y_train, test_size=0.1,
                                                                      random_state=42, stratify=y_train)

    hyperparams = {'n_estimators': 200, 'class_weight': 'balanced', 'random_state': 42}
    mdl = lightgbm.LGBMClassifier(**hyperparams)
    mdl.fit(X_actual_train, y_actual_train, eval_set=(X_eval, y_eval), eval_metric='roc_auc')
    with open(path, "wb") as f:
        pickle.dump(mdl, f)
pred_proba = mdl.predict_proba(X_test)[:, 1]
n_pred_proba_95 = sum(pred_proba >= 0.95)
print(f'stats: {n_pred_proba_95} out of {len(pred_proba)} are 95% objects of class 1')
return pred_proba

Basically either I fit new model and dump it via pickle, or I use already trained one.
I tried to reproduce it in local environment only and couldn't, and now I understand why - I have python 3.7.3 as local environment and python 3.7.0 as production environment. So, actual steps to reproduce are:

train model in python 3.7.3
load it in python 3.7.0
results of predictions vary

I guess it's not really a bug then or very minor one and issue can be closed.

@slam3085

I tried to reproduce it in local environment only and couldn't, and now I understand why - I have python 3.7.3 as local environment and python 3.7.0 as production environment.

Makes sense! Thanks a lot for your investigation.

We are having the same problem. It happens when we try to deploy our models to production. Observed this issue two times:

Trying to deploy a model trained in a system (a node part of a Hadoop cluster with CentOS, 512GB RAM and a 32 cores Xeon) and loading the pickle/text in the rest of the nodes. Predictions are correct only in the same system where the model was trained and inconsistent in the rest (imagine, a continuous variable with values like [123.5, 143.2, 456.4] got predictions like [-4.0, 5.0, -9.0]).

Today while deploying using MLFlow and setting up an endpoint, exactly same problem.

Using both scikit-learn API and classic Python API. Changing the model in our pipeline for a typical scikit-learn RandomForestRegressor and everything is OK.
Any idea about what is happening? It looks like we are missing something important...

What is the difference for the prediction results?

For example, predictions on the same machine with the same environment the model was trained:

[123.5, 143.2, 456.4] (reasonable predictions, close to the original values we try to predict)

Predictions loading the model (joblib or txt) in another machine/environment, always installing LightGBM with conda or pip:

[-4.0, 5.0, -9.0] (weird predictions, negative and integer-like with the .0)

with this large gap, I think it is not the same problem as this issue.
did you try to save and then load from the same machine/env?

Yes, in the same machine and using the same environment there is no problem, in the same machine and using a different environment (for example, deploying the model using MLFlow) the results are different, same as saving the model and loading it in other machines...
The thing is, the models are supposed to be portable, at least saving it as a .txt file.

@dcanones could you try the cli version in the same machine? I think it is possible the bug of env.


[REG-503] Remove matchInfoEmitter

Make RGBot an eventemitter itself
Is backwards compatible with old bots by passing non RGBot/RGMatchInfo event listeners through to mineflayer automatically
This also happens to cleans up the issue where passing in a standalone EventEmitter for matchInfo events felt weird

@vontell See updated code for .on/.off/.once that now handles our events vs mineflayer events automatically without the user needing to write special code.


Asset select menu on Transfer page shows USD when balance is 0.

I Transferred all of my USD to another account. Then, I wanted to do the same thing with the EUR. So, I went back to the Transfer screen, clicked on the asset select menu, and the USD was still an option in that menu even though I emptied it already.

Another high priority bug IMO. Inconsistencies like these hurt confidence. Adding to current Milestone.


chore(registry): bump confd to v0.8

Just to note, this still has some ERROR output that ideally we will silence:
--- Run deisci/registry:jenkins-test-ec2-238 at <IP_ADDRESS>:57185
Error response from daemon: No such container: deis-registry-jenkins-test-ec2-238
2015-03-24T22:27:57Z a70b2fad3602 confd[23]: WARNING Skipping confd config file.
2015-03-24T22:27:57Z a70b2fad3602 confd[23]: ERROR exit status 1
registry: waiting for confd to write initial templates...
2015-03-24T22:28:02Z a70b2fad3602 confd[33]: WARNING Skipping confd config file.
deis-registry running...
2015-03-24T22:28:02Z a70b2fad3602 confd[41]: WARNING Skipping confd config file.
[2015-03-24 22:28:02 +0000] [46] [INFO] Starting gunicorn 19.1.1

Code LGTM.

Code LGTM.

That looks better:
--- Run deisci/registry:jenkins-test-ec2-241 at <IP_ADDRESS>:42695
Error response from daemon: No such container: deis-registry-jenkins-test-ec2-241
registry: waiting for confd to write initial templates...
deis-registry running...
2015-03-25T16:23:35Z 9a82e8802bd5 confd[40]: WARNING Skipping confd config file.
[2015-03-25 16:23:36 +0000] [44] [INFO] Starting gunicorn 19.1.1

Code LGTM++.


Would you like to improve your file structure?

If you want to display question & your source code, you can use comment

Thanks for your issues.
I may fill them in the winter vacation. For now, I am still struggling with how to improve my Linear Algebra.


Add attr.fields_dict()

This patch adds attr.fields_dict(), which returns an OrderedDict of {attribute_name: Attribute} for all attrs attributes of a class.
Closes #290.
I'm bad at describing dictionaries in prose, so feel free to improve. :)
Pull Request Check List
This is just a reminder about the most common mistakes.  Please make sure that you tick all appropriate boxes.  But please read our contribution guide at least once, it will save you unnecessary review cycles!

[x] Added tests for changed code.
[x] New features have been added to our Hypothesis testing strategy.
[x] Updated documentation for changed code.
[x] Documentation in .rst files is written using semantic newlines.
[x] Changed/added classes/methods/functions have appropriate versionadded, versionchanged, or deprecated directives.
[x] Changes (and possible deprecations) have news fragments in changelog.d.

If you have any questions to any of the points above, just submit and ask!  This checklist is here to help you, not to deter you from contributing!

@hynek updated 🎉

Have you pushed? 😇

I think so? The review comments show as outdated except for https://github.com/python-attrs/attrs/pull/349#discussion_r170957703 which is on the line above the newline

Thanks!


Update Suggestion: "Getting Started" on Get Bitcoin page

#191 removes "Bittr in Europe" from "Getting Started" at the top of the page.
What European exchange should take its place?

Good question Pardus, do you have any suggestion? I am interested in the future to encourage users to use P2P exchanges rather than these kyc services, do you have any thoughts on that?

I don't have any suggestions on European exchanges since I'm not in Europe.
I think non-KYC exchanges are great but are there any that are easy for precoiners to use?

May I humbly suggestion Coinfloor?
We support UK and European customers on our exchange since launch.  Our DCA (Autobuy) product will soon support EURO DCA as well.

We actually removed this banner recently so this issue can be closed. Thanks anyway


update k8s deployment readme and add kind cluster script

Add Installation script for 3 nodes kind cluster.
Update aperture installation Readme for  k8s.

This change is

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

As @sbienkow-ninja  mention that we don't need the docker-network code in this script. So I have removed that.
@seunghyupoh3517 added the kind-config.yaml file which will create the simple 3 node kind cluster , which he added for test purpose. So after having discussion with @hdkshingala , we decided to continue with this script as we will require docker-registry which is been created in this script .
cc @tanveergill

Hey, @sbienkow-ninja can  you review the PR. I have made the expected changes .

Just saw that @harjotgill have recently made some changes in the README.md file.
kind-config.yaml  is used to launch the 3 node kind cluster.
Can you confirm are we going with script to just the  kind-config.yaml  file.
cc @harjotgill , @sbienkow-ninja


cannot add WIFI connection on RG351V

Before submitting your issue or feature request.  DO NOT REQUEST ASSISTANce FOR PRELOADED IMAGES.  I do not support or condone the use of preloaded ArkOS images.  Preloaded meaning images that come with copyrighted roms preincluded.  Look for support from the creator or supplier of those preloaded images, not from here!  If a ticket is submitted for assistance for such images or if there's an indiciation that a preloaded image is what the support request is for, the issue ticket will be closed with no further response.

This is not the Stock OS that comes with your device, AmberElec, JelOS, or UnOfficialOS.  ArkOS is a different distribution from those systems and will have differing features.  The only somewhat similarity is that they both use EmulationStation as a frontend.  Even with that, the versions of EmulationStation that's used in between ArkOS and these other distributions is different with different capabilities.  If there are features you prefer with the other available distributions, just use those distributions instead.  The choice is yours.

Have you reviewed the Frequently Asked Questions section available for your supported device in the wiki?

If your issue is related to being able to run certain emulators or ports, have you reviewed the Emulators and Ports section available at https://github.com/christianhaitian/arkos/wiki/ArkOS-Emulators-and-Ports-information for needed bios files and supported rom extensions?

Have you reviewed the known issues section available at https://github.com/christianhaitian/arkos/wiki/Known-issues ?

Describe the Issue (If applicable)
I am trying to update my device from 16/10/2023 version to latest.
I need to update the WIFI settings first.
When I enter Wifi manager, none of the inputs work, I cannot select anything or move the cursor. I can only hold the power button to force shutdown.
This issue is also present when I press update, I cannot interact, only force shutdown.
How can the issue be reproduced? (If applicable)
What device are you using? (Only the Gameforce Chi, RK2020, OGA 1.0, OGA 1.1, RG351MP, RG351V, RGB10, R35s, RG353V/VS, RG353M, RG503, RGB30, and RK2023 units are officially supported)
RG351V
What version of ArkOS are you on?

You can find this during the initial boot process right before the Emulationstation loading screen
16102023

Anything else you'd like to include that may help to review this issue or feature request?
Is there a way to manually update without refreshing the whole card?

Try retroarch or manual option 1 from the FAQ for the RG351V: https://github.com/christianhaitian/arkos/wiki/Frequently-Asked-Questions---RG351V#manual-option-1.  After updating to the 01272024 release, that issue should be resolved.


Release Controller Should Enforce State For All Releases Based On Their Successor If Found

Right now Shipper only reacts to the last two releases, the incumbent and contender. This means that if there is a problem in a release that is not either the incumbent and contender, it will stay there for ever.
This issue showed itself with a Release object that was supposed to be completely rolled out. however, for some reason, the spec of the CapacityTarget and TrafficTarget were still saying that this release should have 10% capacity and 10% traffic.
In a perfect world, Shipper would do the following:

Receive an update event for an old Release
Find out the next release in the history, the newer Release
Figure out what state the associated target objects (InstallationTarget, CapacityTarget, and TrafficTarget) of the old release should be, based on the newer release
Apply that to the target objects

This issue is planned to be fixed with the upcoming merge request #70. The relevant e2e tests are to be delivered asap.


Allow to filter null in array_agg

Is your feature request related to a problem or challenge?
Yes, I want nulls to be filtered from array_agg when I specify with_ignore_nulls: true in AggregateExprBuilder to have behavior similar to Spark, this should also work when with_distinct is false or true
Describe the solution you'd like
Specifing with_ignore_nulls: true should ignore
Describe alternatives you've considered

Adding a filter for removing nulls to the child before creating the array_agg
Copy the implementation and added in myself as an UDAF

Additional context
This PR might be relevant:

#11447

I'm willing to open a pull request

AFAICT, the with_ignore_nulls: true is currently... ignored?
Given it's part of the "logical plan", it should be obeyed. It's probably part of LP because the LP is half-way between parser's AST and real plan  (see also https://github.com/apache/datafusion/issues/12604 and cc @alamb).
I'm OK for respecting  with_ignore_nulls in the plan, but I would actually consider removing it completely. Simpler things are better. The same functionality is available using AggregateFunction.filter.
What calls for keeping with_ignore_nulls is that aggregate functions -- in general -- skip over null inputs. SQL standard array_agg is one of few exceptions (and some systems like Snowflake and apparently Spark skip nulls for array_agg). Ideally this could be decided by the function itself, so could be a parameter of the ArrayAgg struct.

I like the idea that we leverage existing filter in AggregateExec for this usecase
https://github.com/apache/datafusion/blob/c030bfe1e197fdbb385511a2e4fedf753f284d6f/datafusion/physical-plan/src/aggregates/mod.rs#L355-L356
I guess you can create AggregateExec with predicate that only non-nulls goes to array-agg, similar to the following syntax
select array_agg(column1) from table where column1 != NUll;

To be clear, I think the SQL parser supports predicates like ARRAY_AGG(x) IGNORE NULL  and we have logical plan support for them.
However, some aggregates (silently) ignore this, see

https://github.com/apache/datafusion/issues/9924

I think filling out support (or at least making it clear that this syntax is ignored) would be a great idea

Thank you @alamb. do you have an example of an agg that doesn't ignore them?
I tried find one, but maybe i was searching the wrong way

Thank you @alamb. do you have an example of an agg that doesn't ignore them? I tried find one, but maybe i was searching the wrong way

I think first_value supports nulls / ignore nulls


Mutual Authentication with MBEDTLS

Hi,
Is it possible to perform mutual authentication with the Trust X on mbedtls, with the certificate and private key loaded into the trust X? I'm unsure since as far as I know the process would involve exposing the private key to mbedtls for the signing of the certificate when it's used to authenticate the client. Any help is appreciated.
Thanks,

Hello @EnkyuuBi ,
yes that would be possible

Thanks @ayushev
That's good to know, are there any guides that I can refer to to implement the functionality? Thanks again for the help.

the mbedtls works with OPTIGA already in the way you said, by letting the crypolibrary to build up a TLS connection (prepare and send mesages, do some intermediate crpypto etc) and OPTIGA does only the most roublesome parts, like a signature generation and ECDH
More on technical implementation can be found in the wiki section of this repo https://github.com/Infineon/mbedtls-optiga-trust-x/wiki#how-we-use-optiga-trust-x-with-mbedtls
Also you can refer to the figure here

Does this mean that for mutual authentication the certificate and private key are not exposed to Mbedtls?

The certificate contains only public data and read out during the mbedtls initialization (here and here), the private key isn't exposed and used on-demand when it's needed over the optiga_crypt_ecdsa_sign() function here

Understood, thank you very much! That clears up most of my questions.
One side question: is it possible to expose the optiga_crypt_ecdsa_sign() function to use in mbedtls' CSR generation process?

Unfortunately there is no mbedtls PSA API available for OPTIGA Trust X

Understood, thanks!
That's all the question I have.


Tests examples extended (do not merge)

added tests to check if use_inflows and use_traffic_lights is working in the bay bridge example

Pull Request Test Coverage Report for Build 1257

4 of 4 (100.0%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.2%) to 76.778%

Totals

Change from base Build 1213:
0.2%

Covered Lines:
5085

Relevant Lines:
6623

💛  - Coveralls

dibs

LGTM

@AboudyKreidieh Is there any reason this still has a DO NOT MERGE flag?

Hey sorry missed the message. Something's wrong with the starting positions on the bay bridge. Haven't had time to look at it yet, but it's a priority thing for me for sure. More soon hopefully.


Fixed #23816 -- Add support for default deferred fields

Ticket: https://code.djangoproject.com/ticket/23816
Already existing PR: #9309

Thanks for your review. AFAIK, a custom manager would not help preventing field loading in select_related.
Currenly, adding a call to defer is required:
Publication.objects.select_related("user").defer("user__nickname")

With this PR content, you can simply write:
Publication.objects.select_related("user")

Basically, i would like to have a simple way to tell django: never load these fields unless explicitly requested.
This can be useful for fields used for filtering on the db side (e.g. tsvectors) which can have a huge size and are useless on the django side.

@manubb thanks for the extra details.
I understand your need but it feels like the point above about why field deferral should be supported  through Meta and not other queryset achievable options still stand. For example, you might want to make sure that a field is always select_related or that an annotation is always included when select_related is used.
I fear we might be heading a slippery slope like we did with ModelForm.Meta's myriad of options and I'd like us to make sure we know we're aware of that. Seeing how Meta.ordering has been a constant source of confusion which forced the adoption of hard to document special rules (e.g. wrt to distinct and aggregation) opening this door should not be taken lightly. Anything that has global effects on the base and default managers behavior will have ripple effects through many parts of the ORM.

@charettes I took some time to think about this and you are probably right: if we are to add this in Meta, there is no clear reasons not to add a bunch of other things.
In we want to avoid adding data in Meta, the other options i can think of are:

adding a boolean param in the fields indicating that the field should always be default deferred (this is the initial proposal of the ticket)
adding an optional method on the model returning the set of default deferred fields (which i think is very close to adding the data in Meta)

What is your feeling on this?

@charettes ?

Hello Manuel, currently in the middle of moving to a new place and I introduced a few regressions in 3.2 which takes most of my "Django" time.

adding a boolean param in the fields indicating that the field should always be default deferred (this is the initial proposal of the ticket)

I'm personally not against the idea TBH but another previously core developer thought it might not be appropriate. To me the characteristic of querying point is not as strong as it uses to be if we think of options such as related_query_name.

adding an optional method on the model returning the set of default deferred fields (which i think is very close to adding the data in Meta)

I don't think we've used this pattern before for anything else so that might be alien to the other Model APIs.

If you want to reach a larger audience to gather feedback and traction about your proposal I suggest you post to the developer mailing list as way more folks monitor this channel than Github.

@charettes Sorry for being a little pushy. (I had some time last week to move forward on this.)
I'm going to follow your suggestion then!

I think it would be preferable to invest time in allowing managers to be defined with lazy queryset method calls resolved at contribute_to_class time instead to remove the necessity to define Manager subclasses for the sole purpose of overriding .get_queryset() to filter() or defer() from super().get_queryset().

I am slightly against this. Django has tons of "shortcuts" to things that are just a couple extra lines of code. This is another such case. One can always introduce
def Manager_with_deferred(deferred):
    class Manager:
        ...
    return Manager

@adamchainz Thanks for your review. I provide some more context: this PR add a feature that can not be implemented with custom managers (see this comment).
Do you think that:

the deferred fields should be specified in the Meta class of a model?
the deferred fields should be specified with a boolean attribute on the fields?
django does not need to support default deferred fields?


Add inList support

This pr adds the following:

Add an alias for substr to perform substring
Adds support for inList rex call operation: SELECT column in (val1, val2, val3) from table. Fixes #531.
Adds support for negated between SELECT column NOT BETWEEN low AND high from table. Fixes #519.

Todo: Add tests

Codecov Report

:exclamation: No coverage uploaded for pull request base (datafusion-sql-planner@ac6f940). Click here to learn what that means.
The diff coverage is n/a.

@@                    Coverage Diff                    @@
##             datafusion-sql-planner     #604   +/-   ##
=========================================================
  Coverage                          ?   60.99%
=========================================================
  Files                             ?       71
  Lines                             ?     3571
  Branches                          ?      723
=========================================================
  Hits                              ?     2178
  Misses                            ?     1249
  Partials                          ?      144

:mega: Codecov can now indicate which changes are the most critical in Pull Requests. Learn more


Add tooltip probably to WalletDetails page, like on the Address field, explaining that this is an "XMR"-only address and that it doesn't support other forks

The problem is figuring out how to fit the tooltip in... where would it go?


Add ShapeCompletion Action

Resolves 7586

@adlarkin thoughts on pointcloud vs point_cloud?

I think pointcloud (what you have) is fine!


only load tracepoint arg structs if args builtin is used

Fixes: https://github.com/iovisor/bpftrace/issues/422

I'm not happy with the amount of duplicated code between SemanticAnalyser and TracepointArgsVisitor though... I think we can improve it.

thanks!


Fix a few formatting errors that were showing up as warnings.

About the PR

Fixes cases where if/while/for expressions were missing a space before the opening paren. (ie: for(blah)
Fixes cases where the result of n if statement were inline with the if clause. (ie: if (blah) foo();
Fixes a couple places that weren't properly tab-aligned.
Adds empty lines between if-return clauses and other code.
Switches a couple random tabs to spaces.
Switches one of the integration tests to use Assert.That(value, Is.Condition) because I was in the area.
Random assorted warning fixes depending on what I was feeling motivated to bother with while I was in the area.
99% of this PR is just rearranging whitespace so it shouldn't have any actual behavior changes.
Does cull a couple hundred warnings.

Media

[X] I have added screenshots/videos to this PR showcasing its changes ingame, or this PR does not require an ingame showcase

Changelog

This will conflict with a lot of prs with little gain since most of the conflicts would be about whitespaces/lines and other minor things like that, fixes like these are acceptable as part of more condensed and specific prs (i.e cleaning up a specific system of obsolete methods so you might as well fix the formatting), though

Switches one of the integration tests to use Assert.That(value, Is.Condition) because I was in the area.
Random assorted warning fixes depending on what I was feeling motivated to bother with while I was in the area.

would be wanted, so split those into other prs

Conflict purgatory


Add Fashion Platform GW headers

Fashion Platform GW headers and exclude X-Zalando-Request-URI from the Rule 166,
Fixes #1216

:+1:

👍


Allow a request executor to be passed

This allows the developer to pass in a custom RequestExecutor implementation, which we do in IAMUI.

Coverage decreased (-0.1%) to 76.954% when pulling 09619daf8f49be22028e928da1d41e9536271e97 on request-executor-fix into 492969188a57e22fb74334180d5485f6ff2957e1 on master.

Looks good to me!


#375 Add support for deleting offsets

Closes: https://github.com/kcctl/kcctl/issues/375
Must be merged after https://github.com/kcctl/kcctl/pull/393 to let tests on TRUNK run

@C0urante can you please take a look?

@C0urante thanks for the suggestions. I have addressed it and also I have updated the Debezium version to 2.4.0.Final to use the new STOPPED state. If you want to have this update on separate PR, I can do it.

Regarding the bump in Debezium version--I'm fine with that in this PR 😄
It looks like we can also start work on https://github.com/kcctl/kcctl/issues/346 now that we're on a version of Debezium that has native support for the STOPPED state! 🎉


Support AF_UNIX for windows

#22285
Unix domain socket has been added to windows: AF_UNIX comes to Windows
Golang net pkg has adopted this long ago:
https://go-review.googlesource.com/c/go/+/125456
https://go-review.googlesource.com/c/sys/+/132555
grpc-go already support this.
AF_UNIX on windows is seamlessly integrated with winsock API.
The modification needed are:

Set the right address family AF_UNIX depending on the address config, instead of using AF_INET6 all the time.
Ignore socket options for tcp.

@drfloob Can you review?

Hi @youyuanwu. This feature has been in discussion for a while. In short, we need to consider whether unix sockets can be treated identically within gRPC across platforms, or if there are subtle semantic differences that require a bit more scrutiny. There are a few folks reviewing the concept now, and we'll get back with you soon.
Thank you for the contribution regardless!

The PR has been tested manually with grpc-go, which already supports AF_UNIX on windows.
run_tests.py with c++ tests passes locally.

I have also done some testing of this with my own C++ server and C# client (grpc-dotnet which already supports AF_UNIX). No issues found.

Sorry for the delay, @youyuanwu. We've gotten consensus that this seems pretty safe to implement, so we can continue with the code review. Unfortunately, I'm going to be out for the remainder of the year, so we'll have to pick it up in January, but I'll do a quick first pass today.

@drfloob The last test run had failures when event engine listener is turned off. I cannot see the test run links anymore and do not remember the config to turn on/off, so please rerun the pipeline again.
Let me know if that use case needs to be supported as well.

I am updating this page 100 times a day, looking for the moment when the PR will be merged in production. This is a great achievement and I am looking forward to using it.

@youyuanwu
You can see the previous failures by hovering your mouse over the red "x" on the right, next to every individual commit

I do not remember the config to turn on/off, so please rerun the pipeline again.

I kicked off CI again, but you can disable the event_engine_listener experiment with the env var GRPC_EXPERIMENTS=-event_engine_listener, or if you're running tests through bazel: bazel test --test_env=GRPC_EXPERIMENTS=-event_engine_listener.
You'll likely want to test with:

GRPC_EXPERIMENTS=event_engine_client,event_engine_listener
GRPC_EXPERIMENTS=-event_engine_client,-event_engine_listener

@drfloob All failed windows tests has the config: experiment=no_event_engine_listener
Does this feature need to support no_event_engine_listener? I only changed event engine listener. I assume that there is another iomgr path for listener that I never changed. If this is required I will give a shot later this week.

Does this feature need to support no_event_engine_listener? I only changed event engine listener. I assume that there is another iomgr path for listener that I never changed. If this is required I will give a shot later this week.

Support for iomgr is necessary (that experiment enables iomgr listeners, effectively). There is indeed another file that I missed. src/core/lib/iomgr/tcp_server_windows.cc. I'm guessing we're looking at a too-small sockaddr.

Status: experiment=-event_engine_listener is supported now.
One more finding:
This change might have exposed a separate potential deadlock bug in windows event engine client.
With $env:GRPC_EXPERIMENTS="event_engine_client enabled. The client of uds will hit assert in debug mode about potential deadlock if server is not listening. This is related to grpc_core::ExecCtx dispatching pending tasks that uses locks. Will create a spearate issue about this, since I do not think the current change is the culprit.
@drfloob Can you rerun the ci?

I see the mutex re-acquisition problem https://source.cloud.google.com/results/invocations/419395f4-3a67-4bd1-9c73-f3fe2a43c1e5/targets/%2F%2Ftest%2Fcore%2Fend2end:connectivity_test@experiment%3Dpromise_based_client_call/log

I see the mutex re-acquisition problem https://source.cloud.google.com/results/invocations/419395f4-3a67-4bd1-9c73-f3fe2a43c1e5/targets/%2F%2Ftest%2Fcore%2Fend2end:connectivity_test@experiment%3Dpromise_based_client_call/log

Yes. This is the symptom. The easiest repro is to create any uds client that try connect to a non-existent server. Somewhere inside "event_engine_tcp_client_connect()" has the problem.

@youyuanwu I have a fix in the linked PR. Thanks for helping us find some bugs!

@drfloob I rebased with master with your change. The locking issue has been fixed by your change.

@youyuanwu Your change is merged, thanks for adding this feature!

@drfloob Thanks for the review and help. I will leave the documentation change to you, to say that uds is no longer unix only.

Hey, did this make it into main?

Hey, did this make it into main?

Yes. https://github.com/grpc/grpc/commit/a58f83c947be52d4b622aa0d32435323b81a8e5e


[BUG] Assembly broken

The NASM 2.14.02 gives just dots as output
What ever i write into the compiler explorer the NASM option only gives dots as output and when put "Compiler into binary" option is on it shows that the dots have the right value on them but the compiler explorer just seems to show dots as output.
I'm making my own programming language so i use compiler explorer to help me seeing what other compilers do on certain input.
So today (21/03/2020) i opened compiler explorer on my chrome as always and started to write this code here:
"a dd 12
b dd a + 3"
and for some reason NASM just gives this:
"a:
....
b:
...."
and so i checked for the binary using the show binary option and it gave me this:
"

"
Normally it works fine, but when i started to put some opcodes like:
"mov eax, 1"
and it giving me just more dots, i feel that something is now wrong.

My side info:

OS- Windows 10
Browser- chrome
Version 80.0.3987.149

I hope you can fix this because i need this as a tool and probably a lot of other people too.
:D

A link showing this: https://godbolt.org/z/KRWdM2

Seems like the disassembly is failing somewhere and instead we're getting the hex dump of what's going on.

Hi!
objdump shows those dots and that's what Compiler Explorer shows you in return
More or less, in order to assemble your code and show it we do this:
/t/nasmtmp $ cat example.asm
a dd 12
b dd a + 3

mov eax, eax
/t/nasmtmp $ /nasm-2.14.02/nasm -f elf -g -F stabs example.asm
/t/nasmtmp $ /gcc-8.2.0/bin/objdump -d example.o -l --insn-width=16

example.o:     formato del fichero elf32-i386

Desensamblado de la sección .text:

00000000 <a>:
example.asm:1
   0:	0c 00 00 00                                         ....

00000004 <b>:
example.asm:2
   4:	03 00 00 00 89 c0                                   ......

(Note that -z does not change the output)
Asking objdump to dissasemble all sections shows that it's perfectly capable of interpreting them too
t/nasmtmp $ /opt/compiler-explorer/gcc-8.2.0/bin/objdump -D -z example.o -l --insn-width=16

example.o:     formato del fichero elf32-i386

Desensamblado de la sección .text:

00000000 <a>:
example.asm:1
   0:	0c 00                                           	or     $0x0,%al
   2:	00 00                                           	add    %al,(%eax)

00000004 <b>:
example.asm:2
   4:	03 00                                           	add    (%eax),%eax
   6:	00 00                                           	add    %al,(%eax)
example.asm:4
   8:	89 c0                                           	mov    %eax,%eax

(Omitting stabs)
So it's not clear to me if this is a bug at all. What can I be missing?

In the case of just data (the a dd, b dd parts), I think that works much differently from it has in the past.
You do seem to need a new label when starting actual code, not sure that's preventable.
Note that it also makes sense to have a label, otherwise, how are you going to call the code? call/jump to address b+4?
It also does help to select Compile to binary for the codelens feature, I think that's supposed to be on by default (because it always uses objdump), but for some reason that's off by default.
https://godbolt.org/z/EosQtA


haversine error

Bug/Feature Request Description
Haversine primitive raises an error in the following use case:
latlong1 = pd.Series({71: nan, 72: nan, 73: nan, 74: nan, 75: nan})
latlong2 = pd.Series({71: (40.798587799072266, -73.9694061279297), 72: (40.8021354675293, -73.96797943115234), 73: (40.77003860473633, -73.96251678466798), 74: (40.77336883544922, -73.96420288085938), 75: (40.73043823242188, -73.98625183105469)})

TypeError: 'float' object is not subscriptable

Issues created here on Github are for bugs or feature requests. For usage questions and questions about errors, please ask on Stack Overflow with the featuretools tag. Check the documentation for further guidance on where to ask your question.

is there a way to vectorize the whole haversine primitive while also not erroring out on nans? any help on this is appreciated!

It looks like I got it working by representing LatLong missing values as (np.nan, np.nan), however I still get this error when I use the first or last agg_primitives.

There's a pending PR about this issue: https://github.com/FeatureLabs/featuretools/pull/693
I don't know if it also fixes the problem with the first and last primitives.

Thanks for bringing this up again @ekerazha!
Some extra info on how this happens with first and last primitives would be helpful.  Does this only happen with LatLong data?  Do you have an example of when this occurs?

@rwedge

Does this only happen with LatLong data?

This was basically my experience: I had missing LatLong values, represented as np.nan, and I got that error for haversine. So I tried to use (np.nan, np.nan) instead of np.nan and it fixed the haversine issue. Then I added all the available aggregation primitives instead of using the default ones and I got that error again. After removing the primitives one-by-one, I found that the first and last primitives caused the "new" issue.

@ekerazha thanks for the info.  Do you have some example code I could run to try to reproduce this.  Also knowing what kind of environment you have would be helpful.  Running ft.show_info() in a python shell or featuretools info from a terminal would provide some useful info about your featuretools version, etc.  Thanks!

@ekerazha I believe the issue you were having should now be resolved with Featuretools v0.16.0 which was recently released. If you are still having issues with version 0.16.0, please let us know and we can reopen an issue.


Put the zkproof and anchor check back in

https://github.com/Zondax/ledger-zcash-rs/blob/69cc7504a7a6a1e4083f374987c8e2a40f196925/zcash-hsmbuilder/src/txbuilder_ledger.rs#L519-L531
https://github.com/Zondax/ledger-zcash-rs/blob/69cc7504a7a6a1e4083f374987c8e2a40f196925/zcash-hsmbuilder/src/prover_ledger.rs#L154-L184

What's the status here?

Currently for CI tests we created fake transaction data to work with. We can re-enable these checks and close the issue when we have ''real'' transaction data to use in the CI tests. Note that these checks are only ''additional sanity checks'' on blockchain data.

The todo's would be to:

Go through the process of generating proper zcash blocks and how rseed is stored in these blocks
Generate multiple good blocks (as we use more than one spend note in CI tests), having the same root/anchor
Integrate those blocks (with witnesses) in the CI tests.


Abstract away regular expressions for path validation in a programming language agnostic way

See https://github.com/INCF/pybids/issues/131#issuecomment-370916864

Is this issue telling just to create a json file that looks like
[
  {
    "name": "is_cont",
    "pattern": "^\\/(sub-[a-zA-Z0-9]+)",
    "description": "Verifies that the file is..."
  },
  ...
]

where all the regex from the entire bids-validator tool will be kept.
Then this file created will be imported at appropriate locations and the variables will be fed at appropriate locations. Am I right?
Also I want to ask that if the value for name here could be any variable name comfortable with the programmer's point of view.

This should go to https://github.com/INCF/bids-validator-common

The patterns are ready - we should switch to using them.

This will probably require https://github.com/INCF/bids-validator-common/issues/2 to be done first.


Issue for macos spotify

doesnt work with muting ads on spotify, but it saysit sworking on the terminal. i refreshed the app, but still no use

Hey, allow me to ask a few questions to try to help you:

Are you using the desktop Spotify app, right?
Could you copy and paste the output of the console here after you run the script?
Can you tell me what version of osx are you running. You can check it out by running defaults read loginwindow SystemVersionStampAsString in the terminal. Then copy the result here

Close due to inactivity


libSyntax: Support tuple expression.

Tuple expression essentially has the same underlying structure as
function call arguments in libSyntax. However, we separate them as
different libSyntax kinds for better usability.
Different from AST, libSyntax currently allows single-child,
label-free tuple expressions (represented as ParenExpr in AST). This is
subject to change if we need to adopt the same differentiation in
libSyntax in the future.

@swift-ci please smoke test and merge


[BUG] CHIP_ERROR_OUTBOUND_MESSAGE_TOO_BIG when querying attributes

Reproduction steps

arm 32-bit environment build
Commissioning specific devices (type: thermo-hygrometer, vid:65521, pid: 32773)
Attribute Query

...
Command failed chip.devicecontroller.ChipClusterException: CHIP cluster error: 195

Bug prevalence
About five minutes after the Commissioning
GitHub hash of the SDK that was being used
2e9c162
Platform
android
Platform Version(s)
No response
Anything else?
I'm using an unauthenticated Matter device. However, for other unauthenticated devices (light bulbs), I could see that the attributes are searched normally. I want an answer about what the difference is and why the CHIP_ERROR_OUTBUND_MESSAGE_TOO_BIG error occurs. If possible, please also give me a solution.
CHIPError.h

CHIP_ERROR_OUTBOUND_MESSAGE_TOO_BIG can happen for two reasons:

Over BLE, if there are chained buffers involved and CompactHead can't fit all the data into a single buffer.
Over a UDP transport, if sendmsg doesn't send the full message.

The one line from the log above is missing too much context to be able to tell which of those cases it falls into.  Please attach (not paste) complete logs.

@bzbarsky-apple
Thank you for your answer.
It's not what I tested, I requested the log and I'll attach it additionally when I get it. Thank you.


horovod mode increase lr

Not really a 🐛 Bug
https://github.com/PyTorchLightning/pytorch-lightning/blob/master/pytorch_lightning/trainer/distrib_parts.py#L299
Under horovod mode, the learning rate will automatically be increased by hvd.size().
This behavior is different from ddp, so it may confuse the users.

mind check @tgaddair @SkafteNicki  ^^

I was not aware of this. It's seems that it is a good practice, that really can help boost training speed (see this paper https://arxiv.org/abs/1706.02677 and this issue https://github.com/horovod/horovod/issues/384). So maybe we should implement something similar for ddp backend.

Yes, in Horovod (and I believe DDP), increasing number of workers is analogous to increasing the total batch size during training.  As such, scaling the learning rate proportionately is considered a best practice.  It's good to handle it internally to the distributed_backend, because some backends may behave differently.  For example, in Horovod, enabling the Adasum optimizer only requires scaling by the number of GPUs per host.

I agree it is a good practice. However if it's not the only way, I don't think it should be the default, Especially without notifying the users.
It could instead be an argument of the trainer.

I think making it configurable is reasonable.  However, I do think it should be enabled by default.  Part of the goal of the Trainer abstraction is to make distributed training accessible to people who are not familiar with distributed training concepts / best practices.
For most users, unless they are using custom learning rate schedules or unusual optimizers, they will want to scale the learning rate.  At the same time, most users would not know to do this themselves, so I fear without enabling it by default, they would not do so, and their models would converge worse as a result.

BTW, there may be a problem when lr_scheduler is LambdaLR. It seems LambdaLR will collect the lrs in optimizer and save as base_lrs. The lambda function will take place on the base_lrs. Even you change the lr later, the lr scheduler would ignore it.
https://github.com/pytorch/pytorch/blob/879cf0b15a54c7848ae710e3d0ec62c4a9d7d3dd/torch/optim/lr_scheduler.py#L43
LambdaLR scheduler I believe is a commonly used scheduler. For now I think it's safer to delete that line for now, and then think of what is the best way to implement it.

Hey @ruotianluo, that's a good point regarding interaction with LambdaLR and other LR schedulers.  Can you take a look at #2626 and see if it addresses your concern?

@tgaddair I still want to defend against scaling learning rate by default. By primitive search, it doesn't seem to me that in nlp, people do the same learning rate scaling. Bert uses batch size 256 and learning rate 1e-4; Roberta uses batch size 8k and max learning rate is 4e-4/6e-4(depending on the model size). I think it may be related to optimizer(in nlp it's usually adam). I don't know if this fact can convince you.

Hey @ruotianluo, even when training BERT with Horovod, it's common practice to scale the learning rate.  See:

https://github.com/LeoWood/bert-horovod/blob/master/optimization_hvd.py#L61
https://github.com/google-research/bert/pull/568/files#diff-717a6b63e0d2e51c2ff68a440534783eR61

Fundamentally, when you add more workers, you are increasing the batch size.  That holds true whether it is a vision task, NLP, or other scenarios.  So you need to account for that somehow (most commonly through LR scaling, though I imagine other means are possible as well).
I do agree we should make this configurable, though.  I'm interested in putting together a separate PR for this, but it should include changes to DDP as well.

https://arxiv.org/pdf/1904.00962.pdf. This paper uses square root scaling for bert(and also imagenet classification too). Aand albert(from google) uses this approach (https://arxiv.org/pdf/1909.11942.pdf).
The first link you provide doesn't have any results. For the second, I didn't see any quantitative results either how that would affects.(and it is not merged yet.)
Do other frameworks do learning rate scaling by default too(Keras, fastai?)? If it's common across other libraries, I think it's fine too.

Hey @ruotianluo, in my experience, frameworks that expose distributed training to users as an API (like tf.distributed) will mention in their docs that it's good practice to scale the LR, but will leave it to the user to do so (this is what we do with Horovod as well).
However, frameworks that attempt to completely abstract away distributed training (like PyTorch Lightning is seeking to do) should provide a good reasonable default.
I agree with you that in practice, it may be that linearly scaling the learning rate does not provide the best model performance, in which cases the researchers will often hand-tune the combination of learning rate and total batch size (i.e., number of workers) to obtain the best performance.  To support that, there is definitely a need to make learning rate adjustment configurable.
At the same time, whatever solution we come up with needs to be backend-agnostic.  One of the selling points of PL is the ability to swap out different distributed backends.  If we couple the LR scaling to the backend (e.g., require the user to put lr * hvd.size() in the LightningModule), we lose a lot of the benefit.
With that in mind, here's what I'm currently thinking could be a good solution:

Provide a good reasonable default for users who are not experts in distributed training (linear learning rate scaling) for Horovod and DDP.
Provide an optional method in the LightningModule that allows the user to adjust the learning rate as a function of the number of workers, independent of the specific backend being used, which will override the default in (1):

class MyModule(LightningModule):

...

    def adjust_learning_rate(base_lr, world_size):
        return base_lr * sqrt(world_size)

@ruotianluo @williamFalcon @Borda @SkafteNicki what do you think?

cc: @PyTorchLightning/core-contributors

shall be resolved in #2626


Updated CUDA Name Clash Checker By Added CUDA-specific keywords

This pull request addresses issue #59  by adding  more CUDA-specific keywords to enhance the checking of variable/function names and prevent name clashes

/bot checklist

/bot run docs

/bot run docs

/bot run docs

/bot run docs

Please fix merge conflicts so this can be merged

/bot run pr_tests

/bot run pr_tests


AxonDeepSeg to Keras Framework

This PR and branch aims at introducing AxonDeepSeg in Keras framework .
The following thing have been done: -

Creation of model in Keras framework implementing AxonDeepSeg(mini version)
Metrics implementation such as pw_dices_axon and pw_dices_myelin.
Model is working with sample data hosted at OSF https://osf.io/uw5hv/?action=download

Things TODO:-

Weighted Cross Entropy for misclassification, adaptive learning rate and adaptive decay rate.
Data Augmentation to improve the model.
Integration of model(written in keras) into the base model(written in tensoflow).
Generalization of the model (notebook) and appropriate refactoring.
Testing module for jupiter notebook.

Coverage decreased (-0.04%) to 82.38% when pulling 89081a186dfe35da2d34681a6ddaa7ddb08587f6 on vs74:axondeepseg_keras into 804ca0b9141f52590c64ac3a39b97c85782b91de on neuropoly:axondeepsegkeras.


Feature/serial

Here, we extend and improve the serialization and deserialization of pydantic models.
This includes domain, models, and strategies.
Helpers for (de)serialization can be found in bofire.serial.
To make this possible, we cleaned up the AnyX definitions (now found in bofire.any) and split some existing modules, e.g., bofire.domain.features.
Furthermore, we added capabilities to (de)serialize pandas DataFrame and Series.
Note that the necessary restructuring of some modules can lead to broken imports in code using the previous version.

@bertiqwerty : can you have a look at this PR from Benni which is adding the serialization to bofire and tidies up the code a bit.
For me it is fine, and we should merge it is in as soon as possible as this will also cause some restructuring in other PRs.


Current Issues wıth Scan

[ ] - Settıng sensıtıvıty ıs problematıc, cancel ıt for now.
[ ] - Set TC at the begınıng to 100 ms, and default dropdown value should be the same as well.
[ ] - Always read tc, and sens ın the beggınnıng and show ın the panel.
[ ] - Check scan method.
[ ] -Convert step ınto ps ın fıle.
[ ] -Signal is all positive, no negative signals are observed. Why? Check locking screen while measuring.

#12


Support for python 3.10?

Thanks again for maintaining a very useful library that is often used for SAML SSO.
I noticed when installing xmlsec on CPython 3.10 that a wheel is not available:
Installed /opt/hostedtoolcache/Python/3.10.2/x64/lib/python3.10/site-packages/python3_saml-1.14.0-py3.10.egg
Reading http://github.com/mehcode/python-xmlsec/tarball/master
Searching for xmlsec>=1.3.12
Reading https://pypi.org/simple/xmlsec/
Downloading https://files.pythonhosted.org/packages/70/d7/83ac0d43bb92333c8ba3d9be97b85d1b[285]
...
Best match: xmlsec 1.3.12
Processing xmlsec-1.3.12.tar.gz
Writing /tmp/easy_install-bey4yd1m/xmlsec-1.3.12/setup.cfg
Running xmlsec-1.3.12/setup.py -q bdist_egg --dist-dir /tmp/easy_install-bey4yd1m/xmlsec-1.3.12/egg-dist-tmp-umgx26l1
WARNING: The wheel package is not available.
WARNING: The wheel package is not available.
WARNING: The wheel package is not available.

I understand that only 3.5 to 3.9 is supported. Are there plans to support for 3.10?

I second this. For my project I only use >=3.10.
Because of the missing wheel, it is tried to create one locally upon installation (xmlsec is no direct dependency for me). The wheel creation crashes (on windows) because "stdio.h" is missing, which in itself is a whole 'nother issue, I know.

I 3rd it 😂 or any advice on workarounds / self-compilation etc.

Depends on linux it was pretty easy to self-compile. On windows I was not able to get it to work.
So if you are on linux I may be able to help you.

Thanks, but I'm on Windows

Hi there 👋 wonderful codebase maintainers. Just letting you know I am still really looking forward to 3.10 support on this. ❤

This is blocking my project from upgrading to 3.10 as there are no windows py310 binaries in the latest release

Please upgrade for 3.10!
we have a project with 3 major components and in one of them, xmlsec is the only library out of 35 that prohibits the upgrade of our platform to 3.10!

Would really love support for >=3.10! I have a hard requirement to use 3.10 at work, and would really love to be able to use xmlsec on a project.

Hi all, I've uploaded windows wheels including 3.10 ones with the release of 1.3.13.

Can confirm! Thanks for that!


problem

Who will add a code to open the exe program in the singing?
For example, the third stage of the Bob mod dies to open a web page, I want to change it to open the exe program.

maybe try this


Fixed module build error

When building in a Swift project, the module would fail to build with the error Could not build Objective-C module 'MSSTabbedPageViewController' and complain about interface declarations.
Adding imports to a few files fixed these errors and the module builds correctly.

Thanks!


FCS-87 - Prevent path parameters from being lower cased

:tada: This PR is included in version 4.0.10 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


tar-gnu dependency

tar-gnu is needed for extract, along with some others.

This is only required for build and not necessary anymore.


No module named qurator when callling eynollah CLI

I get the following error when trying to call eynollah in the windows terminal:
Traceback (most recent call last):
  File "C:\src\python3\lib\runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\src\python3\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "C:\Users\Moi4\Desktop\code\misc\eynollas\venv\Scripts\eynollah.exe\__main__.py", line 4, in <module>
ModuleNotFoundError: No module named 'qurator'

The error happens regardless of the arguments.
I installed eynollah (and nothing else) in a fresh virtual environment with Python 3.10.11.
If that helps, here are the windows specs:
Edition	Windows 10 Home
Version	22H2
Installed on	05/05/2021
OS build	19045.4780
Experience	Windows Feature Experience Pack 1000.19060.1000.0

We have removed the qurator namespace recently in #133.
Can you try pip uninstall eynollah, git pull and pip install . to see if the error persists?

Unfortunately, building directly from the repo gives me the following error
Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [39 lines of output]
      Traceback (most recent call last):
        File "C:\Users\Moi4\Desktop\code\misc\eynollas\venv\lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 353, in <module>
          main()
        File "C:\Users\Moi4\Desktop\code\misc\eynollas\venv\lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "C:\Users\Moi4\Desktop\code\misc\eynollas\venv\lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
        File "C:\Users\Moi4\AppData\Local\Temp\pip-build-env-fzdy36xu\overlay\Lib\site-packages\setuptools\build_meta.py", line 332, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=[])
        File "C:\Users\Moi4\AppData\Local\Temp\pip-build-env-fzdy36xu\overlay\Lib\site-packages\setuptools\build_meta.py", line 302, in _get_build_requires
          self.run_setup()
        File "C:\Users\Moi4\AppData\Local\Temp\pip-build-env-fzdy36xu\overlay\Lib\site-packages\setuptools\build_meta.py", line 503, in run_setup
          super().run_setup(setup_script=setup_script)
        File "C:\Users\Moi4\AppData\Local\Temp\pip-build-env-fzdy36xu\overlay\Lib\site-packages\setuptools\build_meta.py", line 318, in run_setup
          exec(code, locals())
        File "<string>", line 1, in <module>
        File "C:\Users\Moi4\AppData\Local\Temp\pip-build-env-fzdy36xu\overlay\Lib\site-packages\setuptools\__init__.py", line 117, in setup
          return distutils.core.setup(**attrs)
        File "C:\Users\Moi4\AppData\Local\Temp\pip-build-env-fzdy36xu\overlay\Lib\site-packages\setuptools\_distutils\core.py", line 146, in setup
          _setup_distribution = dist = klass(attrs)
        File "C:\Users\Moi4\AppData\Local\Temp\pip-build-env-fzdy36xu\overlay\Lib\site-packages\setuptools\dist.py", line 283, in __init__
          _Distribution.__init__(self, dist_attrs)
        File "C:\Users\Moi4\AppData\Local\Temp\pip-build-env-fzdy36xu\overlay\Lib\site-packages\setuptools\_distutils\dist.py", line 279, in __init__
          self.finalize_options()
        File "C:\Users\Moi4\AppData\Local\Temp\pip-build-env-fzdy36xu\overlay\Lib\site-packages\setuptools\dist.py", line 635, in finalize_options
          ep(self)
        File "C:\Users\Moi4\AppData\Local\Temp\pip-build-env-fzdy36xu\overlay\Lib\site-packages\setuptools_ocrd\_integration\setuptools.py", line 27, in get_ocrd_tool_version
          dist.metadata.version = _get_version()
        File "C:\Users\Moi4\AppData\Local\Temp\pip-build-env-fzdy36xu\overlay\Lib\site-packages\setuptools_ocrd\_get_version.py", line 11, in _get_version
          return json.load(fp)["version"]
        File "C:\src\python3\lib\json\__init__.py", line 293, in load
          return loads(fp.read(),
        File "C:\src\python3\lib\json\__init__.py", line 346, in loads
          return _default_decoder.decode(s)
        File "C:\src\python3\lib\json\decoder.py", line 337, in decode
          obj, end = self.raw_decode(s, idx=_w(s, 0).end())
        File "C:\src\python3\lib\json\decoder.py", line 355, in raw_decode
          raise JSONDecodeError("Expecting value", s, err.value) from None
      json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

because windows fails to recognize the ocrd-tool.json symlink. I've simply overwritten the symlink with the actual file and eynollah now works in the command line! (at least the --help command does work)

because windows fails to recognize the ocrd-tool.json symlink. I've simply overwritten the symlink with the actual file and eynollah now works in the command line! (at least the --help command does work)

Ah, right, Windows does not like those. And yes, that would have been my recommended workaround. The symlink is only relevant at build time, once it has been installed, no symlinks are involved.
@mikegerber Would it be possible to extend setuptools-ocrd to dereference the symlink OS-independently at build time? Or to look for the ocrd-tool.json in the package directory directly?

@mikegerber Would it be possible to extend setuptools-ocrd to dereference the symlink OS-independently at build time?

That's a bit of a hack :\ How does the top-level ocrd-tool.json look like in Windows? If it's easy to do, I'll consider it.

Or to look for the ocrd-tool.json in the package directory directly?

Then it would need to find the package directory first :\
Note that I do most of my work in WSL and I would 100% recommend using that over native Windows.

@lmarti-dev What git did you use here? Or is it included in the Python install from e.g. the Microsoft Store?
(Not my platform and I need some more information to maybe fix this in setuptools-ocrd (the component that fails here))

(This was already an issue with setup.py as I understand it.)

@mikegerber Would it be possible to extend setuptools-ocrd to dereference the symlink OS-independently at build time?

That's a bit of a hack :\ How does the top-level ocrd-tool.json look like in Windows? If it's easy to do, I'll consider it.

In a checkout using the official Windows version of git, the file looks like this:
C:\Users\b-mg106\Downloads\eynollah>more ocrd-tool.json
src/eynollah/ocrd-tool.json

At least this looks easy enough to resolve "by hand".

Hi @lmarti-dev and thank you for your interest in Eynollah!
I am sorry but we can not support Windows (or Mac) and even if the build and executing --help work, there may still occur other issues further down the line when running Eynollah. As mentioned by @mikegerber, it works nicely in WSL which we strongly recommend to use, alternatively Docker should also do the trick.
I'll leave this open for a couple of days more for feedback and otherwise hope for your understanding when I close this issue.


Initial setup is not user-friendly

@caleywoods mentioned that one step in the setup can be problematic for new users: specifying the global admins (which will be hardcoded in config.json). Currently, this is done by asking the user to enter a JSON array of user IDs, but figuring out one's user ID is not very simple.
Here's an idea for solving the problem: when the user reaches that point, they can either choose an "expert mode" where they enter the list of administrators by hand or a "guided setup" where all they have to do is send a message to the bot. Under the hood, the setup process would spawn a temporary bot whose only role is to collect the user ID and pass it back to the setup process.

This is being fixed in #148.