Dynamic indexing into a StorageBuffer's contents won't compile

Expected Behaviour
The shader to compile with dynamic indexing into a StorageBuffer's contents
Example & Steps To Reproduce
Source

#![cfg_attr(target_arch = "spirv", no_std)]
#![feature(lang_items)]
#![feature(register_attr)]
#![register_attr(spirv)]

use spirv_std::glam::Vec4;
use spirv_std::{Output, StorageBuffer};

#[derive(Copy, Clone)]
pub struct DataBuffer {
    pub data: [i16;6935*3351],
}

#[allow(unused_attributes)]
#[spirv(fragment)]
pub fn main_fs(
  mut output: Output<Vec4>,
  #[spirv(descriptor_set = 1, binding = 0)] storage_buffer: StorageBuffer<DataBuffer>,
) {
    let data = storage_buffer.load();

    let x = data.data[0] as usize;
    let v = data.data[x] as f32;
    output.store(Vec4::new(1.0, v, v, 1.0))

}

#[cfg(all(not(test), target_arch = "spirv"))]
#[panic_handler]
fn panic(_: &core::panic::PanicInfo) -> ! {
    loop {}
}

#[cfg(all(not(test), target_arch = "spirv"))]
#[lang = "eh_personality"]
extern "C" fn rust_eh_personality() {}

I've tried lots of variations of this, but I think this is the clearest. It produces 3 errors and gives up, but none of the errors seem to make sense for this code:
  error: constant runtime array value

  error: Cannot use this pointer directly, it must be dereferenced first

  error: Cannot use this pointer directly, it must be dereferenced first
    --> src/lib.rs:23:13
     |
  23 |     let v = data.data[x] as f32;
     |             ^^^^^^^^^^^^

  error: aborting due to 3 previous errors

I know this is probably a known limitation, but I didn't find reference to it so I thought I'd mention it.
System Info

Rust: rustc 1.50.0-nightly (98d66340d 2020-11-14)
OS: macOS 10.14.6
GPU: Radeon Pro 555 2
SPIR-V: ? not sure but I used the setup script provided by rust-gpu only a few days ago

Backtrace

Backtrace

  --- stderr
      Updating crates.io index
     Compiling core v0.0.0 (/Users/isaac/.rustup/toolchains/nightly-2020-11-15-x86_64-apple-darwin/lib/rustlib/src/rust/library/core)
     Compiling rustc-std-workspace-core v1.99.0 (/Users/isaac/.rustup/toolchains/nightly-2020-11-15-x86_64-apple-darwin/lib/rustlib/src/rust/library/rustc-std-workspace-core)
     Compiling compiler_builtins v0.1.36 (/Users/isaac/.rustup/toolchains/nightly-2020-11-15-x86_64-apple-darwin/lib/rustlib/src/rust/vendor/compiler_builtins)
     Compiling glam v0.10.0 (https://github.com/EmbarkStudios/glam-rs?rev=c9561e4dfd55fa5a9d6838cae3c9e90c8edafaf9#c9561e4d)
     Compiling spirv-std v0.1.0 (/Users/isaac/code/rust-gpu/crates/spirv-std)
     Compiling shader v0.1.0 (/Users/isaac/code/geo-mapper/src/shader)
  error: constant runtime array value

  error: Cannot use this pointer directly, it must be dereferenced first

  error: Cannot use this pointer directly, it must be dereferenced first
    --> src/lib.rs:37:13
     |
  37 |     let v = data.data[x] as f32;
     |             ^^^^^^^^^^^^

  error: aborting due to 3 previous errors

  error: could not compile `shader`

There is this section in rustc_codegen_spirv/codegen_cx/constant.rs:
SpirvType::RuntimeArray { element } => {
    let mut values = Vec::new();
    while offset.bytes_usize() != alloc.len() {
        values.push(
            self.create_const_alloc2(alloc, offset, element)
                .def_cx(self),
        );
    }
    let result = self.constant_composite(ty, values);
    // TODO: Figure out how to do this. Compiling the below crashes both clspv *and* llvm-spirv:
    /*
    __constant struct A {
        float x;
        int y[];
    } a = {1, {2, 3, 4}};

    __kernel void foo(__global int* data, __constant int* c) {
    __constant struct A* asdf = &a;
    *data = *c + asdf->y[*c];
    }
    */
    self.zombie_no_span(result.def_cx(self), "constant runtime array value");
    result
}

The comment seems to indicate that constant pointers cause an issue, but suggests it applies to kernel mode. Perhaps this can be fixed, at least amended to allow this in "shader" mode? I tested it without the zombie and all the tests passed, but I don't know if that's valid.

I do not believe it is valid to create a OpTypeRuntimeArray via OpConstantComposite - further, I don't believe there is any way to create an OpTypeRuntimeArray in user code, all values of the type must come externally. If I'm mistaken, great, but I believe that removing that zombie is incorrect.
Also, kernel mode is generally far more permissive than shader mode, so if something doesn't work in kernel mode, it's highly unlikely to work in shader mode.

I was a bit unsure if I should create this, because one fairly likely explanation is that I was just doing something that's impossible. @khyperia I'm not quite sure if that's what you're saying or not.
I think part of what confused me about this error message though was it wasn't obvious to me why a RuntimeArray was being generated. All the arrays I see declared have sizes and the only way I could see these getting generated was for unsized arrays

No, filing the issue was the right thing, thanks for filing it! I was more responding to @charles-r-earp than the original issue.
And yeah, same, I'm not sure why a RuntimeArray is being generated. Hopefully someone has time to look into this soon.

I did some more looking into this. I think then constant runtime array value is coming from an attempt to compile core::panic::Location which contains a &str. I switched to #278 because I was happing the spans would help (they didn't here) and added some print statements shown below
diff --git a/crates/rustc_codegen_spirv/src/codegen_cx/constant.rs b/crates/rustc_codegen_spirv/src/codegen_cx/constant.rs
index ae6f9773a..4b6a72f08 100644
--- a/crates/rustc_codegen_spirv/src/codegen_cx/constant.rs
+++ b/crates/rustc_codegen_spirv/src/codegen_cx/constant.rs
@@ -263,6 +263,11 @@ impl<'tcx> ConstMethods<'tcx> for CodegenCx<'tcx> {
                                 other.debug(ty, self)
                             )),
                         };
+                        let pointee_type = self.lookup_type(pointee);
+                        if let SpirvType::RuntimeArray{ element } = pointee_type {
+                            let element_type = self.lookup_type(element);
+                            println!("constant pointer to RuntimeArray<{:?}>", element_type);
+                        }
                         let init = self.create_const_alloc(alloc, pointee);
                         let value = self.static_addr_of(init, alloc.align, None);
                         (value, AddressSpace::DATA)
@@ -394,11 +399,13 @@ impl<'tcx> CodegenCx<'tcx> {
                 size,
                 field_types,
                 field_offsets,
+                name,
                 ..
             } => {
                 let base = *offset;
                 let mut values = Vec::with_capacity(field_types.len());
                 let mut occupied_spaces = Vec::with_capacity(field_types.len());
+                println!("constant adt {}", name);
                 for (&ty, &field_offset) in field_types.iter().zip(field_offsets.iter()) {
                     let total_offset_start = base + field_offset;
                     let mut total_offset_end = total_offset_start;
diff --git a/crates/rustc_codegen_spirv/src/codegen_cx/mod.rs b/crates/rustc_codegen_spirv/src/codegen_cx/mod.rs
index 92856ab7f..0d5c96a43 100644
--- a/crates/rustc_codegen_spirv/src/codegen_cx/mod.rs
+++ b/crates/rustc_codegen_spirv/src/codegen_cx/mod.rs
@@ -157,6 +157,7 @@ impl<'tcx> CodegenCx<'tcx> {
         if self.is_system_crate() {
             self.zombie_values.borrow_mut().insert(word, reason);
         } else {
+            println!("non system zombie {}", reason);
             self.tcx.sess.err(reason);
         }
     }

This was after adding a backtrace to zombie_no_span so I knew there were interesting methods in the stacktrace.
I got, well a bunch of things, but the last 4 lines were
  constant adt core::panic::Location
  constant adt &str
  constant pointer to RuntimeArray<Integer(8, false)>
  non system zombie constant runtime array value

I sort of wonder if this might be what's going on with #211 too, but with basically no evidence

I traced down at least one of the Cannot use this pointer directly, it must be dereferenced first to be about this too since it looks like the thing it's trying to deference points to a core::panic::Location.

Right, then this is a partial dupe of https://github.com/EmbarkStudios/rust-gpu/issues/186#issuecomment-720416686, strings and therefore panics aren't supported yet. (I'm guessing the panic is happening from generating out-of-bounds-index error path)

I think this is resolved as of #305 though copying the entire buffer into the shader before indexing into it cause other issues which I ended up resolving by adding a custom storage_buffer wrapper class like so
#[allow(unused_attributes)]
#[spirv(storage_buffer)]
pub struct StorageBufferArray<'value> {
    value: &'value mut DataBuffer,
}
impl StorageBufferArray<'_> {
    #[allow(unused_attributes)]
    #[spirv(really_unsafe_ignore_bitcasts)]
    fn get(&mut self, index: usize) -> i16 {
        let mut result: i16 = 10;
        let zero_index = 0;
        unsafe {
            asm!(
                "%result_type = OpTypePointer StorageBuffer typeof*{result_ptr}",
                "%result_prt = OpAccessChain %result_type {base} {zero} {index}",
                "OpCopyMemory {result_ptr} %result_prt",
                base = in(reg) self.value,
                zero = in(reg) zero_index,
                index = in(reg) index,
                result_ptr = in(reg) &mut result
            );

            result
        }
    }
}

Come to think of it I'm not sure that I tested just doing self.value[index] rather than the complicated business with asm! in this version of the compiler


Bad code generated for byte comparison

This prints false when it should print true.
import ballerina/io;

public function main() {
    io:println(testByte(0x20));
}

function testByte(byte b) returns boolean {
    if b >= 0x20 {
       return true;
    }
    return false;
}

Changing byte to int or changing 0x20 to decimal both make the problem go away.

@warunalakshitha Can we add this to the current sprint?

Had a quick check. There is a bug in codegen specific to byte. We plan to revisit codegen and test all byte specific operations since we found multiple issues related to byte operations.
@hasithaa since today is last date of sprint we will add as high priority item in next sprint. BTW @Nadeeshan96 will work on this.

If it’s a bug in codegen, why does changing 0x20 to 32 avoid the bug?

In BIR, Operand type for '0x20' is byte but for '32' is int. During codegen we specifically handle byte type. As an example following code will fail too :(.
public function main() {
    byte a =32;
    byte b = 32;
    io:println(a >= b);
}

Sounds like the BIR is wrong then. Whether a integer constant is decimal or hexadecimal shouldn't affect whether it is treated as a byte. Any integer in the range 0 to 255 belongs to the byte type regardless of whether it is written in decimal or hex.

Sounds like the BIR is wrong then. Whether a integer constant is decimal or hexadecimal shouldn't affect whether it is treated as a byte. Any integer in the range 0 to 255 belongs to the byte type regardless of whether it is written in decimal or hex.

Created a new issue #31750 to handle this.


Jira ghae register

This has been open a long time without updates. Please raise again if needed 👍


gyp: update gyp to 0.6.2

Refs: https://github.com/nodejs/gyp-next/releases/tag/v0.6.2

Checklist

[ ] npm install && npm test passes
[ ] tests are included
[ ] documentation is changed or added
[ ] commit message follows commit guidelines

Description of change

@targos macos 14.x with 3.8 is consistenly failing with a timeout. Any idea why?

ugh, I can repro on macos for Node 10 but using master, so it's not this PR. @MylesBorins that probably means you have a buggy macOS + Node 10 for npm 7 at the moment. Would you rather us get a node-gyp out to deal with the windows problem and deal with this one separately, or attempt to get them both together? I don't mind shipping a node-gyp with just this update and then try and resolve the other problem separately (hopefully ASAP!).

I wasn't experiencing locally just noticed failing CI.
npm has already floated and shipped the gyp 0.6.2 update to fix windows so I think it is fine to ship what we have here and try and fast follow the Mac fix

bah, it was the Catalina problem I was encountering, now having to run the macOS_Catalina.md steps to rectify (first time I've experienced this tbh).

ok, I can't reproduce locally, I'm going to merge and release


Reolink E1 not responding

Hello
I think ive compiled FFmpeg to be able to handle aac. I visited this link and followed all instructions: https://trac.ffmpeg.org/wiki/CompilationGuide/macOS
PLEASE HELP IT FEELS LIKE IM NEARLY THERE !!! @KhaosT
I now have this log for trying to access my Reolink E1
[2/2/2020, 9:30:56 AM] [Config] [HapClient] [<IP_ADDRESS>:51826 (CC:22:3D:E3:CE:30)] Disconnected
[2/2/2020, 9:30:56 AM] [Config] [HapClient] [<IP_ADDRESS>:55588 (DF:CD:15:22:7B:7A)] Disconnected
[2/2/2020, 9:31:07 AM] [Config] Changes to config.json saved.
[2/2/2020, 9:31:09 AM] [Config] Homebridge restart request received
[2/2/2020, 9:31:09 AM] [Config] No restart command defined, killing process...
[2/2/2020, 9:31:09 AM] [HB Supervisor] Stopping services...
[2/2/2020, 9:31:09 AM] Got SIGTERM, shutting down Homebridge...
[2/2/2020, 9:31:14 AM] [HB Supervisor] Homebridge Process Ended. Code: 143, Signal: null
[2/2/2020, 9:31:15 AM] [HB Supervisor] Homebridge Storage Path: /Users/leighhounsome/.homebridge
[2/2/2020, 9:31:15 AM] [HB Supervisor] Homebridge Config Path: /Users/leighhounsome/.homebridge/config.json
[2/2/2020, 9:31:15 AM] [HB Supervisor] Logging to /Users/leighhounsome/.homebridge/homebridge.log
[2/2/2020, 9:31:15 AM] [HB Supervisor] Homebridge Path: /usr/local/lib/node_modules/homebridge/bin/homebridge
[2/2/2020, 9:31:15 AM] [HB Supervisor] UI Path: /usr/local/lib/node_modules/homebridge-config-ui-x/dist/bin/standalone.js
[2/2/2020, 9:31:15 AM] [HB Supervisor] Started Homebridge with PID: 20866
[2/2/2020, 9:31:16 AM] Loaded config.json with 0 accessories and 3 platforms.
[2/2/2020, 9:31:16 AM] ---
[2/2/2020, 9:31:17 AM] [Config] [HapClient] Discovery :: Started
[2/2/2020, 9:31:17 AM] [Config] Homebridge Config UI X v4.9.0 is listening on :: port 8581
[2/2/2020, 9:31:17 AM] Loaded plugin: homebridge-camera-ffmpeg
[2/2/2020, 9:31:17 AM] Registering platform 'homebridge-camera-ffmpeg.Camera-ffmpeg'
[2/2/2020, 9:31:17 AM] ---
[2/2/2020, 9:31:17 AM] Loaded plugin: homebridge-config-ui-x
[2/2/2020, 9:31:17 AM] Registering platform 'homebridge-config-ui-x.config'
[2/2/2020, 9:31:17 AM] ---
[2/2/2020, 9:31:17 AM] Loaded plugin: homebridge-tuya-web
[2/2/2020, 9:31:17 AM] Registering platform 'homebridge-tuya-web.TuyaWebPlatform'
[2/2/2020, 9:31:17 AM] ---
[2/2/2020, 9:31:17 AM] Loading 3 platforms...
[2/2/2020, 9:31:17 AM] [TuyaWebPlatform] Initializing TuyaWebPlatform platform...
[2/2/2020, 9:31:17 AM] [Config] Initializing config platform...
[2/2/2020, 9:31:17 AM] [Config] Running in Standalone Mode.
[2/2/2020, 9:31:17 AM] [Camera ffmpeg] Initializing Camera-ffmpeg platform...
[2/2/2020, 9:31:17 AM] Loading 0 accessories...
[2/2/2020, 9:31:17 AM] [TuyaWebPlatform] Configuring cached accessory [Jens Desk] bf4cd96cb140b86263o6s6 e1af25f1-6755-4541-95f8-d5fc871463da
[2/2/2020, 9:31:17 AM] [TuyaWebPlatform] Configuring cached accessory [Lamp] bf6235a725da3bd972knma 0e174c25-1f0a-446f-88ce-6837e5c5d121
[2/2/2020, 9:31:17 AM] [TuyaWebPlatform] Configuring cached accessory [Lights] 107200463c71bf3da227 c0c518dc-3780-4db3-95de-238e7952141d
Setup Payload:
X-HM://0023ISYWYDP9O
Enter this code with your HomeKit app on your iOS device to pair with Homebridge:
┌────────────┐
│ 031-45-154 │
└────────────┘

[2/2/2020, 9:31:17 AM] [TuyaWebPlatform] Initializing TuyaWebPlatform...
[2/2/2020, 9:31:17 AM] Homebridge is running on port 51826.
[2/2/2020, 9:31:17 AM] TestCam1 is running on port 56474.
[2/2/2020, 9:31:17 AM] Please add [TestCam1] manually in Home app. Setup Code: 031-45-154
[2/2/2020, 9:31:17 AM] [TuyaWebPlatform] Adding: Jens Desk (switch / bf4cd96cb140b86263o6s6)
[2/2/2020, 9:31:17 AM] [TuyaWebPlatform] Existing Accessory found [Jens Desk] [bf4cd96cb140b86263o6s6] [e1af25f1-6755-4541-95f8-d5fc871463da]
[2/2/2020, 9:31:17 AM] [TuyaWebPlatform] Adding: Lamp (switch / bf6235a725da3bd972knma)
[2/2/2020, 9:31:17 AM] [TuyaWebPlatform] Existing Accessory found [Lamp] [bf6235a725da3bd972knma] [0e174c25-1f0a-446f-88ce-6837e5c5d121]
[2/2/2020, 9:31:17 AM] [TuyaWebPlatform] Adding: Lights (switch / 107200463c71bf3da227)
[2/2/2020, 9:31:17 AM] [TuyaWebPlatform] Existing Accessory found [Lights] [107200463c71bf3da227] [c0c518dc-3780-4db3-95de-238e7952141d]
[2/2/2020, 9:31:18 AM] [Config] [HapClient] Cannot load accessories. No Homebridge instances have been discovered.
[2/2/2020, 9:31:18 AM] [Config] [HapClient] Cannot load accessories. No Homebridge instances have been discovered.
[2/2/2020, 9:31:18 AM] [Config] [HapClient] Discovery :: Re-broadcasting discovery query
[2/2/2020, 9:31:18 AM] [Config] [HapClient] Discovery :: Found HAP device with username DF:CD:15:22:7B:7A
[2/2/2020, 9:31:18 AM] [Config] [HapClient] Discovery :: Testing DF:CD:15:22:7B:7A via http://<IP_ADDRESS>:56474/accessories
[2/2/2020, 9:31:18 AM] [Config] [HapClient] Discovery :: Found HAP device with username CC:22:3D:E3:CE:30
[2/2/2020, 9:31:18 AM] [Config] [HapClient] Discovery :: Testing CC:22:3D:E3:CE:30 via http://<IP_ADDRESS>:51826/accessories
[2/2/2020, 9:31:18 AM] [Config] [HapClient] Discovery :: Success CC:22:3D:E3:CE:30 via http://<IP_ADDRESS>:51826/accessories
[2/2/2020, 9:31:18 AM] [Config] [HapClient] Discovery :: [<IP_ADDRESS>:51826 (CC:22:3D:E3:CE:30)] Instance Registered
[2/2/2020, 9:31:18 AM] [Config] [HapClient] Discovery :: Success DF:CD:15:22:7B:7A via http://<IP_ADDRESS>:56474/accessories
[2/2/2020, 9:31:18 AM] [Config] [HapClient] Discovery :: [<IP_ADDRESS>:56474 (DF:CD:15:22:7B:7A)] Instance Registered
[2/2/2020, 9:31:19 AM] [Config] [HapClient] [<IP_ADDRESS>:51826 (CC:22:3D:E3:CE:30)] Connected
[2/2/2020, 9:31:19 AM] [Config] [HapClient] [<IP_ADDRESS>:56474 (DF:CD:15:22:7B:7A)] Connected
[2/2/2020, 9:31:19 AM] [Config] [HapClient] Discovery :: Re-broadcasting discovery query
[2/2/2020, 9:31:37 AM] [Camera ffmpeg] Snapshot from TestCam1 at 480x270
ffmpeg -i http://<IP_ADDRESS>:80/cgi-bin/api.cgi?cmd=Snap&channel=0&user=admin&password=PASSWORD -t 1 -s 480x270 -f image2 -
[2/2/2020, 9:31:43 AM] [Camera ffmpeg] Snapshot from TestCam1 at 480x270
ffmpeg -i http://<IP_ADDRESS>:80/cgi-bin/api.cgi?cmd=Snap&channel=0&user=admin&password=PASSWORD -t 1 -s 480x270 -f image2 -
[2/2/2020, 9:31:44 AM] [Camera ffmpeg] Start streaming video from TestCam1 with 1280x720@299kBit
ffmpeg -re -i rtsp://admin:PASSWORD@<IP_ADDRESS>:554/h264Preview_01_main -map 0:0 -vcodec libfdk_aac -pix_fmt yuv420p -r 10 -f rawvideo -tune zerolatency -b:v 299k -bufsize 299k -maxrate 299k -payload_type 99 -ssrc 13630497 -f rtp -srtp_out_suite AES_CM_128_HMAC_SHA1_80 -srtp_out_params gVRK3snNK665lR+FcO2c1Wc8ArSmVV35wIO4UbOd srtp://<IP_ADDRESS>:52179?rtcpport=52179&localrtcpport=52179&pkt_size=1316 -map 0:1 -acodec libfdk_aac -profile:a aac_eld -flags +global_header -f null -ar 16k -b:a 24k -bufsize 24k -ac 1 -payload_type 110 -ssrc 11571864 -f rtp -srtp_out_suite AES_CM_128_HMAC_SHA1_80 -srtp_out_params p2DRNPHF4QI6pY2cbX9ppbGxsL0h2ckK/WaGDnu0 srtp://<IP_ADDRESS>:56024?rtcpport=56024&localrtcpport=56024&pkt_size=1316 -loglevel debug
[2/2/2020, 9:31:44 AM] [Config] [HapClient] [<IP_ADDRESS>:56474 (DF:CD:15:22:7B:7A)] Got Event: [{"aid":1,"iid":17,"value":"AQEB"}]
[2/2/2020, 9:31:44 AM] [Config] [HapClient] [<IP_ADDRESS>:56474 (DF:CD:15:22:7B:7A)] Got Event: [{"aid":1,"iid":17,"value":"AQEB"}]
ffmpeg version 4.2.2 Copyright (c) 2000-2019 the FFmpeg developers
built with Apple clang version 11.0.0 (clang-1<IP_ADDRESS>)
configuration: --prefix=/usr/local/Cellar/ffmpeg/4.2.2_1 --enable-shared --enable-pthreads --enable-version3 --enable-avresample --cc=clang --host-cflags='-I/Library/Java/JavaVirtualMachines/adoptopenjdk-13.0.1.jdk/Contents/Home/include -I/Library/Java/JavaVirtualMachines/adoptopenjdk-13.0.1.jdk/Contents/Home/include/darwin -fno-stack-check' --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libbluray --enable-libmp3lame --enable-libopus --enable-librubberband --enable-libsnappy --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librtmp --enable-libspeex --enable-libsoxr --enable-videotoolbox --disable-libjack --disable-indev=jack
libavutil      56. 31.100 / 56. 31.100
libavcodec     58. 54.100 / 58. 54.100
libavformat    58. 29.100 / 58. 29.100
libavdevice    58.  8.100 / 58.  8.100
libavfilter     7. 57.100 /  7. 57.100
libavresample   4.  0.  0 /  4.  0.  0
libswscale      5.  5.100 /  5.  5.100
libswresample   3.  5.100 /  3.  5.100
libpostproc    55.  5.100 / 55.  5.100
Splitting the commandline.
Reading option '-re' ... matched as option 're' (read input at native frame rate) with argument '1'.
Reading option '-i' ... matched as input url with argument 'rtsp://admin:OurHome173@<IP_ADDRESS>:554/h264Preview_01_main'.
Reading option '-map' ... matched as option 'map' (set input stream mapping) with argument '0:0'.
Reading option '-vcodec' ... matched as option 'vcodec' (force video codec ('copy' to copy stream)) with argument 'libfdk_aac'.
Reading option '-pix_fmt' ... matched as option 'pix_fmt' (set pixel format) with argument 'yuv420p'.
Reading option '-r' ... matched as option 'r' (set frame rate (Hz value, fraction or abbreviation)) with argument '10'.
Reading option '-f' ... matched as option 'f' (force format) with argument 'rawvideo'.
Reading option '-tune' ...
matched as AVOption 'tune' with argument 'zerolatency'.
Reading option '-b:v' ... matched as option 'b' (video bitrate (please use -b:v)) with argument '299k'.
Reading option '-bufsize' ... matched as AVOption 'bufsize' with argument '299k'.
Reading option '-maxrate' ... matched as AVOption 'maxrate' with argument '299k'.
Reading option '-payload_type' ...
matched as AVOption 'payload_type' with argument '99'.
Reading option '-ssrc' ...
matched as AVOption 'ssrc' with argument '13630497'.
Reading option '-f' ... matched as option 'f' (force format) with argument 'rtp'.
Reading option '-srtp_out_suite' ... matched as AVOption 'srtp_out_suite' with argument 'AES_CM_128_HMAC_SHA1_80'.
Reading option '-srtp_out_params' ... matched as AVOption 'srtp_out_params' with argument 'gVRK3snNK665lR+FcO2c1Wc8ArSmVV35wIO4UbOd'.
Reading option 'srtp://<IP_ADDRESS>:52179?rtcpport=52179&localrtcpport=52179&pkt_size=1316' ... matched as output url.
Reading option '-map' ... matched as option 'map' (set input stream mapping) with argument '0:1'.
Reading option '-acodec' ... matched as option 'acodec' (force audio codec ('copy' to copy stream)) with argument 'libfdk_aac'.
Reading option '-profile:a' ... matched as option 'profile' (set profile) with argument 'aac_eld'.
Reading option '-flags' ... matched as AVOption 'flags' with argument '+global_header'.
Reading option '-f' ... matched as option 'f' (force format) with argument 'null'.
Reading option '-ar' ... matched as option 'ar' (set audio sampling rate (in Hz)) with argument '16k'.
Reading option '-b:a' ... matched as option 'b' (video bitrate (please use -b:v)) with argument '24k'.
Reading option '-bufsize' ... matched as AVOption 'bufsize' with argument '24k'.
Reading option '-ac' ... matched as option 'ac' (set number of audio channels) with argument '1'.
Reading option '-payload_type' ... matched as AVOption 'payload_type' with argument '110'.
Reading option '-ssrc' ...
matched as AVOption 'ssrc' with argument '11571864'.
Reading option '-f' ... matched as option 'f' (force format) with argument 'rtp'.
Reading option '-srtp_out_suite' ...
matched as AVOption 'srtp_out_suite' with argument 'AES_CM_128_HMAC_SHA1_80'.
Reading option '-srtp_out_params' ...
matched as AVOption 'srtp_out_params' with argument 'p2DRNPHF4QI6pY2cbX9ppbGxsL0h2ckK/WaGDnu0'.
Reading option 'srtp://<IP_ADDRESS>:56024?rtcpport=56024&localrtcpport=56024&pkt_size=1316' ... matched as output url.
Reading option '-loglevel' ... matched as option 'loglevel' (set logging level) with argument 'debug'.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option loglevel (set logging level) with argument debug.
Successfully parsed a group of options.
Parsing a group of options: input url rtsp://admin:PASSWORD@<IP_ADDRESS>:554/h264Preview_01_main.
Applying option re (read input at native frame rate) with argument 1.
Successfully parsed a group of options.

Parsing a group of options: output url srtp://<IP_ADDRESS>:52179?rtcpport=52179&localrtcpport=52179&pkt_size=1316.
Applying option map (set input stream mapping) with argument 0:0.
Applying option vcodec (force video codec ('copy' to copy stream)) with argument libfdk_aac.
Applying option pix_fmt (set pixel format) with argument yuv420p.
Applying option r (set frame rate (Hz value, fraction or abbreviation)) with argument 10.
Applying option f (force format) with argument rawvideo.
Applying option b:v (video bitrate (please use -b:v)) with argument 299k.
Applying option f (force format) with argument rtp.
Successfully parsed a group of options.
Opening an output file: srtp://<IP_ADDRESS>:52179?rtcpport=52179&localrtcpport=52179&pkt_size=1316.
Unknown encoder 'libfdk_aac'
[2/2/2020, 9:31:48 AM] [Camera ffmpeg] ERROR: FFmpeg exited with code 1
[2/2/2020, 9:31:48 AM] [Config] [HapClient] [<IP_ADDRESS>:56474 (DF:CD:15:22:7B:7A)] Got Event: [{"aid":1,"iid":17,"value":"AQEA"}]
[2/2/2020, 9:32:17 AM] [Config] [HapClient] Discovery :: Ended
Originally posted by @leesoyavh in https://github.com/KhaosT/homebridge-camera-ffmpeg/issues/421#issuecomment-581117466

Reolink E1 (not PRO) - it's trash. E1 - don't transmit streaming video, does not work in the browser. TRASH.

I sent E1 back and upgraded to E1 Pro
Have you got E1 Pro and did you get it to work ?

FYI, I have a E1 Zoom running same params as 410

Hi there @iuliusmaximilian
What are the params for 410?

see https://github.com/KhaosT/homebridge-camera-ffmpeg/wiki/Tested-Configurations and my latest comment of the bottom of the page

Yeah, the E1 doesn't support RTSP :/ I'm probably going to send mine back now too.


[DOC] Add Document for min_rollover_age

What do you want to do?

[ ] Request a change to existing documentation
[x] Add new documentation
[ ] Report a technical problem with the documentation
[ ] Other

Tell us about your request.
ISM team discovers the documents of min_rollover_age has not been added. One customer is going to use it.
Do you have any related resources to help us get started?
https://github.com/opensearch-project/index-management/pull/215

@Angie-Zhang: Are you ok being the Subject Matter Expert on this one?

Fixed by #1751


Refactor bigbrain livequery to streamline results and data sources

Pull profile data from EBRAINS https://search.kg.ebrains.eu/instances/26d25994-634c-40af-b88f-2a36e8e1d508,
If a higher level region is queried, choose a joint space while prioritizing BigBrain space
Pull mesh data from bigbrain ftp, https://ftp.bigbrainproject.org/bigbrain-ftp/BigBrainRelease.2015/3D_Surfaces/Apr7_2016/gii/

TODO:

[ ] Determine the thickness files for left and right hemispheres. Potentially, calculate from the mesh data (but this might not be the same as how the authors did it, i.e., they might have used equivolumetric distance calculation)
[ ] https://github.com/FZJ-INM1-BDA/siibra-python/issues/295

Codecov Report

Merging #456 (fbe8298) into main (5a0e6b3) will increase coverage by 0.67%.
Report is 99 commits behind head on main.
The diff coverage is 26.54%.

:exclamation: Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the GitHub App Integration for your organization. Read more.
@@            Coverage Diff             @@
##             main     #456      +/-   ##
==========================================
+ Coverage   36.81%   37.49%   +0.67%
==========================================
  Files          61       61
  Lines        5421     5558     +137
==========================================
+ Hits         1996     2084      +88
- Misses       3425     3474      +49

Files Changed
Coverage Δ

siibra/configuration/configuration.py
42.35% <ø> (ø)

siibra/configuration/factory.py
22.22% <0.00%> (+0.24%)
:arrow_up:

siibra/core/atlas.py
85.71% <ø> (+1.23%)
:arrow_up:

...a/features/connectivity/functional_connectivity.py
55.55% <0.00%> (ø)

siibra/livequeries/ebrains.py
25.35% <0.00%> (+0.35%)
:arrow_up:

siibra/locations/boundingbox.py
19.88% <ø> (ø)

siibra/volumes/neuroglancer.py
21.42% <0.00%> (ø)

siibra/volumes/volume.py
50.29% <0.00%> (ø)

siibra/features/tabular/cortical_profile.py
26.43% <3.70%> (-4.23%)
:arrow_down:

siibra/features/tabular/tabular.py
23.68% <3.70%> (-6.32%)
:arrow_down:

... and 24 more

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


helm: expose enable-certificate-owner-ref and -dns01-recursive-nameservers as helm value

Pull Request Motivation
extraArgs is currently a list of args. In helm, lists are not mergable. Currently, our teams is providing a default helm chart with our internal best practice cert-manager helm values.
Product teams are not able to extend extraArgs of cert-manage, instead they have to override the whole extraArgs value.
By exposing the three parameters, our teams are able to toggle some certain features.
An absolute fix would be that extraArgs is a map instead a list which is a breaking change.
Kind

feature
Release Note

[helm] expose enable-certificate-owner-ref and dns01-recursive-nameservers as helm value

/assign @irbekrm

hey @SgtCoDFish, are you able to take a look on this?

Hey! This PR totally passed me by!
I think this makes sense (especially when our own tests would make use of it, as you've shown in this PR!).
Unfortunately this just missed the cutoff for inclusion in cert-manager 1.11, but IMO it should be included in 1.12.
/ok-to-test

Rebased as requested

/test pull-cert-manager-master-make-test
/test pull-cert-manager-master-e2e-v1-26
/test pull-cert-manager-master-e2e-v1-26-upgrade
These look like strange test failures - could be related, could be flakes

/kind feature


Update to pre-release givenergy-modbus

The current v0.x release contains a growing list of issues and is unsupported. There is a new version in the works, but it has not been released yet.
This integration could be updated to use the pre-release code.
Advantages:

Improved reliability
Compatibility with a wider range of inverters

Disadvantages:

Interfaces may change
Copying & pasting a library in to another codebase 🤮

@alfwro13 If the fix worked for GivTCP, it would work here too

There is now a branch with WIP on this issue.
This takes the newer version of givenergy_modbus that was due to become v1.0 - more or less a complete rewrite compared to the original. Sadly it was never quite finished or released, but I've grabbed the source and attempted to fill some of the few remaining gaps to make it work with the integration.
I've included the CRC calculation fix for reading inverter data, but there's still a chance it won't be able to write new values - more work required there.
This successfully reads and writes values on my inverter with the old firmware. If someone could give this a run against some newer firmware, I'd be interested to hear whether it works at all. This is available through HACS by enabling beta releases on the download page. Look for v2.0.0-alpha1.

Great stuff, thanks got getting to work on this so quickly 👍
I'll try and install this tomorrow and let you know how I get on...

So I've upgraded the integration to the alpha version, and restarted, then reloaded the integration for good measure.
Unfortunately it seems we're not able to read from the inverter on the new firmware (at least in my case).
With debug logging turned on for the integration, these are the logs following a reload (of the integration):
tail -f /home/homeassistant/.homeassistant/home-assistant.log | grep givenergy

2023-12-14 11:54:04.710 INFO (SyncWorker_36) [custom_components.givenergy_local.coordinator] Fetching data from <IP_ADDRESS>
2023-12-14 11:54:04.710 DEBUG (SyncWorker_36) [custom_components.givenergy_local.coordinator] Performing full refresh
2023-12-14 11:54:06.717 ERROR (SyncWorker_36) [givenergy_modbus] Transaction failed
2023-12-14 11:54:06.719 ERROR (SyncWorker_36) [givenergy_modbus] Modbus Error: [Input/Output] Modbus Error: [Invalid Message] No response received, expected at least 164 bytes (0 received)
2023-12-14 11:54:06.720 ERROR (SyncWorker_36) [givenergy_modbus] Did not receive expected response type: ReadInputRegistersResponse != ModbusIOException
2023-12-14 11:54:09.228 ERROR (SyncWorker_36) [givenergy_modbus] Modbus Error: [Input/Output] No Response received from the remote unit/Unable to decode response
2023-12-14 11:54:09.228 ERROR (SyncWorker_36) [givenergy_modbus] Did not receive expected response type: ReadInputRegistersResponse != ModbusIOException
2023-12-14 11:54:11.736 ERROR (SyncWorker_36) [givenergy_modbus] Modbus Error: [Input/Output] No Response received from the remote unit/Unable to decode response
2023-12-14 11:54:11.736 ERROR (SyncWorker_36) [givenergy_modbus] Did not receive expected response type: ReadHoldingRegistersResponse != ModbusIOException
2023-12-14 11:54:14.245 ERROR (SyncWorker_36) [givenergy_modbus] Modbus Error: [Input/Output] No Response received from the remote unit/Unable to decode response
2023-12-14 11:54:14.245 ERROR (SyncWorker_36) [givenergy_modbus] Did not receive expected response type: ReadHoldingRegistersResponse != ModbusIOException
2023-12-14 11:54:14.711 ERROR (MainThread) [custom_components.givenergy_local.coordinator] Error fetching Inverter data: Error communicating with API:
2023-12-14 11:54:14.711 DEBUG (MainThread) [custom_components.givenergy_local.coordinator] Finished fetching Inverter data in 10.002 seconds (success: False)
2023-12-14 11:54:16.755 ERROR (SyncWorker_36) [givenergy_modbus] Modbus Error: [Input/Output] No Response received from the remote unit/Unable to decode response
2023-12-14 11:54:16.756 ERROR (SyncWorker_36) [givenergy_modbus] Did not receive expected response type: ReadHoldingRegistersResponse != ModbusIOException
2023-12-14 11:54:19.261 ERROR (SyncWorker_36) [givenergy_modbus] Modbus Error: [Input/Output] No Response received from the remote unit/Unable to decode response
2023-12-14 11:54:19.262 ERROR (SyncWorker_36) [givenergy_modbus] Did not receive expected response type: ReadInputRegistersResponse != ModbusIOException

Please let me know if there is anything else that would help 👍

Ah, that would be because I messed up the release, so you were actually still running 1.9.0.
I've now published v2.0.0-alpha2.

Great! I've updated however I get a new error, perhaps just a typo?
Logger: homeassistant.setup
Source: setup.py:251
First occurred: 23:42:27 (1 occurrences)
Last logged: 23:42:27

Setup failed for custom integration 'givenergy_local': Unable to import component: No module named 'givenergy_modbus.client.client'; 'givenergy_modbus.client' is not a package
Traceback (most recent call last):
  File "/srv/homeassistant/lib/python3.11/site-packages/homeassistant/setup.py", line 251, in _async_setup_component
    component = integration.get_component()
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/srv/homeassistant/lib/python3.11/site-packages/homeassistant/loader.py", line 814, in get_component
    ComponentProtocol, importlib.import_module(self.pkg_path)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1206, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1178, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1149, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/homeassistant/.homeassistant/custom_components/givenergy_local/__init__.py", line 10, in <module>
    from .coordinator import GivEnergyUpdateCoordinator
  File "/home/homeassistant/.homeassistant/custom_components/givenergy_local/coordinator.py", line 8, in <module>
    from givenergy_modbus.client.client import Client
ModuleNotFoundError: No module named 'givenergy_modbus.client.client'; 'givenergy_modbus.client' is not a package

Great! I've updated however I get a new error, perhaps just a typo?
Logger: homeassistant.setup
Source: setup.py:251
First occurred: 23:42:27 (1 occurrences)
Last logged: 23:42:27

Setup failed for custom integration 'givenergy_local': Unable to import component: No module named 'givenergy_modbus.client.client'; 'givenergy_modbus.client' is not a package
Traceback (most recent call last):
  File "/srv/homeassistant/lib/python3.11/site-packages/homeassistant/setup.py", line 251, in _async_setup_component
    component = integration.get_component()
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/srv/homeassistant/lib/python3.11/site-packages/homeassistant/loader.py", line 814, in get_component
    ComponentProtocol, importlib.import_module(self.pkg_path)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1206, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1178, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1149, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/homeassistant/.homeassistant/custom_components/givenergy_local/__init__.py", line 10, in <module>
    from .coordinator import GivEnergyUpdateCoordinator
  File "/home/homeassistant/.homeassistant/custom_components/givenergy_local/coordinator.py", line 8, in <module>
    from givenergy_modbus.client.client import Client
ModuleNotFoundError: No module named 'givenergy_modbus.client.client'; 'givenergy_modbus.client' is not a package

Not a typo as far as I can tell, but something to do with how Python dependencies are defined and downloaded.
I was able to reproduce this by having 1.9.0 installed, then upgrading to 2.0.0-alpha2 via HACS. Removing and re-adding the integration seems to clear things up, but this shouldn't be required and will need addressing.
If you're keen, you may want to try a completely separate HA instance and install the new version directly in that. I've just been running up a Docker container for it, installing HACS, then installing the integration to play around.
In the next version I'll change the approach to avoid this problem.

Thanks, I've tried removing the integration, but it wants the config deleted first. I've tried that, and confirmed the deletion, but then - nothing - the config doesn't appear to be deleted so I can't remove the integration. Any thoughts?

Assuming we're on alpha2 and it's not behaving, the least disruptive thing to do is perhaps to force it on to the correct version of givenergy_modbus. You can confirm the version with:
pip list installed | grep giv

I found that, after the upgrade, it was stuck on 0.10.1. To install the correct version for alpha2, run:
pip install givenergy-modbus@git+https://github.com/cdpuk/givenergy-modbus@aa5af8f8038408ebec07bfcea834b6b4a8493f25

Then restart HA.
It's odd how this works for fresh installs but not upgrades.

It's a (partial) Christmas miracle 😁🎄
That has worked and (almost) all entities appear to be as they should 👍
However, I'm seeing some issues relating to control - so far I have discovered that:

The charge and discharge power limits have been changed to 2600 and can't be set back to 3600. I guess the integration thinks the inverter is Gen 1 for some reason?
If I adjust the booleans (eco mode, ac charging, dc discharging), this fails and they revert to their original setting.
If I adjust them from givenergy.cloud, they update in HA, but still cannot be controlled.
Adjusting the booleans appears to also cause the integration to lose contact with the inverter, in fact the app and givenergy.cloud also appear to lose contact at the same time. I'm not sure how consistent this is yet.

I've also redownloaded the alpha-2, and restarted Home Assistant several times.
In the logs I've seen the following:
Logger: homeassistant.components.websocket_api.http.connection
Source: components/websocket_api/commands.py:238
Integration: Home Assistant WebSocket API ([documentation](https://www.home-assistant.io/integrations/websocket_api), [issues](https://github.com/home-assistant/core/issues?q=is%3Aissue+is%3Aopen+label%3A%22integration%3A+websocket_api%22))
First occurred: 11:06:40 (4 occurrences)
Last logged: 11:08:42

[140516901194560] Value 3600.0 for number.battery_charge_power_limit is outside valid range 0.0 - 2600
[140516901194560] Value 3000.0 for number.battery_charge_power_limit is outside valid range 0.0 - 2600
[140516901194560] Value 3600.0 for number.battery_discharge_power_limit is outside valid range 0.0 - 2600
Traceback (most recent call last):
  File "/srv/homeassistant/lib/python3.11/site-packages/homeassistant/components/websocket_api/commands.py", line 238, in handle_call_service
    response = await hass.services.async_call(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/srv/homeassistant/lib/python3.11/site-packages/homeassistant/core.py", line 2067, in async_call
    response_data = await coro
                    ^^^^^^^^^^
  File "/srv/homeassistant/lib/python3.11/site-packages/homeassistant/core.py", line 2104, in _execute_service
    return await target(service_call)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/srv/homeassistant/lib/python3.11/site-packages/homeassistant/helpers/entity_component.py", line 272, in handle_service
    return await service.entity_service_call(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/srv/homeassistant/lib/python3.11/site-packages/homeassistant/helpers/service.py", line 878, in entity_service_call
    single_response = await _handle_entity_call(
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/srv/homeassistant/lib/python3.11/site-packages/homeassistant/helpers/service.py", line 948, in _handle_entity_call
    result = await task
             ^^^^^^^^^^
  File "/srv/homeassistant/lib/python3.11/site-packages/homeassistant/components/number/__init__.py", line 96, in async_set_value
    raise ValueError(
ValueError: Value 3600.0 for number.battery_charge_power_limit is outside valid range 0.0 - 2600

The following errors both appear to occur on integration reload:
Logger: givenergy_modbus.client.client
Source: runner.py:188
First occurred: 11:02:29 (2 occurrences)
Last logged: 11:03:56

network_consumer reader at EOF, cannot continue

Logger: custom_components.givenergy_local.coordinator
Source: helpers/update_coordinator.py:332
Integration: GivEnergy Local ([documentation](https://github.com/cdpuk/givenergy-local), [issues](https://github.com/cdpuk/givenergy-local/issues))
First occurred: 11:02:29 (2 occurrences)
Last logged: 11:03:56

Error fetching Inverter data: Error communicating with inverter:

Not sure about the following, perhaps related to the controls not functioning?
Error doing job: _GatheringFuture exception was never retrieved
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/asyncio/tasks.py", line 490, in wait_for
    return fut.result()
           ^^^^^^^^^^^^
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/srv/homeassistant/lib/python3.11/site-packages/givenergy_modbus/client/client.py", line 207, in send_request_and_await_response
    await asyncio.wait_for(response_future, timeout=timeout)
  File "/usr/local/lib/python3.11/asyncio/tasks.py", line 492, in wait_for
    raise exceptions.TimeoutError() from exc
TimeoutError

Logger: homeassistant
Source: custom_components/givenergy_local/coordinator.py:37
Integration: GivEnergy Local ([documentation](https://github.com/cdpuk/givenergy-local), [issues](https://github.com/cdpuk/givenergy-local/issues))
First occurred: 10:02:12 (2 occurrences)
Last logged: 10:05:36

Error doing job: Task exception was never retrieved
Traceback (most recent call last):
  File "/home/homeassistant/.homeassistant/custom_components/givenergy_local/coordinator.py", line 37, in async_shutdown
    await self.client.close()
  File "/srv/homeassistant/lib/python3.11/site-packages/givenergy_modbus/client/client.py", line 68, in close
    await self.writer.wait_closed()
  File "/usr/local/lib/python3.11/asyncio/streams.py", line 350, in wait_closed
    await self._protocol._get_close_waiter(self)
  File "/usr/local/lib/python3.11/asyncio/selector_events.py", line 995, in _read_ready__data_received
    data = self._sock.recv(self.max_size)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ConnectionResetError: [Errno 104] Connection reset by peer

A further update, delayed by festivities and travel: as well as not being able to perform any control functions as described above, the read functionality doesn't last long, and the integration fails again completely, though clearly we're on the right track...

Thanks for the update. I too haven't had much time to make any further changes over the festive period.

Power limits - probably easy to fix. I'm not too worried about this.
Changing booleans - I didn't get around to implementing the CRC correction code for write operations, so I expect that's why.
Updates via the app - good, glad to hear they're being reflected in HA.
Probably related to 2.

I've noticed that modbus, or this specific modbus implementation, is pretty dumb. There doesn't appear to be any isolation between clients when multiple connections are established with an inverter. e.g. while running my development HA, the modbus connection on that receives responses to requests made from my live HA instance. This appears to cause the givenergy_modbus library to get out of sync (receiving a response for a request it never made) and future comms fail. I wouldn't be surprised if failure to write values correctly causes the same thing and also interrupts cloud comms temporarily.
Regardless, the fact you can read data again is a massive step forwards. Next up I'll sort out the CRCs for control operations. Thanks again for your help testing this.

Hi,
Just checking in - is there any update on this please? I failed to check outstanding issues before upgrading my firmware last night and would be super keen to get this back up and running! Happy to help out with testing if I can - I am running Hy-5-G2 inverter and 9.5-G2 battery.
Thanks in advance!

No updates yet, but I'm keen to upgrade my own firmware so this is very much on the list of things to sort out as soon as I've got time to do so.

I've just published v2.0.0-alpha3.

givenergy_modbus has crudely been copied in to the codebase to avoid the dependency issues we saw in alpha2.
A possible fix for write operations has been added. A test would be appreciated to see if values can now be updated.

I've not yet looked at the instability over time, or the issue with the power limits.

Sensors now visible! And only after simple upgrade through HACS, no additional commands required.
Am getting regular error though: "ERROR (MainThread) [custom_components.givenergy_local.coordinator] Error fetching Inverter data: Error communicating with inverter:"
I will have a look through the entities and services in the morning to see if there is anything awry there.
Thanks for the step in the right direction!

Hmm, sensors died overnight, with the following debug errors occurring every 30 seconds:
2024-01-21 06:13:02.364 DEBUG (MainThread) [custom_components.givenergy_local.givenergy_modbus.client.client] Cancelling existing in-flight request and replacing: 2:4/ReadInputRegistersRequest(slave_address=0x32 base_register=0) 2024-01-21 06:13:02.364 DEBUG (MainThread) [custom_components.givenergy_local.givenergy_modbus.client.client] Cancelling existing in-flight request and replacing: 2:4/ReadInputRegistersRequest(slave_address=0x32 base_register=180) 2024-01-21 06:13:02.365 DEBUG (MainThread) [custom_components.givenergy_local.givenergy_modbus.client.client] Cancelling existing in-flight request and replacing: 2:3/ReadHoldingRegistersRequest(slave_address=0x32 base_register=0) 2024-01-21 06:13:02.365 DEBUG (MainThread) [custom_components.givenergy_local.givenergy_modbus.client.client] Cancelling existing in-flight request and replacing: 2:3/ReadHoldingRegistersRequest(slave_address=0x32 base_register=60) 2024-01-21 06:13:02.366 DEBUG (MainThread) [custom_components.givenergy_local.givenergy_modbus.client.client] Cancelling existing in-flight request and replacing: 2:3/ReadHoldingRegistersRequest(slave_address=0x32 base_register=120) 2024-01-21 06:13:02.366 DEBUG (MainThread) [custom_components.givenergy_local.givenergy_modbus.client.client] Cancelling existing in-flight request and replacing: 2:4/ReadInputRegistersRequest(slave_address=0x32 base_register=120) 2024-01-21 06:13:02.367 DEBUG (MainThread) [custom_components.givenergy_local.givenergy_modbus.client.client] Cancelling existing in-flight request and replacing: 2:4/ReadInputRegistersRequest(slave_address=0x32 base_register=60) 2024-01-21 06:13:02.367 DEBUG (MainThread) [custom_components.givenergy_local.givenergy_modbus.client.client] Cancelling existing in-flight request and replacing: 2:4/ReadInputRegistersRequest(slave_address=0x33 base_register=60) 2024-01-21 06:13:02.368 DEBUG (MainThread) [custom_components.givenergy_local.givenergy_modbus.client.client] Cancelling existing in-flight request and replacing: 2:4/ReadInputRegistersRequest(slave_address=0x34 base_register=60) 2024-01-21 06:13:02.368 DEBUG (MainThread) [custom_components.givenergy_local.givenergy_modbus.client.client] Cancelling existing in-flight request and replacing: 2:4/ReadInputRegistersRequest(slave_address=0x35 base_register=60) 2024-01-21 06:13:02.372 DEBUG (MainThread) [custom_components.givenergy_local.givenergy_modbus.client.client] Cancelling existing in-flight request and replacing: 2:4/ReadInputRegistersRequest(slave_address=0x36 base_register=60) 2024-01-21 06:13:12.361 DEBUG (MainThread) [custom_components.givenergy_local.coordinator] Finished fetching Inverter data in 10.002 seconds (success: False)
Looking further back, the following critical occurred on boot:
2024-01-20 21:35:46.743 CRITICAL (MainThread) [custom_components.givenergy_local.givenergy_modbus.client.client] network_consumer reader at EOF, cannot continue
I reloaded the integration - debug log attached, if that helps!
2024-01-21-06-21-GivEnergy_Debug.log
Will report back later on any further status - let me know if you want me to try/lookout for anything else.

Connection now failed, with same issue from last night:
2024-01-21 07:44:27.259 INFO (MainThread) [custom_components.givenergy_local.coordinator] Fetching data from <IP_ADDRESS> 2024-01-21 07:44:28.279 ERROR (MainThread) [custom_components.givenergy_local.coordinator] Error fetching Inverter data: Error communicating with inverter: 2024-01-21 07:44:28.280 DEBUG (MainThread) [custom_components.givenergy_local.coordinator] Finished fetching Inverter data in 1.021 seconds (success: False)
and continues with this every 30 seconds:
2024-01-21 07:45:29.272 INFO (MainThread) [custom_components.givenergy_local.coordinator] Fetching data from <IP_ADDRESS> 2024-01-21 07:45:30.302 DEBUG (MainThread) [custom_components.givenergy_local.coordinator] Finished fetching Inverter data in 1.029 seconds (success: False)
Wifi is fine, I've monitored on my router and can connect to the inverter using GivEnergy App. Reloading the service restores connection... lets see for how long...

So last few hours have been on and off - big gap in connectivity for about an hour and Error communicating with inverter every 2-10 minutes...
I've not yet tried any service calls, but noticed that the Charge and Discharge limits are set to 2600, when they should be (and were previously) 3600 as per the Gen2 capability.

The poor reliability is still an issue that needs looking at. The alpha3 release should enable settings to be written from HA. It'd be useful for someone to test this, but it needs doing during what might be a brief window when data is actually being read successfully.
alpha4, just published, attempts to restore the inverter model identification, which should correctly identify things like power limits.

The poor reliability is still an issue that needs looking at. The alpha3 release should enable settings to be written from HA. It'd be useful for someone to test this, but it needs doing during what might be a brief window when data is actually being read successfully.
alpha4, just published, attempts to restore the inverter model identification, which should correctly identify things like power limits.
The device info should display some of this info:

Can confirm the ID of the inverter amd this the power limits is now correct 👍
As you say no change on the reliability side, and I'm away so have not got time to check the write access properly I'm afraid - doesn't appear to work but I've not looked at logs and I have other things set up that maybe interfering.

I can also confirm the device info and power limits are now correct, another step forward! Thank you.
I have tried adjusting the charge/discharge limits and toggling the Charge/Discharge/Eco switches and these appear to work. second step forward! Thank you!
However, connection stability is still low.

I've now had time to check the write actions, and can also confirm those I have tested (forcing a charge, forcing an export, returning to auto/dynamic/eco mode) work fine 👍
I'm assuming for now that they all do.
Stability is indeed still a significant issue, so I will also try setting up an automation to Reload periodically if the integration has been unavailable for a while.
I'll look out for the peaks mentioned above too.
Happy to help test anything to help with that - pretty busy at the moment though I'll do my best!

v2.0.0-alpha5 is now ready for testing. This completely clears up the stability issues for me - keen to hear whether this works for other too.
The spikes may be due to the fact I removed some error checking code that attempts to identify improbable values. Do other entities also report garbage at exactly the same time? The checks used to look for things like:

Inverter temperature == 0
Battery temperature == 0
Inverter total output < 0

It would be simple enough to reinstate such checks. I'll try to run mine for a bit longer to capture some data.

Great news, I will install now and let you know how it goes.
And yes, multiple givenergy_local entities report garbage/spikes at the same time - suggest the error checking is needed, but I'll report back tomorrow to confirm still an issue in alpha5.

Quick report on first 8 hours, sorry it's not quite going to plan:

Initial warnings on boot:
 2024-01-26 06:04:31.513 WARNING (MainThread) [py.warnings] /config/custom_components/givenergy_local/switch.py:69: RuntimeWarning: coroutine 'GivEnergyUpdateCoordinator.execute' was never awaited 2024-01-26 06:04:31.695 WARNING (MainThread) [py.warnings] /config/custom_components/givenergy_local/switch.py:103: RuntimeWarning: coroutine 'GivEnergyUpdateCoordinator.execute' was never awaited 2024-01-26 06:05:31.119 WARNING (MainThread) [py.warnings] /config/custom_components/givenergy_local/switch.py:65: RuntimeWarning: coroutine 'GivEnergyUpdateCoordinator.execute' was never awaited
None of the switches are working - coinciding with a fresh log entry as above:
2024-01-26 06:45:47.991 WARNING (MainThread) [py.warnings] /config/custom_components/givenergy_local/switch.py:137: RuntimeWarning: coroutine 'GivEnergyUpdateCoordinator.execute' was never awaited
Changing variables (charge/discharge rate etc) appears to work - although the exact figure does "stick". If I put 2000, it returns 1999, if I put 1800, it returns 1714, 1000 returns 952...
Attempting to change variables and toggle switches occasionally causes things to become unavailable, but return within 10 seconds
A number of deprecated unit types are used (POWER_WATT, ELECTRIC_POTENTIAL_VOLT, ENERGY_KILO_WATT_HOUR, FREQUENCY_HERTZ, TEMP_CELCIUS) will be removed from HA in 2025.1 - I will raise another ticket for these as it's not critical to the modbus issue.

Hopefully easy to identify!

I can also confirm that stability is still poor, I'm afraid I am super-busy and will be for at least the next week, so cannot spare time to dig into logs etc, sorry!

Another day, another attempt: v2.0.0-alpha6
I'm surprised how unstable alpha5 has been for others, given the difference it made for me, but I've been able to address a few more reliability issues after throwing more abuse at my test instance, e.g. delaying responses, breaking the connection unexpectedly. Perhaps these will help things, but it'll be interesting to see logs from any further stability problems.
On the points from @comm:

My bad, missed some awaits. Now fixed.
Probably due to the above - should be fine now.
This should't have changed as part of this v2.0.0 series. It's a "feature" of the API: the power limits can only be represented by numbers 0-100, and IIRC, only 0-50 is actually used. An increment of this number corresponds to a much larger incremet in terms of watts. When setting the value in HA, the nearest 0-50 value gets picked and converted back for display in the UI.
Works for me, which isn't helpful, but let's see whether alpha6 improves anything for you.
Fixed on main and ported forward to v2.0.0-alpha6 to cut down on errors/warnings.

Thanks for persisting! Stability now appears good, although not perfect (5 disconnects over 8 hours rather than every 10/15 minutes) - see how it goes.
Other feedback:

No worries, things will always slip through when you're trying to solve another issue - glad we could sort them - all gone now.
Switches now working again - thanks!
My bad, maybe I just didn't notice it before, I don't really use it to be fair.
This appears to be fixed.
Thanks for the quick implementation!

Still seeing the random spikes in some entities:

Logs attached for the 4 events this afternoon.
(2024-01-27-17-51-GivEnergy_Debug.log)
Side note: Interesting entity labelling I just noticed though - why is it called "Grid Export Power" when its Positive for Export and Negative for Import? Where is the "Grid Import Power"? Should this not simply be called "Grid Power"? Anyway...

Yes, thanks for getting this up and running again 👍
Stability with alpha6 is much, much better 👏 As at 08.22 today my automation to restart the integration if unavailable for 2.minutes has only been triggered 5 tines since 23:14 last night rather than around every 10 minutes (plus when switches were used).
Regarding spikes, I've only looked at sensor.grid_power so far, but this does spike frequently to the same figure each time: 23,583 W.

This all sounds fairly promising 😄
I updated my firmware last night, which in the end I think was only a battery firmware update, but everything continues to work smoothly after doing that.
We're past getting the basic comms right, and now just dealing with data quality issues. Missing responses and bad values seem far too common. The ideal solution would be for GivEnergy to spend a bit of effort resolving these in firmware, but I think  we've just got to handle those as transparently as possible for now.
v2.0.0-alpha7 is now available, which expands data quality checks to cover the examples seen further up this thread. It will also reattempt any requests that resulted in bad data, up to 3 times. After that you'll still see unavailable entities in HA, though hopefully only for a short period until the next refresh (~10s).
If you're still having to automate reconnections at this point, some logs would be handy (ideally without the automation running) to understand why it gets stuck.
@comm on the Grid Export Power sensor - this is just one of those things where different people have different opinions on how it should work. At the moment it does the simplest thing possible: takes the value straight from the inverter with no transformation and sticks it in a sensor. Some people think it should never go negative, some people would prefer the +/- reversed, some want an import sensor and an export sensor, and some want a combination of all of that. My normal response is that you can do all of that using template sensors, and the integration isn't the right place to be transforming values. Hope that makes sense.

One further observation to mention: I've noticed the odd glitch I do get often coincides with something else reading data from the inverter (e.g. 2 HA instances, or perhaps GivTCP running if anyone is trying that). Just thought I'd mention in case that's what's causing a larger number of glitches for others.


add mac resource fork reader (py23 compatible); drop mac classic support

I added a new fontTools.misc.macRes module with a ResourceReader class which takes in a file object or path, reads data from a Mac resource file and builds a dictionary of resources keyed by type.
On HFS+ filesystem, I try to read from a file's ..namedfork/rsrc, and if that fails I then read the data fork. Reading data-fork resources works on Windows and Linux as well -- as well as on Python 3.
I dropped the Carbon.Res module and other old stuff which has been deprecated and is no longer used in Python 3.
In macCreatorType, I added support for xattr, a Python wrapper for extended filesystem attributes which -- unlike MacOS module -- works with Python 3 as well. It comes pre-installed on OS X's Python, and can be easily installed via pip install xattr, if required.
For reading 'sfnt' resources from suitcase or DFont, I've kept the old ttLib.macUtils interface, at least for now -- until we define a TTCollection interface.
The tests only cover the generic ResourceReader, as I didn't know how to create custom .suit or .dfont fonts... but we can add these later.
@brawer, would you like to have a look when you have some time?
Of course, I'm greatly indebted to your Java implementation of a Mac resource reader ;)
Cheers!
C.
Related issues:
https://github.com/behdad/fonttools/issues/229#issuecomment-150890828
https://github.com/behdad/fonttools/issues/228
https://github.com/behdad/fonttools/issues/221

Oooh! :+1:
Just one little thing: Try decoding resource names as mac-roman. On a few Macs, the system encoding was different, but those were quite rare.

Ok, thanks! Shall I do tostr or tounicode? I guess the former is fine for this purpose.

:+1:


use benefits/calculateBenefit Decision Service for phlPreK and paSeniorFoodBox results

This is to prevent duplication and make it a little easier to create new benefits (less boilerplate).
Note there is a testBenefit decision service in benefits.dmn that already uses the calculateBenefit decision service (proof of concept).
May be prudent to do this together with #18

@sierscse we can discuss this adn #18 when we sync next; idea is to follow the path started by the testBenefit in benefits.dmn


Update and Correction of Aztec Files

This pull request includes grammatical and formatting corrections to the documentation for the Aztec Packages. The changes aim to enhance clarity, consistency, and readability throughout the document.
Key updates include:

Improved sentence structure and punctuation for better flow.
Consistent use of capitalization in headings and important terms.
Enhanced clarity in technical instructions and descriptions.
Minor adjustments to improve formatting and visual structure.
These modifications ensure that the documentation is user-friendly and easier to understand for developers working with Aztec.

Sorry, not much we want here


Add CI to run tests for PRs

Once tests have been created, we can add a CI (free) to run checks across new PRs.
I would recommend CircleCI as it's free to integrate.

happy to pick this one up for you, will require some CircleCI setup your part though?

@adeora7 If you register for CircleCI, sync your github to it and then add this repo as an app - that will allow @joseph-allen to start creating the pipeline.

PR in :)

PR has been merged - pending CircleCI setup from @adeora7

Sure, will do it today itself !!

you can also change the PR settings so that passing tests are required before a merge 😈

@joseph-allen definitely needs to be done - best to hold off until the CI is in place or nothings getting merged

@lmcjt37  I have synced my account with Circle CI and added this project. Let me know if anything else needs to be done.
@joseph-allen FYI.

@adeora7 just need to add the status checks for CircleCi in the settings for this repo - https://circleci.com/docs/2.0/workflows-waiting-status/

@adeora7 the checks don’t appear to be notifying github. Could you screenshot the branch checks section where you select which checks are needed for merge?

@adeora7 also worth checking that the builds are being run in the CircleCi dashboard and succeeding

Looking into this more, not that we should have to, but may need to add a workflow to the YAML config to maybe kick this into life

@adeora7 Hi again. This is still and issue, would be good to either fix the CI or Github integration.
Failing that, you should disable the CI branch check so that the PRs can be approved and merged.

@adeora7 I know how to fix this having done it recently on my own project.
CircleCI > Settings > Projects > (click cog of project) > Advanced settings > Turn "Build forked Pull Requests" on.
This will resolve the forked PRs not being picked up.


Google form link opening in the same Tab

Description
The google form links in the Join Our Team section and the Volunteer group in the footer section are opening in the same tab. Therefore after submitting the form when trying to return to the home page, the Google form shows an error as everybody knows we cannot press the back button after submitting the google form else it will show an error. So it is better for the forms to open in a separate tab to resolve this issue.
Please include this under the HACTOBERFEST tag and assign it to me @SauravMukherjee44 so that I could resolve this issue for I am first in open source contribution.
Screenshots

@SauravMukherjee44 please assign me this under the Hactoberfest tag.

Make sure all of the forms opens in different tabs !

I want to fix the issue please assign me

All the links on the website are opening in the same tab, which is a huge task to fix, So, Please assign me the work under hack October fest so, that I in resolving the issue.

Please assign it to me,I'd like to work on this issue.


Decouple publishing from instantiating data transfer

Goals
In the reference provider, we need to serve go-legs while also serving retrievals (this will be needed in filecoin integration as well). This means we can't instantiate go-data-transfer inside the LegsTransport.
Implementation
I'm not sure if this is the right approach -- ideally, what I'd like to do is get rid of LegsTransport altogether -- I'm not sure I understand the value of this abstraction, especially since it's main method is not used. I would lean instead toward passing the dependencies directly and assume higher level code manages setup & teardown.
BUT we have code expecting go-legs to behave the way it does so I chose an initial path that avoids breaking anything. Open to other suggestions.

another alternative would be to have an interface that matches whatever is needed:
type LegTransport interface {
Reference
DataTransfer() dt.Manager
Topic() *pubsub.Topic
}
That could enable the MakeLegTransport method while allowing someone else to implement another version.

We're probably partially running into edges that are beyond the initial complexity legs was aiming to handle, which was a very simple interface to let a publisher and set of subscribers keep an IPLD dag in sync.
At the top level, it seems like the thing we'd like to do is be able to pass in the existing go-data-transfer along with the host if desired when making the legs handler for that.
The initial design didn't expose a 'legs transport' at all, and i'm happy with that evolving to be hidden back in to the top level goals a user would have of 'make a publisher' and 'make a subscriber' against a topic and a host. The machinery/complexity exposed in the transport were I believe a response to trying to handle multiplexing on the publisher side.


Update bridged metadata resolution & templates

Closes: #114
Description

Adds a central metadata resolver FlowEVMBridgeResolver contract which handles view resolution for bridged assets

This allows for the updates of placeholder metadata views including Display, NFTCollectionDisplay, and FTDisplay without the need for contract updates

Updates template contracts with support for Display, routing resolution through the new FlowEVMBridgeResolver contract

Also includes the template args for each network

Adds args for initial bridged Display, NFTCollectionDisplay, and FTDisplay placeholder metadata views with plans for a fast follow once we have official art to use as placeholder images

For contributor use:

[ ] Targeted PR against main branch
[ ] Linked to Github issue with discussion and accepted design OR link to spec that describes this work.
[ ] Code follows the standards mentioned here.
[ ] Updated relevant documentation
[ ] Re-reviewed Files changed in the Github PR explorer
[ ] Added appropriate labels

Codecov Report
Attention: Patch coverage is 76.00000% with 6 lines in your changes missing coverage. Please review.

Project coverage is 86.98%. Comparing base (45c5da5) to head (d81748d).

Files with missing lines
Patch %
Lines

cadence/contracts/bridge/FlowEVMBridgeResolver.cdc
76.00%
6 Missing :warning:

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #116      +/-   ##
==========================================
- Coverage   87.28%   86.98%   -0.30%
==========================================
  Files          17       18       +1
  Lines         920      945      +25
==========================================
+ Hits          803      822      +19
- Misses        117      123       +6

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Move the type information from the names into metadata.

This metadata was originally encoded in the name as there was nowhere else to store it. Now that VPR has metadata support, we should store the information there.
Fixes #561.

@acomodi + @kgugala - FYI

FYI @acomodi - I squashed the comments.

FYI @acomodi - I squashed the comments.

Ok, LGTM. I would say it can be merged after CI is green.


Libretro - More build errors

np2kai: [Status: fail] [recipes/playstation/psp] [Finished: 2020-04-14 23:24:20.130478+00:00] [http://paste.libretro.com/200654]
retroarch: [Status: start] [recipes/playstation/psp] [Started: 2020-04-14 23:25:24.414964+00:00]
retroarch: [Status: fail] [recipes/playstation/psp] [Finished: 2020-04-14 23:28:10.542635+00:00] [http://paste.libretro.com/200656]
retroarch: [Status: fail] [recipes/playstation/vita] [Finished: 2020-04-14 23:43:09.216244+00:00] [http://paste.libretro.com/200659]

I commited 'fix 17'.
I add fix for Vita.
About PSP, difficult to error resolving.
Please remove from PSP build cycle.
It seems can't use -std=c++11, -std=gnu++, then int64_t.
(But on my PSP building, success with -std=c++11.)
Regard.

Did a language spec bump happen? I don't think C++11 was required before.
I don't think this was a good idea to do a language spec bump, that means it will compile on fewer platforms that won't ever have C++11 support in their toolchains.
I can take PSP out of the build cycle, but Vita still doesn't compile.

You are right. I removed language spec.
And I could solve errors on PSP.
Sorry to you made wait long time.
I send you pull req now.
It'll solve link error on Vita.

I saw error on
emscripten
armhf
arm7neonhf
windows-x86_dw2
I'm looking and ready others.

I more fixed.

If not C++11, use cstdint
For MSC (fmgen_types.h)
(other) Reoprted bug fix

I more fixed.

For PSP, remove tick count
For Windows, mod to UWP, typedef fix, version bug fix

I add fixed.

fix Windows Desktop & UWP

I add fixed.

fix Windows Desktop & UWP

Lastest Windwos version does not boot any hdi/fdi/m3u

Probably spaced name file was read correctly.
I fixed.

I fixed for windows-x86_dw2.
(I understood x86_dw2 is MinGW32.)

I've update retroarch (windows) core with ver.086 kai rev.21 5dde5c7 and hdi working but fdd/fdi/m3u no.

I tried 'ReadOnly' fdi, it couldn't be opened.
I cheking now.

I fixed that 'read only floppy image' can be opened (and set protect flag).
I'll send pull request soon.


Update sbt-scala-native to 0.4.17

About this PR
📦 Updates org.scala-native:sbt-scala-native from 0.3.9 to 0.4.17
📜 GitHub Release Notes - Version Diff
Usage
✅ Please merge!
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

⚙ Adjust future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "org.scala-native", artifactId = "sbt-scala-native" } ]

Or, add this to slow down future updates of this dependency:
dependencyOverrides = [{
  pullRequests = { frequency = "30 days" },
  dependency = { groupId = "org.scala-native", artifactId = "sbt-scala-native" }
}]

labels: sbt-plugin-update, early-semver-major, semver-spec-minor, commit-count:1

Superseded by #702.


[input-date] make the input date look more material

From @TomGallon on October 6, 2015 9:34

Copied from original issue: KleeGroup/focus-components#490

https://github.com/Hacker0x01/react-datepicker/

http://airbnb.io/react-dates/


[LocalNet] Run Relayer and AppGateServer

Summary
Human Summary
As we've got relayer moved from the alpha repo, and appgateserver merged into main, we need to be able to run them on LocalNet. Support for this on helm-charts side have been merged.
AI Summary

Summary generated by Reviewpad on 11 Nov 23 17:40 UTC
This pull request introduces changes related to the relayer and appgateserver.
It adds new values files for the appgateserver and relayer in the localnet/kubernetes directory and sets their enable values to true.
Additionally, it modifies the Tiltfile to include the appgateserver helm resource, which uses the new values file and sets the replicaCount based on the localnet configuration.
Overall, this patch adds support for the appgateserver and enables the relayer in the localnet setup.

Type of change
Select one or more:

[x] New feature, functionality or library
[ ] Bug fix
[ ] Code health or cleanup
[ ] Documentation
[ ] Other (specify)

Testing

[x] Run all unit tests: make go_develop_and_test
[x] Verify Localnet manually: See the instructions [here](TODO: add link to instructions)

Sanity Checklist

[x] I have tested my changes using the available tooling
[x] I have performed a self-review of my own code
[ ] I have commented my code, updated documentation and left TODOs throughout the codebase

Based on the state of charts/poktroll/templates/scripts.yaml on main, it looks like we still need to:

Update the subcommand name relayer -> relayminer
Decide when/how to workaround integrate with the solution to the supplier1/app1 vs. supplier2/app2 for the relayminer's and appgate server's signing-key flags, respectively. See #pokrtoll in discord and https://github.com/pokt-network/poktroll/pull/177/files#r1392304589.
Ensure that the suppler and app both have public keys on chain.

@bryanchriswhite Agreed regarding s/relayer/relayminer  in this PR.
Re (2) & (3) - I believe it's outside the scope of this PR and we'll capture it in #180.

Per discussion above, I'm going to tweak the helm charts to use relayminer instead of relayer.

I ended up splitting helm charts for appgate-server and relayminer - https://github.com/pokt-network/helm-charts/pull/6/files. The services are going to have different configs, ports exposed so it made sense to separate them. Also, this should potentially simplify key management for node runners. We will get to that point in a few weeks. :)
Just need a review on that PR and we should be good.

@okdas Anything blocking from getting this in?

nope, let's get it in!


Incremental rebuild performance regression in v5.54.0

Bug report

What is the current behavior?
Webpack spends several seconds inside of lib/Watching.js after an incremental rebuild:
This profile is from 5.54.0:

And I verified it's still an issue in 5.67.0 (as well as a few versions in between):

If the current behavior is a bug, please provide the steps to reproduce.
I'm not sure how to reproduce this outside of our app, I think it's only really noticeable by it being a large application. I've included the profiles from the debugger, so I hope that's enough to go on? If not let me know if there's anything else I can do to help diagnose the root cause.

What is the expected behavior?
Webpack should spend only about a second on this step. This profile is from 5.53.0 (but also seeing the same lower speeds on 5.50.0 and 5.52.1 so I'm confident it's not a fluke)

Other relevant information:
webpack version:  All versions of webpack from 5.54.0
Node.js version: v16.13.1
Operating System: MacOS Monterey
Additional tools: This is being run from the angular cli

Is it happen before 5.54?

Does this happen before 5.54?

Nope, I tried a few versions to be sure and 5.50.0, 5.52.1 and 5.53.0 don't have the issue. This chart is from 5.53.0 but it's a similar story on the other earlier versions:

On all versions from 5.54.0 onwards that I tried, I saw the consistently slower times.

Very strange because we don't change nothing related to watch in 5.53, maybe something in plugins, can you locally debug?

@mattlewis92 Does you use exactly same project/setup ? Is it possible reduce issue to reproducible repo ?

Very strange because we don't change nothing related to watch in 5.53, maybe something in plugins, can you locally debug?

I'll do what I can to debug locally and see if I can work out what's wrong.

@mattlewis92 Do you use exactly same project/setup ? Is it possible reduce issue to reproducible repo ?

Yep exactly the same each time, all I did between each of those profiles was run npm i -D webpack@version and then run an incremental rebuild. I'm not sure how to reproduce it standalone in a repo as our app is enormous, and I suspect the problem is caused by it being a large app, rather than a specific issue within it.

https://github.com/webpack/webpack/pull/15991 may address this partially. I don't know if it'll bring the 7.65s Watching.js call back down to 1.02s, but it should be much less.


Ensure gzip writer is closed in influx_inspect export

Required for all non-trivial PRs

[x] Rebased/mergable
[ ] Tests pass
[x] CHANGELOG.md updated
[x] Sign CLA (if not already signed)

Previously, the gzip writer may have had unflushed data when the
underlying writer was closed, and the output stream would have been
truncated early.

👍

👍


Update readme and package.json

Which version? I don't think it necessarily justifies a v2.0, but it is a rewrite.

I think v2.0 is the safe way to go, I'd prefer to err on the safe side with changes to utilities like this.

and thank you!!!

Done. I bumped to 2.0 in package.json and added you to npm as a maintainer, so I'll leave the honors to you to publish. thanks again!

Published. Thank you as well!

perfect! thanks


Remove AP fragments w/o aromatic rings

Description
For recombination, we want to set up a reduced fragment library (to save computational costs). Special focus was already given to the AP subpocket pool, since this is (a) the most important pocket for kinase inhibitor binding and (b) also the starting point for our recombination algorithm.
In order to only recombine hinge-like fragments, we already filtered the AP pool for fragments that have at least one HBA or HBD. Since we still have small, linear fragments in the AP pool, we here filter additionally for fragments that show at least one aromatic ring.
ToDos

[x] Add column to fragment_library_concat_filtered for number of aromatic rings (and number of heavy atoms - this was our first idea for filtering)
[x] Drop fragments that show no HBA/D and/or no aromatic ring
[x] Run notebook with six different values for parameters N_REPRESENTED_FRAGMENTS and INCLUDE_SINGLETONS

Questions

[ ] @AndreaVolkamer: What do you think of the "number of aromatic rings" > 0 instead of "number of heavy atoms" > 3 criteria?

Status

[x] Ready to go

Hi @AndreaVolkamer,
Please take a look at the extended hinge filtering.
Filtering by number of heavy atoms still left some fragments that I would prefer to filter out:

Filtering by number of aromatic rings (as shown in the notebook in this PR) did the job in my opinion:

What do you think?

Hi @AndreaVolkamer, these are the "hinge-unlike" fragments if filtered by

at least one HBA or HBD
at least one aromatic or aliphatic ring

View changes in notebook:
https://app.reviewnb.com/volkamerlab/KinFragLib/pull/13/files/
Notebook: notebooks/3_1_fragment_library_reduced.ipynb
View final AP fragment selection:
https://github.com/volkamerlab/KinFragLib/blob/hinge_fragments/notebooks/3_1_fragment_library_reduced.ipynb

@dominiquesydow I only adapted the text for the filtered out cases slightly.


Create stable and incubator directories

Fixes https://github.com/kubernetes/charts/issues/39
I'd like to get this change in so that we can start merging in charts from #38, #37 and #41.

We should also update https://github.com/kubernetes/charts/blob/master/CONTRIBUTING.md to explain the purpose of incubator and path to stable - would you be able to do that?

Yep. I'll take care of that. Good call.

I took a first stab at it.

Updated and moved the structure breakdown to the README.

This looks really good to me. I just left a few small comments on the doc.

Updated based on feedback and opened https://github.com/kubernetes/charts/issues/44 so that we don't forget to make the Best Practices doc.

Latest looks great to me.


Feat/support claim hierarchy

This PR brings the support to VC containing a claim with a hierarchy based on named nodes.
Context
When a claim express a hierarchy through named nodes it poses the problem of the credential isolation once stored in the cognitarium. We want to have as named node subject only the credential identifier, any hierarchy descending from the credential shall be based on blank nodes to ensure consistency at revocation and security by avoiding extending other credential properties.
To this point claim containing named nodes hierarchies were rejected, a special treatment to support them is proposed hereafter.
Proposed solution
The idea is to replace the named node making the hierarchical link by a blank node, and then to add a triple linking the original named node to the replacing blank node using the dedicated predicate <dataverse:claim#original-node>, for example:
Taking the following credential RDF in input:
<https://w3id.org/okp4/ontology/vnext/schema/credential/digital-service/description/72cab400-5bd6-4eb4-8605-a5ee8c1a45c9> <https://www.w3.org/2018/credentials#credentialSubject> <did:key:zQ3shhb4SvzBRLbBonsvKb3WX6WoDeKWHpsXXXMhAJETqXAfB> .
<did:key:zQ3shhb4SvzBRLbBonsvKb3WX6WoDeKWHpsXXXMhAJETqXAfB> <https://w3id.org/okp4/ontology/vnext/schema/credential/digital-service/description/hasCategory> <https://w3id.org/okp4/ontology/vnext/thesaurus/digital-service-category/Storage> .
<did:key:zQ3shhb4SvzBRLbBonsvKb3WX6WoDeKWHpsXXXMhAJETqXAfB> <test:claim#named-hierarchy> <test:named-link> .
<test:named-link> <test:claim#nested-predicate> "nested value" .

Will be serialised and stored in the cognitarium this way:
<https://w3id.org/okp4/ontology/vnext/schema/credential/digital-service/description/72cab400-5bd6-4eb4-8605-a5ee8c1a45c9> <dataverse:credential#subject> <did:key:zQ3shhb4SvzBRLbBonsvKb3WX6WoDeKWHpsXXXMhAJETqXAfB> .
<https://w3id.org/okp4/ontology/vnext/schema/credential/digital-service/description/72cab400-5bd6-4eb4-8605-a5ee8c1a45c9> <dataverse:credential#claim> _:c0 .
_:c0 <https://w3id.org/okp4/ontology/vnext/schema/credential/digital-service/description/hasCategory> <https://w3id.org/okp4/ontology/vnext/thesaurus/digital-service-category/Storage> .
_:c0 <test:claim#named-hierarchy> _:a0 .
_:a0 <test:claim#nested-predicate> \"nested value\" .
_:a0 <dataverse:claim#original-node> <test:named-link> .

Security
This PR also brings a security fix ensuring that dataverse reserved predicates are refused as inputs.

Summary by CodeRabbit

New Features

Introduced enhanced serialization for Dataverse credentials, ensuring better handling of reserved predicates.
Added support for named and blank issuers in credential triples.

Refactor

Updated method calls for credential serialization to improve code efficiency.

Tests

Implemented new tests to validate serialization and reserved predicates handling.

Documentation

Added new RDF triples for digital service credentials, showcasing the structure and expected attributes.

size-limit report 📦

Path
Size

target/wasm32-unknown-unknown/release/okp4_objectarium.wasm
440.47 KB (0%)

target/wasm32-unknown-unknown/release/okp4_law_stone.wasm
654.01 KB (0%)

target/wasm32-unknown-unknown/release/okp4_cognitarium.wasm
835.63 KB (0%)

target/wasm32-unknown-unknown/release/okp4_dataverse.wasm
674.44 KB (+0.09% 🔺)

:tada: This PR is included in version 5.0.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


chore: revert concating vendor files

as requested :)

I've submitted this PR by creating https://github.com/angular/angular-cli/pull/231, as it was blocking the release. Thanks @cironunes (put your name in the PR).


add an acpi bgrt example

it'll read the firmware image, and its offsets from the acpi bgrt table and show it in
the same place on the framebuffer.
further development could make it a tool to read a luks passphrase or
similar

Hello gdamjan, thank you for your pull request. I tried running the example on 3 different systems (Arch Linux desktop, Arch Linux laptop and a Raspberry Pi 3B) but none of them seem the have the directory /sys/firmware/acpi/bgrt so the example panics. I couldn't find the bgrt directory anywhere on my systems.
I think it would be cool to get this example working, but if it panics on most systems then I don't think it is a good example.

It can't work on a raspberry since it doesn't have UEFI (and perhaps ACPI 5.0 either).
The existence of /sys/firmware/acpi/bgrt depends on the firmware of course, so not all computers will have it. On my Thinkpad I had to configure it for UEFI only booting (in which case the boot screen is full resolution). https://www.kernel.org/doc/Documentation/ABI/testing/sysfs-firmware-acpi
I could add some explicit error handling when the directory doesn't exist, but being an example seemed like an overkill.

/sys/firmware/acpi/bgrt is available in qemu though:
qemu-system-x86_64 -enable-kvm -m 512  -bios /usr/share/ovmf/x64/OVMF_CODE.fd -cdrom alpine-standard-3.8.1-x86_64.iso


Test/reminders

Added tests for reminders, might have merge conflicts with the PR that edits the invalidReminderException

lgtm! Perhaps can add reminders to some of the original tests for tasks to ensure they work there! eg tests for EditCommand

Ok I will likely add it in another PR


Fix missing status column

PR closed since this is already been implemented in the #180

PR closed since this is already been implemented in the #180


Updating to Java 11

With support for Java 8 ending in January, 2019, I think it is a good time to discuss updating Luna to use Java 11. As Luna does not seem to be a commercial product, there shouldn't be an issue using Oracle's official JDK releases, even with their new licensing.
Besides the support deadline, there are many other reasons to update:

Modularization (Java 9)
G1 Garbage Collector (Java 9)
Collection Factory Methods (Java 9)
Type Inference Enhancements (Java 9)
Latin1 Encoded Strings (Java 9)
GraalVM (Java 10)
Local Variable Type Inference (Java 10)
Built-in HttpClient (Java 11)
An extensive amount of new methods (Java 9-11)

I would recommend using the OpenJDK Java 11. Just to prevent issues with the license in the scenario that someone would like to use Luna for commercial use, or in the scenario that a commercial entity would like to run a community OSRS server. Not that I would recommend any commercial entity do this.

I just realized that it probably doesn't matter which JDK we use to build it, so long that we don't create a self-contained executable with it. It would essentially be up to the user to select their JDK.
Side-note: I was able to build Luna on JDK 11, but I had to remove the following dependencies:
<dependency>
    <groupId>org.functionaljava</groupId>
    <artifactId>functionaljava-java8</artifactId>
    <version>4.7</version>
</dependency>
<dependency>
    <groupId>org.functionaljava</groupId>
    <artifactId>functionaljava-quickcheck</artifactId>
    <version>4.7</version>
</dependency>
<dependency>
    <groupId>org.functionaljava</groupId>
    <artifactId>functionaljava-java-core</artifactId>
    <version>4.7</version>
</dependency>```
No code changes were required (other than the addition of the `module-info.java`).

The functional java dependencies are only used in one or two situations (one of which Java 11 can replace) so I think we'd be okay removing them altogether. I think there are a few things that need to be considered when updating

[ ] - Changing local variables to use 'var' syntax
[ ] - Using Java 9 modules
[ ] - Using new additions to the Optional API

Once issue #80 is completed I'll create a new branch for upgrading to Java 11. Do you guys have any suggestions or objections for code that should be updated while transitioning?

@lare96 From what I've seen, the functional dependencies are only used for <Integer, Integer> tuples (which should definitely be their own class to prevent boxing) and to get a list of lines from a String (which is built in to Java 11 with String#lines).
I don't think any code modifications should occur until the server is successfully built on the new branch, but I do agree that we can take advantage of var (so long as we stick to the style guide).
At first, the server should also be one big module and we can modularize it from there, if needed.

It seems that the Scala dependencies that we rely on do not fully support Java 11 yet. Their next version milestones are set for late January and early February, 2019, so it would be best to put this issue on hold until then.

Ammonite unblocks this issue, but the only problem is we'll have to remain on Scala 2.12.6 until a version is compatible with Java 11


Opencensus Tracing: fix nested traces + ApiTracer types

Complex callable chains like LRO callables need to be able to create a hierarchy of spans.
For example LROs will need to create the following heirarchy:

one span for the entire logical operation
a child for the initial rpc to start the operation
a child for each poll attempt

This PR enables the behavior by extracting the parent span from the ApiCallContext. If the ApiCallContext doesn't have a parent span, it will fallback to the ambient parent span that the caller might've set in the GRPC context.
It also introduces ApiTracer types to modify the behavior of tracing for different operation types. Currently this only affects Batching by forcing it be a root span. In a follow up PR it also be used for LRO tracing to change the annotations for polling.
The two changes (nesting & tracer types) are unrelated, but they both modify the same factory methods, so I figured I would bundle them together. If it makes it easier to review, I can split the PR.

All feedback should be addressed. PTAL


added photo

I am from Brl AKGEC

This is not the right way to commit changes. For now just add your card (including both the image and the spotify playlist link)


Improvement documentation: ballerina.lang.messages

Documentation of ballerina.lang.messages is a bit ambiguous:
Example:
function getXmlPayload(message m) (xml )
Gets the message payload in XML format
...
The XML representation of the message payload

It should be stated clearly that there is no implicit casting in place. If message is of type application/json it will not result in an XML representation, it will fail instead. Same applies to other functions accordingly.

message is no longer there


Update  dapr-1.0.0-rc.1.tgz

The checksum may change, so you probably have to do a helm merge or calculate a new checksum?

The checksum may change, so you probably have to do a helm repo index --url ${{ env.DAPR_HELM_REPO }} --merge index.yaml ., or calculate a new checksum?
yes.

@artursouza you need to generate index again.

Done. I thought it was 2 separate PRs for that. Now I get it.

Please, merge.

Please, merge.

DONE


cheerio: The find function can also take a cheerio element as an argument

cf. documentation : https://github.com/cheeriojs/cheerio#findnode

@GMouron thanks mate!


Add econo toggle for stylish

Is your feature request related to a problem? Please describe:

Describe the solution you'd like:

Describe alternatives you've considered:

Additional context:

Could you please fill more information about your enhancement report? It ain't gonna happen without some more intel about what you're expecting

@viliamcarnoky I understand that Stylish is not an AC? Could you post the logs with debug mode enabled?

nIt is the BRP069B4x model it is an AC. It supports Econo mode. Also i Dont have the Fan speed control. Ill try to send you the logs.

Apparently I have a Stylish device example: https://github.com/JeroenVdb/homebridge-daikin-cloud/blob/main/docs/api-response-with-multiple-devices-incl-heatpump.json
This indeed has a econoMode. So I don't know why it's now showing. Have you set the showExtraFeatures option to true?

yes

Ok, then I will need to see the logs :)

Im sending you the logs
homebridge.log.txt


chore: verify user is logged in to npm before local publish

Npm login guard

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 97.60%. Comparing base (e3c4886) to head (98c800f).
Report is 3 commits behind head on master.

Additional details and impacted files
@@           Coverage Diff           @@
##           master     #182   +/-   ##
=======================================
  Coverage   97.60%   97.60%
=======================================
  Files          17       17
  Lines         209      209
  Branches       29       29
=======================================
  Hits          204      204
  Misses          5        5

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


add Recommended Response content for monitoring metrics [#152615851]

Signed-off-by: Glenn Oppegard<EMAIL_ADDRESS>

@vikafed please review and merge when you get a chance.  thanks!

Thanks, @robertjsullivan!


Request fails if user date is after st louis date because of time zones

If the user date is after St Louis date (b/c in europe after midnight) then the request fails.
Need to change some of the defaults that are using DateTime.Now. Perhaps just set to max real-time date of 9999-12-31.
> fred.Series.Search("10-year").Summary()

Error: System.Net.WebException: The remote server returned an error: (400) Bad Request.
Response from https://api.stlouisfed.org/fred/series/search?&search_text=10-year&search_type=full_text&realtime_start=2023-09-27&realtime_end=2023-09-27&limit=20&order_by=search_rank&sort_order=desc&api_key=dcad55219c8a9de0d22274de20bd4187&file_type=json:
{"error_code":400,"error_message":"Bad Request. Variable realtime_start can not be after today's date (2023-09-26) unless it's equal to the real-time max date 9999-12-31."}

@DavideGCosta do you have any interest in fixing this?
The API thinks it is in St. Louis and it won’t accept start or end dates after the current St. Louis date. So default start and ends of DateTime.Now can fail if you’re not in the St. Louis time zone (As is happening to me now).

@nhirschey yes! Will take a look and lyk

Great! I think .net 6 added some support that will be useful. https://devblogs.microsoft.com/dotnet/date-time-and-time-zone-enhancements-in-net-6/#time-zone-conversion-apis

BTW, no rush.

Will be out in NY till next Tuesday! Will have a look after that


feature: spreadsheet support

https://github.com/PHPOffice/PhpSpreadsheet

{
  "filename": "export",
  "writer": "xlsx",
  "active_sheet": 0,
  "sheets": [
    {
      "name": "Export form",
      "color": "#FF0000",
      "rows": [
          ["a1", "a2"],
          ["b1", "b3"]
      ]
    },
    {
      "name": "Export form",
      "rows": [
          ["a1", "a2"],
          ["b1", "b3"]
      ],
    }
  ]
}

Add spreadsheet namespace to the Common namespace
Add contract interface for implementation in the core

Released in 1.8.58