Test fails on Windows 8.1 64-bit - GL_INVALID_OPERATION error generated.  is not a valid framebuffer object.

T:\Data\Projects\glium>cargo test --verbose
       Fresh compile_msg v0.1.3 (https://github.com/huonw/compile_msg#b19da50c)
       Fresh bitflags v0.1.0
       Fresh glium_macros v0.0.1 (file:///T:/Data/Projects/glium)
       Fresh khronos_api v0.0.5
       Fresh rustc-serialize v0.2.10
       Fresh log v0.2.1
       Fresh gl_common v0.0.3
       Fresh winapi v0.1.6
       Fresh xml-rs v0.1.16
       Fresh num v0.1.10 (https://github.com/rust-lang/num#c05cd530)
       Fresh cgmath v0.0.1 (https://github.com/bjz/cgmath-rs#9bb0f789)
       Fresh nalgebra v0.2.2 (https://github.com/sebcrozet/nalgebra#14df1380)
       Fresh kernel32-sys v0.0.5
       Fresh gdi32-sys v0.0.3
       Fresh user32-sys v0.0.5
       Fresh gl_generator v0.0.13
       Fresh image v0.2.0-alpha.8 (https://github.com/PistonDevelopers/image#b29
d0706)
       Fresh glutin v0.0.4-pre (http://github.com/tomaka/glutin#be092183)
       Fresh glium v0.0.2 (file:///T:/Data/Projects/glium)
     Running `T:\Data\Projects\glium\target\attrs-96fa59aaedf34481.exe`

running 9 tests
Debug message: `Buffer detailed info: Buffer object 1 (bound to GL_ARRAY_BUFFER_
ARB, usage hint is GL_STATIC_DRAW) will use VIDEO memory as the source for buffe
r object operations.`
Debug message: `Buffer detailed info: Buffer object 2 (bound to GL_ARRAY_BUFFER_
ARB, usage hint is GL_STATIC_DRAW) will use VIDEO memory as the source for buffe
r object operations.`
test attribute_vec2_tuple ... ok
Debug message: `Buffer detailed info: Buffer object 1 (bound to GL_ARRAY_BUFFER_
ARB, usage hint is GL_STATIC_DRAW) will use VIDEO memory as the source for buffe
r object operations.`
Debug message: `Buffer detailed info: Buffer object 1 (bound to GL_ARRAY_BUFFER_
ARB, usage hint is GL_STATIC_DRAW) will use VIDEO memory as the source for buffe
r object operations.`
Debug message: `Buffer detailed info: Buffer object 2 (bound to GL_ARRAY_BUFFER_
ARB, usage hint is GL_STATIC_DRAW) will use VIDEO memory as the source for buffe
r object operations.`
test attribute_types_mismatch ... ok
Debug message: `Buffer detailed info: Buffer object 2 (bound to GL_ARRAY_BUFFER_
ARB, usage hint is GL_STATIC_DRAW) will use VIDEO memory as the source for buffe
r object operations.`
test attribute_float ... ok
test attribute_vec2 ... ok
Debug message: `Buffer detailed info: Buffer object 1 (bound to GL_ARRAY_BUFFER_
ARB, usage hint is GL_STATIC_DRAW) will use VIDEO memory as the source for buffe
r object operations.`
Debug message: `Buffer detailed info: Buffer object 2 (bound to GL_ARRAY_BUFFER_
ARB, usage hint is GL_STATIC_DRAW) will use VIDEO memory as the source for buffe
r object operations.`
test attribute_vec3_tuple ... ok
Debug message: `Buffer detailed info: Buffer object 1 (bound to GL_ARRAY_BUFFER_
ARB, usage hint is GL_STATIC_DRAW) will use VIDEO memory as the source for buffe
r object operations.`
Debug message: `Buffer detailed info: Buffer object 2 (bound to GL_ARRAY_BUFFER_
ARB, usage hint is GL_STATIC_DRAW) will use VIDEO memory as the source for buffe
r object operations.`
test attribute_vec3 ... ok
Debug message: `Buffer detailed info: Buffer object 1 (bound to GL_ARRAY_BUFFER_
ARB, usage hint is GL_STATIC_DRAW) will use VIDEO memory as the source for buffe
r object operations.`
Debug message: `Buffer detailed info: Buffer object 2 (bound to GL_ARRAY_BUFFER_
ARB, usage hint is GL_STATIC_DRAW) will use VIDEO memory as the source for buffe
r object operations.`
test attribute_vec4 ... ok
Debug message: `Buffer detailed info: Buffer object 1 (bound to GL_ARRAY_BUFFER_
ARB, usage hint is GL_STATIC_DRAW) will use VIDEO memory as the source for buffe
r object operations.`
Debug message: `Buffer detailed info: Buffer object 2 (bound to GL_ARRAY_BUFFER_
ARB, usage hint is GL_STATIC_DRAW) will use VIDEO memory as the source for buffe
r object operations.`
test attribute_vec4_tuple ... ok
test missing_attribute ... ok

test result: ok. 9 passed; 0 failed; 0 ignored; 0 measured

     Running `T:\Data\Projects\glium\target\backface-culling-53f05dbce04756d9.ex
e`

running 4 tests
Debug message: `Buffer detailed info: Buffer object 1 (bound to GL_ARRAY_BUFFER_
ARB, usage hint is GL_STATIC_DRAW) will use VIDEO memory as the source for buffe
r object operations.`
Debug message: `Buffer detailed info: Buffer object 2 (bound to GL_ARRAY_BUFFER_
ARB, usage hint is GL_STATIC_DRAW) will use VIDEO memory as the source for buffe
r object operations.`
thread 'glium rendering thread' panicked at 'Debug message with high or medium s
everity: `GL_INVALID_OPERATION error generated. <framebuffer> is not a valid fra
mebuffer object.`.
Please report this error: https://github.com/tomaka/glium/issues', src\lib.rs:16
56
Debug message: `Buffer detailed info: Buffer object 1 (bound to GL_ARRAY_BUFFER_
ARB, usage hint is GL_STATIC_DRAW) will use VIDEO memory as the source for buffe
r object operations.`
Debug message: `Buffer detailed info: Buffer object 2 (bound to GL_ARRAY_BUFFER_
ARB, usage hint is GL_STATIC_DRAW) will use VIDEO memory as the source for buffe
r object operations.`
thread 'glium rendering thread' panicked at 'Debug message with high or medium s
everity: `GL_INVALID_OPERATION error generated. <framebuffer> is not a valid fra
mebuffer object.`.
Please report this error: https://github.com/tomaka/glium/issues', src\lib.rs:16
56
stack backtrace:
   1:         0x69bec62b - sys::backtrace::write::h5d8eeb6b31ece965cqu
   2:         0x69c026eb - rt::unwind::register::h49d2602058168ec5c7A
   3:         0x69b8360f - rt::unwind::begin_unwind_inner::h1772a00c8d230691N4A
   4:         0x69b83dd1 - rt::unwind::begin_unwind_fmt::h84a2c35fac0712d0j3A
   5:         0x69c01d9a - rust_begin_unwind
   6:         0x69c17489 - panicking::panic_fmt::had385503c1077d42BRv
   7:           0x49425b - main
   8:           0x5fdadf - main
   9:           0x5f5a74 - main
  10:           0x5f3d55 - main
  11:           0x568ede - main
  12:           0x568e79 - main
  13:           0x44c3a9
  14:           0x44c36d
  15:           0x4830e1
  16:         0x6979f65e - Bencher::ns_per_iter::h22bba3f257e5ac85udc
  17:         0x697a87d9 - Bencher::ns_per_iter::h22bba3f257e5ac85udc
  18:         0x697a0034 - Bencher::ns_per_iter::h22bba3f257e5ac85udc
  19:         0x697a057a - Bencher::ns_per_iter::h22bba3f257e5ac85udc
  20:         0x69c2f46c - rust_try
  21:         0x69c2f449 - rust_try
  22:         0x697a0baf - Bencher::ns_per_iter::h22bba3f257e5ac85udc
  23:         0x69bf2e12 - sys::time::_&'a SteadyTime.Sub::sub::h818476a6d1b7066
augx
  24:     0x7ff86d9213d2 - BaseThreadInitThunk
thread panicked while panicking. aborting.
Debug message: `Buffer detailed info: Buffer object 1 (bound to GL_ARRAY_BUFFER_
ARB, usage hint is GL_STATIC_DRAW) will use VIDEO memory as the source for buffe
r object operations.`
Debug message: `Buffer detailed info: Buffer object 2 (bound to GL_ARRAY_BUFFER_
ARB, usage hint is GL_STATIC_DRAW) will use VIDEO memory as the source for buffe
r object operations.`
thread 'glium rendering thread' panicked at 'Debug message with high or medium s
everity: `GL_INVALID_OPERATION error generated. <framebuffer> is not a valid fra
mebuffer object.`.
Please report this error: https://github.com/tomaka/glium/issues', src\lib.rs:16
56
Debug message: `Buffer detailed info: Buffer object 1 (bound to GL_ARRAY_BUFFER_
ARB, usage hint is GL_STATIC_DRAW) will use VIDEO memory as the source for buffe
r object operations.`
stack backtrace:
   1:         0x69bec62b - sys::backtrace::write::Debug message: `Buffer detaile
d info: Buffer object 2 (bound to GL_ARRAY_BUFFER_ARB, usage hint is GL_STATIC_D
RAW) will use VIDEO memory as the source for buffer object operations.`
h5d8eeb6b31ece965cqu
   2:         0x69c026eb - rt::unwind::register::h49d2602058168ec5c7A
   3:         0x69b8360f - rt::unwind::begin_unwind_inner::h1772a00c8d230691N4A
   4:         0x69b83dd1 - rt::unwind::begin_unwind_fmt::h84a2c35fac0712d0j3A
thread 'glium rendering thread' panicked at 'Debug message with high or medium s
everity: `GL_INVALID_OPERATION error generated. <framebuffer> is not a valid fra
mebuffer object.`.
Please report this error: https://github.com/tomaka/glium/issues', src\lib.rs:16
56
   5:         0x69c01d9a - rust_begin_unwind
   6:         0x69c17489 - panicking::panic_fmt::had385503c1077d42BRv
   7:           0x49425b - main
   8:           0x5fdadf - main
   9:           0x5f5a74 - main
  10:

T:\Data\Projects\glium>rustc --version
rustc 1.0.0-nightly (52c74e63d 2015-01-29 22:54:19 +0000)

GPU: Geforce GTX 660 Ti

Thanks for reporting!
Render to texture doesn't seem to work at all.
Could you manually run the four tests starting by texture by launching their executable from the target directory?

Here you go.
T:\Data\Projects\glium\target>texture_draw-ebe0990174773ff8.exe

running 2 tests
Debug message: `Buffer detailed info: Buffer object 1 (bound to GL_ARRAY_BUFFER_
ARB, usage hint is GL_STATIC_DRAW) will use VIDEO memory as the source for buffe
r object operations.`
Debug message: `Buffer detailed info: Buffer object 2 (bound to GL_ARRAY_BUFFER_
ARB, usage hint is GL_STATIC_DRAW) will use VIDEO memory as the source for buffe
r object operations.`
thread 'glium rendering thread' panicked at 'Debug message with high or medium s
everity: `GL_INVALID_OPERATION error generated. <framebuffer> is not a valid fra
mebuffer object.`.
Please report this error: https://github.com/tomaka/glium/issues', src\lib.rs:16
56
Debug message: `Buffer detailed info: Buffer object 1 (bound to GL_ARRAY_BUFFER_
ARB, usage hint is GL_STATIC_DRAW) will use VIDEO memory as the source for buffe
r object operations.`
Debug message: `Buffer detailed info: Buffer object 2 (bound to GL_ARRAY_BUFFER_
ARB, usage hint is GL_STATIC_DRAW) will use VIDEO memory as the source for buffe
r object operations.`
thread 'glium rendering thread' panicked at 'Debug message with high or medium s
everity: `GL_INVALID_OPERATION error generated. <framebuffer> is not a valid fra
mebuffer object.`.
Please report this error: https://github.com/tomaka/glium/issues', src\lib.rs:16
56
stack backtrace:
   1:         0x69bec62b - sys::backtrace::write::h5d8eeb6b31ece965cqu
   2:         0x69c026eb - rt::unwind::register::h49d2602058168ec5c7A
   3:         0x69b8360f - rt::unwind::begin_unwind_inner::h1772a00c8d230691N4A
   4:         0x69b83dd1 - rt::unwind::begin_unwind_fmt::h84a2c35fac0712d0j3A
   5:         0x69c01d9a - rust_begin_unwind
   6:         0x69c17489 - panicking::panic_fmt::had385503c1077d42BRv
   7:           0x48c54b - main
   8:           0x5f5dcf - main
   9:           0x5edd64 - main
  10:           0x5ec045 - main
  11:           0x5611ce - main
  12:           0x561169 - main
  13:           0x44cef9
  14:           0x44cebd
  15:           0x4470ea
  16:         0x6979f65e - Bencher::ns_per_iter::h22bba3f257e5ac85udc
  17:         0x697a87d9 - Bencher::ns_per_iter::h22bba3f257e5ac85udc
  18:         0x697a0034 - Bencher::ns_per_iter::h22bba3f257e5ac85udc
  19:         0x697a057a - Bencher::ns_per_iter::h22bba3f257e5ac85udc
  20:         0x69c2f46c - rust_try
  21:         0x69c2f449 - rust_try
  22:         0x697a0baf - Bencher::ns_per_iter::h22bba3f257e5ac85udc
  23:         0x69bf2e12 - sys::time::_&'a SteadyTime.Sub::sub::h818476a6d1b7066
augx
  24:     0x7ff86d9213d2 - BaseThreadInitThunk
thread panicked while panicking. aborting.
stack backtrace:
   1:         0x69bec62b - sys::backtrace::write::h5d8eeb6b31ece965cqu
   2:         0x69c026eb - rt::unwind::register::h49d2602058168ec5c7A
   3:         0x69b8360f - rt::unwind::begin_unwind_inner::h1772a00c8d230691N4A
   4:         0x69b83dd1 - rt::unwind::begin_unwind_fmt::h84a2c35fac0712d0j3A
   5:         0x69c01d9a - rust_begin_unwind
   6:         0x69c17489 - panicking::panic_fmt::had385503c1077d42BRv
   7:           0x48c54b - main
   8:           0x5f5dcf - main
   9:           0x5edd64 - main
  10:           0x5ec045 - main
  11:           0x5611ce - main
  12:           0x561169 - main

T:\Data\Projects\glium\target>texture_read-02be6fb63ab0151b.exe

running 3 tests
Debug message: `Buffer detailed info: Buffer object 1 (bound to GL_PIXEL_UNPACK_
BUFFER_ARB, usage hint is GL_DYNAMIC_DRAW) will use VIDEO memory as the source f
or buffer object operations.`
Debug message: `Buffer detailed info: Buffer object 1 (bound to GL_PIXEL_UNPACK_
BUFFER_ARB, usage hint is GL_DYNAMIC_DRAW) will use VIDEO memory as the source f
or buffer object operations.`
test empty_pixel_buffer ... ok
thread 'glium rendering thread' panicked at 'Debug message with high or medium s
everity: `GL_INVALID_OPERATION error generated. <framebuffer> is not a valid fra
mebuffer object.`.
Please report this error: https://github.com/tomaka/glium/issues', src\lib.rs:16
56
thread 'glium rendering thread' panicked at 'Debug message with high or medium s
everity: `GL_INVALID_OPERATION error generated. <framebuffer> is not a valid fra
mebuffer object.`.
Please report this error: https://github.com/tomaka/glium/issues', src\lib.rs:16
56
stack backtrace:
   1:         0x69bec62b - sys::backtrace::write::h5d8eeb6b31ece965cqu
   2:         0x69c026eb - rt::unwind::register::h49d2602058168ec5c7A
   3:         0x69b8360f - rt::unwind::begin_unwind_inner::h1772a00c8d230691N4A
   4:         0x69b83dd1 - rt::unwind::begin_unwind_fmt::h84a2c35fac0712d0j3A
   5:         0x69c01d9a - rust_begin_unwind
   6:         0x69c17489 - panicking::panic_fmt::had385503c1077d42BRv
   7:           0x47378b - main
   8:           0x5dd00f - main
   9:           0x5d4fa4 - main
  10:           0x5d3285 - main
  11:           0x54840e - main
  12:           0x5483a9 - main
  13:           0x457ae9
  14:           0x457aad
  15:           0x446df8
  16:         0x6979f65e - Bencher::ns_per_iter::h22bba3f257e5ac85udc
  17:         0x697a87d9 - Bencher::ns_per_iter::h22bba3f257e5ac85udc
  18:         0x697a0034 - Bencher::ns_per_iter::h22bba3f257e5ac85udc
  19:         0x697a057a - Bencher::ns_per_iter::h22bba3f257e5ac85udc
  20:         0x69c2f46c - rust_try
  21:         0x69c2f449 - rust_try
  22:         0x697a0baf - Bencher::ns_per_iter::h22bba3f257e5ac85udc
  23:         0x69bf2e12 - sys::time::_&'a SteadyTime.Sub::sub::h818476a6d1b7066
augx
  24:     0x7ff86d9213d2 - BaseThreadInitThunk
thread panicked while panicking. aborting.
stack backtrace:
   1:         0x69bec62b - sys::backtrace::write::h5d8eeb6b31ece965cqu
   2:         0x69c026eb - rt::unwind::register::h49d2602058168ec5c7A
   3:         0x69b8360f - rt::unwind::begin_unwind_inner::h1772a00c8d230691N4A
T:\Data\Projects\glium\target>texture_write-a025a483811cf67c.exe

running 1 test
thread 'glium rendering thread' panicked at 'Debug message with high or medium s
everity: `GL_INVALID_OPERATION error generated. <framebuffer> is not a valid fra
mebuffer object.`.
Please report this error: https://github.com/tomaka/glium/issues', src\lib.rs:16
56
stack backtrace:
   1:         0x69bec62b - sys::backtrace::write::h5d8eeb6b31ece965cqu
   2:         0x69c026eb - rt::unwind::register::h49d2602058168ec5c7A
   3:         0x69b8360f - rt::unwind::begin_unwind_inner::h1772a00c8d230691N4A
   4:         0x69b83dd1 - rt::unwind::begin_unwind_fmt::h84a2c35fac0712d0j3A
   5:         0x69c01d9a - rust_begin_unwind
   6:         0x69c17489 - panicking::panic_fmt::had385503c1077d42BRv
   7:           0x46497b - main
   8:           0x5ce1ff - main
   9:           0x5c6194 - main
  10:           0x5c4475 - main
  11:           0x5395fe - main
  12:           0x539599 - main
  13:           0x449309
  14:           0x4492cd
  15:           0x446f6f
  16:         0x6979f65e - Bencher::ns_per_iter::h22bba3f257e5ac85udc
  17:         0x697a87d9 - Bencher::ns_per_iter::h22bba3f257e5ac85udc
  18:         0x697a0034 - Bencher::ns_per_iter::h22bba3f257e5ac85udc
  19:         0x697a057a - Bencher::ns_per_iter::h22bba3f257e5ac85udc
  20:         0x69c2f46c - rust_try
  21:         0x69c2f449 - rust_try
  22:         0x697a0baf - Bencher::ns_per_iter::h22bba3f257e5ac85udc
  23:         0x69bf2e12 - sys::time::_&'a SteadyTime.Sub::sub::h818476a6d1b7066
augx
  24:     0x7ff86d9213d2 - BaseThreadInitThunk
thread panicked while panicking. aborting.

T:\Data\Projects\glium\target>texture-f6a19f2b5f66cd28.exe

running 6 tests
test render_to_texture2d ... ignored
test texture_1d_creation ... ok
test compressed_texture_2d_creation ... ok
test texture_2d_creation ... ok
test empty_texture2d ... ok
test texture_3d_creation ... ok

test result: ok. 5 passed; 0 failed; 1 ignored; 0 measured

Could you add a breakpoint at rust_panic in gdb and print the backtrace when it's triggered?
Or alternatively (more useful), if you have apitrace or glintercept installed, could you run it with just texture_write-a025a483811cf67c.exe so that I can see the list of the functions being called?

Sorry for taking this long to reply. I tested again with newest glium (commit f498289) and rust nightly.
rustc 1.0.0-nightly (b63cee4a1 2015-02-14 17:01:11 +0000)
Gist of running apitrace (don't know if useful)
Here is link to apitrace texture_write-490f14605af0c1bd.trace

Ugh. Sorry for wasting your time, but could you run with gdb and put a breakpoint at rust_panic?
I thought that apitrace would capture the faulty instruction, but it didn't. There's no way glGenFramebuffers could produce this error message.

It's all good. :) I'm not very familiar with gdb but I hope this is enough.
backtrace on rust_panic breakpoint

Backtrace stopped: previous frame inner to this frame (corrupt stack?)

This is problematic, and there's nothing we can do about it :-/
So, to sum up:

In texture_write, the last two calls before the error are glGenerateMipmaps and glGenFramebuffers. The next one is glNamedFramebufferTextureEXT or equivalent.
Considering that the texture tests are passing, glGenerateMipmaps is probably not at fault.
The follow-up is: glBindFramebuffer, glDrawBuffers, glBindFramebuffer, glReadBuffer (off-topic: this could be optimized), glPixelStorei, and finally glReadPixels. It would be surprising if an error triggered by glReadPixels produced such a shallow trace.
The error is probably triggered by glNamedFramebufferTextureEXT, glBindFramebuffer, glFramebufferTexture2D, glFramebufferTexture, or glFramebufferTexture2DEXT.
Since glGenFramebuffers is called, that means we have a 3.0 context or more. We can exclude glFramebufferTexture2DEXT.
All the functions above trigger an invalid operation only if the framebuffer ID is wrong.

Conclusion: I don't know!

I made some changes. Could you run apitrace again, but with the GLIUM_DISABLE_DEBUG_OUTPUT environment variable set to 1 (or any other value)?

Looks like the error messages are now more useful.
logs
traces

Thanks again for giving me these!
After some discussion on IRC, the only hypothesis as to why that happens is a bug in the nvidia drivers (which wouldn't be that surprising).
When I have some time I'm going to make some changes in another branch, so you can see if it fixes it.

Wait, I didn't read the trace correctly. There's something very weird.
Could you checkout the branch issue-423-debug and run apitrace with GLIUM_DISABLE_DEBUG_OUTPUT on texture_write again?

log
trace

Thanks!
That means that this code and this code don't show up in the trace.
The first one could be explained if attachments was empty (which would be a pretty huge bug too), but the most likely explanation is that apitrace doesn't trace calls that generate an error.
I don't think the faulty call is glBindFramebuffer, as a call to glBindFramebuffer with the same framebuffer is being traced right after. Therefore it's probably either glNamedFramebufferTexture or glNamedFramebufferTextureEXT that produces the error.

I've changed the content of the branch (by force-pushing). Could you git fetch origin, git checkout origin/issue-423-debug and try again ? :-/
I understand if you're getting tired of this. It's not a problem if you want to stop playing this back-and-forth game.

I don't mind it. I just appreciate you taking the time to look at this. We can continue as long as you're willing.
logs
trace

You used the old issue-423-debug branch instead of new one. Please delete the branch before pulling.

@isurakka Thanks for the help, this is now fixed!

Good to hear!


Move driver interface code from the Coupler to ClimaLSM

Purpose
Add Coupled Driver types to ClimaLSM (currently they are defined in ClimaCoupler). There are also methods that are defined in ClimaCoupler for these types, which this PR moves into ClimaLSM.
Lastly, we unify use of turbulent_fluxes and surface_air_density between standalone models in ClimaLSM here, which slightly changes how these are stored in the Bucket model cache.
See issue #449
To-do
[] unit tests
Content
I think self-explanatory from the description of purpose. some variables are present only in prescribed vs coupled runs, but ideally we can handle that with dispatch off of the Atmos type. (rho_sfc, e.g.)
Note that the PrescribedAtmos etc driver types have methods for update_drivers! which are called during callbacks. In coupled runs, we do not need to do any updating on the ClimaLSM side, so these methods do not exist and we would not set a callback.
Review checklist
I have:

followed the codebase contribution guide: https://clima.github.io/ClimateMachine.jl/latest/Contributing/
followed the style guide: https://clima.github.io/ClimateMachine.jl/latest/DevDocs/CodeStyle/
followed the documentation policy: https://github.com/CliMA/policies/wiki/Documentation-Policy
checked that this PR does not duplicate an open PR.

In the Content, I have included

relevant unit tests, and integration tests,
appropriate docstrings on all functions, structs, and modules, and included relevant documentation.

[ ] I have read and checked the items on the review checklist.

This looks good to me!!
Maybe we want to hold off on merging until we've had a chance to think about whether we want to include the coupled driver case in the Interpolators work? @Sbozzolo

for sure, we can hold off to be sure!
My 2c is that we dont need any interpolation for the forcing variables (atmos/radiation) in coupled simulations, so it might be confusing to have them be related to each other conceptually. We are not doing any update_drivers! or interpolations internally to climalsm with coupled runs currently - the coupler just updates the values in p.
That said, it would make sense to include this in the Interpolators work if we wanted to store the multiple values of the fluxes that the coupler gets from Atmos in coupled runs, and then interpolate to the land simulation time in between the two atmos times, and write an update_drivers! method for CoupledDrivers that does the interpolation. But I dont think this is planned.
and we wont have any update_drivers! function for these types (where the interpolations would be called). I think they might be different concepts.

This looks good to me!!
Maybe we want to hold off on merging until we've had a chance to think about whether we want to include the coupled driver case in the Interpolators work? @Sbozzolo

fwiw, it is slightly confusing to call evaluating a forcing function f(t) an interpolation if it is an analytic function, but it's probably ok since it is only internal and we can document it well. But I think forcing/driving is a separate concept than interpolating data (even if some types of forcing require interpolations).

That said, it would make sense to include this in the Interpolators work if we wanted to store the multiple values of the fluxes that the coupler gets from Atmos in coupled runs, and then interpolate to the land simulation time in between the two atmos times, and write an update_drivers! method for CoupledDrivers that does the interpolation. But I dont think this is planned.

I also don't think this is planned either - all component models are just stepped using the same timestep right now, and I think it will stay like that for a while.
I agree that the coupled vs prescribed atmos are different concepts. I'm just not sure if we want to have a uniform interface for both cases, or if we're okay with having different interfaces for the two.
Either way, I agree that the terminology of interpolate to update a forcing when interpolation is only one of the update options is perhaps misleading.

I haven’t look at this yet, but I also wanted to bring up that
interpolate is not the best name and I wanted to solicit better names
that encode evaluating analytical functions and interpolated data (I
couldn’t find anything particularly suitable).
On Thu, Jan 18, 2024 at 12:41 PM Julia Sloan @.***>
wrote:

That said, it would make sense to include this in the Interpolators work
if we wanted to store the multiple values of the fluxes that the coupler
gets from Atmos in coupled runs, and then interpolate to the land
simulation time in between the two atmos times, and write an
update_drivers! method for CoupledDrivers that does the interpolation. But
I dont think this is planned.
I also don't think this is planned either - all component models are just
stepped using the same timestep right now, and I think it will stay like
that for a while.
I agree that the coupled vs prescribed atmos are different concepts. I'm
just not sure if we want to have a uniform interface for both cases, or if
we're okay with having different interfaces for the two.
Either way, I agree that the terminology of interpolate to update a
forcing when interpolation is only one of the update options is perhaps
misleading.
—
Reply to this email directly, view it on GitHub
https://github.com/CliMA/ClimaLSM.jl/pull/453#issuecomment-1899173348,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACF6E7OSZRNI43E3ARQC6S3YPGCHLAVCNFSM6AAAAABB5WAXOSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOJZGE3TGMZUHA
.
You are receiving this because you were mentioned.Message ID:
@.***>

The point of view I am taking is the one of model developers and my goal is to ensure that model developers do not have to think about the source of the data that they need to use.
For example, with my interpolator interface, the update drivers for atmosphere will always look like:
function make_update_drivers(a::PrescribedAtmosphere{FT}) where {FT}
    function update_drivers!(p, t)
        interpolate!(p.drivers.P_liq, a.liquid_precip, t)
        interpolate!(p.drivers.P_snow, a.snow_precip, t)
        interpolate!(p.drivers.T, a.T, t)
        interpolate!(p.drivers.P, a.P, t)
        interpolate!(p.drivers.u, a.u, t)
        interpolate!(p.drivers.q, a.q, t)
        interpolate!(p.drivers.c_co2, a.c_co2, t)
    end
    return update_drivers!
end

We don't have to worry if a.T is a function, a 1D data, or 2D data.
I think we could extend this to also work with coupled runs. Internally, interpolate! will just fetch the correct information. This would unify the interface for coupled runs as well and make for a nice user/developer-experience (there's fewer types, only one way to do things, responsabilities are reasonably well segregrated, ...). There are a couple of technical challenges (+ unforseen ones) in implementing this.
In any case, what name could I pick instead of Interpolator? DataEvaluator ?

The point of view I am taking is the one of model developers and my goal is to ensure that model developers do not have to think about the source of the data that they need to use.
For example, with my interpolator interface, the update drivers for atmosphere will always look like:
function make_update_drivers(a::PrescribedAtmosphere{FT}) where {FT}
    function update_drivers!(p, t)
        interpolate!(p.drivers.P_liq, a.liquid_precip, t)
        interpolate!(p.drivers.P_snow, a.snow_precip, t)
        interpolate!(p.drivers.T, a.T, t)
        interpolate!(p.drivers.P, a.P, t)
        interpolate!(p.drivers.u, a.u, t)
        interpolate!(p.drivers.q, a.q, t)
        interpolate!(p.drivers.c_co2, a.c_co2, t)
    end
    return update_drivers!
end

We don't have to worry if a.T is a function, a 1D data, or 2D data.
I think we could extend this to also work with coupled runs. Internally, interpolate! will just fetch the correct information. This would unify the interface for coupled runs as well and make for a nice user/developer-experience (there's fewer types, only one way to do things, responsabilities are reasonably well segregrated, ...). There are a couple of technical challenges (+ unforseen ones) in implementing this.
In any case, what name could I pick instead of Interpolator? DataEvaluator ?

What you have so far makes a lot of sense (prescribed data forcing the simulation, with the same interface if it is an anlytic function, 1d data, 2d, etc). evaluate! is a nice name over interpolate!, or even update_field!
But Im not sure coupled forcing needs to fit the same pattern because it has quite different requirements (aside from being "forcing"). With coupled runs/coupled forcing data, we do not need to have an update_drivers! function at all. With prescribed forcing, we need to call update_drivers! using a callback, and this is not required in coupled runs.
I would propose that we merge this PR as it maintains the current state of how we run coupled simulations. it just moves the definition of these coupled types etc to ClimaLSM instead of ClimaCoupler. This would be a necessary step to changing how we handled coupled forcing, because moving forwards any changes we make (extending evaluate! to coupled forcing, e.g.) is internal to ClimaLSM. And then moving forwards we can figure out what is best - it would a separate PR to the GPU/interpolation work I think as well?

I do think that it would be nice to handle the coupled case since Interpolators (or whatever we name it) will handle all other atmos/radiative forcing cases. What technical challenges are you referring to? Would the coupled case add new ones, or are they more general?
I'm hesitant about DataEvaluator because in the case of an analytical function there's no data involved. Maybe just Evaluator, or Updater? But those might be too vague :/

I would propose that we merge this PR as it maintains the current state of how we run coupled simulations. it just moves the definition of these coupled types etc to ClimaLSM instead of ClimaCoupler. This would be a necessary step to changing how we handled coupled forcing, because moving forwards any changes we make (extending evaluate! to coupled forcing, e.g.) is internal to ClimaLSM. And then moving forwards we can figure out what is best - it would a separate PR to the GPU/interpolation work I think as well?

Yes, I agree.

What technical challenges are you referring to? Would the coupled case add new ones, or are they more general?

I see at least three:

If we add a fourth interpolator (CoupledInterpolator), the interpolator object will essentially have to contain pointers to the source of where we take the data from, so that we can the evaluate function just being x .= y, where y is the data independently handled by the coupler
related, this might require ClimaLSM to know more about the details of the coupler, so it might end up that the ClimaLSM will have to depend on the coupler, which depends on ClimaLSM (not possible, we would have to have a 3rd package)
it might be tricky to guarantee that when evaluate! is called the data is ready and up to date

I don't have the full implementation in mind, so I am not sure if these are really problems or not (and if there are even more problems)

I'm hesitant about DataEvaluator because in the case of an analytical function there's no data involved. Maybe just Evaluator, or Updater? But those might be too vague :/

Yes, I am not fond of any of these either


Rails 6 Support

It appears that Shopify 8.0.0 gem is not compatible with Rails 6.x.
Is there any plan for when this will be supported?
    shopify_api (>= 8.0) was resolved to 8.0.0, which depends on
      activeresource (>= 4.1.0, < 6.0.0) was resolved to 5.1.0, which depends on
        ....

I just ran a clean installation and can confirm that you use this with Rails 6 just fine. activeresource will resolve to 5.1.0 which is the latest version.


Consume HDWallet isMultiAccountEnabled

Overview
Requires https://github.com/shapeshift/hdwallet/issues/567
There's a hack in web to allow adding accounts. Remove it and ask HDWallet
References and additional details

modify this to consume from hdwallet https://github.com/shapeshift/web/pull/2702/files#diff-4b0df2c06aadf5250d39147ae27bbd1f2f5aa5482340a8baa2dd9d38568cf74fR19

Acceptance Criteria

We ask HDWallet for this rather than hard coding in the client

Need By Date
No response
Screenshots/Mockups
No response
Estimated effort
No response

done


Feature: clearCache for specific entity instead of all entities

Hi,
This a question / feature request.
Any specific reason why u clear all entities related to the service in the clearCache?
clearCache() {
this.broker.emit("cache.clean", this.name + ".*");
return this.Promise.resolve();
},
Have u considered removing just the entity being impacted by an update/delete?
I'm considering doing a PR with this feature, but would like to hear from u first.

Hi,
it is not so easy. Because if the list and find actions store the whole results in the cache. If one record changed, you need to clear these entries as well.
But you can change these logic, if you overwrite the clearCache method in your service.
module.exports = {
  mixins: [DbService],
  methods: {
    clearCache() {
        ...
    }
}
}


Add a test framework

I recommend adding some unit test framework and start creating assertions on the src files in the repo. Look into https://www.npmjs.com/package/tape or https://www.npmjs.com/package/mocha

Added Mocha, building tests as I go now.

Awesome. I would add to the .gitignore file:
node_modules/

It's common to not include this directory in the git repo so that it doesn't bloat the project. People can do an npm install to download the dependencies in the package.json file

Good point, thx!


"definitions" support

It would be great is jjv supported "definitions" (http://json-schema.org/latest/json-schema-validation.html#anchor94).
Definitions allow using a schema defined elsewhere for multiple properties. Referring to a definition is done using a JSON pointer.

jjv works correctly with definitions in my personal experience.
var subschema = { type: 'string' };
var schema = {
  type: 'object',
  definitions: { subschema: subschema },
  properties: {
    x: { $ref: '#/definitions/subschema' }
  },
  required: ['x']
};
env.validate(schema, { x: 4 });

>> { validation: { x: { type: 'string' } } }

Correct. It does Support definitions already.

Sorry, my bad


[BUG] - misleading error message for missing --fee or --ttl arguments

Internal
Area
Other   Any other topic (Delegation, Ranking, ...).
Summary
When trying to build a transaction without specifying fee (using --fee argument), the error message used to contain "fee must be specified".
Now, after merging https://github.com/input-output-hk/cardano-node/pull/4516/, the error message is Transaction _ fee not supported in Babbage, which doesn't make sense in this context.
$ cardano-cli transaction build-raw --invalid-hereafter 2033 --out-file 221101_122717084_qskb_tx.body --tx-in "c6eef2089120ed25b6b86e239a43a914e1c4f66efa1d44c688545c2ad89aef6e#0" --tx-out "addr_test1qpzluvlfcmauftc93zc3e309qwkx8zwx8gfnnr54f0ydyav9ha685w7u3ssnlpehwkz0sz7rlx8r2v7vq80zat5aatzq6nkr8l+999821387"

Command failed: transaction build-raw  Error: Transaction _ fee not supported in Babbage

EDIT:
Similar situation with missing --ttl (--invalid-hereafter) for Shelley-era transactions:
$ cardano-cli transaction build-raw --fee 178613 --out-file 221101_125508477_ibeb_tx.body --tx-in "ffdf0ecac69a6a50f1730272b832b62650db08eed4fb85b93ce9e9fce70061fc#0" --tx-out "addr_test1qq3x3ezmxvg6swp4na4w5gvnm7hfrtppuyxcrd3lrcdt5ukph6j2qr546cy2q7zlwcw227tmcgup99y7l7g7fygdtensw3tn69+999821387" --shelley-era

Command failed: transaction build-raw  Error: Transaction validity upper bound not supported in Shelley

Expected behavior
A clear message that fee is missing is printed out.
A clear message that --ttl (--invalid-hereafter) is missing is printed out.
System info:

OS Name: Fedora 36
Node version: 0e11afc07f7c5939d291d56e6bd1abd62439ad63
CLI version: 0e11afc07f7c5939d291d56e6bd1abd62439ad63

@mkoura bump


Problem: pg_yregress not collecting notices done during commit (deferred triggers)

Given a deferred trigger that may issue a notice, when we have a commit: true test, these notices are seemingly not collected by pg_yregress.
This impedes testing of the trigger.
Proposed solution: ensure we collect notices issued at commit time

Bounty size: $100
Read more about bounties

I tested with this
$schema: "https://raw.githubusercontent.com/omnigres/omnigres/master/pg_yregress/schema.json"

instance:
  init:
  - create table test(id int)
  - |
    create function raise_notice() returns trigger as $$
    begin
      raise notice 'raising notice';
      return new;
    end;
    $$ language plpgsql
  - |
    create constraint trigger test_trigger after insert on test
    deferrable initially deferred
    for each row execute function raise_notice()

tests:
- name: notices
  query: insert into test values (1)
  commit: true
  notices:
  - raising notice

The test is passing which implies notices are being collected. Is there an example where it is not able to collect notices ?
Another thing i observed is the test still passes when removing commit: true line. There is no notice raised on aborting a tx, are expected notices checked only when actual output contains notices ?

@yrashk is this still a bug ? From what i have tested doesn't seem like it.

What was your test? I don't recall fixing it yet, but I'm curious what was your test to see if the issue is defined accurately.

What was your test? I don't recall fixing it yet, but I'm curious what was your test to see if the issue is defined accurately.

https://github.com/omnigres/omnigres/issues/261#issuecomment-1809943345

Oh. I apologize, I was looking at the notification alone and not the whole issue. My bad. I'll get back to you shortly.

The best context I can provide at the mlment is this:
To the best of my recollection, I ran into this issue when I was doing a multistep test (steps)
But the exact shape is escaping me at the moment.

@yrashk multistep tests always pass even if notices don't match, could you please assign it to me

Solved in #383. 🎊 🍸


Remove old derrida playbooks and variables

These changes look fine, though looking at the new state of the inventory prompts two questions: Do we only run the 'crawl' in staging? Why do we have 2 boxes in derrida_archive_staging but only one in derrida_archive_production?

Ah, these are good questions!

The 'crawl' was used to generate the web archive that is deployed for playback on the other derrida archive servers / playbooks.  It was more of a worker machine than a true staging; it doesn't need a production equivalent, and it can probably be archived now. I'm not sure I would do it the same way if/when we do this again, but a worker machine for running the crawls is nice!

@kayiwa recommended adding a second box in production and I think agreed but we didn't get it set up yet.

I'm going to merge this, since the questions are not directly related to the changes in this PR.


Fixes #316 : set return null for HydratingResultSet::current() on no data

Provide a narrative description of what you are trying to accomplish:

[x] Are you fixing a bug?

[x] Detail how the bug is invoked currently.
call HydratingResultSet::current() on no data in current record.
[x] Detail the original, incorrect behavior.
it returns false.
[x] Detail the new, expected behavior.
it returns null, the "AbstractResultset" and "ResultSet" class returns null on no data, so it returns null for consistency/fix.
[x] Base your feature on the master branch, and submit against that branch.
[x] Add a regression test that demonstrates the bug, and proves the fix.
[x] Add a CHANGELOG.md entry for the fix.

If this is considered a BC break, I can target to develop. Fixes #316

@samsonasik yes, this is a BC break. Please send it to develop. Thanks!

@ezimuel I've rebased against develop branch and change target to develop branch.

This repository has been closed and moved to laminas/laminas-db; a new issue has been opened at https://github.com/laminas/laminas-db/issues/30.

This repository has been moved to laminas/laminas-db. If you feel that this patch is still relevant, please re-open against that repository, and reference this issue. To re-open, we suggest the following workflow:

Squash all commits in your branch (git rebase -i origin/{branch})
Make a note of all changed files (`git diff --name-only origin/{branch}...HEAD
Run the laminas/laminas-migration tool on the code.
Clone laminas/laminas-db to another directory.
Copy the files from the second bullet point to the clone of laminas/laminas-db.
In your clone of laminas/laminas-db, commit the files, push to your fork, and open the new PR.
We will be providing tooling via laminas/laminas-migration soon to help  automate the process.


[front] - fix: remove redundant outline class from button

Description
This PR aims at fixing the website's advanced settings modal.
Old:

New:

References:

https://github.com/dust-tt/tasks/issues/1532#issuecomment-2441652953

Risk
None
Deploy Plan
Deploy front

PMRR

👍


add dependabot as default plugin

I thought it may be nice to add Dependabot as default plugin. It's basically CompatHelper but for GitHub actions. See also https://github.com/JuliaCI/PkgTemplates.jl/pull/392 where I setup Dependabot for this repo.
This is my first real PR to this repo, so please tell me if there is anything else to do.

Can we break this into two PRs?
One to just add this, (which can be this one)
and another to add it as a default plugin?
Adding this is a clear good, adding it as a default might be more debatable.
and i don't want the former to be blocked on the latter.

Sure, good point. I updated the PR accordingly.


Account scan runs in a loop

I've connected my Ledger Live with my bitcoind node and satstack started to scan the accounts but runs in a loop:
...
2022-09-08T22:08:45Z [satstack] Still rescanning. At block 752734. Progress=0.998890
2022-09-08T22:09:07Z [satstack] Rescan completed in        15115194ms
2022-09-08T22:09:07Z [satstack] RescanFromTime: Rescanning last 496208 blocks
2022-09-08T22:09:07Z [satstack] Rescan started from block<PHONE_NUMBER>000029b44a34ed1e591566244a501a962e8b3dcc93f110c9470872...
2022-09-08T22:10:07Z [satstack] Still rescanning. At block 270775. Progress=0.036118
2022-09-08T22:11:07Z [satstack] Still rescanning. At block 281167. Progress=0.040967
2022-09-08T22:12:07Z [satstack] Still rescanning. At block 289515. Progress=0.045150
2022-09-08T22:12:30Z New outbound peer connected: version: 70016, blocks=753216, peer=228 (block-relay-only)
2022-09-08T22:13:07Z [satstack] Still rescanning. At block 297642. Progress=0.049267
2022-09-08T22:14:07Z [satstack] Still rescanning. At block 306354. Progress=0.053655
2022-09-08T22:15:07Z [satstack] Still rescanning. At block 314971. Progress=0.058091
2022-09-08T22:16:07Z [satstack] Still rescanning. At block 323089. Progress=0.062694

Any idea why this happens?

Hi @chuegel, not really tbh. What is your setup please?

yes, it depends how your hardware and if you don't set a birthday, it can be very long.

I just ran into this. One comment is that it would be helpful to have some messaging around this. I.e., to indicate that this process is on loop 1 of 5 (or whatever the number is). Without that information, the looping seems odd/broken to the end-user.


Accessing the webcam during testing

I'm developing a library that activates a user's webcam and analyzes the captured images.  I'd like to test that portion of my code in mocha, but I'm not sure how.  I don't imagine it will be possible in the command line (npm test).  The browser-based test runner (npm run test-browser) seems plausible, except that it loads with the file:// protocol, instead of localhost.  That means the browser prevents the code from accessing the webcam.
Any reccomendations on modifying this boilerplate to run the test runner at http://localhost/test/runner.html (or similar) instead of file:///path/to/project/test/runner.html?  Thanks.

I've tried integrating karma, as a replacement, but so far I'm not able to get tests run by karma to inject the


Fix IllegalArgumentException with non-ascii paths

Fix #1194

Looks like I have some maintenance tasks before the CI will run.

@Thrameos  I'm working on a test and it seems emoji still don't work although that appears to be a different reason.

The emoji thing seems to be a problem with java itself
 java -classpath "😊/mypackage.jar" mypackage.MyClass
Error: A JNI error has occurred, please check your installation and try again
Exception in thread "main" java.lang.IllegalArgumentException: Error decoding percent encoded characters
        at java.base/sun.net.www.ParseUtil.decode(ParseUtil.java:218)
        at java.base/jdk.internal.loader.FileURLMapper.getPath(FileURLMapper.java:64)
        ...

https://youtrack.jetbrains.com/issue/IJPL-19330/Scratch-Java-file-wont-run-if-theres-an-emoji-in-the-project-name

I am hoping it is safe to ignore emoji.   Although a coworker of mine had great fun using the poop emoji when we added internationalization to string conversions, I think that it is an edge case.
It is also a good idea to add a line to "doc/CHANGELOG.rst"  describing the change so that others will know it is fixes.  Thanks for taking the time to add tests.

Since Java appears to have difficulties with emojis in classpaths, it's probably impossible to fix this issue. The important thing is to support non-English character sets that are likely in user paths. That is the problem that prompted me to open this issue in the first place. Although a poop emoji directory does sound like a good place to store certain parts of java :D.
I've added my changes to the changelog. Let me know if its not in the right place or the wording is bad.

Honestly I do not understand right now, why the pipeline isnt converging. On the main branch everything was fine.

I fixed the hanging issue and packaged it with the related tests in test_startup.py. These are run as sub-processes using `subrun'. I'm running into an issue with them failing on Windows. I'll try to fix this later this week.

Last week I tried fixing this on windows but ran into some other issues related to UTF-8 vs UTF-16 character encoding on Windows in the JNI.

@tachyonicClock Thanks for your work on this so far! Any updates on this one? We would like to conclude a set of changes for the next release and I would like to include this fix if it also works on Windows.

Launching Java with non-ascii class paths appears impossible on Windows (https://stackoverflow.com/questions/20052455/jni-start-jvm-with-unicode-support). To resolve this issue, I tried adding the classpath after initialization since this supports Unicode. But the change to how jpype starts up is significant.
I fixed a problem in Python 3.8 with JDK8 todo with "zip file system provider returns doubly % encoded URIs" (native/java/org/jpype/pkg/JPypePackageManager.java). A pre-existing fix removed % signs if '%2520' was present. This did not fix doubly encoded Unicode characters and risked removing % signs that were intentional. The new version will work in these.
Emoji now work.
I added project/jars/unicode_à😎  to describe how to build the Unicode jar test case. I tried to follow the pattern of other examples, but I'm not confident I did this correctly.


Can't release microphone on iOS after opening audio session for recording

Hi @ryanheise ,
I have been stuck at trying to identify why I can't get the microphone to be release after stopping recording. A user can see this in the form of an orange dot on iOS devices (indicating mic is in use).
I am initialising an audio session usingaudio_session. Then I record using flutter_sound. On stop, I make sure I call  stopRecorder, closeRecorder and close and cancel all streams.
Is there a way to use AudioSession.instance.setActive(false) in the dispose call or some other way?
This is audio_session initialisation:
 void initRecorder() async {     await _createRecordingFile.openRecorder();     await _recorder.openRecorder();     final session = await AudioSession.instance;     await session.configure(AudioSessionConfiguration(       avAudioSessionCategory: AVAudioSessionCategory.record,       avAudioSessionCategoryOptions:           AVAudioSessionCategoryOptions.allowBluetooth,       avAudioSessionMode: AVAudioSessionMode.spokenAudio,       avAudioSessionRouteSharingPolicy:           AVAudioSessionRouteSharingPolicy.defaultPolicy,       avAudioSessionSetActiveOptions:           AVAudioSessionSetActiveOptions.notifyOthersOnDeactivation,       androidAudioAttributes: AndroidAudioAttributes(           contentType: AndroidAudioContentType.speech,           flags: AndroidAudioFlags.none,           usage: AndroidAudioUsage.voiceCommunication),       androidAudioFocusGainType: AndroidAudioFocusGainType.gain,       androidWillPauseWhenDucked: true,     ));   }

Are you sure it's not flutter_sound that's not releasing the microphone?

Hi Ryan,
I verified my code again and aggressively called all stop and cancel methods.
void stopAndRelease() async {     await _createRecordingFile.stopRecorder();     await _recorder.stopRecorder();     await _createRecordingFile.closeRecorder();     await _recorder.closeRecorder();     await _audioStream.close();     await _recordingDataController.close();     await _recordingDataSubscription.cancel();   }
I got this from the logs so but was not able to understand them
[VERBOSE-2:ui_dart_state.cc(198)] Unhandled Exception: PlatformException(2003329396, The operation couldn’t be completed. (OSStatus error<PHONE_NUMBER>.), null, null) #0      StandardMethodCodec.decodeEnvelope (package:flutter/src/services/message_codecs.dart:607:7) #1      MethodChannel._invokeMethod (package:flutter/src/services/platform_channel.dart:167:18) <asynchronous suspension> #2      AudioSession.configure (package:audio_session/src/core.dart:195:5) <asynchronous suspension> #3      AudioPlayerService.initAudioSession (package:sixthousandthoughts/services/audioplayer_service.dart:15:5) <asynchronous suspension>

I found this S/O answer: https://stackoverflow.com/questions/21893007/how-to-consistently-stop-avaudiosession-after-avspeechutterance
It basically says you may get this error if you try to deactivate your audio session too soon before the audio sources have been completely closed.

Thanks Ryan,
I am not sure how I can check whether it is too soon but I will review the code to reorder the close/cancel methods and check again to make sure I am waiting for them to finish.
Is it ok if I keep the issue open until then?
If you have any suggestions or have faced this in any of your work, I would be very grateful for any directions on this.

I would also check that flutter_sound also actually releases the resources before the session is deactivated. This may involve looking at flutter_sound's code. A quick test you can do is to insert a delay before deactivating the session and see if it has any effect.

Hi @ryanheise ,
It was not related to audio session plug-in . I believe it had something to do with not canceling the recording stream correctly. For now, I can see that the mic resources are being released correctly. 🤞
I will close the issue. Thank you for your help and suggestions.


tileSprite using bitmapData as texture

this worked on 2.3.0 but not on versions > 2.3.0
bmd = game.add.bitmapData(400, 100);
bmd.fill(0, 0, 0, 1);
text = game.make.text(0, 0, "hello!" );
bmd.draw(text, 0, 0);
ts=game.add.tileSprite(10, 10, 400, 100, bmd);//<-this throws a message to console (WIP?)

try to add bitmap data to cache
var bmd = game.add.bitmapData(400, 100);
game.cache.addBitmapData('myBmd', bmd);
bmd.fill(255, 255, 255, 1);
var text = game.make.text(0, 0, "hello!" );
bmd.draw(text, 0, 0);
var ts=game.add.tileSprite(10, 10, 400, 100, bmd);

console message is a leave behind console.log

it works this way, THANKYOU!!!
but about key parameter docs say:

This is the image or texture used by the TileSprite during rendering. It can be a string which is a reference to the Phaser Image Cache entry, or an instance of a RenderTexture, PIXI.Texture or BitmapData.

Should it work without game.cache.addBitmapData(...)?
thankyou again

Thanks for submitting this issue. We have fixed this, and the fix has been pushed to the dev branch.


Added Median Algorithm

feat: added median.go algorithm
test: added test cases in median_test.go for median.go

Something broke with the CI because of my twitchy finger. Please request another PR.


Set PRAGMA when using sqlite3, avoiding thousands of "database is locked" warnings in logs

Discussed in https://github.com/juicedata/juicefs/discussions/2615

Originally posted by solracsf August 28, 2022
When using Litestream, there are some parameters that can be optimized : https://litestream.io/tips/
Is there any way to pass PRAGMA parameters? Could this be an enhancement?
This would avoid lines like:
juicefs[634272] <WARNING>: Transaction succeeded after 3 tries (3.671605ms), inodes: [1], last error: database is locked [sql.go:654]

See https://github.com/mattn/go-sqlite3/blob/master/README.md#connection-string
If this is already possible, document it.

I've tested a mount with
sqlite3://juicefs.db?_journal_mode=WAL&_busy_timeout=5000&_synchronous=NORMAL
and it seem to work flawlessly.

Let me add this here that could help for a better handling of these:
https://github.com/mattn/go-sqlite3/issues/632#issuecomment-450585303

Can you add -q to ignore these warning?

Why hide them instead of avoid them? 🥴


Bookie Handle not available

Don't know if it's related to #258, since this afternoon after some of our bookies crashed unexpectedly, we can't consume messages from a specific partition of a topic. We get this error on the brokers:
ERROR - [BookKeeperClientWorker-17-1:PersistentDispatcherMultipleConsumers@316] - [persistent://fury/global/apicoremisc_listing_sort__listing_sort_api/apicoremisc_listing_sort__listing_sort_api-partition-7 / apicoremisc_listing_sort_saas_consumer] Error reading entries at 141134:25599 : Bookie handle is not available,
And lots of these on bookies:
2017-03-02 00:13:13,429 - ERROR - [BookKeeperClientWorker-22-1:LedgerFragmentReplicator$2@252] - BK error reading ledger entry: 44413 org.apache.bookkeeper.client.BKException$BKBookieHandleNotAvailableException         at org.apache.bookkeeper.client.BKException.create(BKException.java:62)         at org.apache.bookkeeper.client.LedgerFragmentReplicator$2.readComplete(LedgerFragmentReplicator.java:253)         at org.apache.bookkeeper.client.PendingReadOp.submitCallback(PendingReadOp.java:430)         at org.apache.bookkeeper.client.PendingReadOp.access$000(PendingReadOp.java:59)         at org.apache.bookkeeper.client.PendingReadOp$LedgerEntryRequest.sendNextRead(PendingReadOp.java:171)         at org.apache.bookkeeper.client.PendingReadOp$LedgerEntryRequest.logErrorAndReattemptRead(PendingReadOp.java:227)         at org.apache.bookkeeper.client.PendingReadOp.readEntryComplete(PendingReadOp.java:380)         at org.apache.bookkeeper.proto.BookieClient$2$1.safeRun(BookieClient.java:312)         at org.apache.bookkeeper.util.SafeRunnable.run(SafeRunnable.java:31)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)         at java.lang.Thread.run(Thread.java:745)
Trying to get the metadata of the mentioned ledger on the log using bookkeeper shell I get a not found error. Looks like a few ledgers dissapeared for some reason.
Any help on finding the root cause of this issue will be much appreciated. If you need more information please let me know.

Bookie handle is not available

This error is printed when the client fails to connect to a particular bookie for reading/writing. If the process is crashing/restarting, it's to be expected to see that in broker logs.

And lots of these on bookies:
2017-03-02 00:13:13,429 - ERROR - [BookKeeperClientWorker-22->1:LedgerFragmentReplicator$2@252] - BK error reading ledger entry: 44413 >org.apache.bookkeeper.client.BKException$BKBookieHandleNotAvailableException at >org.apache.bookkeeper.client.BKException.create(BKException.java:62) at or

These are probably related to the bookie auto-replication (it's very noisy on the logs). Are you running that in the same process as the bookies?

Trying to get the metadata of the mentioned ledger on the log using bookkeeper shell I get a not found error. Looks like a few ledgers dissapeared for some reason.

Are these from data ledgers and were supposed not to be deleted? Can you grep for the ledgerId in the broker/bookie logs to check when (an possibly why) it was deleted?

Yes we're running the auto-recovery in the same process as the bookies. I had to disable it, because several bookies wouldn't stop logging that exception and the CPU got to a 100%.
Checking the logs I couldn't find a reason for the deletion of the ledger. Maybe it was supposed to be deleted and in that case why it was trying to replicate unexistent ledgers.

@merlimat None of our bookies have crashed for the past 18hs and we are still getting these messages:
ERROR - [BookKeeperClientWorker-17-1:PersistentDispatcherMultipleConsumers@316] - [persistent://fury/global/apicoremisc_listing_sort__listing_sort_api/apicoremisc_listing_sort__listing_sort_api-partition-7 / apicoremisc_listing_sort_saas_consumer] Error reading entries at 141134:0 : Bookie handle is not available, Read Type Normal - Retrying to read in 63.609 seconds

After a lot of digging we found that the servers that contained those ledgers were accidentally deleted by us. The whole ensemble was removed with no chance of replication.


Fix example_async_file_sensor

We have an issue in example_async_file_sensor due to a recent merged PR: https://github.com/astronomer/astronomer-providers/pull/489.
for the integration test of FileSensorAsync in a recent, we made the following improvement

trigger FileSensorAsync task
in parallel with FileSensorAsync trigger a bash task which will wait for 10 sec and create a file example_file_async_sensor.txt in the dags folder so that the FileSensorAsync task will get time to defer
once the file is created  FileSensorAsync task would find it and mark itself a success
then delete the file with bash operator

It is working fine on local setup but on astro-cloud FileSensorAsync is not able to find a created file by bash also the delete bash operator fails with an error that the file is not found
below is a graph view of the updated dag

it is working on local but not astro-cloud
I tried adding more tasks to log the file in the /usr/local/airflow/dags folder and I can see the required file has been created at the desired place.
[2022-07-04, 17:21:36 UTC] {subprocess.py:74} INFO - Running command: ['bash', '-c', 'cd /usr/local/airflow/dags && ls']
[2022-07-04, 17:21:36 UTC] {subprocess.py:85} INFO - Output:
[2022-07-04, 17:21:36 UTC] {subprocess.py:92} INFO - __pycache__
[2022-07-04, 17:21:36 UTC] {subprocess.py:92} INFO - astronomer_dags
[2022-07-04, 17:21:36 UTC] {subprocess.py:92} INFO - example_adf_run_pipeline.py
[2022-07-04, 17:21:36 UTC] {subprocess.py:92} INFO - example_aws_nuke.py
[2022-07-04, 17:21:36 UTC] {subprocess.py:92} INFO - example_azure_test_file.txt
[2022-07-04, 17:21:36 UTC] {subprocess.py:92} INFO - example_batch.py
[2022-07-04, 17:21:36 UTC] {subprocess.py:92} INFO - example_bigquery_queries.py
[2022-07-04, 17:21:36 UTC] {subprocess.py:92} INFO - example_bigquery_query.sql
[2022-07-04, 17:21:36 UTC] {subprocess.py:92} INFO - example_bigquery_sensors.py
[2022-07-04, 17:21:36 UTC] {subprocess.py:92} INFO - example_create_eks_kube_namespace_with_role.sh
[2022-07-04, 17:21:36 UTC] {subprocess.py:92} INFO - example_databricks.py
[2022-07-04, 17:21:36 UTC] {subprocess.py:92} INFO - example_dataproc.py
[2022-07-04, 17:21:36 UTC] {subprocess.py:92} INFO - example_delete_eks_cluster_and_role_policies.sh
[2022-07-04, 17:21:36 UTC] {subprocess.py:92} INFO - example_emr_eks_containers_job.py
[2022-07-04, 17:21:36 UTC] {subprocess.py:92} INFO - example_emr_sensor.py
[2022-07-04, 17:21:36 UTC] {subprocess.py:92} INFO - example_external_task.py
[2022-07-04, 17:21:36 UTC] {subprocess.py:92} INFO - example_external_task_wait_for_me.py
[2022-07-04, 17:21:36 UTC] {subprocess.py:92} INFO - example_file_async_sensor.txt
[2022-07-04, 17:21:36 UTC] {subprocess.py:92} INFO - example_file_sensor.py
[2022-07-04, 17:21:36 UTC] {subprocess.py:92} INFO - example_gcs.py
[2022-07-04, 17:21:36 UTC] {subprocess.py:92} INFO - example_hive.py
[2022-07-04, 17:21:36 UTC] {subprocess.py:92} INFO - example_http.py
[2022-07-04, 17:21:36 UTC] {subprocess.py:92} INFO - example_kubernetes_engine.py
[2022-07-04, 17:21:36 UTC] {subprocess.py:92} INFO - example_kubernetes_pod_operator.py
[2022-07-04, 17:21:36 UTC] {subprocess.py:92} INFO - example_livy.py
[2022-07-04, 17:21:36 UTC] {subprocess.py:92} INFO - example_redshift_cluster_management.py
[2022-07-04, 17:21:36 UTC] {subprocess.py:92} INFO - example_redshift_data.py
[2022-07-04, 17:21:36 UTC] {subprocess.py:92} INFO - example_redshift_sql.py
[2022-07-04, 17:21:36 UTC] {subprocess.py:92} INFO - example_s3.py
[2022-07-04, 17:21:36 UTC] {subprocess.py:92} INFO - example_s3_test_file.txt
[2022-07-04, 17:21:36 UTC] {subprocess.py:92} INFO - example_snowflake.py
[2022-07-04, 17:21:36 UTC] {subprocess.py:92} INFO - example_wasb_sensors.py
[2022-07-04, 17:21:36 UTC] {subprocess.py:92} INFO - master_dag.py
[2022-07-04, 17:21:36 UTC] {subprocess.py:92} INFO - nuke-config.yml
[2022-07-04, 17:21:36 UTC] {subprocess.py:96} INFO - Command exited with return code 0
[2022-07-04, 17:21:37 UTC] {taskinstance.py:1395} INFO - Marking task as SUCCESS. dag_id=example_async_file_sensor, task_id=list_file, execution_date=20220704T172127, start_date=20220704T172136, end_date=20220704T172137
[2022-07-04, 17:21:37 UTC] {local_task_job.py:156} INFO - Task exited with return code 0
[2022-07-04, 17:21:37 UTC] {local_task_job.py:273} INFO - 0 downstream tasks scheduled from follow-on schedule check

Also, the delete file task is throwing the below error which is weird
[2022-07-03, 03:06:32 UTC] {subprocess.py:62} INFO - Tmp dir root location:
 /tmp
[2022-07-03, 03:06:32 UTC] {subprocess.py:74} INFO - Running command: ['bash', '-c', 'rm /usr/local/airflow/dags/example_file_async_sensor.txt']
[2022-07-03, 03:06:32 UTC] {subprocess.py:85} INFO - Output:
[2022-07-03, 03:06:32 UTC] {subprocess.py:92} INFO - rm: cannot remove '/usr/local/airflow/dags/example_file_async_sensor.txt': No such file or directory
[2022-07-03, 03:06:32 UTC] {subprocess.py:96} INFO - Command exited with return code 1
[2022-07-03, 03:06:32 UTC] {taskinstance.py:1889} ERROR - Task failed with exception
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/airflow/operators/bash.py", line 194, in execute
    raise AirflowException(
airflow.exceptions.AirflowException: Bash command failed. The command returned a non-zero exit code 1.

If my assumption is correct the dags folder is shared across the container then this might be some issue with astro-cloud

The dag folder in our cloud infra is not shared across container
Any file created by a task gets clean up after the task is finished for this reason the delete task is failing
Revering the PR => https://github.com/astronomer/astronomer-providers/pull/501


Added robotnik_sensors to indigo/distribution

Please, tell me that everything is correct xD

Looks good.

Yay!


Request: Add test-state-machine-client to ic-nix as part of the packaged binaries

Hi @ninegua
I am referring to this new binary. Thoughts?

I don't think it is a widely used utility, so I'm hesitant to include it as part of the distribution, because we can't include everything in ic-nix build. It also looks like a straight-forward rust repo that can be built using rustup or cargo. So if you want the binary, you can go ahead a build it yourself.

I'm using this shell.nix to add the additional packages I need to my dev environment and then using direnv to setup a dev environment.
{ pkgs ? import <nixpkgs> { } }:
let
  dfx-env = import (fetchTarball https://github.com/ninegua/ic-nix/releases/latest/download/dfx-env.tar.gz) { version = "20230101"; };
in
dfx-env.overrideAttrs (old: {
  nativeBuildInputs = old.nativeBuildInputs ++ [ pkgs.rustup pkgs.pkg-config pkgs.openssl pkgs.protobuf pkgs.cmake pkgs.cachix pkgs.killall ];
})

How would I add the prebuilt binary for test-state-machine-client package to this and make the binary available in the environment?
I believe this package should be added to ic-nix since it's the official successor to ic-state-machine-tests which is the accepted way to perform integration testing for Rust projects and it's part of the core set of tools one would expect to be present for IC development using Rust

If you are looking for ways of including rust packages in your nix setup, you can start from this guide https://ryantm.github.io/nixpkgs/languages-frameworks/rust/. Your above shell.nix is already a good demonstration that not every dependencies for your own setup have to come from ic-nix itself.

Hi @ninegua
The thing is, this is this is not a rust package. It is a IC specific binary like ic-repl or candid. It would make a lot of sense for this binary to be available as part of ic-nix.
For example, the constructor for the state machine here requires the binary to be present for the library to be initialized.
The internet-identity repo uses it like this:
https://github.com/dfinity/internet-identity/blob/e9cbaae4f52e65abd9e7f00aa622da74b88437c3/src/canister_tests/src/framework.rs#L125-L151
It requires the binary to be present in the path.
With ic-nix, this could really improve usability if this binary were made available as part of the Nix shell environment.
Thoughts?

I think I finally understood what you are asking for. This https://github.com/dfinity/test-state-machine-client is just a library, and the actual binary is built as part of https://github.com/dfinity/ic repository.
Also, according to this Cargo.toml, the binary is call ic-state-machine-tests instead. Probably the internet-identity repo is using it under a different name?

@ninegua So, ic-state-machine-tests is an older dependency that was needed to be built from the dfinity/ic repo directly by referencing a commit hash. This new library/binary combo provides the binary in a prebuilt format. And it has all its dependencies updated to use more modern counterparts. Here is a thread from the forums that summarizes this:
https://forum.dfinity.org/t/ic-state-machine-tests-has-a-lot-of-outdated-dependencies/17909
Please add the binary to ic-nix as this would alleviate a lot of the headache and then we can just reference the binary using an environment variable as illustrated in this code here:
https://github.com/dfinity/internet-identity/blob/main/src/canister_tests/src/framework.rs#L125-L151

@ninegua Thank you so much. I will try this and come back in case any issues

@ninegua After updating to the above, this is what I am getting:

This is my shell.nix
{ pkgs ? import <nixpkgs> { } }:
let
  ic-nix = pkgs.fetchFromGitHub {
    owner = "ninegua";
    repo = "ic-nix";
    rev = "0c8c6808a88d650b5546a60a073853f6ceae97cc";
    sha256 = "sha256-MiK4vw4TLjXN3VH24+yeUvV7Qy2zbtKr5io/lafXjVA=";
  };
  dfx-env = pkgs.callPackage "${ic-nix}/dfx-env.nix" {
    force = true;
    inherit pkgs ic-nix;
  };
in
dfx-env.overrideAttrs (old: {
  nativeBuildInputs = old.nativeBuildInputs ++ [ pkgs.rustup pkgs.pkg-config pkgs.openssl pkgs.protobuf pkgs.cmake pkgs.cachix pkgs.killall ];
})

@ninegua I am also running into this error and unable to start dfx when I revert to my older shell.nix
Please advise on how to proceed

Hi @ninegua
I also tried specifying the newer version in my old shell.nix file like this:
{ pkgs ? import <nixpkgs> { } }:
let
  dfx-env = import (fetchTarball https://github.com/ninegua/ic-nix/releases/latest/download/dfx-env.tar.gz) { version = "20230314"; };
in
dfx-env.overrideAttrs (old: {
  nativeBuildInputs = old.nativeBuildInputs ++ [ pkgs.rustup pkgs.pkg-config pkgs.openssl pkgs.protobuf pkgs.cmake pkgs.cachix pkgs.killall ];
})

This upgraded me to DFX v0.13.1 but now builds are breaking with this message.

And this has broken my local dev environment. How do I go back to using the earlier ic-nix version. I tried simply updating the version to the older version but my build environment is still using the newer dfx version.
Please help

@ninegua After updating to the above, this is what I am getting:
This is my shell.nix
{ pkgs ? import <nixpkgs> { } }:
let
  ic-nix = pkgs.fetchFromGitHub {
    owner = "ninegua";
    repo = "ic-nix";
    rev = "0c8c6808a88d650b5546a60a073853f6ceae97cc";
    sha256 = "sha256-MiK4vw4TLjXN3VH24+yeUvV7Qy2zbtKr5io/lafXjVA=";
  };
  dfx-env = pkgs.callPackage "${ic-nix}/dfx-env.nix" {
    force = true;
    inherit pkgs ic-nix;
  };
in
dfx-env.overrideAttrs (old: {
  nativeBuildInputs = old.nativeBuildInputs ++ [ pkgs.rustup pkgs.pkg-config pkgs.openssl pkgs.protobuf pkgs.cmake pkgs.cachix pkgs.killall ];
})

Which NixOS version are you using?

Hi @ninegua
I also tried specifying the newer version in my old shell.nix file like this:
{ pkgs ? import <nixpkgs> { } }:
let
  dfx-env = import (fetchTarball https://github.com/ninegua/ic-nix/releases/latest/download/dfx-env.tar.gz) { version = "20230314"; };
in
dfx-env.overrideAttrs (old: {
  nativeBuildInputs = old.nativeBuildInputs ++ [ pkgs.rustup pkgs.pkg-config pkgs.openssl pkgs.protobuf pkgs.cmake pkgs.cachix pkgs.killall ];
})

This upgraded me to DFX v0.13.1 but now builds are breaking with this message.
And this has broken my local dev environment. How do I go back to using the earlier ic-nix version. I tried simply updating the version to the older version but my build environment is still using the newer dfx version.
Please help

This particular error message doesn't look like related to ic-nix. Please check your own program to make sure you didn't have anything like principal "", which is not allowed.

Which NixOS version are you using?

I am on NixOS unstable

The following error seems to be an open issue on "unstable" https://github.com/Nix-QChem/NixOS-QChem/issues/262#issuecomment-1458263684
error: The ‘env’ attribute set cannot contain any attributes passed to derivation. The following attributes are overlapping: NIX_CFLAGS_COMPILE

To fix this, you will need to pin nixpkgs to nixos-22.11 channel. Here is one way of doing this:
nix-channel --add https:///nixos.org/channels/nixos-22.11
nix-channel --update nixos-22.11

This will download a copy of the latest nixos-22.11 to your computer. Then with the new version of the shell.nix as you gave here, modify the first line to be:
{ pkgs ? import <nixos-22.11/nixpkgs> { } }:

Then you may try running nix-shell and see if this pinning works for you.

Hi @ninegua
The nixpkgs is not found as shown in the error. What is the right way to set the $NIX_PATH?

Hi @ninegua
I have tried pinning the nixpkgs to nixos-22.11 as you suggested but I'm still seeing this error:

My shell.nix looks like this now:
{ pkgs ? import <nixos-22.11/nixpkgs> { } }:
let
  dfx-env = import (fetchTarball https://github.com/ninegua/ic-nix/releases/latest/download/dfx-env.tar.gz) { version = "20230101"; };
in
dfx-env.overrideAttrs (old: {
  nativeBuildInputs = old.nativeBuildInputs ++ [ pkgs.rustup pkgs.pkg-config pkgs.openssl pkgs.protobuf pkgs.cmake pkgs.cachix pkgs.killall ];
})

Please change the dfx-env = ... line to the following
  ic-nix = pkgs.fetchFromGitHub {
    owner = "ninegua";
    repo = "ic-nix";
    rev = "0c8c6808a88d650b5546a60a073853f6ceae97cc";
    sha256 = "sha256-MiK4vw4TLjXN3VH24+yeUvV7Qy2zbtKr5io/lafXjVA=";
  };
  dfx-env = pkgs.callPackage "${ic-nix}/dfx-env.nix" {
    force = true;
    inherit pkgs ic-nix;
  };

@ninegua The above is working for me now. Just an added suggestion.
The binary path is currently made available using the env variable DFX_IC_STATE_MACHINE_TESTS_PATH
However, the II repo uses this env variable for referencing to it: STATE_MACHINE_BINARY
as seen here
Do you think this package should use the same? Or should the II repo update theirs. I believe the name you set is more verbose and self-explanatory and would lead to a lesser chance of collision with another variable set.
Anyway, please feel free to close this issue.
Thank you for patiently walking through the issue with me and helping reach a resolution. :)

Hi @ninegua
Please update ic-test-state-machine-client to the latest version v2.1.0 as mentioned here:
https://github.com/dfinity/test-state-machine-client/issues/13#issuecomment-1506833532
Thank you


Lock first column from editing

Is is possible to have cells editable but lock  first column?
I was wondering if rowMarkers could be custom because I'd like to use those as "row headers". If that's not possible, is it possible to change theme on first column to match headers and disable editing for the column?

If you want custom rowMarkers and to keep row selection behavior when clicking on them, I would use something like patch-package to customize their rendering.


support removing service from mesh

https://github.com/istio/istio/issues/15134
provide corresponding remove command.
[ ] Configuration Infrastructure
[ ] Docs
[ ] Installation
[ ] Networking
[ ] Performance and Scalability
[ ] Policies and Telemetry
[ ] Security
[ ] Test and Release
[ X] User Experience
[ ] Developer Infrastructure

@irisdingbj: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

integ-mixer-k8s-presubmit-tests-master
2bdb030caefc4abbf390a6b0141584d723b36689
link
/test integ-mixer-k8s-presubmit-tests-master

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.


process datamodel xml flh in codegen

process the datamodel xml field level help during codegen in order to generate the docstrings for the generated code

Hi @mkundu1

is this still pending?
is this something we should do?


Scourge + Ghastly Jewels

Ghastly jewels can roll the property
"15-20% increased minion damage if you've used a Minion skill recently"
There needs to be a tick or some check to "Have used a minion skill recently" so that the 20% per jewel is applied through the Scourge's "Increases and Reductions to Minion Damage also affect you"
Also idk if the ability is working or not in PoB cause it doesn't seem that damage goes up that much, the minion damage % is treated as a global damage increase right?

But there is already a "Have you used a Minion Skill Recently?" tick.

And you might just already have a ton of increased damage on your tree/gear, so an additional 20% isn't that significant of an increase.

I have around 335% increased minion damage with the build in PoB, with another 10 Abyss jewels setup with 20% minion damage.
Also with a "extra damage as ___" stat stick just for the highest possible damage this could pull off. And full build its saying like 100k dps

You do realize that extra damage as ELEMENT does NOT count for your minions and only counts from phys damage you (or your totems) inflict?

I am not using minions really at all, I am using the Scourge claw which changes Minion Damage increases and reductions into a Global Damage modifier for the player.

Then with what minion skill to you trigger the "blah blah minion skill recently"?

Summon Ice Golem

You might wanna choose something else. As "recently" refers to the last 4 seconds. So you can't keep that buff up. But for your problem. Can you please post a pastebin with your build where this won't work?

https://pastebin.com/4DmEyKtb

http://prntscr.com/ix8o4q
As you can see the function for the "minion skill recently" is in the config tab and activated.
If you really don't see it your PoB has to be outdated.
btw. ticking that off will cost you 9k dps... so you can ignore that completely

Everything seems to be working correctly here; the Minion Skill used Recently option is present and functioning, and all the calculations appear to be correct.


Allow passing a closure for users.

Hi,
Thank you for your package.
By allowing a closure for users, we can let the dev directly fetch the users from the DB, or interact with the app:
FilamentDeveloperLoginsPlugin::make()
     ->users(fn () => User::pluck('email', 'name')->toArray())

Thanks,
Mathieu.

Thanks for your PR, my only gripe with this is that the query is always executed, even when the package is disabled. We don't want users of the package to retrieve all database users on production environments. Can you add the enabled() check to the getUsers() method so we can prevent this?

Ah yes, you are correct. Can you document this feature in the readme?

Done!


 I think it deserves the same tile-based-cache as background/window/spritesheet. Thus this is only rudimentary "cache all"

Closed by https://github.com/fengb/fundude/commit/d259b7e2e9ddcb20523475cddfdad6a67a735787
Super simple dirty checks actually improved render time by a lot. Marking this resolved and moving the hard stuff to other issues.


httpCredentials with $variable is not working

When using the following code, the credential variables are not resolved and the variable name is used instead of actual value:
    ${user}    Set Variable    myuser
    ${pw}    Set Variable    mypassword
    New Browser    chromium  headless=false
    New Context    httpCredentials={'username': '$user', 'password': '$pw'}

I tested this by temporarily disabling _mask_credentials function.

Credentials are also hidden from the logs, is there something else that would indicate that credentials are not working correctly?

For the site I am testing it is not working unless I am using old way, i.e. httpCredentials={'username': '${user}', 'password': '${pw}'}
That's why I disabled the _mask_credentials function from playwright_state.py file, so that to see what's wrong. In this way you can see that actually the password used by New Context is literally '$pw' and not 'mypassword' as expected.

I looked the code and could not see anything obvious.  But there is hole in our test and they do not ensure that we actually pass the argument value to Playwright side. Need to investigate this more deeply.

@all-contributors please add @adrian-evo  for code bugs

Found the bug, good catch.


Fix ValueError when inputting an integer to disk properties

Related to Issue #593.

Codecov Report

Merging #594 into master will decrease coverage by 0.00%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master     #594      +/-   ##
==========================================
- Coverage   88.78%   88.77%   -0.01%
==========================================
  Files          24       24
  Lines        5153     5150       -3
==========================================
- Hits         4575     4572       -3
  Misses        578      578

Impacted Files
Coverage Δ

ross/disk_element.py
97.77% <100.00%> (ø)

ross/rotor_assembly.py
88.38% <0.00%> (-0.04%)
:arrow_down:

ross/api_report.py
84.28% <0.00%> (-0.03%)
:arrow_down:

ross/stochastic/st_rotor_assembly.py
92.99% <0.00%> (+0.03%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 54a4792...cdd2ff2. Read the comment docs.


internal/redesign deploy with .env, CMD, entrypoint and where to set which variables

Some variables are in dspace-ui.json, but are also set through .env. Some variables in docker composes are set several times and are not clear. Needs to be fixed and unified in some sendible way. Brief analysis should be first, of course.

All variables used in docker compose(s)
INSTANCE
TIMEZONE
REST_URL
UI_URL
S3 related (not necessary to list those)
DSPACE_REST_IMAGE
DSPACE_REST_HOST (only in FE)
DSPACE_UI_IMAGE
Variables that could be added
SSL/NONSSL (in FE)
ports (in FE)

also move changes into branch customer/TUL


Chore: [M3-6828]: Cloud Manager Linting POC

Description 📝
This POC implements the new linting for the package/manager codebase. Eventually we can extend the config to other packages but this is a start. In essence it does nothing but warn about linting improvements and commits those improvements to a few files as an example. It could be merged as is if everyone is comfortable with it, and engineers will be responsible to fix the linting suggestions in the files they're working in (one shortcut away!), or ignore them until we lint the codebase as a whole, or at least portions of it.
Let's recap the benefits:

Readability: Finding declarations in a sorted, large list is a little faster. Remember that you read the code much more often than you write it.
Maintainability: Sorting imports and properties is considered a good practice in software development, contributing to code quality and consistency across the codebase.
Code Review and Collaboration: If you set rules that say you can only do things one way, then no one will have to spend time thinking about how to do it.
Code Uniformity: When all code looks exactly the same, it is very hard to see who wrote it, which makes achieving the lofty goal of collective code ownership easier.
Aesthetics: This not only provides functional benefits, but also gives the code an aesthetic appeal, visually pleasing and harmonious structure. Take your code to the beauty salon!

Structure
The linting alphabetizes everything, from type & interfaces (sorry @bnussman it's a pure asc sorting for those, no grouping of functions but also it makes sense to keep it that way), to imports & exports, arguments and arrays element. The plugin is extremely smart at not sorting such instances of spread properties and keeps comments very much aligned with their relevant code.
The most customization is done to the grouping of imports which is a very prominent section of the file structure, outlined in the screenshot before for a file with many imports). I do like the spacing between groups as it helps locating import sections easily. It's all customizable, to a certain extent, and I spent a fair amount of time thinking about those and configuring groups, but it's not perfect, nor will it ever be. It's also quite subjective.

Testing
I recommend pulling this code, update your dependencies and open a variety of files to get a holistic sense of the experience developing, updating and writing code while getting those suggestions.
Next steps
We have a fe options as mentioned above

Merge this as it and update as we go
Lint the whole thing and make it an error instead of warning (I'd like to keep it a warning but it also means we have to be diligent either linting on our PRs or respecting the rules while developing
Lint only portions of the codebase at a time to be careful not to introduce potential bugs the plugin could introduce

Thanks for the feedback so far! FWIW, I have not had any performance issue with my VScode during linting but that's a valid concern. My take is that if it becomes a problem we can always turn everything back off in the future and go back to business as usual.
A couple outstanding questions remain, which I'd like to make decisions on, for which I'd like @jaalah-akamai to be the mighty ruler:

Types import: they sit at the top (which is the default config) and I find it to be good for me, but @bnussman-akamai has objections.
Deploy: sweeping change VS incremental changes. Both have pros & cons. I like the sweeping change as well, but only we have the confidence in our regression tooling, because the diff will be so big it will be impossible to spot a possible issue. After spending time with the plugin it looks like it's smart enough to do everything right, but who knows what could happen that wouldn't be spotted by either unit or e2e suites. All in all i want to do what's the least distracting to the team. Bonus point for a sweep: @bnussman-akamai is going away for a week so there will be twice less PRs with merge conflict 🤣

Can we hold on merging until this PR merges ? Thinking we could avoid merge conflicts.


Polish DefaultTaskOperationDescriptor

Fixes  #21983

@bot-gradle test RFM

OK, I've already triggered the following builds for you:

PullRequestFeedback build

@bot-gradle test and merge

Your PR is queued. See the queue page for details.

OK, I've already triggered a build for you.

Pre-tested commit build failed.

Your PR is queued. See the queue page for details.

OK, I've already triggered a build for you.


remove old page title presumably from other project

https://github.com/altafino/react-webpack-5-tailwind-2/blob/950dcfda92acd89c36f26b196b0456df31922409/config/webpack.common.js#L44

Tks @brumm , Issue is fixed

Tks @brumm , Issue is fixed


Fixes the Broken link in blog: k8s.gcr.io Redirect to registry.k8s.i…

Fixes issue Broken link in blog: k8s.gcr.io Redirect to registry.k8s.io - What You Need to Know #40071

/retitle Fix broken link in blog: k8s.gcr.io Redirect to registry.k8s.io - What You Need to Know


V3.3.0

Release a new minor version of datascience-python with civisml-extensions. To be merged after https://github.com/civisanalytics/civisml-extensions/pull/13.

Holding off on this release until the new version of the API client.


Model improves worse with GradientTape than with fit()

Click to expand!
Issue Type
Bug
Source
binary
Tensorflow Version
tf 2.9.1
Custom Code
Yes
OS Platform and Distribution
Linux Ubuntu 22.04.1
Mobile device
No response
Python version
3.8 & 3.10
Bazel version
No response
GCC/Compiler version
No response
CUDA/cuDNN version
No response
GPU model and memory
No response
Current Behaviour?
I already opened a question on stackoverflow for this, but I believe it could be some issue with tensorflow.

https://stackoverflow.com/questions/73440964/model-not-improving-with-gradienttape-but-with-model-fit

The same model using either custom training with GradientTape or keras' model.fit() will perform differently. The custom run, will stop improving earlier and will take longer to improve at all.

I've also included a Google Colab notebook in order to reproduce the issue:
https://colab.research.google.com/drive/1pk66rbiux5vHZcav9VNSBhdWWIhQM-nF?usp=sharing

Standalone code to reproduce the issue
def build_model(kernel_regularizer=l2(0.0001), dropout=0.001, recurrent_dropout=0.):
    x1 = Input(62)
    x2 = Input((62, 3))

    x = Embedding(30, 100, mask_zero=True)(x1)
    x = Concatenate()([x, x2])

    x = Bidirectional(LSTM(500,
                           return_sequences=True,
                           kernel_regularizer=kernel_regularizer,
                           dropout=dropout,
                           recurrent_dropout=recurrent_dropout))(x)

    x = Bidirectional(LSTM(500,
                           return_sequences=False,
                           kernel_regularizer=kernel_regularizer,
                           dropout=dropout,
                           recurrent_dropout=recurrent_dropout))(x)

    x = Activation('softmax')(x)

    x = Dense(1000)(x)
    x = Dense(500)(x)
    x = Dense(250)(x)
    x = Dense(1, bias_initializer='ones')(x)

    x = tf.math.abs(x)
    return Model(inputs=[x1, x2], outputs=x)

optimizer = Adam(learning_rate=0.0001)

model = build_model()
model.compile(optimizer=optimizer, loss='mse', metrics='mse')

options = tf.data.Options()
options.experimental_distribute.auto_shard_policy = AutoShardPolicy.DATA
dat_train = tf.data.Dataset.from_generator(
    generator= lambda: <load_function()>
    output_types=((tf.int32, tf.float32), tf.float32)
)
dat_train = dat_train.with_options(options)

# keras training
model.fit(dat_train, epochs=50)

# custom training
for epoch in range(50):
    for (x1, x2), y in dat_train:
        with tf.GradientTape() as tape:
            y_pred = model((x1, x2), training=True)
            loss = model.loss(y, y_pred)
        grads = tape.gradient(loss, model.trainable_variables)
        model.optimizer.apply_gradients(zip(grads, model.trainable_variables))

Relevant log output
No response

I have found the solution, it was a simple shape mismatch, which was somehow not picked up by any error check and worked both with my custom loss function and MSE. Using x = Reshape(())(x) as final layer did the trick.


Add more components to 2.0.1 manifest and test manifests

Signed-off-by: Zelin Hao<EMAIL_ADDRESS>Description
Add more components to 2.0.1 manifest and test manifests.
After this PR only two OpenSearch plugins (Security, Cross cluster replication) are missing because of https://github.com/opensearch-project/OpenSearch/issues/3554 with Core and security plugin.
Two dashboards plugins need to be added later as well.
https://github.com/opensearch-project/index-management-dashboards-plugin/pull/202
https://github.com/opensearch-project/security-dashboards-plugin/pull/1010
Issues Resolved
Part of #2165
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Codecov Report

Merging #2178 (df7714c) into main (7ad040e) will increase coverage by 5.43%.
The diff coverage is n/a.

@@              Coverage Diff              @@
##               main     #2178      +/-   ##
=============================================
+ Coverage     94.56%   100.00%   +5.43%
=============================================
  Files           204         6     -198
  Lines          4253       105    -4148
  Branches         29        19      -10
=============================================
- Hits           4022       105    -3917
+ Misses          225         0     -225
+ Partials          6         0       -6

Impacted Files
Coverage Δ

src/build_workflow/builders.py

src/system/os.py

tests/jenkins/jobs/DownloadFromS3_Jenkinsfile

src/test_workflow/test_recorder/log_recorder.py

src/run_assemble.py

src/run_checkout.py

src/test_workflow/integ_test/distribution_rpm.py

src/test_workflow/test_result/test_result.py

...eg_test/integ_test_runner_opensearch_dashboards.py

src/ci_workflow/ci_check_list_dist.py

... and 188 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7ad040e...df7714c. Read the comment docs.

I think without security plugin we expect to see build and test failures

Security and CCR plugins have been added here! :D
https://github.com/opensearch-project/security/pull/1881

Ohh I read this and assumed they aren't
After this PR only two OpenSearch plugins (Security, Cross cluster replication) are missing because of https://github.com/opensearch-project/OpenSearch/issues/3554 with Core and security plugin.


Deadlock problem

The TM open transaction calls an RM, and the RM operates three table update operations. The data with the two table operations has the same primary key id.
After the first request is successful, the second request will report a lock exception.

Confirmed.
Triggering conditions:
1.store.mode=file
2.Superimposed spring @Transaction
3.Different tables with the same pk

fixed.
release in 090


Add internationalization support

The DisplayAttribute and other non-ValidationAttributes are not localised.
It would be nice to have something to make it available.

Hi @Levvy055 ,
I tested that the formgenerator works fine with translated DataAttributes and ValidationAttributes. Just add some resources, and follow (this)[https://docs.microsoft.com/en-us/aspnet/core/fundamentals/localization?view=aspnetcore-5.0].
       [Display(ResourceType = typeof(Resources.Address), Name = nameof(Resources.Address.FIRSTNAME_LABEL))]
        // Add element to row 1 with automatic width based on number of items in a row
        [VxFormElementLayout(RowId = 1)]
        public string SurName { get; set; }
        [Display(Name = "Street")]
        // Add element to row 2 and set the width to 9 of 12 columns
        [VxFormElementLayout(RowId = 2, ColSpan = 9)]
        [MinLength(5, ErrorMessage = nameof(Resources.Address.FIRSTNAME_LABEL), ErrorMessageResourceType = typeof(Resources.Address))]
        public string Street { get; set; }

See image, where Firstname is translated to Voornaam in Dutch.

if you have any other remark, please let me know

Hi @Levvy055 ,
I tested that the formgenerator works fine with translated DataAttributes and ValidationAttributes. Just add some resources, and follow (this)[https://docs.microsoft.com/en-us/aspnet/core/fundamentals/localization?view=aspnetcore-5.0].
       [Display(ResourceType = typeof(Resources.Address), Name = nameof(Resources.Address.FIRSTNAME_LABEL))]
        // Add element to row 1 with automatic width based on number of items in a row
        [VxFormElementLayout(RowId = 1)]
        public string SurName { get; set; }
        [Display(Name = "Street")]
        // Add element to row 2 and set the width to 9 of 12 columns
        [VxFormElementLayout(RowId = 2, ColSpan = 9)]
        [MinLength(5, ErrorMessage = nameof(Resources.Address.FIRSTNAME_LABEL), ErrorMessageResourceType = typeof(Resources.Address))]
        public string Street { get; set; }

See image, where Firstname is translated to Voornaam in Dutch.

if you have any other remark, please let me know

I'm closing this because internationalization is not an issue related to the component (at this moment).

I'm closing this because internationalization is not an issue related to the component (at this moment).


Quantized ops?

In the data types section, there are quantized ops
http://tensorflow.org/resources/dims_types.md#data_types
Found a note here:
https://github.com/tensorflow/tensorflow/blob/d6357a5849db980df51d00d8a9ff874cda2faeb3/tensorflow/cc/ops/const_op.h#L65
Can you explain what these are?

We are working on support for operators and kernels that take quantized (fixed-point) data -- we're still working on the right APIs to expose them, and with that we'll have more documentation on why and how they are used.  Stay tuned!

Awesome, this is great work :open_mouth:
Please give my sincere thanks to :santa:

Closing for now. When the quantized ops come online, they'll come with documentation.

@vrv Any news on fixed-point / quantized data type support?
I am designing an FPGA-based accelerator for CNNs as my master thesis in the next few months. It would be awesome to be able to train the models with fixed-point weights and activations.
Do you have any roadmap for fixed-point support?

@dgschwend have you find the answer?

@vrv could you please repoen, point to the correct doc, or confirm that is still should be considered internal (though it is exposed in many place by now)


Add expanse support

Description
add Expanse (EXP) support

Homepage: https://www.expanse.tech/
Block Explorer : https://gander.tech
API node: https://api.gander.tech/
BIP44 40 ( satoshilabs/slips#125 )
Chainid 2 (https://chainid.network/)
Ticker: EXP
CMC: https://coinmarketcap.com/currencies/expanse/

Testing instructions
./bootstrap.sh or make -C build tests
Types of changes
Expanse support added

sorry, it seems that expanse doesn't meet our requirements, close for now

@hewigovens Because of docs? Or what?


Add inline-display extension

please see the discussion on the LV2 email list for details

updated with falkTX' comment. see #8


feat(core): Implement parsing for enrollment transaction

Closes #414

Both unit and integration tests are fine locally. I am not sure why CircleCI is complaining.

300 points dauTT
100 points snowypowers for review

https://neoscan.io/transaction/c382c87ce7282664dc849fef5cb8c8de22cf3b596dd1b4e85c87efb194bb2003


switching to OS mode without reason

At first, many, many thanks for your time and your effort in this project.
I have the problem, that the WomoLIN box is loosing the WiFi connection to router, althrough it is approx. 10cm away. After this it is rebooting in "OS mode"
So I have two questions:
a) Does the check box "LAN support" also works on the WomoLIN-box. Doesn't seam to work for me.
b) Will the box restart in "Normal mode" after repowering? In this case I can program a relay to interrupt the power supply via VRM in case this happens, while I'm not in the camper.
Many thanks for your time and best wishes
Carsten
https://paulette.die-oswalds.net/umbauten/inet-box-alternative

Addendum: Release is 2.6.4

Hi Carsten,
I'm pleased that you like the software. I developed the LAN configuration especially for Jan's board, which he provided me with for this purpose. However, you definitely need to change the version so that the connection errors can be rectified. In the end, I had a very stable connection. Please make sure that you specify the correct hardware, see the instructions. When rebooting, the box starts in "Normal mode". Only if there is no MQTT connectivity to the broker over 10 reboots does the fallback to OS mode take place. In most cases, the credentials were then set incorrectly.
I like your article and I'm very pleased that there is also an Openhab version. If you would like to publish the connection, you are welcome to do so via the repo. If you have any other questions, please don't hesitate to contact me.
Magnus

Hello Magnus,
I would like to update to the release 2.6.5 , but I have the WomoLIN hardware.
I'ved tried it with the [flash_esp32_inetbox2mqtt_v265_4M.bin.zip] and I can connect to the WLAN and the MQTT broker, but not to the TiN-interface.
The [flash_womolin_inetbox2mqtt_v264_4M.bin.zip] is reporting it's release 2.64, as predictable.
Do you have a 2.6.5 version for WomoLIN hardware?
I hope it's not to hard to update this branch.
In the moment, I spepped back to WomoLIN 2.6.4 this works, als long as thr router and the VenusOS don't get a restart.
Many thanks in advance.
Carsten

Hello Carsten,
sorry that you are running into problems.

Option:
If you want to use a LAN connection, V2.6.4 should be sufficient.

Option:
The images are predefined for a special HW configuration. This can be changed as a parameter. In order for the LIN interface to be recognised correctly, the hardware must be changed to the WOMOLIN interface assignment. After burning, you will find a file arg.dat in the file directory, in which the setting must be changed to hw=WOMOLIN. I didn't really understand why Jan chose this configuration.

Good luck
Magnus

Hello Magnus,
that was easy! I though, i have to compile the whole project. I will document it on my website.
Are you interested in the Node-RED setup?

Many, many thanks!

Works!

Great - have fun


Feature update rxjava and play services

Update rxjava to version 1.0.1
Update play services to 7.8

Play services needs to be updated to 7.8...