Feature request: fast inflate that discards data, to find length

Some file formats, such as the git packfile format, have a series of DEFLATE-compressed streams within them, and don't include lengths that would allow skipping over them to the next piece of data. Git's index file format provides those offsets, but when creating that index file, you have to walk over the entire pack file, deflating each object sequentially, in order to find where the next object starts. That's a major bottleneck in some common git operations.
Would it be possible to have a function in zlib-ng that would do the fastest possible inflate of a DEFLATE-compressed stream without actually reconstructing the decompressed data? This could just decode the block headers, decode any dynamic Huffman tables, skip over literal symbols and match/distance pairs, and look for the end-of-block symbol. No decompressed data, no window of data for backreferences, no copy operations, just looking for the end of each block as fast as possible.
If the resulting function ended up being substantially faster, would that be a reasonable thing to add?

It would still need to partially reconstruct the decompressed data to make sure no-one tries to pass in corrupted or invalid data, but copying of decompressed data to user-supplied output buffer could be skipped to improve the speed.

It seems like that could already be accomplished with inflateBack with user specified write callback.

Calling a callback function would probably be even slower than using normal optimized memcpy(). This is because callback functions need own stack frame.

We can check to see if the callback is NULL and if it is then not attempt to call it.

@nmoinvaz That would actually make it slightly faster when callback is NULL even though it introduces conditional jump. I just don't know how big speed penalty it would incur if the callback is non-NULL.

 It ends up making things a bit faster for one scenario and a bit slower for another. Either way I think @joshtriplett should be able to use inflateBack in its current state and he can profile it to see if checking for out is NULL is that much better for his scenario.

@mtl1979 wrote:

It would still need to partially reconstruct the decompressed data to make sure no-one tries to pass in corrupted or invalid data, but copying of decompressed data to user-supplied output buffer could be skipped to improve the speed.

That's not quite what I'm looking for. If I need to validate the data, I can do a full inflate; the optimization of avoiding the final copy might be nice, but generally if I'm validating the data I'll also want to do something else with it, such as hash it, so I need the data anyway.
In this case, I really do just want to know the length of the compressed data stream, so that I can skip past it and start processing the next record. I'd like to avoid not just the overhead of the final copy, but also the overhead of the internal copies to process backreferences.
If it'd be trivial to do, reconstructing the match-length and summing up the uncompressed length would be nice. Beyond that, I'd rather skip any checks for the validity of the deflate stream, if they'd add any overhead at all.

@nmoinvaz I had a quick glimpse at inflateBack() and no-op'ing the output callback would require more than just checking that output callback is non-NULL as the code blindly writes to output buffer until it's full...

@joshtriplett I think easiest way would be to duplicate inflateBack() and strip anything that writes data to output window. Some lines would need to be modified to make sure input pointer is advanced even if copying and advancing output pointer is removed.
I'm not sure what is the best way to test the modified code actually works as expected except linking the code against git...

@mtl1979 I'd be happy to give that a try and benchmark it. (I'm sure there's a more maintainable way to avoid the code duplication, but it'll help to have numbers for how much this could help first.)

@joshtriplett We have used various tricks to avoid code duplication elsewhere in the code, but for initial testing, it is easier to just duplicate the code... When there is just two alternatives, it doesn't make sense to use macros to hide the differences especially when the code already uses a lot of macros....

Some preliminary results using a large (914MB compressed, 2592MB uncompressed) DEFLATE stream:

About 8.3s using inflate.
About 8.9s using inflateBack with a no-op in and out callback.

Changing inflateBack to check for a NULL callback seemed to add a few milliseconds to the timing, whether passing NULL or a no-op. Looks like inflateBack isn't the right starting point.

About 7.4s using a version of inflate that checks a flag (state->skipout, cached in a local variable) to see if it should skip output.
About 6.4s using a version of inflate if that flag is hardcoded (setting the local to 1 unconditionally), or if the output code is deleted entirely (no performance difference, compilers are smart enough). (Hardcoding it only in inflate_fast gives part but not all of that speedup.)

Not as much as I'd expected, but still fast enough to be worthwhile, and it seems quite likely that it could be optimized further.
Here's the WIP patch/hack: zlib-ng-skipout.patch.txt
To test it, you need a raw deflate stream (not zlib or gzip). Call inflateInit2 with windowbits set to -256-15.
It's currently in "hardcoded skipout" mode; search for the FIXME comment to find that.
Thoughts?
Also, in the course of working on this, I found an unrelated optimization, which I'll send a PR for.

I am noticing that in inflateBack that there might be some continual checks that call ROOM. And it may still copy bytes to s->window using the put variable.  So s->window is being used as internal buffer before writing to out it seems. Removing that might give significant gains. Also I am wondering what is the size of the input buffer given to inflateBack? If you increase it do you notice speed?
Nice work on the improvement. I will try and do some performance testing on the PR soon.

@nmoinvaz I gave inflateBack all of the input data at once, and made the input callback a no-op.

@joshtriplett do you plan on trying to take out the copying of data to s->window and the checks to ROOM? Perhaps the modifications might be better suited for a new function like inflateSize.

@nmoinvaz I didn't touch the inflateBack function, only inflate, so there are no calls to ROOM. I removed all of the copying of data to s->window, and as far as I can tell, the window should never get allocated at all.

I ran across this in the FAQ.zlib which may be related/useful:

Can I access data randomly in a compressed stream?
No, not without some preparation.  If when compressing you periodically use
Z_FULL_FLUSH, carefully write all the pending data at those points, and
keep an index of those locations, then you can start decompression at those
points.  You have to be careful to not use Z_FULL_FLUSH too often, since it
can significantly degrade compression...

... Alternatively, you can scan a deflate stream once to generate an index, and then use that index for
random access.  See examples/zran.c .
https://github.com/madler/zlib/blob/master/examples/zran.c

I repeated performance tests related to deflate decompression performance on Apple Silicon (MacBook Air) using the system provided git executable for comparison. It appears it got even faster in scanning through tiny deflate streams, leaving gitoxide in the dust.
See this comment for more details.
To my mind the efficient handling of decompressing small streams is vital to further improving on gitoxides performance both when streaming a pack file as well as when decoding objects. The latter at least is quite perfectly parallelised which helps offsetting some cost.
It is surprising to me how git can be this fast given that it appears to only use zlib without any special handling.


Update README.md

Corrected spell..

Thanks for pointing out the spelling error in the README.
I am not going to accept the pull request. I don't think it is appropriate to have you listed as a contributor for pointing out a single spelling error in the documentation. Next time please submit these sorts of things a issues and not pull requests.
I have fixed the error.
Thanks again


[app] Support for multiple files

Started in #11

WONTFIX. Given we have https://github.com/frictionlessdata/datapackagist let's keep the demo super simple.


workloadentry handler is registered separately in serviceentry store

/retest

@hzxuzhonghu: The following test failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

integ-pilot-k8s-tests_istio
e5db56e4069bf7f9aab9e779a21feacf156833a4
link
/test integ-pilot-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@hzxuzhonghu: The following test failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

integ-pilot-k8s-tests_istio
e5db56e4069bf7f9aab9e779a21feacf156833a4
link
/test integ-pilot-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@hzxuzhonghu: The following test failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

integ-pilot-k8s-tests_istio
31a87fcccf6cc0c73d49b05df7a66a7e48f387a8
link
/test integ-pilot-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@hzxuzhonghu: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

integ-pilot-k8s-tests_istio
31a87fcccf6cc0c73d49b05df7a66a7e48f387a8
link
/test integ-pilot-k8s-tests_istio

integ-pilot-multicluster-tests_istio
31a87fcccf6cc0c73d49b05df7a66a7e48f387a8
link
/test integ-pilot-multicluster-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/retest

@hzxuzhonghu: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

integ-pilot-multicluster-tests_istio
31a87fcccf6cc0c73d49b05df7a66a7e48f387a8
link
/test integ-pilot-multicluster-tests_istio

integ-pilot-k8s-tests_istio
31a87fcccf6cc0c73d49b05df7a66a7e48f387a8
link
/test integ-pilot-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@hzxuzhonghu: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

integ-pilot-k8s-tests_istio
31a87fcccf6cc0c73d49b05df7a66a7e48f387a8
link
/test integ-pilot-k8s-tests_istio

integ-pilot-multicluster-tests_istio
31a87fcccf6cc0c73d49b05df7a66a7e48f387a8
link
/test integ-pilot-multicluster-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.


Support js target

Hello, I would like to use cps with javascript but the project seems not to be compatible so I made the issue to suggest that.
Tried compiling cps with js, at first it complains about stdmsg usage in spec.nim (writeStackFramesImpl and writeTraceDeque), bodged it with a js "console.error".
Now I get undeclared identifier: 'copyMem' expected something like that, still doesn't help though.
Could you if you have time tell me what it would take to succeed the compilation, I'd do it if I feel capable enough.

I added your bodge, or at least what I imagine it looked like.  It looks like most of the tests now pass.  I don't see a copyMem error -- do you have a specific piece of code that you can share which raises that defect?  It looks to me like we merely need to workaround an issue with getCurrentException() and some strange problem with the fn field in Continuation.

If you have some thoughts on how the javascript backend should work with respect to exceptions or CPS in general, please share them.  You must be the only user of this stuff, so any feedback on how it does or doesn't work well would be appreciated.  For that matter, any feedback on how easy or hard it was to get going with CPS would be helpful.  Confirmed sightings of CPS users are few and far between, to say the least. :wink:


Pin position is not changed on iOS

Changing pin Positionat android target works fine, however changing the pin Positionwill not effect the pin location at map in iOS, it seems that the pin Positionbinding is not working for iOS ?! :cry:

Thanks for reporting this, I will investigate and release a hotfix if this is really the case.

Thank you, we will be waiting to test the hotfix.
Actually, a very important decision is literary pending because of this issue!

I this issue fixed? I'm seeing this issue in iOS in the latest nuget (1.5.0) - if i can't find a workaround by next week I wont be able to use this package in my app..
thanks!

Hello. I did changes regarding this but I didn't test it yet. There is also no nuget package avaiable containing the fix. You can, of course, download the dev branch and do your own build. I don't think I'll be able to work on this next two weeks.

Ok - I  downloaded the latest Dev Build and used those updated libs in my Xamarin Forms project - However I'm still seeing a lack of update for pin position when zooming the map in or out. See attached movie of the map running on android device. Any Suggestions?? -is there some way to manually update the pin position on zoom or pinch?  I'm about to release the app to my client next week.
Movie of issue: https://www.dropbox.com/s/gnanxmym1nvt0mx/TKMapPinError.mov?dl=0

I'm not able to reproduce this. It looks like your pin is static on some screen coordinates, I actually don't know what is going on there and I dont think it is atually related to this issue.
The issue with the pin not moving on iOS when the position is changed, it will be fixed in the next release.

1.5.1.


Modifying awkward expressions in two ko.md files

I corrected a simple grammar error in korean markdown files.

Thanks :-)


winrestcmd() sequence is repeated twice

Steps to reproduce

Run vim

 vim -g --clean +'split' +'echo winrestcmd()'

Sequence of resize commands will be printed:

1resize 40|vert :1resize 175|:2resize 39|vert :2resize 175|:1resize 40|vert :1resize 175|:2resize 39|vert :2resize 175|

Expected behaviour
1resize 40|vert :1resize 175|:2resize 39|vert :2resize 175

Version of Vim
9.0.0035
Environment
OS: Debian 11
Terminal: GUI
Logs and stack traces
No response

This is intentional.  I can't remember the example, but in some cases resizing one window cause another window, that was resized already, to change.  Therefore it is done twice.
Besides the longer string, this has no real problems, right?

No any real problem. Just an observation.


build on github pages

task is to setup automation to build and deploy this to a branch and then serve through github pages
example script here: https://github.com/Lullabot/information-security-policy/blob/main/.github/workflows/BuildGitHubPages.yml

I think that I need to have the Pages set to use a GitHub action.

https://github.com/Lullabot/lb-ebook-poc/settings/pages

I've created a workflow and got it to successfully run.

https://github.com/Lullabot/lb-ebook-poc/blob/main/.github/workflows/BuildGitHubPages.yml
https://github.com/Lullabot/lb-ebook-poc/actions/runs/9813831663

However I'm still not getting a site.

https://lullabot.github.io/lb-ebook-poc

@deviantintegral would you mind looking over this and seeing if you can spot what I may be doing wrong?

It looks like GitHub has changed things around. Here's what we have on the security handbook:

That whole dropdown is new - and the security handbook is using what they call the "classic" experience. So, I think you either have to change that setting, or you need to find a new workflow that works with the new direct-publish system they have.

OK, yeah I had tried that, and it automatically runs a job that tries to run the site through jekyll which fails out because this isn't a jekyll site. I'll keep digging to see if I can find a way to change that.

found that this works if I add a .nojekyll file to the root.


Suggestion: Adjust header bar and buttons dependend on content size

I think that iOS does a really good job on optimizing the content's size of the header bar. It automatically extends the available space of the title if it's too long and removes the back button's label if needed.
This article describes the problem very well and provides a good solution (paragraph 3.2. Titlebox).
What do you think, is this something for Ionic? I'd love the Ionic framework even more!

This is now a part of Ionic with the beta14 updates


Add help paginator & other QOL updates

Due to Github being bugged for me at the moment, I can't PR multiple branches, so I gotta do this all in one PR :/
If you wish to not include any proposed feature, comment so I can revert it
Look at the commit history for updates

Yay, there's no conflicts

Is this working?

@Oji0721 Generally, you should assume a pull request isn't stable yet if it isn't approved or merged.

@yogurtsyum I'm finally done with this PR, may you review it?

It looks like you've also pushed your group ban feature.

No? I just added the group ban status to the /user API endpoint

So what'll happen with this PR?

I'll just remove the 1 feature that was implemented already. Sorry I couldn't use this one, but I was reviewing this and noticed some issues, so I figured I would just push my own solution for now because I needed it done.
And for the review for this, I will have it posted sometime tomorrow.

Alright

I wonder, should I just remake the PR excluding the features in other PRs/in the bot already?


Replace ADD with COPY in dockerfiles

Issue #926  , if available:
Description of changes:
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

cc @micahhausler

/test golang-1-16-tooling-presubmit

/approve


Remove Redundant Types from tx-construction

Describe changes
Removed redundant types from tx-construction package

Let's do it!


Fix the build

I'm no TS expert but compile/test tasks weren't working (probably required global settings I don't have)

added missing type definitions
added missing mocha import
slight change on Promise.reject call because AFAICT the definition is wrong:
(should be reject<T>(error: *any*): Promise<T>, as the type of the error has no effect on the type of the promise)

Need that before tackling either #14 or #15

That one is probably just me as Travis was running fine w/o it, npm run typings may be the step I was missing, will check

and that it is, sorry about that one, got confused as apollostack/subscriptions-transport-ws didn't need this step.


Packages missing from repos

Hi guys,
So far we are considering Oracle Linux as a viable option to replace CentOS.
Presently we have two questions if you are able to assist:

I don't think this centos2ol repo is the correct place for some Q&A, so is there any community forums, mailing list or sort where we can engage with other community members?
We cannot find the fail2ban package in any repo for OL8. Maybe we're looking wrong, could you please confirm its availability?
Thank you.

@Djelibeybi many thanks for your quick reply!
About 1. amazing I'll probably put my next questions there instead of here!
About #2 prioritising fail2ban would be super we use it a lot and missing it on OL!
Great job you guys in putting OL up to speed for those coming from CentOS! Kudos!

@Djelibeybi many thanks for your quick reply!
About 1. amazing I'll probably put my next questions there instead of here!
About #2 prioritising fail2ban would be super we use it a lot and missing it on OL!
Great job you guys in putting OL up to speed for those coming from CentOS! Kudos!


boards/x86/acrn: build it in x86_64 mode

added CONFIG_X86_64 since ACRN hypervisor supports x86_64 arch only.
removed CONFIG_SET_GDT because Zephyr doesn't support it in x86_64,
and ACRN doesn't need the guest to setup GDT during boot time.
added CONFIG_X2APIC since it's preferred by ACRN.
include sysapic.h in x86_64 as well so that it gets built if
CONFIG_LOAPIC is enabled in x86_64 build.

Signed-off-by: Zide Chen<EMAIL_ADDRESS>

[28/77] Building C object zephyr/CMakeFiles/zephyr.dir/subsys/debug/openocd.c.obj
FAILED: zephyr/CMakeFiles/zephyr.dir/subsys/debug/openocd.c.obj
ccache /opt/sdk/zephyr-sdk-0.11.1/x86_64-zephyr-elf/bin/x86_64-zephyr-elf-gcc -DBUILD_VERSION=zephyr-v2.1.0-2100-g04084a1f02a1 -DKERNEL -D_FORTIFY_SOURCE=2 -D__ZEPHYR__=1 -I../../../../../../../include -Izephyr/include/generated -I../../../../../../../soc/x86/ia32 -isystem ../../../../../../../lib/libc/minimal/include -isystem /opt/sdk/zephyr-sdk-0.11.1/x86_64-zephyr-elf/bin/../lib/gcc/x86_64-zephyr-elf/9.2.0/include -isystem /opt/sdk/zephyr-sdk-0.11.1/x86_64-zephyr-elf/bin/../lib/gcc/x86_64-zephyr-elf/9.2.0/include-fixed -m64    -Os -imacros/home/buildslave/src/github.com/zephyrproject-rtos/zephyr/sanity-out/acrn/tests/subsys/debug/tracing/tracing.backends.openocd/zephyr/include/generated/autoconf.h -ffreestanding -fno-common -g -m64 -Wa,--divide -imacros/home/buildslave/src/github.com/zephyrproject-rtos/zephyr/include/toolchain/zephyr_stdint.h -Wall -Wformat -Wformat-security -Wno-format-zero-length -Wno-main -Wno-address-of-packed-member -Wno-pointer-sign -Wpointer-arith -Wno-unused-but-set-variable -Werror=implicit-int -Werror -fno-asynchronous-unwind-tables -fno-pie -fno-pic -fno-strict-overflow -fno-reorder-functions -fno-defer-pop -fmacro-prefix-map=/home/buildslave/src/github.com/zephyrproject-rtos/zephyr/tests/subsys/debug/tracing=CMAKE_SOURCE_DIR -fmacro-prefix-map=/home/buildslave/src/github.com/zephyrproject-rtos/zephyr=ZEPHYR_BASE -ffunction-sections -fdata-sections -march=x86-64 -std=c99 -nostdinc -MD -MT zephyr/CMakeFiles/zephyr.dir/subsys/debug/openocd.c.obj -MF zephyr/CMakeFiles/zephyr.dir/subsys/debug/openocd.c.obj.d -o zephyr/CMakeFiles/zephyr.dir/subsys/debug/openocd.c.obj   -c /home/buildslave/src/github.com/zephyrproject-rtos/zephyr/subsys/debug/openocd.c
In file included from ../../../../../../../include/kernel_includes.h:16,
                 from ../../../../../../../include/kernel.h:17,
                 from /home/buildslave/src/github.com/zephyrproject-rtos/zephyr/subsys/debug/openocd.c:7:
/home/buildslave/src/github.com/zephyrproject-rtos/zephyr/subsys/debug/openocd.c:58:20: error: 'struct _callee_saved' has no member named 'esp'; did you mean 'rsp'?
   58 |       callee_saved.esp),
      |

Looks like this is trying to build some openocd code that assumes 32-bit (note esp and not rsp)

[28/77] Building C object zephyr/CMakeFiles/zephyr.dir/subsys/debug/openocd.c.obj
FAILED: zephyr/CMakeFiles/zephyr.dir/subsys/debug/openocd.c.obj
ccache /opt/sdk/zephyr-sdk-0.11.1/x86_64-zephyr-elf/bin/x86_64-zephyr-elf-gcc -DBUILD_VERSION=zephyr-v2.1.0-2100-g04084a1f02a1 -DKERNEL -D_FORTIFY_SOURCE=2 -D__ZEPHYR__=1 -I../../../../../../../include -Izephyr/include/generated -I../../../../../../../soc/x86/ia32 -isystem ../../../../../../../lib/libc/minimal/include -isystem /opt/sdk/zephyr-sdk-0.11.1/x86_64-zephyr-elf/bin/../lib/gcc/x86_64-zephyr-elf/9.2.0/include -isystem /opt/sdk/zephyr-sdk-0.11.1/x86_64-zephyr-elf/bin/../lib/gcc/x86_64-zephyr-elf/9.2.0/include-fixed -m64    -Os -imacros/home/buildslave/src/github.com/zephyrproject-rtos/zephyr/sanity-out/acrn/tests/subsys/debug/tracing/tracing.backends.openocd/zephyr/include/generated/autoconf.h -ffreestanding -fno-common -g -m64 -Wa,--divide -imacros/home/buildslave/src/github.com/zephyrproject-rtos/zephyr/include/toolchain/zephyr_stdint.h -Wall -Wformat -Wformat-security -Wno-format-zero-length -Wno-main -Wno-address-of-packed-member -Wno-pointer-sign -Wpointer-arith -Wno-unused-but-set-variable -Werror=implicit-int -Werror -fno-asynchronous-unwind-tables -fno-pie -fno-pic -fno-strict-overflow -fno-reorder-functions -fno-defer-pop -fmacro-prefix-map=/home/buildslave/src/github.com/zephyrproject-rtos/zephyr/tests/subsys/debug/tracing=CMAKE_SOURCE_DIR -fmacro-prefix-map=/home/buildslave/src/github.com/zephyrproject-rtos/zephyr=ZEPHYR_BASE -ffunction-sections -fdata-sections -march=x86-64 -std=c99 -nostdinc -MD -MT zephyr/CMakeFiles/zephyr.dir/subsys/debug/openocd.c.obj -MF zephyr/CMakeFiles/zephyr.dir/subsys/debug/openocd.c.obj.d -o zephyr/CMakeFiles/zephyr.dir/subsys/debug/openocd.c.obj   -c /home/buildslave/src/github.com/zephyrproject-rtos/zephyr/subsys/debug/openocd.c
In file included from ../../../../../../../include/kernel_includes.h:16,
                 from ../../../../../../../include/kernel.h:17,
                 from /home/buildslave/src/github.com/zephyrproject-rtos/zephyr/subsys/debug/openocd.c:7:
/home/buildslave/src/github.com/zephyrproject-rtos/zephyr/subsys/debug/openocd.c:58:20: error: 'struct _callee_saved' has no member named 'esp'; did you mean 'rsp'?
   58 |       callee_saved.esp),
      |

Looks like this is trying to build some openocd code that assumes 32-bit (note esp and not rsp)

This is not an issue introduced by this patch, is it correct? In master, I saw some problem with "west build -b qemu_x86_64 samples/hello_world/" if I enabled CONFIG_OPENOCD_SUPPORT.
Probably need to fix it in openocd in a separate patch?

This is not an issue introduced by this patch, is it correct? In master, I saw some problem with "west build -b qemu_x86_64 samples/hello_world/" if I enabled CONFIG_OPENOCD_SUPPORT.
Probably need to fix it in openocd in a separate patch?

For some reason this PR is causing this to induce a CI failure when it didn't before. We can't check anything in which breaks CI, so this PR will need at least a workaround.

This seems to fix it:
diff --git a/subsys/debug/openocd.c b/subsys/debug/openocd.c
index 9ed76ddf09..bf1acf9c04 100644
--- a/subsys/debug/openocd.c
+++ b/subsys/debug/openocd.c
@@ -54,8 +54,13 @@ size_t _kernel_openocd_offsets[] = {
        [OPENOCD_OFFSET_T_STACK_PTR] = offsetof(struct k_thread,
                                                callee_saved.sp),
 #elif defined(CONFIG_X86)
+#if defined(CONFIG_X86_64)
+       [OPENOCD_OFFSET_T_STACK_PTR] = offsetof(struct k_thread,
+                                               callee_saved.rsp),
+#else
        [OPENOCD_OFFSET_T_STACK_PTR] = offsetof(struct k_thread,
                                                callee_saved.esp),
+#endif
 #elif defined(CONFIG_NIOS2)
        [OPENOCD_OFFSET_T_STACK_PTR] = offsetof(struct k_thread,
                                                callee_saved.sp),

There's another build issue with the loapic timer driver and some hard-coded assembly for reading the TSC if CONFIG_EXECUTION_BENCHMARKING is enabled, why this is not an inline function baffles me. I'm looking into it.
There's also an issue with how tests/arch/x86/static_idt is filtered.
Im working on patches for all of these.

This seems to fix it:
diff --git a/subsys/debug/openocd.c b/subsys/debug/openocd.c
index 9ed76ddf09..bf1acf9c04 100644
--- a/subsys/debug/openocd.c
+++ b/subsys/debug/openocd.c
@@ -54,8 +54,13 @@ size_t _kernel_openocd_offsets[] = {
        [OPENOCD_OFFSET_T_STACK_PTR] = offsetof(struct k_thread,
                                                callee_saved.sp),
 #elif defined(CONFIG_X86)
+#if defined(CONFIG_X86_64)
+       [OPENOCD_OFFSET_T_STACK_PTR] = offsetof(struct k_thread,
+                                               callee_saved.rsp),
+#else
        [OPENOCD_OFFSET_T_STACK_PTR] = offsetof(struct k_thread,
                                                callee_saved.esp),
+#endif
 #elif defined(CONFIG_NIOS2)
        [OPENOCD_OFFSET_T_STACK_PTR] = offsetof(struct k_thread,
                                                callee_saved.sp),

There's another build issue with the loapic timer driver and some hard-coded assembly for reading the TSC if CONFIG_EXECUTION_BENCHMARKING is enabled, why this is not an inline function baffles me. I'm looking into it.
There's also an issue with how tests/arch/x86/static_idt is filtered.
Im working on patches for all of these.

Then this PR can wait until your patch is merged. I can close the PR for now.

The fixes I made in #22560 will need to be applied before this will pass.
Also, you made several changes which are not closely related, I would appreciate if you split your patch into individual logical changes.

I can wait until #22560 is done as well. And will split it two patches: 1) acrn specific  2) header changes in arch.h.
Or even better, if you can address this generic issue into your #22534, then I can only touch board acrn.

The fixes I made in #22560 will need to be applied before this will pass.
Also, you made several changes which are not closely related, I would appreciate if you split your patch into individual logical changes.

I can wait until #22560 is done as well. And will split it two patches: 1) acrn specific 2) header changes in arch.h.
Or even better, if you can address this generic issue into your #22534, then I can only touch board acrn.

Mine should be in fairly soon, rebase and force push after that and we'll see where we are at with CI.

@ZideChen0 just push a new revision to this PR when it's ready, no need to close this and open another. We prefer code review history to not span multiple PRs.

@ZideChen0 my PR has been merged, can you rebase?

@ZideChen0 my PR has been merged, can you rebase?

Yes, noticed that this morning. Rebased, and doing sanity tests.

Looks like a potentially incorrect rebase? I'm removing myself from review

@ZideChen0 I don't think your rebase was executed properly.
Usually you just need to do, while on your forked branch:
git rebase origin/master
git push -f

@ZideChen0 I don't think your rebase was executed properly.
Usually you just need to do, while on your forked branch:
git rebase origin/master
git push -f

Done. Thanks for your help.


Move across drives in Windows

As mentioned in #10 there is still a problem when using rename across drives in Windows. This error occurs when using bosh-cli add-blobs.

@fydon can you please add a windows specific mover_test.go

@cppforlife Sure I can try. How would I go about doing this across two drives? I assume that these would need to be real, rather than fake_file_system?

@Fydon i think it will be fine to use fake file sys just like we do for other branches in this class: https://github.com/cloudfoundry/bosh-utils/blob/master/fileutil/mover_test.go#L48

@cppforlife Sure. I copied that context and its tests and used the Windows specific error. Are those tests  correct?


master -> live

Docs Build status updates of commit aba1eda:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs/api/_data/catalog-package-details.json
:white_check_mark:Succeeded
View

docs/api/_data/package-registration-page.json
:white_check_mark:Succeeded
View

docs/api/catalog-resource.md
:white_check_mark:Succeeded
View

docs/api/registration-base-url-resource.md
:white_check_mark:Succeeded
View

docs/tools/dotnet-Commands.md
:white_check_mark:Succeeded
View

docs/tools/ps-ref-update-package.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Morph: use a uniform instead of a define for the number of active targets

See https://forum.babylonjs.com/t/babylon-blendshapes-animation-player-fps-drops-and-glitch/41884/13
Doing so will avoid recompiling a new effect each time the number of active targets changes.

Closed by #14734


RenderingServer.material_set_param doesn't work with texture based objects

Tested versions

Reproducible in master, 4.4 dev 4, 4.3 stable

System information
Godot v4.4.dev4 - Windows 10.0.19045 - Multi-window, 2 monitors - Vulkan (Forward+) - dedicated AMD Radeon RX 580 2048SP (Advanced Micro Devices, Inc.; 31.0.21921.1000) - AMD Ryzen 5 3600 6-Core Processor (12 threads)
Issue description
As this question  in Godot forum shows, RenderingServer.material_set_param don't allow to set sampler uniforms with a texture object, which diverges from Material.set_shader_parameter that works normally with textures as parameter. That happens because RenderingServer.material_set_param rejects any Object as value:
https://github.com/godotengine/godot/blob/e65a23762b36b564eb94672031f37fdadba72333/servers/rendering/renderer_rd/storage_rd/material_storage.cpp#L2223-L2232
Steps to reproduce

Create a shader with an uniform variable of type Sampler2D and assign this shader to a node
Load any image (the Godot icon as example)
Try to set the shader uniform variable using RenderingServer.material_set_param and the loaded image
Godot will fail with error:  Condition "p_value.get_type() == Variant::OBJECT" is true.

Or play the MRP below.
Minimal reproduction project (MRP)
test-uniform-set-with-rendering-server.zip

Servers don't use objects. You need to pass a RID instead.
RenderingServer.material_set_param($ColorRect.get_material(), "sample", texture.get_rid())

Servers don't use Objects. You need to pass a RID instead.

True, if change this behavior is not desired an alternative could be use a more clear error message.


Calrify how to login with federated token

Clarify how to login with federated credential tokens with az cli and note that it applies to both Service Principals and Managed Identities
az login --service-principal -u $CLIENT_ID -t $TENANT --federated-token $TOKEN

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: eeedb9a7-2aed-6727-4cfb-1515809d676e
Version Independent ID: 2c83180d-22ad-db42-e2eb-f27f11977963
Content: az
Content Source: latest/docs-ref-autogen/reference-index.yml
GitHub Login: @rloutlaw
Microsoft Alias: routlaw

@jiasli for awareness

The help message of --service-principal was provided by federated credential service team. We will refine it once we get more information.

Wait, can it be used with managed identities without the proxy sidecar?

Federated token value seems to be : api://AzureADTokenExchange if you're using a managed identity and a federated token

Running az login --federated-token "$(cat $AZURE_FEDERATED_TOKEN_FILE)" --service-principal -u $AZURE_CLIENT_ID -t $AZURE_TENANT_ID returns:
Interactive authentication is needed. Please run:
az login
Still unable to login using user assigned identity using workload identity authentication method. The documented way:
Log in using a VM's user-assigned managed identity. Client or object ids of the service identity
also work.
az login --identity -u /subscriptions//resourcegroups/myRG/providers/Microso
ft.ManagedIdentity/userAssignedIdentities/myID
yields:
Failed to connect to MSI. Please make sure MSI is configured correctly.
Get Token request returned http error: 400, reason: Bad Request
I verified the environment variables and they are correctly set. The token file is readable.
By now just going in circles as specific documantation about az cli login via workload identity with AKS is very scarce or esluding me.

Running az login --service-principal --tenant "${AZURE_TENANT_ID}" --username "${AZURE_CLIENT_ID}" --federated-token "$(cat ${AZURE_FEDERATED_TOKEN_FILE})" works on my side.
I'm using the latest version of az cli, if you used an old version it seems to redirect you in an interactive login.
You can use this docker image: mcr.microsoft.com/azure-cli:latest
Or curl the latest version: curl -sL https://aka.ms/InstallAzureCLIDeb
On my side, I was not able to use az login --identity -u <identity_id>, it returns the same error code.

Running az login --service-principal --tenant "${AZURE_TENANT_ID}" --username "${AZURE_CLIENT_ID}" --federated-token "$(cat ${AZURE_FEDERATED_TOKEN_FILE})" works on my side.
I'm using the latest version of az cli, if you used an old version it seems to redirect you in an interactive login.
You can use this docker image: mcr.microsoft.com/azure-cli:latest Or curl the latest version: curl -sL https://aka.ms/InstallAzureCLIDeb
On my side, I was not able to use az login --identity -u <identity_id>, it returns the same error code.

Thanks @pachr I finally managed to make it work. I had the Wrong ID set as AZURE_CLIENT_ID. This is all very confusing and with this ID mess Azure identity system is, plus lack of good documentation, makes mistakes like mine all too easy to make.

Running az login --service-principal --tenant "${AZURE_TENANT_ID}" --username "${AZURE_CLIENT_ID}" --federated-token "$(cat ${AZURE_FEDERATED_TOKEN_FILE})" works on my side.
I'm using the latest version of az cli, if you used an old version it seems to redirect you in an interactive login.
You can use this docker image: mcr.microsoft.com/azure-cli:latest Or curl the latest version: curl -sL https://aka.ms/InstallAzureCLIDeb
On my side, I was not able to use az login --identity -u <identity_id>, it returns the same error code.

Thanks @pachr I finally managed to make it work. I had the Wrong ID set as AZURE_CLIENT_ID. This is all very confusing and with this ID mess Azure identity system is, plus lack of good documentation, makes mistakes like mine are way too easy to make.

@fabio-s-franco yes it's a bit confusing between CLIENT ID, TENANT ID, PRINCIPAL ID, OBJECT ID 😅 glad it works for you.

@GabriFedi97 I am sorry I couldn't get to your question sooner. I have hit this error many times I don't even remember what causes it. Likely many possibilities. Problem of Bad Request is that it doesn't say much. Ideally we could get debug logs for details of the error, but I am unsure I ever got to enable it, or if it is possible to obtain more information.
But my command is different than yours. I never got the command you are using to work. Instead I used the one that authenticates with --service-principal. The example above from pachr. What it does is to use the token mounted on disk, for pods that have the annotation to use workload identity.
The problem I had was about using the correct value for AZURE_CLIENT_ID in the command.
It is all too confusing (which I believe motivated this issue to be created). The error you are seeing is a step that requires to use MSI endpoint to obtain the tokens you should already have. When using workload identity you don't need to obtain the Entra ID/Azure AD token, because that gets projected to the pod. It's the part that "should" make it easier, if only this was explained somewhere.
This token is used to obtain and renew the token that identifies the workload as a service principal (JWT), which then has roles assigned to it, and will be used to access other Azure services which are controlled using RBAC.
So if you're seeing any message about MSI, it's because it's trying to do the step in between, which is not what workload identity mechanism provides. So check these:
1 - Federated Identity is correctly configured, your pod is annotated to use workload identity and the tokens are mounted on disk/volume. You can literally shell into your pod and see them. I don't remember exactly where now.
2 - Make sure the environment variables are set with the right values (tenant id, client id) and your pass the federated token like in the command above. You can't authenticate directly with service principals using managed identities, they are kind of an abstraction on top of them. They still have the associated service principal (which adds to the confusion), but the authentication flow is different if as you had a standalone service principal.
3 - You see that on my post that I have very different error messages. One of them which errors with the MSI endpoint was not suitable to it, so you shouldn't use it. The first one was the correct one, but I had AZURE_CLIENT_ID using the wrong value (I can't even tell you where it is from, I don't remember, where from the pot I obtained, and would likely have to scavenge it all over again if I were to configure this again). TLDR: Use the correct command, if you see MSI error, you're using the wrong command (if you're trying to authenticate managed/user assigned identity).
These 3 pointers will give you enough (if you hadn't given up or figured out) clues to go through the maze. I hope...

Federated token value seems to be : api://AzureADTokenExchange if you're using a managed identity and a federated token

Do we need to specify this as federated token?

@GabriFedi97 I am sorry I couldn't get to your question sooner. I have hit this error many times I don't even remember what causes it. Likely many possibilities. Problem of Bad Request is that it doesn't say much. Ideally we could get debug logs for details of the error, but I am unsure I ever got to enable it, or if it is possible to obtain more information.
But my command is different than yours. I never got the command you are using to work. Instead I used the one that authenticates with --service-principal. The example above from pachr. What it does is to use the token mounted on disk, for pods that have the annotation to use workload identity.
The problem I had was about using the correct value for AZURE_CLIENT_ID in the command.
It is all too confusing (which I believe motivated this issue to be created). The error you are seeing is a step that requires to use MSI endpoint to obtain the tokens you should already have. When using workload identity you don't need to obtain the Entra ID/Azure AD token, because that gets projected to the pod. It's the part that "should" make it easier, if only this was explained somewhere.
This token is used to obtain and renew the token that identifies the workload as a service principal (JWT), which then has roles assigned to it, and will be used to access other Azure services which are controlled using RBAC.
So if you're seeing any message about MSI, it's because it's trying to do the step in between, which is not what workload identity mechanism provides. So check these:
1 - Federated Identity is correctly configured, your pod is annotated to use workload identity and the tokens are mounted on disk/volume. You can literally shell into your pod and see them. I don't remember exactly where now.
2 - Make sure the environment variables are set with the right values (tenant id, client id) and your pass the federated token like in the command above. You can't authenticate directly with service principals using managed identities, they are kind of an abstraction on top of them. They still have the associated service principal (which adds to the confusion), but the authentication flow is different if as you had a standalone service principal.
3 - You see that on my post that I have very different error messages. One of them which errors with the MSI endpoint was not suitable to it, so you shouldn't use it. The first one was the correct one, but I had AZURE_CLIENT_ID using the wrong value. I can't even tell you where it is from, I don't remember, where I got it from. If I had to do it again, I would likely have to scavenge it all over again to configure it. TLDR: Use the correct command, if you see MSI error, you're using the wrong command (if you're trying to authenticate managed/user assigned identity).
These 3 pointers will give you enough clues to go through the maze. If you haven't given up already or figured out yourself, or used another solution altogether...

How do we get federated token in azure devops pipeline? Any idea how to extend lifetime of idtoken in azure devops pipeline?

How do we get federated token in azure devops pipeline? Any idea how to extend lifetime of idtoken in azure devops pipeline?

Federated tokens are never gotten, they are issued (for example, you can issue one manually and add to your devops pipeline). It depends on the identity provider to manage the lifetime of the token. Maybe what you want to do would benefit more from using MSI instead.
There is a github integration example, which may be relevant to you to see if it is appropriate to your use case:
https://learn.microsoft.com/en-us/azure/developer/github/connect-from-azure?tabs=azure-portal%2Cwindows

@pallavi-vetal Did you get it working on your pipelines?

If anyone wants to login to the az cli using a federated credential I've written up a series of blog posts on doing this "manually" so to speak.
You'll first need to setup your OIDC Identity Documents and then configure a User Assigned Managed Identity to "map" to the OIDC document endpoints.
Once you have all of this in place you can then generate a signed client assertion which you pass to the az login command.
You can find all the different steps documented here: https://finarne.wordpress.com/2024/07/25/acquire-an-entra-id-token-using-federated-credentials-part-1-oidc-discovery-documents/


Option to Omit Setup Scripts from magento2:generate:schema-upgrade

Something like --skip-versioned-scripts.  This will let people create a default UpgradeSchema/UpgradeData class to use however they want.

Added


Fix WLED light transition duration

Description:
Fixes WLED transitions durations. WLED and the wled package expects them in ms, while Home Assistant works in seconds. This caused it to appear like the transition wasn't working (which it did, but 1000x faster).
It also addresses an issue where transitions on turning the light off were not implemented at all.
Related issue (if applicable): #30489 (related, does not fix it). fixes #30488
Checklist:

[x] The code change is tested and works locally.
[x] Local tests pass with tox. Your PR cannot be merged unless tests pass
[x] There is no commented out code in this PR.
[x] I have followed the development checklist

Wow, superfast fix, thanks!


Controlling Aubo i5  using joint velocities

I am doing a visual servoing project in Aubo i5 robot. I need to control the robot based on visual servoing outputs from the camera. I have obtained both joint velocities and end effector velocities through visual servoingand robot jacobian.   Is it possible to control Aubo i5 using joint velocities ?

what do you mean by control i5 with joint velocities?


[WIP] Implement Sort Groups for Sprite3D and Label3D render priority.

Implements https://github.com/godotengine/godot-proposals/issues/3986

Adds SortGroup node, to allow nested render priority sorting of the Sprite3D and Label3D nodes (or any other material with, if appropriate group ID is set in the material settings).
Increases render priority range from 8-bits to 32-bits.

With SortGroups:
https://user-images.githubusercontent.com/7645683/166207628-4a0103ee-e2c1-4061-a1a1-f26d5286e53e.mov
Without SortGroups:
https://user-images.githubusercontent.com/7645683/166207677-a025e14b-2eb8-4b96-9d21-49a19da0fb01.mov
Test Project:
test_project.zip
Note: Node selection seems to override rendering priority and will cause Z-fighting until it's deselected.

Outdated, only works in some cases, and doesn't seem like a right way to implement it in general.


Update to substrate 3.0

Updates to Substrate 3.0 by changing dependencies and fixing any lingering type errors.

I will bump the substrate frontier branch and reset the peg once I get this to compile.

Quite confused why a 2.0.1 version of parity-scale-codec keeps showing up, any ideas @sorpaas ?
error[E0277]: the trait bound `ExitReason: WrapperTypeEncode` is not satisfied
   --> primitives/evm/src/lib.rs:45:2
    |
45  |     pub exit_reason: ExitReason,
    |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^ the trait `WrapperTypeEncode` is not implemented for `ExitReason`
    |
   ::: /Users/drewstone/.cargo/registry/src/github.com-1ecc6299db9ec823/parity-scale-codec-2.0.1/src/codec.rs:216:21
    |
216 |     fn encode_to<T: Output + ?Sized>(&self, dest: &mut T) {
    |                     ------ required by this bound in `encode_to`
    |
    = note: required because of the requirements on the impl of `Encode` for `ExitReason`

error[E0277]: the trait bound `[evm::backend::Log]: Encode` is not satisfied
   --> primitives/evm/src/lib.rs:48:2
    |
48  |     pub logs: Vec<Log>,
    |     ^^^^^^^^^^^^^^^^^^ the trait `Encode` is not implemented for `[evm::backend::Log]`
    |
   ::: /Users/drewstone/.cargo/registry/src/github.com-1ecc6299db9ec823/parity-scale-codec-2.0.1/src/codec.rs:216:21
    |
216 |     fn encode_to<T: Output + ?Sized>(&self, dest: &mut T) {
    |                     ------ required by this bound in `encode_to`
    |
    = help: the following implementations were found:
              <[T; 0] as Encode>
              <[T; 100] as Encode>
              <[T; 101] as Encode>
              <[T; 1024] as Encode>
            and 263 others
    = note: required because of the requirements on the impl of `Encode` for `Vec<evm::backend::Log>`

error[E0277]: the trait bound `ExitReason: WrapperTypeDecode` is not satisfied
   --> primitives/evm/src/lib.rs:45:2
    |
45  |     pub exit_reason: ExitReason,
    |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^ the trait `WrapperTypeDecode` is not implemented for `ExitReason`
    |
   ::: /Users/drewstone/.cargo/registry/src/github.com-1ecc6299db9ec823/parity-scale-codec-2.0.1/src/codec.rs:277:18
    |
277 |     fn decode<I: Input>(input: &mut I) -> Result<Self, Error>;
    |                  ----- required by this bound in `sp_core::Decode::decode`
    |
    = note: required because of the requirements on the impl of `Decode` for `ExitReason`

error[E0277]: the trait bound `Vec<evm::backend::Log>: Decode` is not satisfied
   --> primitives/evm/src/lib.rs:48:2
    |
48  |     pub logs: Vec<Log>,
    |     ^^^^^^^^^^^^^^^^^^ the trait `Decode` is not implemented for `Vec<evm::backend::Log>`
    |
   ::: /Users/drewstone/.cargo/registry/src/github.com-1ecc6299db9ec823/parity-scale-codec-2.0.1/src/codec.rs:277:18
    |
277 |     fn decode<I: Input>(input: &mut I) -> Result<Self, Error>;
    |                  ----- required by this bound in `sp_core::Decode::decode`
    |
    = help: the following implementations were found:
              <Vec<T> as Decode>

error: aborting due to 4 previous errors

That's probably evm crate issue. I'll take a look.

This is what I got stuck at @sorpaas
https://github.com/paritytech/frontier/issues/308


Place username and password in config file

As a developer, I'm a bit worried about committing my account credentials accidentally. There should be a config file and it should be added to the .gitignore to prevent that. Unless there's a safer idea.

Significant discussion has taken place in #71. I have some new ideas, but we should continue on that chain instead.


Update impactPage

Type of Changes

Type

✓
:sparkles: New feature

✓
:link: Update dependencies

Description
This pull request:
1- Adds new dependencies: react-datamaps and d3.
2- Moves the worldMap component from web-client to site.
3- Modifies path/to/impactPage.js to display the map.
4- Adds new supporters to the partners section and removes the old ones.

Testing Guidelines

From your terminal, pull down this branch with git pull origin site-design-update-impact and check that branch out with git checkout site-design-update-impact
Then yarn to install the newly added dependencies locally and yarn start to launch the app.
Go to http://localhost:8000/ to see the map on the landing page.

Please let me know if anything needs to change. Thanks.

Closing this since it was merged with site-design

Closing this since it was merged with site-design


Generate PDF always leave first page as blank page

In the element, I saw that it inject the div.html__page-break inside. How to remove that?

That only happens if your 1st child element height /w margin top and bottom on your pdf-content slot is greater than your prop paginate-elements-by-height value

Feel free to open this, if the issue persist!


5 best cards dont win..

There is an issue that if two Players have the same 5 best cards but one of them has a higher handcard he win. But thats not right cause just the highest 5 cards are important for winning so it should be split between that. Saw it on a straight in middle cards, so all had the same straight but the bot with the highest cards won.

I have corrected your code in kind of youre Style.

@Simi2437 Hi can you share your fix here? I encounter the same issue with the current version///

@allenfrostline im not shure anymore if I have that code. I can search for it if the following is no opinion to you. I remember that this Cardevaluater was realy slow too and switcht to deuces Cardevaluater. So I believe it whould be better using a other solution.

@allenfrostline im not shure anymore if I have that code. I can search for it if the following is no opinion to you. I remember that this Cardevaluater was realy slow too and switcht to deuces Cardevaluater. So I believe it whould be better using a other solution.

@Simi2437 Thanks. Also, agree on the speed thing. I'm currently trying to rewrite the evaluation in treys, which is a py3-equivalence of deuces. It'd be great if you could find the code or just tell me the logic you changed in the game evaluation part.

@allenfrostline a fast look at the code let me think, that the problem is in hand_evaluater.py there is the eval_hand function. This allways include to the score of the two handcards in the lower signifikant bits. That causes that if you have a straight, that score is right, get shifted 8 bits left and than the score of the to hand cards add up to that, this causes the problem. So you have to change it so that if its a straight, a full house etc where just the 5 highest cards count and the first 8 bits are 0 or the same in some way.

There was a other failure too if i remember right. Ass 2 3 4 5 dont work am i right ? You maby have to change this too if you want to use that evaluator

@Simi2437 Thanks a lot! I fixed the A-5 straight problem yesterday.

Just thinking this hole idea of use handcards in the first 8 bit makes no sense. You have to override all eval hands anyways and use the first 8 bit for real highcardhandling.


Add hidden flag for the delayed compaction random time window

Improve a bit https://github.com/prometheus/prometheus/issues/8278 and https://github.com/prometheus/prometheus/pull/12532 by adding an hidden flag to configure how much delay to add to the compaction.
It seems that 10% is not enough for our use cases (we have tons of prometheus sending all blocks to an object storage using thanos), we are still filling our network bandwidth, we would like to try with big values.
The hidden flags add the ability to override the default 10% to any percentage.
Also already discussed in https://github.com/prometheus/prometheus/pull/12532/files#r1512749547
Thanks, Alban

I think its ready for review and merge. Can I please ask for your review @machine or @jesusvazquez ?
Thanks


Added the 1.7 proposed timeline

Just getting the ball rolling
Needs to incorporate feedback from 1.6 retrospective and needs approval at community meeting

Let's start with it, thanks @grodrigues3!

@grodrigues3

Needs to incorporate feedback from 1.6 retrospective and needs approval at community meeting
One more question - when we are going to discuss and approve it? Next community meeting will happen on April 13, while the iteration starts on April 3.

In that case, maybe we send it out to K8s-dev sooner, get some feedback and merge when we have general consensus

@grodrigues3 let's do it the next week.

Pulling some conversations out of collapsed threads:
I recommend to keep periodic targets in units of weeks. It helps minimize cognitive load if we all know "release cuts are supposed to happen on Wednesdays". Wednesday is nice because it gives you time to ramp up to the cut (Mon-Tue) and then time to react to new issues (Thu-Fri) without a momentum-breaking weekend.
I suggest alphas should be every 2 weeks by default, but allow additional alphas in between if needed or requested by downstream integrators (e.g. kubeadm). I think betas should be every 1 week by default. During the code slush, a lot of people tend to be focused on the release so fixes come in at a high enough rate to be worth cutting every week.
In retrospect, I think we should have reopened the master branch sooner than we did for the 1.6 cycle. Code slush (milestone restriction) is a fairly low barrier against new changes, and is difficult to police. We resisted reopening master because we didn't want to increase friction on fixes, but I think the tradeoff is that we lost out on time where the branch is truly quiet/stable leading up to release.
Working backward from the final release target on Wed 6/28, my proposed schedule would be:

Wed 4/19: v1.7.0-alpha.2
Wed 5/3: v1.7.0-alpha.3
Wed 5/17: v1.7.0-alpha.4
Wed 5/31: v1.7.0-beta.0

Create release branch.
Set up CI for branch.
Begin regular fast-forwards (at least daily to keep CI up-to-date).

Thu 6/1: Begin code slush.

Deadline for feature work
Add milestone restriction on submit queue.

Wed 6/7: v1.7.0-beta.1
Wed 6/14: v1.7.0-beta.2
Thu 6/15: End code slush.

Final fast-forward of release branch.
All changes for the release must now be cherrypicked in batches by branch manager.
Remove milestone restriction on submit queue. Bot drains backlog over the weekend.

Mon 6/19: v1.8.0-alpha.1
Wed 6/21: v1.7.0-rc.1

RC means no known release-blockers outstanding.
Only accept cherrypicks for release-blockers.
Announce on mailing lists, Twitter, etc. to beg people to try the RC for real.
Perhaps managed k8s providers make rc.1 available on early access channels.
More RCs as needed to verify release-blocking fixes.

Wed 6/28: v1.7.0

Final release target. Official marketing, blogs, etc.

@enisoc

Deadline for feature work

What does it mean? We have two feature-related deadlines: feature submission deadline ("feature freeze") and feature development deadline ("coding freeze"). Here "coding freeze" is meant, right?

@idvoretskyi Yes, I meant coding. I've edited that line to be more specific.

lgtm
On Thu, Mar 23, 2017 at 11:14 AM grodrigues3<EMAIL_ADDRESS>wrote:

Just getting the ball rolling
Needs to incorporate feedback from 1.6 retrospective and needs approval
at community meeting
You can view, comment on, or merge this pull request online at:
https://github.com/kubernetes/features/pull/223
Commit Summary

Added the 1.7 proposed timeline

File Changes

A release-1.7/release-1.7.md
https://github.com/kubernetes/features/pull/223/files#diff-0 (58)
A release-1.7/release_team.md
https://github.com/kubernetes/features/pull/223/files#diff-1 (12)

Patch Links:

https://github.com/kubernetes/features/pull/223.patch
https://github.com/kubernetes/features/pull/223.diff

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/kubernetes/features/pull/223, or mute the thread
https://github.com/notifications/unsubscribe-auth/AACDCAQvxKlkI7_rgoBVbzOGbfrqtmsHks5rorZjgaJpZM4MnKID
.

Updated with @enisoc's changes.  Will merge later today and iterate as needed.

Besides the comments. LGTM


y_sort: make clear which item has to be drawn first when two have the same y-coordinate

If you use y_sort and you have the sprites of two entities with the same y-coordinate overlapping horizontally, it is undefined which one has to be drawn first which causes flickering.
This is especially bad when you are using tiles which are larger than the cell size of the grid so they can overlap (which I do a lot).
I added an additional comparation between the x-coordinates to the y-sorter. Not sure if this is wanted in the official build (dont know how much it actually affects performance), but I decided to give it a try.

Just happen that I wanted to report this as well. Ysort with TileMap is troublesome especially when making an isometric map. Tile object "with height" (in isometric) tends to produce the "z-fighting" issue even though they are created on a different grid (by projection wise is overlapping due to the "height" of the tile)

a nice addition in my opinion
this might be more performant by the way
float dy = p_right->xform.elements[2].y - p_left->xform.elements[2].y;
return CMP_EPSILON < dy || (dy >-CMP_EPSILON && p_left->xform.elements[2].x < p_right->xform.elements[2].x);

@tamkcr I bet Math::abs is already inlined by the compiler, so, unless you mind a few cpu cycles going to waste while compiling.... Also, yours would be slower in this case, as the compiler would be able to cache p_left->xform.elements[2].y - p_right->xform.elements[2].y

@bojidar-bg my intention is to try eliminating the branch which seem a sensible thing to do for a sort function? (just out of curiosity, I doubt y-sort for canvas items will ever cost anything significant on today's hardware outside benchmarks)
the dy is there for readability and I just trust the compiler will turn it to something just as fast (mind you my knowledge on compilers is very limited)

@tamkcr Well, if you really want to prove which method is faster, you should do something like:
int a = 0, b = 0; // Stuff that never gets actually used, but anyway
for(unsigned i = 0; i < 10000000; i++) {
  int y1 = rand() % 8000;
  int y2 = rand() % 8000; // Randomness to throw off computer if-branch prediction
  if (abs(y1 - y2) < 1) { // Test goes here
    a ++;
  } else {
    b ++;
  }
}
cout << a << b; // Use them, just in case the compiler gets smart enough to remove the if

And then run it through some timing command a few times, and average it. :smiley:
But, yeah, it is hardly any use for this, so, maybe better not (i.e. this is not the part that would bottleneck first).

just tried, in ~100000 range and below they are close, I would say the branched code is faster on average, the variance is quite large relatively speaking. branchless one is about 10-20% faster at 1000000+ iterations. (msvc)


how to train sigma lora?

I modify the lora code and import the sigma model success，but the valuation save image is not chance！

How about the inference output? Any changing?

@lawrence-cj
thanks your  response.
when I modify the sigma pipeline the lora work， but I can not train 1girl like SD, The inference result not much change

when I modify the sigma pipeline the lora work， but I can not train 1girl like SD

Sry, but I don't understand very clearly. Would you like to explain more?

@lawrence-cj like this lora https://civitai.com/models/104759/easy-age-slider-for-1girl,when I train sigma lora, the 1girl not change.


TST: PROJ 8.0.1 test_osgb_1936 failure

test_osgb_1936
https://github.com/snowman2/pyproj/runs/2250339603

Likely related: https://github.com/OSGeo/PROJ/pull/2625


fix bug in handler

The handler always fails when it is executed.
| bool fixes this.

LGTM, thanks!


fix lidar1_points collisions with tf

Tf axes appear to be solid to the lidar in the simulation and some points appear around them.

Solved by 7ca05b42d4ecfe67b7fa9bbbcb26117128149015


[wip] Add full version and init status to admin app

Initial proof of concept, lpc55-only.
$ nitropy nk3 version
Command line tool to interact with Nitrokey devices 0.4.32
status = 0b100
Warning: non-zero initialization status
v1.2.2
1.2.2-alpha.2023-01-31

→ https://github.com/Nitrokey/nitrokey-3-firmware/pull/172


Multiple Scom Interfaces

Hi,
First of all, many thanks for providing this module. It works pretty well!
My current setup is a bit overloaded. I run 3 xtender and each has its own Bus (xtender + BSP). With that setup, I currently have a Raspberry PI Zero for each xtender. Of course thats not ideal, especially because of maintenance, but also because of the power consumption and the coordination between the 3 xtenders (which needs another device). I would rather use one device (maybe a raspi 3 or 4) to communicate with all xtenders.
So my question: is there a way to run multiple scom interfaces in parallel? e.g. xtender 1 runs on tty0, xtender 2 on tty1 etc.
Since only one devicemanager and therefore one scom is initialized, I would say that's currently not possible.

Hi Mr Phil,
I'll not reply on your question as I don't know the module yet, sorry  ;-)
But about the Xtender system I have experience. There exists an "independant" mode when you have multiple units: param 1462 With that mode you can have all three Xtenders working together on the same Studer-bus but not synchronized as if it would be in parrallel or in threephased. This will be OK if you have one battery and one BSP in a system as multi BSP is not possible.
Maybe that could fit your case
Regards

Hi moixpo
Thanks a lot for the hint. I didn't know that. I would stick to the 3 independent battery systems, since that setup has also some advantages. especially if you have huge stack of lead-acid batteries that need a full load every couple of days, that maintenance load can be done independently.
Regards,
Phil


Release build includes orca.zip inside app folder

There is a 52mb zip in one of the folders that contain another build.
Accidental?
..Orca-win32-x64.zip\Orca-win32-x64\resources\app\orca.zip

Good spot thanks.
New builds much smaller :)


Add support for progress notifiers

Replaces https://github.com/realm/realm-cocoa/pull/4509.
TODO:

API discussion
Binding-level tests

I added tests for the "streaming" notifiers.
Unfortunately, the data transfer rate between the app and the prepackaged realm object server is way too fast to test the non-streaming notifiers. Testing those requires being able to register a notifier at a point in time when sync has begun, but not yet finished, an upload or download. This is impossible to do reliably, as far as I can tell, and we have no APIs that would facilitate that task.

PF can be used to throttle network connections (including on the loopback interface) so in theory we could do something like limit the connection to 1 KB/sec and then make a several KB commit and then have a several-second window where syncing should be in progress. Might be worth doing manually once to at least minimally verify that things work, but figuring out what testing hooks we need in the sync library would be a better use of time for building something robust that'd live in the actual test suite.

@bdash, if you could take a look at my usage of the CFRunLoop APIs I would appreciate it. I had a bit of trouble getting the two sync notifier tests to work properly, so I might have been doing something incorrectly.

Any ideas for better case names than RLMSyncNotifierModeAlwaysReportLatest and RLMSyncNotifierModeProgressIndicator are very welcome.

This PR depends on https://github.com/realm/realm-object-store/pull/332, which should be going in soon.

CI is still problematic. OS X tests passed locally.


Chrome 129 headless Error

Today, I updated the Chrome browser to the latest version (129).
After the update, running Chrome in headless mode produces an unusual result: a blank white window appears, matching the size of the browser.

We don't have support for 129 yet. Upstream change in puppeteer might come soon.

I tested on chrome128 and chrome130. there was no problem. I think chromium(129) has a problem.

Do you know how to fix this issue interim?

So you are using your installed browser rather than the one downloaded by puppeteer right?

Yes we are using installed browser

I tried to reproduce that. But I'm not getting that on Chrome 129.0.6668.60 on MacOS.

It seems this was reported on Selenium here. It was also reported on chromium bugs here.
One potential, an ugly, workaround is to add this command line argument --window-position=-2400,-2400.

It is also reported that you could replace Headless = false with HeadlessMode = HeadlessMode.Shell.

It is also reported that you could replace Headless = false with HeadlessMode = HeadlessMode.Shell.

That is not correct way, since that will point to old headless mode which is going to be out of support from chrome soon.
By default parameters, i.e new headless we are getting this white screen.

It is also reported that you could replace Headless = false with HeadlessMode = HeadlessMode.Shell.

That is not correct way, since that will point to old headless mode which is going to be out of support from chrome soon. By default parameters, i.e new headless we are getting this white screen.

Ok so you'll need to wait till the chromium team fixes the bug.

Can confirm that current build as of 26 Oct 2024 have this issue
I'm running next code
var browserFetcher = new BrowserFetcher();
await browserFetcher.DownloadAsync()
    .ConfigureAwait(false);

var launchOptions = new LaunchOptions()
{
    Args = new string[]
    {
        "--no-sandbox"
    },
    Headless = true
};
browser = await Puppeteer.LaunchAsync(launchOptions).ConfigureAwait(false);

Fetching loads Chrome/129.0.6668.100

Windows 11 23H2 (22631.4317)
net6.0

Waiting for new PuppeteerSharp version?


Lora resume training from checkpoint error

System Info
torch 2.1.2+cu121
transformers 4.33.0
transformers-stream-generator 0.0.4
peft 0.7.1
accelerate 0.25.0
deepspeed 0.12.5
nvcc --version release 12.2, V12.2.128
Build cuda_12.2.r12.2/compiler.33053471_0
nvidia-smi  525.147.05 Driver Version: 525.147.05 CUDA Version: 12.0
Who can help?
No response
Information

[ ] The official example scripts
[X] My own modified scripts

Tasks

[ ] An officially supported task in the examples folder
[X] My own task or dataset (give details below)

Reproduction
Code base I'm using
https://github.com/shibing624/MedicalGPT/blob/main/supervised_finetuning.py

Shell for training stage1
CUDA_VISIBLE_DEVICES=2,3 torchrun --nproc_per_node 2 ../supervised_finetuning.py \
    --model_type baichuan \
    --model_name_or_path /data/llm/models/Pretrained/Baichuan2-7B-Base/ \
    --train_file_dir ../data/finetune/blossom_stg1/ \
    --per_device_train_batch_size 10 \
    --do_train \
    --use_peft True \
    --num_train_epochs 1 \
    --learning_rate 2e-5 \
    --warmup_ratio 0.03 \
    --weight_decay 0. \
    --bf16 \
    --logging_strategy steps \
    --logging_steps 10 \
    --save_strategy epoch \
    --save_total_limit 5 \
    --gradient_accumulation_steps 1 \
    --preprocessing_num_workers 8 \
    --output_dir ../outputs/20231225_exp1_stg1 \
    --overwrite_output_dir \
    --ddp_timeout 30000 \
    --logging_first_step True \
    --target_modules all \
    --lora_rank 64 \
    --lora_alpha 128 \
    --lora_dropout 0.05 \
    --torch_dtype bfloat16 \
    --device_map auto \
    --report_to tensorboard \
    --ddp_find_unused_parameters False \
    --gradient_checkpointing True \
    --cache_dir ./cache \
    --model_max_length 4096 \
    --deepspeed ../deepspeed_zero_stage2_config_no16.json \
    --template_name baichuan2

shell for resume training stage2
CUDA_VISIBLE_DEVICES=5,7 torchrun --nproc_per_node 2 ../supervised_finetuning.py \
    --model_type baichuan \
    --model_name_or_path /data/llm/models/Pretrained/Baichuan2-7B-Base/ \
    --train_file_dir ../data/finetune/blossom_stg2/ \
    --peft_path ../outputs/20231225_exp1_stg1/ \
    --per_device_train_batch_size 10 \
    --do_train \
    --use_peft True \
    --max_train_samples -1 \
    --num_train_epochs 3 \
    --learning_rate 2e-5 \
    --warmup_ratio 0.03 \
    --weight_decay 0. \
    --bf16 \
    --logging_strategy steps \
    --logging_steps 10 \
    --save_steps 500 \
    --save_strategy steps \
    --save_total_limit 5 \
    --gradient_accumulation_steps 1 \
    --preprocessing_num_workers 8 \
    --output_dir ../outputs/20231225_exp1_stg2 \
    --overwrite_output_dir \
    --ddp_timeout 30000 \
    --logging_first_step True \
    --target_modules all \
    --lora_rank 64 \
    --lora_alpha 128 \
    --lora_dropout 0.05 \
    --torch_dtype bfloat16 \
    --device_map auto \
    --report_to tensorboard \
    --ddp_find_unused_parameters False \
    --gradient_checkpointing True \
    --cache_dir ./cache \
    --model_max_length 4096 \
    --deepspeed ../deepspeed_zero_stage2_config_no16.json \
    --template_name baichuan2

Error info
{'loss': 1.1006, 'learning_rate': 1.4442364268520997e-05, 'epoch': 0.9}
{'loss': 1.0321, 'learning_rate': 1.4415619149505215e-05, 'epoch': 0.9}
30%|█████████████████▍                                        | 2326/7710 [2:16:18<4:40:30,  3.13s/it]                                                                                ../aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [550,0,0], thread: [0,0,0]                                                                                Assertion `srcIndex < srcSelectDimSize` failed.
../aten/src/ATen/native/cuda/Indexing.cu:1292: indexSelectLargeIndex: block: [550,0,0], thread: [1,0,0]                                                                                Assertion `srcIndex < srcSelectDimSize` failed.

***multiple similar lines omitted***

Traceback (most recent call last):
  File "/data/mn/shibing624/MedicalGPT-1.6.3-231215/shell/../supervised_finetuning.py", line 1383, in <                                                                                module>
    main()
  File "/data/mn/shibing624/MedicalGPT-1.6.3-231215/shell/../supervised_finetuning.py", line 1344, in m                                                                                ain
    train_result = trainer.train(resume_from_checkpoint=checkpoint)
  File "/data/mn/anaconda3/envs/manuo/lib/python3.9/site-packages/transformers/trainer.py", line 1553,                                                                                in train
    return inner_training_loop(
  File "/data/mn/anaconda3/envs/manuo/lib/python3.9/site-packages/transformers/trainer.py", line 1835,                                                                                in _inner_training_loop
    tr_loss_step = self.training_step(model, inputs)
  File "/data/mn/anaconda3/envs/manuo/lib/python3.9/site-packages/transformers/trainer.py", line 2679,                                                                                in training_step
    loss = self.compute_loss(model, inputs)
  File "/data/mn/anaconda3/envs/manuo/lib/python3.9/site-packages/transformers/trainer.py", line 2704,                                                                                in compute_loss
    outputs = model(**inputs)
  File "/data/mn/anaconda3/envs/manuo/lib/python3.9/site-packages/torch/nn/modules/module.py", line 151                                                                                8, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/data/mn/anaconda3/envs/manuo/lib/python3.9/site-packages/torch/nn/modules/module.py", line 152                                                                                7, in _call_impl
    return forward_call(*args, **kwargs)
  File "/data/mn/anaconda3/envs/manuo/lib/python3.9/site-packages/torch/nn/parallel/distributed.py", li                                                                                ne 1519, in forward
    else self._run_ddp_forward(*inputs, **kwargs)
  File "/data/mn/anaconda3/envs/manuo/lib/python3.9/site-packages/torch/nn/parallel/distributed.py", li                                                                                ne 1355, in _run_ddp_forward
    return self.module(*inputs, **kwargs)  # type: ignore[index]
  File "/data/mn/anaconda3/envs/manuo/lib/python3.9/site-packages/torch/nn/modules/module.py", line 151                                                                                8, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/data/mn/anaconda3/envs/manuo/lib/python3.9/site-packages/torch/nn/modules/module.py", line 152                                                                                7, in _call_impl
    return forward_call(*args, **kwargs)
  File "/data/mn/anaconda3/envs/manuo/lib/python3.9/site-packages/accelerate/utils/operations.py", line                                                                                680, in forward
    return model_forward(*args, **kwargs)
  File "/data/mn/anaconda3/envs/manuo/lib/python3.9/site-packages/accelerate/utils/operations.py", line                                                                                668, in __call__
    return convert_to_fp32(self.model_forward(*args, **kwargs))
  File "/data/mn/anaconda3/envs/manuo/lib/python3.9/site-packages/torch/amp/autocast_mode.py", line 16,                                                                                in decorate_autocast
    return func(*args, **kwargs)
  File "/data/mn/anaconda3/envs/manuo/lib/python3.9/site-packages/peft/peft_model.py", line 1073, in fo                                                                                rward
    return self.base_model(
  File "/data/mn/anaconda3/envs/manuo/lib/python3.9/site-packages/torch/nn/modules/module.py", line 151                                                                                8, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/data/mn/anaconda3/envs/manuo/lib/python3.9/site-packages/torch/nn/modules/module.py", line 152                                                                                7, in _call_impl
    return forward_call(*args, **kwargs)
  File "/data/mn/anaconda3/envs/manuo/lib/python3.9/site-packages/peft/tuners/tuners_utils.py", line 10                                                                                3, in forward
    return self.model.forward(*args, **kwargs)
  File "/home/manuo/.cache/huggingface/modules/transformers_modules/modeling_baichuan.py", line 684, in                                                                                forward
    outputs = self.model(
  File "/data/mn/anaconda3/envs/manuo/lib/python3.9/site-packages/torch/nn/modules/module.py", line 151                                                                                8, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/data/mn/anaconda3/envs/manuo/lib/python3.9/site-packages/torch/nn/modules/module.py", line 152                                                                                7, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/manuo/.cache/huggingface/modules/transformers_modules/modeling_baichuan.py", line 420, in                                                                                forward
    attention_mask = self._prepare_decoder_attention_mask(
  File "/home/manuo/.cache/huggingface/modules/transformers_modules/modeling_baichuan.py", line 349, in                                                                                _prepare_decoder_attention_mask
    combined_attention_mask = _make_causal_mask(
  File "/home/manuo/.cache/huggingface/modules/transformers_modules/modeling_baichuan.py", line 62, in                                                                                _make_causal_mask
    mask = torch.full((tgt_len, tgt_len), torch.tensor(torch.finfo(dtype).min, device=device), device=device)
RuntimeError: CUDA error: device-side assert triggered
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below mig                                                                                ht be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.

Expected behavior
Problem Description:
I intend to perform two-stage SFT training with Lora. The base model is Baichuan-2-7B-Base,  the first stage uses Lora and Alpaca to train for one epoch, then the second stage uses Sharegpt and previous Lora weight to resume training for 2 more epochs.
The problem is that, during the second stage of training, when the process reaches close to 1epoch, the training process was terminated with the exception. My guess from the error message is that it has to do with peft, have you guys seen similar things?
thanks!

update:
I tried using a different dataset, and no error occurred. I then tried the original dataset (original error), only to swap the order of jsonl file for stage 1 and stage 2, and same error information occurred.
This is even more weird, since the sharegpt format file should be processed by datasets at the very beginning of the training. Why it waited till the very end of 1 epoch for the error message to show up?
Example jsonl
{"conversations": [{"from": "human", "value": "Mr. and Mrs. Boyden take their 3 children to a leisure park. They buy tickets for the whole family. The cost of an adult ticket is $6 more than the cost of a child ticket. The total cost of the 5 tickets is $77. What is the cost of an adult ticket?"}, {"from": "gpt", "value": "Let's say the cost of a child ticket is x dollars.\nThe cost of an adult ticket is x+6 dollars.\nThe total number of child tickets is 3.\nThe cost of the 3 child tickets is 3x dollars.\nThe total number of adult tickets is 2 because a family has 2 parents.\nThe cost of the adult tickets is 2(x+6) dollars.\nThe total cost of all the tickets is 77 dollars.\nThe expression representing the total cost of the tickets is 3x+2(x+6)=77\nSimplifying the expression gives us 3x+2x+12=77\nCombining like terms gives us 5x+12=77\nSubtracting 12 from both sides gives us 5x=65\nDividing both sides by 5 gives us x=13\nThe cost of an adult ticket is x+6 = 13+6 = $19."}]}
{"conversations": [{"from": "human", "value": "Hannah is at a restaurant. She spends $23 in total on an entree and a dessert. If the entree costs $5 more than the dessert, how much does the entree cost, in dollars?"}, {"from": "gpt", "value": "Let E be the cost of the entree and D be the cost of the dessert. We know that E + D = $23 and E = D + $5.\nSubstituting the second equation into the first equation, we get D + $5 + D = $23\nCombining like terms, we get 2D + $5 = $23\nSubtracting $5 from both sides, we get 2D = $18\nDividing both sides by 2, we get D = $9\nSubstituting D = $9 into the equation E = D + $5, we get E = $9 + $5 = $14."}]}

the issue seems to be my SFT data.


navbar: Replace gear menu with avatar and show status icon.

commit 1: Replace gear-menu with user avatar.
commit 2: Display status icon beside user avatar only when user has a valid user_status.
Testing plan:
local dev env
GIFs or screenshots:

@timabbott @amanagr It would be awesome if I can get a review on this !

LInk to the czo discussion of this PR?

LInk to the czo discussion of this PR?

@amanagr Here is the link to the conversation: https://chat.zulip.org/#narrow/stream/3-backend/topic/status-expire/near/1114757

This is the updated UI for the settings dropdown.

Current UI:

Setting margin-left of user_circle_green user_circle to 0 will make the name and status icons start in the same vertical line.

@ligmitz looks promising!

I think the space for your name isn't configured properly; I tried changing my name to something extra long and there's a lot of extra space:


 It is recommended to rebase your git history and force push to overwrite this history. Besides, remember to revoke your DeepL API key and generate a new one.
To avoid this issue happen again, it is recommended to use a crate like https://github.com/allan2/dotenvy and read the API key from environment variable.

Got it, thanks for making me aware of the problem. I'll see if I can revoke the API key and get a new one


Docker-compose container doesn't connect to IRC over IPv6 and times out

Describe the bug
So I just stumbled onto this very strange issue. When I run the container using docker run the container gets an IPv6 address and can connect to IRC successfully, but when started using docker-compose it times out.
To Reproduce
The working docker run command is: docker run -v /home/user/docker/matterbridge/matterbridge.toml:/matterbridge.toml 42wim/matterbridge -debug
The compose file that times out:
version: '2.4'

services:
  matterbridge:
    image: 42wim/matterbridge
    restart: unless-stopped
    container_name: matterbridge
    command: "-debug"
    volumes:
      - ${DOCKER_DIR}/matterbridge/matterbridge.toml:/matterbridge.toml
    networks:
      default:
        ipv6_address: 2001:db8:123:d008:4:455c:75c9:2818

networks:
  default:
    enable_ipv6: true
    driver: bridge
    ipam:
      driver: default
      config:
      - subnet: 2001:db8:123:d008:4::/80
        gateway: 2001:db8:123:d008:4::1

After that docker-compose up -d.
Expected behavior
It is expected to connect to the IRC server as it would when run using docker run. It can also successfully connect to other sites using IPv6, I've ran this setup for a few years now with other containers.
Screenshots/debug logs
[0000]  INFO main:         Enabling debug logging.
[0000]  INFO main:         Running version 1.17.5-dev 52a071e3
[0000]  INFO main:         WARNING: THIS IS A DEVELOPMENT VERSION. Things may break.
[0000]  INFO router:       Parsing gateway gateway1
[0000]  INFO router:       Starting bridge: irc.myirc
[0000]  INFO irc:          Connecting irc.snt.utwente.nl:6697
[0005] ERROR irc:          disconnect: error: dial tcp: i/o timeout
[0005] FATAL main:         Starting gateway failed: Bridge irc.myirc failed to start: connection failed dial tcp: i/o timeout

This timeout message appears fairly quick after starting the container. When running a tcpdump on the host I don't see any traffic going out to the IRC server, but I do see outgoing traffic to Telegram (the other protocol configured) and I also see dns requests for the Telegram API.
Environment (please complete the following information):

OS: Debian 10.3
Matterbridge version: 1.17.5-dev
If self compiled: output of git rev-parse HEAD

Additional context
[irc.myirc]
Nick="<nick>"
NickServNick="<nickname>"
NickServPassword="<password>"
Server="irc.snt.utwente.nl:6697"
UseTLS=true
UseSASL=true
SkipTLSVerify=false
RemoteNickFormat=""

[telegram.mytelegram]
Token="<token>"
RemoteNickFormat="<{NICK}> "

[[gateway]]
name="gateway1"
enable=true

[[gateway.inout]]
account="irc.myirc"
channel="#<channel>"

[[gateway.inout]]
account="telegram.mytelegram"
channel="<channel>"

Have you seen https://docs.docker.com/config/daemon/ipv6/ ?

Hi @qaisjp, yes I have. Outgoing network traffic (both IPv6 and IPv4) works fine, I can see the container requesting dns records (over IPv6) for the Telegram API and it also connects to the Telegram API (also over IPv6).
Other containers on the same bridge work perfectly fine with IPv6 too. It just doesn't request dns records for irc.snt.utwente.nl and it never makes outgoing requests to the irc server.
Even if IPv6 wasn't working it would be able to fall back to IPv4 (the docker-compose does work with an IPv4 bridge for some reason).
The thing that I find really strange is that when I use the docker run command it also automatically gets an IPv6 address (the prefix is configured in the /etc/docker/daemon.json file) but then it can connect to the IRC server (I haven't ran across this problem with all of the other containers I'm running).

Timeout seems like a firewall issue.
Sorry but debugging docker issues is out of scope for the project.
You can always just run the binary, no need for a docker container.

In case anyone stumbles across this problem in the future, I did manage to find a workaround for this issue.
Turns out it wasn't a firewall issue, but the usage of a named Docker network.
When the default bridge gets used (also configured with IPv6) for the matterbridge container it is able to connect with IRC servers over IPv6. Only when a named Docker network gets used matterbridge will fail to connect to IRC servers over IPv6.


HyprLandAPI registerCallBackDynamic API change

Hyprland Recently changed their API and Hyprgrass callbacks need to be changed accordingly.

Fixes building Hyprgrass against Hyprland (Git/HEAD).

LGTM. thanks!

oops, closed it by accident - I can't merge it anyway. that's on your end. lol
( sorry for the extra noise )


Add json.json

Because markup languages count too!

Hello world in every programming language.
You need another programming language to print the "Hello World", so -1 from me.

I would have to agree with @greensn0w the json is not a programming language.  The reason that we should not accept this change is because if the project goes in a direction of providing a sample program for each language we will be unable to write one for json.

Other non-programming languages in this repo:
https://github.com/leachim6/hello-world/blob/master/x/xml.xml
https://github.com/leachim6/hello-world/blob/master/h/html.html
https://github.com/leachim6/hello-world/blob/master/c/css.css
https://github.com/leachim6/hello-world/blob/master/x/xslt.xslt
Please let me know why these qualify and JSON does not.

I totally see where you are coming from. Perhaps we should accept this pull request right now because of the above mentioned languages. However take a look at issue #158 in which we are discussing if this project will morph into something more than printing "hello world".

I think neither this nor the xml.xml should be (have been) accepted. The html, css and xslt are acceptable because

xslt is actually Turing complete so is literally a programming language.
The html and css when processed "in the normal way" both provide just "Hello World!" with no extra characters.

The XML could be rescued if it's reformulated as xhtml; possibly.
However, I don't see any 'normal processing' that would make the json work.


Logo image tries to load from localhost even on remote machine

Describe the bug
If I set the logo to a custom image and save the changes, I can only see the custom logo when viewing BookStack from the local machine (navigating to localhost). On a remote machine on the local network the GET request for the image fails because the URL in the <img> tag is still using http://localhost.
Steps To Reproduce
Steps to reproduce the behavior:

Change the BookStack logo on a fresh install to some custom image
Go to BookStack on a another machine using the BookStack machine's IP address and port number
The logo doesn't load because the <img> tag src attribute starts with http://localhost and of course since we are on a remote machine this doesn't work
Try copying the URL into the browser and changing localhost to the correct IP address and port and verify that the image can still be accessed as expected... it should work fine

Expected behavior
The logo shows up no matter where the page is loaded from. The src attribute of the <img> tag containing the logo should start with something like http://<ip-address> like every other URL on the page. Even the parent <a> element with the logo class just outside the logo <img> tag has its src attribute correctly set to the IP address of the BookStack machine.
Screenshots

Your Configuration (please complete the following information):

Exact BookStack Version (Found in settings): v0.27.4
PHP Version: 7.2.22
Hosting Method (Nginx/Apache/Docker): Docker on Windows, Linux Containers

Additional context
Of course when viewing the page from the same machine as the server is running on there is no issue because http://localhost will have no issue resolving.

Ok so I've just discovered that if I set the logo image from the other machine then everything works fine because the src attribute URL then correctly has the IP address in it. Interestingly, the IP address also shows up in the src attribute URL when the page is viewed from localhost on the local BookStack machine too.
So it seems the first part of the URL is not dynamically changing based on where the page is viewed from like all the other links on the page. I dug through the source code a little bit and although I'm not familiar with PHP it seems that the problem is likely in SettingController.php on line 70 where the image URL is saved to the app-logo setting with setting()->put('app-logo', $image->url);.
I'm not sure how to confirm this, but I suspect that the value saved to the app-logo setting is something like http://<ip-address>/uploads/images/system/2019-10/logo.png rather than simply /uploads/images/system/2019-10/logo.png.
I'll bet that in header.blade.php when url(setting('app-logo', '')) is run the url function does nothing because the value of app-logo is already a complete URL.

Ok so it seems that changing line 70 in SettingController.php from this...
setting()->put('app-logo', $image->url);

...to this fixes the problem.
setting()->put('app-logo', parse_url($image->url)['path']);

Any comment on whether or not I'm doing the correct thing here?

Hi there, you mentioned that you're hosting with Docker. Do you have a proxy in front of the container that is forwarding requests to it?
If so, you may need to tell BookStack which URL is actually used to access it (rather than what it sees in the request)
Check out this line in your configuration to see if it applies to you: https://github.com/BookStackApp/BookStack/blob/master/.env.example#L11
You may also be able to configure your proxy to forward the headers "as-is" in the original request from your browser. However, the APP_URL configuration would be the more straightforward fix of these two.
One gotcha to watch out for is that this will only apply to newly uploaded images. I keep stumbling across pages with references to images that were uploaded when my BookStack instance was accessed with a different URL, and just needs manually updated.

I am not using any proxies whatsoever. The Docker container is simply run with the -p 80:80 argument to bind to port 80 on the host and make the web server accessible anywhere the host is accessible.
As you can see in my previous comments, I believe the issue is that the logo image is incorrectly saved to the application settings as a full URL instead of just a path.

Thanks for clarifying. I have seen this behaviour before with regular images, but not the logo which is why I suggested the config check. I've done a couple of tests with my own instance, and here are my findings:

Uploading an image with APP_URL set uses that URL, even when accessing the site with another URL.
Uploading an image with one URL, then changing APP_URL does not change the path of the logo URL.
The default logo uses APP_URL no matter what URL you actually access it from, so is behaving correctly.

So I can reproduce the issue you're facing with custom logos. One for @ssddanbrown to triage as a potential bug?

Thanks for trying to assist here @cw1998.
The fundamental thing is that BookStack does not support being access via multiple base URLs. The app logo won't be the only thing that you'll have issues with if you're not using a fixed base URL via setting APP_URL.  Issue #1342 exists as a request to allow a relative base.

Since the last comment on this issue is relatively old I'm going to close this. If the issue remains and is something you still require to be fixed please open a new issue, referencing this one.


fixed cpc module issue

What does this PR do?

Fixes #679
Before submitting

[x] Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
[x] Did you read the contributor guideline, Pull Request section?
[x] Did you make sure your PR does only one thing, instead of bundling different changes together?
[n] Did you make sure to update the documentation with your changes?
[n] Did you write any new necessary tests? [not needed for typos/docs]
[n] Did you verify new and existing tests pass locally with your changes?
[ ] If you made a notable change (that affects users), did you update the CHANGELOG?

PR review

[x] Is this pull request ready for review? (if not, please submit in draft mode)

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
Did you have fun?
Yes :)
Make sure you had fun coding 🙃

Oops sorry I didnt mean to request review again from @Borda 😬


[Feature/extensions] Modified Environment settings support

Description
Environment Settings support for extensibility will now pass the entire Settings object from OpenSearch node environment to the SDK.
Companion PR : https://github.com/opensearch-project/OpenSearch/pull/4731
Issues Resolved
https://github.com/opensearch-project/opensearch-sdk-java/issues/78
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Hey @joshpalis I just noticed the following in some gradle checks:

/home/runner/work/opensearch-sdk-java/opensearch-sdk-java/src/main/java/org/opensearch/sdk/handlers/EnvironmentSettingsResponseHandler.java:70: warning: no @throws for java.lang.InterruptedException

Can you please add that @throws in one of your next PRs?

@dbwiddis Sure thanks for catching that, I'll make sure to add this