How can we close gap on client performance vs blocking sockets?

pgjdbc-ng uses Netty in a client environment and when we compare performance with that of pgjdbc it can be any where from 100% slower (on localhost) to 15% slower (on any remote host).
After extensive investigation and many failed implementations, we're at a dead end with making it any faster using the currently provided Netty facilities.  We desperately want to keep Netty because it makes the implementation drastically simpler but at some point the performance gap needs to be closed.
The mainline driver uses non-blocking sockets and "no threads" which makes it limiting and unable to implement some features of the JDBC specification correctly. Clearly we need to retain the asynchronous nature of the driver and realize that may impose a performance cost above an unthreaded non-blocking implementation; but we need it to be much closer than it is.
FYI, I tried OioSocketChannel and OioEventLoopGroup because the thread-per-connection model seemed promising; realizing they are deprecated.  Unfortunately they appear broken in Netty 4.  Due to writes being directed to the event loop thread, and the fact that it uses blocking reads with a periodic timeout, the writes are starved until the read time's out and the event loop thread can process them. I hacked a version that allow writes from any thread but it provides only a small performance improvement (and is a dubious solution given Netty's requirements).
So, the basic question is... what are we doing wrong and how can we solve this problem?  After reading everything available here, on netty.io, on stackoverflow and pretty much any related hit on Google... this is my last hope.
Minimal yet complete reproducer code (or URL to code)
pgjdbc-ng
Netty version
4.1.42
JVM version (e.g. java -version)
8, 9, 10, 11, 13
OS version (e.g. uname -a)
Darwin .local 18.7.0 Darwin Kernel Version 18.7.0: Thu Jun 20 18:42:21 PDT 2019; root:xnu-4903.270.47~4/RELEASE_X86_64 x86_64

@kdubb how do you compare performance? do you have benchmarks that you can share?

For anybody examining the code:
Bootstrap
The basic creation via Bootstrap is here:
https://github.com/impossibl/pgjdbc-ng/blob/735348296c148f93ad89f5007bdeca15965963f1/driver/src/main/java/com/impossibl/postgres/protocol/v30/ServerConnectionFactory.java#L289
SSL is dynamically added here
https://github.com/impossibl/pgjdbc-ng/blob/735348296c148f93ad89f5007bdeca15965963f1/driver/src/main/java/com/impossibl/postgres/protocol/v30/ServerConnectionFactory.java#L170
Channel Handler
The protocol handler is a ChannelDuplexHandler located here:
https://github.com/impossibl/pgjdbc-ng/blob/develop/driver/src/main/java/com/impossibl/postgres/protocol/v30/MessageDispatchHandler.java

I've created a simple test project that displays the same issue. ehttps://github.com/impossibl/netty-test
TestServer is a simple server that responds to requests by sending a requested amount of data back to the client with an id. TestClient uses Netty to generate & send requests and wait for responses like pgjdbc-ng does.  TestBlockingClient uses a Socket to send the request then does blocking reads to read the response, like pgjdbc does.
TestClient: ~5.8s
TestBlockingClient: ~3.7s

Mark.
This is an interesting issue. I haven't researched your code, but I will try some in the near future...

@carryxyh I'm willing to help all I can.  I have hacked together a number of alternatives in the driver and ended up throwing them away as they didn't change much.

@kdubb
Hi, unfortunately, I'm not familiar with the gradle project. I have built it in my idea for a long time and haven't built it successfully.
But I still have some findings that might help.
It looks like your pipeline is very simple, there is only a LengthFieldBasedFrameDecoder and a handler (as you said: MessageDispatchHandler). Can you try to use ThreadPoolExecutor to process the message when MessageDispatchHandler receives the message instead of processing it directly in channelRead (Or you have already done this, if you have used ThreadPoolExecutor, please ignore me) ?

Another thing, it seems that the flushTrace method does not need to be sync called in the channelReadComplete. Can u try to put it in the ThreadPoolExecutor?

@Liyixin95 Apparently I missed this comment many months ago... Yes TestClient is synchronous, like TestBlockingClient is, to show that merely switching between Netty and a blocking socket has a performance penalty.
The actual driver implementation pipelines request whenever possible.


Have question some device problem in visual studio  code pynvme plug in

Hi
Some enterprise essd have problem in visual studio code plug in queue window
These device print some hex code at device active (nvme0=d.Contoller(...)) unlike normal device
(After MSIX address print)
Device operation no problem.  But not show at queue pair status window
How can i see all log at this time?
Thank you

Hi,
Could you please upload or email us a screenshot and the test.log generated in the test? We can learn more details of your situations.
Thanks,
-Crane

Hi.
no test.log.
I only run  'sudo python3' and 'import nvme as d' , ' nvme0=d.Controller(b'0c:00.0')
result is
INFO intr_mgt.c(176) msix_intc_int: vextor_num 32
INFO intr_mgt.c(183) msix_intc_int: msix table addr<PHONE_NUMBER>00
INFO intr_mgt.c(192) msix_intc_int: msix table addr 2 0x201000800000
INFO intr_mgt.c(195) msix_intc_int: msix table addr<PHONE_NUMBER>00
00000000 18 62 32 61 13 5b 45 5f b9 4e c5 34 66 1b 9e 69 .b2a.[E_.N.4f..i
INFO driver.c(601) nvme_init: found device: 0000:0c:00.0, 0x2000201e6140

other device not print abnormal hex code. only this device occur this issue
operation no problem.

Thanks for reporting the detail finding. We will investigate it.

Controller Attributes (CTRATT): Bit 0 if set to ‘1’, then the controller supports a 128-bit Host Identifier.

Your enterprise SSD likely supports this feature, and so SPDK generates the ID, and print it in the log.
So, it is just normal output.

Thank you
If ctratt is set then unavailable visual code plugin?

Could you please upload a screenshot? I may need your help to reproduce and test this issue. I dont have the device to reproduce it.
Thanks,
-Crane

Sorry~ can not take screenshot because company security reason
normal sample all show Qpair Status after nvme0=d.Controller(b'xx:xx.0')
but some sample not show qpair status after nvme0=d.Controller(b'xx:xx.0').
only progress bar blink upper "PYNVME QPAIR" status window name
other enterprise sample and consumer ssd sample no problem.
how can i see driver level detail log?

Thank you
i tested debug driver.
i compared issue devices to normal device debug message. no difference devices except one thing
(i disabled 128bit Host ID to issue device. but still remain issue)
issues device have special unicode in fw name place last character (0xfffd)
each device print DEBUG driver.c(510) attach_cb: attached device...
normal device : 0000:be:00.0:  device name 1.0.4  , 1 namespaces, pid xxx
issue device : 0000:bc:00.0: device name 1.0.4.�, 1 namespaces, pid xxx
Can certain Unicode cause problems?

Hi,
Can you share with me the debug log of the device with issues? Maybe I can find some more interesting points.
For those Unicode device name, it violates the spec, but I don't think it would cause any problem.
AQA set before base address: it is also fine, only if those registers are written before CC.EN
In you first message, you mentioned:

Device operation no problem. But not show at queue pair status window
So, the devices with symptoms of unicode/nvme-mi/AQA still can work as usual, but only vscode plugin does not display qpairs of them?

You can send me email if it is more efficient for you.
Thanks,
-Crane

Here is my test.log file when I run the commands below with branch issue_31:
make setup
make test TESTS=scripts/demo_test.py::test_debug

test.log

ok i see seems like this log
but can  not find difference  point
i dont know why can not display vsc plugin queue starus

If you can send me the log file, I may find more details.

Ok i will try logfile to you

Hi, Catogts,
We just meet a similar issue recently that vscode cannot display controllers and qpairs. We have fixed it in the branch: https://github.com/cranechu/pynvme/tree/pynvme_1.5
Could you please repeat the test with your SSD?

Thank you!
Works great

Perfect!


Allow some basic HTML tags in story description

Nice work, @julinvictus!
Can you make this work on the React side as well?

Looks good! We'll just need to make sure this also works in the React front end. I think it'd be fine to accept this as-is, but to merge we should first do a follow-up PR, or amend this PR to also get the HTML tag working in React.

I can take a look at the react part

When you say React you mean this part here, right?

When you say React you mean this part here, right?

Correct


zqd support for space ids

Add support for space ids, uuid-like immutable identifiers that will be used instead of space names as the resource id in api calls. The id should be created by zqd when the space is returned, and passed back in the create response struct.
To make the transition easy from the existing name-based calls, I suggest that we continue to treat the directory name under the spaces directory as the space's id. So before this work goes in, if there was a directory spaces/foo, because space foo was created, we treat foo as the id for that space going forward. After this work goes in, though, when a new space is created with name bar, the fs would look like:
space/ae123428d33.../
space/ae123428d33.../config.json
where config.json now has an entry name: "bar".

I triggered the creation of a couple Spaces, one via pcap import and one via Zeek log import. I can see the opaque Space names now below my data/ directory.
~/Library/Application Support/Brim/data/spaces$ tree
.
├── sp_1c8HwI2VOUQWlgvIoELq51XrFZG
│   ├── all.zng
│   ├── config.json
│   ├── info.json
│   └── packets.idx.json
└── sp_1c8Hz0OM81fitoUQagFvW0uLKrk
    ├── all.zng
    ├── config.json
    └── info.json

2 directories, 7 files

Thanks @mattnibs!


'Client' object has no attribute 'get_trafficmanager'

Sorry,I meet some problem in the program: The problem is in tools.modules.py 1065 lines,  in class ModuleWorld() self.client.get_trafficmanager.And I use the carla0.9.9.

Sorry,I meet some problem in the program: The problem is in tools.modules.py 1065 lines, in class ModuleWorld() self.client.get_trafficmanager.And I use the carla0.9.9.

I do got this erro, did you work out this error?


Suggestion: Improve generated code comments for godoc

Current behavior: https://pkg.go.dev/github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/keyvault/armkeyvault#KeysClient.Get

Comparison of before vs. after:

go has support for some document formatting. See here. I'm suggesting two changes to take use of formatting:

API version generated should be its own paragraph. This allows the first sentence to accurately describe method behavior with API version gen as an addendum.
Parameter lists are formatted as a list

When I look at this locally it's slightly different.  Looks like a code block, not a list.  How did you create your snippet?

When I look at this locally it's slightly different. Looks like a code block, not a list. How did you create your snippet?

That's weird. This exact code snippet looks correct on mine:
// GetEmpty - Get complex types with array property which is empty
// If the operation fails it returns an *azcore.ResponseError type.
//
// Generated from API version 2016-02-29
//   - options - ArrayClientGetEmptyOptions contains the optional parameters for the ArrayClient.GetEmpty method.
func GetEmpty() {
}

@jhendrixMSFT go would detect that as a codeblock if the bullet list is somehow unrecognized:
From the docs on Code Blocks:

A code block is a span of indented or blank lines not starting with a bullet list marker or numbered list marker. It is rendered as preformatted text (a <pre> block in HTML).

I could reproduce this behavior but only if I made the bullet list invalid (like not introducing a space after the bullet list marker):
// GetEmpty - Get complex types with array property which is empty
// If the operation fails it returns an *azcore.ResponseError type.
//
// Generated from API version 2016-02-29
//	-options - ArrayClientGetEmptyOptions contains the optional parameters for the ArrayClient.GetEmpty method.

@weikanglim did you generate your snippet running a local instance of pkgsite or is this on the official site?

My bad, I was running an older version of pkgsite. Things look as expected.


Preview can sometimes get too large

An image will explain this one.
Just needs a better way to check bounds and apply an even border around the preview.

Fixed by #39


Frontend/login reset

:tickets: Ticket(s): Closes #51

:construction_worker: Changes
A brief summary of what changes were introduced.
Created Login and reset pages. Button should send axios request to backend. This will be implemented after endpoints are set up.
:flashlight: Testing Instructions
Explain how to test your changes, if applicable.
Look and marvel.

For some strange reason though codegen still fails. This branch doesn't have any proto rebuilds since it's just a fresh merge pull from master + frontend changes.

@LynxHack try to update to go protoc plugin:
go get -u github.com/golang/protobuf/protoc-gen-go

ill look into resolving the silent protoc-gen-go updates at some point


Add a link to open the RSS feed in a new tab in addition to copying it

My Feeds now displays a hyper link that a user can click on which can take them to the RSS feed


Add language to the historic underinvestment to "explain" why it is missing for some tracts

Is your feature request related to a problem? Please describe.
The HOLC data only exists for certain tracts (mostly large cities). This means for some tracts it is not expected and it has to be not shown on the side panel.
Describe the solution you'd like
Update the content 1.0 doc to include some language around this.
Describe alternatives you've considered
Kameron would like to propose another solution for "not expected" data at some time.
Additional context
Emma said it has to work like this.

@BethMattern can you put the language we wordsmithed in this ticket?


Change fullscreen icon

This came up in a Twitter discussion:
https://twitter.com/01241/status/666901750557892609
Proposal is: http://fiddle.jshell.net/Markus_Steiger/8q4mtxL3/
I agree that the new icon is better, what do others think?

Agreed. Strange to get this as tweet instead of a pull request though.

Working on PR now on this one.

Instead of the implementation in the fiddle, I'd suggest using a different label:
⤢
NORTH EAST AND SOUTH WEST ARROW
Unicode: U+2922, UTF-8: E2 A4 A2


Save Correct Output In JSON When Selecting Color-Space

Is your feature request related to a problem? Please describe.
Currently, a user has to manually paste the RGBA or HSL value in the field to get the selected space's value as output in json. However, when using the color-picker the plugin saves it as a hex-value in json regardless of which space is selected.
Describe the solution you'd like
Support for saving the corresponding value depending on what space is selected.
Screenshots & Recordings:
Selecting space via the color-picker:

Color-picker only gives hex-value as output
https://github.com/tokens-studio/figma-plugin/assets/93133012/418ba3b5-deca-4eff-8e23-20566f5ebd99

This request has been moved to our new feedback tool. Go check out https://tokensstudio.featurebase.app/ to create a free account. You can upvote and comment on existing posts, make a new request, and subscribe to be notified when requests move into our roadmap.

Added to FeatureBase Roadmap : 🎨 Expanded support for color spaces


[wip] fix: ignore package lock

The update-electron-dependencies script is not installing devDependencies. Removing the package lock file ensures that the script is doing a fresh npm install from scratch.
🍐 @vanessayuenn

This turned out to be a red herring in our investigation of #1323


feat: adds base64 support in webui token (#840)

Type: Feature
The following has been addressed in the PR:

There is a related issue #840
Unit or Functional tests are included in the PR

Description:

adds base64 support in webui token.

For more information:
It's a small library<EMAIL_ADDRESS>I tried the solution mentioned here https://developer.mozilla.org/en-US/docs/Web/API/WindowBase64/Base64_encoding_and_decoding but tests output are different in our case.

How we can test this?

From MDN
Solution #1 – escaping the string before encoding it
function b64EncodeUnicode(str) {
    // first we use encodeURIComponent to get percent-encoded UTF-8,
    // then we convert the percent encodings into raw bytes which
    // can be fed into btoa.
    return btoa(encodeURIComponent(str).replace(/%([0-9A-F]{2})/g,
        function toSolidBytes(match, p1) {
            return String.fromCharCode('0x' + p1);
    }));
}

b64EncodeUnicode('✓ à la mode'); // "4pyTIMOgIGxhIG1vZGU="
b64EncodeUnicode('\n'); // "Cg=="

To decode the Base64-encoded value back into a String:
function b64DecodeUnicode(str) {
    // Going backwards: from bytestream, to percent-encoding, to original string.
    return decodeURIComponent(atob(str).split('').map(function(c) {
        return '%' + ('00' + c.charCodeAt(0).toString(16)).slice(-2);
    }).join(''));
}

b64DecodeUnicode('4pyTIMOgIGxhIG1vZGU='); // "✓ à la mode"
b64DecodeUnicode('Cg=='); // "\n"

but output were different in our test case so I ended up using<EMAIL_ADDRESS>


Change download file name

Hey, great script, not at issue at all just a question.
Currently the file downloaded when an .ics is created is named 'download'. Is there any option available to change it? Or maybe to make it dynamic?
Thanks

On your download link you need to specify the download attribute
For example:
`Save to calendar

My bad, thanks


feat(secureHeader): add Report-Only support

close #3412
The author should do the following, if applicable

[x] Add tests
[x] Run tests
[x] bun run format:fix && bun run lint:fix to format the code
[x] Add TSDoc/JSDoc to document the code

Hi @isoppp
Thank you for the PR!
@watany-dev Can you review this?

Thank you, I think this option looks good. The implementation also seems promising.


Everything is dark when shadow isn't enabled

Godot 3.0 master
nVidia 940M
Windows 10 64 bits
I noticed that if I disable shadows on a directional light, everything is dark as if all was in shadow:

ShadowQuality.zip

Bisected:
git bisect start
# good: [5d85108f9448ac47b871f19c8af4652b7d3dea9d] Merge pull request #10428 from akien-mga/minizip
git bisect good 5d85108f9448ac47b871f19c8af4652b7d3dea9d
# bad: [28c85922b12cb0fa8c2cd1465cbeb0c0d226b880] Merge pull request #10716 from Noshyaar/pr-update
git bisect bad 28c85922b12cb0fa8c2cd1465cbeb0c0d226b880
# good: [78d9936f8d63754a1480f99ad14a5eee642380a9] Merge pull request #10573 from tagcup/gizmo_aabb_scaling_order
git bisect good 78d9936f8d63754a1480f99ad14a5eee642380a9
# good: [5976e8d7ded314e56a2bb80406ea7eee9354e80c] Merge pull request #10625 from Rubonnek/fixed-leaks
git bisect good 5976e8d7ded314e56a2bb80406ea7eee9354e80c
# bad: [59d1f96418540b6d1b8fec5892b3c1b7d1ae69e1] Merge pull request #10688 from Noshyaar/pr-csv2
git bisect bad 59d1f96418540b6d1b8fec5892b3c1b7d1ae69e1
# bad: [560fc0f19932a7b51e02e9eda8a64f284f41e6d0] stb: Update to upstream stb_truetype 1.17 and stb_vorbis 1.11
git bisect bad 560fc0f19932a7b51e02e9eda8a64f284f41e6d0
# bad: [9c71da00aa02680659131656f662462f37235bd3] Merge pull request #10590 from MasonAsh/fix-10589
git bisect bad 9c71da00aa02680659131656f662462f37235bd3
# bad: [612099e3771578fd3e78171f1ca1444f6c41cef3] Merge pull request #10591 from Rubonnek/possible-null-ptr-dereference
git bisect bad 612099e3771578fd3e78171f1ca1444f6c41cef3
# good: [ea55b400d908c01345a974cae82d8a8720e9904d] Merge pull request #10598 from Rubonnek/free-memory
git bisect good ea55b400d908c01345a974cae82d8a8720e9904d
# bad: [7a07895920196c00d1ee14187e4ccdb2a6f0d0b9] Added/Fixed null pointer checks
git bisect bad 7a07895920196c00d1ee14187e4ccdb2a6f0d0b9
# first bad commit: [] Added/Fixed null pointer checks

So, 7a07895920196c00d1ee14187e4ccdb2a6f0d0b9 is the culprit, somehow, ping @Rubonnek.
Guess it is related to this patch:
@@ -1482,11 +1482,10 @@ void VisualServerScene::_render_scene(const Transform p_cam_transform, const Cam

                        if (light && p_shadow_atlas.is_valid() && VSG::storage->light_has_shadow(E->get()->base)) {
                                lights_with_shadow[directional_shadow_count++] = E->get();
-                       }
-
-                       //add to list

-                       directional_light_ptr[directional_light_count++] = light->instance;
+                               //add to list
+                               directional_light_ptr[directional_light_count++] = light->instance;
+                       }
                }

Hm yeah this change did look kind of buggy, bbut I assumed that it had been done for a reason,
I guess it needs to be changed to this instead:
if (light) {
    if (p_shadow_atlas.is_valid() && VSG::storage->light_has_shadow(E->get()->base)) {
        lights_with_shadow[directional_shadow_count++] = E->get();
    }

    //add to list
    directional_light_ptr[directional_light_count++] = light->instance;
}

@akien-mga It was done to mitigate a cppcheck message related to null pointer dereferencing. I forgot to pull the light pointer check to another if statement just like you did.


Remove unused/broken references to util_dynamic_array.h

I assume that external sources of util_dynamic_array.h still exist. If not, I'm happy to remove the preprocessor macro related to this also.

Suggestion: to help make the Git history more readable, I recommend rebasing changes on the main branch before merging them.

Hi @johnmcfarlane what I did was pull util::dynamic_array into the header file uintwide_t. And the uintwide_t class needs util::dynamic_array for very large digit counts.
If the project has an external source of util::dynamic_array, then WIDE_INTEGER_DISABLE_IMPLEMENT_UTIL_DYNAMIC_ARRAY should be defined and the header uintwide_t will not implement its own copy of util::dynamic_array.

So your change is fine for the CMake build, but the proprocessor maccro needs to remain. Personally, i'm not very happy with the way i use util::dynamic_array and wish that C++ would offer such a fixed-size array class with dynamic, one-shot, fixed-width allocation. Maybe there is such a container by now, but I personally am not aware of it.

@ckormanyos yes there are many nearly-the-same containers and proposals for many of them. However vector covers so much area that the are not super-urgent.
Have you considered vector? I know it's larger than {pointer, size} but if you were to implement an elastic number which is growing exponentially (e.g. many successive calls to n *= 42;), then this would involve fewer dynamic allocations.
Alternatively, does unique_ptr<T[]> help in any way?

Have you considered vector? I know it's larger than {pointer, size} ...

Yes John, this is actually a long story.
Believe it or not, in fact, some of the systems that I originally used (and one system I still use today) with uintwide_t do not have STL implementations --- so no std::vector available either. So I opted to write a custom container as well as a bunch of easy-to-write other STL stuff like array, etc. With these methods, all my projects and samples --- even for the most tiny microcontrollers --- work fine.
In cnl, I might recommend to eventually kick out the self-written container (or switch it for a replacement enabled/disabled via  preprocessor option) and go with a constant-sized kind of a std::vector. When doing this sort of thing, I usually end up wrapping std::vector with a templated Size parameter.
The code using the container should be container independent. What we ultimately do in cnl is not yet known to me. But at least I got the whole story out there.

Got it. Then can I suggest the adaptor pattern as used by std::queue where you pass the container type into the number type as a template parameter?
I would recommend you split up the type into two types - roughly along the lines of P1889's bounded and unbounded types. Here's an example of how you might go about this without duplicating too much code.

... recommend you split up the type into two types - roughly along the lines of P1889's bounded and unbounded types. Here's an example of how you might go about this without duplicating too much code.

Hi @johnmcfarlane I am confused how/if there is a template parameter for is_signed/is_unsigned? Is that the parameter S in the PR? What type is it? I had expected something like template parameter bool is_unsigned = false.

Please see separate issue for latest CE prototype. It has a bool parameter. It's up to you if that's the most appropriate interface. I'd say it's a good fit of you firmly believe that a sequence of unsigned limbs is the best representation by a fair distance. Because then how do you know if the ripper is signed otherwise?

Here's the thread with the example: https://github.com/ckormanyos/wide-integer/issues/91#issuecomment-968116351


documentation: cors issue with upload examples

Getting Cors Issues with the example file upload at the documentation site using Safari
OS/Browers version
macOS 10.2, safari 10/tech preview.
Vue version
Irrelevant
Reproduction Link
Visit: http://element.eleme.io/#/en-US/component/upload
Steps to reproduce
Try using the file upload examples
What is Expected?
A working example
What is actually happening?
See screenshot:

@jbergstroem  I can not reproduce your situations, it works fine for me. Maybe u can try it again.

@liril-net just to confirm: did you try uploading a file through the examples?

@jbergstroem Yes, i have uploaded an image, everything works fine except the original image can not be loaded at start.

The POST is missing the Access-Control-Allow-Origin header after the OPTIONS. This is a cors bug https://github.com/expressjs/cors/issues/91 .
jsonplaceholder is online fake REST API server. cors is one of its dependencies.


Support concurrency for IAVL and fix Racing conditions

cref: https://github.com/cosmos/iavl/issues/696
This PR fixes the problem stated above by replacing maps that causes racing conditions to sync map to be thread safe.
Confirmed and tested on a node with the following procedure:

Have node running
Spam it with tx bots and query bots
Previously node would crash within 1, maximum 2 minutes with the racing condition logs mentioned above
With fix applied node would not crash for 40+ minutes.

After 40+ minutes, node does seem to crash, but suspecting this may be to different concurrency issues (still doing more investigation)

I think this should target master instead of the release branch, this issue is probably present on v0.20 as well and v1.0.0.

@julienrbrt We can also backport this to older release versions as well though right?

@julienrbrt We can also backport this to older release versions as well though right?

Right!

@julienrbrt Awesome, let's do this, I'll close this issue for now and create a PR pointing at main. After we have that reviewed we can probably backport this to all necessary branches

Awesome, let's do that!


Sdk not working (Grade Error)

"Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
For more on this, please refer to https://docs.gradle.org/8.9/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation.
BUILD FAILED in 2s"
"Multiple build operations failed.
Could not create task ':FtcRobotController:generateDebugRFile'.
Could not create task ':TeamCode:processDebugResources'.
Could not create task ':FtcRobotController:generateDebugRFile'.
Cannot use @TaskAction annotation on method IncrementalTask.taskAction$gradle_core() because interface org.gradle.api.tasks.incremental.IncrementalTaskInputs is not a valid parameter to an action method."
these are the errors that I am getting.

This issue is caused by the new version of Android Studio automatically upgrading the Gradle JDK version that it uses. Take a look at the Gradle section of cookbook, it contains instructions on how to resolve the issue.

Also please view this android studio installation note on ftc-docs:
https://ftc-docs.firstinspires.org/en/latest/programming_resources/tutorial_specific/android_studio/installing_android_studio/Installing-Android-Studio.html#configuring-android-studio-ladybug-and-later

The repo's README says you need "Android Studio 2021.2 (codename Chipmunk) or later":

Requirements
To use this Android Studio project, you will need Android Studio 2021.2 (codename Chipmunk) or later ...

Most teams will not read past "or later" so perhaps the cautions concerning Ladybug should be noted in a more prominent location.

You'll notice that the last update to this repo was prior to Ladybug coming
out.   We are working on a solution that will work with Ladybug.
Thanks for your patience!
On Mon, Oct 28, 2024 at 12:42 PM Alec @.***> wrote:

The repo's README http://../#Requirements says you need "Android
Studio 2021.2 (codename Chipmunk) or later":
Requirements
To use this Android Studio project, you will need Android Studio 2021.2
(codename Chipmunk) or later ...
Most teams will not read past "or later" so perhaps the cautions
concerning Ladybug should be noted in a more prominent location.
—
Reply to this email directly, view it on GitHub
https://github.com/FIRST-Tech-Challenge/FtcRobotController/issues/1218#issuecomment-2442099942,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAF3Y76DOIG7AP6NAJNP5P3Z5ZSP7AVCNFSM6AAAAABQVIZFL6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINBSGA4TSOJUGI
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***
.com>

You'll notice that the last update to this repo was prior to Ladybug coming out...

I'm so out of touch; I'm still running Hedgehog:

Addressed in 10.1.1


Fix select dropdown in dialog gets cut off

Description
Fixes: https://github.com/orchest/orchest/issues/508
Checklist

[x] I have manually tested the application to make sure the changes don’t cause any downstream issues, which includes making sure ./orchest status --ext is not reporting failures when Orchest is running.

Thanks @jacobodeharo! Great fix

@ricklamers I thought there was a missing scenario I didn't check :)


support zadd options

redis 3.0.2 has released recently. Will redis-py supports this feature about zadd ? It's really useful.

I also need this feature too

I suppose no one would mind if I try to do that on a weekend

@chester89 Much appreciated if you do this :smile:

ok, I'll try
2015-07-17 12:10 GMT+03:00 sinchb<EMAIL_ADDRESS>
@chester89 https://github.com/chester89 Much appreciated if you do this [image:
:smile:]
—
Reply to this email directly or view it on GitHub
https://github.com/andymccurdy/redis-py/issues/625#issuecomment-122225421
.

--
Yours faithfully,
Gleb

Any cloud providers already use 3.0.2? I don't have Linux VM at the moment
2015-07-17 12:30 GMT+03:00 Gleb Chermennov<EMAIL_ADDRESS>
ok, I'll try
2015-07-17 12:10 GMT+03:00 sinchb<EMAIL_ADDRESS>
@chester89 https://github.com/chester89 Much appreciated if you do
this [image: :smile:]
—
Reply to this email directly or view it on GitHub
https://github.com/andymccurdy/redis-py/issues/625#issuecomment-122225421
.

--
Yours faithfully,
Gleb

--
Yours faithfully,
Gleb

So I've written the code and tests, let me try if it works with Redis
server and PR will be ready in 2 or 3 days.
2015-07-17 17:40 GMT+03:00 Gleb Chermennov<EMAIL_ADDRESS>
Any cloud providers already use 3.0.2? I don't have Linux VM at the moment
2015-07-17 12:30 GMT+03:00 Gleb Chermennov<EMAIL_ADDRESS>
ok, I'll try
2015-07-17 12:10 GMT+03:00 sinchb<EMAIL_ADDRESS>
@chester89 https://github.com/chester89 Much appreciated if you do
this [image: :smile:]
—
Reply to this email directly or view it on GitHub
https://github.com/andymccurdy/redis-py/issues/625#issuecomment-122225421
.

--
Yours faithfully,
Gleb

--
Yours faithfully,
Gleb

--
Yours faithfully,
Gleb

thanks a lot@chester89

hi @chester89 , how about this feature :)

I made [a PR] (https://github.com/andymccurdy/redis-py/pull/638), there's still some finishing touches to be made

hi @chester89 , how about this feature. I cannot find this feature in the newest release.

I didn't know the release was coming, unfortunately I didn't finish the PR.
Will do that this week
2015-11-03 5:58 GMT+03:00 sinchb<EMAIL_ADDRESS>
hi @chester89 https://github.com/chester89 , how about this feature. I
cannot find this feature in the newest release.
—
Reply to this email directly or view it on GitHub
https://github.com/andymccurdy/redis-py/issues/625#issuecomment-153228508
.

--
Yours faithfully,
Gleb


'sample-project' cleanup

Coverage remained the same at 32.816% when pulling 3fbdb548122c45676d61676cd2bc923a99df49d6 on sample_project_ondate into 5601ae9ae1d8b21830973ecc82776db04f4ed3c7 on master.


feat(robot-server): include device serial number in /health response

Overview
Blocks #8763
Implementation details

Can choose to also update the shape of the /health response under HTTP API v3

Acceptance criteria

GET /health provides device serial number under serialNumber key

Already returned in different response.


Goes mdl.vim check github markdown(ghmarkdown) in Syntastic?

hey markdownlint and @mivok
I couldn't find any specific mention of this in the documentation so I was hoping it would be okay to ask here.
Thanks 🙂

Markdownlint doesn't treat github flavored markdown differently than any other markdown file, but most of the rules are applicable to github flavored markdown and it should work fine.

@mivok thanks a lot of the fast response 🖐🏻


Allow a Campaign Manager User to modify Campaign Goals

Fixed #2146

@VishalMadhvani thanks for this fix, looks good save the NREs in the tests. Can you have a look into those?

@VishalMadhvani - Just checking if to see if you think you'll have time to fix up the unit tests on this one?

If @VishalMadhvani doesn't have time, I can take care of that later this week if you want.

No sorry i haven't had time to look at the tests at all... feel free to grab it @GProulx


Role Template Create/Edit UI should allow you to set "apiGroup" if desired

What kind of request is this (question/bug/enhancement/feature request):
Enhancement
Steps to reproduce (least amount of steps as possible):
Create Role Template in Rancher 2.1.1 UI
Result:
You are able to set Verbs and Resources, but apiGroup is implicitly set to *
Desired Result:
You should be able to set Verbs, Resources, and apiGroups

Specific use case that requires this:
Admin needs to create a custom project role for users managing Istio resources like gateways, virtualservices etc. that live under the networking.istio.io ApiGroup.

Version: Master (v2.3) (6/11/19)
I confirmed what was stated in the issue description (launch older version of Rancher, notice we cannot configure API Groups).
I then launched Master (v2.3) (6/11/19)
We now have the option to set API groups:

I was able to successfully create a roletemplate with a custom string for API Groups (root>rules>0>apiGroups>0 in my case). Also tested with multiple rules. This is of type array[string] according to the API schema (v3/schemas/policyRule) and appears to work correctly based on my testing done from a UI perspective. Create (POST) and Edit (PUT) and Delete (DELETE) all seem to work as expected.


Do you have any plans to separate the text editor as a individual text editor?

Hi,
I am looking for a text editor to replace QScintilla in PyQt5 so that I can transpose to PySide. But I didn't get any power editor until I meet the spyder.
What a wonderful text editor it is. And this is what I need to replace QScintilla. I have tried to separate the text editor from spyder project, but it's difficult for me.
Do you provide any exist text editor or have similar plan to do it?
Thank you very much.

If you need a text editor I recommend Atom. In fact I wish there was a way to integrate spyder around it since spyder's editor is missing some important functionality

Hi @Mr-ZBin,
In fact with one enhacement we've been working on during the past months you will be able to choose the plugins that you want to load Spyder with. This means, that you can start Spyder with only the editor. To do so, please go to Preferences > Plugins and uncheck all the ones you don't want to use while using Spyder.
Please let me know if this is something like you had in mind.


Failed to inspect image "ranchercharts/confluentinc-cp-kafka:5.3.0": rpc error: code = Unknown desc = Error response from daemon: readlink /var/lib/docker/overlay2/l: invalid argument Error: ImageInspectError

i've got this error in my pods Failed to inspect image "ranchercharts/confluentinc-cp-kafka:5.3.0": rpc error: code = Unknown desc = Error response from daemon: readlink /var/lib/docker/overlay2/l: invalid argument
Error: ImageInspectError
Failed to inspect image "ranchercharts/confluentinc-cp-kafka-connect:5.3.0": rpc error: code = Unknown desc = Error response from daemon: readlink /var/lib/docker/overlay2/l: invalid argument

not quite sure.
what's ranchercharts?

Looks like a fork - https://github.com/rancher/charts/tree/master/charts/kafka/v0.8.0


Kubeflow is not published for arm64

With SQA, we were trying to deploy ckf 1.7 on an arm64 machine:
2023-08-08-10:40:46 root DEBUG [localhost]: juju deploy -m foundations-kubernetes:kubeflow --debug --channel=1.7/stable --overlay ./config/kubeflow_options.yaml --trust ch:kubeflow
2023-08-08-10:40:46 root DEBUG 10:40:46 INFO  juju.cmd supercommand.go:56 running juju [3.2.2 bbad9d12de0ebefbd33af942599cc7760c59a1a3 gc go1.20.6]
2023-08-08-10:40:46 root DEBUG 10:40:46 DEBUG juju.cmd supercommand.go:57   args: []string{"/snap/juju/23846/bin/juju", "deploy", "-m", "foundations-kubernetes:kubeflow", "--debug", "--channel=1.7/stable", "--overlay", "./config/kubeflow_options.yaml", "--trust", "ch:kubeflow"}
2023-08-08-10:40:46 root DEBUG 10:40:46 INFO  juju.juju api.go:86 connecting to API addresses: [<IP_ADDRESS>:17070]
2023-08-08-10:40:46 root DEBUG 10:40:46 DEBUG juju.api apiclient.go:639 starting proxier for connection
2023-08-08-10:40:46 root DEBUG 10:40:46 DEBUG juju.api apiclient.go:643 tunnel proxy in use at localhost on port 33165
2023-08-08-10:40:46 root DEBUG 10:40:46 DEBUG juju.api apiclient.go:1166 successfully dialed "wss://localhost:33165/model/9250114a-82f8-486b-8a08-de414dabbfab/api"
2023-08-08-10:40:46 root DEBUG 10:40:46 INFO  juju.api apiclient.go:1068 cannot resolve "localhost": lookup localhost: operation was canceled
2023-08-08-10:40:46 root DEBUG 10:40:46 INFO  juju.api apiclient.go:701 connection established to "wss://localhost:33165/model/9250114a-82f8-486b-8a08-de414dabbfab/api"
2023-08-08-10:40:46 root DEBUG 10:40:46 INFO  juju.juju api.go:86 connecting to API addresses: [<IP_ADDRESS>:17070]
2023-08-08-10:40:47 root DEBUG 10:40:47 DEBUG juju.api apiclient.go:639 starting proxier for connection
2023-08-08-10:40:47 root DEBUG 10:40:47 DEBUG juju.api apiclient.go:643 tunnel proxy in use at localhost on port 44021
2023-08-08-10:40:47 root DEBUG 10:40:47 DEBUG juju.api apiclient.go:819 looked up localhost -> [<IP_ADDRESS>]
2023-08-08-10:40:47 root DEBUG 10:40:47 DEBUG juju.api apiclient.go:1166 successfully dialed "wss://localhost:44021/api"
2023-08-08-10:40:47 root DEBUG 10:40:47 INFO  juju.api apiclient.go:701 connection established to "wss://localhost:44021/api"
2023-08-08-10:40:48 root DEBUG 10:40:48 INFO  cmd bundle.go:351 Located bundle "kubeflow" in charm-hub, revision 340
2023-08-08-10:40:48 root DEBUG 10:40:48 DEBUG juju.cmd.juju.application.deployer bundlehandler.go:259 model: &bundlechanges.Model{
2023-08-08-10:40:48 root DEBUG     Applications: {
2023-08-08-10:40:48 root DEBUG     },
2023-08-08-10:40:48 root DEBUG     Machines: {
2023-08-08-10:40:48 root DEBUG     },
2023-08-08-10:40:48 root DEBUG     Relations:        nil,
2023-08-08-10:40:48 root DEBUG     ConstraintsEqual: func(string, string) bool {...},
2023-08-08-10:40:48 root DEBUG     Sequence:         {},
2023-08-08-10:40:48 root DEBUG     sequence:         {},
2023-08-08-10:40:48 root DEBUG     MachineMap:       {},
2023-08-08-10:40:48 root DEBUG     logger:           nil,
2023-08-08-10:40:48 root DEBUG }
2023-08-08-10:40:48 root DEBUG 10:40:48 DEBUG juju.api monitor.go:35 RPC connection died
2023-08-08-10:40:48 root DEBUG 10:40:48 DEBUG juju.api monitor.go:35 RPC connection died
2023-08-08-10:40:48 root DEBUG ERROR cannot deploy bundle: cannot resolve charm or bundle "admission-webhook": selecting releases: charm or bundle not found for channel "1.7/stable", platform "arm64"
2023-08-08-10:40:48 root DEBUG 10:40:48 DEBUG cmd supercommand.go:548 error stack:
2023-08-08-10:40:48 root DEBUG selecting releases: charm or bundle not found for channel "1.7/stable", platform "arm64"
2023-08-08-10:40:48 root DEBUG github.com/juju/juju/cmd/juju/application/store.(*CharmAdaptor).ResolveCharm:75:
2023-08-08-10:40:48 root DEBUG github.com/juju/juju/cmd/juju/application/deployer.(*bundleHandler).resolveCharmsAndEndpoints:355: cannot resolve charm or bundle "admission-webhook"
2023-08-08-10:40:48 root DEBUG github.com/juju/juju/cmd/juju/application/deployer.bundleDeploy:94:
2023-08-08-10:40:48 root DEBUG github.com/juju/juju/cmd/juju/application/deployer.(*deployBundle).deploy:176: cannot deploy bundle
2023-08-08-10:40:48 root ERROR [localhost] Command failed: juju deploy -m foundations-kubernetes:kubeflow --debug --channel=1.7/stable --overlay ./config/kubeflow_options.yaml --trust ch:kubeflow
2023-08-08-10:40:48 root ERROR 1[localhost] STDOUT follows:

2023-08-08-10:40:48 root ERROR 2[localhost] STDERR follows:
10:40:46 INFO  juju.cmd supercommand.go:56 running juju [3.2.2 bbad9d12de0ebefbd33af942599cc7760c59a1a3 gc go1.20.6]
10:40:46 DEBUG juju.cmd supercommand.go:57   args: []string{"/snap/juju/23846/bin/juju", "deploy", "-m", "foundations-kubernetes:kubeflow", "--debug", "--channel=1.7/stable", "--overlay", "./config/kubeflow_options.yaml", "--trust", "ch:kubeflow"}
10:40:46 INFO  juju.juju api.go:86 connecting to API addresses: [<IP_ADDRESS>:17070]
10:40:46 DEBUG juju.api apiclient.go:639 starting proxier for connection
10:40:46 DEBUG juju.api apiclient.go:643 tunnel proxy in use at localhost on port 33165
10:40:46 DEBUG juju.api apiclient.go:1166 successfully dialed "wss://localhost:33165/model/9250114a-82f8-486b-8a08-de414dabbfab/api"
10:40:46 INFO  juju.api apiclient.go:1068 cannot resolve "localhost": lookup localhost: operation was canceled
10:40:46 INFO  juju.api apiclient.go:701 connection established to "wss://localhost:33165/model/9250114a-82f8-486b-8a08-de414dabbfab/api"
10:40:46 INFO  juju.juju api.go:86 connecting to API addresses: [<IP_ADDRESS>:17070]
10:40:47 DEBUG juju.api apiclient.go:639 starting proxier for connection
10:40:47 DEBUG juju.api apiclient.go:643 tunnel proxy in use at localhost on port 44021
10:40:47 DEBUG juju.api apiclient.go:819 looked up localhost -> [<IP_ADDRESS>]
10:40:47 DEBUG juju.api apiclient.go:1166 successfully dialed "wss://localhost:44021/api"
10:40:47 INFO  juju.api apiclient.go:701 connection established to "wss://localhost:44021/api"
10:40:48 INFO  cmd bundle.go:351 Located bundle "kubeflow" in charm-hub, revision 340
10:40:48 DEBUG juju.cmd.juju.application.deployer bundlehandler.go:259 model: &bundlechanges.Model{
    Applications: {
    },
    Machines: {
    },
    Relations:        nil,
    ConstraintsEqual: func(string, string) bool {...},
    Sequence:         {},
    sequence:         {},
    MachineMap:       {},
    logger:           nil,
}
10:40:48 DEBUG juju.api monitor.go:35 RPC connection died
10:40:48 DEBUG juju.api monitor.go:35 RPC connection died
ERROR cannot deploy bundle: cannot resolve charm or bundle "admission-webhook": selecting releases: charm or bundle not found for channel "1.7/stable", platform "arm64"
10:40:48 DEBUG cmd supercommand.go:548 error stack:
selecting releases: charm or bundle not found for channel "1.7/stable", platform "arm64"
github.com/juju/juju/cmd/juju/application/store.(*CharmAdaptor).ResolveCharm:75:
github.com/juju/juju/cmd/juju/application/deployer.(*bundleHandler).resolveCharmsAndEndpoints:355: cannot resolve charm or bundle "admission-webhook"
github.com/juju/juju/cmd/juju/application/deployer.bundleDeploy:94:
github.com/juju/juju/cmd/juju/application/deployer.(*deployBundle).deploy:176: cannot deploy bundle

Traceback (most recent call last):
  File "/usr/local/bin/fce", line 33, in <module>
    sys.exit(load_entry_point('foundationcloudengine', 'console_scripts', 'fce')())
  File "/home/ubuntu/cpe/foundation/foundationcloudengine/foundationcloudengine/main.py", line 217, in entry_point
    sys.exit(main(sys.argv[1:]))
  File "/home/ubuntu/cpe/foundation/foundationcloudengine/foundationcloudengine/main.py", line 208, in main
    opts.func(opts)
  File "/home/ubuntu/cpe/foundation/foundationcloudengine/foundationcloudengine/command.py", line 94, in run
    self.run_per_layer(layer, args)
  File "/home/ubuntu/cpe/foundation/foundationcloudengine/foundationcloudengine/commands/build.py", line 46, in run_per_layer
    build_and_validate_if_needed(
  File "/home/ubuntu/cpe/foundation/foundationcloudengine/foundationcloudengine/commands/build.py", line 98, in build_and_validate_if_needed
    layer.build_outer(only_steps)
  File "/home/ubuntu/cpe/foundation/foundationcloudengine/foundationcloudengine/layers/baselayer.py", line 160, in build_outer
    self.build(only_steps=only_steps)
  File "/home/ubuntu/cpe/foundation/foundationcloudengine/foundationcloudengine/layers/jujuworkloadlayer.py", line 1412, in build
    self.run_steps(only_steps)
  File "/home/ubuntu/cpe/foundation/foundationcloudengine/foundationcloudengine/layers/steppedbaselayer.py", line 110, in run_steps
    step.build()
  File "/home/ubuntu/cpe/foundation/foundationcloudengine/foundationcloudengine/layers/jujuworkloadlayer.py", line 1004, in build
    deploy(
  File "/home/ubuntu/cpe/foundation/foundationcloudengine/foundationcloudengine/juju_cli.py", line 191, in deploy
    local(command, output_mode="live")
  File "/home/ubuntu/cpe/foundation/foundationcloudengine/foundationcloudengine/remotehelpers.py", line 278, in local
    return run_cmd(cmd, target_machine, **kwargs)
  File "/home/ubuntu/cpe/foundation/foundationcloudengine/foundationcloudengine/remotehelpers.py", line 209, in run_cmd
    raise subprocess.CalledProcessError(
subprocess.CalledProcessError: Command '['juju', 'deploy', '-m', 'foundations-kubernetes:kubeflow', '--debug', '--channel=1.7/stable', '--overlay', './config/kubeflow_options.yaml', '--trust', 'ch:kubeflow']' returned non-zero exit status 1.
Build step 'Execute shell' marked build as failure

Are there plans for supporting arm64? Would it make sense for SQA to step testing ckf on ARM64 for now?

Thank you the question @Basdbruijne. Currently, we do not support CKF on arm64 architectures. There are going to be efforts towards this direction, but until we confirm that we fully support it, it doesn't make sense to test that on ARM64. Closing this issue but feel free to re-open it if you have any more questions.


Multiple serialization 'profiles' for one type

Hi
I was wondering if it were somehow possible to define multiple serialization profiles for one specific type. Having done a lot of work with NewtonSoft JSON in C#, I got used to being able to define different profiles for a specific type. For example an encrypted messages:

Against the database, I need to store additional information like 'nonce' or 'salt' etc. As my database is storing JSON directly, I use serde to create the database entry
Sending the encrypted message to the user (after decryption of course) however, should not include things like nonce or salt.

The same approach could be taken for a Json-API like web API: the request contains information about what fields should be pulled from the database. So the field-set could be different for every request. One solution of course would be to make all the fields optional, but sometimes I need to make sure that the field actually exists (so it should not be Option<...>), but still the end-user needs to be able to chose not to receive it in the JSON response.
I haven't seen any other way than creating different types (like data transfer objects) and implementing something like From to make a conversion from the actual struct to the stripped down transfer struct. But then I believe that this is quite an overhead for just selecting the fields to be serialized.
Is there a better way to achieve that?

This isn't supported in serde, but I would be interested in someone developing a different library that is more suited to this use case.


Java2Swift: Swift does not support overriding with covariant array result types

works for me, thank you!


Capitalize Activate.ps1

For #2607
This is to allow Activate.ps1 to work appropriate on Linux for PowerShell Core.

[x] Pull request represents a single change (i.e. not fixing disparate/unrelated things in a single PR)
[x] Title summarizes what is changing
[x] Has a news entry file (remember to thank yourself!)
[x] Appropriate comments and documentation strings in the code
[x] Has sufficient logging.
[x] Has telemetry for enhancements.
[x] Unit tests & system/integration tests are added/updated
[x] Test plan is updated as appropriate
[x] package-lock.json has been regenerated by running npm install (if dependencies have changed)
[x] The wiki is updated with any design decisions/details.

Codecov Report

Merging #9911 into master will decrease coverage by 0.02%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master    #9911      +/-   ##
==========================================
- Coverage   61.21%   61.19%   -0.03%
==========================================
  Files         564      564
  Lines       30061    30061
  Branches     4545     4545
==========================================
- Hits        18403    18396       -7
- Misses      10629    10635       +6
- Partials     1029     1030       +1

Impacted Files
Coverage Δ

...al/environmentActivationProviders/commandPrompt.ts
92.85% <100%> (ø)
:arrow_up:

src/client/common/utils/platform.ts
64.7% <0%> (-11.77%)
:arrow_down:

src/client/linters/pylint.ts
94.82% <0%> (-3.45%)
:arrow_down:

src/client/linters/pydocstyle.ts
86.66% <0%> (-2.23%)
:arrow_down:

src/client/datascience/debugLocationTracker.ts
76.56% <0%> (-1.57%)
:arrow_down:

src/client/common/process/proc.ts
14.49% <0%> (-0.73%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f551a73...78c1568. Read the comment docs.


vmap doesn't handle named arguments.

Minimalish example:
def f(a, b):
  return (2*a, 3*b)

jax.vmap(f)(jnp.array([1]), jnp.array([2]))  # works
jax.vmap(f)(a=jnp.array([1]), b=jnp.array([2]))  # doesn't work

The last line fails with TypeError: reduce() of empty sequence with no initial value.

I came across a situation where vmap will appear to allow keyword arguments but produce incorrect results. In my case the incorrect results were valid values which allowed learning to continue (badly). Ideally an exception would be raised until keyword arguments are supported.
def wrapped_argmax(dummy, array):
  return jnp.argmax(array)

batch_wrapped_argmax = jax.vmap(wrapped_argmax)

array = jnp.array([[0, 3, 0], [0, 0, 2], [1, 0, 0]], dtype=jnp.float32)
dummy = jnp.zeros_like(array)

no_kw = batch_wrapped_argmax(
    dummy,
    array,
)
print("good", no_kw)

kw = batch_wrapped_argmax(
    dummy,
    array=array,
)
print("bad ", kw)

kw_slice = batch_wrapped_argmax(
    dummy[2:3],
    array=array[2:3],
)
print("good", kw_slice)

try:
  batch_wrapped_argmax(
      dummy=dummy,
      array=array,
  )
except Exception as err:
  print(err)

# Output:

# good [1 2 0]
# bad  [1 1 1]
# good [0]
# reduce() of empty sequence with no initial value

@mattjj Is it possible to revert this change or (probably better) replace the NotImplementedError by a warning? Some code that we use in jaxmd relies on passing kwargs to vmapped function (with the expectation that they will not be vmapped over, so for us it's working as expected).

Some code that we use in jaxmd relies on passing kwargs to vmapped function (with the expectation that they will not be vmapped over, so for us it's working as expected).

I don't know the details of what you are trying to do, but would it be perhaps easier/clearer to use something like functools.partial to feed in the non-vmapped arguments in such cases?
So, something like the following?
g = functools.partial(f, a=a, b=b)  # setting generic, non-vmapped, kwargs
h = jax.vmap(g)
z = h(x, y)

I appreciate this might be slightly less elegant for your use case. But I think the alternative may be confusing at times, if the code behaves unintuitively to users (but doesn't fail)?
I think ideally named arguments would eventually be supported to vmap over.  I suppose that would break your current code, since it relies on them behaving differently?

Ah, thanks for the recommendation. If it feels like my suggestion will adversely affect user experience then I can use a pattern like that.
When calling f(x, y, a=a, b=b) in a loop, however, it feels a bit awkward to call jit(vmap(g)) in the loop (though I guess it will hit the cache and so shouldn't be costly). I am definitely not proposing that jax be silent re: use of kwargs. I just think a warning stating that kwargs are not batched over might suffice to let users know about that behavior and reason about it accordingly.
Regarding future compatibility, I would assume - as is the case with positional arguments - that eventually vmap will allow one to decide whether or not kwargs are to be vmapped over.

@sschoenholz actually jit(vmap(g)) won't hit the cache, because vmap (like other transformations) produces a new Python callable object each time it's applied. (I promised Adam I'd write a doc explaining when to expect cache hits...)
As for the main topic of the thread: the main reason kwargs are hard to support with vmapped functions is that the way in_axes are specified really depends on arguments being passed as positional. But we can cover some special cases, including the one in the OP. I'll try that!

@sschoenholz actually jit(vmap(g)) won't hit the cache, because vmap (like other transformations) produces a new Python callable object each time it's applied. (I promised Adam I'd write a doc explaining when to expect cache hits...)
As for the main topic of the thread: the main reason kwargs are hard to support with vmapped functions is that the way in_axes are specified really depends on arguments being passed as positional. But we can cover some special cases, including the one in the OP. I'll try that!


Merge E2E tests from test branch to main branch

Just a ugly as the one from forked branch... closing


fixed lodash case sensitive require statements and bumped lodash version

Changes the lodash imports to be case sensitive and bumps lodash version for mapValues
This will address build issues on case sensitive operating systems.

Close #1

Close #2


How to deploy the controllers as daemonsets or at least redeploy in case of node failure?

In my testing, I created a cluster of three master nodes, all are untainted and can schedule normal pods.
Flux is only ever running on the node it was originally deployed on via timoni.
If this node goes down, the controllers are not deployed to other nodes.
flux events - shows logs until the node went down.
The pods show running on the node that is down:
stream logs failed Get "https://<IP_ADDRESS>:10250/containerLogs/flux-system/flux-57bd866b6d-zbrfc/helm-controller?follow=true&sinceSeconds=300&tailLines=100&timestamps=true": dial tcp <IP_ADDRESS>:10250: connect:

Here is the cue bit for timoni, I use:
  "flux": {
          module: {
                  url:     "oci://ghcr.io/stefanprodan/modules/flux-aio"
                  version: "2.1.2"
          }
          namespace: "flux-system"
          values: {
                  hostNetwork:     true
                  securityProfile: "privileged"
                  controllers: notification: enabled: false
          }
  }

If this node goes down, the controllers are not deployed to other nodes.

Is the Kubernetes control plane still working? I expect it to reschedule the pod on a different node. Maybe the toleration we set in Flux is too broad, I set it like this https://github.com/stefanprodan/flux-aio/blob/aedf966e28a5f7170e3e737d7d52fa5815c8cfad/modules/flux-aio/templates/config.cue#L132

It may well be that since etcd has no quorum, the control plane will no longer schedule pods anywhere. I suggest creating a cluster with 2 worker nodes, deploy Flux on one of the workers, make that node fail and see if it gets reschedule to the healthy node.

I can still schedule pods.
Weave Dashboard, AWX, kubernetes dashboard are all rescheduled to other nodes. Only Flux does not and shows "running".

If you describe the Flux pod, is there any hint in the events about some blocker to rescheduling?

Events show this
Events:
  Type     Reason          Age                From             Message
  ----     ------          ----               ----             -------
  Normal   Created         39m                kubelet          Created container kustomize-controller
  Normal   Pulled          39m                kubelet          Container image "ghcr.io/fluxcd/helm-controller:v0.36.2" already present on machine
  Normal   Pulled          39m                kubelet          Container image "ghcr.io/fluxcd/source-controller:v1.1.2" already present on machine
  Normal   Created         39m                kubelet          Created container source-controller
  Normal   Started         39m                kubelet          Started container source-controller
  Normal   Pulled          39m                kubelet          Container image "ghcr.io/fluxcd/kustomize-controller:v1.1.1" already present on machine
  Normal   SandboxChanged  39m                kubelet          Pod sandbox changed, it will be killed and re-created.
  Normal   Started         39m                kubelet          Started container kustomize-controller
  Normal   Started         39m                kubelet          Started container helm-controller
  Normal   Created         39m                kubelet          Created container helm-controller
  Warning  Unhealthy       38m                kubelet          Liveness probe failed: Get "http://<IP_ADDRESS>:9794/healthz": dial tcp <IP_ADDRESS>:9794: connect: connection refused
  Warning  Unhealthy       38m (x5 over 39m)  kubelet          Readiness probe failed: Get "http://<IP_ADDRESS>:9794/readyz": dial tcp <IP_ADDRESS>:9794: connect: connection refused
  Warning  Unhealthy       38m                kubelet          Liveness probe failed: Get "http://<IP_ADDRESS>:9792/healthz": dial tcp <IP_ADDRESS>:9792: connect: connection refused
  Warning  Unhealthy       38m (x8 over 39m)  kubelet          Readiness probe failed: Get "http://<IP_ADDRESS>:9790/": dial tcp <IP_ADDRESS>:9790: connect: connection refused
  Warning  NodeNotReady    16m (x3 over 93m)  node-controller  Node is not ready

The full description:
[disi@vmalmakw1s ~]$ kubectl describe pod flux-57bd866b6d-zbrfc -n flux-system
Name:                 flux-57bd866b6d-zbrfc
Namespace:            flux-system
Priority:<PHONE_NUMBER>
Priority Class Name:  system-cluster-critical
Service Account:      flux
Node:                 vmalmakms.home/<IP_ADDRESS>
Start Time:           Sat, 25 Nov 2023 22:11:15 +0000
Labels:               app.kubernetes.io/name=flux
                      pod-template-hash=57bd866b6d
Annotations:          cluster-autoscaler.kubernetes.io/safe-to-evict: true
                      prometheus.io/scrape: true
Status:               Running
IP:                   <IP_ADDRESS>
IPs:
  IP:           <IP_ADDRESS>
Controlled By:  ReplicaSet/flux-57bd866b6d
Containers:
  source-controller:
    Container ID:    containerd://f95a5ff962bb8b4697cc3b3b933b2c45499f594b65802f113aebb587ff822b61
    Image:           ghcr.io/fluxcd/source-controller:v1.1.2
    Image ID:        ghcr.io/fluxcd/source-controller@sha256:b776e085ac079bf22ed23afe2874aebd10efcfaa740ec25748774608bbc79932
    Ports:           9790/TCP, 9791/TCP, 9792/TCP
    Host Ports:      9790/TCP, 9791/TCP, 9792/TCP
    SeccompProfile:  RuntimeDefault
    Args:
      --watch-all-namespaces
      --log-level=info
      --log-encoding=json
      --enable-leader-election=false
      --metrics-addr=:9791
      --health-addr=:9792
      --storage-addr=:9790
      --storage-path=/data
      --storage-adv-addr=flux.$(RUNTIME_NAMESPACE).svc.cluster.local.
      --concurrent=5
      --requeue-dependency=30s
      --watch-label-selector=!sharding.fluxcd.io/key
      --helm-cache-max-size=10
      --helm-cache-ttl=60m
      --helm-cache-purge-interval=5m
    State:          Running
      Started:      Sun, 26 Nov 2023 08:33:56 +0000
    Last State:     Terminated
      Reason:       Completed
      Exit Code:    0
      Started:      Sun, 26 Nov 2023 08:33:25 +0000
      Finished:     Sun, 26 Nov 2023 08:33:56 +0000
    Ready:          True
    Restart Count:  15
    Limits:
      memory:  1Gi
    Requests:
      cpu:      100m
      memory:   64Mi
    Liveness:   http-get http://:healthz-sc/healthz delay=0s timeout=1s period=10s #success=1 #failure=3
    Readiness:  http-get http://:http-sc/ delay=0s timeout=1s period=10s #success=1 #failure=3
    Environment:
      SOURCE_CONTROLLER_LOCALHOST:  localhost:9790
      RUNTIME_NAMESPACE:            flux-system (v1:metadata.namespace)
      TUF_ROOT:                     /tmp/.sigstore
      NO_PROXY:                     .cluster.local.,.cluster.local,.svc
    Mounts:
      /data from data (rw)
      /tmp from tmp (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-nznjt (ro)
  kustomize-controller:
    Container ID:    containerd://997b139a3e85abd2da13c1d95fbb585bf6cfe29967bcd241d95a885493213971
    Image:           ghcr.io/fluxcd/kustomize-controller:v1.1.1
    Image ID:        ghcr.io/fluxcd/kustomize-controller@sha256:e2b3c9e1292564bbfaa513f3cc6fa1df1194fae8ba9483fbe581099d0c585d94
    Ports:           9793/TCP, 9794/TCP
    Host Ports:      9793/TCP, 9794/TCP
    SeccompProfile:  RuntimeDefault
    Args:
      --watch-all-namespaces
      --log-level=info
      --log-encoding=json
      --enable-leader-election=false
      --metrics-addr=:9793
      --health-addr=:9794
      --watch-label-selector=!sharding.fluxcd.io/key
      --concurrent=5
      --requeue-dependency=30s
    State:          Running
      Started:      Sun, 26 Nov 2023 08:33:56 +0000
    Last State:     Terminated
      Reason:       Completed
      Exit Code:    0
      Started:      Sun, 26 Nov 2023 08:33:25 +0000
      Finished:     Sun, 26 Nov 2023 08:33:56 +0000
    Ready:          True
    Restart Count:  15
    Limits:
      memory:  1Gi
    Requests:
      cpu:      100m
      memory:   64Mi
    Liveness:   http-get http://:healthz-kc/healthz delay=0s timeout=1s period=10s #success=1 #failure=3
    Readiness:  http-get http://:healthz-kc/readyz delay=0s timeout=1s period=10s #success=1 #failure=3
    Environment:
      SOURCE_CONTROLLER_LOCALHOST:  localhost:9790
      RUNTIME_NAMESPACE:            flux-system (v1:metadata.namespace)
      TUF_ROOT:                     /tmp/.sigstore
      NO_PROXY:                     .cluster.local.,.cluster.local,.svc
    Mounts:
      /tmp from tmp (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-nznjt (ro)
  helm-controller:
    Container ID:    containerd://c6a9a4ec46740520acc2f70a763259469830add59b7904a4ee0b00e8e97d2dd1
    Image:           ghcr.io/fluxcd/helm-controller:v0.36.2
    Image ID:        ghcr.io/fluxcd/helm-controller@sha256:6ee7e590e57350ac91cfdeee4587d0e9e6f52e723c56d4b7878c59279bd36f00
    Ports:           9795/TCP, 9796/TCP
    Host Ports:      9795/TCP, 9796/TCP
    SeccompProfile:  RuntimeDefault
    Args:
      --watch-all-namespaces
      --log-level=info
      --log-encoding=json
      --enable-leader-election=false
      --metrics-addr=:9795
      --health-addr=:9796
      --watch-label-selector=!sharding.fluxcd.io/key
      --concurrent=5
      --requeue-dependency=30s
    State:          Running
      Started:      Sun, 26 Nov 2023 08:33:25 +0000
    Last State:     Terminated
      Reason:       Unknown
      Exit Code:    255
      Started:      Sun, 26 Nov 2023 07:49:37 +0000
      Finished:     Sun, 26 Nov 2023 08:33:09 +0000
    Ready:          True
    Restart Count:  13
    Limits:
      memory:  1Gi
    Requests:
      cpu:      100m
      memory:   64Mi
    Liveness:   http-get http://:healthz-hc/healthz delay=0s timeout=1s period=10s #success=1 #failure=3
    Readiness:  http-get http://:healthz-hc/readyz delay=0s timeout=1s period=10s #success=1 #failure=3
    Environment:
      SOURCE_CONTROLLER_LOCALHOST:  localhost:9790
      RUNTIME_NAMESPACE:            flux-system (v1:metadata.namespace)
      TUF_ROOT:                     /tmp/.sigstore
      NO_PROXY:                     .cluster.local.,.cluster.local,.svc
    Mounts:
      /tmp from tmp (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-nznjt (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             False
  ContainersReady   True
  PodScheduled      True
Volumes:
  data:
    Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:
    SizeLimit:  <unset>
  tmp:
    Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:
    SizeLimit:  <unset>
  kube-api-access-nznjt:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   Burstable
Node-Selectors:              <none>
Tolerations:                 op=Exists
Events:
  Type     Reason          Age                From             Message
  ----     ------          ----               ----             -------
  Normal   Created         39m                kubelet          Created container kustomize-controller
  Normal   Pulled          39m                kubelet          Container image "ghcr.io/fluxcd/helm-controller:v0.36.2" already present on machine
  Normal   Pulled          39m                kubelet          Container image "ghcr.io/fluxcd/source-controller:v1.1.2" already present on machine
  Normal   Created         39m                kubelet          Created container source-controller
  Normal   Started         39m                kubelet          Started container source-controller
  Normal   Pulled          39m                kubelet          Container image "ghcr.io/fluxcd/kustomize-controller:v1.1.1" already present on machine
  Normal   SandboxChanged  39m                kubelet          Pod sandbox changed, it will be killed and re-created.
   Can also describe the ReplicaSet/flux-57bd866b6d and the flux deployment please.

Replicaset
Name:           flux-57bd866b6d
Namespace:      flux-system
Selector:       app.kubernetes.io/name=flux,pod-template-hash=57bd866b6d
Labels:         app.kubernetes.io/name=flux
                pod-template-hash=57bd866b6d
Annotations:    app.kubernetes.io/role: cluster-admin
                deployment.kubernetes.io/desired-replicas: 1
                deployment.kubernetes.io/max-replicas: 1
                deployment.kubernetes.io/revision: 1
Controlled By:  Deployment/flux
Replicas:       1 current / 1 desired
Pods Status:    1 Running / 0 Waiting / 0 Succeeded / 0 Failed
Pod Template:
  Labels:           app.kubernetes.io/name=flux
                    pod-template-hash=57bd866b6d
  Annotations:      cluster-autoscaler.kubernetes.io/safe-to-evict: true
                    prometheus.io/scrape: true
  Service Account:  flux
  Containers:
   source-controller:
    Image:           ghcr.io/fluxcd/source-controller:v1.1.2
    Ports:           9790/TCP, 9791/TCP, 9792/TCP
    Host Ports:      0/TCP, 0/TCP, 0/TCP
    SeccompProfile:  RuntimeDefault
    Args:
      --watch-all-namespaces
      --log-level=info
      --log-encoding=json
      --enable-leader-election=false
      --metrics-addr=:9791
      --health-addr=:9792
      --storage-addr=:9790
      --storage-path=/data
      --storage-adv-addr=flux.$(RUNTIME_NAMESPACE).svc.cluster.local.
      --concurrent=5
      --requeue-dependency=30s
      --watch-label-selector=!sharding.fluxcd.io/key
      --helm-cache-max-size=10
      --helm-cache-ttl=60m
      --helm-cache-purge-interval=5m
    Limits:
      memory:  1Gi
    Requests:
      cpu:      100m
      memory:   64Mi
    Liveness:   http-get http://:healthz-sc/healthz delay=0s timeout=1s period=10s #success=1 #failure=3
    Readiness:  http-get http://:http-sc/ delay=0s timeout=1s period=10s #success=1 #failure=3
    Environment:
      SOURCE_CONTROLLER_LOCALHOST:  localhost:9790
      RUNTIME_NAMESPACE:             (v1:metadata.namespace)
      TUF_ROOT:                     /tmp/.sigstore
      NO_PROXY:                     .cluster.local.,.cluster.local,.svc
    Mounts:
      /data from data (rw)
      /tmp from tmp (rw)
   kustomize-controller:
    Image:           ghcr.io/fluxcd/kustomize-controller:v1.1.1
    Ports:           9793/TCP, 9794/TCP
    Host Ports:      0/TCP, 0/TCP
    SeccompProfile:  RuntimeDefault
    Args:
      --watch-all-namespaces
      --log-level=info
      --log-encoding=json
      --enable-leader-election=false
      --metrics-addr=:9793
      --health-addr=:9794
      --watch-label-selector=!sharding.fluxcd.io/key
      --concurrent=5
      --requeue-dependency=30s
    Limits:
      memory:  1Gi
    Requests:
      cpu:      100m
      memory:   64Mi
    Liveness:   http-get http://:healthz-kc/healthz delay=0s timeout=1s period=10s #success=1 #failure=3
    Readiness:  http-get http://:healthz-kc/readyz delay=0s timeout=1s period=10s #success=1 #failure=3
    Environment:
      SOURCE_CONTROLLER_LOCALHOST:  localhost:9790
      RUNTIME_NAMESPACE:             (v1:metadata.namespace)
      TUF_ROOT:                     /tmp/.sigstore
      NO_PROXY:                     .cluster.local.,.cluster.local,.svc
    Mounts:
      /tmp from tmp (rw)
   helm-controller:
    Image:           ghcr.io/fluxcd/helm-controller:v0.36.2
    Ports:           9795/TCP, 9796/TCP
    Host Ports:      0/TCP, 0/TCP
    SeccompProfile:  RuntimeDefault
    Args:
      --watch-all-namespaces
      --log-level=info
      --log-encoding=json
      --enable-leader-election=false
      --metrics-addr=:9795
      --health-addr=:9796
      --watch-label-selector=!sharding.fluxcd.io/key
      --concurrent=5
      --requeue-dependency=30s
    Limits:
      memory:  1Gi
    Requests:
      cpu:      100m
      memory:   64Mi
    Liveness:   http-get http://:healthz-hc/healthz delay=0s timeout=1s period=10s #success=1 #failure=3
    Readiness:  http-get http://:healthz-hc/readyz delay=0s timeout=1s period=10s #success=1 #failure=3
    Environment:
      SOURCE_CONTROLLER_LOCALHOST:  localhost:9790
      RUNTIME_NAMESPACE:             (v1:metadata.namespace)
      TUF_ROOT:                     /tmp/.sigstore
      NO_PROXY:                     .cluster.local.,.cluster.local,.svc
    Mounts:
      /tmp from tmp (rw)
  Volumes:
   data:
    Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:
    SizeLimit:  <unset>
   tmp:
    Type:               EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:
    SizeLimit:          <unset>
  Priority Class Name:  system-cluster-critical
Events:                 <none>

Deployment
Name:               flux
Namespace:          flux-system
CreationTimestamp:  Mon, 20 Nov 2023 15:16:57 +0000
Labels:             app.kubernetes.io/managed-by=timoni
                    app.kubernetes.io/name=flux
                    app.kubernetes.io/part-of=flux
                    app.kubernetes.io/version=v2.1.2
                    instance.timoni.sh/name=flux
                    instance.timoni.sh/namespace=flux-system
Annotations:        app.kubernetes.io/role: cluster-admin
                    deployment.kubernetes.io/revision: 1
Selector:           app.kubernetes.io/name=flux
Replicas:           1 desired | 1 updated | 1 total | 0 available | 1 unavailable
StrategyType:       Recreate
MinReadySeconds:    0
Pod Template:
  Labels:           app.kubernetes.io/name=flux
  Annotations:      cluster-autoscaler.kubernetes.io/safe-to-evict: true
                    prometheus.io/scrape: true
  Service Account:  flux
  Containers:
   source-controller:
    Image:           ghcr.io/fluxcd/source-controller:v1.1.2
    Ports:           9790/TCP, 9791/TCP, 9792/TCP
    Host Ports:      0/TCP, 0/TCP, 0/TCP
    SeccompProfile:  RuntimeDefault
    Args:
      --watch-all-namespaces
      --log-level=info
      --log-encoding=json
      --enable-leader-election=false
      --metrics-addr=:9791
      --health-addr=:9792
      --storage-addr=:9790
      --storage-path=/data
      --storage-adv-addr=flux.$(RUNTIME_NAMESPACE).svc.cluster.local.
      --concurrent=5
      --requeue-dependency=30s
      --watch-label-selector=!sharding.fluxcd.io/key
      --helm-cache-max-size=10
      --helm-cache-ttl=60m
      --helm-cache-purge-interval=5m
    Limits:
      memory:  1Gi
    Requests:
      cpu:      100m
      memory:   64Mi
    Liveness:   http-get http://:healthz-sc/healthz delay=0s timeout=1s period=10s #success=1 #failure=3
    Readiness:  http-get http://:http-sc/ delay=0s timeout=1s period=10s #success=1 #failure=3
    Environment:
      SOURCE_CONTROLLER_LOCALHOST:  localhost:9790
      RUNTIME_NAMESPACE:             (v1:metadata.namespace)
      TUF_ROOT:                     /tmp/.sigstore
      NO_PROXY:                     .cluster.local.,.cluster.local,.svc
    Mounts:
      /data from data (rw)
      /tmp from tmp (rw)
   kustomize-controller:
    Image:           ghcr.io/fluxcd/kustomize-controller:v1.1.1
    Ports:           9793/TCP, 9794/TCP
    Host Ports:      0/TCP, 0/TCP
    SeccompProfile:  RuntimeDefault
    Args:
      --watch-all-namespaces
      --log-level=info
      --log-encoding=json
      --enable-leader-election=false
      --metrics-addr=:9793
      --health-addr=:9794
      --watch-label-selector=!sharding.fluxcd.io/key
      --concurrent=5
      --requeue-dependency=30s
    Limits:
      memory:  1Gi
    Requests:
      cpu:      100m
      memory:   64Mi
    Liveness:   http-get http://:healthz-kc/healthz delay=0s timeout=1s period=10s #success=1 #failure=3
    Readiness:  http-get http://:healthz-kc/readyz delay=0s timeout=1s period=10s #success=1 #failure=3
    Environment:
      SOURCE_CONTROLLER_LOCALHOST:  localhost:9790
      RUNTIME_NAMESPACE:             (v1:metadata.namespace)
      TUF_ROOT:                     /tmp/.sigstore
      NO_PROXY:                     .cluster.local.,.cluster.local,.svc
    Mounts:
      /tmp from tmp (rw)
   helm-controller:
    Image:           ghcr.io/fluxcd/helm-controller:v0.36.2
    Ports:           9795/TCP, 9796/TCP
    Host Ports:      0/TCP, 0/TCP
    SeccompProfile:  RuntimeDefault
    Args:
      --watch-all-namespaces
      --log-level=info
      --log-encoding=json
      --enable-leader-election=false
      --metrics-addr=:9795
      --health-addr=:9796
      --watch-label-selector=!sharding.fluxcd.io/key
      --concurrent=5
      --requeue-dependency=30s
    Limits:
      memory:  1Gi
    Requests:
      cpu:      100m
      memory:   64Mi
    Liveness:   http-get http://:healthz-hc/healthz delay=0s timeout=1s period=10s #success=1 #failure=3
    Readiness:  http-get http://:healthz-hc/readyz delay=0s timeout=1s period=10s #success=1 #failure=3
    Environment:
      SOURCE_CONTROLLER_LOCALHOST:  localhost:9790
      RUNTIME_NAMESPACE:             (v1:metadata.namespace)
      TUF_ROOT:                     /tmp/.sigstore
      NO_PROXY:                     .cluster.local.,.cluster.local,.svc
    Mounts:
      /tmp from tmp (rw)
  Volumes:
   data:
    Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:
    SizeLimit:  <unset>
   tmp:
    Type:               EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:
    SizeLimit:          <unset>
  Priority Class Name:  system-cluster-critical
Conditions:
  Type           Status  Reason
  ----           ------  ------
  Progressing    True    NewReplicaSetAvailable
  Available      False   MinimumReplicasUnavailable
OldReplicaSets:  <none>
NewReplicaSet:   flux-57bd866b6d (1/1 replicas created)
Events:          <none>

Really odd, the replicaset says Pods Status: 1 Running which is really strange, but the Deployment say Replicas: 1 unavailable but it doesn't create a new replicaset.

I will see later today, if I start over and redeploy the entire cluster. Then test again and see if it has the same behaviour.

I guess if you delete pod it will get rescheduled, this looks like some race condition in the Kubernetes scheduler or the toleration makes it trip.

Correct :)
I did delete the pod.
flux-system    flux-57bd866b6d-j6z7x                              3/3     Running       0                56s     <IP_ADDRESS>      vmalmakw2s.home   <none>           <none>
flux-system    flux-57bd866b6d-zbrfc                              3/3     Terminating   43 (75m ago)     11h     <IP_ADDRESS>      vmalmakms.home    <none>           <none>

A new was created and it's working fine.
But it did not happen automatically.

Hmm so it looks like it got stuck in Terminating, but why wasn't this status reflected in the replicaset and why it didn't timeout. I wander if this is some bug in Kubernetes.

Terminating status only after ran:
$ kubectl delete pod flux-57bd866b6d-zbrfc -n flux-system

i.e. the kubernetes dashboard and other pods also linger some time in terminating. Default is some 15min, before those get removed by Kubernetes?

If you managed to reproduce this, it would be good to take snapshots of the deployment and replicaset and see what events are issued for those, I guess those expired and that's why there are no events listed now.

If you can reproduce this, please add with kubectl edit a toleration like so, and retest please:
- effect: NoExecute
    key: node.kubernetes.io/unreachable
    operator: Exists
    tolerationSeconds: 30

Here some attempts to log events:
Flux is now running on vmalmakw2s and if I shut that down, there is no event to the Replicaset or Deployment after ~15min. I ran this:
$ watch "kubectl describe replicasets.apps -n flux-system flux-57bd866b6d | grep Events"

And no events on the Deployment either.
I then started the node again and still no events.
Then I shutdown the node with awx-operator deployment and monitored.
The only event on the new ReplicaSet for awx-operator that is deployed after ~6min:
Normal  SuccessfulCreate  89s   replicaset-controller  Created pod: awx-operator-controller-manager-5cd65bb78d-7wn64`

I hope this helps.
Now, I'll edit Flux as you stated above and test again.

[disi@vmalmakw1s ~]$ kubectl edit deployments.apps  -n flux-system flux
deployment.apps/flux edited

Still running fine. Tested sync with git and events.
Deployment log:
Normal  ScalingReplicaSet  3m12s  deployment-controller  Scaled down replica set flux-57bd866b6d to 0 from 1
 Normal  ScalingReplicaSet  3m12s  deployment-controller  Scaled up replica set flux-5c4dd674fc to 1

ReplicaSet log:
  Normal  SuccessfulCreate  6m11s  replicaset-controller  Created pod: flux-57bd866b6d-qj947
  Normal  SuccessfulDelete  3m49s  replicaset-controller  Deleted pod: flux-57bd866b6d-qj947

Running on node "vmalmakms".
Monitoring:
$ watch "kubectl describe replicasets.apps -n flux-system flux-57bd866b6d | grep -A 6 Events"
$ watch "kubectl describe deployments.apps -n flux-system flux | grep -A 6 Events"

Now shutting down "vmalmakms"...
It's working :)
New pod
Events:
  Type    Reason     Age   From               Message
  ----    ------     ----  ----               -------
  Normal  Scheduled  62s   default-scheduler  Successfully assigned flux-system/flux-5c4dd674fc-dfpqk to vmalmakw2s.home
  Normal  Pulled     62s   kubelet            Container image "ghcr.io/fluxcd/source-controller:v1.1.2" already present on machine
  Normal  Created    62s   kubelet            Created container source-controller
  Normal  Started    61s   kubelet            Started container source-controller
  Normal  Pulled     61s   kubelet            Container image "ghcr.io/fluxcd/kustomize-controller:v1.1.1" already present on machine
  Normal  Created    61s   kubelet            Created container kustomize-controller
  Normal  Started    61s   kubelet            Started container kustomize-controller
  Normal  Pulled     61s   kubelet            Container image "ghcr.io/fluxcd/helm-controller:v0.36.2" already present on machine
  Normal  Created    61s   kubelet            Created container helm-controller
  Normal  Started    61s   kubelet            Started container helm-controller

old pod
  Type     Reason        Age    From               Message
  ----     ------        ----   ----               -------
  Normal   Scheduled     9m36s  default-scheduler  Successfully assigned flux-system/flux-5c4dd674fc-bxx65 to vmalmakms.home
  Normal   Pulled        9m36s  kubelet            Container image "ghcr.io/fluxcd/source-controller:v1.1.2" already present on machine
  Normal   Created       9m36s  kubelet            Created container source-controller
  Normal   Started       9m36s  kubelet            Started container source-controller
  Normal   Pulled        9m36s  kubelet            Container image "ghcr.io/fluxcd/kustomize-controller:v1.1.1" already present on machine
  Normal   Created       9m36s  kubelet            Created container kustomize-controller
  Normal   Started       9m36s  kubelet            Started container kustomize-controller
  Normal   Pulled        9m36s  kubelet            Container image "ghcr.io/fluxcd/helm-controller:v0.36.2" already present on machine
  Normal   Created       9m36s  kubelet            Created container helm-controller
  Normal   Started       9m35s  kubelet            Started container helm-controller
  Warning  NodeNotReady  64s    node-controller    Node is not ready

No events on the Deployment, a new ReplicaSet is created:
flux-system    flux-57bd866b6d                              0         0         0       5d20h
flux-system    flux-5c4dd674fc                              1         1         1       15m


Rename touch_set_state() to standby()

Description
Rename touch_set_state() to standby()
Changes include

[ ] Bugfix (non-breaking change that solves an issue)
[ ] New feature (non-breaking change that adds functionality)
[ ] Breaking change (change that is not backwards-compatible and/or changes current functionality)
[ ] Tests
[ ] Documentation
[ ] Other (for changes that might not fit in any category)

Breaking changes
Please complete this section if any breaking changes have been made, otherwise delete it.
Additional comments
Please post additional comments in this section if you have them, otherwise delete it.

le template avec les cases a cocher est pas la pour faire joli ;)


Wallet integration

Integrate with keplr. Apps wont need to.

User has option to use browser extension instead


Error activating nu-plugin-bash-env nu::shell::io_error

Hi,
I installed your plugin (following the PR #29 I made) but it resulted in errors (nu v0.97.1): the home-manager service can be started.
systemctl status home-manager-clementpoiret.service:
Sep 19 08:58:21 desktop hm-activate-clementpoiret[1835]: Activating nu-plugin-bash-env
Sep 19 08:58:21 desktop hm-activate-clementpoiret[2403]: Error: nu::shell::io_error
Sep 19 08:58:21 desktop hm-activate-clementpoiret[2403]:   × I/O error
Sep 19 08:58:21 desktop hm-activate-clementpoiret[2403]:   help: Broken pipe (os error 32)
Sep 19 08:58:21 desktop systemd[1]: home-manager-clementpoiret.service: Main process exited, code=exited, status=1/FAILURE
Sep 19 08:58:21 desktop systemd[1]: home-manager-clementpoiret.service: Failed with result 'exit-code'.
Sep 19 08:58:21 desktop systemd[1]: Failed to start Home Manager environment for clementpoiret.

My nix-config is similar to yours:
    activation = let
      nu-plugin = path:
        lib.hm.dag.entryAfter [ "writeBoundary" ] ''
          run ${pkgs.nushell}/bin/nu --no-config-file --no-history --no-std-lib -c 'plugin add --plugin-config ~/.config/nushell/plugin.msgpackz ${path}'
        '';
    in {
      nu-plugin-bash-env =
        nu-plugin "${pkgs.flake.nu_plugin_bash_env}/bin/nu_plugin_bash_env";
    };

Which helps loading env vars such as:
    extraEnv = ''
      # Source /etc/set-environment to get environment variables
      bash-env /etc/set-environment | load-env

      # Same thing but for session variables set using home-manager
      bash-env ${profileDirectory}/etc/profile.d/hm-session-vars.sh | load-env
    ''

Oops nevermind, I thought nix-unstable already has nu v0.98, but it's still v.0.97.1. It's certainly the issue...


Custom templates gets ignored

I have created a custom template, but when I create a new file from that template the files is blank.
The template is stored in User/SublimeTmpl/template as blogpost.tmpl
I also added a new item to the MENU as:
{
"caption": "BLOG POST",
"command": "sublime_tmpl",
"args": {
"type": "blogpost"
}

Hi,
I have try created blogpost.tmpl in Packages/User/SublimeTmpl/templates/, and added the item in menu. It works normally.

I had a isse with an variable, working now. Thank you.


chore: (video) removed unsupported samples

go/cloud-ai-2021-snippet-cleanup
Devsite update : cl/364902130 [submitted & published]

PTAL @dwsupplee

I do not agree that these samples should be removed so you will need to find someone else to approve and merge this


New fontbakery-fix-whitespace-characters.py script

In discussing #597, @rsheeter noted there are 3 nbsp problems:

fonts without an
fonts with an   that is not the same width as space
fonts with an   that has ink (that is, draws something. e.g. Voces)

Today we cover 1 & 2 in https://github.com/googlefonts/fontbakery/blob/master/bakery_cli/fixers.py#L370
So there should be a new script to detect if whitespace characters have ink: non-0 contours or non-0 composites for glyf, and similarly for the cff table equivalents (eg FontTools glyph.isComposite() glyf.numberOfContours > 0)
https://en.wikipedia.org/wiki/Whitespace_character provides the following list:
U+0009
U+000A
U+000B
U+000C
U+000D
U+0020
U+0085
U+00A0
U+1680
U+2000
U+2001
U+2002
U+2003
U+2004
U+2005
U+2006
U+2007
U+2008
U+2009
U+200A
U+2028
U+2029
U+202F
U+205F
U+3000
U+180E
U+200B
U+200C
U+200D
U+2060
U+FEFF

Related to https://github.com/googlefonts/fontbakery/issues/535

@felipesanches this should be part of the main ttf checker tool

We also need to check anything with a width matches space

We already cover all of the above-listed issues in the ttf checker.
We have these checks:

"Font contains glyphs for whitespace characters?"
"Font has proper whitespace glyph names?"
"Whitespace glyphs have ink?" (also checks coherence of advanceWidth values)

I think that the only thing missing is the list of whitespace characters and making sure the tests are run on any of those.


M3-901 Update disk creation drawer help text.

add: Update help text for Linode disk resizing.

Coverage decreased (-0.005%) to 26.302% when pulling 01ad505a313d0805121e64c8ff699e6edc1adf21 on rjstires:M3-901 into 2117c93dcadf744b0ae123d6157ab23291f05974 on linode:develop.

I resized my linode from Nanode 25GB to 50GB, but when i click "Resize" on my disk the maximum disk size it still my original maximum linode disk size.
(TOTAL_LINODE_SIZE - EXISTING_DISKS) + UNALLOCATED_SIZE

What @patthiel said


hugo-hooks: folders as hooks

Is your feature request related to a problem? Please describe.
Hooks map onto files with the same name as the hook (and its cached variant). For instance, taking the head-start hook from the Best Practice list, the hook will load:

layouts/partials/hooks/head-start.html
layouts/partials/hooks/head-start-cached.html

But there can only be a single file implementing that hook as a result, which means that the hook implementation is custom to a site or theme and not really pluggable.
Describe the solution you'd like
What if we would change this to work with folders? Taking the same head-start hook, resolving this hook would load all files in this folder(s):

layouts/partials/hooks/head-start/
layouts/partials/hooks/head-start-cached/

More than one Hugo module could then contribute snippets to the same hook.
For backwards compatibility, we could still resolve the files (current implementation) as a last resort.
Describe alternatives you've considered
Making a "clone" of your hugo-hooks, but I rather bundle forces instead of creating my own hugo-hooks variant.
Additional context
I use hugo-netlification. If we would implement folders as hooks, the netlification module could be updated to provide the snippets placed in the correct folder(s):

layouts/partials/hooks/setup/netlification.html
layouts/partials/hooks/head-start/netlification.html

After adding the hugo-netlification module to my site, only the netlification configuration is required separately. Files are already at the correct place for inclusion.

Closing this in favour of config based hook setups.
Note that in a subsequent release the requirement of a namespace was introduced. See docs...


Question regarding self.frame_image_transform in vos_dataset.py

in Line 252 in vos_dataset.py, do we need to apply a new random seed before calling self.frame_image_transform(), otherwise each frame in the sequence would still receive the same set of color jitter parameters? Thanks.

Nice catch! This indeed looks like a bug. We do seed it again with np.random (which itself does not get reseeded) in an earlier version of this code: https://github.com/hkchengrex/XMem/blob/a4427dbd748f66ed8fef159b529527e23060c666/dataset/vos_dataset.py#L145
Most likely, we introduced this bug during refactoring. All the models are trained with this version of vos_dataset though.


improve PCM plugin compatiblity with ALSA rate plugin

Thanks for this PR, and sorry for delayed merge (I'm really, really busy lately :/)


CLOUD-44323 added ability to define scopes

@keyki

Jenkins build finished, all tests passed.
Refer to this link for build results: http://ci.sequenceiq.com/job/cloudbreak-pull-request/1832/


I'm using Redux-form to build a relatively large form that includes a slider.
Been trying to implement the redux-form custom component example http://redux-form.com/5.3.1/#/faq/custom-component?_k=qnjmi9 and I have found two issues.
If I try to use the onUpdate method without binding this and adding the redux-form model to a static property. It updated the model and also causes a re-render of the slider.. which then renders back at the start values (looks like you slide to make a change and the slide jumps back to its starting position).
If I bind this then the render method is called in a loop causing 'call stack exceeded' error.
Just using a basic example as below
render method returns

                <Nouislider
                    range={{min: 10, max: 100}}
                    pips={{ mode: 'range', density: 11, format: wNumb({ prefix: '£', postfix: 'k' }) }}
                    step={10}
                    connect={true}
                    start={[40, 70]}
                    onUpdate={this.update.bind(this)}
                    />

                    40k - 70k

update method
    update(textVal, handle, values) {
        this.props.model.onChange(`${values[0]}k - ${values[1]}k`);
    }

Wondering if anyone has encountered or got around this before?
Thanks in advance
James

Hi @james-dowell, thanks for reporting this.
Could you first maybe have a look at http://react-component.github.io/slider/ which is a more React friendly slider than react-nouislider?
Then if you really want to use react-nouislider, maybe you could dig more into the issue because we never encountered such issue.

Hi @vvo,
Thanks for getting back to me :)
Yeah, I'm not sure the  http://react-component.github.io/slider/ is going to cut it but thanks for the suggestion.
I'll do some more investigation into it and just have a play to see if I can get it to work. If you have any thoughts or pointers in the meantime let me know, likewise I'll report back if I get to the bottom of the issue or get a workaround.
All the best!

Okay, so I have done something rather horrible to get it working.. not sure I can leave it like this but for clarity's sake I'll post my findings.
Forgetting the binding of this, if I force the component to not update then I have the desired effect. It appears as if the callback I pass down for either 'onChange' or 'onUpdate' is invoked when the slider is rendered. which triggers my update method to run. Calling onChange method on redux-form (somehow - not sure about this yet) then triggers a re-render of my component.. which creates a circular dependency.
In some ways forcing the component to not update makes sense as I want nouislider to handle the changing and controlling of the slider but it feels really wrong and dangerous to have something like the following:
    shouldComponentUpdate() {
        return false;
    }

Not sure if that has sparked an idea for anyone, or if anyone has a better solution but I'm all ears.
Cheers,

Steps:

Add onUpdate() handler function in a parent component, that modifies that component's state
Create a slider, and give it that onUpdate handler
As per noUiSlider docs - onUpdate is called instantly, when attached
onUpdate() modifies the parent components state, and react calls it's render function
In React the parent's render causes the child (slider) to render, which in your wrapper is triggering the code listed below
The slider is destroyed and re-created, thus calling it's onUpdate() again, which repeats the whole cycle

  componentDidUpdate() {
    if (this.props.disabled) this.sliderContainer.setAttribute('disabled', true);
    else this.sliderContainer.removeAttribute('disabled');
    this.slider.destroy();
    this.createSlider();
  }

Realistically, what you probably want to do is to replace componentDidUpdate() with componentWillReceiveProps() and check if there are changes that actually require re-creating the slider.
Additionally, seems like noUiSlider provides an API to update it's settings without destroying it - http://refreshless.com/nouislider/more/#section-update
In general, calling renders very often is normal React behavior, so you definitely do not want to to assign such heavy operations as re-creating something to a handler that could be called many times per second.

I am not sure if we as react-nouislider need to take actions on the issues you are facing, if you feel like we need, then open a PR and explain it. Thanks!

I am not sure if we as react-nouislider need to take actions on the issues you are facing

Allow me to disagree, your component pretty much guarantees a crash (with call stack exceeded) if it's passed any update handler, which works with the parent component's state (which is very common practice in react)
E.g. roughly like this, if you want a label showing the current value:
onUpdate(value) {
  this.setState({value: value});
}

render() {
  return(

      <Slider onUpdate={this.onUpdate}/>
      {this.state.value}

  )
}

@darkalor To be clear when I said "I am not sure" I did not meant "I doubt you are saying the right things".
On this thread I am lost because you guys are far more comfortable with React than me. I will follow and read any of you valuable advice regarding how we need to update react-nouislider if that's the case.
Thanks!

@vvo Sorry, I interpreted your response wrong. I'll try to provide some more details and a possible solution during the weekend, when I have the time for that.
In short, my solution is to change your componentDidUpdate handling that always re-created the slider to:

check if new settings were received
if anything changed, check if any of the changes require re-creating the slider
otherwise just apply changes via updateOptions/re-attach events/set handle values

Sorry, have been busy the last week. I have the thing working, but just didn't get the time to make a proper clean PR

Really not managing to deal with this for now.
Here's a gist (not properly cleaned up or anything) of what I'm using in my project, take a look if you can.
Note that I'm relying on lodash for working with arrays/objects.
https://gist.github.com/darkalor/12edbb03a347bf8b99b3135ae1daba5e

Sorry we are not maintaining this repository as much as we should. I recommend moving to https://github.com/airbnb/rheostat


fix(index): merge input and instrumented sourcemaps

Based on the work by @adamburgess in https://github.com/webpack-contrib/istanbul-instrumenter-loader/issues/73#issuecomment-427665948.
Added a minimal update to work with the latest webpack (used in CI tests), updated a dependency with a security issue,, fixed a test, and added a fix for the sourcemaps file names.
Type

[x] Bug

Issues

Fixes #73

SemVer

[x] Patch

Heya @michael-ciniawsky, any update on when this might be merged/released?

@d3viant0ne, @michael-ciniawsky any updates on this?

I created a new npm package named coverage-istanbul-loader.  It includes the fix mentioned above as well as other fixes and updated dependencies.

Hey, @filipesilva,
Why to not to use coverage-instanbul-loader?
In my point of vision it's great way to get code coverage. Either way Angular pin all dependencies.
(just I'm happened with broken test after migration to Angular 8 and sourcemap is really broken :( )


Export Module

A new feature to download the channel content as excel would be helpful

Hello CMS Coder,
Thank you for your input regarding the download feature for channel content as an Excel file. I appreciate your suggestion, and I want to assure you that we will be working on developing this feature shortly.

Good work team


fix rename directory and result of deleteDir()

Fix problem that deleteDir() result is false always
Fix problem that can not rename the directory

Thanks!

@frankdejonge What do you think about this PR?

This PR is outdated and maybe no longer necessary