KubeDirector migrated to operator-sdk version v0.8.1

From the testing I've been able to do in my environment I believe the code is fully functional but there are still a number of issues that need to be cleaned up. I've marked all of the known issues with TODO: comments.
Per Joel's comments last week, I would like to get the majority of the changes checked in and clean the code up over time so that other people can work on this version of the code.

I'm going to jump this PR to a non-master branch so when we eventually merge it, it can still have some ongoing development before we promote it to master.

Do you remember where build.sh and docker_build.sh are invoked from? I noticed that the new SDK normally puts those files in build/bin/ but you have placed them just in build/.

While we're messing around in the Makefile we should also fix the behavior of the "compile" target. It works ok as a sanity check done during CI, but it is broken as a target that you might want to invoke manually (doesn't cross-compile and writes to wrong file). It should do the same go build command as is done inside build.sh. Or maybe it should just invoke build.sh?

Do we really need the empty example CR yaml files (like deploy/kubedirector/kubedirector_v1alpha1_kubedirectorapp_cr.yaml) in the deploy/kubedirector directory?

Not done w/ review pass yet... will return for more later.

I have to pause my review again for a bit. I really only have these three files to look at, although I've been intentionally saving the more interesting ones to the last:

pkg/controller/kubedirectorcluster/cluster.go
pkg/controller/kubedirectorcluster/kubedirectorcluster_controller.go
pkg/executor/cluster.go

There's stuff to discuss around "status gen" UUIDs and valid-markers. I'm going to type up a description of this functionality here for both of our benefits, because it's been a while since I set up this arrangement and I need to wake up those brain cells.
The status gen tracking served two purposes:

We don't want clients other than KubeDirector making changes to the status block of a KDC resource.

We don't want to process a stale poll update from the framework.

Both of those purposes require a bit more discussion.

For item 1:
The status block of a KDC resource is essentially KD's persistent state storage for managing that resource. If that got corrupted then things would break. So we don't want other clients writing to the status block. But clients should be allowed to change the spec and metadata.
If the status was a "subresource" then it could be protected with ACLs, but a) that's kind of fragile as it relies on the K8s system being configured correctly and staying correct every time ACLs are created or modified, and b) originally we supported running on versions of K8s that didn't support subresources for CRs. Issue "b" there is now a non-issue, but issue "a" is still worth thinking about.
The general implementation of this is that the the reconciler and validator share access to the "status gen" data for that KDC in memory. When the reconciler wants to make an update to the status, it can store in memory the expected value of a special GenerationUID property that it is going to include in its write. When the validator sees an incoming write that changes the status, it can check to see whether the GenerationUID value is as expected. If not then the write will be rejected.
The "validated" flag is used to make sure that we only accept one change for any given GenerationUID.
This all happens in validator.admitClusterCR in the old code.
I don't think we want to move to using subresources quite yet, both because of issue "a" above and also because that would probably involve a bit more code churn and unknown knock-on issues on top of all this other churn. So I think the status gen is still important to have.
(If it does turn out to be problematic to keep doing status-protection this way however, then we may need to look at moving over to the use of subresources.)

For item 2:
The operator SDK's framework fires off the reconciler handler when a watched resource changes. It also just fires the handler periodically. It's possible in this case to have one of those periodic "refresh" polls coming in at about the same time that a previously triggered handler is writing some changes to the resource.
With the old framework, the resource itself would be passed in along with the handler invocation. So in the case described above, the incoming poll triggering the handler could be carrying a version of the resource that did not include the changes that we just wrote into the resource. It is stale and should be ignored.
We can check the GenerationUID value to see if it is the most recent value that we remember writing to the resource. If it isn't, then this is a stale polling-driven update and we should bail out of the handler and do nothing. This happens in reconciler.handleStatusGen in the old code.
With the new framework, the resource is not given to us and we have to fetch it ourselves. This removes the possibility of this stale-resource race and it looks to me like item 2 no longer matters.
That does NOT mean that handleStatusGen can just be thrown away, as it does a few different tasks.

Given that item 2 is out the window, we could rename handleStatusGen to something like handleNewCluster and simplify it down to this (untested example code):
// handleNewCluster looks in the cache for the last-known status generation
// UID for this CR. If there is one, return true to keep processing the CR.
// If there is not any last-known UID, this is either a new CR or one that
// was created before this KD came up. In the former case, where the CR status
// itself has no generation UID: set the cluster state to creating (this will
// also trigger population of the generation UID) and return false to cause
// this handler to exit; we'll pick up further processing in the next handler.
// In the latter case, sync up our internal state with the visible state of
// the CR and return true to continue processing.
func handleNewCluster(
    cr *kdv1.KubeDirectorCluster,
    reconciler *ReconcileKubeDirectorCluster,
) bool {

    lastKnown, ok := ReadStatusGen(cr, reconciler)
    if ok {
        return true
    }
    incoming := cr.Status.GenerationUID
    if incoming == "" {
        shared.LogInfo(
            cr,
            shared.EventReasonCluster,
            "new",
        )
        cr.Status.State = string(clusterCreating)
        return false
    }
    shared.LogWarnf(
        cr,
        shared.EventReasonNoEvent,
        "unknown with incoming gen uid %s",
        incoming,
    )
    writeStatusGen(cr, reconciler, incoming)
    ValidateStatusGen(cr, reconciler)
    ensureClusterAppReference(cr, reconciler)
    return true
}

Now let's turn to the issue you noted in a comment in kubedirectorcluster.Reconcile. It's no longer possible to do deleteStatusGen when the resource has been deleted, because in the new framework we don't know the resource UID of the deleted CR.
One possibility is to index the cache by a resource's NamespacedName rather than by UID. This would work fine in 99.999% of the situations, but there's a tiny race window where another CR with the same NamespacedName could be created and processed before we process the delete of the previous resource.
We can instead do deleteStatusGen in handleFinalizers when we are removing our finalizer. Once a CR has been deletion-timestamped we will no longer do anything interesting with its status block, and so the status gen is no longer important. Especially since we don't have to worry about stale poll updates anymore.
This does raise the question: what happens if someone else removes our finalizer? That's an issue in any situation where a finalizer is actually useful, so I guess it's just something you live with. Should the KDC validator explicitly prevent other clients from removing the KD finalizer (similar to how we protect the status block)? I thought about that a little in issue #38 and decided not to implement that protection yet, but maybe that's a decision we should revisit.
Anyway, let's say we move deleteStatusGen into handleFinalizers, right after the call to RemoveFinalizer. Like this (untested example code):
// handleFinalizers will remove our finalizer and clear the cached status gen
// if deletion has been requested. Otherwise it will add our finalizer if it
// is absent.
func handleFinalizers(
    cr *kdv1.KubeDirectorCluster,
    reconciler *ReconcileKubeDirectorCluster,
) (bool, error) {

    if cr.DeletionTimestamp != nil {
        // If a deletion has been requested, while ours (or other) finalizers
        // existed on the CR, go ahead and remove our finalizer.
        removeErr := executor.RemoveFinalizer(cr, reconciler.Client)
        if removeErr == nil {
            shared.LogInfo(
                cr,
                shared.EventReasonCluster,
                "greenlighting for deletion",
            )
        }
        // Also clear the status gen from our cache, regardless of whether
        // finalizer modification succeeded.
        deleteStatusGen(cr, reconciler)
        return true, removeErr
    }

    // If our finalizer doesn't exist on the CR, put it in there.
    ensureErr := executor.EnsureFinalizer(cr, reconciler.Client)
    if ensureErr != nil {
        return true, ensureErr
    }

    return false, nil
}

Last thing: With that change we now have the situation where we could get a poll-triggered handler for a KDC CR that has a deletion timestamp and no finalizer, which has no entry in our status gen cache. We need to make sure that is handled correctly, i.e. ignored.
So in the KDC reconciler, we want to make sure we don't do any processing for a CR that has a deletion timestamp, beyond (if necessary) removing its finalizer and removing its status gen from the cache. It looks like handleFinalizers already takes care of that. However, the current ordering of the calls to handleNewCluster (used to be handleStatusGen) and handleFinalizers is unfortunate in that handleNewCluster would re-add the status gen back into the cache. I believe the simplest answer to that problem is to switch the order that those functions are called -- always do handleFinalizers first. I think the reason that the invocations were in their current order was to properly ignore stale poll handlers (which is no longer an issue).
I'm wondering if we need any change to the KDC validator. I think the validator will be fine as-is, since we won't be trying to make any changes to the status block after clearing the cached status gen UID. In fact the deferred func in syncCluster that does status update will explicitly do nothing if the deletion timestamp is set. But if for some reason the validator gets cantankerous about something related to these changes, we can look into fixes that change its behavior for CRs that have a deletion timestamp.

Some discussion of the problems with controller-runtime not yet supporting PATCH:
The potential gotcha in all of these cases (where we try to do a PATCH without actually doing a PATCH) is that there could be a race where some other client has updated the object since we last fetched it, and if we write back an object that we've been holding in memory then our write can fail -- and if we take steps to make it not fail (more on that in a minute) we can be clobbering someone else's changes.
Doing a GET of the object right before we change and update the object does reduce that race window. Although it doesn't technically eliminate it.
For objects like services that KD creates, I'm fine with clobbering someone else's changes, because other clients should not be modifying the objects anyway. For a KDC however we do want and expect clients to change the object.
So we should have awareness that the race can happen, and we really should move to using PATCH when controller-runtime enables that. Or move to using a status subresource, which has been discussed elsewhere -- although that would only help us for status-block updates and doesn't help us when it comes to doing finalizer updates and other stuff.

What does that mean we should do for all these wannabe-PATCH cases?
I think we should get some shared clarity on what is up with controller-runtime. The 0.2.0 milestone on their GitHub only has one remaining issue, but I don't know if that's necessarily an indication of an imminent stable release. Possibly we could/should use a beta release? But there might also be unknown issues with moving our whole build (including the framework) over to that new version.
Unless it looks really tempting to switch over to 0.2.0 ASAP, we should implement a common PATCH utility function in our code. It could take as input a) the current known object and b) a func for appropriately modifying the object to apply the change.
Now as mentioned above, even if we do a GET right before the PUT, there's a race possibility. And I think the CR that we receive from a GET will include a metadata.resourceVersion value, which automatically changes on every write. If someone else sneaks in an update to the CR before we do our PUT, our PUT will be rejected because it has a non-matching resourceVersion.
Maybe we should do something like:

Rather than starting out with a GET, just change the initial input object and try to PUT that.
If that is rejected because of non-matching resourceVersion, do a GET to fetch the more recent object, and try changing and PUT-ting from that.
If THAT also fails because of non-matching resourceVersion, hmm. We'd need to discuss what the retry and final error behavior is.

Done w/ review pass. I'm also going to ask Swami to take a look in here, at least at the particular spots I commented on.
Once we all chew on these issues a bit and knock out any low-hanging fruit and major changes that we all agree on... we can talk about which remaining things really need to be tackled before at least settling this into its branch here, and which things can just be recorded as new issues. Then I can look at getting this branch up to sync with the current master.

One more thing. In the old framework we could specify the resyncPeriod for watching a particular CR, so that we could control how frequently our handler might get triggered. Is there an equivalent in the new framework?

One general comment on the client object. This seems like an ideal candidate for a global client object.
This way we don't need to be passing down through all the functions. This is how we were using it before

One general comment on the client object. This seems like an ideal candidate for a global client object.
This way we don't need to be passing down through all the functions. This is how we were using it before

One more thing. In the old framework we could specify the resyncPeriod for watching a particular CR, so that we could control how frequently our handler might get triggered. Is there an equivalent in the new framework?

Good catch. That was on my todo list but I didn't get to it. I've checked in the changes but I have a couple of questions.
I currently have the polling cycle set to 30 secs for both the Config and Cluster CRs.

Is that polling cycle OK for the cluster CRs?

Can we not poll at all for the config CRs? There's no status to check and we can just reconcile when a change is posted.

BTW I notice that the Travis CI build is failing with this error message:
build github.com/bluek8s/kubedirector/cmd/manager: cannot load github.com/Azure/go-autorest/autorest: ambiguous import: found github.com/Azure/go-autorest/autorest in multiple modules:
	github.com/Azure/go-autorest v11.7.0+incompatible<EMAIL_ADDRESS>	github.com/Azure/go-autorest/autorest v0.5.0<EMAIL_ADDRESS>Makefile:266: recipe for target 'compile' failed

Kinda weird. Does "make compile" work for you locally?

Just FYI my plan is to still land this on a branch this evening and move any unresolved issues to tracked Issues on GitHub. Any remaining work on this branch today should focus on low-hanging fruit.

List of things that will be converted to GitHub issues tonight (unless there's some in-flight change or response that lands soon addressing it):

What deletes the metrics service object on cleanup?
Why is the storage.size KDC property always required in Don's setup?
Why is a "status" subresource required to be declared in Don's setup?
Does KD correctly watch all namespaces for KDCluster? What is the namespace passed to the manager used for?
Does KD correctly only watch its OWN namespace for KDConfig?
Fixes for object-update behavior (e.g. handle necessary retries across multiple handler invocations, instead of hardcoded single-retry within handler).
Set polling interval to 30 seconds for KDCluster. Don't poll for KDConfig.
Set correct owner ref for created secret/service/webhook and verify they go away when KD deployment is deleted.
Management of the K8s client (shared global vs. passed around).
Remove replicasets from the RBAC, or determine why that is not possible.
Fix Travis CI build.

Set polling interval to 30 seconds for KDCluster. Don't poll for KDConfig.
I checked in a fix for this issue earlier this week.

Issues filed: https://github.com/bluek8s/kubedirector/labels/Project%3A Operator SDK


Flaky hang on Travis during class tests run

I have seen this situation at least 4 times recently

The worst part that it's not consistent and happens sporadically with about 20% chance.
It affects CI and master branch builds.
The particular tests is different, but it's always during class language tests.

I saw this too in a recent PR.

Per @lzybkr

I don't know, but a search of 'create linux core dump' suggests that gcore or kill may be capable of creating the core dump.

It is dup #3670


google sign in error code 10:

When user signs in through Google. A toast message is shown and user cannot log in

Fixed but needs more testing

doesn't work on machines that differ from original local

UPDATE

Works if I add your SHA keys to firebase
Works if used a signed app


Add missing release to ecma_op_container_foreach

The result of the callback function was not freed
Fixes #2910
Co-authored-by: Tibor Dusnoki<EMAIL_ADDRESS>JerryScript-DCO-1.0-Signed-off-by: Adam Szilagyi<EMAIL_ADDRESS>

@szilagyiadam it seems there is something wrong the test fails on the CI.

@szilagyiadam the test is placed in the tests/jerry/fail/  directory indicating that this should fail. However it does not fail (that is why the CI fails). Is this really the place where the test should be?

The test file is still in the incorrect place. It should be in the tests/jerry/es2015 subdir (and not in the tests/jerry/ dir)

Did the requested changes.


HttpPostRequestEncoderTest failures introduced by 08748344d852a07011be6e74bb5ddc6dcf221c45

After commit 08748344d852a07011be6e74bb5ddc6dcf221c45 some tests in HttpPostRequestEncoderTest are now failing.

Failed tests:
HttpPostRequestEncoderTest.testDataIsMultipleOfChunkSize2:412->checkNextChunkSize:423 Chunk is not 8096 bytes expected:<8096> but was:<8094>

Failed tests:
HttpPostRequestEncoderTest.testDataIsMultipleOfChunkSize1:390->checkNextChunkSize:423 Chunk is not 8096 bytes expected:<8096> but was:<8093>

/cc @violetagg

@Scottmitch strange not failing here :( Will investigate

@Scottmitch I tested yesterday on my local machine and they were ok + the PR voter also passed with all tests green. I'll also check again.

I'm not able to reproduce this failure neither on Mac nor Ubuntu
maven 3.3.9
java version "1.8.0_131"
Java(TM) SE Runtime Environment (build 1.8.0_131-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode)
Linux bosh-lite 4.2.0-36-generic #42~14.04.1-Ubuntu SMP Fri May 13 17:27:22 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
maven 3.3.9
java version "1.8.0_141"
Java(TM) SE Runtime Environment (build 1.8.0_141-b15)
Java HotSpot(TM) 64-Bit Server VM (build 25.141-b15, mixed mode)
Darwin MacBook-Pro-2.local 16.7.0 Darwin Kernel Version 16.7.0: Thu Jun 15 17:36:27 PDT 2017; root:xnu-3789.70.16~2/RELEASE_X86_64 x86_64

Fixed in https://github.com/netty/netty/pull/7005.

Fixed by https://github.com/netty/netty/pull/7005


ntop: fix alignment issues

ntop would behave unpredictably when used with if/else statements for
differently-sized ntop buffers (8/20, depending on inet AF type).
Upcasting to 20 fixes the issue for some tools, but caused other tools
to stop translating the address correctly. This second issue was caused
by LLVM adding pad between elements of the printf buffer, which caused
our printf implementation to read from incorrect memory addresses. To
prevent LLVM from adding paddings, we can change the size of ntop to a
stack-aligned size. 24 is the lowest stack-aligned size we can use for
ntop, so the new buffer size now is always 24.
Fixes: https://github.com/iovisor/bpftrace/issues/809

doesn't work for me:
# ../build/src/bpftrace -v tcptest.bt
[...]
44: (85) call bpf_probe_read#4
45: (61) r1 = *(u32 *)(r10 -92)
46: (67) r1 <<= 32
47: (61) r2 = *(u32 *)(r10 -96)
48: (4f) r1 |= r2
49: (7b) *(u64 *)(r10 -136) = r1
50: (61) r1 = *(u32 *)(r10 -84)
invalid read from stack off -84+0 size 4

Error loading program: kprobe:tcp_set_state

tcptest.bt is in the ticket.

Yeah, something is still not right. Closing, will open a new one when I figure it out.


Consider to refactor webview.h

Though webview.h includes C specific part, it is not possible to include this header and compile a pure C program because there are C++ code in this header.
It would be better to split webview.h into a header for C, a header for C++ and a source for C++ so that the users of webview.h can include it with a pure C program.
Still thanks to your hard work.

Currently the way to exclude C++ code is to define WEBVIEW_HEADER.


SvelteKit 3 various breaking changes notes

A list of things to keep an eye out for when doing 3.0

check codebase for TODO 3.0 comments

#11794 is a potential thing to change. I don't have a very strong opinion about what the behavior should be here, but I do have a strong opinion that we not change it except in a major version.


distribution: simplify

Many existing functionalities aren't actually used by anyone.
This commit delete these and simplify the rest.

@mdietz94 in case you also want to take a look

Codecov Report

Merging #428 into master will decrease coverage by 0.31%.
The diff coverage is 50%.

@@             Coverage Diff              @@
##             master     #428      +/-   ##
============================================
- Coverage     70.53%   70.22%   -0.32%
+ Complexity      634      622      -12
============================================
  Files           136      136
  Lines          2912     2878      -34
  Branches        220      215       -5
============================================
- Hits           2054     2021      -33
+ Misses          776      774       -2
- Partials         82       83       +1

Impacted Files
Coverage Δ
Complexity Δ

...ain/java/com/google/api/gax/core/Distribution.java
50% <50%> (-27.59%)
4 <1> (-12)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 485b717...aea9c2d. Read the comment docs.

@garrettjonesgoogle PTAL

cc @jba


Adding Max-Age in Set-Cookie function

When setting the cookie in:
response.set_cookie(app.session_cookie_name, session_id,                             expires=expires, httponly=httponly,                             domain=domain, path=path, secure=secure)
Would it be possable to add a option parameter of max_age when setting the cookie for browsers that support it?
I have a embedded system where the user can manually set the server clock. If the user sets the server clock to a point in time after the current time some browsers (e.x. Firefox) deletes the cookie every time and you cant login.
Instead of only using a absolute time via timestamp it would be great if max_age was added as an option where a number of seconds untill the cookie expires can be set.
if the browser support both and both are set. The Max-Age parameter till override the Expires parameter.
Source: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Set-Cookie

@pokpe the cookie length is coming from this setting in your flask app: https://flask.palletsprojects.com/en/2.3.x/api/?highlight=session#flask.sessions.SessionInterface.get_expiration_time


Improved extension checking logic

This changes the logic that checks and enabled extensions in our plugin.
It works with a map to which we add the extensions we want.
The entry then points to a boolean that the logic either sets or clears depending on whether the extension is available.
If this pointer is a nullptr we deem the extension as a requirement and it not being available will fail initialization of the plugin.
Finally we enabled all extensions that were requested and found.
The boolean flags allow further logic to actually implement the usage of the extension.
I've also moved most of the FB extensions out of the Android only sphere, I suspect a number of these will become extensions also implemented by other vendors as has happened with a number of Microsoft and Valve extensions that still bear their names (especially around refresh rates and such, really weird those aren't part of the core spec already).

I also snuck in setting the view configuration type...


T-joint implementation

Hi,  I noticed that the T-joints still need to be implemented. Do you have any rough implementations of T-joints? Or can you guide me through how to implement it myself?
Thank you

I guess one should start with a SemiCylinder but one face should be inclined at 45 degrees. The easiest would be if there was a skew transform so I guess that would be the first thing to add.
Then, everything else is just copy/rotate/mirror.
Note that a T-joint is a sort of collection of Shapes so it's nothing like current classes (Operation, Shape, Stack). I guess this still requires some thought.
What do you have in mind?

Thank you for the response. I will think about your idea, try implementing it, and see where it goes.

This might be a nice starting point: "pipeline mesh generator based on blockmesh"
https://oar.ptb.de/files/download/530.20230512.pdf
Although I don't know how well the code is documented using equations.

Fantastic, this will be helpful to me. I will start scripting and get things done, and once I have everything, I will discuss the object-oriented framework. As I have a lot of  features in my geometry

T-joint
Pipe branches to two pipes
multiple pipes merge in one big cylinder

Thank you.

I suggest:

Add a shear transform to ElementBase
Create a 'private' SemiCylinder Shape with an (arbitrarily) tilted end face
Make a new entity type, 'Assembly', that holds multiple Shapes
Create Assemblies, starting with a T-joint and expanding to other stuff

Check out the feature/assembly branch. I whipped a rough draft of what I suppose T-joint might look like. Read through the commit message to see what's left to be done. Next week I'll be on holidays, hence - offline.

Hello, this is nice. I'll be sure to see how I can take it from here with my rusty Python knowledge.
Have a nice holiday

Aye. I'm working on a nicer way of doing everything now. Looks like the final implementation will be far from the initial makeshift stuff.

Merged new stuff with the development branch - T-joint, L-joint (a sharp elbow) and an arbitrary N-joint. Will be pushed into master after some more testing.

Next, the cylinder thing. This is a bit more complicated one.

This looks fantastic. Thanks a lot.


[CS-1099] fix: security vulnerabilities in golang packages

fixing security vulnerabilities:

github.com/hashicorp/go-getter from 1.7.0  to 1.7.4
golang.org/x/net from 0.17.0 to anything above 0.23.0
google.golang.org/grpc from 1.51.0 to 1.58.3
google.golang.org/protobuf from 1.28.1  to 1.33.0
from<EMAIL_ADDRESS>to<EMAIL_ADDRESS>via upgrading github.com/hashicorp/vault/api from v1.8.2 to v1.14.0

also have upgraded go.mod file from 1.19 golang to 1.21
im unable to test things, how can we review these fixes are good enough and make a new release? @vipul-sharma20 / @sreeram-narayanan
attaching the trivy vulnerabilities found on the master branch:
go.mod (gomod)
==============
Total: 6 (MEDIUM: 4, HIGH: 1, CRITICAL: 1)

trivy_before_fix.txt
and trivy didn't find anything on this new vanta_security_upgrades branch. thus getting rid of 2 vulnerabilities (high + critical)
this is the command used btw:
trivy fs ./ --scanners vuln --severity HIGH,CRITICAL,MEDIUM  > trivy_before_fix.txt

a lot of prometheus libraries have vanished for some reason. did i do something wrong? or someone else didn't do go mod tidy in the previous commits?


:sparkles: feat(config): Support configuring more than one address

Description
Concise description of proposed changes
Testing
Explain the quality checks that have been done on the code changes
Additional Information

[ ] I read the contributing docs (if this is your first contribution)

Your ENS/address:

Current dependencies on/for this PR:

main

PR #184   👈

This comment was auto-generated by Graphite.


[ENH] - Adding a licences (BSD?)

A licence is currently missing and should be added for the user to know what are the constraints.

c68f4e8 closes this issue


minor issue in font color in version2 , pls look at the below pic its very difficult to read them

minor issue in font color in version2 , pls look at the below pic its very difficult to read them

Hey @Manik400, thanks for pointing it out.
Actually there's an issue when the System is on Light Mode.
In dark mode / expected:

duplicate #184


Expose AudioStreamPlaybackInteractive::get_current_clip_index().

Closes godotengine/godot-proposals#10494
Add a way to get what is the currently playing clip from AudioStreamPlaybackInteractive.
Exposes get_current_clip_index()

Just a head up, this is failing checks now because the added const needs to be matched in the docs.

Just a head up, this is failing checks now because the added const needs to be matched in the docs.

Oh thanks, I didn't realize that. So many mistakes for such a simple PR.
I think it should be fixed now.

Thanks!


Specimen plugins

It's currently possible to add your own specimens but I'm not super-happy with the current implementation and therefore the API that is exposed. I'm opening this issue as for tracking progress on this feature.
This is on top of our priority list because people keep asking for new Specimens or even make pull requests to add them. Rather than adding more Specimens to Catalog core, I'd like to build the Specimen API in a way that is flexible enough for the various use cases and document how to do it. This way, we don't bloat Catalog with more functionality that only a part of its users need and can enable a Specimen ecosystem that is built by the community.

@herrstucki I couldn't find anything about adding custom specimen in the docs :/ Did I just miss it or is there another issue that describes how to do that?


Point triangulation solving

Hi all,
I'm trying to use the triangulation method on my own dataset but I struggle to understand how can the third component of the 2D homogeneous keypoints be ignored.
The triangulation problem aims to solve the equation AX=0 where A is constructed from the known projection matrices P and the image coordinates of the 2D keypoints x.
The matrix A is constructed from the fact that we want X such that x ^ PX = 0 for ^ denotes the cross product between two vectors.
As far as I understand, x=[u, v, w] where u and v are the image coordinates of the 2D keypoints but I don't understand how to get the value of w.
From the code, I understand that w is assumed to be equal to 1 but in that case,  u and v are only known up to a factor from the 2D keypoints, is that correct? Is so, how can I consider w as an additional unknown and solve for it alongside with the 3D keypoint coordinates?
Thanks

Hi, @nisace! Thank you for thoughtful question.
You're completely right, except that X=[u, v, w] (3-dim vector). Actually it's a 4-dim vector X=[x, y, z, w] (or 3D homogenous vector).
You can refer to this function for more details about algebraic triangulation.

Thanks @karfly for your answer.
I agree that X=[x, y, z, w] is the solution of AX=0 and that we can get the 3D point coordinates as [x/w, y/w, z/w].
My question is about the 2D keypoint coordinates x (or the points argument of the function). These coordinates are of dimension 3 [u, v, w]. However, the function takes points of shape (n ,2) because it assumes that xcan be built from the image coordinates u and v and assumes that w=1.
It's the w=1 that I don't understand. What allows such an assumption?
Thanks again for your quick response.

The homogeneous scale factor is eliminated by a cross product. (from "Multiple view geometry in computer vision", Richard Hartley and Andrew Zisserman, 12.2, p. 312).

Yes I saw that but the question then is: why would the pixel coordinates of the 2D keypoints on the images correspond to a w value of 1?
I made an experiment where I simulate multiple images by projecting a 3D point with different matrices which gives me a set of "2D" points of dimension 3. Then I give the matrices and the 2D points as input to your function to see if I can reconstructed the 3D point.
If I understand correctly, the first two components of the vector given by the projection are the pixel coordinates. The third component can be ignored (see https://www.khronos.org/registry/OpenGL-Refpages/gl2.1/xhtml/gluProject.xml). Thus I thought your function would expect the first two components as inputs. However the result is wrong if I do so.
I need to divide the first two components of the 2D points by the third component in order to get the correct result.

@karfly your help would be much appreciated if possible. Thanks in advance !

Why is P[-1][-1] equal to 1? I understand that P is constructed as follows (from http://ksimek.github.io/2013/08/13/intrinsic/) and your code seems to follow this convention.

And even P[-1][-1] = 1 does not imply w = 1, we need the P[-1] = [0, 0, 0, 1] for that.
Or maybe you have some assumptions on the matrix [ R | t ]?
Thanks

@karfly Hi! I just read this issue and I admit I don't understand why would P[-1][-1] be 1 and why would the last component of x be 1 in this case without any other assumption.
Thanks!

sorry for bothering you guys, but I am also confused about the triangulation process.
In the triangulate_point_from_multiple_view_linear(pro_matricies, points), why the points' shape is (n,2)but not (n,32,2)(n for views, 32 for numbers of human3.6 2d points, 2 for x and y)? or I have some misunderstanding about this? hope you guys can help,thanks~


ProhibitPlainJunitAssertionsRule reports AssertJ assertion as plain JUnit assertion

I'm getting Avoid using Plain JUnit assertions (ProhibitPlainJunitAssertionsRule) PMD error for the following code:
import static org.assertj.core.api.Assertions.assertThat;
import org.junit.Test;

public class DummyTest {
    @Test
    public void test() {
        assertThat(Boolean.TRUE).isTrue();
    }
}

My assertion however is not a plain JUnit assertion, but it comes from AssertJ framework.

@yegor256/z please, pay attention to this issue

@grzechol/z this project will fix the problem faster if you donate a few dollars to it; just click here and pay via Stripe, it's very fast, convenient and appreciated; thanks a lot!

Sorry guys, I meant to report this in quile project instead of this one.

Job gh:yegor256/takes#858 is not assigned, can't get performer

This job is not in scope


Implement projection of k-vectors to nlm

This computes the radial integral and spherical harmonics for the k-vectors. The resulting values agree with pylode k_dep_factor_reordered with G_k set to 1.0.

LGTM!


optimize cross-examine features queries

empty or too loosely defined queries can kill the backend - need to investigate why and optimize query handling

Rewritten the whole backend from scratch and refined the fronend.
The current version discourages very generic queries (block you from submitting the form without a query criteria,
though you can still search data for the whole country) and even full Tunisia search does not seem to kill the backend, though the markers query is still not very fast.
Frontend was connected in thes PR:
https://github.com/acdh-oeaw/vicav-vue3/pull/163
New backend PR:
https://github.com/acdh-oeaw/vicav-app/pull/90


Racon gives empty output (fasta file)

Hi,
I am very new to genome assembly, I have an assembly I created with NextDenovo with PacBio data.
I used minimap2 (the command I used: minimap2 -ax map-pb -t 20 --cs=long /dorotheeh/hadjez/nd24.asm.fasta /dorotheeh/hadjez/FilteredResults_Q20.fastq > pb_alignment_minimap2.sam)
Now I am trying to run Racon but the output file I get is empty, this is my command: racon -m 8 -x -6 -g -8 -w 500 -t 14 -q 20 -u /dorotheeh/hadjez/FilteredResults_Q20.fastq /dorotheeh/hadjez/pb_alignment_minimap2.sam /dorotheeh/hadjez/nd24.asm.fasta > nd_Racon.fasta
Can you help me understand what I'm doing wrong please?
Thank you in advance,
Rachel

Hi Rachel,
Your command lines look correct.
Unfortunately, I cannot reproduce your issue using your command lines on small sample data (available in the repo).
Does this produce output for you as well?
minimap2 -ax map-pb -t 20 --cs=long test/data/sample_layout.fasta.gz test/data/sample_reads.fastq.gz > aln.sam
racon -m 8 -x -6 -g -8 -w 500 -t 14 -q 20 -u test/data/sample_reads.fastq.gz aln.sam test/data/sample_layout.fasta.gz > out.fasta

A couple of questions:

Does Racon finish successfully the entire process?
Does the SAM file contain alignments that look reasonable?
What happens if you remove the -q 20 option from Racon?
What is the size of your input reads file, the input SAM file and the input reference file? (It is possible that, if they are very large, that they cannot fit into memory and the process gets killed.)

Best regards,
Ivan.

Hi Ivan!
First of all thank you for your reply! I ran the command you wrote but still it gave the same results.

Racon ran for about 2 days and generated an empty file so I "killed" the job. After it stopped it generated an error file (size 136360KB) that I couldn't open.
The SAM file looks like this:
@SQ     SN:ctg000900    LN:437423
@SQ     SN:ctg000910    LN:102996
@SQ     SN:ctg000920    LN:1577448
@PG     ID:minimap2     PN:minimap2     VN:2.17-r941    CL:minimap2 -ax map-pb -t 20 --cs=long /dorotheeh/hadjez/nd24.asm.fasta /dorotheeh/hadjez/FilteredResults_Q20.fastq
m64047_230223_061922/29/ccs     16      ctg000300       4682658 60      1288M1D3493M1I3M1I2692M1D877M1I322M1D9337M      *       0       0       GTTACGTTAATTTACAGCTTTTAAAATTCCTTTTTATATCTATTCATCAGAAATAAAACAATTTAATTTAAATATATCAACAACATATGAATCAATGTGCATCCAAAATGTACGGGATTAGTTGGAATTTTTATAATGATTTTAATTTATGATCTTTCAATAATTTTTGATTTAATTAATAAATGTAATAAATAAAAGAGTTTTGGTCAAATATA.....

I copied a part of it so you can see, it looks different than SAM file I generated with BWA (short reads) for example.
Does the format I pasted look normal for long read alignment?

Nothing changes when I ran it without -q 20...
Sizes are:
reads (long reads) in fastq- 67,773,498 KB
genome assembly in fasta- 72,934 KB
SAM file- 119,878,001 KB

I think its not a problem of memory, or else it would stop running no? Maybe I'm mistaking (once again I'm very new to this).
Thank you for your patience and help!!
Sincerely,
Rachel

Hi Rachel!
Thanks for getting back!
Can you also copy/paste the output of Racon from your terminal if you still have it, so I can see where it is hanging?
Did it reach the step with the sliding arrow that says "generating consensus"?
The SAM file you sent looks normal, thanks for sharing it!
Racon will try to load all reads into memory at the beginning of the process. Your FASTQ file is ~64GB large and the input SAM file is ~114GB in size. How much memory does your machine have available?
If your machine doesn't have this much RAM (actually, somewhat more than this because of data structures and algorithms), the process could get killed by the system, or it could go into swap and become extremely laggy to the point that it looks frozen.
In this case, the solution is to separate contigs from your reference file into multiple files, and extract only the alignments from the SAM file which map to each contig group, and then run them separately. (This can be scripted.)
Still, this does not explain why the small test example doesn't produce any output for you.
Can you rerun it with /usr/bin/time so we can record the exit status? For example, like this:
minimap2 -ax map-pb -t 20 --cs=long test/data/sample_layout.fasta.gz test/data/sample_reads.fastq.gz > aln.sam
/usr/bin/time --format="cmd: %C\\nreal_time: %e s\\nuser_time: %U s\\nsys_time: %S s\\nmax_rss: %M kB\\nexit_status: %x" -o out.memtime \
racon -m 8 -x -6 -g -8 -w 500 -t 14 -q 20 -u test/data/sample_reads.fastq.gz aln.sam test/data/sample_layout.fasta.gz > out.fasta

After this, please copy the out.memtime contents and the stderr output from your terminal (the output from Racon).
Best regards,
Ivan.

Hi Ivan!
Sadly I could not run it, and then we had a problem with the linux cluster at our university so finally I managed to run it on "galaxy" website and it gave me a great output!
Thank you so much for your help!!!
Rachel

Glad to hear it's resolved!
Best regards,
Ivan.


ThunderFX splitter should send no_grad regions to Thunder

🐛 Bug
Splitter for Thunder as a Dynamo backend should send regions of code under a "no_grad" context manager to Thunder. Currently, it chooses to send these computations to Inductor.
from thunder.dynamo import ThunderCompiler
import thunder
import torch

def f(x):
    with torch.no_grad():
        return x * x

jit_f = thunder.jit(f)
backend = ThunderCompiler()
compile_f = torch.compile(backend=backend)(f)

x = torch.randn(3, 3, requires_grad=True)

out = jit_f(x) # Works with thunder.jit with a warning
out_1 = compile_f(x) # Works with torch.compile but sends the computation to the Inductor instead of Thunder

print(backend.subgraph_infos[0].split_graph_module.print_readable())

prints:
class GraphModule(torch.nn.Module):
    def forward(self, l_x_: "f32[3, 3]"):
        # No stacktrace found for following nodes
        inductor_1 = self.inductor_1(l_x_);  l_x_ = None
        return (inductor_1,)

    class inductor_1(torch.nn.Module):
        def forward(self, l_x_: "f32[3, 3]"):
            # No stacktrace found for following nodes
            _set_grad_enabled = torch._C._set_grad_enabled(False);  _set_grad_enabled = None

             # File: <ipython-input-3-f5a278fc59c9>:7 in f, code: return x * x
            mul: "f32[3, 3]" = l_x_ * l_x_;  l_x_ = None

            # No stacktrace found for following nodes
            _set_grad_enabled_1 = torch._C._set_grad_enabled(True);  _set_grad_enabled_1 = None
            return mul

HF's Qwen 2 model added in https://github.com/Lightning-AI/lightning-thunder/pull/1406 creates a small Inductor region because Qwen2RotaryEmbedding.forward is decorated with a torch.no_grad.

Sending no_grad to Thunder was disabled in https://github.com/Lightning-AI/lightning-thunder/pull/1282 for a good reason (https://github.com/Lightning-AI/lightning-thunder/issues/1219). Maybe it's time to properly support the no_grad context in Thunder?


Fix SM4-XTS build failure on Mac mini M1

Fix https://github.com/openssl/openssl/pull/19914#issuecomment-1413360822

It's just the change of the way to load const, it looks good to me.

This pull request is ready to merge

Close/open to kick the CIs into another build

Close/open to kick the CIs into another build

Now we can see the M1 in action...


Op for scaled attention

Description
Operator for scaled attention
Checklist
Essentials
Please feel free to remove inapplicable items for your PR.

[ ] The PR title starts with [MXNET-$JIRA_ID], where $JIRA_ID refers to the relevant JIRA issue created (except PRs with tiny changes)
[x] Changes are complete (i.e. I finished coding on this PR)
[x] All changes have test coverage:
Unit tests are added for small changes to verify correctness (e.g. adding a new operator)
Nightly tests are added for complicated/long-running ones (e.g. changing distributed kvstore)
Build tests will be added for build configuration changes (e.g. adding a new build option with NCCL)
[x] Code is well-documented:
For user-facing API changes, API doc string has been updated.
For new C++ functions in header files, their functionalities and arguments are documented.
For new examples, README.md is added to explain the what the example does, the source of the dataset, expected performance on test set and reference to the original paper if applicable
Check the API doc at http://mxnet-ci-doc.s3-accelerate.dualstack.amazonaws.com/PR-$PR_ID/$BUILD_ID/index.html
[x] To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change

Changes

[x] div_sqrt_dim, tests, (and when applicable, API doc)

Comments

If this change is a backward incompatible change, why must this change be made.
Interesting edge cases to note here

I think it's better to have an op like get_shape and do
x = x/mx.sym.get_shape(x)[-1]


lint: inconsistent fenced code block -line indentation

A problem after implementing #372 is that some lines inside md fenced code blocks starting with - get indented for unknown reasons by Prettier.
E.g. by finding the regex ^- cache in all files we can see that static/docs/commands-reference/commit.md has the unindented lines:

Similar cases in static/docs/get-started/connect-code-and-data.md and static/docs/tutorial/define-ml-pipeline.md

However static/docs/commands-reference/add.md has them indented:

I don't see a clear pattern explaining this apparently erratic behavior but it's all over the docs.
Again, this i work of Prettier which has been run on all md files in #512. None of the DVC-file samples had indentation before - lines originally. We decided that the indentation was OK in https://github.com/iterative/dvc.org/issues/512#issuecomment-518878493 but the fact that it indents some and unindents others troubles me a little. Perhaps we should use <!-- prettier-ignore -->s before all YAML code blocks after all?
All these code samples are still valid YAML though so please just close this issue if the decision is to live with it and let prettier do whatever it wants 😋

Originally reported partially as lint: don't indent fenced code block contents

I would actually create a ticket on the Prettier's repo to be honest. For us it does not look very important to include special instructions.

Closing this for now :)

Good call, I started opening an issue at https://github.com/prettier/prettier/issues/new?template=formatting.md but in the process I discovered why this is happening!
 Prettier tries to run the YAML linter inside, but simply fails to parse them with error message SyntaxError: Map keys must be unique; (See this playground).
Should we be worried that some of our DVC-file aren't technically valid YAML @shcheklein ? More of a Q for the core repo but asking here anyway.

there is a bug in docs - in the first example, third md5 from the top should not be indented at all. It's a top-level md5.

reopening it to check the docs and linting for this bug

Hmm I see. OK I fixed that in 5f13d5c but what about the other 2 examples I mentioned?
Here's a playground for static/docs/get-started/connect-code-and-data.md. In this case it complaints that SyntaxError: Document is not valid YAML (bad indentation?) (5:1)
And in this playground for static/docs/tutorial/define-ml-pipeline.md quite simply since the block langauge is ```dvc (unknown to Prettier) it ignores it. I can fix that by making sure all YAML samples are in '''yaml language blocks.

The first one should be like this:
cmd: python src/prepare.py data/data.xml
deps:
- md5: b4801c88a83f3bf5024c19a942993a48
  path: src/prepare.py
- md5: a304afb96060aad90176268345e10355
  path: data/data.xml
md5: c3a73109be6c186b9d72e714bcedaddb
outs:
- cache: true
  md5: 6836f797f3924fb46fcfd6b9f6aa6416.dir
  metric: false
  path: data/prepared
wdir: .

or like this:
cmd: python src/prepare.py data/data.xml
deps:
  - md5: b4801c88a83f3bf5024c19a942993a48
    path: src/prepare.py
  - md5: a304afb96060aad90176268345e10355
    path: data/data.xml
md5: c3a73109be6c186b9d72e714bcedaddb
outs:
  - cache: true
    md5: 6836f797f3924fb46fcfd6b9f6aa6416.dir
    metric: false
    path: data/prepared
wdir: .

Great. Yeah, Prettier forces it into the 2nd form you provided. Fixed all in b5cd910


Log is too flatten to understand.  All the operations seems are for repository, but some of them are for project.

I agree this implementation is adhoc, Ideally the log should be "activity log" and be broken down into  different tables for better tracking and display.

Can we add a target column which would identify what object the activity is for? can the filter show just activity for projects or repos or all? would this help for this release?


[BUG] AssertionError: Unable to pre-compile ops without torch installed. Please install torch before attempting to pre-compile ops.

When I run pip install deepspeed on my Windows 11 machine in a virtual environment with Python 3.12.4, PyTorch 2.4.1 and CUDA 12.4 installed, I get the below error:
Collecting deepspeed
  Downloading deepspeed-0.15.1.tar.gz (1.4 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.4/1.4 MB 8.7 MB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [23 lines of output]
      [WARNING] Unable to import torch, pre-compiling ops will be disabled. Please visit https://pytorch.org/ to see how to properly install torch on your system.
       [WARNING]  unable to import torch, please install it if you want to pre-compile any deepspeed ops.
      DS_BUILD_OPS=1
      Traceback (most recent call last):
        File "D:\[REDACTED]\[REDACTED]\.venv\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 353, in <module>
          main()
        File "D:\[REDACTED]\[REDACTED]\.venv\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "D:\[REDACTED]\[REDACTED]\.venv\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
                 ^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\[REDACTED]\AppData\Local\Temp\pip-build-env-bsf_1y1b\overlay\Lib\site-packages\setuptools\build_meta.py", line 332, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=[])
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\[REDACTED]\AppData\Local\Temp\pip-build-env-bsf_1y1b\overlay\Lib\site-packages\setuptools\build_meta.py", line 302, in _get_build_requires
          self.run_setup()
        File "C:\Users\[REDACTED]\AppData\Local\Temp\pip-build-env-bsf_1y1b\overlay\Lib\site-packages\setuptools\build_meta.py", line 503, in run_setup
          super().run_setup(setup_script=setup_script)
        File "C:\Users\[REDACTED]\AppData\Local\Temp\pip-build-env-bsf_1y1b\overlay\Lib\site-packages\setuptools\build_meta.py", line 318, in run_setup
          exec(code, locals())
        File "<string>", line 155, in <module>
      AssertionError: Unable to pre-compile ops without torch installed. Please install torch before attempting to pre-compile ops.
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

[notice] A new release of pip is available: 24.0 -> 24.2
[notice] To update, run: python.exe -m pip install --upgrade pip

@loadams

Hi @umarbutler - looks like you're on Windows.  The wheel we publish on Windows has a specific set of requirements, and it looks like you are pulling down the linux wheel.
See the Windows version here, you may need to modify your python version to get it to grab this
https://pypi.org/project/deepspeed/0.15.0/#files

Hi @umarbutler - did that resolve your issue?


Feature/home page text

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: Constantino sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.


Allow optional properties in actor and item data for updates

I'm not 100 % sure about this.
I describe the problem first: If there is an optional property in ActorData.data or ItemData.data that has an object type, it is not possible to update this object partially.
If I'm right, there are can not be optional properties defined through template.json (and in fact it was just mistyped as optional in my code).
But I guess this should be fixed anyway, especially because DeepPartial is used not exclusively on actor and item data.

Is this still relevant?


Error Runing In Pypy3 "TypeError: expected c_char_p instance instead of int"

Use Linux(centos7), Runing In Pypy3,  An exception occurs when I run the following code
import joblib

lgb = joblib.load('./catigory_xh.pkl')

for k in lgb.feature_name():
    print(k);

Error Info
Traceback (most recent call last):
  File "/usr/local/src/pypy3.6/lib_pypy/_ctypes/basics.py", line 56, in from_param
    as_parameter = value._as_parameter_
AttributeError: 'int' object has no attribute '_as_parameter_'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "model.py", line 5, in <module>
    for k in lgb.feature_name():
  File "/usr/local/src/pypy3.6/site-packages/lightgbm/basic.py", line 3655, in feature_name
    ptr_string_buffers = (ctypes.c_char_p * num_feature)(*map(ctypes.addressof, string_buffers))
  File "/usr/local/src/pypy3.6/lib_pypy/_ctypes/array.py", line 203, in __init__
    self[i] = arg
  File "/usr/local/src/pypy3.6/lib_pypy/_ctypes/array.py", line 229, in __setitem__
    cobj = self._type_.from_param(value)
  File "/usr/local/src/pypy3.6/lib_pypy/_ctypes/primitive.py", line 372, in from_param
    return super(SimpleType, self).from_param(value)
  File "/usr/local/src/pypy3.6/lib_pypy/_ctypes/basics.py", line 59, in from_param
    self.__name__, type(value).__name__))
TypeError: expected c_char_p instance instead of int

LightGBM version:
Name: lightgbm
Version: 3.3.1
Summary: LightGBM Python Package
Home-page: https://github.com/microsoft/LightGBM
Author:
Author-email:
License: The MIT License (Microsoft)
Location: /usr/local/src/pypy3.6/site-packages
Requires: numpy, scikit-learn, scipy, wheel
Required-by:

Pypy Version:
Python 3.6.12 (db1e853f94de, Nov 18 2020, 09:49:19)
[PyPy 7.3.3 with GCC 7.3.1 20180303 (Red Hat 7.3.1-5)]

In fact, I have tried in other versions, and the error is the same
catigory_xh.pkl

Thanks very much for using LightGBM! This project doesn't currently have any tests using alternative implementations of Python like Pypy, so incompatibilities are definitely possible.
We'd like to help, but need some additional information to help reproduce the issues you're seeing. Please "How to create a minimal, reproducible example" (Stack Overflow docs) for some guidance on this.
I attempted to create a reproducible example tonight using the official pypy docker images, but found that it was taking too long as scipy had to be built from source in containers using these images (at least the ones I tried). You can see the code below for a starting point.
what I tried (click me)
docker run \
    --rm \
    --workdir /opt \
    --entrypoint="" \
    -it pypy:3.7 \
    /bin/bash

# get system dependencies
# NOTE: need openblas and gfortran for scipy
apt-get update -y
apt-get install -y \
    build-essential \
    cmake \
    gfortran \
    libopenblas-dev

# check versions
cmake --version
gcc --version
git --version
pypy --version

# cmake version 3.18.4
# gcc (Debian 10.2.1-6) 10.2.1 20210110
# git version 2.30.2
# Python 3.7.12 (44db26267d0a, Oct 24 2021, 14:21:50)
# [PyPy 7.3.7 with GCC 7.3.1 20180303 (Red Hat 7.3.1-5)]

# install lightgbm's dependencies
# NOTE: numpy.distutils.system_info.NotFoundError: No lapack/blas resources found.
pypy -m pip install \
    --no-cache-dir \
    numpy \
    scikit-learn \
    scipy

# install LightGBM 3.3.1 with pypy
git clone \
    --depth 1 \
    --recursive \
    -b v3.3.1 \
    https://github.com/microsoft/LightGBM.git

cd LightGBM/python-package
pypy setup.py install

# confirm that pypy can see load LightGBM and its dependencies
pypy -c "import lightgbm"

Please also note that maintainers here (and in other projects) are unlikely to be willing to directly load a pickle file you provide. From the pickle docs

It is possible to construct malicious pickle data which will execute arbitrary code during unpickling. Never unpickle data that could have come from an untrusted source, or that could have been tampered with.


Issue using latest version with Sinatra

/gems/backports-3.6.4/lib/backports/std_lib.rb:9:in require': cannot load such file -- rails/railtie (LoadError) gems/backports-3.6.4/lib/backports/std_lib.rb:9:in require_with_backports'
gems/activesupport-4.2.2/lib/active_support/dependencies.rb:274:in block in require' gems/activesupport-4.2.2/lib/active_support/dependencies.rb:240:in load_dependency'
gems/activesupport-4.2.2/lib/active_support/dependencies.rb:274:in require' gems/google-api-client-0.8.6/lib/google/api_client/railtie.rb:1:in <top (required)>'
Would be nice if this line would be only required if Rails is defined
require 'rails/railtie'

Can this be fixed please?

this was happening with activesupport 4.2.2 and i was able to fix it by adding to Gemfile, the gem 'railties' and i had to require before this gem
require 'rails/all'

and that seems to do the trick.  I can close this issue.


CouchDB 2.3 purge_seq transition to "string" from "number" breaks Java client(s)

According to the docs, purge_seq is still reported as a "number". Reality has it - it's now ( and probably other fields as well) are now "strings". This is probably related to the "Clustered purge" feature. IBM's cloudant-java client is currently the only maintained java client for couchdb, and it cannot properly work with db-info queries (by the look of the stacktrace, it also affects the lightcouch driver, meaning both of these drivers will fail to work.
This leaves the couchdb-java ecosystem in a problematic state for 2.3 and future releases.
While I opened this issue for the java driver, it will best to update the documentation for 2.3 and above release. and perhaps cover additional breaking changes.

As this will be resolved by a documentation patch, moving this to apache/couchdb-documentation.

@wohali The change of purge_seq from JSON integer to JSON string also broke our CouchDB clients when we upgraded to CouchDB 2.3. Please mention the change in the v2.3 release notes.

hey @denyeart, the change was described via commit https://github.com/apache/couchdb-documentation/commit/cc9feb13068e63332c9a1e81de95f17b31ef12c1#diff-6b4a3e899b13e42489f90830ed0a4c25R77. Is it not enough?

@jiangphcn I don't think it is enough. Databases must not introduce incompatible API changes in a minor release that break existing applications.  And if they do so, it should be mentioned in release notes / upgrade notes so that people upgrading know about it.

We can't go back and update the 2.3 release notes now that the label has been placed on the repository, I'm sorry. It's an ASF limitation.


Iwd says disconnected then it connected

network module cant see if iwd connected to the net for some reason. waybar 0.10.4-2

forgot to say, idk means disconnected

this was my issue, sorry


Added test for TraceCommand together with potential fix

@dsyer , @spencergibb - WDYT?

LGTM. Why doesn't it suffer from the same problem as the TraceCallable and TraceRunnable (#61) - we don't detach the parent trace from the thread at the end of the command?

I've added a test and it's working thanks to the changes in the close method. But actually there would be the very same issue as in [#61]

Done in https://github.com/spring-cloud/spring-cloud-sleuth/commit/c1eb04b68aa3cabfe34fae6699e68d8f29055ac3


Cache not cleared after native-craft edit of image asset

When editing an image assets' crop, rotation etc. via the native Craft tools, the cache is not invalidated. Even after manually using 'Clear Imager transforms' it keeps the cached version.
It will only update after a full imager-cache wipe.

Still works for me, can you please provide some more information about your setup. Which platform? What versions of PHP, Craft and Imager? Which transformer are you using? Where do you store your assets?

It happens on both local Valet and on production. Both behind nginx 1.15. php 7.3.x. I have updated Craft and your plugin to the latest versions as of writing (3.1.27 and 2.1.10).
I'm now also using your preparse field plugin, to create transforms on save. But this also won't invalidate the cache with the (for example) cropped image.
If you want I could grab a screenrecording if you want, but only privately.

Same here, cropped an image in Craft but it isn't updating (even after clearing all possible caches).
My answers for the questions above are:

both local (valet) and production nginx/1.15.6 and php 7.2.12
Craft 3.1.5 and Imager 2.1.4
imager is imgix
images are stored in a default Craft way

Revisited this, and I'm still not able to reproduce.
Was thinking that maybe it was a race condition with preparse (if it's on the asset itself), but it doesn't make sense that the Clear Imager transforms wouldn't work either.
Did you check the folder on your disk after doing Clear Imager transforms? Did the files get recreated or did nothing happen? Also, there's a fair amount of logging in those methods, did you check your logs?


stable/grafana helm chart job to inject data source is broken

Is this a request for help?:
No
Is this a BUG REPORT or FEATURE REQUEST? (choose one):
Bug report.
Version of Helm and Kubernetes:
helm 2.7.2
kubernetes 1.9.3
Which chart:
stable/grafana
What happened:
The changes done on stable/grafana/templates/_helpers.tpl (see commit https://github.com/kubernetes/charts/commit/69a3d76d14a8dddfca11418daadc871e41ea45ad#diff-fc31bc5a3c35dc9e17100f97040adfd6) (https://github.com/kubernetes/charts/pull/4062) to align the update grafana fullname helper to follow standard broke the master/stable/grafana/templates/job.yaml template.
From https://github.com/kubernetes/charts/blob/master/stable/grafana/templates/job.yaml#L40
          - "http://$(ADMIN_USER):$(ADMIN_PASSWORD)@{{ template "grafana.fullname" . }}:{{ .Values.server.service.httpPort }}/api/datasources"
The job will use the above URL to inject the datasource when server.setDatasource.enabled is true.  The problem is that the "template "grafana.fullname" . " after the refactoring is not aligned with template "grafana.server.fullname" .  anymore and the service name is defined as template "grafana.server.fullname" .
The result is that, when  server.setDatasource.enabled is set to true, that the url will point to the wrong endpoint.
What you expected to happen:
In https://github.com/kubernetes/charts/blob/master/stable/grafana/templates/job.yaml#L40
          - "http://$(ADMIN_USER):$(ADMIN_PASSWORD)@{{ template "grafana.fullname" . }}:{{ .Values.server.service.httpPort }}/api/datasources", the url should be aligned with the service name definition for grafana server.
How to reproduce it (as minimally and precisely as possible):
Deploy chart with:
--set server.setDatasource.enabled=true
Anything else we need to know:
No.

/close


Make Container constructor constant

var x = const Container();

I will implement this change myself and create a pull request. 🙂

pull request was closed, so closing this issue.


Figure out a better way to handle window focus

Right now the focus settings are a mess. When the app launches I had to blur the focus, which is annoying, but this was an attempt to fix an annoying setup where every hot module load would re focus the app. This was also annoying.

Establish a  component as a default on the app. When you first load the React App you should just see a loading spinner.

This is much simpler now and the hmr problem isn't an issue any longer.


[ADAM-776] Rename duplicate OUTPUT metaVar in ADAM2Fastq.

Resolves #776.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1358/
Test PASSed.

+1

Merged commit 9c72620d006ee118125f8476cf8e4632618ac4ca
Thank you, @fnothaft!


How do you even go about configuring this?

Hi im on arch and have an apex pro 5 keyboard with the id of 1038:161c. I tried to follow your instructions for installing apex-tux through Udev and also tried the apex-tux-git aur package, im stuck on step 2 when it comes to "Enter the following data from here". What do you mean by here? this just links me to a python script where there is seemingly no spot to enter my keyboard's ID. I also looked at the issues section and have the same error about the settings file missing upon running apex-tux or sudo apex-tux. This whole thing feels poorly written and rather cryptic trying to understand even the converstations going on in the issues section. You guys keep reffering to a settings.toml file needing to be copied to the target /release folder or asking if they are in the current directory. What is the current directory? where is the target/release folder and why is this all so poorly explained and cryptic? this almost works just I need to get past the whole your settings.toml is missing.

If you're stuck at step 2, you shouldn't worry about the settings.toml for now, what you should do is skip the python script (while it works for some, it doesn't for other), and should create /etc/udev/rules.d/97-steelseries.rules and put
SUBSYSTEM=="input", GROUP="input", MODE="0666"

SUBSYSTEM=="usb", ATTRS{idVendor}=="1038", ATTRS{idProduct}=="161c", MODE="0666", GROUP="plugdev"
KERNEL=="hidraw*", ATTRS{idVendor}=="1038", ATTRS{idProduct}=="161c", MODE="0666", GROUP="plugdev"

after that, reload the udev rules / restart
and then you should worry about compiling it / figuring where the settings.toml is
however, I do agree that the installation is quite a ride, I will try to rewrite it, but english is not my main language

Thank you for getting back to me, I'll try the steps you listed also sorry if I came off a bit harsh I just felt like I was misunderstanding something that was simple for the others in the github post.  I hadn't considered the dev might not have English as thier first language.

I got it working, thank you for getting back to me in such a short time!


Support for Aliyun OSS as a storage backend

This pull request depends on https://github.com/livekit/protocol/pull/234 .
@davidzhao , please help to review the two pull requests.
Thanks.

The related issue is: https://github.com/livekit/egress/issues/114

hey @tomxiong, can you update protocol to 41f5b55c9507c061d4b3c45d80aefccf915ae388

@frostbyte73 ， Done. Please help to review.

@frostbyte73 , I update the params.go as you mentioned. Please help to review again.

@frostbyte73 , I update the params.go as you mentioned. Please help to review again.

it looks like the changes haven't been pushed yet

@frostbyte73 , sorry about that, I push code.

Done @frostbyte73


Doesn't work with Faraday 0.7.6

It is tested and works with 0.8.8 and 0.9.0

Error:
Speechmatics::Client#test_0001_lists jobs:
NameError: wrong constant name mashify
/Users/andrew/.rbenv/versions/2.2.1/lib/ruby/gems/2.2.0/gems/faraday-0.7.6/lib/faraday.rb:32:in const_get' /Users/andrew/.rbenv/versions/2.2.1/lib/ruby/gems/2.2.0/gems/faraday-0.7.6/lib/faraday.rb:32:inlookup_module'
/Users/andrew/.rbenv/versions/2.2.1/lib/ruby/gems/2.2.0/gems/faraday-0.7.6/lib/faraday/builder.rb:151:in use_symbol' /Users/andrew/.rbenv/versions/2.2.1/lib/ruby/gems/2.2.0/gems/faraday-0.7.6/lib/faraday/builder.rb:107:inresponse'
/Users/andrew/dev/projects/speechmatics/lib/speechmatics/connection.rb:43:in block in connection' /Users/andrew/.rbenv/versions/2.2.1/lib/ruby/gems/2.2.0/gems/faraday-0.7.6/lib/faraday/connection.rb:46:inblock in initialize'
/Users/andrew/.rbenv/versions/2.2.1/lib/ruby/gems/2.2.0/gems/faraday-0.7.6/lib/faraday/builder.rb:12:in block in create' /Users/andrew/.rbenv/versions/2.2.1/lib/ruby/gems/2.2.0/gems/faraday-0.7.6/lib/faraday/builder.rb:65:inbuild'
/Users/andrew/.rbenv/versions/2.2.1/lib/ruby/gems/2.2.0/gems/faraday-0.7.6/lib/faraday/builder.rb:54:in initialize' /Users/andrew/.rbenv/versions/2.2.1/lib/ruby/gems/2.2.0/gems/faraday-0.7.6/lib/faraday/builder.rb:12:innew'
/Users/andrew/.rbenv/versions/2.2.1/lib/ruby/gems/2.2.0/gems/faraday-0.7.6/lib/faraday/builder.rb:12:in create' /Users/andrew/.rbenv/versions/2.2.1/lib/ruby/gems/2.2.0/gems/faraday-0.7.6/lib/faraday/connection.rb:46:ininitialize'
/Users/andrew/dev/projects/speechmatics/lib/speechmatics/connection.rb:39:in new' /Users/andrew/dev/projects/speechmatics/lib/speechmatics/connection.rb:39:inconnection'
/Users/andrew/dev/projects/speechmatics/lib/speechmatics/api.rb:39:in request' /Users/andrew/dev/projects/speechmatics/lib/speechmatics/api.rb:68:inlist'
/Users/andrew/dev/projects/speechmatics/test/user/jobs_test.rb:91:in `block (2 levels) in <top (required)>'

addressed in a0c272fb32e6405af63546a24b4c9134bb77d957


End to End Flow

Modify the users' application code

Users' application implements logic to get the users' consent to collect opt-in telemetry.
If the users' application is using Golang, they can import our provide client package. The client package is safe for calling concurrently by multiple goroutines. It is also safe to call start() or stop() on the same client object multiple times. Calling start() on already started client or calling stop() on already stopped() client are no-op. Application can call start() and stop() the Binoculars client when users' consent is changed.
User's application implements the MetricHandler interface and passes it into the Binoculars client so the Binoculars client knows how to gather metrics and process the response. See example package for an example
The Binoculars client starts sending updated metrics to the Binoculars server in the following JSON format:[
    {
        "name": "storage_capacity",
        "tags": {"instance_uuid": "123e4567-e89b-12d3-a456-426614174000"},
        "fields": {"value":<PHONE_NUMBER>000}
    },
    {
        "name": "enabled_feature",
        "tags": {"instance_uuid": "123e4567-e89b-12d3-a456-426614174000"},
        "fields": {"node_selector": 1, "data_locality": 1}
    }
]

The Binoculars client will receive a response from the Binoculars server that contains the time duration that the client should send the next metric update to the server, RequestIntervalInMinutes.

Binoculars server

Binoculars server creates a database with the name APPLICATION_NAME_binoculars in the InfluxDB instance to store all related telemetrics for the users' application.

Parse the metric request
The Binoculars server receive the metrics in the following JSON format
[
    {
        "name": "storage_capacity",
        "tags": {"instance_uuid": "123e4567-e89b-12d3-a456-426614174000"},
        "fields": {"value":<PHONE_NUMBER>000}
    },
    {
        "name": "enabled_feature",
        "tags": {"instance_uuid": "123e4567-e89b-12d3-a456-426614174000"},
        "fields": {"node_selector": 1, "data_locality": 1}
    }
]

The server parses the metrics and creates a point for each metric in the JSON request's body as follows:

name is the name of measurement (table) of the point in the InfluxDB
tags are tags of the point in the InfluxDB
fields are fields of the point in the InfluxDB

The server then stores the points to the InfluxDB. With the above example request, the server stores 2 points in the InfluxDB:
> select * from  storage_capacity
name: storage_capacity
time                instance_uuid                        value
----                -------------                        -----
1622017251731519223 123e4567-e89b-12d3-a456-426614174000<PHONE_NUMBER>000

> select * from enabled_feature
name: enabled_feature
time                auto_salvage data_locality instance_uuid
----                ------------ ------------- -------------
1622017251731530324 1            1             123e4567-e89b-12d3-a456-426614174000

The server returns the time duration that it wants the client to send the next metric update, RequestIntervalInMinutes, in the response's body.

InfluxDB queries
Now that we have the raw data in the storage_capacity and enabled_feature measurements, it is time for the application developers to use their InfluxDB expertise to get useful information out of the raw data.
Let's take a look at the enabled_feature metric. The goal of this metric is so that the application developers can know which features of the application are most used and which features are not used so the developers can plan their development plan. In this example, assume that the query-period is 1 hour so the Binoculars server is getting the updated metric from the client every hour.
First, create a continuous query that counts the number of each enabled feature every 1h
CREATE CONTINUOUS QUERY cq_enabled_feature_downsampling ON <APPLICATION_NAME>_binoculars
BEGIN
SELECT sum(*) INTO enabled_feature_downsampling
FROM enabled_feature
GROUP BY time(1h)
END

The following measurement is created in the InfluxDB (note that I changed the grouping time to 5 mins to save testing time)
> select * from enabled_feature_downsampling
name: enabled_feature_downsampling
time                sum_auto_salvage sum_data_locality sum_node_selector sum_toleration
----                ---------------- ----------------- ----------------- --------------
1621971600000000000                  1
1621971900000000000 3                4                 3                 1
1621973400000000000 2                6                                   4
1621978500000000000 4                8                                   3
1621978800000000000 6                9                                   8
1621979100000000000 6                6                                   6
1621979400000000000 11               14                                  5

Grafana panels

Create a data source that points to the database <APPLICATION_NAME>_binoculars in the InfluxDB instance.
Continue on the enabled_feature metric from the previous section. In Grafana, the application developers can use this query to create a panel from the enabled_feature_downsampling (Note that I actually group by 5m to save testing time):SELECT last(*) FROM "enabled_feature_downsampling" WHERE $timeFilter GROUP BY time(1h) fill(null)

What are the differences between tag and field? By defining those in Metric we're tying ourselves to Influxdb (which is also a valid option).

@yasker
tag are indexed in the InfluxDB for faster lookup and grouping while fields are not. tags are usually used for things that have many different values (e.g, the UUID) so that searching doesn't require scanning over the whole database.
defining tag and field doesn't tie us to InfluxDB since we can implement the PostgresSQL database client to convert all tags and fields into regular columns when we store data to the database. User can then manually create indexes on the tags column` inside PostgresSQL

I meant the user needs to know the differences, and they need to know there must be at least one field (IIRC). Those concepts are tie to InfluxDB. We can still switch to another DB later, but users always need to follow the InfluxDB concept for tag vs field, no matter what's the backend DB is.
It's not a major issue. As long as we explain what is a tag and what is a field and how users should choose between the two for things they want to record, that would be fine.

@yasker
I see. Thank you! I will add a note to the README.md when I create it.

The top level object you send should be a map.  If you need to add some more info later you're in for a bad time. {"metrics": [ ... ]}
...for example, tags/install uuid seems like it could be sent once at the root instead of duplicating with every data point.  So you could add a top-level "tags" map that applies to every metric.  But if your top-level is an array, you're just stuck because there's nowhere to put it and will have to break the API.
Think about data types, are fields values always an integers or can they be sent as strings or bools, etc?  I don't know what I'm allowed to send.  Handling very large numbers (> 2^52) in JSON is error-prone/implementation-specific.  If "storage capacity" is in bytes that's "only" 4,500 TB (half a rack of backblaze storage pods) before some clients can't accurately represent it.  But losing a bit of accuracy for a metric is probably not a critical problem unless you try to compare/find exact data points.
When should I (as a person adding this to my product) send multiple top-level entries in the array versus one entry with multiple "fields"?  Assume I (a person reading the docs) know and care nothing about Influx and how it works.
Same for "tags", when do I want a tag vs a field?  Calling them "labels" might make more sense to our k8s-centric crowd.
For rancher we often look at the min/mean/median/max value for a metric, to get an idea of the distribution for how a feature is used.  I assume you can calculate that out of influx somehow.

Thanks @vincent99!

Yes, I will make the top level object a map. Great point!
Data types can be: string, bool, float 64-bit. (It is possible to support int64 but I am not sure if it is needed). Yeah, it is up to the app developer to choose the unit for fields. As you mentioned, if "storage capacity" can be more than 4500TB, app developer might want to consider using MB as unit instead of bytes.
App developer need to know how InfluxQL works since they will design the measurement ( the table) schema in InfluxDB. They will also be the one who design continuous queries to extract useful information out of the raw measurement. So, knowing InfluxQL is required. But sure, I will have some document and example about how to choose between tags/fields; multiple metrics or a single metric with multiple fields.
Yes, we can calculate basic stats min/max/mean/median using InfluxQL https://docs.influxdata.com/influxdb/v1.8/query_language/functions/

user's application will create its own random instance_uuid and maintain it?

@tybalex Yeah
In Longhorn, we are thinking of using namespace UUID where Longhorn is install to be the UUID of the Longhorn instance.


Tree shaking seems not working.

I use core-decorators in create-react-app project, and use function in this way:
import {autobind} from 'core-decorators';

But the bundle size doesn't reduce.
When I change in this way:
import autobind from 'core-decorators/lib/autobind;'

The bundle size seems good to me.
Did I misunderstanding tree-shaking?
Let me know if you need more infomation.

And rollup doesn't seem to work for me, too.
config:
import resolve from 'rollup-plugin-node-resolve';

export default {
    input: 'index.js',
    output: {
        file: 'test.js',
        format: 'umd',
        name: 'yo',
    },
    plugins: [
        resolve({
            jsnext: true,
            main: true,
            browser: true
        })
    ]
}

index.js:
import {autobind} from 'core-decorators'

export default autobind

The output is the whole deps.

hmm if it works in rollup, but not webpack2, that seems to suggest webpack2 is the culprit. If you have time to make an example I can use to reproduce I'll take a closer look?
This might be the issue: https://github.com/facebookincubator/create-react-app/issues/2748 and https://github.com/webpack/webpack/issues/2867

Yeah, it is really easy to reproduce. Run all these lines:
create-react-app react-demo
cd react-demo
npm install core-decorators --save

App.js:
...
import {autobind} from 'core-decorators';
...

Then run npm run build, see the bundle size:

Edit App.js:
...
import autobind from 'core-decorators/lib/autobind';
...

Then run npm run build, see the bundle size:

I solve this issue by using babel-plugin-transform-imports.

Any news on this?


HotChocolate fails to deserialize a list of ID types.

Is there an existing issue for this?

[X] I have searched the existing issues

Describe the bug
HotChocolate is unable to deserialize a ListType<NonNullType<IdType>> field the way it deserializes a simple NonNullType<IdType>.
I've created a repository where you can test this out.
Steps to reproduce

Run the project
Do the following GraphQL request: mutation { doTest(ids: ["VGVzdAppMQ==", "VGVzdAppMg=="]) }
Get slammed with an error, when you shouldn't.

Relevant log output
No response
Additional Context?
The closest thing to a workaround is doing all the heavy-lifting yourself, but it's not perfect:
public static class IdSerializerExtensions
    {
        public static IEnumerable<TKey> DeserializeIds<TKey>(this IIdSerializer serializer, IReadOnlyList<IValueNode> ids)
        {
            for (int i = 0; i < ids.Count; i++)
            {
                yield return (TKey)serializer.Deserialize((string)ids[i].Value).Value;
            }
        }
    }

public class SampleResolver
{
	public bool ExampleResolverMethod(
	   IResolverContext context,
	   IIdSerializer idSerializer,
	   IExampleService service)
	{
		// The solution is not perfect because this line can throw an error
		// if the end-user has supplied a different type of argument with the same name.
		var argument = context.ArgumentLiteral<ListValueNode>("<argumentName>");

		return service.DoExampleWork(idSerializer.DeserializeIds<int>(argument.Items));
	}
}

Product
Hot Chocolate
Version
13.0.0

Another workaround would be to modify your resolver :
       descriptor.Field("doTest")             .Argument("ids", descriptor => descriptor.Type<NonNullType<ListType<NonNullType<IdType>>>>())             .Type<BooleanType>()             .Resolve(context =>             {                 var ids = context.ArgumentValue<IEnumerable<object>>("ids");                 return new Resolver().DoTests(ids.ToArray());             });

@Nivisan-Jeyananthan adding object[] ids still fails like before with the error message The IDs 'XXXXXX' have an invalid format.

@Nivisan-Jeyananthan was this issue solved? HotChocolate is unable to automatically do what it does for a single ID parameter.
There are workarounds, but they all involve working with the raw payload, meaning you have to:

Use an IIdSerializer to deserialize the strings.
Check if the deserialized id is of the type you are expecting and handle exceptions if deserialization straight-out fails.
Hard-code the error message the framework usually returns for invalid id's (The IDs `XXXXXX, YYYYYY` have an invalid format.) so there are no inconsistencies with how it handles invalid id's and deal with IResolverContext to report it.

I don't think this is desirable? Especially the part where you have to copy an error that can at any point in the future change.

#5160

@SpiritBob sorry, thought your problem was fixed with this.


Routes with children cannot have a name property.

This should resolve nuxt-community/nuxt-i18n#98
Sets route name before checking for children, and if children exist, delete the name property from the route.
cc @htchtc052

Thanks @elrolito !

Glad I could contribute.


[BUG] Run BlockBlobClient.QueryAsync() on a big csv file, then write the output stream to a file fail.

Describe the bug
Run BlockBlobClient.QueryAsync() on a big csv file (~230MB), then write the output stream to a file fail with  "Destination array was not long enough. Check the destination index, length, and the array's lower bounds."
And it seems the “ProgressHandler” won’t be called until the request finish, and start to read from output stream. I expect it will be invoke on download the result from server. Any comments?
Expected behavior
write the stream to a local file should success.
Actual behavior (include Exception or Stack Trace)
Failed with
System.ArgumentException: Destination array was not long enough. Check the destination index, length, and the array's lower bounds.
Parameter name: destinationArray
   at System.Array.Copy(Array sourceArray, Int32 sourceIndex, Array destinationArray, Int32 destinationIndex, Int32 length, Boolean reliable)
   at Azure.Storage.Blobs.BlobQuickQueryStream.ReadInternal(Boolean async, Byte[] buffer, Int32 offset, Int32 count)
   at Azure.Storage.Blobs.BlobQuickQueryStream.Read(Byte[] buffer, Int32 offset, Int32 count)
   at System.IO.Stream.CopyTo(Stream destination, Int32 bufferSize)
   at testTracking.Program.Main(String[] args) in C:\Users\weiwei\source\repos\testTracking\testTracking\Program.cs:line 74

To Reproduce
Steps to reproduce the behavior (include a code snippet, screenshot, or any additional information that might help us reproduce the issue)
Following is my repro program, I will share my test csv file Uri by mail.
The failure is on line reader.CopyTo(fs);
           string bloburi = "https://xxx.blob.core.windows.net/weiqatest/testcsv.csv?sv=2019-12-12&se=2020-12-29T16%3A00%3A00Z&sr=b&sp=r&sig=xxx";
           BlockBlobClient blob = new BlockBlobClient(new Uri(bloburi), new BlobClientOptions());

            // prepare query Option
            IProgress<long> progressHandler = new Progress<long>((finishedBytes) =>
            {
                Console.WriteLine(finishedBytes);
            });
            BlobQueryOptions queryOption = new BlobQueryOptions
            {
                InputTextConfiguration = new BlobQueryCsvTextConfiguration() { HasHeaders = true },
                OutputTextConfiguration = new BlobQueryCsvTextConfiguration() { HasHeaders = true },
                ProgressHandler = progressHandler
            };

            string query = "SELECT * FROM BlobStorage WHERE _1 LIKE '1%%'";
            Task<Response<BlobDownloadInfo>>  task =blob.QueryAsync(query, queryOption);
            task.Wait();

            var reader = task.Result.Value.Content;
            FileStream fs = File.Create(@"C:\temp\QA_result.csv");
            reader.CopyTo(fs);
            fs.Close();
           reader.Close();

Environment:

Name and version of the Library package used: [e.g. Azure.Storage.Blobs 12.4.0]
Azure.Storage.Blobs/12.5.0-preview.5
Hosting platform or OS and .NET runtime version (dotnet --info output for .NET Core projects): [e.g. Azure AppService or Windows 10 .NET Framework 4.8]

.NET Core SDK (reflecting any global.json):
 Version:   2.2.108
 Commit:    33ed5b90ce

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.18363
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\2.2.108\

Host (useful for support):
  Version: 2.2.6
  Commit:  7dac9b1b51

.NET Core SDKs installed:
  2.2.108 [C:\Program Files\dotnet\sdk]

.NET Core runtimes installed:
  Microsoft.AspNetCore.All 2.2.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.2.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 2.2.6 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]

To install additional .NET Core runtimes or SDKs:
  https://aka.ms/dotnet-download

IDE and version : [e.g. Visual Studio 16.3]
VS 15.7.4

@blueww, unfortunately I am not able to repo.
using Azure;
using Azure.Storage.Blobs;
using Azure.Storage.Blobs.Models;
using Azure.Storage.Blobs.Specialized;
using System;
using System.IO;
using System.Threading.Tasks;

namespace ConsoleApp16
{
    class Program
    {
        static async Task Main(string[] args)
        {
            string bloburi = " https://xxx.blob.core.windows.net/weiqatest/testcsv.csv?sv=2019-12-12&se=2020-12-29T16%3A00%3A00Z&sr=b&sp=r&sig=yyy";
            BlockBlobClient blob = new BlockBlobClient(new Uri(bloburi), new BlobClientOptions());

            // prepare query Option
            IProgress<long> progressHandler = new Progress<long>((finishedBytes) =>
            {
                Console.WriteLine(finishedBytes);
            });
            BlobQueryOptions queryOption = new BlobQueryOptions
            {
                InputTextConfiguration = new BlobQueryCsvTextConfiguration() { HasHeaders = true },
                OutputTextConfiguration = new BlobQueryCsvTextConfiguration() { HasHeaders = true },
                ProgressHandler = progressHandler
            };

            string query = "SELECT * FROM BlobStorage WHERE _1 LIKE '1%%'";

            Response<BlobDownloadInfo> response = await blob.QueryAsync(query, queryOption);

            var reader = response.Value.Content;
            FileStream fs = File.Create(@"C:\temp\QA_result.csv");
            reader.CopyTo(fs);
            fs.Close();
            reader.Close();
        }
    }
}

And it seems the “ProgressHandler” won’t be called until the request finish, and start to read from output stream. I expect it will be invoke on download the result from server. Any comments?

This is by design.  The ProgressHandler is used to report progress events in the data stream.

@seankane-msft
Thanks for the explain!
For ProgressHandler, it's used to show progress to customer. So we might should use it on the item which will use most time.
In Query(), per my test, normally the time mostly be used on network request than write to local disk, so can we use ProgressHandler for progress of the network request instead of disk write, so customer can get the really progress of whole Query process. Or the customer will see the progress keeps 0% for a long time, then go to 100% very quickly.
Is there anything blocks we use the network request progress in ProgressHandler?  As I see, in other API like BlobClient.Upload(), the ProgressHandlerwork can show the correct progress.

The functionality of ProgressHandler is different for Query by design.  This was specifically specified by the Quick Query backend team.

@seanmcc-msft
Thanks for the quick fix!
For the progress handler, as it can't show the real progress of the query, I will not use it to show the progress bar in PSH, to avoid misleading user.


Fix ToggleCheckbox value

Issue
Steps to reproduce:

Set the default value of ToggleCheckbox as true
Clear the value
Change/toggle value
Value will be set to false

Fix

Clicking when value is null should change the value to true
Fixed Warning: A component is changing a controlled input to be uncontrolled. This is likely caused by the value changing from a defined to undefined, which should not happen error message

fix #645

Thank you for fixing this!
I'll leave it up to you, but what do you think about starting e2e tests on this component, because the toggle-logic becomes more and more complicated? Doesn't need to be big, but mainly testing the 3 states ...

was my first time on e2e tests, let me know if it's look good to you @gu-stav  🌞


IBX-163: Total amount for order is hardcoded to 99999.99

Added parameter to set max orderable amount

Precision has been decreased to 10