removeBoneFromList - potential error

Hi,
I ported the project to C#/Unity, it seems to run correctly, albeit very slow (around 5-6FPS if solving every frame, on a high-end laptop).
One line I had to comment out is in bold below:
public void removeFromBoneList(AbstractBone abstractBone) {
if(bones.contains(abstractBone)) {
bones.remove(abstractBone);
boneMap.remove(abstractBone);
}
}
Is this line correct? It appears to be attempting to move an element from a HashMap by value not key, which I presume will not do anything? I do very little Java, so I maybe missing something.
Out of interest, what sort of performance would you expect to get when solving every frame (I have been trying ConstraintExample_Kusudama and SimpleExample as my tests). I don't know if performance issues have been introduced in the Java -> C# process, or whether this library is meant more for offline calculations than runtime (using it for things like working out where a human skeleton is based on three pieces of tracking data for head and both hands).

The line you identified is a bug, yes (good catch!). However, that function
is for user convenience and (if I recall correctly) not called internally,
so your modification probably shouldn't be affecting performance.
The library was created for an animation program, so its default settings
prefer quality and consistency first, and speed second (aiming for
interactive rather than real time). That said it should be usable for real
time applications where processor cycles aren't at a premium (or you're
willing to sacrifice quality). If you're porting this specifically for use
in a one-time project, and don't care so much about efficiency beyond your
particular project, then here are the lowest effort + high reward things
you'll want to take a look at:

What value have you assigned to the iterations parameter? Depending on
how your armature is set up, there is at the very least a linear
relationship between the number of iterations per solution and the number
of solutions per second. So if you need the solver to respond 5 times as
fast, try setting the "iterations" parameter to a fifth of your current
value.

However, decreasing the number of iterations has the drawback that your
results are less likely to converge by the time the solver has finished,
which might result in bones that seem to chase (instead of track) fast
moving targets. This can be avoided by increasing the "dampening"
parameter. The dampening parameter establishes a hard limit on how many
degrees the bones are allowed to rotate per iteration. The larger the
dampening parameter, the fewer iterations will be required for the solver
to converge. However, the higher this parameter is set, the less evenly
distributed / graceful the solution will be (particularly, more of the
rotation will occur in the outer bones, and less in the inner bones; so
setting this parameter too high will result in situations where the
armature will, for example, move the arms towards the targets, but not
bother to include rotation in the spine if it can get away with it. This
can lead to somewhat robotic results). How much grace you're willing to
sacrifice for speed is a judgement call that you'll have to evaluate
empirically for your use case.

In short, your best bet for attaining the biggest performance increase with
the least additional effort is to first relax (increase) your dampening
parameter to the largest value that still yields subjectively acceptable
pose quality. Then, having decided on that acceptable level, decrease the
solver's "iterations" parameter until you achieve the minimum acceptable
level of chasing (frames where the solver hasn't quite reached the target).
If at the end of this you find you have performance to spare, then great!
You can use that to bump up whichever of the two (pose quality or
convergence rate) you care more about.
If on the other hand you find that you end up in a situation of mutual
exclusivity between pose quality and convergence rate, or if performance is
at a premium such that spending fewer cycles in the solver would allow you
to spend more elsewhere, then here are some additional optimization details
to consider:

How are you calling the solver? Once per frame? If so, you can likely
increase apparent convergence rate by accounting for the peculiarities of
realtime use cases. Particularly, try setting the iteration parameter to an
extremely low value (ideally 1, but maybe a bit higher depending on
implementation specific overhead), then set the rest of your code up so
that you call the solver on a loop (perhaps in its own dedicated thread)
which runs regardless of whether or not a new frame is available. This way,
the solver will not waste a bunch of iterations trying to converge on the
targets of an old frame when there is a new and more recent frame available
that it should be trying to converge to, and as a bonus, instead of waiting
for the next frame when it's done, it can can spend that additional time on
another iteration to further converge on the available frame.
This approach is definitely more ideal for realtime use, however, the
version of the code you've ported might require you to work around
potential concurrency issues if you try to read the armature's pose while
the solver is still working on it (I can't remember if the last version I
pushed includes changes that alleviate the need for this). A trivial,
low-effort hack around possible concurrency issues might be to create two
copies of the armature: one on which you run the solver, and one which
mimics the pose of the solved armature after every solution.
The tranquil solver and ambitious solver have different performance
characteristics. I believe the tranquil solver should also be less prone to
concurrency issues if you want to minimize workarounds.
Use floats instead of doubles (might lead to instability, haven't
tested).

I suspect that suggestions 2 and 3 should be enough to address your
performance issues (potentially with room to spare). If your test is much
faster in the java / processing demo, and presuming there isn't any
peculiar weird thing in C# or your port, then I suspect 3 is the culprit.
If the java version of your armature is just as slow for you as tour port,
then I will have to look at the code to make sure I didn't accidentally
leave some debugging procedures in or something.
All of that being said, I have expanded this library significantly since I
last pushed it, but have been neglecting updates. If things like target
orientation (instead of just position) are important to you as well, and
you're willing to port another few classes / changes, I can clean up and
push the latest version (which also has some potential efficiency
improvements, particularly in backing the scene graph by actual matrices,
thereby opening up the potential for SIMD optimizations when porting (or
reusing existing scene graph implementations in Unity or whatever)).
Note also that while this library should allow for real time use, the way
its set up is going to be a huge resource hog on the processor. So games
are kind of a poor fit for how it's structured. But this is more a
byproduct of the use cases it targets than fundamental limitations to the
algorithms its backed by. If you're interested in making this a plugin or
something, it might be more appropriate to implement the concepts rather
than the code. I've been meaning to write a blog post exploring /
explaining how it works, so let me know if that's something you're looking
for and I'll take it as a hint from the universe that I should move it up
the priority queue.
On Thu, Apr 18, 2019, 7:50 PM Steven Craft<EMAIL_ADDRESS>wrote:

Hi,
I ported the project to C#/Unity, it seems to run correctly, albeit very
slow (around 5-6FPS if solving every frame, on a high-end laptop).
One line I had to comment out is in bold below:
public void removeFromBoneList(AbstractBone abstractBone) {
if(bones.contains(abstractBone)) {
bones.remove(abstractBone);
boneMap.remove(abstractBone);
}
}
Is this line correct? It appears to be attempting to move an element from
a HashMap by value not key, which I presume will not do anything? I do very
little Java, so I maybe missing something.
Out of interest, what sort of performance would you expect to get when
solving every frame (I have been trying ConstraintExample_Kusudama and
SimpleExample as my tests). I don't know if performance issues have been
introduced in the Java -> C# process, or whether this library is meant more
for offline calculations than runtime (using it for things like working out
where a human skeleton is based on three pieces of tracking data for head
and both hands).
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/EGjoni/Everything-WIll-Be-IK/issues/5, or mute the
thread
https://github.com/notifications/unsubscribe-auth/AAKKPI27JWY77JKCJDIAS4LPRECMPANCNFSM4HHBCJEA
.

Thank you for your detailed response.

What value have you assigned to the iterations parameter? Depending on
how your armature is set up, there is at the very least a linear
relationship between the number of iterations per solution and the number
of solutions per second. So if you need the solver to respond 5 times as
fast, try setting the "iterations" parameter to a fifth of your current
value.

I have been using the defaults in the sample, which seems to be 0.1/50 during init, and then 0.01/25 during each update. As a reference point, in a 'basic' FABRIK solver I have experimented with, I was using 5 iterations, so that does line up with what you are suggesting.

Dampening

Thank you for explaining how this works, based on this I can certainly tweak this parameter.

How I update

I am calling the solver once per frame, in my test bed (basically your samples) I have changed it so rather than clicking a point and then letting it solve, it solves every frame and I can real-time adjust the position of the rootBone pin. If I can get to a nice, stable solution, looking at threading is certainly a possibility.

Which solver

I am currently using the tranquil and must admit I haven't experimented much here yet

Use floats

So this was my first go-to for trying to improve performance, and it did seem to yield a small win (although on some test runs the win sometimes was close to zero). This did not put me off, as, with Unity, whilst user scripts are C#, the backend is optimized native code. Unity works with floats, not double, so once I had a floating point version working, I was able to substitute out the DVector and PVector classes for the Unity native Vector3 class. This has yielded a vast performance win, the stats look something like:

52 FPS - Unity backed float version
13 FPS - standard c# float version
11 FPS - direct Java -> C# port (using doubles)

There is still a lot of math taking place within the solver, some of which potentially could be replaced with Unity native math (perhaps part of the Rot class could be more directly replaced with Quaternions).

If things like target orientation (instead of just position) are important to you

That would be fantastic.  I would like to feed that data into Everything-Will-Be-IK and get back a predicted position of the skeleton. I have been going through a through different solvers for this and stumbled upon Everything-Will-Be-IK and from the video, it looked like it had more power than other simple implementations (in particular with regards to the constraints).
Now, onto the bad news, although I have made progress with the float version, and whilst I believe it is doing the right thing (it is completely possible there are bugs of course) it does feel quite unstable. For reference, here is the straight port (double):
https://youtu.be/Q8lcaImdsH0
Here is the float version:
https://youtu.be/uOkqoLDD87w
If you have any thoughts about the above it would be good to hear (i.e. do you feel the above is caused by float vs double issues, or do you feel there are some bugs in the float version). Also, one function I wasn't sure how to port the best way is:
public static float linearCombination(float a1_, float b1_, float a2_, float b2_)
    {
        double a1 = (double)a1_;
        double a2 = (double)a2_;
        double b1 = (double)b1_;
        double b2 = (double)b2_;

        // the code below is split in many additions/subtractions that may
        // appear redundant. However, they should NOT be simplified, as they
        // use IEEE754 floating point arithmetic rounding properties.
        // The variable naming conventions are that xyzHigh contains the most significant
        // bits of xyz and xyzLow contains its least significant bits. So theoretically
        // xyz is the sum xyzHigh + xyzLow, but in many cases below, this sum cannot
        // be represented in only one double precision number so we preserve two numbers
        // to hold it as long as we can, combining the high and low order bits together
        // only at the end, after cancellation may have occurred on high order bits

        // split a1 and b1 as one 26 bits number and one 27 bits number
        double a1High = System.BitConverter.Int64BitsToDouble(System.BitConverter.DoubleToInt64Bits(a1) & ((-1L) << 27));
        double a1Low = a1 - a1High;
        double b1High = System.BitConverter.Int64BitsToDouble(System.BitConverter.DoubleToInt64Bits(b1) & ((-1L) << 27));
        double b1Low = b1 - b1High;

        // accurate multiplication a1 * b1
        double prod1High = a1 * b1;
        double prod1Low = a1Low * b1Low - (((prod1High - a1High * b1High) - a1Low * b1High) - a1High * b1Low);

        // split a2 and b2 as one 26 bits number and one 27 bits number
        double a2High = System.BitConverter.Int64BitsToDouble(System.BitConverter.DoubleToInt64Bits(a2) & ((-1L) << 27));
        double a2Low = a2 - a2High;
        double b2High = System.BitConverter.Int64BitsToDouble(System.BitConverter.DoubleToInt64Bits(b2) & ((-1L) << 27));
        double b2Low = b2 - b2High;

        // accurate multiplication a2 * b2
        double prod2High = a2 * b2;
        double prod2Low = a2Low * b2Low - (((prod2High - a2High * b2High) - a2Low * b2High) - a2High * b2Low);

        // accurate addition a1 * b1 + a2 * b2
        double s12High = prod1High + prod2High;
        double s12Prime = s12High - prod2High;
        double s12Low = (prod2High - (s12High - s12Prime)) + (prod1High - s12Prime);

        // final rounding, s12 may have suffered many cancellations, we try
        // to recover some bits from the extra words we have saved up to now
        double result = s12High + (prod1Low + prod2Low + s12Low);

        if (double.IsNaN(result))
        {
            // either we have split infinite numbers or some coefficients were NaNs,
            // just rely on the naive implementation and let IEEE754 handle this
            result = a1 * b1 + a2 * b2;
        }

        return (float)result;
    }

At the moment, I am just converting to and from doubles for this function, but as this function is designed to allow more stability/accuracy, I wondered whether it could be an initial problem area. I wondered whether this function could be updated to work with floats, there is information about the method being used here:
http://www.ti3.tu-harburg.de/paper/rump/OgRuOi05.pdf
But it wasn't clear to me how it would be adapted to floats.
My hope is somehow there is a way to improve the stability of the float version so it is useable, as I feel if this was the case, there would be a path to get the performance acceptable for my use case.

Oh wow yeah that is hilariously bad performance (especially for so simple an armature). If Unity has a profiler or something that lets you look at how much time is spent in each function, I would be very interested to see where exactly it's choking.
It's good that using Unity backed floats results in such a performance win, but I think it might still be competing against a huge performance loss elsewhere. I say this primarily because even using doubles and no SIMD instructions on a considerably more complex armature than the one you linked to, the Java version is easily able to handle 1500+ iterations per second on a 6 year old core i5.
For reference, here is a video of the tranquil solver and its performance characteristics over a range of parameters as I tweak them on the left hand panel (note that the dampening value in the UI has been converted to degrees for display purposes, you should not actually set the dampening to a higher value than PI, as this amounts to no dampening. Also note that in this example, the solver is invoked once per mouse input event, not once per frame, but even so your port should still be getting much better performance than 53 fps).

There is still a lot of math taking place within the solver, some of which potentially could be replaced with Unity native math (perhaps part of the Rot class could be more directly replaced with Quaternions).

Rot is mostly just a wrapper around the Apache Commons Rotations class, which is itself essentially just a collection of Quaternion and Matrix convenience functions; so yes, wherever you find opportunity to use whatever Quaternion functions Unity offers, absolutely do. But as a heads up, to be mindful that some libraries use the convention q0=x, q1=y, q2=z, q3=w, while some others use q0=w, q1=x, q2=y, q3=z. (this can be a huge pain if you forget to swap a q3 for a q0 somewhere).

Now, onto the bad news, although I have made progress with the float version, and whilst I believe it is doing the right thing (it is completely possible there are bugs of course) it does feel quite unstable.

If both are using the tranquil solver with Kusudama constraints (instead of Euler constraints, which were added on a whim and aren't very well tested), then I'm inclined to suspect that the instability is due to how floating point errors are being handled, yes. Using the built in Unity quaternion implementation could help with this.

At the moment, I am just converting to and from doubles for this function

I think this is probably a very bad idea (given the IEEE754 reliance). Converting floats to doubles and then operating on the bits as if they were doubles is quite plausibly introducing further instability. If you can avoid that function altogether, and rely instead on some built in Unity or C# library to do a basic linear combination, that would be ideal. Another option would be to replace
public Rot(DVector iv1, DVector iv2, DVector iu1, DVector iu2) {
		Vector3D v1 = new Vector3D((double) iv1.x, (double) iv1.y, (double) iv1.z);
		Vector3D v2 = new Vector3D((double) iv2.x, (double) iv2.y, (double) iv2.z);
		Vector3D u1 = new Vector3D((double) iu1.x, (double) iu1.y, (double) iu1.z);
		Vector3D u2 = new Vector3D((double) iu2.x, (double) iu2.y, (double) iu2.z);
		try {
			rotation = new Rotation(v1, v2, u1, u2);
		} catch(Exception e) {
			rotation = new Rotation(v1, 0f);
		}
}

To something that manually does the same thing in terms of two separate rotations (and wrapping around Unity's built in Quaternion functions). Like:
public Rot(Vector3 iv1, Vector3 iv2, Vector3 iu1, Vector3 iu2) {

	Rotation firstRotation = new Rotation(iv1, iu1);
        firstRotation.normalize();
	Vector3 t_v2 = firstRotation.applyTo(iv2);
	Rotation secondRotation = new Rotation(t_v2, iu2);
	Rotation result = secondRotation.applyTo(firstRotation);  //composes the two rotations
	rotation  =  result.normalize();
}

I have been going through a through different solvers for this and stumbled upon Everything-Will-Be-IK and from the video, it looked like it had more power than other simple implementations  (in particular with regards to the constraints).

Yup. Basically Inverse Kinematics is super easy right up until you have to worry about constraints. At which point, it is suddenly super hard.

My hope is somehow there is a way to improve the stability of the float version so it is useable, as I feel if this was the case, there would be a path to get the performance acceptable for my use case.

I feel like there isn't any inherent algorithmic reason to expect huge stability issues when going from double to float in your ultimate use case (they would be more likely in your test case though, depending on how your armature is constrained). In general. the nature of the two solvers is such that they should inherently correct for numeric instabilities whenever a solution is readily reachable. For a head and hands rig, a solution should pretty much always be readily reachable (I guess unless the player is a contortionist).
Beyond that, it's also possible that there is a bug in the version you're porting from which I have since inadvertently corrected. I'm working on cleaning up the new version's code now, so I guess we'll see. In the meantime, you wouldn't be wasting effort if you began / continued hooking things into Unity's quaternion implementation, as the new version still relies heavily on quaternion math.

I did a profile, however, C# managed code is just really slow (under Unity at least), it is just a huge mess of boxing/unboxing values, indirect memory access to ensure no out of bounds issues are hit, etc. I have seen this exact issue in the past where maths code in a C# script performs 10X (or more) worse than the same code written natively, put in a DLL and called from the C# script. There is a large amount of Garbage Collection being generated and cleared up too.
I noticed whilst comparing different versions, that if I just left the float version (the version where all I have done is basically replace double's with float's, although the same issue also occurs in the version where I converted to Unity native types for Vectors etc) going for several minutes, it all just falls apart (you may want to speed YouTube up to double speed or more):
https://youtu.be/9bVnV_t69AU
I have combed through the changes between the double and float versions several times now, but I am failing to see any real difference, nearly all the changes are just things like:

float -> double
1.0d -> 1.0f
Math.Sin -> Mathf.Sin

When your new version hits, I'll basically convert that to C#, get that working, then do the same double -. float and then float -> Unity types and see what results I get. That will naturally replace anything I have looked at so far, that said, if you can suggest anything for me to look at, from the above video, I am happy to take a look. As I fear the above issue is going to come back with any float version until a diagnosis can be made of what is causing it. FWIW, I changed the linear combination function to just do a1 * b1 + a2 * b2 and still get the issue, so that isn't the code causing the issue.
Your video looks great and represents exactly what I would like to achieve, a really stable, robust approach to IK, which maintains target position and orientation whilst obeying a set of well-defined constraints. It is also exciting to think it should be possible to get decent speed ultimately (I will worry more about speed once I have got a float version robust).

Oh: a little question, in terms of this C#/Unity version, I am more than happy to put the source on GitHub, would it make the most sense for it to be a new project (EWBIK-C#), or a fork of this project with some branches (branches/c#-double branches/c#-float branches/c#-unity-native-types) or some other variations?

I realize I didn't answer one part of the question, which is to say my faster version is using UnityEngine.Vector3 and UnityEngine.Quaternion as direct substitutes for what was the DVector/PVector and Quaternion classes. I did indeed run into the issue of Unity having quaternions set up so the first three elements are the XYZ and the forth is the W, however I was lucky that this was actually easier than I had expected it to be to work around, with some getQ0(), getQ1(), getQ2(), getQ3() methods returning the right thing, and being careful on the order of parameters to the constructor.
So, I have three versions, the straight port (double) version, the conversion to float (singles) and the conversion (from the float version) to the version which uses Unity.Vector3 and Unity.Quaternion. The double version seems to behave correctly, the float and Unity.Vector3/Unity.Quaternion versions behave the same and is captured in the above video where things just fall apart. If I move the target around, the skeleton updates and attempts to follow, and gives the impression it is generally working relatively well, but something somewhere is causing this constant issue which completely destroys the implementation. Hopefully, that is all clear.

I have seen this exact issue in the past where maths code in a C# script performs 10X (or more) worse than the same code written natively, put in a DLL and called from the C# script. There is a large amount of Garbage Collection being generated and cleared up too.

Ah, well that solves that mystery. Though, with regard to the Garbage Collection, this code's current but somewhat inconsistent philosophy is "whatever, that's what the garbage collector is for."

That will naturally replace anything I have looked at so far, that said, if you can suggest anything for me to look at, from the above video, I am happy to take a look.

Nope! The video you just posted is perfect. I've seen that drift issue before and I believe I know how to fix it. The only thing I'm unsure of is if it will still need fixing in the new version. (The solution is to just orthonormalize each bone's local axes every time it gets touched by iterateCCDStrand(), and possibly forcibly translate each child back to the parent bone tip [and I can give you more detailed instructions if you'd prefer to try it ahead of time than just wait for the new version]).

Oh: a little question, in terms of this C#/Unity version, I am more than happy to put the source on GitHub, would it make the most sense for it to be a new project (EWBIK-C#), or a fork of this project with some branches (branches/c#-double branches/c#-float branches/c#-unity-native-types) or some other variations?

Hmm. I can't really come up with any reason why users would want a jar and java source in some dusty directory their C# code never touches, so I think a fork would probably be the more sensible approach? Preferably with a link in the readme back to this project or something so people can check for (and maybe help with) feature lag between the two.

The double version seems to behave correctly, the float and Unity.Vector3/Unity.Quaternion versions behave the same and is captured in the above video where things just fall apart. If I move the target around, the skeleton updates and attempts to follow, and gives the impression it is generally working relatively well, but something somewhere is causing this constant issue which completely destroys the implementation. Hopefully, that is all clear.

I think I understand, yes. Though just to be clear, the Vector3/Quaternion version is sufficiently fast for your purposes, and the issue right now is primarily stability correct?

I have seen this exact issue in the past where maths code in a C# script performs 10X (or more) worse than the same code written natively, put in a DLL and called from the C# script. There is a large amount of Garbage Collection being generated and cleared up too.

Ah, well that solves that mystery. Though, with regard to the Garbage Collection, this code's current but somewhat inconsistent philosophy is "whatever, that's what the garbage collector is for." So optimizations are possible.

Yes, and on my end, I'm hopeful I can find more optimizations. I will look more into the profiler side, I have just avoided diving too deep into any one area whilst things are in flux. Seeing the performance win from using Unity Native stuff makes me feel that is the direction I want to end up going in (and it makes sense anyway, better to use a single Vector/Quaternion/Maths lib than have a new version bundled).

That will naturally replace anything I have looked at so far, that said, if you can suggest anything for me to look at, from the above video, I am happy to take a look.

Nope! The video you just posted is perfect. I've seen that drift issue before and I believe I know how to fix it. The only thing I'm unsure of is if it will still need fixing in the new version. (The solution is to just orthonormalize each bone's local axes every time it gets touched by iterateCCDStrand(), and possibly forcibly translate each child back to the parent bone tip [and I can give you more detailed instructions if you'd prefer to try it ahead of time rather than just wait for the new version]).

That's a relief! I was worried you would have no idea and it would be a tricky problem to work out how to track down and resolve. If there is a straight forward way for me to test your idea I am happy to do so, equally I am happy to wait for your big code drop and see whether that resolves the issue or not. Although it took a while to get the code working the first time around in C#/Unity, I have spent long enough porting and reporting and fixing issues that I suspect I'll be able to bring over the latest version without too many issues (unless brand new complicated issues spring up, which can happen sometimes)!

Oh: a little question, in terms of this C#/Unity version, I am more than happy to put the source on GitHub, would it make the most sense for it to be a new project (EWBIK-C#), or a fork of this project with some branches (branches/c#-double branches/c#-float branches/c#-unity-native-types) or some other variations?

Hmm. I can't really come up with any reason why users would want a jar and java source in some dusty directory their C# code never touches, so I think a new project would probably be the more sensible approach? Preferably with a link in the readme back to this project or something so people can check for (and maybe help with) feature lag between the two.

Perfect, I'll wait until the new code drops, then get the port working, once things are looking good I'll get it into a GitHub project and link it back to this main project.

The double version seems to behave correctly, the float and Unity.Vector3/Unity.Quaternion versions behave the same and is captured in the above video where things just fall apart. If I move the target around, the skeleton updates and attempts to follow, and gives the impression it is generally working relatively well, but something somewhere is causing this constant issue which completely destroys the implementation. Hopefully, that is all clear.

I think I understand, yes. Though just to be clear, the Vector3/Quaternion version is sufficiently fast for your purposes, and the issue right now is primarily stability correct?

Well, it isn't fast enough as is, but as is it is doing 25X iterations per frame, so getting that down to 5X iterations is going to help a lot. Further to that, Unity has a 'Burst Compiler' now, which can compile subsets of C# down to natively level performing bytecode. It isn't an area I have looked into very much, but I think this will be a good excuse for me to look into it. Even without the Burst Compiler, I will look further into the profiler and see what can be done. It feels like it should be possible to get a decently performing project though. I need to be able to target 90 FPS for Virtual Reality, and I can't use the entire frame time calculating IK, however, my hope is it will possible to get this running at a decent enough rate. But yes, before anything can happen, I need to get it stable using floats. Once the stability is there, I will be able to commit and have a baseline to compare any optimizations against.

Oh, just to keep you in the loop I'm hoping to push the new version maybe tonight or tomorrow. But, I was thinking about your use case last night and I'm concerned you have a strange additional challenge that might require you to do something additionally clever on top of just the IK system. Specifically, while you know the positions and orientations of the head and hands, you probably don't have advanced knowledge of the physical proportions of the body they're attached to, and this can have significant impact on pose quality (most worryingly, if your user is shorter than you anticipate, the virtual armature's spine will basically crumple inward, because as far as the solver is aware, your user isn't short, they're just trying to retract their head into their own torso).
I think you might be able to alleviate some of that issue by having some callibration step where you ask the user to mimic some prespecified poses so that you can infer their proportions -- but do be aware that it is something you'll likely need to think about.
I'm adding a few minor features as I clean up which should hopefully make it easier to avoid unnatural configurations resulting from minor deviations, but the problem domain is interesting and I'm vaguely beginning to suspect that you might find that constraining your armature won't be the best approach, or might even be counterproductive for your purposes. What you might end up finding gives better results is first creating a couple dozen prespecified known-good poses corresponding to different end-effector positions. Then doing some procedure like

for each sensor,
i.take the four poses whose effector's bound that sensor's position in an n-simplex (probably a tetrahedron).
ii. find the barycentric coordinates of the sensor within that simplex.
iii. pose the armature to the weighted (by corresponding barycentric coordinate) average of those bounding poses.
average the three poses from those sensors.
run the IK solver unconstrained beginning from THAT averaged pose.

I think I remember reading a paper that did something like that a couple of years ago, I'll see if I can find it.

Oh, just to keep you in the loop I'm hoping to push the new version maybe tonight or tomorrow.

Amazing, thank you, I look forward to porting it over to C# / Unity and seeing if the floating point issue has been resolved.

But, I was thinking about your use case last night and I'm concerned you have a strange additional challenge that might require you to do something additionally clever on top of just the IK system. Specifically, while you know the positions and orientations of the head and hands, you probably don't have advanced knowledge of the physical proportions of the body they're attached to, and this can have significant impact on pose quality (most worryingly, if your user is shorter than you anticipate, the virtual armature's spine will basically crumple inward, because as far as the solver is aware, your user isn't short, they're just trying to retract their head into their own torso).

Yes, so at the moment, with a basic FABRIK solver that is in now, there are avatars floating around, clearly the avatars were not designed for my 6'5" stature! So the initial plan was to have a calibrate button, the user stands naturally, presses calibrate, and then the avatar is scaled so the head sensor position matches the avatars head position whilst keeping feet grounded. I had appreciated that the entire skeleton wouldn't be a perfect match, but it wasn't clear how much of a problem it would cause. It sounds like more of a problem than I hoped. As an initial pass I could also ask the user to stand with arms fully extended horizontally, to get arm length, and then to bend elbows 90°, to get the distance to the elbow. As we only have tracking data for head and hands, perhaps these measurements would do enough?

I think you might be able to alleviate some of that issue by having some callibration step where you ask the user to mimic some prespecified poses so that you can infer their proportions -- but do be aware that it is something you'll likely need to think about.

Let me know if you feel more than the above would be required.

I'm adding a few minor features as I clean up which should hopefully make it easier to avoid unnatural configurations resulting from minor deviations, but the problem domain is interesting and I'm vaguely beginning to suspect that you might find that constraining your armature won't be the best approach, or might even be counterproductive for your purposes. What you might end up finding gives better results is first creating a couple dozen prespecified known-good poses corresponding to different end-effector positions. Then doing some procedure like

for each sensor,
i.take the four poses whose effector's bound that sensor's position in an n-simplex (probably a tetrahedron).
ii. find the barycentric coordinates of the sensor within that simplex.
iii. pose the armature to the weighted (by corresponding barycentric coordinate) average of those bounding poses.
average the three poses from those sensors.
run the IK solver unconstrained beginning from THAT averaged pose.

I think I remember reading a paper that did something like that a couple of years ago, I'll see if I can find it.

Certainly with basic FABRIK, without correct bone dimensions, and limited constraint support (some homemade constraint support has been added, a bit of a WIP with lots of limitations) various issues were coming up, and it felt like the issue was whilst the solution reached was possible to mimic, it simply wasn't natural. It felt like we needed to work out a hint system, sort of like if the hands are in front and close to the chest, elbows are more likely to point outwards (even tho you can point your elbows downwards and reach many of the same points). I wonder if your suggestion could be an improved way to resolve this as we would essentially be suggesting these hints via the initial pose we solve from. Any helper methods you can add to achieve the above, or samples, would be highly appreciated. IK is a domain with quite a lot of deep knowledge required, lots of solutions available to achieve a basic, constraintless, rotationless, solution, but when you have a real world case with constraints, desired poses to reach, etc there doesn't seem to be so much around. I was surprised to i. find this project and ii. for it to be so actively supported. I don't know for sure, but I suspect the C# / Unity port may get more eyes as it has become the defacto way most people get into games development now.

Let me know if you feel more than the above would be required.

You might also want arms stretched sideways while shrugging, arms stretched upwards while shrugging, arms stretched downward while shrugging, and arms stretched forward while shrugging. With these you should be able to construct a sphere from four points for each arm. The center of this sphere, +  (arms stretched outwards while shrugging - arms stretched outwards as much as possible)  should roughly = the location of shoulder, allowing you to infer shoulder width.
Alternatively, you could ask the user to just touch the controllers to their shoulders, but this has the disadvantage of not being as funny to everyone else in the room.

Certainly with basic FABRIK, without correct bone dimensions, and limited constraint support (some homemade constraint support has been added, a bit of a WIP with lots of limitations)

I suggest not bothering trying to get constraints working with basic FABRIK. Deadlocks and race conditions are basically inevitable, and it's kind of annoying that the author simply published as if this weren't the case.

it simply wasn't natural. It felt like we needed to work out a hint system, sort of like if the hands are in front and close to the chest, elbows are more likely to point outwards (even tho you can point your elbows downwards and reach many of the same points).

Yeah, you're likely going to end up encountering the same sorts of "unnaturalness" out of the box with this library. Constraints can prevent poses that amount to hyper-extension, but they have no sense of "preference" or even "mild discomfort." There is a bunch of inactive / unfinished code in the new version for "soft" kusudama constraints, which essentially work by establishing some pressure to keep bones away from the constraint edges (here's an inactive visualization [which you should probably mute before watching]). I realized after like a day of working on the the way I needed it to behave to not to be a pain for artists was mathematically impossible. However, I think the way you would need it to behave (presuming your use case doesn't require that keyframes take precedence over IK solutions) basically amounts to the trivial case, and should help keep things more natural. I can leave the bulk of that code in and maybe add some explanation, but I can't make any guarantee as to its stability.

Any helper methods you can add to achieve the above, or samples, would be highly appreciated.

Yeah, I'll have to think through the best data structure for it. Something similar would be useful from an artist perspective, so I'll need something general enough to be usable across domains. I managed to find that paper I vaguely remembered reading, but it turns out to be a bit too "out there" for most purposes..

I don't know for sure, but I suspect the C# / Unity port may get more eyes as it has become the defacto way most people get into games development now.

 Since you're working with Unity I imagine FinalIK would have been one of them. The videos I'd seen of it a couple of years back looked pretty robust, did it turn out to be limiting in some unanticipated way?

I've noted your thoughts on shoulders and grabbed a copy of the PDF linked to.
The main issue with FinalIK is the license, having code that has no restrictions is always helpful.
The video is certainly interesting, it will be nice if we can get the Unity version synced with the Java version (including getting the same sort of tools you are demonstrating in the video, for editing and visualising joints, constraints, etc). At the moment my thinking is if you make smaller updates (after the big drop) they may be able to be manually applied to the C# in the appropriate places. If any large changes are made, then it'll be likely instead I'd reconvert the project (which involves running some scripts I have written and then fixing up a bunch of issues manually that I have become aware of). Either way, it would be nice if there was some parity - one step at a time of course. Perhaps we can work out some basic unit tests, i.e. if you import skeleton s and then move the IK effectors to a series of specific rotations and positions, does each bone appears at the correct place/orientation (where correct is within a low epsilon like tolerance).

The constraint visualizations are basically entirely in glsl. So porting to Unity should be trivial. I'll upload those with the next piush.
Unit tests are a good idea. I'll put in some of the JSON code I've set up for saving / loading armatures into the abstract classes. (I assume Unity/C# have reasonably painless JSON parsers?)
On the topic of parity, I realized this is going to be very inefficient to debug unless there's a float based version in Java, so I'm untangling a few things to make it easy to swap between the two. This means it'll probably be another day.
P.S. I think I've figured out a really good and simple way to give the user very reliable and versatile control over poses / rules across a wide variety of body types, proportions, and use cases.

This all sounds very promising - it was on my mind that having a float Java version would have a lot of benefits for parity - I did try and get the Java version running myself but failed (I have done very little Java, with the majority of it being within Android apps, not standalone desktop stuff). So if you are able to run a float version at your end, we could get near identical results which should massively help track down issues.

New version is up. I've added but have not yet tested support for floats. Though I suspect drift won't be an issue.
I will upload an example humanoid armature with hands and head targets tomorrow. And probably clean the code a bit more.
With regard to porting, "Basis.java" and "AbstractAxes.java" are likely the places to start. Let me know if there's anything weird or confusing, some of the structure is there to make the library extensible to other java frameworks, and can likely be safely simplified when porting.

Fantastic, I hope to have time after work tonight to take a look at this and see if I can get it ported over and tested!


fix: Removed archived project

Removed the reference to the conversion tool as it's been merged into the sdk repo / functionality.
PR fulfills these requirements

[x] Commit message(s) and PR title follows the format [fix|feat|ci|chore|doc]: TICKET-ID: Short description of change made ex. fix: DEV-XXXX: Removed inconsistent code usage causing intermittent errors
[ ] Tests for the changes have been added/updated (for bug fixes/features)
[x] Docs have been added/updated (for bug fixes/features)
[x] Best efforts were made to ensure docs/code are concise and coherent (checked for spelling/grammatical errors, commented out code, debug logs etc.)
[x] Self-reviewed and ran all changes on a local instance (for bug fixes/features)

Change has impacts in these area(s)
(check all that apply)

[ ] Product design
[ ] Backend (Database)
[ ] Backend (API)
[ ] Frontend

Describe the reason for change
(link to issue, supportive screenshots etc.)
What does this fix?
(if this is a bug fix)
What is the new behavior?
(if this is a breaking or feature change)
What is the current behavior?
(if this is a breaking or feature change)
What libraries were added/updated?
(list all with version changes)
Does this change affect performance?
(if so describe the impacts positive or negative)
Does this change affect security?
(if so describe the impacts positive or negative)
What alternative approaches were there?
(briefly list any if applicable)
What feature flags were used to cover this change?
(briefly list any if applicable)
Does this PR introduce a breaking change?
(check only one)

[ ] Yes, and covered entirely by feature flag(s)
[ ] Yes, and covered partially by feature flag(s)
[x] No
[ ] Not sure (briefly explain the situation below)

What level of testing was included in the change?
(check all that apply)

[ ] e2e
[ ] integration
[ ] unit

Which logical domain(s) does this change affect?
(for bug fixes/features, be as precise as possible. ex. Authentication, Annotation History, Review Stream etc.)

Please don't remove this. Still applies, I think...


Completed Feature: Image Size Threshold (Issue : #3)

Here's a revised version of your PR description:

PR Description

[x] Main Feature: Implemented RemoveExceedSizeLimit() to address the image size threshold for clearing unused images.
[x] Code Refactor 01: Replaced log.Fatalf with log.Printf to improve error handling by returning errors to main.go. Additionally, added utils.go in the Docker package to consolidate reusable functions.
[x] Code Refactor 02: Removed all fmt package usages and switched to the Go log library for logging.

Testing

Let me know if you need any more adjustments!

Hi @mmuazam98, I have completed the Issue #3 also which has been assigned to me. Could you please review and merge this if it looks good.
I'll complete the goroutines issue #6 and update you soon
Thanks

Hi @mmuazam98 , could you also add hacktoberfest-accepted label after merging this PR.
To make it valid for Hacktoberfest
Thanks!

@deexithparand can you create a new PR that updates the README file?

Sure @mmuazam98 , I'll raise a new PR and update the README


401: Unauthorized when calling hassio API

In HA0.77.3:
curl -H "Content-Type" -X POST --data '{"protected":"false"}' http://hassio/addons/local_nrfnet/options
return 401.
In HA0.77.2: this command works well.
I add  "x-ha-access:${HASSIO_TOKEN}" into the parameters but the command returns 401 also.

You need include the hassio token + the api call have moved to /addons/.../security. Look at changelog of 129

https://github.com/home-assistant/developers.home-assistant/blob/master/docs/hassio_addon_communication.md#hassio-api

I think that would resolve your issues @tomcatGu, therefore closing this issue.
If you have additional questions, feel free to re-open or (probably better), join us at Discord, so we can help you directly.


Deleting forms and reuploading the same one resulted in error

It shown errors that previous form already exist (although I've deleted it previously in odk central web) :

I only managed to reuploaded after changing the form_id, converted it to xml and re-upload it again.

as described in the message, this is intentional. you must change either the form id or the version in order to upload the same form again.
this is because if somebody has already downloaded the form to their mobile device before you deleted it from the server, and then you create a new different form with the same form id and version, then there is no way for the system to know that the form on the device is different than the form on the server when submissions to the form are uploaded, and data can get mixed and confused.

oh ok, that's make sense although I wish I can override it while still in development mode. Is it possible if i redownload the docker images or delete the specific data from the posgres db directly?

yeah—eventually if/when we have form preview on the central web panel itself, we may introduce some sort of draft state where you can replace over a form as many times as you'd like because it won't be visible for download onto mobile devices.
for right now, you can delete the data from postgres directly, yes: you can log into postgres with this command:
docker-compose exec postgres psql -U odk -W odk
and giving the password (by default also just odk). then you can cascading-delete the appropriate records from the forms table.

thanks, I've managed to delete the previous form now.


Revise Delta distribution to support ADVI + Transforms

This refactors our Delta distribution to work with transforms in ADVI.
The Problem
Before this PR, ADVI used TransformeDistribution(Delta, ...), however this combination fails due to finite precision in the x -> t(x) -> t.inv(t(x)) round trip, leading to sampled values accidentally missing the infinitesimally narrow delta from whence they came. The reason we are using Deltas at all is to allow the ADVI guide to keep all randomness in a single packed Normal or MultivariateNormal latent, and to unpack this into multiple Delta sample statements (one per sample site in the model).
Proposed Solution
This PR adds a bit of functionality to our Delta distribution so that it can function as a proper TransformedDistribution(Delta, ...). This is mathematically appealing, since after this PR, the class of Delta distributions is closed under transformations by differentiable functions. Specifically, this PR:

Adds a log_density parameter to Delta, so we can replace- TransformedDistribution(Delta(v), t)
+ Delta(t(v), log_density=-t.log_abs_det_jacobian(v, t(v)))

Adds an event_dim parameter to Delta, so we can support multivariate transforms (and their Jacobians)
Removes the [unused] .enumerate_support() method since the log_density scales log_prob such that it is no longer normalized when considered as a discrete distribution. If we have use for discrete delta distributions in the future, we should consider creating a new class without .rsample().

Alternate solutions include:

Creating a separate TransformedDelta distribution
Restricting our use of TransformedDistribution(Delta(v), transform) to cached transforms so they can avoid the round-trip.
Using fuzzy comparison in Delta.log_prob(-). This is a bad idea.

Tested

[x] added shape tests for the new Delta parameters
[x] added loss assertions to existing ADVI tests
[x] added a test with Dirichlet distribution

Ok, I'm happy with this PR now that the tests are all passing (and not xfailing).


Restore updating chain head during backward sync

PR description
Updating chain head during backward sync was wrongly removed during the removal of the finalized block rule, this PR restore it and also complete the removal of the remaining dead code that was meant to manage the finalized rule., and some renaming to improve semantic.
Removed a log statement that was keeping reference to blocks sent by newPayload, that causes high memory consumption during initial sync.
Fixed Issue(s)

Documentation

[x] I thought about documentation and added the doc-change-required label to this PR if
updates are required.

Changelog

[x] I thought about the changelog and included a changelog update if required.

renamed the branch and recreated here https://github.com/hyperledger/besu/pull/4718


Shortcodes: Add support for only showing teaser in content shortcode

Fixes #843.
The ticket is asking for tagging support in the excerpt, which doesn't really go with how WordPress thinks of the excerpt's use (just text, no links etc), but we have something similar built-in to the content. The More tag let's users define a teaser for a post and only shows that teaser on index and archive pages. We can use the same mechanism to allow Activitypub users to limit the amount of the content that's getting sent out with the [ap_content] shortcode.
This PR introduces a new, undocumented, shortcode attribute called more_tag, that will split out the content to only return the teaser, if a more tag is defined in the content.
Proposed changes:

Adds a shortcode attribute to selectively only show the post's teaser.
Updates shortcode docs to account for the new attribute.
Adds unit tests

Docs entry after:

Other information:

[x] Have you written new tests for your changes, if applicable?

Testing instructions:

Go to Activitypub settings, select "Note" as Activity-Object-Type, and save changes.
In post content, insert a ap_content shortcode with the more_tag attribute set to "yes", and save changes.
Create a new post that includes a more tag and publish.
Make sure that it only shows the teaser in Outbox.

This is already possible if you use the template. Simply use the [ap_excerpt] and add @-replies and hashtags to the code. The mention and hashtag code will be also executed on the excerpt!

Huh, I guess I took the bug report at face value instead of probing further. Let me do that

Okay, it looks like the premise for this PR is moot. I don't have a strong preference for this still to be merged. Happy to just close it and write it off as a learning exercise for Cursor and shortcodes.

Here’s my personal take on shortcodes in general: they were a creative early solution that offered a lot of flexibility, but I’d like to see them phased out over time. As the community focuses on making long-form content a first-class citizen of the fediverse, I believe we’ll eventually no longer need to rely on exceptions. Templating was essential for crossposting scenarios, but on the fediverse, the focus should be on federating original content as is. Platforms like Mastodon, for instance, don’t have a template feature to dictate how content appears on other servers.

Thanks, that makes sense. Let's just close it :)


Is there a command doing the same thing as git add -A ?

In a terminal, "git add -A" not only stages newly added/modified files but also stages deleted files. That way we don't have to call "git rm" to stage deleted files. How do I achieve this in nodegit ? Does Index.addAll() only stages newly added/modified files?

@calvinhmw that is correct. git add -A is syntax sugar for what you want to do. We don't have a built-in way currently to do what you're asking. One thing you can do is iterate over all of the files and call Index#remove on the files you want to delete.

@johnhaley81 I'm also looking for a simple way to stage all new / changed / deleted files in HEAD and commit them if needed. Could you point to an example of that? Could not find anything simple in examples, so what do you think of adding it?


[5.1] [Should be 5.0] Allow Toggling Route Filtering

This is useful for testing filters separately.

?

Not sure why this was deleted, but it would be really useful.

I've opened a new issue for this: #7360.

Sorry, didn't realize it close the issue when I was spring cleaning my PR branches.

Don't worry about it. :)


POC to post provider bundle with header to save data from MyCarePlanner into SDS

Reference #266

Discuss this with Dan

@swmuir to ask matt for information on the header to manipulate to save data into the SDS.

Need a UX design to demonstrate what this would look like. (Ex: button that says "share my data.")

Sean met with Matt. Sean is working on backend. There may be a link in the code already that's commented out.

Sean ran into a Node issue. Need at least Node 18, to use the Fetch feature. Since we're updating, we should jump up to the latest version of Node.
Other approach is to try FHIR client to do $everything operation. There might be something built into React to select a certain API if it's available. @swmuir will try the FHIR client first.

Ready for QA (but probably needs to be done by Matt).


Add a schedule-aware repeat

While the retry mechanism allows for repeating an operation - according to a Schedule in case of error, we'd also like to have a schedule-aware repeat, which would run an operation according to a Schedule or perhaps until a stop condition is met.
Some things to consider:

what happens if the operation fails before the scheduled number of repetitions? Maybe this could be expressed with a multi-policy (see: #55) that combines repeat and retry?
perhaps the stop condition could be thought of yet another Schedule like UntilResult[T](p: T => Boolean) or Until(p: => Boolean), or both?

Some operations that we'd like to have:

schedule at fixed rate
schedule with an initial delay

Should we measure time between start of each operation (start2-start1), or the pause between operation invocations (start2-end1)? Or maybe this should be configurable?

I did some API prototypic which led to few observations:

We could refactor current retry* functions to scheduled* and reimplement retry* on top of them, specifically:

def schedule[T](
    schedule: Schedule,
    onTick: (Int, Either[Throwable, T]) => Unit = (_: Int, _: Either[Throwable, T]) => (),
    shouldRetryOnError: Throwable => Boolean = (_: Throwable) => false,
    shouldContinue: T => Boolean = (_: T) => true
)(operation: => T): T =
  scheduleEither(schedule, onTick, shouldRetryOnError, shouldContinue)(Try(operation).toEither).fold(throw _, identity)

def scheduleEither[E, T](
    schedule: Schedule,
    onTick: (Int, Either[E, T]) => Unit = (_: Int, _: Either[E, T]) => (),
    shouldRetryOnError: E => Boolean = (_: E) => false,
    shouldContinue: T => Boolean = (_: T) => true
)(operation: => Either[E, T]): Either[E, T] =
  scheduleWithErrorMode(EitherMode[E])(schedule, onTick, shouldRetryOnError, shouldContinue)(operation)

def scheduleWithErrorMode[E, F[_], T](em: ErrorMode[E, F])(
    schedule: Schedule,
    onTick: (Int, Either[E, T]) => Unit = (_: Int, _: Either[E, T]) => (),
    shouldRetryOnError: E => Boolean = (_: E) => false,
    shouldContinue: T => Boolean = (_: T) => true
)(operation: => F[T]): F[T] = ...

Then retryWithErrorMode could reuse  scheduleWithErrorMode like this:
def retryWithErrorMode[E, F[_], T](em: ErrorMode[E, F])(policy: RetryPolicy[E, T])(operation: => F[T]): F[T] =
  scheduleWithErrorMode(em)(
    policy.schedule,
    policy.onRetry,
    policy.resultPolicy.isWorthRetrying,
    t => !policy.resultPolicy.isSuccess(t)
  )(operation)

The language used in Schedule docs/symbol names couples it to the retry concept, so that would need to be refactored. For example maxAttempts instead of maxRetries
Then Schedule trait can be extended with additional variants like FixedRate, initial delay, etc. The problem is the additional variants wouldn't make sense for retry as well as some of the existing variants, like Backoff wouldn't make sense for scheduling, so I'm not sure about it.

See #172

I still think repeat and retry are in their essence the same thing. You're right we might need to change the vocabulary a bit, but both of these perform the same basic steps:

run a computation
inspect the result, and either:
3a. sleep and restart
3b. finish

The difference is that for repeat the inspection in step 2. checks if the result is successful - and if so, continues, otherwise finishes with an error. For retry, it's the opposite: it checks if the result is an error, and if so continues, otherwise finishes with success.
So with the definition of "success" and "error" being user-provided (in the form of a function: Either[successValue, errorValue] => shouldContinue) it seems that retry and repeat are both specialisations of this runWithSchedule function (or however we call it). Maybe even retry is a specialisation of repeat?
I think the Backoff also makes sense for repeat, just the naming might need to be different - maybe simply Exponential? You might repeat something with exponentially increasing pauses, why not?
FixedRate is really what we currently call Delay.forever, no? We just might need two variants of this, one which would measure the delay between end-of-previous and start-of-next, and another one which would measure the delay between start-of-previous and start-of-next (that is, disregarding the time it takes to actually run the computation). But that would be useful for retries as well.
What we could have is use-case-specific DSLs for retries & repeats, using retry or repeat-specific vocabulary for naming the constructor methods, but under the covers yield Schedule instances.
Another thing to consider: it might be useful to linearly combine schedules. Each schedule would have to be combined with the number of repetitions it handles (0, 1, 2, ... infinity). Then, we might want to say: run schedule1, followed by schedule2, etc. The schedules would be applied basing on the current repetition number, as long as they are available. Then we could express things like "initial delay" (handles 0 repetitions, but includes a delay), or "retry 3 times, then with backoff indefinitely".

FixedRate is really what we currently call Delay.forever, no? We just might need two variants of this, one which would measure the delay between end-of-previous and start-of-next, and another one which would measure the delay between start-of-previous and start-of-next (that is, disregarding the time it takes to actually run the computation). But that would be useful for retries as well.

I wanted to distinguish these two options by putting them into separate Schedule variants, so that:

Delay remains as it was originally (sleep after previous operation ends)
FixedRate implements the other option, where we sleep [interval - (now - start time of prev operation)] or immediately (when it's <= 0)

Also, for now I created runScheduled which is called by repeat and retry, but it can be "reduced" to retry using repeat, but I think the first approach is more clean.

I think both delay strategies can be useful in both situations (repeat/retry). So ideally we want to be able to use both.

By design all the strategies can be used in repeat and retry

Right, in case maybe it would make sense to parametrise all of Delay, Backoff/Exponential, FixedRate with how the time between subsequent attempts is measured - either end <-> start, or start <-> start?

removed Schedule.FixedRate
added DelayPolicy as a parameter of runScheduled (decoupled from Schedule)

enum DelayPolicy:
  case SinceTheStartOfTheLastInvocation, SinceTheEndOfTheLastInvocation


Can't get to work on Chrome, Cypress 8.7.0

Hello, I encountered an issue where this plugin just doesn't work with provided instructions on the newer versions of Cypress Chrome. Could someone confirm that it works properly on the Cypress Chrome newer versions please?

This is working in my PR #37 with Chrome 95 and Cypress 9. What issues are you running into?


fix: Support 'generate_name' when saving template from guest

这个 PR 实现什么功能/修复什么问题:

是否需要 backport 到之前的 release 分支:
NONE

/area region
/cc @zexi

Codecov Report

Merging #6736 into master will increase coverage by 0.01%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##           master    #6736      +/-   ##
==========================================
+ Coverage    7.13%    7.15%   +0.01%
==========================================
  Files         648      648
  Lines      109488   109488
==========================================
+ Hits         7817     7830      +13
+ Misses     101072   101059      -13
  Partials      599      599

Flag
Coverage Δ

#aFlag
7.15% <0.00%> (+0.01%)
:arrow_up:

Impacted Files
Coverage Δ

pkg/compute/models/guest_template.go
2.28% <0.00%> (ø)

pkg/cloudcommon/validators/validators_cert.go
39.77% <0.00%> (ø)

pkg/appsrv/response.go
69.11% <0.00%> (+2.94%)
:arrow_up:

pkg/cloudcommon/cronman/cronman.go
53.95% <0.00%> (+5.11%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0d75932...27f746e. Read the comment docs.


Reset zoom button resets to startScale

For #361.

This sounds like something that could be merged upstream?


PartsUnlimited project is created with an empty repo

Since this week we have more and more occurrences in which the creation of a project using the "PartsUnlimited" template is successful but doesn't have the repository imported/created. It's apparently not happening all the time, but when I tried it just now it did not import the repo.
AB#3631

Hi @mkennten, Thank you for reaching us, could you please try it in production site

I was using this site before and after clicking on "Sign in" got automatically re-directed to https://azuredevopsdemogenerator-core.azurewebsites.net. Now that you provided that link it did not redirect me and the import worked successfully. Did you change something already? Can we use the URL without being re-directed?

We are migrating the application to Asp.Net Core and doing beta test with real-time user traffic https://azuredevopsdemogenerator-core.azurewebsites.net
But we didn't find any issue with importing repo for PartsUnlimited template. Could you please try again and let us know if the issue persists.
Thanks.

I tested the -core link again and the repo is still NOT imported (don't receive the mail, files are not there). It works with the production site, not with the -core one. And btw. it takes much longer using the -core site (up to 8 min) than the production site (2-4 min).

Is it happening only for PartsUnlimited template? Could you please try the SmartHotel360 template as well, if not working we will assign this ticket to the team.

No, I just learnt that it happens also for a number of students (10%) for other template, e.g. Whitesource lab

Thanks for more info. We will look into this.

Just tested SmartHotel360, same issue, no repo imported. It looks like the -core site only causes this issue.

For a while it looked like no forwarding to the -core site happened and everything seemed okay, but now I learnt from peers that they got forwarded and then the repo import still fails. In this case for the AKS template. What's the status?

Could you please verify the repo import is working in https://azuredevopsdemogenerator-core.azurewebsites.net/ ??

@mkennten - Status update... @aksh-h is actively working on this...  The -core site appears to currently be offline with production traffic going through the stable "legacy" deployment.  But we are working on determining the issues with the -core implementation and fixing them.  Thank you!

The issue has been resolved. Root cause for this issue was the configuration setting were wrongly updated.
Closing the issue.

Just tested it, it works as expected, repo is imported. Thanks!


Update description

Changed the name in the function description:

StartPointX, StartPointY;
EndPointX, EndPointY.

Changed the range of angles in the Angle function description.

Updated the description to reflect the change in the extension in the "header" file

Excellent 👍


Revert "MWPW-154363: Add param anaylitics sandbox"

Reverts adobecom/bacom#239

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 95.95%. Comparing base (229705f) to head (5a76b49).

Additional details and impacted files
@@            Coverage Diff             @@
##            stage     #242      +/-   ##
==========================================
- Coverage   95.98%   95.95%   -0.03%
==========================================
  Files          11       11
  Lines        1095     1088       -7
==========================================
- Hits         1051     1044       -7
  Misses         44       44

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Fluent trait methods

Why not make Trait API fluent?
Eg. in EntrustRoleTrait the attachPermissions method could look like
    public function attachPermissions($permissions)
    {
        foreach ($permissions as $permission) {
            $this->attachPermission($permission);
        }

        return $this; // <---
    }

I haven't found any similar issues. Am I the only one that finds this useful:
$role = Role::create($roleAttributes)->attachPermissions($permissions);
Will you do it or should I make a PR? Are there any downsides to this approach?

If you want i have a fork of this code and with some features solved made the PR and i could take a look

@KKSzymanowski the fluent api is already implemented in my code

Thank you very much. I will definitely take a look at your fork.


No debug output: Unsupported protocol "vscode-file:"

Does this issue occur when all extensions are disabled?: Yes

VS Code Version:
Version: 1.61.2
Commit: 6cba118ac49a1b88332f312a8f67186f7f3c1643
Date: 2021-10-20T00:21:59.656Z
Electron: 13.5.1
Chrome: 91.0.4472.164
Node.js: 14.16.0
V8: 9.1.269.39-electron.0
OS: Linux x64 5.11.0-38-generic
OS Version: Ubuntu 20.04

I tried debugging VS Code itself. But I don't get any output in the debug console. Only this error:
Could not read source map for vscode-file://vscode-app/home/max/repos/vscode/out/vs/base/browser/dompurify/dompurify.js: Unexpected 503 response from vscode-file://vscode-app/home/max/repos/vscode/out/vs/base/browser/dompurify/purify.es.js.map: Unsupported protocol "vscode-file:"
 INFO [FileService]: Using sandbox ready file system provider
[Extension Host] debugger listening on port 5870
 INFO [LocalProcessExtensionHost]: IExtensionHostStarter.createExtensionHost() took 871 ms.
 INFO [LocalProcessExtensionHost]: IExtensionHostStarter.start() took 26 ms.
[Extension Host] debugger inspector at chrome-devtools://devtools/bundled/inspector.html?experiments=true&v8only=true&ws=<IP_ADDRESS>:5870/8b30cbb9-60e9-40a6-b577-0e5ee91abacb

(I expect to see the lines printed with console.log and console.info in the VS Code instance being debugged.)
This seems to be similar to #125841, but #125841 has been fixed/closed already.
Steps to Reproduce:

Start VS Code (in my case: Codium)
Open folder with git repository of VS Code (in my case /home/max/repos/vscode/, commit 45fdad3a10d)
Click "Run and Debug"
Select "Launch VS Code"
Click "Start Debugging" (or press F5)

/gifPlease

Here's a gif:

Okay, thanks. The log line you see is fine. Where are the console.log's that you're expecting to see? I'm guessing they're being hit in a process other than the renderer, so when you debug only the renderer you don't see them.

When I add a line like this:
$ git diff
diff --git a/src/vs/workbench/contrib/preferences/browser/keybindingWidgets.ts b/src/vs/workbench/contrib/preferences/browser/keybindingWidgets.ts
index 0b996a9ea8e..274fb7bcad7 100644
--- a/src/vs/workbench/contrib/preferences/browser/keybindingWidgets.ts
+++ b/src/vs/workbench/contrib/preferences/browser/keybindingWidgets.ts
@@ -276,6 +276,7 @@ export class DefineKeybindingWidget extends Widget {
          this._chordPart = chordPart;
          dom.clearNode(this._outputNode);
          dom.clearNode(this._showExistingKeybindingsNode);
+         console.log(this._firstPart);

          this.keybindingLabelStylers.clear();

How/where can I see this console.log's output? (Of course, I tried changing some keybindings for provoking a logging output.)
How can I debug another component than the renderer?


Sep state vars

Separates colorMaps, model, transferFunction, etc. into their own state variables. This will make separating the app into different exports much easier but that's left for another PR

The reset button is moved up into Controls.jsx and now remounts the entire component when clicked. This resets the model's position in the scene

RE the useEffect and useCallback it still seems like this is the best way to handle it. If I do const [model, setModel] = useState({ ...DEFAULT_MODEL, ...modelProp }); for example, it won't re-fire when modelProp changes. It needs to be a setModel and in order to do that without constantly forcing a re-render it needs to be inside a useEffect.
The only simplification I could do is remove the function (getModel) from the useEffect dependency array. This would let me eliminate the usecallback step (making it a basic arrow function) but ESLint shows me a warning when the function is missing from the dependency array.
I'm learning on the fly with all this so I very well might not be looking at the best solution but I don't know any way to change the props from another component.

useEffect with the correct dependencies sounds pretty idiomatic to me.  You can put a comment on the line about the dependencies to tell eslint to ignore that error.  eslint is not all knowing 😉

Actually I think I found a better solution. Having the getter function inside the closure meant React was re-creating the function definition for each render. Pulling them outside of the closure means there's a single function definition, I just need to pass the prop in as a parameter since it's no longer in the same scope. Since the function will never change it's not actually a dependency for useEffect and, even better, the function will get called whenever the prop changes so I can use that return to update state. It's a similar solution as toAframeString in the other branch and is way cleaner.
I'm not sure what to call the return variable though. They're prepended with init right now but I'm not sure that's the clearest name


state: add vote_state

[!WARNING]
This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

#10
#9
#8
#7
#5
#11  👈
#4
#3
#2
#1
#6
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @buffalojoec and the rest of your teammates on  Graphite


Installing multiple simulators in one command?

I'd like to use xcversion to setup my CI nodes, but installing all of the simulators is currently a fairly manual process. My attempts to chain installs didn't work, and xcversion isn't able to install multiple sims at once. Anyone have a solution for this? Additionally, is there a way to authenticate once for multiple invocations?

Manually chaining install commands works fine, in an authenticated shell, it's just confusing due to the password prompt coming up even when authenticated (#245). It would still be great for xcversion to take an array of versions to install instead of just one at a time.


Show header in profile pages when personal details are loading

Details

Fixed Issues

$ https://github.com/Expensify/App/issues/24642
PROPOSAL: https://github.com/Expensify/App/issues/24642#issuecomment-1681164078
Tests

Logout and then login
Turn off network
Visit the personal details page settings/profile/personal-details and verify that it shows the loading spinner and the header is visible and clickable.
Visit the personal address details page settings/profile/personal-details/address from deep link and verify that it shows the loading spinner and the header is visible and clickable.
Visit the date of birth details page settings/profile/personal-details/date-of-birth from deep link and verify that it shows the loading spinner and the header is visible and clickable.
Visit the legal name details page settings/profile/personal-details/legal-name from deep link and verify that it shows the loading spinner and the header is visible and clickable.

Sidenote:
To visit the desktop deep link, right click > inspect element and add window.location = '{The url you want to visit}' and press enter, and to visit native mobile deep-link, use npx uri-scheme open "new-expensify://settings/profile/personal-details/address" --android, replace the path with the path you want to visit. For IOS, use --ios instead of --android

[ ] Verify that no errors appear in the JS console

Offline tests

Same as the "Tests" section above (but we need to make sure that personal details are not loaded after login)
QA Steps

Same as the "Tests" section above.

[ ] Verify that no errors appear in the JS console

PR Author Checklist

[x] I linked the correct issue in the ### Fixed Issues section above
[x] I wrote clear testing steps that cover the changes made in this PR

[x] I added steps for local testing in the Tests section
[x] I added steps for the expected offline behavior in the Offline steps section
[x] I added steps for Staging and/or Production testing in the QA steps section
[x] I added steps to cover failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[x] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)
[x] I tested this PR with a High Traffic account against the staging or production API to ensure there are no regressions (e.g. long loading states that impact usability).

[x] I included screenshots or videos for tests on all platforms
[x] I ran the tests on all platforms & verified they passed on:

[x] Android / native
[x] Android / Chrome
[x] iOS / native
[x] iOS / Safari
[x] MacOS / Chrome / Safari
[x] MacOS / Desktop

[x] I verified there are no console errors (if there's a console error not related to the PR, report it or open an issue for it to be fixed)
[x] I followed proper code patterns (see Reviewing the code)

[x] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick)
[x] I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
[x] I verified that comments were added to code that is not self explanatory
[x] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[x] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method

[x] If any non-english text was added/modified, I verified the translation was requested/reviewed in #expensify-open-source and it was approved by an internal Expensify engineer. Link to Slack message:

[x] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[x] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is approved by marketing by adding the Waiting for Copy label for a copy review on the original GH to get the correct copy.
[x] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[x] I verified the JSDocs style guidelines (in STYLE.md) were followed

[x] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[x] I followed the guidelines as stated in the Review Guidelines
[x] I tested other components that can be impacted by my changes (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar are working as expected)
[x] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[x] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[x] I verified that if a function's arguments changed that all usages have also been updated correctly
[x] If a new component is created I verified that:

[x] A similar component doesn't exist in the codebase
[x] All props are defined accurately and each prop has a /** comment above it */
[x] The file is named correctly
[x] The component has a clear name that is non-ambiguous and the purpose of the component can be inferred from the name alone
[x] The only data being stored in the state is data necessary for rendering and nothing else
[x] If we are not using the full Onyx data that we loaded, I've added the proper selector in order to ensure the component only re-renders when the data it is using changes
[x] For Class Components, any internal methods passed to components event handlers are bound to this properly so there are no scoping issues (i.e. for onClick={this.submit} the method this.submit should be bound to this in the constructor)
[x] Any internal methods bound to this are necessary to be bound (i.e. avoid this.submit = this.submit.bind(this); if this.submit is never passed to a component event handler like onClick)
[x] All JSX used for rendering exists in the render method
[x] The component has the minimum amount of code necessary for its purpose, and it is broken down into smaller components in order to separate concerns and functions

[x] If any new file was added I verified that:

[x] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[x] If a new CSS style is added I verified that:

[x] A similar style doesn't already exist
[x] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(themeColors.componentBG))

[x] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[x] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[x] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[x] If the PR modifies a component or page that can be accessed by a direct deeplink, I verified that the code functions as expected when the deeplink is used - from a logged in and logged out account.
[x] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[x] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.
[x] I have checked off every checkbox in the PR author checklist, including those that don't apply to this PR.

Screenshots/Videos

Web
Chrome:
https://github.com/Expensify/App/assets/68777211/abcd964e-8213-4b8b-bb53-b037808f75e7
Safari:
https://github.com/Expensify/App/assets/68777211/9b91294a-16d9-4169-b796-a2b8715fe79f

Mobile Web - Chrome
https://github.com/Expensify/App/assets/68777211/df6bddac-63cb-46d6-b2c6-227645a95c62

Mobile Web - Safari
https://github.com/Expensify/App/assets/68777211/d273de25-5c3c-43fa-9f15-82ecbd6fb0b8

Desktop
https://github.com/Expensify/App/assets/68777211/54b9b286-b2f0-4c65-a48d-48f2a7cbe3c3

iOS
https://github.com/Expensify/App/assets/68777211/1b9972e8-a08a-4e35-8373-e5dad9f0d9d5

Android
https://github.com/Expensify/App/assets/68777211/b0186a01-2c13-47d5-9fe7-54d60db25347

Reviewer Checklist

[ ] I have verified the author checklist is complete (all boxes are checked off).
[ ] I verified the correct issue is linked in the ### Fixed Issues section above
[ ] I verified testing steps are clear and they cover the changes made in this PR

[ ] I verified the steps for local testing are in the Tests section
[ ] I verified the steps for Staging and/or Production testing are in the QA steps section
[ ] I verified the steps cover any possible failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[ ] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)

[ ] I checked that screenshots or videos are included for tests on all platforms
[ ] I included screenshots or videos for tests on all platforms
[ ] I verified tests pass on all platforms & I tested again on:

[ ] Android / native
[ ] Android / Chrome
[ ] iOS / native
[ ] iOS / Safari
[ ] MacOS / Chrome / Safari
[ ] MacOS / Desktop

[ ] If there are any errors in the console that are unrelated to this PR, I either fixed them (preferred) or linked to where I reported them in Slack
[ ] I verified proper code patterns were followed (see Reviewing the code)

[ ] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick).
[ ] I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
[ ] I verified that comments were added to code that is not self explanatory
[ ] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[ ] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method
[ ] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[ ] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is approved by marketing by adding the Waiting for Copy label for a copy review on the original GH to get the correct copy.
[ ] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[ ] I verified the JSDocs style guidelines (in STYLE.md) were followed

[ ] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[ ] I verified that this PR follows the guidelines as stated in the Review Guidelines
[ ] I verified other components that can be impacted by these changes have been tested, and I retested again (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar have been tested & I retested again)
[ ] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[ ] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[ ] If a new component is created I verified that:

[ ] A similar component doesn't exist in the codebase
[ ] All props are defined accurately and each prop has a /** comment above it */
[ ] The file is named correctly
[ ] The component has a clear name that is non-ambiguous and the purpose of the component can be inferred from the name alone
[ ] The only data being stored in the state is data necessary for rendering and nothing else
[ ] For Class Components, any internal methods passed to components event handlers are bound to this properly so there are no scoping issues (i.e. for onClick={this.submit} the method this.submit should be bound to this in the constructor)
[ ] Any internal methods bound to this are necessary to be bound (i.e. avoid this.submit = this.submit.bind(this); if this.submit is never passed to a component event handler like onClick)
[ ] All JSX used for rendering exists in the render method
[ ] The component has the minimum amount of code necessary for its purpose, and it is broken down into smaller components in order to separate concerns and functions

[ ] If any new file was added I verified that:

[ ] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[ ] If a new CSS style is added I verified that:

[ ] A similar style doesn't already exist
[ ] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(themeColors.componentBG)

[ ] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[ ] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[ ] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[ ] If the PR modifies a component or page that can be accessed by a direct deeplink, I verified that the code functions as expected when the deeplink is used - from a logged in and logged out account.
[ ] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[ ] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.
[ ] I have checked off every checkbox in the PR reviewer checklist, including those that don't apply to this PR.

Screenshots/Videos

Web

Mobile Web - Chrome

Mobile Web - Safari

Desktop

iOS

Android

Hmm, I just remembered there is this merge freeze: https://expensify.slack.com/archives/C01GTK53T8Q/p1693347701566439 🤔

Asked here: https://expensify.slack.com/archives/C03TQ48KC/p1693505399925859
We will HOLD the merge as this is not fixing a critical issue.

🚀 Deployed to staging by https://github.com/aldo-expensify in version: 1.3.68-0 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

🚀 Deployed to production by https://github.com/Julesssss in version: 1.3.68-17 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
failure ❌

🕸 web 🕸
success ✅


PacMan Game

Created Pacman Game Using CPP.

please add labels hacktoberfest and hacktoberfest-accepted in this PR


Consider Setting Console CP to UTF-8 on Windows to enable UTF-8 Console Output

Expected Behavior
When I output UTF-8 text from a Gradle script, or a process that is launched from a Gradle script, this output should show correctly even in Windows terminals and console hosts.
Current Behavior (optional)
Currently, Gradle neither uses the Win32 API to write console messages to the current console (effectively enabling Unicode output without the use of codepages), nor does it set the current console codepage to UTF-8, leading to "garbled text" on many occasions.
Context
I have a command line tool that tries to output Emojis to enrich the user experience. I am running this from within a Gradle build, but due to Gradles daemon architecture, neither my tool nor my build logic can switch the console to UTF-8 mode on Windows, leading to Emojis being shown as:

The corresponding Win32 API function would be:
https://learn.microsoft.com/en-us/windows/console/setconsoleoutputcp
I.e.:
SetConsoleOutputCP(CP_UTF8); // CP_UTF8 is 65001

This feature request is in the backlog of the relevant team and is prioritized by them.

Thanks for taking the time to craft a reproducer and show different outputs with different setups.
Relates to:

#28959
#19952


Digi app v. 1.1.30 novel taxon name notes are not populated.

Describe the bug
For new names the notes field in the specimen table is not populated.
To Reproduce
Steps to reproduce the behavior:

Log in as NHMA or NHMD
Create a record with a novel name
For NHMA:
tab out of the 'Taxon number' field
See the Taxon name being deleted.

Expected behavior
When a name not existing in the taxonomy is submitted, it should:

Be created in the specimen table in the app DB
Name marked with a note/remark stating that the name is new: | Verbatim_taxon: [the new name]

Please complete the following information:

version 1.1.30
specify is not getting data flagging taxonomic names not already in the taxonomic spine.

This is no longer required as per #457


llama 3.1 405B fp8 support

I have been staging some updates testing the tgi-gaudi software with llama 405B fp8, i am waiting for habana optimum to approve the PR, and then I will submit a pr for huggingface/tgi_gaudi and will submit a PR for TGI in the microservices.
I got it running on xeon with llama_cpp (which is what ollama is based on) at 1 tok/s on sapphire rapids, but am going to test speculative decoding for llama 3.1 8b, which should improve performance 10-20 times depending on how many tokens can be completed by the draft model. However ollama is broken and that will need to be investigated further.

I have thoroughly gone through all of the examples and interfaces with regards to
optimum-habana
Intel Neural Compressor (3.0 and 2.4)
tgi-gaudi
Currently the situation is that the libraries are in a state of disrepair for everything but bf16, as a result of a lack of unit testing, integration testing, regression testing. The examples do not work because they were designed for previous versions of libraries that are no longer working with the other versions of the libraries, the only thing that does work with regards to quantization is compile time quantization, which does not actually reduce the number of devices needed to run a model, but does increase the inference speed of the models, however with llama 3.1 405b it is currently impossible to run it on a single node, but only because the software packages are not being maintained in a functioning state.
I have spend 3 days so far on this endeavor, and I am unwilling to take the time needed to become a maintainer of those libraries, even if I do want to reduce hallucinations in my language modeling tasks, as I have been getting asked to complete my AGPL edge oriented mlops infrastructure package more quickly by @jaanli so that he can migrate away from google tpu cloud.
https://github.com/endomorphosis/ipfs_transformers/issues/1#issuecomment-2282238714

Thanks so much @endomorphosis and on behalf of @onefact! Giving a talk on Thursday if it's possible to demo any edge models at https://duckdb.org/2024/08/15/duckcon5
Even just a encoder-only small transformer like what I did before: https://arxiv.org/abs/1904.05342 (let me know if you need HF links :)

Thanks so much @endomorphosis and on behalf of @onefact! Giving a talk on Thursday if it's possible to demo any edge models at https://duckdb.org/2024/08/15/duckcon5
Even just a encoder-only small transformer like what I did before: https://arxiv.org/abs/1904.05342 (let me know if you need HF links :)

I have no idea what hardware you are running it on.

Ah yes, sorry - iPhone pro max 15 with latest firmware.

https://github.com/HabanaAI/vllm-fork/pull/144
I am going to continue with attempting to get llama 405b with speculative decoding with llama 8b working, and process some Wikipedia datasets and embeddings.


cache checks

Hello,
is it possible to cache checks? I am attempting to move from using my own s3 cache to this to simplify a bit the setup but I am having an issue because the result of running the checks doesn't seem to be cached. Because some of my tests requires a bit of infrastructure I was doing something like the following:
    - name: "Verify if we need to build"
      id: verify-build
      run: |
        export check_dry_run_output=$(nix build \
                --dry-run \
                --json \
                --print-build-logs \
                .\#checks.$(ARCH)-$(OS).$(NAME))
        export drvPath=$(echo "$check_dry_run_output" | grep -oE '"out":.*"' | awk -F\" '{ print $4 }')
        nix store verify --no-trust --store s3://..... $drvPath \
          && export BUILD_NEEDED=no \
          || export BUILD_NEEDED=yes
        echo BUILD_NEEDED=$BUILD_NEEDED >> $GITHUB_OUTPUT

Then in the rest of the workflow I can just use BUILD_NEEDED to skip unnecessary tests.
However, this doesn't seem to work after moving to use this store instead of s3, I suspect because the check derivation isn't being cached.

Ok, I feel a bit dumb now. There was so much "magic" here that it didn't occur to me I could just keep copying to this store myself like I was doing before... Apologies for the noise.


Use spotless to run diktat

diktatCheck -> spotlessCheck
diktatFix -> spotlessApply
Current limitations of spotless: cannot enable SARIF reporting, no control of JVM version since Spotless doesn't use forking.

May be Nariman will help you to finish it? @nulls