Example 5bis fails

Creating an issue for this for notes and discussion.
Example 5bis fails in PubGrub.unitPropagationLoop by PartialSolution.relation priorCause updatedModel.partialSolution returning Satisfies instead of expected AlmostSatisfies.
Here partialSolution is
foo   NOT  1
bar   NOT  [ 2, 3 [
baz   IS   [ 1, 2 [
foo   IS   [ 1, 2 [
root  IS   1

and priorCause is
{ root: IS 1, baz: NOT none }

Manual checking confirms that partialSolution indeed satisfies priorCause, so one of these is wrong.

Thanks for the creation of the issue. I suppose you figured out that the binary tree displayed in the debug output for priorCause is the chain of derivation leading to the actual incompatibility creating the issue.

Yes it's the "Derivation Graph" mentioned at PubGrub documentation[1]. I'm still reading this while comparing it to the implementation.
While current debug output is quite complete, it would be more useful if it used exact same terms as the PubGrub documentation. For example terms changed, package, incompatibility etc. in "Unit Propagation" section.
[1] https://github.com/dart-lang/pub/blob/master/doc/solver.md

PubGrub documentation says that

The process of returning the partial solution to a state where the incompatibility is no longer satisfied is known as conflict resolution.

So error here is in conflictResolution which doesn't return correct value. I see it has some notes about things which are guaranteed, but those are not checked, so adding more error reporting there would be first step to do.

Adding this debug check makes Example 5bis fail earlier, and also makes Examples 3 and 4 fail: https://github.com/malaire/elm-pubgrub/commit/02dc6fc403974fef10f36b9eccc67ba9ab7f6955
p.s. I havn't yet learned how to make pull requests in GitHub.

:)

Why can't official guide tell simple solution for making PR, but instead has longwinded 8-step process which requires base branches, head forks and compare branches?
Anyway, here's the PR (https://github.com/mpizenberg/elm-pubgrub/pull/2), and I think I'll continue debugging with Example 3 which now fails also and is simpler than 5bis.

So I've done a full re-read of the PubGrub algorithm. The last step of the conflict resolution is to return the current incompatibility of the loop and to backtrack the partial solution. That incompatibility has been obtained through successive "priorCause" computations and should be named "rootCause" to stick to PubGrub namings. If we believe PubGrub unit propagation explanation, the returned root cause should be guaranted to almost satisfy the backtracked partial solution. I'll probably want to verify why that is, but let's assume that is indeed the case (they should know ^^). In that case, these are the things that could go wrong.

[ ] findSatisfier is not correct?
[ ] findPreviousSatisfier is not correct?
-> for this one, make sure about what happens if we search to the right (older)
and reach the last assignment (first root derivation before root decision).
[ ] Are we in the edge case when previousSatisfier == satisfier?
[ ] I don't understand point of parenthesis text in the phrase:
"If incompatibility is different than the original input, add it to the solver's incompatibility set. (If the conflicting incompatibility was added lazily during decision making, it may not have a distinct root cause.)"
Does this refer to the incompatibilities with no available versions?
[ ] backtrack is not correct?
[ ] priorCause is not correct?
Does "union of the terms in incompatibility and ..." means something different
than how its implemented here?
[ ] not (satisfier \ term) is incorrect?
[ ] Term.satisfies is incorrect?
[ ] Term.union is incorrect?
[ ] (related to 4 previous checks)
When building priorCause last term in conflict resolution,
we should just build {q, r, t1 union t2} where t1 and t2 are the terms referring
to the package in the "satisfier" assignment, meaning t1 = term (in incompatibility)
and t2 = not satisfier.
-> Actually shouldn't this be t2 = satisfier? if we are following
the rule of resolution saying that {q, t1} and {r, t2} => {q, r, t1 U t2}

I'll want to check those things manually and maybe add unit tests.
There are also few remarks:

[ ] While searching for satisfier, we could go through assignments one by one
and only recheck for the package of the term that was just added/removed
once we are making dichotomic steps smaller than the size of the incompatibility.
This should be correct since there is no change regarding other packages.
This should also be faster if the incompatibility is big
(which I guess is not often since most are just 2 elements).
[ ] Incompatibilities added when backtracking could be merged smartly?
At least for the "no version" case? https://github.com/dart-lang/pub/issues/2485#issuecomment-640886792

About {q, r, t1 U t2}: Documentation says

In fact, we can generalize this: given any incompatibilities {t1, q} and {t2, r}, we can derive {q, r, t1 ∪ t2}, since either t1 or t2 is true in every solution in which t1 ∪ t2 is true. This reduces to {q, r} in any case where not t2 ⊆ t1 (that is, where not t2 satisfies t1), including the case above where t1 = t and t2 = not t.

The way I understand this is that {q, r, t1 ∪ t2} is just part of the explanation - it's never actually used in the algorithm. Instead in algorithm {q, r} is used, but only when not t2 satisfied t1.

Yes it is actually used, that's the else block here: https://github.com/mpizenberg/elm-pubgrub/blob/master/src/PubGrub.elm#L331
It's just that the naming of the algorithm calls it "priorCause" even before it is actually a correct prior cause, which I called newIncompat later because I was just out of names and didn't fully understand it yet: https://github.com/mpizenberg/elm-pubgrub/blob/master/src/PubGrub.elm#L346

Ah, true. priorCause is initially set to {q, r}, but the third term can be added a bit later.
Although the added term isn't t1 U t2 but corresponding not (not t2 \ t1).

But elm-pubgrub is adding the union instead of not (satisfier \ term) as algorithm says?
https://github.com/mpizenberg/elm-pubgrub/blob/3a36cd3ed2dd4c0278fff6316b57b9411c35b1cf/src/PubGrub.elm#L344

The logic is explained just above I believe this is correct
https://github.com/mpizenberg/elm-pubgrub/blob/3a36cd3ed2dd4c0278fff6316b57b9411c35b1cf/src/PubGrub.elm#L339-L341

To expand a bit the explanation. I do not have the \ Set operator but I have the union and intersection operations on ranges Sets, so with A for satisfier and B for term we have:
A \ B == intersection A (not B)
and
union (not A) (not B) == not (intersection A B)
so
not (A \ B) == union (not A) B

Yes, even the algorithm description says that they correspond to each other. But algorithm description clearly says that the added term is not (satisfier \ term) and not the union. So elm-pubgrub is wrong here.
From https://github.com/dart-lang/pub/blob/master/doc/solver.md#conflict-resolution:

If satisfier doesn't satisfy term, add not (satisfier \ term) to priorCause.

Note: not (satisfier \ term) corresponds to t1 ∪ t2 above with term = t1 and satisfier = not t2, by the identity (Sᶜ \ T)ᶜ = S ∪ T.

This is the second place I've seen where elm-pubgrub deliberately implements the algorithm incorrectly.
I see no reason to debug elm-pubgrub further as it doesn't even attempt to implement Pubgrub correctly.

I'm sorry if this frustrates you but I'm trying to do the correct implementation (with the time I can put in it) and to strictly follow the PubGrub algorithm, the explanation above that I'm re-adding below seems correct to me so I am actually adding not (satisfier \ term). If you've seen another place delibarately different than PubGrub, please do tell.
A \ B == intersection A (not B)
and
union (not A) (B) == not (intersection A (not B))
so
not (A \ B) == union (not A) B

If you've seen another place delibarately different than PubGrub, please do tell.

The other place is about expanding/merging version ranges: https://github.com/mpizenberg/elm-pubgrub/issues/3#issuecomment-639600678

... the explanation above that I'm re-adding below seems correct to me so I am actually adding not (satisfier \ term) ...

For one thing you are adding potentially POSITIVE term while algorithm clearly says to add NEGATIVE term.
Also doing things differently than what is said in algorithm makes debugging a lot harder as it's not possible to compare the behavior of your implementation to the exact steps mentioned in documentation.

For one thing you are adding potentially POSITIVE term while algorithm clearly says to add NEGATIVE term.

This isn't sayed anywhere, or it may be your interpretation that differs from mine. I don't see a reason for satisfier or term to be positive terms. Maybe that's my interpretation that's wrong and that's why having a second view is great. satisfier and term could very well be negative terms, and in that case not (satisfier \ term) is positive.
As for the equivalent code in the actual dart pub code base, it turns out the Set complement operator is called difference there and is implemented as follows (exactly how I did).
https://github.com/dart-lang/pub/blob/master/lib/src/solver/term.dart#L149

Also doing things differently than what is said in algorithm makes debugging a lot harder as it's not possible to compare the behavior of your implementation to the exact steps mentioned in documentation.

Sorry about that, I thought the comment just above in the code would suffice to follow easily. Obviously things need to be different between an imperative description and a functional implementation. I didn't thought that line would be much trouble compared to the structural changes to the algorithm required by the Elm language.

The other place is about expanding/merging version ranges: #3 (comment)

I can understand if you consider this deliberately different. Actually the description of the PubGrub algorithm leaves a lot of grey areas where I just have to make some choices, like version choosing heuristics and other things. It also indicates some things as being uneeded for correctness but good for performances. I interpreted this explanation as such one of those performance-only improvements, especially given that no such collapse is part of the algorithm steps below the textual explanation.

Pubgrub collapses identical dependencies from adjacent package versions into individual incompatibilities. This substantially reduces the total number of incompatibilities and makes it much easier for Pubgrub to reason about multiple versions of packages at once. For example, rather than representing foo 1.0.0 depends on bar ^1.0.0 and foo 1.1.0 depends on bar ^1.0.0 as two separate incompatibilities, they're collapsed together into the single incompatibility {foo ^1.0.0, not bar ^1.0.0}.

Since I have a limited time implementing all this, I've reduced things to what needs to be done, and added some TODOS in the code base for those as in that example you point out:
https://github.com/mpizenberg/elm-pubgrub/blob/master/src/Incompatibility.elm#L190
But again, if this is a misinterpretation on my part, you're right and that's why I've added your point to the things to check in this issue (see "absence of smart merging" there: https://github.com/mpizenberg/elm-pubgrub/issues/1#issuecomment-640913440)
Anyway, I just thought that your comment was a bit harsh and wanted to let you know that it hurt me. If you do not feel like helping anymore I totally understand.

When I mentioned following the steps, I meant the steps in Examples section.

But again, if this is a misinterpretation on my part, you're right and that's why I've added your point to the things to check in this issue (see "absence of smart merging" there: #1 (comment))

Can you confirm whether relation-checking at any point has access to the list of all available versions? Because if not, then current implementation of elm-pubgrub can't work correctly, as I said in the other thread (https://github.com/mpizenberg/elm-pubgrub/issues/3#issuecomment-639641827).

As for the equivalent code in the actual dart pub code base, it turns out the Set complement operator is called difference there and is implemented as follows (exactly how I did).

I was wrong here then. I thought it was important when algorithm description clearly made the point that different term needs to be used instead of union.

Can you confirm whether relation-checking at any point has access to the list of all available versions? Because if not, then current implementation of elm-pubgrub can't work correctly, as I said in the other thread (#3 (comment)).

It doesn't. And it appears that pub also doesn't have access at that time:
https://github.com/dart-lang/pub/blob/master/lib/src/solver/version_solver.dart#L156
Then I suppose this means that incompatibilities should be more descriptive and collapsing is needed? Merging a new incompatibility (Incompatibility.merge and in PubGrub.backtrack that should use Incompatibility.merge) is a place where this could be done I suppose. I didn't take the time yet to check that since I wanted to first get a better overview of the algorithm.
I'll take some time next weekend on specifically this issue https://github.com/mpizenberg/elm-pubgrub/issues/3#issuecomment-639641827 that you pointed out.

I'll expand on my understanding of bar any vs. bar 1.0.0 issue if you didn't understand what I think is wrong here.
Basic question in conflict resolution is whether certain term satisfies another term.
IN ALGORITHM
In Example 3 the question is whether bar ^1.0.0 satisfies bar any.
In other words the question is: If bar is 1.0.0 <= x < 2.0.0 is true, does that mean bar is any version is always true? Clearly answer is YES, because if bar is a version within range 1.0.0 <= x < 2.0.0 then bar does have a version.
In terms of sets we can say that bar ^1.0.0 is SUBSET of bar any.
IN ELM-PUBGRUB
elm-pubgrub uses bar 1.0.0 instead of bar any. Let's see how this works out:
The question now is whether bar ^1.0.0 satisfies bar 1.0.0.
In other words the question is: If bar is 1.0.0 <= x < 2.0.0 is true, does that mean bar is version 1.0.0 is always true? Clearly answer is NO, because if bar is a version within range 1.0.0 <= x < 2.0.0 then bar could be for example 1.5.0 which is not 1.0.0.
In terms of sets we can say that bar ^1.0.0 is SUPERSET of bar 1.0.0.
THE BUG
So by deciding to use bar 1.0.0 instead of bar any, elm-pubgrub arrives at completely wrong conclusion IF it only uses set operations to check for set relations.
HOWEVER if elm-pubgrub uses list of available versions when checking set relations,
then and only then, can it conclude that in Example 3 bar ^1.0.0 actually is same as bar 1.0.0 because there is only one version of bar, and so answer is YES.

It doesn't. And it appears that pub also doesn't have access at that time:

Of course not, it is only needed when not merging/expanding version ranges.

And it appears that pub also doesn't have access at that time:

pub doesn't need to because it would only be needed when not merging/expanding version ranges.

Note that bar ^1.0.0 above comes from dependencies, so it's range both in pub and elm-pubgrub.
So stated another way:
in pub:
termFromDepencencies: bar ^1.0.0
termFromPickedVersion: bar any
in elm-pubgrub:
termFromDepencencies: bar ^1.0.0
termFromPickedVersion: bar 1.0.0
Problem:
However you implement relations-checking, it must conclude that in context of Example 3, termFromDepencencies satisfies termFromPickedVersion.
In case of pub this can be done using only set operations, without context.
In case of elm-pubgrub this can't be done using only set operations, without context, but needs to use context (i.e. the set of available versions).
p.s. It's not actually relevant that range is any. This issue would arise also with smaller range than any which is still superset of ^1.0.0.

You could also say that pub takes the context of other available versions into account when merging/expanding picked version to a range.
elm-pubgrub doesn't take this context into account at that point, so it must take that context into account at some other point. Currently it doesn't, which is why relation checking, and in extension conflict resolution, doesn't work.
While this was from Example 3, I believe it's highly likely that this is also behind this issue of Example 5bis failing.

I'm pretty confident I've identified the issue. It has to do with prior cause computation but not what we thought on collapsing incompatibilities.
First the issue you raised is actually not a problem it turns out. In fact the step-by-step examples are a little bit idealized to be more easily readable. In practice, there is another kind of incompatibility in the algorithm that is left out of the explanations for matters of simplicity. These are "noVersion" incompatibilities and are mentioned in the detailed algorithm in the following decision making step:

If there is no such version, add an incompatibility {term} to the incompatibility set and return package's name. This tells Pubgrub to avoid this range of versions in the future.

In pub code this appears here: https://github.com/dart-lang/pub/blob/master/lib/src/solver/version_solver.dart#L380. In elm-pubgrub this corresponds to the Unknown variant of an incompatibility kind that should be renamed to NoVersion (or better, there should be some changes in the Incompatibility type).
What's interesting is that a non-optimal incompatibility (such as { bar 1, foo not (1 <=v < 2) }) will later trigger search for a version of bar in the range 1.0.1 <= v < 2 that is the rest of the search space. At the time of picking a version, which has access to packages and versions it will realize that no such version exist. As such, a new incompatibility will be generated of the form { bar : the rest of the search space }. This is what happens in the debug logs here:
   bar: 1.0.0 <= v < 2.0.0, foo: Not ( 1.0.0 <= v < 2.0.0 )  <<<  derived from:
      bar: 1.0.0, foo: Not ( 1.0.0 <= v < 2.0.0 )  <<<  from dependency of bar at version 1.0.0
      bar: 1.0.1 <= v < 2.0.0  <<<  from unknown reason ...: ""

Now onto the issue.
Remember the question whether or not the satisfier term has to be added to the initialization of the prior cause with both incompatibilities without their terms related to satisfier package? It turns out the same should be done for any package. Let me develop that. In example 5bis, after getting through the transitive bar dependency, the algorithm realizes that it will not be possible. At that point the following detailed logs for prior cause computation give:
previousLevel == satisfierLevel: ""
   satisfier bar Not ( 2.0.0 ): ""
   cause
      bar: 2.0.0, baz: Not ( 3.0.0 <= v < 4.0.0 )  <<<  from dependency of bar at version 2.0.0: ""
   incompat
      bar: 2.0.1 <= v < 3.0.0, baz: Not ( 2.0.0 <= v < 3.0.0 )  <<<  derived from:
         bar: 2.1.0, baz: Not ( 2.0.0 <= v < 3.0.0 )  <<<  from dependency of bar at version 2.1.0
         bar: [ 2.0.1, 2.1.0 [  [ 2.1.1, 3.0.0 [  <<<  from unknown reason ...: ""
   priorCause
   bar: 2.0.0 <= v < 3.0.0, baz: Not ( ∅ )  <<<  derived from:
      bar: 2.0.0, baz: Not ( 3.0.0 <= v < 4.0.0 )  <<<  from dependency of bar at version 2.0.0
      bar: 2.0.1 <= v < 3.0.0, baz: Not ( 2.0.0 <= v < 3.0.0 )  <<<  derived from:
         bar: 2.1.0, baz: Not ( 2.0.0 <= v < 3.0.0 )  <<<  from dependency of bar at version 2.1.0
         bar: [ 2.0.1, 2.1.0 [  [ 2.1.1, 3.0.0 [  <<<  from unknown reason ...: ""

Here the satisfier package is bar so the rest of the two incompatibilities used for priorCause initialization are
{ baz: Not ( 3.0.0 <= v < 4.0.0 ) }
and
{ baz: Not ( 2.0.0 <= v < 3.0.0 ) }

And the union of those to terms (logical "or") gives "not none". Actually, it turns out "not none" is always satisfied. Indeed it is satisfied if no version is selected (because it's negative) and it is satisfied if any version is selected. Instead, just like for the satisfier term that we do not add when not t1 is included in t2, we should not add a term to an incompatibility when the union is "not none". This is actually exactly the same thing, just expressed differently (here we could say that not (not (3 <= v < 4)) which is 3 <= v < 4 is included in not (2 <= v < 3)).
Because we do not do that, we have the incompatibility {bar: 2.0.0 <= v < 3.0.0, baz: Not ( ∅ )} in our set instead of just {bar: 2.0.0 <= v < 3.0.0}. And that incompatibility propagates later to {root: 1.0.0, baz: Not ( ∅ )} just before reaching the impossible state. Indeed the incompatibility {root:  1.0.0, baz: Not ( ∅ )} is a disguised terminal case, that would have been detected by the first condition of conflictResolution if it was in its canonical form {root: 1.0.0}.
if Dict.isEmpty (Incompatibility.asDict incompat) || Incompatibility.singlePositive root incompat then

I'm very confident that this will solve that issue. I just started to add notes to the code for places that need to change. Next weekend I can't work on this, the weekend after it all should be solved. Then there will only be error reporting left :)

Actually the fix is pretty trivial, I just replaced the Incompatibility.fuse code by
fuse : String -> Term -> Term -> Incompatibility -> Incompatibility
fuse name t1 t2 incompatibility =
    let
        termUnion =
            Term.union t1 t2
    in
    if termUnion == Term.Negative Range.none then
        incompatibility

    else
        insert name (Term.union t1 t2) incompatibility

And it works on example 5bis, i.e. it stops and identify that root package can't be selected before reaching any impossible state.

Also PS, there was a bug in Term.union. It was returning a positive term instead of a negative term when we have one of each. I've fixed it by union t1 t2 = not ( intersection (not t1) (not t2)). I'll also fix that properly in two weekends.

wow, this pubgrub is really tricky endevour. I guess this should not be too suprising as it is NP-hard. Is very impressive the progress made so far :+1:

Solved in https://github.com/mpizenberg/elm-pubgrub/pull/5

I should just stick with the math I know. This algorithm is far too complex for me.


feat(crud): compute and save screenshot diff

What kind of change does this PR introduce?

Add 2 columns in screenshot table to store png_diff_data and png_mask_diff_data.

CRUD now supports calculation and save screenshots diff data on the flight.

Summary

This PR solves https://github.com/OpenAdaptAI/OpenAdapt/issues/348. The main logic behinds the change is to introduce the function to calculate and then eventually store png_data difference between 2 linked screenshots. Motivation for the design:

2 columns that store png_diff_data and png_mask_diff_data is BLOB/binary to conform with existing png_data.
However, the Screenshot.diff and Screenshot.diff_mask are kept as PIL Image for convenience of usage.

Side effects:

If called, the calculation of screenshot diff is resource intensive for first time calculation, as a pairs of linked screenshot are compared, then converted to binary and store in database.
The size of the storage significantly increases. As of now, png_data is the column that consumes the most space.
It will makes space-storage worst, which is already limited by the fact that the max length of recording is ~1 minute for now.
However, those concerns are mitigated by the fact that precompute_diff flag is opt-in an can be used only for case where optimization is required.

Checklist

[x] My code follows the style guidelines of OpenAdapt
[x] I have performed a self-review of my code
[x] If applicable, I have added tests to prove my fix is functional/effective
[x] I have linted my code locally prior to submission
[x] I have commented my code, particularly in hard-to-understand areas
[x] I have made corresponding changes to the documentation (e.g. README.md, requirements.txt)
[x] New and existing unit tests pass locally with my changes

How can your code be run and tested?
To activate screen diff storage, set SAVE_SCREENSHOT_DIFF environment variable to True
export SAVE_SCREENSHOT_DIFF=True
python -m openadapt.replay DemoReplayStrategy

This feature can be activated by an environment variable Flag. Discussion in the PR.

The first run should take long time as the screenshot diffs are calculated and stored.
The second run should be instant, as the diff are only loaded from existing database.

Diff stored as BLOB in database:

Other information
Your Hackernews message brought me here, knowing that you folks are seeking paid-freelancers for this project. I am interested as the codebase looks great and the core features are extremely interesting.
Please reach out to me via email at<EMAIL_ADDRESS>if you need a hand on this project.

Thanks @LaPetiteSouris !
Please grab some time on my calendar: https://www.getclockwise.com/c/richard-abrich/quick-meeting


CORE-18535: Update event mediator to remove shared mutable state and remove dead code

Problem description
As part of the flow retry and timeout enhancements, any timed out synchronous event should cause processing of the corresponding events to fail and the state to be marked as failed. At a later point, it will additionally be required to time out the event processing in the short running task if this goes on too long and also mark the state as failed. These two operations require modification of the same state which is currently being processed in different threads. Making these changes in the existing code will leave a situation where the change is difficult to test and could fail in some edge cases.
Solution
Refactor the event mediator to do the following:

In the event processor, return an object containing the new state, any asynchronous events, and whether the state operation is a create/update/delete
In the consumer processor, merge all the event processor results together and handle accordingly.
Remove the ConsumerProcessorState
Ensure Metadata is merged in StateManagerHelper rather than overwritten.
Eliminate dead code.

This change also adds the proposed shared metadata key, called FAILED_STATE. This is introduced in the API module so that clients using the state manager to inspect states it owns can use it to identify anything that has failed.
Note that this change does not include the work to fail processing if the client times out - this will come in a follow up PR.

build e2e

build e2e


As a user I can add/edit a task's starttime/endtime without specifying the time

default to 12:00am

Should support "today" or "tomorrow"

So should we let the program know what day today is?
Currently I just set a specific datetime as a default.
It's in EditCommandparser and AddCommand.

yes the program needs to know what today is

Date date = new Date() gets current time

Closing as per Andy's PR

default to 12:00am of the day specified

overwriting tags removes s/ e/ too - this is a bug

@pengchengan can we close this


Shorten simplification warning and error messages

When equation application fails with TooManyIterations or another data constructor of EquationFailure, we just show the data constructor, log it as a warning or return as error response (whichever is appropriate). However, it is usually only the data constructor tag (i.e. TooManyIterations) and not its arguments (usually huge terms) that is useful.
After:
[Warn#proxy] Problem with simplify request :  Aborted - TooManyRecursions [Term (TermAtt..............................................................................................................goes on forever.............................................

After:
[Warn#proxy] Problem with simplify request :  Aborted - TooManyRecursions [Term (TermAtt...truncated

@goodlyrottenapple because we need to then specifically handle every error case and have specific fragile code that prints the constructors.


initial secretmanager-retrieve-secrets

What does this PR do?

Motivation

Checklist

[ ] The attack technique emulates a single attack step, not a full attack chain
[ ] We have factual evidence & references that the attack technique was used by real malware, pentesters, or attackers
[ ] The attack technique makes no assumption about the state of the environment prior to warming it up

stale


API for removing all VNFRs of a NSR

In some cases, there is a need to remove all VNFR from a NSR. It should be possible to do that with a single API call.

Possibly a duplicate of #81?

Indeed.


Support WebHosting Certificate Store

Currentyl we are importing our IIS Certificates into the WebHosting store (which is also used, when requesting a certificate from IIS Console).
The WinRM IIS Web App Management currently support only the Personal Store (MY) which ist hardcoded in AppCmdOnTargetMachines.ps1

That's an issue for us too. IIS uses/supports both stores, so the desired one should be selectable. According to this IIS documentation page, the WebHosting store is the preferred one when using SNI because of scalability:

The Web Hosting store works like the Personal store, so all of the existing tools to import and export certificates work the same way. The key difference between Web Hosting store and Personal store is that Web Hosting store is designed to scale to higher numbers of certificates.

So it would be really important to support this out of the box.


Game refuses to load with mod installed

I'm running 1.19 with Forge 41.0.110; whenever I enabled bdlib and Factorium, my game refuses to launch, stating that bdlib and Factorium "failed to initialize."  I've got Bdlib <IP_ADDRESS> and Factorium <IP_ADDRESS>; I can upload the log if needed.
Other mods:
Advanced Chimneys <IP_ADDRESS>
Catalogue 1.6.3
Craftable Horse Armor and Saddle 1.9
Configured 1.5.5
Elevator Mod 1.8.6
ForgeEndertech <IP_ADDRESS>
Journeymap 5.8.5rc5
JEI <IP_ADDRESS>
Macaw's Bridges2.0.3
ScalableCatsForce 2.13.8 build 6
(Copied/pasted from my comment on Curseforge)

You need newer versions for forge newer than 41.0.64
Factorium <IP_ADDRESS> and BDLib <IP_ADDRESS> are the current latest versions.
Lots of breaking changes around that forge version so will likely need to update other mods too


Server Selection drop down not showing in Opera & Canary

The Server Selection dropdown does not appear in opera browser and in chrome canary (on mac).
The dropdown is hidden behind the big webmin and virtualmin tabs on top left.
Please check the image for reference.

What theme version are you running? It's been fixed a long time ago.

Oh! thanks for responding. I just checked that it never informed me to upgrade, as my virtualmin / webmin versions are not updated too.
I am confused, how can i update virtualmin/webmin to latest versions?
I am running webmin 1.90 and webmin 6.05, and everything else is updated. But no option / way to update them?


Added IRC nick colors

This greatly increases the readability in IRC channels (at least when using irssi). "npm run coverage" could still fail, couldn't get it to work for some magical reason.

Awesome! If you look at the output of npm run coverage you can see what tests are failing, which at the moment are these two:
https://github.com/reactiflux/discord-irc/blob/master/test%2Fbot.test.js#L67
https://github.com/reactiflux/discord-irc/blob/master/test%2Fbot.test.js#L114
If you update those test cases to include the correct color it should work. Shout out if you need any help :)

Oh, I didn't notice that I can click those tests open.
I have absolutely no experience with tests, and since it seems to be impossible to install this by cloning the repo, I have no way to do this with trial and error.

Here's how to work on the tests after cloning the repository:
# In the repository folder (after git cloning):
$ npm install
$ npm test

If you don't have time to look at it I can fix the tests for you, just thought you might want to experiment with it yourself.

Yeah, I really have no time to learn tests now. I'll learn by seeing your fix. :)

Updated the failing tests and merged it into master now: https://github.com/reactiflux/discord-irc/commit/5869801d41d978ef7f04634fefd0fab5fad5229a
Released version 0.4.0 with the changes. Thank you for the pull request!


Add Jetty

https://github.com/gazebo-tooling/release-tools/issues/1186

I think the test failed as this depends on https://github.com/gazebo-tooling/gazebodistro/pull/181


Snapshot compare fails when simulator running on external monitor.

I have a number of snapshot tests that will successfully complete multiple times when run on the same display, but fail when run on a different display. In my case, when the initial test snapshots are recorded while the simulator is running on my 5K LG external monitor then subsequent runs will pass.  However, if I disconnect the laptop from the external monitor and try to run the tests again they will fail.  Below is an example set of images showing the failed comparison.
Reference Image:

Test Image:

Here is what Image Magick shows as the difference when comparing the images:

It looks like an anti-aliasing issue with how the rounded rectangle is rendered.
Any suggestions on how to work around this?  Is there some setting I'm missing when capturing the snapshots?

@BennettSmith There are a number of things that can cause images to render slightly differently, and this is unfortunately one of them. The best you can do is always run snapshot tests on a consistent environment, or change the precision argument on the snapshot strategy to allow for some pixel variation.
Going to close this out as it isn't an underlying bug with the library. If you'd like to continue the discussion, feel free to open one on the forum!

The issue is, adding (changing) precision makes the test take x6-x10 times longer, that means more pipeline costs.
Furthermore, we cannot adjust the pipeline logic (display) if it has at all any displays.


Allow images to be generated without width and height attributes

Issue Summary
Allow images to be rendered without width and height attributes when calling Rendition.img_tag.
Responsive images (that use srcset and sizes attributes) should not have width and height attributes.  All examples I have found exclude these attributes, and it makes sense to exclude them as the browser will be selecting the best size image to render on the fly. There is a template tag that we have used on various projects at Torchbox (and which would be useful to get into Wagtail core) which relies on calling Rendition.img_tag to render a responsive image with srcset and sizes attributes. But Rendition.img_tag always returns a width and height. A change was made in this pull request to allow attributes to be overridden, but you can't remove them all together.
I have not included the other parts of the bug report template as this isn't a bug so much as change request.

Thanks for raising this, @helenb. Can you support this using the as syntax? See "Generating the image 'as foo' to access individual properties" in the image docs.

@tomdyson yes, and that is how I have used it before. But it would be nice if we could get the responsiveimage tag into core (and perhaps resolve my ancient issue here: https://github.com/wagtail/wagtail/issues/285), and it ought to work as expected by default if it is added.

Thanks for raising this, @helenb. Can you support this using the as syntax? See "Generating the image 'as foo' to access individual properties" in the image docs.

@tomdyson @helenb
Hey guys, this approach only supports {% image .. %} tag. What if I want the same feature when inserting images in RichTextField??
The height and width attribute also generated by wagtail, which is not responsive and annoying.

Fixed in #10781.


added jsconfig file to manage files path and resolve merge issue

Description

Fixes # (issue)
Type of change

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update

How Has This Been Tested?

[ ] Test A
[ ] Test B

Development & Testing Configuration

react version:
react-router version:
SCSS version:

Checklist:

[ ] My code follows the style guidelines of this project

[ ] I have performed a self-review of my own code

[ ] I have commented my code, particularly in hard-to-understand areas

[ ] My changes generate no new warnings

[ ] Any dependent changes have been merged and published in downstream modules

[ ] I have read and agree with Code of Conduct

[ ] I have read and agree with Contributing

Thank you for contributing to @Invoicetor 💜🤍


Switch repository to github.com/integrations

Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

@conda-forge-admin, please rerender


Add Razor project configuration telemetry

Is this a Bug or Feature request?:
Feature
Description of the problem:
We need to know the common project types that are being used with tooling in order to prioritize features.

Telemetry bits have been submitted. We'll know if they're approved Tuesday.


BUG: Module exporting undeclared name should be a syntax error

Another bug I found whilst working on #5759 though I do not currently have a fix planned for this.
Per
https://tc39.github.io/ecma262/#sec-module-semantics-static-semantics-early-errors
It is a Syntax Error if any element of the ExportedBindings of
ModuleItemList does not also occur in either the VarDeclaredNames of
ModuleItemList, or the LexicallyDeclaredNames of ModuleItemList.
POC:
//start.js
import {Fun} from "./other.js";
print(Fun);

//other.js
export {anything as Fun};

Note the same repros if instead of "anything" you use any other undeclared name OR use a global like Number or Array or if you don't alias "anything".
Output with ch: undefined, jsc and v8 throw Syntax Errors
(Can't do an eshost repro at the moment as eshost doesn't yet interact with ch's module flag)
This causes numerous test262 failures.

I've been thinking about how to fix this - it seems it needs to involve adding a step at the end of parsing a module that re-checks any names exported via export {namesHere} without a from clause to confirm if they were locally defined and if not seek back to the relevant line and raise a syntax error.

Seems like something we should be able to do at name binding. We do push a pidref to 'anything' when we export it, right? When we bind names, we could throw a SyntaxError if we find a referenced name with no declaration.

Would that permit a valid case like this:
export {foo};
var foo = 5;

Whilst catching the invalid cases I put in the first post?

Yes, it would. @boingoing is referring to the way the parser binds names to declarations. The "push" records the fact that there was a reference to the given name in the current scope. Binding takes place when parsing of the current scope is completed. So the reference can lexically precede or follow the declaration.

Thanks @pleath that makes sense.
I was going to make a PR to fix this but see it's now assigned to you @boingoing want me to give it a go still or have you got it?

@rhuanjl, feel free to take a stab at it if you're motivated. I haven't looked-into the fix yet. We can help if you need some in that area of the code. If you're not feeling it I can take this one on, as well. 👍

I had a go at trying to do this but didn't get far, unless I'm missing something currently no pid ref is pushed for an exported name unless the export is also variable declaration (in which case it can't possibly hit this error).
And I couldn't quite work out how to push a pid ref for the name and have it be accessible in the right place without making it into a variable declaration which would obviously be wrong.

You're right, no pid ref is pushed in this case. It looks like we implemented it without attempting to bind the exported name to a declaration. Either we missed this requirement in the spec or implemented an earlier spec that didn't contain the requirement.
What you'll want to do is call PushPidRef from Parser::AddModuleImportOrExportEntry, passing in the IdentPtr of the local name, and store the resulting PidRefStack* on the export record. Binding will fill in the Symbol* on the PidRefStack. When we later resolve the export entries, a null Symbol* will indicate an undeclared name. Note that this has to be done whether buildAST is false or true, so the caller of AddModuleImportOrExportEntry will have to make the call in both cases.
Does all that make sense?


SBS Australia On Demand Error: No Formats Are Available

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

[X] I understand that I will be blocked if I intentionally remove or skip any mandatory* field

Checklist

[X] I'm reporting that yt-dlp is broken on a supported site
[X] I've verified that I have updated yt-dlp to nightly or master (update instructions)
[X] I've checked that all provided URLs are playable in a browser with the same IP and same login details
[X] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[X] I've searched known issues and the bugtracker for similar issues including closed ones. DO NOT post duplicates
[X] I've read the guidelines for opening an issue
[X] I've read about sharing account credentials and I'm willing to share it if required

Region
Australia
Provide a description that is worded well enough to be understood
After successfully extracting several video files from SBS Australia there was suddenly an error message that said "No formats are available". I get that error always now from SBS Australia.
Provide verbose output that clearly demonstrates the problem

[X] Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
[x] If using API, add 'verbose': True to YoutubeDL params instead
[X] Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output
[debug] Command-line config: ['-vU', '--verbose', 'https://www.sbs.com.au/ondemand/tv-series/darkness-those-who-kill/season-4/darkness-those-who-kill-s4-ep8/2365873219814']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version<EMAIL_ADDRESS>from yt-dlp/yt-dlp-nightly-builds [d1c4d88b2] (zip)
[debug] Python 3.11.2 (CPython x86_64 64bit) - Linux-6.5.0-5mx-ahs-amd64-x86_64-with-glibc2.36 (OpenSSL 3.0.14 4 Jun 2024, glibc 2.36)
[debug] exe versions: ffmpeg 6.0.1 (fdk,setts), ffprobe 6.0.1, rtmpdump 2.4
[debug] Optional libraries: Cryptodome-3.11.0, brotli-1.0.9, certifi-2022.09.24, mutagen-1.46.0, requests-2.28.1, sqlite3-3.40.1, urllib3-1.26.12, websockets-10.4
[debug] Proxy map: {}
[debug] Request Handlers: urllib
[debug] Loaded 1832 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp-nightly-builds/releases/latest
Latest version<EMAIL_ADDRESS>from yt-dlp/yt-dlp-nightly-builds
yt-dlp is up to date<EMAIL_ADDRESS>from yt-dlp/yt-dlp-nightly-builds)
[debug] Using fake IP <IP_ADDRESS> (AU) as X-Forwarded-For
[SBS] Extracting URL: https://www.sbs.com.au/ondemand/tv-series/darkness-those-who-kill/season-4/darkness-those-who-kill-s4-ep8/2365873219814
[SBS]<PHONE_NUMBER>814: Downloading SMIL file
[SBS]<PHONE_NUMBER>814: Checking geo-restriction
ERROR: [SBS]<PHONE_NUMBER>814: No formats are available; please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
  File "/bin/yt-dlp/yt_dlp/extractor/common.py", line 740, in extract
    ie_result = self._real_extract(url)
                ^^^^^^^^^^^^^^^^^^^^^^^
  File "/bin/yt-dlp/yt_dlp/extractor/sbs.py", line 111, in _real_extract
    self.raise_no_formats('No formats are available', video_id=video_id)
  File "/bin/yt-dlp/yt_dlp/extractor/common.py", line 1263, in raise_no_formats
    raise ExtractorError(msg, expected=expected, video_id=video_id)

The example URL gives me a 404 error (in Australia). Does it work for you?

This content expired on 9/11/24 2:00 PM.

<smil xmlns="http://www.w3.org/2005/SMIL21/Language">
	<head>
	</head>
	<body>
	<seq>
		<ref src="http://link.theplatform.com/s/errorFiles/Unavailable.flv" title="Expired" abstract="This content expired on 9/11/24 2:00 PM.">
			<param name="isException" value="true"/>
			<param name="exception" value="Expired"/>
			<param name="responseCode" value="403"/>
			<param name="correlationId" value="1c1cf619-2b4b-4158-9421-c971f4094fc6"/>
		</ref>
	</seq>
	</body>
</smil>


Login No relevant user found

I've been using the Nginx Proxy Manager as a docker for a couple of years. I haven't been able to log in for a while
No relevant user found
I have the login data in a password manager, so they must be correct
v2.10.2

@poltrian, this may sound a little daft but did you try logging in with the default credentials.
Default Admin User:
Email<EMAIL_ADDRESS>Password: changeme

Thanks,

Hello,
Fresh install on Portainer, same error.


chore(deps): update terraform github.com/terraform-ibm-modules/terraform-ibm-devsecops-alm to v1.4.3

This PR contains the following updates:

Package
Type
Update
Change

github.com/terraform-ibm-modules/terraform-ibm-devsecops-alm
module
patch
v1.4.0 -> v1.4.3

Release Notes

terraform-ibm-modules/terraform-ibm-devsecops-alm (github.com/terraform-ibm-modules/terraform-ibm-devsecops-alm)
v1.4.3
Compare Source
Bug Fixes

catalogjson (#382) (419ee20)

v1.4.2
Compare Source
Bug Fixes

remove unsupported parameter (#381) (5bc301d)

v1.4.1
Compare Source
Bug Fixes

incorrect validation promotion trigger settings (#380) (764de71)

Configuration
📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
👻 Immortal: This PR will be recreated if closed unmerged. Get config help if that's undesired.

[ ] If you want to rebase/retry this PR, check this box

This PR has been generated by Renovate Bot.

/run pipeline

:tada: This PR is included in version 1.5.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


optimize check idle time when get connection

在 DestroyTask 线程中已经对 idle 进行校验，这里将时间调大，减少获取连接时校验的耗时

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.PerryX seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.

no plan.


Uncaught TypeError: LocationServices.lastPosition.timestamp.getTime is not a function

Hi!
Thanks for your great plugin!
I am starting/stoping LocationServices via an ActivityRecognition service and sometimes (cannot reproduce it), I am getting this TypeError "LocationServices.lastPosition.timestamp.getTime is not a function" and LocationServices fail to start.
Plugin version: 1.2.0
Options:  {
enableHighAccuracy: true,
timeout : 12000,
maximumAge : 60000,
priority: LocationServices.priorities.PRIORITY_BALANCED_POWER_ACCURACY,
interval: 10000,
fastInterval: 5000
}
Device: Samsung Duos Android 4.2
Could you please advise?
Thank you!
Regards,
Stelios

Hi!
If you manage to reproduce it, would you mind sending me the stack trace.
Meanwhile I will check all calls to getTime, maybe I can find what's happening.

Thanks!
As a workaround, I replaced line 148 in LocationServices.js with
if (LocationServices.lastPosition && LocationServices.lastPosition.timestamp && LocationServices.lastPosition.timestamp.getTime && options.maximumAge && (((new Date()).getTime() - LocationServices.lastPosition.timestamp.getTime()) <= options.maximumAge)) {
I don't know If it is a correct thing to do but for the last 2 days the issue hasn't been reproduced.

OK, so it's indeed a bug in v1.2.0, it's been fixed in v2.0.0, where timestamp is always time in ms and not a Date object.
I'll make a v1.2 branch a publish a new release.

The new version compatible with Cordova 4 has been published:
cordova plugin add cordova-plugin-locationservices@legacy


Screenshots do not get taken for all failure conditions

What is your Test Scenario?
I am throwing a custom exception in typescript from one of my cucumber steps. I am using the test runner API and I have everything set up to take screen shots on failure. this works when throwing an assertion error, but other error types don't appear to cause the screenshot to be taken.
What is the Current behavior?
A screenshot is not taken in these cases even though the test fails
What is the Expected behavior?
A failure screen should be taken when an exception is thrown from a cucumber step, (while screenshots on failure are enabled)
What is your web application and your TestCafe test code?
I am throwing the exception like so:
throw new Error("Wait Time Out Exception. An element you were waiting for did not appears after " + timeout.toString() + " seconds.")

Your Environment details:

testcafe version:
node.js version:
command-line arguments:
browser name and version:
platform and version:
other:

I think the same behavior can occur when testcafe throws an error (for example what it can't find an action element on the screen or something similar). If possible it should take a screen shot in these cases as well.

I've checked the screnario and it works.
Here is the test code:
fixture `New Fixture`
    .page `http://example.com`;

test('New Test', async t => {
    throw new Error('It is a custom error');
});

And here is my terminal command:
testcafe chrome index.ts -s takeOnFails=true

I've got a screenshot as expected.
Could you please provide with a more detailed description and test code, which demonstrates the issue?

In my case I am setting the screenshot option via the test runner API rather than on the command line. here is that code snippet:
    createTestCafe('localhost')
      .then(function (tc: any) {
        testcafe = tc
        myTestCafe = tc
        runner = tc.createRunner()
        let screenshotPath = 'artifacts/screenshots/testcafe'
        if(process.env.SCREENSHOT_PATH){
          screenshotPath = process.env.SCREENSHOT_PATH
        }
        return runner
          .src(stepDefinitionsPath + '/test.js')
          .browsers(getBrowserSettings())
          .reporter('list')
          .screenshots({path: screenshotPath, takeOnFails: process.env.SCREENSHOT == 'true', fullPage: true})
          .run({
            skipJsErrors: true,
            quarantineMode: true,
            selectorTimeout: 10000,
            assertionTimeout: 10000,
            pageLoadTimeout: 10000
            //speed: 0.1
          })
          .catch(function (error: any) {
            console.error('Runner error count was: ', error)
          })
      })
      .then(function (report: any) {
        console.error('Report data was: ', report)
      })

I've created an example
with Runner API and it works as expected.
Could you please provide a simple example that we can run locally without any additional setup?


unable to install 4.0.1

Hi guys,
I was able to install 4.0.0 but had some errors with 4.0.1. The error message is below. Looks like is-glob is out of range in 4.0.1 under "dependencies":
npm ERR! node v4.4.4
npm ERR! npm  v2.15.1
npm ERR! code ETARGET
npm ERR! notarget No compatible version found<EMAIL_ADDRESS><4.0.0'
npm ERR! notarget Valid install targets:
npm ERR! notarget ["0.1.0","0.2.0","0.3.0","1.0.0","1.1.0","1.1.1","1.1.2","1.1.3","2.0.0","2.0.1"]
npm ERR! notarget
npm ERR! notarget This is most likely not a problem with npm itself.
npm ERR! notarget In most cases you or one of your dependencies are requesting
npm ERR! notarget a package version that doesn't exist.
npm ERR! notarget
npm ERR! notarget It was specified as a dependency of 'copy-webpack-plugin'
npm ERR! notarget
=====
{...
version: '4.0.1',
main: 'dist/index.js',
dependencies:
{ bluebird: '^2.10.2',
'fs-extra': '^0.26.4',
glob: '^6.0.4',
'is-glob': '^3.1.0',
'loader-utils': '^0.2.15',
lodash: '^4.3.0',
minimatch: '^3.0.0',
'node-dir': '^0.1.10' },
...}

This is unrelated to the changes in 4.0.1. It may have been an issue with the npm registry.


Calling method through ajax

Hello,
In PagedListPager we are calling method from URL.Action synchronously like follows,
@Html.PagedListPager((IPagedList)Model, page => Url.Action("Mailbox", new { page = page }), PagedListRenderOptions.Classic)

Can we call the same method through ajax  ?
Regards,
Rohit

Ye, I think you can implement separate view that can loaded by ajax.


Error exit status (1) in NFCORE_CHIPSEQ:CHIPSEQ:CUSTOM_DUMPSOFTWAREVERSIONS

Description of the bug
Hi all, I am trying to process my ChIP-seq data using the nf-core pipeline. It is pretty much finished but running into an error in the second to last step:

The error code I received was this:

I went to the work directory and opened the collated_versions.yml file and went to line 79, column 4 and found this (indicated by red star):

So it looks like the text here is shifted and should be in column 1, however, I cannot find how to fix this issue, and it seems to be doing this for all collated_versions.yml files. If it helps, I added --nomodel and --extsize 147 in the main.nf file in macs2/callpeak since there were too few peaks to build a shifting model. I dont know if this is the cause of the error:

Any help would be greatly appreciated.
Command used and terminal output
No response
Relevant files
*Changed file format to .txt so it can upload here. Please find attached the dumpsoftwareversions.py and the main.nf files.
dumpsoftwareversions.txt
main.txt
System information
Nextflow version: <IP_ADDRESS>36
Hardware: HPC
Executor: Slurm
Container engine: Singularity
Version of nf-core/chipseq: 2.0.0

Sorry for the late reply, for what I understand you are modifying the dumpsoftwareversions.py in principle the original script is working without any issue in most of the nf-core pipelines, and I have never seen the error you reported. I will advice you to use the original released pipeline and if you keep having issues, do not hesitate to ask again.

I didn't modify dumpsoftwareversions.py . I ended up having to shift the two lines of code I entered for main.nf and that fixed the issue.
The original released pipeline did not work for my data, as MACS2 was unable to call peaks. So I had to add --nomodel and --extsize 147 to main.nf .

Closed this issue now as it had no activity recently


[BUG] "no close frame received or sent" in test run

Reproduction steps
Observed the following error when running Test Case SMOKECO-2.5:
Executing Test Step: Step 71a: TH waits 60 Seconds
--
Test Step Completed [PASSED]: Step 71a: TH waits 60 Seconds
Executing Test Step: Step 71b: TH reads DeviceMuted attribute from DUT
Test Step Error: Error occurred during execution of test case TC-SMOKECO-2.5. no close frame received or sent
Test Step Completed [ERROR]: Step 71b: TH reads DeviceMuted attribute from DUT
Test Case Completed[ERROR]: TC-SMOKECO-2.5
Unpairing chip_tool from device
[SMOKECO-2_5_2023_08_21_14_32_54.log](https://github.com/project-chip/connectedhomeip/files/12410115/SMOKECO-2_5_2023_08_21_14_32_54.log)

Complete log is TH log is attached.
Bug prevalence
always
GitHub hash of the SDK that was being used
latest
Platform
raspi, other
Platform Version(s)
test harness
Anything else?
Reducing the wait times from 60 seconds 10 seconds in steps 55a and 71a serves as a workaround in our environment.

SDK_SHA = a795cca7934e8a0429c6366e7d9a2da7d687bacd

I'm also experiencing this problem.
UI Test Run log:
INFO       | 2023-09-01 07:11:42.807485 | Executing Test Step: Step 31: TH reads ExpressedState attribute from DUT
ERROR      | 2023-09-01 07:12:43.269677 | Test Step Error: Error occurred during execution of test case TC-SMOKECO-2.6. no close frame received or sent
INFO       | 2023-09-01 07:12:43.308084 | Test Step Completed [ERROR]: Step 31: TH reads ExpressedState attribute from DUT
INFO       | 2023-09-01 07:12:43.323340 | Test Case Completed[ERROR]: TC-SMOKECO-2.6
INFO       | 2023-09-01 07:12:43.327129 | Unpairing chip_tool from device
ERROR      | 2023-09-01 07:12:53.360470 | Test Suite Error: Error occurred during cleanup of test suite FirstChipToolSuite. Connecting to ws://<IP_ADDRESS>:9002 failed.
INFO       | 2023-09-01 07:12:53.401011 | Test Suite Completed [ERROR]: FirstChipToolSuite
INFO       | 2023-09-01 07:12:54.078359 | Test Run Completed [ERROR]

DUT log:
[00:01:31.278][detail][IN] UDP Message Received packet nb : 46 SrcAddr : fd11:22:0:0:5267:4155:1db8:8eec[60881] DestAddr : fd11:22:0:0:e198:a544:5031:27b3[5540] Payload Length 51
[00:01:36.277][detail][IN] UDP Message Received packet nb : 47 SrcAddr : fd11:22:0:0:5267:4155:1db8:8eec[60881] DestAddr : fd11:22:0:0:e198:a544:5031:27b3[5540] Payload Length 51
[00:01:41.280][detail][IN] UDP Message Received packet nb : 48 SrcAddr : fd11:22:0:0:5267:4155:1db8:8eec[60881] DestAddr : fd11:22:0:0:e198:a544:5031:27b3[5540] Payload Length 51
[00:01:56.278][detail][IN] UDP Message Received packet nb : 49 SrcAddr : fd11:22:0:0:5267:4155:1db8:8eec[60881] DestAddr : fd11:22:0:0:e198:a544:5031:27b3[5540] Payload Length 51
[00:02:11.277][detail][IN] UDP Message Received packet nb : 50 SrcAddr : fd11:22:0:0:5267:4155:1db8:8eec[60881] DestAddr : fd11:22:0:0:e198:a544:5031:27b3[5540] Payload Length 51


[Request] Docker Compose examples

I want to build a compose file that can be used with cosmos server and it's registries (like https://github.com/TinyActive/cosmos-servapps/issues/26)
https://github.com/azukaar/Cosmos-Server

How about https://github.com/factoriotools/factorio-docker/blob/master/docker/docker-compose.yml ?

Factorio Update has support please help me check if it works.... thank you

How about https://github.com/factoriotools/factorio-docker/blob/master/docker/docker-compose.yml ?

It's a terrible example. Environment variables aren't exactly documented/mapped and that opt directory change should be included in the Docker file. I have no idea what you guys are thinking.


Compare performance between beluga_amcl and nav2_amcl

Description
Now that we have a functional ROS2 node beluga_amcl, we should start thinking about a comparison with nav2_amcl. It would be ideal to have a reproducible environment to compare performance and accuracy whenever we want as we improve the software (looking at you lambkin).
QuickMCL already has done this for the ROS1 case, leaving behind a repository we could use as reference.
Definition of Done

[x] Collect performance and accuracy metrics data from both packages.
[x] Create a report that explains the methodology and results.

I don't see a discussion here on the actual localization quality / improvements that Beluga could offer which seem important. Are there any metrics on this?

@SteveMacenski there are! But the reports have not been made properly public (because time). Our benchmarking toolkit has to go open source (soon enough) and beluga_benchmark is due for an update (a merge, actually, with our private, soon public codebase). We are getting there...

OK! Let me know if I can be useful. The things I'm looking for (for reference):

Generally in the same-ish compute regime as AMCL. I don't think it needs to be a race to the bottom, but within reason
Equal or better localization quality over several realistic spaces (warehouse, office, etc) over long periods of time
Shown to resolve AMCL's delocalization jumping issues, or at least the potential path for it in the new framework
Better default tuning which we never really bothered with in Navigation/Nav2 to date

With that, I'm perfectly happy to dump nav2_amcl into its own repository on a deprecation track, add in Beluga as default, and the documentation / tutorials to fit.

S

Equal or better localization quality over several realistic spaces (warehouse, office, etc) over long periods of time

Noted. Warehouses are the trickiest for us right now. We would need a collaborator in that line of business (which we haven't found yet). Speaking of large spaces though, I wonder if any Willow Garage datasets (e.g. the one described here) survived to this day. Do you happen to know @mikeferguson? The ones on https://google-cartographer-ros.readthedocs.io/en/latest/data.html#pr2-willow-garage are gone, and 18 hs x 1600 $m^2$ was pretty decent.

Shown to resolve AMCL's delocalization jumping issues, or at least the potential path for it in the new framework

This one's perhaps the largest stretch. Posteriors may naturally go multi-modal on a particle filter. Maybe the key here is to do so in a controlled manner if it comes that :thinking: CC @glpuga @nahueespinosa.

@hidmic I do not know of anywhere those still exist - perhaps reaching out to some of the authors of Cartographer?

Warehouses are the trickiest for us right now.

Let me inquire to see if I can get a dataset or two, no promises though. I have a few that I have private NDA access to, but nothing I can share

I don't see a discussion here on the actual localization quality / improvements that Beluga could offer which seem important. Are there any metrics on this yet?

@SteveMacenski related to this we just posted the results of a benchmarking campaing validating Beluga AMCl against Nav2 AMCL.
Results summary and commentaries are in the blog post in our site here, and the complete bagfile-by-bagfile results are in this pdf


ML Scoring

We've already discussed what/why/etc. so here I'll just describe how it was implemented and what remains to be done
@sergii-mamedov please review the back-end changes and @lmacielvieira please review the front-end changes
Setup
After restarting graphql to add the new table, the v3_default scoring model can be imported by running this:
dc exec api sh  # If you don't have Python set up outside Docker, open a shell inside the API container

cd /opt/dev/metaspace/metaspace/engine

python -m scripts.import_scoring_model v3_default ../scoring-models/model-2022-01-05T13-45-26.947188-416b1311.cbm sm-engine-dev

Architecture Overview
annotation.stats JSON column
The schema hasn't changed, but msm has been removed (it was unused & also in a column) and more fields have been added:
Old style:
{
  "chaos": 0.9819444444444444,
  "spatial": 0.048468438199788286,
  "spectral": 0.9251577868703323,
  "msm": 0.04403132469,
  "total_iso_ints": [1504149.575683593,<PHONE_NUMBER>492179, 24994.709472656,<PHONE_NUMBER>6875],
  "min_iso_ints": [0, 0, 0, 0],
  "max_iso_ints": [75321.7109375, 45423.57421875, 20302.056640625, 42440.9453125],
}

New style:
{
  "chaos": 0.9976041666666666,
  "spatial": 0.005921380836044965,
  "spectral": 0.7906196869421936,
  "total_iso_ints": [348247.875, 1093766.0, 1901134.625, 36702.76953125],
  "min_iso_ints": [0.0, 0.0, 0.0, 0.0],
  "max_iso_ints": [4206.11474609375, 1619.6641845703125, 1990.939453125, 1607.725341796875],
  "mz_mean": [623.3374633789062,<PHONE_NUMBER>3125,<PHONE_NUMBER>023438,<PHONE_NUMBER>789062],
  "mz_stddev": [0.0003070708771701902, 0.0006269849836826324, 0.0007836259319446981, 0.0011551531497389078],
  "mz_err_abs": 0.0016151343006640673,
  "mz_err_rel": -0.0008197362767532468,
  "theo_mz": [623.3358764648438,<PHONE_NUMBER>984375,<PHONE_NUMBER>507812,<PHONE_NUMBER>328125],
  "theo_ints": [100.0, 37.65089797973633, 37.747135162353516, 13.01543140411377],
}

mz_mean and mz_stddev are new per-image mass error fields that are interesting for diagnostics
mz_err_abs and mz_err_rel are the 2 new metrics
theo_mz and theo_ints are technically redundant, as they can be recalculated, however it has always been a pain to recalculate these fields during data analysis as they depend on dataset metadata.

TBH I'm not happy that this field has become so big. It used to be ~300B/row and now it's ~750B. It contains 33 64-bit floats and 7 arrays, so if it was split into non-JSON columns it would be ~320B with the new data. However, the annotation table is very infrequently accessed (all Annotation queries go to ElasticSearch instead), so it wasn't pragmatic for me to spend time optimizing this.
ScoringModel
This table/class/ds_config item represents a named model that transforms some set of features into a score. By default this is None for V1/V2 and 'v3_default' for V3. Specifying None will revert to the old MSM calculation (spatialspectralchaos). Using any non-null string will look up the ScoringModel row in the database try to load it as a CatBoost model.
The features passed to CatBoost depend on the features specified in the ScoringModel metadata. This is to support new models that may use different configurations of features. Some "derived features" (e.g. the FDR-normalized features) need to be calculated based on the overall results for the dataset, so there's a function add_derived_features that adds these. Additionally, there's remove_uninteresting_features, which removes temporary features that don't need to be saved to stats.
The model that is included in the repository does not use the FDR-normalized metrics I've previously described.
FDR changes
Continuous FDR values + "Rule of succession" approximation
V3 datasets now save exact FDR values (e.g. 0.003812) instead of snapping to the 5/10/20/50% thresholds. Webapp has been changed to snap to 5/10/20/50%, but the raw values can be viewed through the API, CSV export or with the raw_fdr feature flag.
FDR_RESULTS Diagnostics
Every job now produces a FDR_RESULTS diagnostics entry, which links to a set of 3 parquet files for mapping target/decoy adducts to formulas to metrics. The main documentation is in diagnostics.py.
Decoy adduct selection
Under V1, each target adduct gets its own independent ranking where 20 decoy adducts are randomly selected. With 3 target adducts this means 3x20 random selections, averaging 53 decoy adducts per formula after deduplication. This is a wasteful, as there's no dependency between the rankings, so there's no reason to resample the decoy adducts
If V3 mode is selected, the randomly selected decoy adducts are reused for all target adduct rankings. This means  ~60% fewer ion formulas are searched for with 3 target adducts, or ~40% fewer with 2 target adducts.
Lithops pipeline changes

I rewrote MolDB segmentation to use a non-recursive algorithm for splitting the work into segments, as I was having trouble making the old version stable when annotating with 80 decoy adducts. Hopefully now it'll be a bit easier to tune the job size.
Optimized the PNG encoding step - more accurate prediction of CPU time required to encode, and smaller chunks of images
Added a "store images" step that posts images directly from IBM Cloud Functions to S3. This should significantly speed up the last step and decrease CPU usage on the web server for datasets with >5000 images.

sm.fdr_engineering directory
This is the code used for selecting & reprocessing datasets, downloading their FDR Diagnostics, and training new models. It's meant to be run interactively - all the run___.py files are structured like notebooks, using the #%% "percent format" to indicate cells. Use PyCharm Scientific Mode, Spyder or Jupytext to make life easier running these files.
Remaining work
(I'll do these in other PRs)

[x] Add script & documentation for importing ScoringModels
[ ] Investigate inconsistent results between Spark and Lithops implementations - some metrics differ very slightly between the implementations, usually only by 1-2 bits. I suspect it's either due to differences in the order of pixels or floating point numbers having different precisions in different codepaths
[ ] Add a V3 mode to sci-test with reference results
[ ] Add an "FDR-normalized" model

I've added the changes to fix the Spark/Lithops discrepancy. The issue was that the coo_matrix sparse forms of images had their peaks stored in different orders between the two implementations. Spark used quicksort (an unstable sort) to sort peaks by m/z, whereas Lithops used mergesort (a stable sort). Both using 32-bit m/z. With a stable sort, peaks that have the same m/z are kept in their original order (spectrum index ascending), but with unstable sort they're unpredictably ordered. I fixed this by making Spark use mergesort.
This was enough to make them consistent, but the amount of variance caused by the order-dependent operations indicated that it was also a potential issue with accuracy, so I kept digging.
Many floating point operations have order-dependent results. The weighted np.average operation used in weighted_stddev turns out to be exceptionally sensitive, with almost every value being different depending on order. To stabilize the outputs, I made this function use 80-bit longdouble math. Even with this change it's not 100% order-independent, but the order-dependent error is much smaller.
This wasn't previously detected because the original spectral/spatial/chaos metrics use the non-sparse forms of images, where pixels are in grid order. total_iso_ints (calculated from the sparse images) occasionally differed between the implementations because np.sum is also order-dependent, but np.sum has much less order-dependent error than np.average with weights, so we never noticed.

@sergii-mamedov I just pushed the fixes for the two bugs we discussed

There's a merge conflict now, but I'll try to wait until after approval before merging in master to minimize disruption.
With db_mutex.py, the change in master is better than the change in this branch, so this branch's change will be undone.


use secrets from aws secret manager

Could you please help me how can I use secret variables from aws secret manager in terragrunt ?
With terraform I use such approach:
variables.tf
data "aws_secretsmanager_secret_version" "creds" {
  secret_id = "app-prod-config-server"
}

locals {
  app-creds = jsondecode (
    data.aws_secretsmanager_secret_version.creds.secret_string
  )
}

Then I can use secret variables in my tf files.
username                   = local.app-creds.DATASOURCE_USERNAME
password                   = local.app-creds.DATASOURCE_PASSWORD

The best way would be to use run_cmd with the AWS CLI.


tensorflow.python.framework.errors_impl.NotFoundError: Graph ops missing from the python registry ({'Einsum'}) are also absent from the c++ registry

I am trying to creat tensorflow-hub Module using locally saved ALBER pre-trained model and I am getting this error :
tensorflow.python.framework.errors_impl.NotFoundError: Graph ops missing from the python registry ({'Einsum'}) are also absent from the c++ registry.
I have:
tensorflow: 1.14.0
tensorflow-hub: 0.7.0
opt-einsum= 3.1.0
I am working on MAC OS High Sierra 10.13.6

this is the code for creating module:
import tensorflow_hub as hub
import tensorflow as tf
embed= hub.Module(“/path/to/saved/locally/pre-trained/ALBERT/Model/ALBERT_xxlarge")
this is the error that I got
Traceback (most recent call last):
File "/Users/……/python/helpers/pydev/pydevd.py", line 1741, in
main()
File "/Users/……/python/helpers/pydev/pydevd.py", line 1735, in main
globals = debugger.run(setup['file'], None, None, is_module)
File "/Users/……/python/helpers/pydev/pydevd.py", line 1135, in run
pydev_imports.execfile(file, globals, locals) # execute the script
File "/Users/……/python/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc)
File "/Users/…….py", line 3, in
embed= hub.Module("/Users/……/Pretrain models/ALBERT/ALBERT_xxlarge")
File "/Users/……/lib/python3.6/site-packages/tensorflow_hub/module.py", line 176, in init
tags=self._tags)
File "/Users/……/lib/python3.6/site-packages/tensorflow_hub/native_module.py", line 386, in _create_impl
name=name)
File "/Users/……/lib/python3.6/site-packages/tensorflow_hub/native_module.py", line 428, in init
op.name for op in self._meta_graph.meta_info_def.stripped_op_list.op})
File "/Users/……/lib/python3.6/site-packages/tensorflow_hub/native_module.py", line 858, in register_ops_if_needed
"the c++ registry." % missing_ops)
tensorflow.python.framework.errors_impl.NotFoundError: Graph ops missing from the python registry ({'Einsum'}) are also absent from the c++ registry.
I downloaded pre-trained ALBERT model from:
https://tfhub.dev/google/albert_xxlarge/2?tf-hub-format=compressed
thanks.

Same issue with https://github.com/google-research/google-research/issues/108. No solution until now, maybe try https://tfhub.dev/google/albert_xxlarge/1.


fix workspace; fix warning

Project didn't compile because of the workspace = { members = ["sixel_test"] } line introduced when fixing #2, without the accompanying tests, so I've commented it out.
Also there was an unused import.

I should check that project more often :/.
Thanks


Scheduled callback call transfers

Hi guys,
We have recently implemented customer call backs to handle longer than usual wait times on our queues.
When an agent receives a customer callback however, they are unable to transfer the call if required.
When using the same transfer script as our normal calls, we get a console error
[ERROR]: 'api_response' event handler failed.  amazon-connect-v1.6.1.js:21264:38
Has anyone else ran into the same issue and was able to solve the problem?
Below is an example of our quick connect warm transfer:
`var agent = new lily.Agent();
agent.getEndpoints(agent.getAllQueueARNs(), {
  success: function(data){
    agent.getContacts(lily.ContactType.VOICE)[0].addConnection(data.endpoints[qcId], {
      success: function(data) {
        console.log("Call transferred");
        callState("callTransfer");
      },
      failure: function(data) {
        alert("transfer failed");
      }
    });

  },
  failure:function(){
    console.log("failed")
  }
});`

I believe I actually solved this shortly after. It appears that agent.getContacts(lily.ContactType.VOICE)[0] was returning null.
Changing my script to the below has resolved the issue.
`var agent = new lily.Agent();
var agentContacts = agent.getContacts();
agent.getEndpoints(agent.getAllQueueARNs(), {
  success: function(data){
    agentContacts[0].addConnection(data.endpoints[qcId], {
      success: function(data) {
        console.log("Call transferred");
        callState("callTransfer");
      },
      failure: function(data) {
        alert("transfer failed");
      }
    });

  },
  failure:function(){
    console.log("failed")
  }
});`


Add Parameter description in 2016 version

Added description for all parameters in 2016 version and fixed few missing in 2012 R2 version.

Docs Build status updates of commit 8d78c8d:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docset/windows/failoverclusters/remove-clustervmmonitoreditem.md
:white_check_mark:Succeeded

docset/winserver2012r2-ps/failoverclusters/Remove-ClusterVMMonitoredItem.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

@nenonix @JohanFreelancer9 Copy editing is needed for this PR. Thanks.

@dariomws306 @get-itips @tseward
Peer review needed for this PR. Thanks.

Docs Build status updates of commit b87df66:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docset/windows/failoverclusters/remove-clustervmmonitoreditem.md
:white_check_mark:Succeeded

docset/winserver2012r2-ps/failoverclusters/Remove-ClusterVMMonitoredItem.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

Docs Build status updates of commit 05831fc:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docset/windows/failoverclusters/remove-clustervmmonitoreditem.md
:white_check_mark:Succeeded

docset/winserver2012r2-ps/failoverclusters/Remove-ClusterVMMonitoredItem.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


fix sorting bug

As described in #1722 , there is an issue with sorting and DDP.
This is to open up a PR. The goal is not fixing all possibilities - the goal is to restore functionality.
A testing is not yet implemented (needs DDP set-up; that might be a different sort of github workflow altogether).
https://github.com/speechbrain/speechbrain/blob/c229dbcde906a850ffcacd7de2c5b02755f2d74e/speechbrain/core.py#L726
defines a shuffle variable that is simply re-used for the flags that were redefined in #1518 ; defaults to False, such that sorting is kept if wanted througout.

To what is this variable connected? a yaml parameter?

yeah kinda - but it might be in conflict with some other concept beneath.
maybe the idea is bogus. need to test - let's use this as a playground to get sth working
well, actually, the sorting flag is the flag?

just put it to "False" instead; this simply reverts the other PR that caused the bug w/o needing to rethink recipe & hparam logic.

Added a test and an initial fix outline. The test uses a CPU ddp backend, gloo.
I run into these nearby-sortings w/ sorting & ddp:
tensor([2.2050, 2.3400, 2.6000, 2.8700, 3.1050, 3.1500, 3.2200, 3.2800, 3.4550,
        3.6500, 3.7000, 4.2650, 4.4701, 4.7600, 4.8350, 4.8900, 5.1600, 6.1450,
        1.7750], dtype=torch.float64)

tensor([6.1450, 5.1600, 4.8900, 4.8350, 4.7600, 4.4701, 4.2650, 3.7000, 3.6500,
        3.4550, 3.2800, 3.2200, 3.1500, 3.1050, 2.8700, 2.6000, 2.3400, 2.2050,
        6.2000], dtype=torch.float64)

The test needed to be adjusted to DDP & pytorch's DistributedSampler which has:
        drop_last (bool, optional): if ``True``, then the sampler will drop the
            tail of the data to make it evenly divisible across the number of
            replicas. If ``False``, the sampler will add extra indices to make
            the data evenly divisible across the replicas. Default: ``False``.

Test works for the proposed fix on DDP & sorting. Bc of the drop_last & replicas, the final elements of the last batch won't be sorted within that batch. Yet, DDP seems to distribute the batches w/ replicas BUT

for sorted (filtered) datasets, the total number of examples does not change
for random (shuffling), the number of examples increases, as replicas are included.

The sorting asserts are thus tailored to the dummy data (and a bit lazy with "[:-1]" subset testing across all batches, instead of final batch only).
Ready for review.


[embedded] Add support for actors and async let into the embedded Concurrency runtime

This PR:

expands the same "pass type information via taskOptions" approach done in https://github.com/apple/swift/pull/68928 to async let runtime calls
starts building a larger portion of the Swift Concurrency runtime in embedded mode, including Actor.swift
adds simple testcases showing actors and async let working in embedded Swift

@swift-ci please test

@swift-ci please test

@swift-ci please test

Merging to make progress on getting the experimental implementation of embedded Swift to support Swift Concurrency, but happy to take any post-merge feedback/review comments.


Define mach_port_mod_refs

Sure, added that to my TODO list.

Published on crates.io: https://crates.io/crates/mach2/0.4.1


Log errors only when connected to diamond socket

remove parens on :138. otherwise, +1

lgtm (apart from what what ztm has mentioned)

+1


Building docs fails on Readthedocs

See https://readthedocs.org/projects/mdf/builds/. Latest commit to master includes the restructured dependencies which may be the source of the issue (possibly this change: https://github.com/ModECI/MDF/commit/f41a4a30ab316ce2099967cdc96f73bbf9044c85#diff-60f61ab7a8d1910d86d9fda2261620314edcae5894d5aaa236b821c7256badd7L14?)
Could you look into this @parikshit14? Could also be to do with an update to sphinx or other libraries being used on Readthedocs to build...
@davidt0x Could you give pgleeson permission on the https://readthedocs.org/projects/mdf? Can tweak it to test building the development branch too.

just checked the requirements.txt
https://github.com/ModECI/MDF/blob/a4e0e6e65d0d377d941576748de20b3bb00d8e5d/docs/sphinx/requirements.txt#L2
i think it should have all or optional instead of docs

Worked nicely! Thanks @parikshit14


Migrate project to TypeScript

🚨 Please review the guidelines for contributing to this repository.
Hi @ramgrandhi, peeps,
I've taken a bit of time to migrate this project to TypeScript. What's added is I've migrated all the files to TypeScript (shallowly for now).
The project does build and the tests do run. I have configured both eslint and jest to work with typescript as well.
One thing that remains is that we need to figure out how to build it before we publish it of course.

[x] Updated unit tests (*.spec.js)
[] Updated e2e tests (here)
[ ] Updated documentation (here)

🙌 Thank you!

Awesome that you started it! 🤩
Let's pick a moment (after my holidays) and merge this.


COMPONENTS: Remove Post View

@hassanhabib This component should be probably just an icon button taking PostView, or only the Id itself, preferably the first i assume. Any pre work that is required to complete this component. I think with the service method there it should be completable. I would take care of this after you post your opinion on the design ;)

@hassanhabib

Icon Component
[Parameter] PostView / PostViewId
Execute Remove and maybe navigate to other page/refresh or fire and event to remove PostView in Parent Component


Snapshots

I see that snapshot read only transactions are unsupported. Was this based on it not being important for most use cases, or is it too difficult to implement?
Or perhaps is it planned for future versions?

Are you referring to stale reads? They are supported.

That feature is great, and will cover most use cases, but a snapshot, unlike a standard single instance read only transaction, creates a read only transaction that you can use multiple times in a request. It's not essential, but just like you seem to have made provision for other queries to use either the base database object, or a transaction object, it would also be good to have a way for a set of queries to use a snapshot that remains consistent across the handling of a method that contains multiple reads, even if data is modified in between the different reads
Heres the docs about snapshot - https://cloud.google.com/spanner/docs/samples/spanner-read-only-transaction#code-sample

Ah, I see. I've never really had any use for such a case so it's not implemented.
I can put it on the TODO but implementing it seems a bit challenging, so I can't guarantee that it will get implemented anytime soon.

Understandable, its not critical to have in most cases, but good to flag it for the future

@taka-oyama I just want to bump this after further investigation.
It appears that while the existing withStaleness() option is useful for performance, it is still not sufficient and a proper implementation of snapshots is essential.
The primary reason is locking. From the docs here https://cloud.google.com/spanner/docs/timestamp-bounds#timestamp_bound_types

Although reads using these timestamp bound modes are not part of a read-write transaction, they can block waiting for concurrent read-write transactions to commit. Bounded staleness reads attempt to pick a timestamp to avoid blocking, but may still block.

Even with timestamp bounds - there is still potential for locking of other read-write transactions.
On the other hand, snapshots take no locks and never block other transactions. In view of this, it is critical they be implemented

I found that simply adding this to Connection
    public function snapshot($options = [], Closure $callback)
    {
        if($this->currentTransaction)
            throw new Exception('Cant run a snapshot in the middle of a transaction');

        $this->currentTransaction = $this->getSpannerDatabase()->snapshot($options);

        $result = $callback();

        $this->currentTransaction = null;

        return $result;
    }

And then updating the typing of currentTransaction in ManagesTransactions like so
trait ManagesTransactions
{
    /**
     * @var Transaction|Snapshot|null
     */
    protected Transaction|Snapshot|null $currentTransaction = null;

Is sufficient to make a simple way to call a snapshot query in a closure like this
$conn->snapshot(['exactStaleness' => new Duration(15)], function() {
    return Model::all();
});

Even with timestamp bounds - there is still potential for locking of other read-write transactions.

Was there a specific case where it did block?
There is a bug in the current code that causes the staleness to be ignored with inside a read-write transaction.
I am planning to fix that in the next few days.

Btw, if you enable data boost, that's already using snapshot in the background.

There is a bug in the current code that causes the staleness to be ignored when inside a read-write transaction.
I am planning to fix that in the next few days.

This has been fixed in v7.4.2 and v8.1.1

Btw, if you enable data boost in your query, that's using snapshot in the background would this solve your case?

This is not viable since databoost queries are incredibly limited in what kind of queries you can run, due to the requirements of being root partionable etc

Even with timestamp bounds - there is still potential for locking of other read-write transactions.

Was there a specific case where it did block?
There is a bug in the current code that causes the staleness to be ignored when inside a read-write transaction. I am planning to fix that in the next few days.

No, there wasnt a specific case but in a high load application, it is an unacceptable risk, snapshots were designed for read only analytics queries that do not lock rows and are the correct way to run read only queries for optimal performance

I couldn't find any mention about snapshots not locking in the docs but I finally found details  on snapshot transaction in the API reference.

https://cloud.google.com/spanner/docs/reference/rest/v1/TransactionOptions#:~:text=For transactions that,are not needed.

For transactions that only read, snapshot read-only transactions provide simpler semantics and are almost always faster. In particular, read-only transactions do not take locks,

https://cloud.google.com/spanner/docs/reference/rest/v1/TransactionOptions#:~:text=Snapshot transactions do not take locks. Instead%2C they work by choosing a Cloud Spanner timestamp%2C then executing all reads at that timestamp. Since they do not acquire locks%2C they do not block concurrent read-write transactions.

Snapshot transactions do not take locks. Instead, they work by choosing a Cloud Spanner timestamp, then executing all reads at that timestamp. Since they do not acquire locks, they do not block concurrent read-write transactions.


Anvil collection Fix

I think this is a better solution to the way the previous anvil library is autogenerated.
As an added bonus, I think AnVIL_Template can now receive workflow updates from OTTR_Template. Previously, these would have clashed with the references to the anvil-library-build workflow.
Changes include:

Removing anvil-library-build workflow scripts and files
Removing triggers to anvil-library-build in pull-request and render-all workflows
Referencing the new link (https://hutchdatascience.org/AnVIL_Collection/) on the index page

It'll be nice to trim the time down a bit on PR previews too.

Totally. Thanks!

IIRC we had excluded the workflow files from the subscription. I can check on this!


Use constant instead of magic value

This Pull Request has been merged by GitConsensus.
Vote Totals

Yes
No
Abstain
Voters

3
0
0
3

Vote Breakdown

User
Yes
No
Abstain

PostCrafter
✔

dragoonreas
✔

tedivm
✔

Vote Results

Criteria
Result

Has Quorum
True

Has Votes
True


Oauth2 issue

I have a problem with authentications type = oauth2.
Making integration with oracle by instruction: https://www.oracle.com/webfolder/technetwork/tutorials/obe/cloud/idcs/idcs_splunk_obe/splunk.html#section2s1
And there are no incoming http requests, no errors, no logs in _internal.
Is this feature working?
using 1.5.4 and splunk 7.2.4

requests-oauthlib in application don't support 'http' endpoints for oauth2, update to latest and set OAUTHLIB_INSECURE_TRANSPORT env.


Downgrade airbrake to 4.x and restore config

Our errbit installation doesn't like the airbrake 5.x gem and therefore deploys don't get notified properly. Downgrading to 4.x fixes this.
Inspired by https://github.com/alphagov/specialist-publisher/pull/988

@whoojemaflip thank you and fixed!


Feature request: making the PowerToys app systray menu more useful

Summary of the new feature/enhancement
Clicking the PowerToys systray icon opens the settings screen, while right clicking brings up the context menu that provides a way to close the app, open settings or an about dialog. Not super useful.

Proposed technical implementation details
A new context menu could contain shortcuts to various PowerToys tools. This makes it easier for mouse users to access certain features.
The context menu should:

Provide the list of features (such as the Shortcuts Guides) that can be clicked to open it.
The context menu should list the keyboard shortcut of the feature as well so users can use it as a cheatsheet as well
Icons can be used to make the context menu more glanceable.
The about box is redundant, since it will be part of the settings screen (#1086 )

Nice to have:

Use WinUI 3.0 to modernize the context menu so it's more in line with the shell (#891 )
Make the left-mouse click customizable? (E.g. clicking it might bring up the power launcher instead of the default Settings screen?).

UX mock-up:

xref #891 master tracking item.  Good idea

Things we should think about here.

the context menu has icons
WinKey should be an icon too
Utilities disabled should not appear here.

To circle back on #3514 . It would be nice if some features can be quickly turned on/off (like FancyZones or KBM) in the context menu.

Yes i agree, lets scope this to just getting new menu look / feel system online and we can move on from there

Some thoughts from me:

Settings (bold if default action)
Open FancyZones Editor; and include the hotkey
Open Keyboard Manager (related to #3925?)
Open PowerToys Run; and include the hotkey as set for this
Check for updates
Open Website (GitHub)
Exit

Apart from that - I searched for this using the words "tray icon menu" but that didn't bring me to this issue. Maybe having those words in the title might help future searches?

I would really like it if the tray icon acted like Instant Eyedropper! It'd be great if PowerToys could fully replace it.
Seeing that context menu makes me reminds me of how MS closed my Project Reunion issue to update the styles of other UI frameworks. Elements used to have rounded corners and radio buttons used to look the way they do now but they can't be bothered updating the styles again (it's been 5 years since the launch of W10 and it is still far from cohesive). WPF still has the Windows 8 style.

@Poopooracoocoo This issue is tagged in the new one, so no information is lost :).
Have you checked out #6751? It should address all your concerns?


Unable to load vector drawable resources with app:srcCompat="@drawable/my_drawable" from Android Library project

Xamarin.Android Version (eg: 6.0):
Xamarin.Android   <IP_ADDRESS> (7db2aac)
Operating System & Version (eg: Mac OSX 10.11):
Microsoft Visual Studio Enterprise 2015
Version 14.0.25123.00 Update 2
Microsoft .NET Framework
Version 4.6.01055
Support Libraries Version (eg: 23.3.0):
23.4.0
Describe your Issue:
On Android KitKat platforms and above, activities from an Android library project using vector drawables with AppCompat are unable to resolve drawable resources, resulting in an unhandled exception crashing the app.
A simple workaround is to manually add the <AndroidResgenExtraArgs>"--no-version-vectors"</AndroidResgenExtraArgs> to the Android application project file.
Steps to Reproduce (with link to sample solution if possible):
A reproduction sample is available here https://github.com/ggirard07/AppCompatRepro
Or you could do it by yourself by following the steps:

Create a new Android Application project (blank app)
Add a new Android Library project to the solution
Implement an activity in the Library project that consume vector drawables with AppCompat and srcCompat attribute.
Modify the Android application project to start the Android library project activity.

Include any relevant Exception Stack traces, build logs, adb logs:
07-04 12:50:01.359 E/mono-rt (13471): android.content.res.Resources$NotFoundException: File res/drawable/ic_flashlight.xml from drawable resource ID #0x7f02004b
07-04 12:50:01.359 E/mono-rt (13471): 	at android.content.res.Resources.loadDrawable(Resources.java:2235)
07-04 12:50:01.359 E/mono-rt (13471): 	at android.content.res.Resources.getDrawable(Resources.java:733)
07-04 12:50:01.359 E/mono-rt (13471): 	at android.content.res.Resources.getDrawable(Resources.java:711)
07-04 12:50:01.359 E/mono-rt (13471): 	at android.support.v7.widget.ResourcesWrapper.getDrawable(ResourcesWrapper.java:12

I'm seeing the exact same problem including a vector drawable in some menu.xml.  This used to work fine in 23.3 but fails in 23.4.  The workaround above fixed the problem.  This nuget package needs to add the MSBuild argument.
See<EMAIL_ADDRESS>

It looks like there is a .targets file missing in the nuget for Vector Drawable 23.4 which is supposed to take care of adding this for you.
While it's still in the .nuspec: https://github.com/xamarin/AndroidSupportComponents/blob/master/vector-drawable/nuget/Xamarin.Android.Support.Vector.Drawable.nuspec#L21-L22  It looks like there was a packaging issue with that version which caused the assembly and targets file to be omitted.
We will fix this in v<IP_ADDRESS>

Fixed back in v<IP_ADDRESS>


Demonstrate how to use Dask+Holoviews to show RA, Dec density map

Uses Dask+Holoview to visualize RA, Dec using a local Dask cluster.
Can you

try this out and confirm that works for
offer suggestions for anything to do this better

@cwwalter I added some more pedagogical material giving more details of Dask vs. Pandas DataFrame comparisons with discussions of persist, memory usage, disk I/O, and computation speeds.
I rewrote the star truth table comparison to no longer use GCRCatalogs and instead just directly read the star truth table parquet file into a Pandas DataFrame.

@cwwalter Will you have a chance to look at this PR in the next week?


Integration with dolfinx

get libtab from Python
Find header files of libtab

I'll try and get the headers to install.

Excellent. Once that's done, I'll get to work getting everything to work again

Pretty much there


How can I configure the security.toml to start the volumes node with HTTPS?

version：
version 30GB 3.72 e50d85c0f3697caf0ff9aca6662d59b4327a8424 linux amd64
Describe the bug
I have already configured the .crt and .key files for HTTPS in the security.toml and signed them, but after starting, it still runs as an HTTP service.
[https.volume]
cert = "/Users/chris/.seaweedfs/out/volume01.crt"
key  = "/Users/chris/.seaweedfs/out/volume01.key"
System Setup
linux amd64  kernel-version：5.10.0

@jovanLee07 Did you enable https separately using options and set up the port for TLS?

@jovanLee07 Did you enable https separately using options and set up the port for TLS?
I haven’t used the disabledhttp option, but I did set up the TLS port. I also configured the certificates for client, master, volume, and filer through the security.toml file using certstrap. After the service started and I uploaded a file, the volume logs show the following message: http: TLS handshake error from localhost:32900: remote error: tls: bad certificate. I’m not sure which step went wrong.

@kmlebedev Thank you for your reminder. I haven’t used the -disableHttp option, but I did set up the TLS port. I also configured the certificates for client, master, volume, and filer through the security.toml file using certstrap. After the service started and I uploaded a file, the volume logs show the following message: http: TLS handshake error from localhost:32900: remote error: tls: bad certificate. I’m not sure which step went wrong.

@jovanLee07 did you fix the issue? and for every service like master service,filer service and volume service we need to set up tls saperately? and does seaweedfs have the ui for volume service?

certstrap

See an example of creating a correct certificate in the Makefile certstrap
https://github.com/seaweedfs/seaweedfs/blob/master/docker/Makefile#L114

certstrap

See an example of creating a correct certificate in the Makefile certstrap
https://github.com/seaweedfs/seaweedfs/blob/master/docker/Makefile#L114

@kmlebedev certstrap commands provided on this url and on wiki for grpc conn, not for https conns, for https need to generate x509 certs.