Enhancements to one-of construct

Background

The CDM Asset Refactoring Taskforce is working to enhance the modelling of financial assets in the CDM.
This includes adding additional data types to the existing product model which will introduce additional levels into the hierarchy and increased use of the “one of” syntax to provide conditional selection of multiple subsidiary product types.
Example

An example of a new data type introduced in the refactoring is Asset (simplified):

  type Asset
    basket Basket (0..1)
    loan Loan (0..1)
    security Security (0..1)
       condition: one-of

The sub-types in the Asset definition will contain common attributes, for example an identifier.

Rationale

The proposed enhancements to the DSL will:

Increase understanding of modularity in models built using Rune.
Improve readability of Rune DSL code.
Differentiate one-of selections from other data types.
Reduce long path traversals in complex constructs.
Enable ease-of-use enhancements in DSL tools.

Requirements

Simplify the one-of construct by enabling a special kind of data type which:

Is composed of two or more constituents (which are themselves defined as data types).
Contains a choice of one and only one of the constituents that can be used in any instance.
Restricts the cardinality of all constituents to one.
Imposes that the name of a constituent (i.e. the attribute) is the same as that of the data type.

Where multiple child data types of a parent data type have common attributes, it should be possible to access the common attributes directly.

For example, where identifier is common across all Asset sub-types, the current syntax would require IF…THEN logic to identify which sub-type is present, e.g.:

  if basket exists
  then asset -> basket -> identifier
  else if loan exists
    then asset -> loan -> identifier
    else asset -> security -> identifier

A simpler access path is required, for example:

   asset -> identifier
Summary Deck
The attachment documents these requirements.
RUNE DSL Enhancement for One Of.pptx

Related CDM initiative:  https://github.com/finos/common-domain-model/issues/2805

Here is a comparison of three different solutions. I evaluate each of them based on the following five questions.

How is the type represented in the model? How are common attributes represented? Is there duplication?
How can one access common attributes in an expression?
How is the type represented in serial form (JSON)?
How can one discriminate between different types?
Is it possible to guarantee that all cases are covered? (e.g., by red-underlying the Rune code when a modeller forgot to cover a case)

All comparisons are based on a toy version of the Asset (Basket/Loan/Security) model.
Enhance the current solution (one-of)
Summary:

Common attributes are duplicated for each one-of case.
Accessing common attributes is verbose. A proposed enhancement (->>) can eliminate this verbosity.
Serialisation is nested. The upside is that the field name allows us to discriminate different types during deserialisation.
Discriminating types and accessing their attributes is verbose. A proposed enhancement (switch) can eliminate this problem.
No validation to guarantee all cases are covered. A proposed enhancement (switch) can eliminate this problem.

Model
type Asset:
  basket Basket (0..1)
  loan Loan (0..1)
  security Security (0..1)

  condition Choice:
    one-of

type Basket:
  identifier string (1..1)
  basketAttribute int (0..1)

type Loan:
  identifier string (1..1)
  loanAttribute number (0..*)

type Security:
  identifier string (1..1)
  securityAttribute date (2..2)

Access common attributes
if asset -> basket exists
then asset -> basket -> identifier
else if asset -> loan exists
then asset -> loan -> identifier
else if asset -> security exists
then asset -> security -> identifier

Access common attributes (proposed enhancement: automatically detect common attributes and make them directly available)
// Option 1: implicit
asset -> identifier
// Option 2: explicit
asset ->> identifier

Serialisation (of a basket)
{
  "basket": {
    "identifier": "abc123"
    "basketAttribute": 42
  }
}

Discriminate types
if asset -> basket exists
then // do something with asset -> basket -> basketAttribute
else if asset -> loan exists
then // do something with asset -> loan -> loanAttribute
else if asset -> security exists
then // do something with asset -> security -> securityAttribute

Discriminate types (proposed enhancement: switch over one-of types to guarantee coverage and reduce verbosity)
switch asset
  case basket then // do something with basket -> basketAttribute
  case loan then // do something with loan -> loanAttribute
  case security then // do something with security -> securityAttribute

Using extends
Summary:

Common attributes are not repeated. Good!
Accessing common attributes is concise. Good!
Cannot determine the actual type when deserialising. A proposed enhancement (__type__) can eliminate this problem.
Need a way to discriminate between different types, either via an of-type operation, or via "dispatching".
Validation to guarantee all cases are covered is impossible since subtyping is open for extension.

Model
type Asset:
  identifier string (1..1)

type Basket extends Asset:
  basketAttribute int (0..1)

type Loan extends Asset:
  loanAttribute number (0..*)

type Security extends Asset:
  securityAttribute date (2..2)

Access common attributes
asset -> identifier

Serialisation (of a basket) (with proposed solution to determine the actual type)
{
  "__type__": "Basket"
  "identifier": "abc123",
  "basketAttribute": 42
}

Discriminate types (new feature!)
// Option 1: à la `instanceof` (with type narrowing?)
if asset of-type Basket
then // do something with asset -> basketAttribute
else if asset of-type Loan
then // do something with asset -> loanAttribute
else if asset of-type Security
then // do something with asset -> securityAttribute
// else ... (potentially add a default case here)

// Option 2: dispatching
dispatch func DoTheThing(asset Asset):
  output: result Foo (1..1)

  // set result: ... potentially add a default case here

implement func DoTheThing(basket Basket):
  set result: // do something with basket -> basketAttribute

implement func DoTheThing(loan Loan):
  set result: // do something with loan -> loanAttribute

implement func DoTheThing(security Security):
  set result: // do something with security -> securityAttribute

Using union (alternative: choice)
Summary:

Common attributes are duplicated for each union case.
Accessing common attributes is concise. Good!
Need to serialise the type name as well.
Discriminating types is concise. Good!
Validation to guarantee coverage of all cases is possible. Good!

Model
union Asset:
  Basket
  Loan
  Security

type Basket:
  identifier string (1..1)
  basketAttribute int (0..1)

type Loan:
  identifier string (1..1)
  loanAttribute number (0..*)

type Security:
  identifier string (1..1)
  securityAttribute date (2..2)

Access common attributes
asset -> identifier

Serialisation (of a basket)
{
  "__type__": "Basket"
  "identifier": "abc123",
  "basketAttribute": 42
}

Discriminate types
switch asset
  case Basket then // do something with asset -> basketAttribute
  case Loan then // do something with asset -> loanAttribute
  case Security then // do something with asset -> securityAttribute

Based on this analysis, I would propose the following.

The end goal is to support union types as described in the last section.
As a migration strategy, we first support the two enhancements (-> for nested attributes and switch) for one-of types.

as we are currently writing it, will look like this with the common elements in AssetBase:
union Asset:
  Basket
  Loan
  Security

type AssetBase:
  identifier string (1..1)

type Basket extends AssetBase:
  basketAttribute int (0..1)

type Loan extends AssetBase:
  loanAttribute number (0..*)

type Security extends AssetBase:
  securityAttribute date (2..2)

Does this change your analysis at all?
In fact, we have also made this slightly worse, as follows
type AssetBase:
  identifier assetIdentifier (1..1)

type AssetIdentifier extends Identifier:
  identifierType assetIdTypeEnum (1..1)

type Identifier:
  identifier string (1..1)

Which means, as it currently stands, when we need to reference an identifier, we need to do this:
basket -> identifier -> identifier

So there is an even stronger case for
asset ->> identifier

On the proposed migration strategy, can we leverage Minesh’s “pre-processing” concept to implement union on the front end that is actually implemented as one-of in the DSL?  That is:
View in Rosetta:
union Foo:
  Bar1
  Bar2
  Bar3

Implementation
type Foo:
  bar1 Bar1 (0..1)
  bar2 Bar2 (0..1)
  bar3 Bar3 (0..1)
    condition: one-of

Just a clarification on the model, as we are currently writing it, will look like this with the common elements in AssetBase:
union Asset:
  Basket
  Loan
  Security

type AssetBase:
  identifier string (1..1)

type Basket extends AssetBase:
  basketAttribute int (0..1)

type Loan extends AssetBase:
  loanAttribute number (0..*)

type Security extends AssetBase:
  securityAttribute date (2..2)

Does this change your analysis at all?

This should work fine!

In fact, we have also made this slightly worse, as follows
type AssetBase:
  identifier assetIdentifier (1..1)

type AssetIdentifier extends Identifier:
  identifierType assetIdTypeEnum (1..1)

type Identifier:
  identifier string (1..1)

Which means, as it currently stands, when we need to reference an identifier, we need to do this:
basket -> identifier -> identifier

So there is an even stronger case for
asset ->> identifier

Hm, the current proposal adds ->> support for one-of and union types only. Since the type Identifier is neither,
it wouldn't be possible to do that. What you describe here seems like a different use case as the ones in the original issue.

On the proposed migration strategy, can we leverage Minesh’s “pre-processing” concept to implement union on the front end that is actually implemented as one-of in the DSL? That is:
View in Rosetta:
union Foo:
  Bar1
  Bar2
  Bar3

Implementation
type Foo:
  bar1 Bar1 (0..1)
  bar2 Bar2 (0..1)
  bar3 Bar3 (0..1)
    condition: one-of

Currently investigating this. I took a quick look together with Minesh, and we came to the conclusion that it's easier said than done. There is a path I haven't explored yet - more to follow.

@Oblongs With regards to your point about:
basket -> identifier -> identifier

I don't think the proposal would allow simply to shorten as:
asset ->> identifier

Instead, the approach we discussed to eliminate the extra level on this one is to define:
AssetBase extends Identifier:
  identifierType assetIdTypeEnum (1..1)

But it's separate from the issue being discussed here.

 Although it's another "killer-feature", it's independent form the above and not on the critical path of migration:
When defining a union, it should be possible to declare an associated enum:
union Asset:
  Basket
  Loan
  Security
  as-enum AssetTypeEnum

And then it would be possible to use AssetTypeEnum as if it was explicity declared, e.g.:
type Collateral
  assetType AssetTypeEnum

Also how would that work in the "nested" union case?

@lolabeis Great points, some of which I have been consciously "forgetting", given they were not listed as requirements yet.

Proposal looks good. A few comments and questions:
1. In the 3rd proposal (`union`) when you use `switch`, does it assume that the path starts at (in the `Basket` case) `asset -> basket`, so you would directly start typing `basketAttribute`?

I see two options here. Just to recapitulate, the question is: how do I access basketAttribute in the following location? (xxx)
switch asset
  case Basket then xxx
  ...

Either:

In each branch, we use item to indicate asset with its type narrowed down to the specific case, e.g., Basket. This would mean you could directly type basketAttribute, which would be syntactic sugar for item -> basketAttribute:

switch asset
  case Basket then basketAttribute
  ...

In case you are switching on a more complex expression, this improves conciseness even more, e.g.,
switch this -> is -> some -> long -> path -> asset
  case Basket then basketAttribute        // instead of having to write this -> is -> some -> long -> path -> asset -> basketAttribute
  ...

One potential downside is that it redefines item, which when combined with other operations that define item (such as extract) can be confusing. Fictive example:
reportableEvents
  extract switch reportableInformation -> asset // suppose `reportableInformation` has an asset attached to it.
    case Basket then Process(basketAttribute, reportableInformation) // This won't work: reportableInformation suddenly becomes "unavailable". A modeller would have to name their `reportableEvent` explicitly.
    ...

In each branch, the type of asset will change to the actual narrower type, e.g., Basket. This would mean a modeller would have to refer to asset -> basketAttribute to access to attribute:

switch asset
  case Basket then asset -> basketAttribute
  ...

When switching over long expressions, this could be cumbersome, although rewriting the expression can be avoided using extract, e.g.,
this -> is -> some -> long -> path
  extract
    switch asset
      case Basket then asset -> basketAttribute
      ...

Currently I'm leaning towards the first option.

2. Can some of a `union` underlying types be a `union` as well?

Yes, each of the union cases can be of any type, including data types, enumerations, basic types and other union types. In the long term I see additional benefits such as being able to conform to regulations that require us to either output a number or a string, e.g.,
union NumberOrString:
  number
  string

type Foo:
  bar NumberOrString (1..1)

which can then be serialised into
{
  bar: 42
}

or
{
  bar: "42 ounces"
}

This is something which currently is impossible to model with Rune.

In this case, can you specify how the switch statement, which likely needs nesting, would work? Please use the following example:
union Instrument:
  Security
  Loan

union Asset:
  Basket
  Instrument

Great question! I think supporting a "flat" switch, even for nested unions, will be the most easy to read and write, so you would be able to do something like
switch asset
  case Basket then ...
  case Security then ...
  case Loan then ...
// or, if only the common attributes of Instrument are relevant:
switch asset
  case Basket then ...
  case Instrument then ...

Potentially, they could also be "mixed" to provide default cases for nested unions. Suppose that Instrument had a third option called AnotherInstrument, then one could write something like this:
switch asset
  case Basket then ...
  case Security then ...
  case Instrument then ... // this catches both `Loan` and `AnotherInstrument`

Note that the order of cases then starts to matter. Writing case Instrument and then case Security should be forbidden by the DSL, since the latter case will never be reached.

3. Would you allow the following expression: `asset -> basket -> basketAttribute` (which means the DSL must associate some default name to each attribute), or would you only allow `switch` statements or calling the common attributes on a `union` type?

This would be part of the migration strategy, but in the end I would disallow this kind of direct access of attributes that are not common, unless a compelling use case arises.

4. Finally I think the `choice` naming alternative is more appropriate than `union` indeed.

To give it a try, I will use choice in my following responses. :)

On the proposed migration strategy, can we leverage Minesh’s “pre-processing” concept to implement union on the front end that is actually implemented as one-of in the DSL?

Update on this one: this is starting to look promising. We will probably follow this strategy as a quick win, and then incrementally start improving it.

 Although it's another "killer-feature", it's independent form the above and not on the critical path of migration:
When defining a union, it should be possible to declare an associated enum:
union Asset:
  Basket
  Loan
  Security
  as-enum AssetTypeEnum

And then it would be possible to use AssetTypeEnum as if it was explicity declared, e.g.:
type Collateral
  assetType AssetTypeEnum

Also how would that work in the "nested" union case?

Interesting. I think this wouldn't be too hard to add. Like you mention, the trickiness lies in how to handle nested choice types. To take your example from before:
choice Instrument as-enum InstrumentEnum: // another syntax suggestion
  Security
  Loan

choice Asset as-enum AssetEnum:
  Basket
  Instrument

I think the most useful interpretation is to flatten again. I assume the use case of representing a choice type as an enum is to indicate the actual type of an instance. Since an actual will always be either a Basket, a Security or a Loan, and never an Instrument, I think it that should be the case for the enum as well. I.e., AssetEnum would be equivalent to the following.
enum AssetEnum:
  Basket
  Security
  Loan

But perhaps it's best to raise a separate issue for this.

I would like to add an alternative switch syntax to the discussion that @lolabeis proposed, which I also quite like:
asset switch
  Basket then <expr>,
  Security then <expr>,
  Loan then <expr>,
  <default expr> // this is optional

One "use case" I didn't add to the comparison, but which would have been useful, is how to go from a specific type to a choice type, e.g., given a function that accepts an Asset as input, and given a variable of type Basket, how do I call this function?
Current solution (one-of)
Need to wrap it in an Asset constructor.
ProcessAsset(Asset { basket: basket, ... })

This is cumbersome!
Using extends
Works out of the box:
ProcessAsset(basket)

Using the proposed choice types
Works out of the box. No need to wrap!
ProcessAsset(basket)

Summary of the implementation plan
Below are four steps to get us from the current state to full support for choice types.
1. Choice types are supported as syntactic sugar of one-of types.
This is a quick win. E.g.,
choice Asset:
  Basket
  Loan
  Security

is syntactic sugar to
type Asset:
  Basket Basket (0..1)
  Loan Loan (0..1)
  Security Security (0..1)

  condition Choice:
    one-of

2. Killer-feature: common nested attributes of one-of (and choice types) can be accessed via a new ->> operator.
E.g.,
basket ->> identifier

This should also work for attributes that are nested with multiple levels of one-of types.
3. Remove syntactic sugar (dedicated choice types), add support for switch expressions, support accessing common attributes with ->, and support (de)serialisation with @type (except for basic types).
A couple of things change at this point.

A basket can now directly be passed to a function expecting an asset, instead of wrapping it in Asset { basket: basket, ... }. The Asset {...} constructor syntax disappears.
Accessing common attributes in choice types changes from ->> to ->. For one-of types, the ->> syntax stays the same.
Discriminating a choice type now must happen by using a switch. All checks of the form if asset -> basket exists then will need to be refactored. E.g.,

asset switch
  Basket then <item is now of type Basket>,
  Loan then <item is now of type Loan>,
  Security then <item is now of type Security>,
  default then <optional default case>

Nested choice types are flattened out.
4. Serialisation becomes less nested. E.g., what used to be
{
  "basket": {
    "identifier": "abc123"
    "basketAttribute": 42
  }
}

now becomes
{
  "@type": "Basket",
  "identifier": "abc123",
  "basketAttribute": 42
}

Note that this can only be done after the migration to Translate 2.0.
4. Add support for using choice types as enum.
E.g.,
choice Asset as-enum AssetTypeEnum:
  ...

type Underlier:
  assetType AssetTypeEnum (1..1)

In an expression:
if underlier -> assetType = AssetTypeEnum -> Basket
then ...

I would like to add an alternative switch syntax to the discussion that @lolabeis proposed, which I also quite like:
asset switch
  Basket then <expr>,
  Security then <expr>,
  Loan then <expr>,
  <default expr> // this is optional

This would be better aligned with other operators such as extract.

Fully support this, and I was about to suggest it 😄.
I think this would also allow you to do nested choice more elegantly - and I suggest using square bracket [] to be consistent with nesting of list operators. Re-using the same example as above :
asset switch
  Basket then <expr>,
  Instrument then switch [
    Security then <expr>,
    Loan then <expr>
    ],
  ...

Currently I'm leaning towards the first option.

Agree with this.
With regards to the issue of redefining item, I think it's consistent with the nesting of list operators: to access a previously defined item, it must be named.

Also with the switch syntax now redefined to be aligned onto the list operator syntax, all of those should be allowed:
Direct attribute access (in line with simpler rule syntax):
asset switch
  Basket then basketAttribute -> ... ,
  Security then securityAttribute -> ... ,
  Loan then loanAttribute -> ...

Using default item:
asset switch
  Basket then item -> basketAttribute -> ... ,
  Security then item -> securityAttribute -> ... ,
  Loan then item -> loanAttribute -> ...

Using named item:
asset switch a [
  Basket then a -> basketAttribute -> ... ,
  Security then a -> securityAttribute -> ... ,
  Loan then a -> loanAttribute -> ...
  ]

One "use case" I didn't add to the comparison, but which would have been useful, is how to go from a specific type to a choice type, e.g., given a function that accepts an Asset as input, and given a variable of type Basket, how do I call this function?
Current solution (one-of)
Need to wrap it in an Asset constructor.
ProcessAsset(Asset { basket: basket, ... })

This is cumbersome!
Using extends
Works out of the box:
ProcessAsset(basket)

Using the proposed choice types
Works out of the box. No need to wrap!
ProcessAsset(basket)

So you could pass a variable of type basket to a function that takes Asset as input - this is cool!

I think supporting a "flat" switch, even for nested unions, will be the most easy to read and write

With the way you redefined the switch syntax, it allows you to do nesting more easily - See above ☝.
Your flat switch suggestion works and is quite concise, but at the expense of introducing an ordering concern, as you point out. It involves a little magic, whereas the explicit switch nesting is more transparent.

Below are four steps to get us from the current state to full support for choice types.

Your implementation plan looks sensible. There is potentially a step 5, where we may be able to get rid of the one-of syntax (and consequently of ->>) altogether, if we manage to replace all occurences using choice - TBD.

The inverse scenario to defining a choice as also available as an enum also exists.
We already have this enum (simplified):
enum currencyEnum:
   EUR
   GBP
   USD

It might be interesting to be able to say
enum currencyEnum as-choice Cash:
   EUR
   GBP
   USD

Of course, it would be possible to refactor currencyEnum to become a choice data type with as-enum but its primary use will be as an enumerator and only edge case as a choice data type.

Thanks for all of the feedback. Since there is a consensus for the initial plan (steps 1 and 2 of https://github.com/finos/rune-dsl/issues/747#issuecomment-2105050958), I will start development for those. Once we get to a stage were we can start the rest of the proposal, we can summarise and continue these threads in a separate issue.

@SimonCockx Can you create new issues for the later phases, ie switch

Created issues here:
https://github.com/finos/rune-dsl/issues/797
https://github.com/finos/rune-dsl/issues/798


Crash if surface is deleted during debounce

The mod Factory Search (1.11.0) caused a non-recoverable error.
Please report this error to the mod author.

Error while running event FactorySearch::on_tick (ID 0)
LuaSurface API call when LuaSurface was invalid.
stack traceback:
	[C]: in function '__index'
	__FactorySearch__/scripts/search.lua:498: in function 'blocking_search'
	__FactorySearch__/scripts/search.lua:589: in function <__FactorySearch__/scripts/search.lua:581>

To reproduce, sandbox and create a new surface. Initate a search (on all surfaces) and then during the debounce quickly delete the surface. Error appears.
This happens frequently in mods like SE where it's constantly moving space ships around which are creating/deleting surfaces constantly.

Fixed in v1.11.1. Sorry for waiting so long!


Page cache exceptions won't work (ssl)

Hello. I noticed that there is something wrong with page cache exception rules. My website uses ssl. By default the option to cache https requests is disabled, so nothing gets cached. When I enable that option - almost everything gets cached, regardless of exceptions. For example, wp-login.php cached, pages with query strings - cached, even though the option to cache them is disabled. Only logged in users don't get cached. Defining DONOTPAGECACHE constant works, which is what I'm going to use, it is more flexible than regex matching rules anyway. Just wanted to report this issue.

Oops...just checked and noticed it does indeed cache files like wp-login.php   Sorry about that!  The fix has been implemented.  Give it a try to see if it resolves all your other problems too.   If it doesnt i will give it a more thorough check.
To save you time just open the file: w3-total-cache/lib/W3/PgCache.php and remove line 702.  It is the line that says:
$accept_uri[] = $_SERVER['PHP_SELF'];

I can confirm it works properly now. Thank you for fixing this.


task: Update KV functions for new task scheduler

The current task kv service does a significant part of the scheduler's work. This needs to change.  With the update coming to the scheduler we need the kv store to have a few functions defined to help with this new process. Additionally once we switch to using the new scheduler we will be able to remove some of the functions that are no longer required.
Additions so far(#14183):
type Checkpointer interface {
	// Checkpoint saves the last checkpoint a id has reached.
	Checkpoint(ctx context.Context, id ID, t time.Time) error

	// Last stored checkpoint.
	Last(ctx context.Context, id ID) (time.Time, error)
}

We may want to create a wrapper function so the kv service doesn't have a "Last" function that isn't clear what it means or which service it is meant to implement.
additionally the executor will need:
	CreateRun(ctx context.Context, taskID influxdb.ID, scheduledFor time.Time) (*influxdb.Run, error)

	// PopManualRun pulls a manual run from the list and moves it to currently running.
	// Returns the run that was popped.
	PopManualRun(ctx context.Context, taskID influxdb.ID) (*influxdb.Run, error)

these functions are still work in progress and are related to issue #14184
Once we have implemented this new system and are no longer using the old scheduler we will be able to remove:
	CreateNextRun(ctx context.Context, taskID influxdb.ID, now int64) (RunCreation, error)
	NextDueRun(ctx context.Context, taskID influxdb.ID) (int64, error)

closed by #14252  and by #14257


please remove the discription regarding Office 365 authentication

[Enter feedback here]
please refer to the discission IcM 243029169.
Office 365 authentication is dependent on Regional Discovery Service, and shutting down of Regional Discovery Service is ongoing. This, Office 365 authentication of Azure Data Factory doesn't work any more.
https://docs.microsoft.com/en-us/power-platform/important-changes-coming#regional-discovery-service-is-deprecated
Please remove the discription regarding Office 365 authentication, or add the discription to navigate the customer to use Azure AD service-principal and key authentication instead.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 304d4c46-c0ac-05c9-15eb-d8b677ec5e90
Version Independent ID: 36a79d36-2b72-2722-68e5-e167f9a2cd41
Content: Copy data in Dynamics (Microsoft Dataverse) - Azure Data Factory
Content Source: articles/data-factory/connector-dynamics-crm-office-365.md
Service: data-factory
GitHub Login: @jianleishen
Microsoft Alias: jianleishen

thank you @TakiyaNakano-MSFT . We are working on this to guide user migrate from O365 to SP (see this: https://docs.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-guide#error-code-dynamicsfailedtoconnect). Meanwhile, when choosing O365, it's been redirect to GDS already.

@KranthiPakala-MSFT  we don't yet have immediate plan to remove this term on UI but backend it already redirects user to GDS.  refer to the O365 deprecation plan and we will plan it properly along with this. https://docs.microsoft.com/en-us/power-platform/important-changes-coming#deprecation-of-office365-authentication-type-and-organizationserviceproxy-class-for-connecting-to-dataverse

It seems like the feedback has been handled. I'll close the issue for now, assuming there's not any additional follow-up needed, but let us know if that's incorrect. Thanks! Jason
#please-close


Pulmonary veins don’t clip

In some cases for the fire and ice study the veins do not clip - possibly related to the orientation of the file. Specific case folder has been shared with Jose

This issue is thought of being related to the orientation of the files. As part of fixing issue #13, we have included a module where the orientation is set to RAI in every image converted (regardless of the DICOM converter). Thus, I will close this issue, with the possibility if reopening it if the problem is not fully fixed.


Consumer-based "satisfy" assertions for elements in arrays and iterables

Summary
Consider extending the assertions API for arrays and iterables with satisfy* methods accepting multiple  Consumer<? super ELEMENT>s. The consumers probe the individual elements of the tested collection.
The additional methods could be:

satisfy(Consumer<? super ELEMENT>... consumers): For each element of the tested collection the (remaining) consumers are called; if a consumer accepts the element, then the element and that consumer are removed from their collections and the next element is probed.
satisfyExactly(Consumer<? super ELEMENT>... consumers): The tested collection must have the same size as the consumers. Each element of the tested collection is fed to the corresponding consumer.
satisfyExactlyInAnyOrder(Consumer<? super ELEMENT>... consumers): The tested collection must have the same size as the consumers. For each element of the tested collection the (remaining) consumers are called; if a consumer accepts the element, then the element and that consumer are removed from their collections and the next element is probed.

This would allow more flexible assertions on arrays and iterables.
Example
assertThat(trilogy).satisfy(movie -> {
    // the two Towers
    assertThat(movie.getReleaseDate()).isEqualTo(parse("2002-12-18"));
    assertThat(movie.getTitle()).contains("Towers");
  },
  movie -> {
    // the fellowship of the Ring
    assertThat(movie.getReleaseDate()).isBefore(parse("2002-12-17"));
    assertThat(movie.xrated).isFalse();
  }
);

assertThat(trilogy).satisfyExactly(movie -> {
    // the fellowship of the Ring
    assertThat(movie.getReleaseDate()).isBefore(parse("2002-12-17"));
    assertThat(movie.xrated).isFalse();
  },
  movie -> {
    // the two Towers
     assertThat(movie.getReleaseDate()).isEqualTo(parse("2002-12-18"));
     assertThat(movie.getTitle()).contains("Towers");
  },
  movie -> {
    // the Return of the King
    assertThat(movie.getTitle()).endsWith("King");
  }
);

assertThat(trilogy).satisfyExactlyInAnyOrder(movie -> {
    // the two Towers
    assertThat(movie.getReleaseDate()).isEqualTo(parse("2002-12-18"));
    assertThat(movie.getTitle()).contains("Towers");
  },
  movie -> {
    // the fellowship of the Ring
    assertThat(movie.getReleaseDate()).isBefore(parse("2002-12-17"));
    assertThat(movie.xrated).isFalse();
  },
  movie -> {
    // the Return of the King
    assertThat(movie.getTitle()).endsWith("King");
  }
);

Alternatives
The current API provides the following alternatives (for the proposed satisfyExactlyInAnyOrder method):

extracting multiple fields and assert matching tuples:

assertThat(trilogy)
  .extracting(Movie::getTitle, Movie::getReleaseDate, movie -> movie.xrated)
  .containsExactlyInAnyOrder(
    tuple("the Return of the King", parse("2003-12-17"), false),
    tuple("the fellowship of the Ring", parse("2001-12-19"), false),
    tuple("the two Towers", parse("2002-12-18"), false)
  );

This approach requires all values to match exactly, requires all elements to be tested for the same fields, is not type safe, and gets unreadable above three fields.

Individual assertThat for each desired element with filterOn distinguishing characteristic of that element:

assertThat(trilogy).hasSize(3);

assertThat(trilogy)
  .filterOn(movie -> parse("2002-12-17").compareTo(movie.getReleaseDate()) > 0)
  .hasSize(1)
  .element(0)
  .satisfies(movie -> {
    assertThat(movie.xrated).isFalse();
  });

assertThat(trilogy)
  .filterOn("releaseDate", parse("2002-12-18"))
  .hasSize(1)
  .element(0)
  .satisfies(movie -> {
    assertThat(movie.getTitle()).contains("Towers");
  });

assertThat(trilogy)
  .filterOn(movie -> movie.getTitle().endsWith("King"))
  .hasSize(1);

While this approach is more flexible than the first, it is more verbose and does usually not reflect the mental model of how a list is tested.

Hey, I really like the idea and would enjoy to tackle that.

@mgrafl  @lxndrdnxl  I need to look into that and likely split it into different isssues, sorry for the late answer!

@joel-costigliola Would split like to split it into three issues, one per method (satisfy, satisfyExactly, satisfyExactlyInAnyOrder)?

yes that would be good @mgrafl, let's start with satisfiy. thanks.

When I try to implement satisfy, one thing I found is very interesting. Example:
Suppose the matching table is as follows:

Consumer 1
Consumer 2
Consumer 3

Element a
yes
yes
yes

Element b
yes
no
no

Element c
yes
no
no

Then for every consumer, there is at least one element that can match this consumer.
However, I think the assertion should fail, because an element can only be used at most one time.
The problem is what the failure message should be? As shown in this example, every consumer has at least one matched element. As far as I am concerned, we cannot simply say that which consumer is not matched (especially when the matching table is larger and more complicated) because it depends on our algorithm.
It seems that we cannot show details in the failure message.

Hello Ting Sun,
Thanks implementing this! I agree with your matching strategy. It is
consistent with my initial description of satisfy(Consumer<? super
ELEMENT>... consumers): "For each element of the tested collection the
(remaining) consumers are called; if a consumer accepts the element, then
the element and that consumer are removed from their collections and the
next element is probed."
And yes, failure messages are tricky here. Although we could indicate which
elements were not matched, I would also prefer a simple failure message for
now and wait for feedback from users.
Best regards,
Michael
On Fri, May 29, 2020 at 4:14 PM Ting Sun<EMAIL_ADDRESS>wrote:

When I try to implement satisfy, one thing I found is very interesting.
Example:
Suppose the matching table is as follows:
Consumer 1 Consumer 2 Consumer 3
Element a yes yes yes
Element b yes no no
Element c yes no no
Then for every consumer, there is at least one element that can match this
consumer.
However, I think the assertion should fail, because an element can only be
used at most one time.
The problem is what the failure message should be? As shown in this
example, every consumer has at least one matched element. As far as I am
concerned, we cannot simply say that which consumer is not matched
(especially when the matching table is larger and more complicated) because
it depends on our algorithm.
It seems that we cannot show details in the failure message.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/joel-costigliola/assertj-core/issues/1621#issuecomment-635995154,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AEKYEEFLXADJQMO2XPVYK5LRT67NBANCNFSM4I26OZMQ
.

I totally agree with you! And I implemented it in this way. Thanks @mgrafl

@Sunt-ing, your implementation has brought a nuance of satisfy to my attention, that I had not thought of before:
As far as I can see, there are four different strategies:
1. "element-iteration":
For each element of the tested collection the (remaining) consumers are called; if a consumer accepts the element, then the element and that consumer are removed from their collections and the next element is probed. (This was my initial proposal.)
Example a (fails):
List<String> starWarsCharacters = newArrayList("Luke", "Leia", "Yoda");
assertThat(starWarsCharacters).satisfy(
    name -> {
       assertThat(name).contains("L");
     }, name -> {
       assertThat(name).doesNotContain("a");
     });
// "Luke" is probed and accepted by 1st consumer (contains "L").
// --> Element and consumer are removed.
// "Leia" is probed and none of the remaining consumers (i.e., 2nd consumer) accept.
// --> Failure

Example b (also fails):
List<String> starWarsCharacters = newArrayList("Luke", "Leia", "Yoda");
assertThat(starWarsCharacters).satisfy( // order of consumers changed
    name -> {
       assertThat(name).doesNotContain("a");
     }, name -> {
       assertThat(name).contains("L");
     });

This strategy depends on the ordering of elements to be tested.
Example c (succeeds);
List<String> starWarsCharacters = newArrayList("Leia", "Luke", "Yoda"); // order of elements changed
assertThat(starWarsCharacters).satisfy(
    name -> {
       assertThat(name).contains("L");
     }, name -> {
       assertThat(name).doesNotContain("a");
     });
// "Leia" is probed and accepted by 1st consumer (contains "L").
// --> Element and consumer are removed.
// "Luke" is probed and accepted by 2nd consumer (does not contain "a").
// --> Success

This is probably not what programmers would expect, since they usually have no control over the order of elements at this point.
2. "consumer-iteration":
For each consumer of the tested collection the (remaining)  elements are probed; if an element is accepted by the consumer, then that element and the consumer are removed from their collections and the next consumer is probed.
Example a (fails):
List<String> starWarsCharacters = newArrayList("Luke", "Leia", "Yoda");
assertThat(starWarsCharacters).satisfy(
    name -> {
       assertThat(name).contains("L"); // "Luke" matches and is removed from elements
     }, name -> {
       assertThat(name).doesNotContain("a"); // Fails because "Luke" was removed
     });

Example b (succeeds):
List<String> starWarsCharacters = newArrayList("Luke", "Leia", "Yoda");
assertThat(starWarsCharacters).satisfy( // order of consumers changed
    name -> {
       assertThat(name).doesNotContain("a"); // "Luke" matches and is removed from elements
     }, name -> {
       assertThat(name).contains("L"); // "Leia" matches
     });

However, this also depends on the order of elements.
Example c (succeeds):
List<String> starWarsCharacters = newArrayList("Leia", "Luke", "Yoda"); // order of elements changed
assertThat(starWarsCharacters).satisfy(
    name -> {
       assertThat(name).contains("L"); // "Leia" matches and is removed from elements
     }, name -> {
       assertThat(name).doesNotContain("a"); // "Luke" matches
     });

3. "consumers match distinct elements":
Verifies that the all the given consumers can be satisfied by elements in the iterable with an element at most satisfies one consumer. No order requirement. (This is the implementation proposed by @Sunt-ing in #1892.)
In other words, it checks whether there is any re-ordering of elements to be tested so that each consumer accepts the corresponding element.
Above examples a,b & c all succeed with this strategy.
Example d (fails):
List<String> starWarsCharacters = newArrayList("Luke", "Leia", "Yoda");
assertThat(starWarsCharacters).satisfy(
    name -> {
       assertThat(name).contains("k"); // "Luke" matches
     }, name -> {
       assertThat(name).contains("u"); // "Luke" already matched by first consumer
     });

This strategy does not depend on the order elements.
This approach has the highest computational complexity (each consumer is probed with each element and a mutual exclusion of satisfying elements must be checked). But even though the number of elements could be huge, the number of consumers will usually be quite small. So this should not be a problem.
4. "consumers match any elements":
For each consumer check whether at least one element in the iterable satisfies the consumer. One element can satisfy multiple consumers.
Above examples a,b & c all succeed with this strategy.
Example d (succeeds):
List<String> starWarsCharacters = newArrayList("Luke", "Leia", "Yoda");
assertThat(starWarsCharacters).satisfy(
    name -> {
       assertThat(name).contains("k"); // "Luke" matches
     }, name -> {
       assertThat(name).contains("u"); // "Luke" matches
     });

This strategy does not depend on the order elements.
This strategy would also allow for more consumers than elements to be tested.
Example e (succeeds):
List<String> starWarsCharacters = newArrayList("Luke");
assertThat(starWarsCharacters).satisfy( // more consumers than elements
    name -> {
       assertThat(name).contains("k"); // "Luke" matches
     }, name -> {
       assertThat(name).contains("u"); // "Luke" matches
     });

I currently prefer strategy 3 (consumers match distinct elements), but I would appreciate additional opinions @joel-costigliola.
In any case, the method's documentation should clearly explain the chosen approach.

@mgrafl Yes, it needs to be discussed. We need to choose useful ones from all the four strategies and pick appropriate API names for them. Then clarify them in the doc.

I have given it some more thought and I am convinced that the currently implemented strategy 3 (consumers match distinct elements) is most appropriate because it has the least unexpected surprises and it will easily fit with satisfyExactlyInAnyOrder().

At this assertion is named I would expect that all given consumers can be satisfied which means it's okay if multiple elements satisfy the same consumer (I think this is option 4.).
The following assertion would pass as both consumer are met, the first one by two elements.
List<String> starWarsCharacters = newArrayList("Luke", "Leia", "Yoda");
assertThat(starWarsCharacters).satisfy(name -> assertThat(name).contains("L"),
                                       name -> assertThat(name).contains("Y"));

If we want to go for option 3 (one element at most satisfies a consumer), we need to find a non ambiguous name.
@mgrafl @Sunt-ing, thoughts?

My initial idea was to reuse the naming pattern from contains, containsExactly, and containsExactlyInAnyOrder.
For containsExactly and containsExactlyInAnyOrder, the names already suggest a one-to-one correlation between elements and consumers.
But I agree that the method name satisfy is ambiguous.
On option 3: I think the current description is misleading and does not reflect to proposed implementation. It is not about preventing multiple elements from satisfying the same consumer.
I propose to correct the description: "Verifies that all given consumers can be satisfied by separate elements in the iterable under test. No order requirement."
How about subCollectionSatisfyInAnyOrder (the terms "subSet" and "subList" would also be possible but would suggest specific Java types)?
Or someElementsSatisfyInAnyOrder (someElementsIndividuallySatisfyInAnyOrder or someElementsSeparatelySatisfyInAnyOrder is a bit long for my taste)?
List<String> starWarsCharacters = newArrayList("Luke", "Leia", "Yoda");

// assertion passes as both consumer are met, the first one by two elements.
assertThat(starWarsCharacters).someElementsSatisfyInAnyOrder(
                               name -> assertThat(name).contains("L"),
                               name -> assertThat(name).contains("Y"));

// assertion fails because "Leia" satisfies "L", "Luke" satisfies "u", but no separate element satisfies "k"
assertThat(starWarsCharacters).someElementsSatisfyInAnyOrder(
                               name -> assertThat(name).contains("L"),
                               name -> assertThat(name).contains("u"),
                               name -> assertThat(name).contains("k"),
                               name -> assertThat(name).contains("e"));

if we make the parallel between satisfy and contains then my interpretation is correct, after all
contains is like satisfy + isEqualTo:
assertThat(starWarsCharacters).contains("Yoda", "Luke");

vs
assertThat(starWarsCharacters).satisfy(name -> isEqualTo(name).contains("L"),
                                       name -> assertThat(name).isEqualTo("Y"));

I propose to correct the description: "Verifies that all given consumers can be satisfied by separate elements in the iterable under test. No order requirement."

satisfyByDifferentElements is the best that I can find but it still sounds a bit awkward:
assertThat(starWarsCharacters).satisfyByDifferentElements(name -> isEqualTo(name).contains("L"),
                                                          name -> assertThat(name).isEqualTo("Y"));

Unless we find a crystal clear name, I'm not too keen to add this assertion, I'm fine with satisfy, satisfyExactly, satisfyExactlyInAnyOrder although this does not address the previous assertion.

How about containsElementsSatisfying?
assertThat(starWarsCharacters).containsElementsSatisfying(
                               name -> assertThat(name).contains("L"),
                               name -> assertThat(name).contains("Y"));

(Side node: I do not fully understand the consumers in your previous example isEqualTo(name).contains("L") and assertThat(name).isEqualTo("Y").)

Sorry bad copy paste, example updated.
containsElementsSatisfying is a clear name but it does not convey the fact that only a consumers are matched by different elements if that was your intention

containsElementsSatisfyingExactlyInAnyOrder, or is that too long?

A minor variation would be containsElementsThatSatisfyExactlyInAnyOrder (though I slightly prefer the containsElementsSatisfyingExactlyInAnyOrder variant).
Although it is a bit long, it conveys the intention of the option 3 (implemented by #1892).
If somebody finds a better (and shorter) name in the future, an alias method name can be added easily.

It would be great if this functions could ( or had sibling functions that did ) operate not on the collection ELEMENT but on a assertion over that element. It could be achieved by requiring an extra parameter which would be the assert factory. The signature could be something like:
public <A extends AbstractAssert<?,ELEMENT>>
ListAssert<SELF,ACTUAL,ELEMENT,ELEMENT_ASSERT> satisfiesExactly(
            Function<ELEMENT,A> assertFactory,
            Consumer<A>... consumers
)

Such a function would allow to avoid the recurring 'assertThat' such as:
assertThat(starWarsCharacters).someElementsSatisfyInAnyOrder(
                               name -> assertThat(name).contains("L"),
                               name -> assertThat(name).contains("u"),
                               name -> assertThat(name).contains("k"),
                               name -> assertThat(name).contains("e"));

allowing:
assertThat(starWarsCharacters).someElementsSatisfyInAnyOrder(
                               Assertions::assertThat,
                               name -> name.contains("L"),
                               name -> name.contains("u"),
                               name -> name.contains("k"),
                               name -> name.contains("e"));

Unfortunately this functions could not be overloads of the originally suggested ones, as it would make using them ambiguous to the compiler

Hey - was really wanting this assertion and when searching for existing issues before filing one, found this. Any way to get consensus on the naming? FWIW, only the exact versions are interesting to me, satisfyExactly and satisfyExactlyInAnyOrder, and perhaps they aren't as ambiguous? Even just satisfyExactly would work great.

@anuraaga, thanks for reviving the discussion.
If you have any naming suggestions for satisfy (aka. containsElementsSatisfyingExactlyInAnyOrder), please share them.
I agree that the other two methods satisfyExactly and satisfyExactlyInAnyOrder are less controversial and could be implemented right away.
For satisfyExactly, I expect the implementation to be straight forward.
For satisfyExactlyInAnyOrder, the implementation from #1892 could be combined with a length verification.
I will create separate issues for these two methods.

@jbytecoder, I like your idea of having an optional parameter for an assert factory. It makes the code easier to read if you have many consumers.
From a quick prototype, I don't expect any overloading problems if both method variants are introduced simultaneously. The most common usage of parameters will be the one from your example: first the method reference to Assertions::assertThat, followed by lambda expressions for the consumers. This is unambiguous and the compiler handles it well.
Only if the first parameter implements both Function and Consumer, the caller will have to cast it accordingly.
One tiny technical remark: I would specify the method signature as
public final <A extends AbstractAssert<?, ELEMENT>>
SELF satisfyExactly(Function<ELEMENT, A> assertFactory, Consumer<? super A>... consumers)

This would also support consumers of a super-type of A. For example:
// extracted variable also used in other assertions:
Consumer<AbstractCharSequenceAssert<?, ?>> consumer1 = AbstractCharSequenceAssert::isNotBlank;

assertThat(elements).satisfyExactly(
                     Assertions::assertThat,
                     consumer1,
                     s -> s.doesNotContain("a"));

If it's okay for @joel-costigliola, I will include your method variant in the new issues for satisfyExactly and satisfyExactlyInAnyOrder.


fix: add support for JB_ETH_PAYMENT_TERMINAL_V_3_1_1 in distribute payouts

What does this PR do and why?
Provide a description of what this PR does. Link to any relevant GitHub issues, Notion tasks or Discord discussions.
Screenshots or screen recordings
If applicable, provide screenshots or screen recordings to demonstrate the changes.
Acceptance checklist

[ ] I have evaluated the Approval Guidelines for this PR.
[ ] (if relevant) I have tested this PR in all supported browsers.
[ ] (if relevant) I have tested this PR in dark mode and light mode (if applicable).

Current dependencies on/for this PR:

main

PR #3927   👈

This comment was auto-generated by Graphite.


compute spherical grf to use gpu

Earth-2 MIP Pull Request
Description
Compute noise eon GPU
Checklist

[x] I am familiar with the Contributing Guidelines.
[x] New or existing tests cover these changes.
[x] The documentation is up to date with these changes.
[x] The CHANGELOG.md is up to date with these changes.
[x] An issue is linked to this pull request.

Dependencies

I tested this PR and it seems to work fine.

/blossom-ci

/blossom-ci

/blossom-ci

/blossom-ci


Issue #237: Add Discipline filter

Added filter for academic discipline and modified related tests to include this filter field.

Coverage increased (+0.004%) to 92.387% when pulling a37deed288b67279eed9ae01c2ac9389c2f75ab6 on discipline-filter into 30276ec184c691b448037c2379a7215002bd3919 on master.

Coverage increased (+0.003%) to 92.545% when pulling 6f44142160decdc6aec1297c5b7b9aee0293a57f on discipline-filter into 84850deda023501c046b6a6af9f7112594ecded4 on master.


Improved Japanese text slightly.

I fixed some errors in the Japanese text.

Thank you for doing this.  Much appreciated.


Add description and semantic to tables

looks good. before I merge this, can you addapt the somersault example ECU to include the new attributes?

done

thanks. merging

@andlaus could you publish a new version with this change?

Ok. I've just created the 1.6.0 release. it should be uploaded to pypi within a few hours..


imagecodecs.numcodecs.register_codecs: RuntimeError

Hi @cgohlke,
In a CI run, I just saw a RuntimeError: dictionary changed size during iteration in register_codecs
It only failed on Python3.11 on ubuntu. And rerunning the job could not reproduce the error, so this might be just some artefact.
 Anyway, I'll change it to list(globals().items()).
Is register_codecs called from different threads or is the imagecodecs-numcodecs package installed?

imagecodecs-numcodecs is not installed in the CI and the crash occurred when running pytest.
But I can't reproduce locally and it hasn't occurred since in the CI.
Maybe it was just some single event error on the runner 🤷


run it on a GPU

Hi,@wolterlw ,thanks for your nice job.  I want to know why  hand_tracker.py don't use the GPUs (I find it only use computer's CPUs).
If I want to use that on a computer with GPU , how should I do ?
Looking forword your reply! Thanks alot.

I use tflite interpreter to run Mediapipe's models. Currently tflite only supports mobile GPUs so to run the model on a desktop/laptop GPU you'd have to extend tflite with a new "delegate" as they call it, which I believe is far from easy.
I believe it would be possible to use models from this repository and run them as a regular tensorflow model on the GPU, but I've only seen this repo recently so didn't have time to validate those models and incorporate them into the pipeline.

@wolterlw Thanks for your suggestions  ,I will try it .  ^_^

@lilyswang do you, by any chance have that repo forked or downloaded? because the source was removed by the author.


formatted parameter names

Type of Change

[ ] Bug fix (non-breaking change, fixes # )
[x] New feature (non-breaking change, adds functionality, fixes # )
[ ] Breaking change (effects multiple commands or functionality, fixes # )
[ ] Ran manual Pester test and has passed (`.\tests\manual.pester.ps1)
[ ] Adding code coverage to existing functionality
[ ] Pester test is included
[ ] If new file reference added for test, has is been added to github.com/sqlcollaborative/appveyor-lab ?
[ ] Nunit test is included
[ ] Documentation
[ ] Build system

Purpose

Approach

Commands to test

Screenshots

Learning

I love some approved PRs! haha thank you  both🙇🏼


Update DG (Iterations)

Codecov Report

Merging #133 (d0d7122) into master (ca7a249) will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master     #133   +/-   ##
=========================================
  Coverage     61.04%   61.04%
  Complexity      758      758
=========================================
  Files           131      131
  Lines          2973     2973
  Branches        322      322
=========================================
  Hits           1815     1815
  Misses         1067     1067
  Partials         91       91

Impacted Files
Coverage Δ

.../java/seedu/address/model/iteration/Iteration.java
97.14% <ø> (ø)

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


Failed to build flash-attn

torch==2.0.1,cuda11.4

This repository is a total mess.  It's better to use the PyTorch implementation of flash-attention.

Hi @nanowell u are absolutely correct I have been trying to setup flash attn with poetry for at least 5 hours with no luck. Could you please provide a link for pytorch implementation of flash-attention? :/

ONE method I found may solve the error:
my env
OS:linux
torch:2.1.1+cu121
GPU:A800
python:3.9.9
1.update the env config
(bash) cd /usr/local
(bash) ls
you can see more file like bin cuda cuda-11.x cuda-12.x
config your correct version to bashrc
(bash) vim ~/.bashrc
append content like..
export CUDA_HOME=/usr/loca/{your cuda version}
for example to me, 'CUDA_HOME=/usr/loca/cuda-12.2'
save and exit then execute refresh config
(bash) source ~/.bashrc
2.reinstall flash-attn compile with source
(bash) MAX_JOBS=4 python setup.py install
but if you execute the command above you may get another error: cutlass/numeric_types.h:No such file or directory
you could solve through:
(bash) MAX_JOBS=4 pip install flash-attn


Implement resource permission evaluation in security

Description
[Describe what this change achieves]
This change adds resource permission evaluation into the Security plugin framework. This is accomplished by adding a new class ResourcePrivilegesEvaluator and expanding the RoleV7 definition to account for the new Resource class.
Issues Resolved
https://github.com/opensearch-project/security/issues/4562
Testing
[Please provide details of testing done: unit testing, integration testing and manual testing]
Check List

[ ] New functionality includes testing
[ ] New functionality has been documented
[ ] New Roles/Permissions have a corresponding security dashboards plugin PR
[ ] API changes companion pull request created
[ ] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

FYI I have been exploring the idea I posted about here on my fork here.
Ultimately, I think the resource owner needs to be empowered to determine how they want to share the resource with others without going through the admin of the cluster.
In order for ml-commons to adopt the new scheme that security will provide, how will the public, private and restricted attributes translate to this new scheme?

HI @cwperks, thanks for your thoughtful comments.
I tried to address your thoughts below.

Ultimately, I think the resource owner needs to be empowered to determine how they want to share the resource with others without going through the admin of the cluster.

I think most patterns we follow force users to rely on the cluster operator/administrator in some ways. I understand you would like to model resources as registered fields in a per-resource indices which any user can create. This is quite different than how roles work and seems better suited outside of the roles system. Perhaps, you discussed things elsewhere, but based off https://github.com/opensearch-project/security/issues/4500#issuecomment-2233595019 it seemed like this discussion was being tabled.

In order for ml-commons to adopt the new scheme that security will provide, how will the public, private and restricted attributes translate to this new scheme?

I would expect the ML-commons plugin to override ActionRequests for their plugin and add the associated resources to the new method being added (in core):
/**
     * Should be overwritten by implementing plugins with the resources each actionRequest requires.
     * Bypasses resource evaluation by default.
     */
    public List<String> getResources() {
        return Collections.emptyList();
    }

From there, I would expect there to be a role i.e. ml-all-access which included the Resource with a pattern 'ml-commons:*' or whatever the resources were called. Similarly, they would need to define roles which had the appropriate analog for private and restricted as shown in the documentation. The assignment of the appropriate roles would have to be carried out by an administrator.

@DarshitChanpura @cwperks It seems like there is still decisions to be made around the design of the feature. I would describe DC's intention as more of "adding resource controls to role based access controls" while Craig seems to be more interested in adding an independent access system for resources. I suggest we reach consensus on this before moving forward.
It is probably best to carry this on over at https://github.com/opensearch-project/security/issues/4500


Update background colors in Storybook

From Brandon O'Keefe:
There was a question about what colors should we add or have in storybook for backgrounds.  Final decisions on this are still being looked at in contextual scenarios and is requiring consideration for alignment between multiple products.
I plan to have more guidance in the Nimble design spec eventually.
For now variations would be:
Light UI
Background: White | Black7
Header: Black15 | Black30
Dark UI
Background: Black85 | Black88 |
Header: Black80 | Black85 | Black 91

Done - a while back


Update content security policy to display NTAS iframe

🗣 Description

Added www.dhs.gov to the Content Security Policy

💭 Motivation and context
Within the ReadySetCyber Dashboard footer, an Iframe has been added to display the National Terrorism Advisory System (NTAS) widget. This widget is displayed within a local environment, but once in staging it is prevented from loading by the content security policy.
This closes issue #242

✅ Pre-approval checklist

[x] This PR has an informative and human-readable title.
[x] Changes are limited to a single goal - eschew scope creep!
[x] All relevant type-of-change labels have been added.
[x] I have read the CONTRIBUTING document.
[x] These code changes follow cisagov code standards.
[ ] Tests have been added and/or modified to cover the changes in this PR.
[x] All new and existing tests pass.

✅ Pre-merge checklist

[ ] Revert dependencies to default branches.
[ ] Finalize version.

✅ Post-merge checklist

[ ] Create a release.

Closing this Pull request, covered in detail by @Matthew-Grayson on PR #244


Spelling comments

Nothing here should impact the build, however, there are a lot of files. In order to make it possible for me to exclude things which would have, I selectively included changes on a per-directory basis.
This means there are 23 commits, each of which should be fairly easy to review independently.
This is PR 5 of N:
The following is a list of some spelling fixes which have been made to the impacted files. It is not necessarily a complete list of changed words -- if a word is misspelled in code or resource strings the quick tool I used to list what's changed might say "that's still present" and not have highlighted it.

atleast
Defragmentor
deserialization
deserialized
deserializing
fragmentor
ICloneable
IEnumerable
iformattable
kerberos
lifecycle
Microsoft
Minrunspaces
overridee
parameterset
stopprocessing
telementry
typenames
unmarshal

Hi @jsoref, I'm your friendly neighborhood Microsoft Pull Request Bot (You can call me MSBOT). Thanks for your contribution!
You've already signed the contribution license agreement. Thanks!
The agreement was validated by Microsoft and real humans are currently evaluating your PR.
TTYL, MSBOT;

I have gone through the changes, all changes happen in comments. LGTM.


Implement multi-page dashboard as a GUI for duqtools

I was just playing around a bit what this could look like.
Prototype for #210

I re-used some of the code from this PR in #256 to move the merge tool to a new page. This can be closed.


Unable to fetch quotes

The functionality of fetching quotes is not working, would like to work on this issue

i assigned this to you,

make sure you star ⭐ repo if like and follow on github

Hi, I see that this issue is still not resolved. Please assign this to me under hackoctoberfest.
@mohitahlawat2001


[VS Code] Joining collaboration session: Live Share was unable to start the agent fast enough. Reload to try again

Error:
Joining collaboration session: Live Share was unable to start the agent fast enough. Reload to try again
Steps to Reproduce:

Version Data

extensionName
VSLS

extensionVersion
1.0.45

protocolVersion
2.2

applicationName
VSCode

applicationVersion
1.33.1

platformName
Windows

platformVersion
10.0.16299

Thanks for submitting this issue, @subashini08.
Your logs may provide us with helpful information. You can access them via the Live Share: Export Logs command from the command palette (ctrl + shift + p) and attach them to this issue.

I'm closing this issue due to inactivity and lack of information. If you hit this problem again, please feel free to reopen.


User is not alerted when a task fails

What

When a background task fails, the user is not alerted in any way. When editing a product, the user is informed that the change will take place in the background. If an exception occurs, the only way for a user to know is for them to open the DEV mode screen and look at the Pending contributions section. The typical user is not going to do this. Another note that I have not checked yet is whether exceptions clear on their own. Some tasks appear to have a 10 minute wait period assigned to them, though exceptions do not. As these break some functionality of the app, they may need to clear on their own (If they do not already).

Steps to reproduce the behavior

Go to a product
Edit the packaging components
See error in the DEV mode section

Expected behavior
A user should be informed of an issue and be prompted with a potential solution to the issue, whether this is to wait and try again later etc.
Why

Currently the user has no way of knowing if something did not work correctly.

Part of

#4627
#4637
#4639

Smartphone model

Device: Pixel 6
OS: Android 13
App Version: 4.10.0+1230-ZXing-FDroid

Hi @NixedSec!
The first question is: how should we alert the user? A clickable widget that appears all the time when there's an error, something like a red exclamation mark in the app bar?

That sounds like a good idea. At least at the time being, the error will need to persist for as long as the exceptions do. Otherwise the user would be under the belief that everything is working as expected.

@teolemon How would you like the users to be warned of a background task error, UI/UX-wise?


integration list - what about using 2 rows to show all the data

This is a...

[x] Feature request
[ ] Regression (a behavior that used to work and stopped working in a new release)
[ ] Bug report
[ ] Documentation issue or request

Description
@dongniwang @seanforyou23 @mcoker
Could we use like 2 rows within a list item to show all the data and stuff we need to show for an integration on the integration list page?  For example if we had more than one row in a list item, this name maybe wouldn't have to wrap:

and we'd probably have an easier time managing how the app looks at various widths.

If I make it a bit narrower, I do get rows, and look the list items are almost more consistent in height:

Name still wraps though.  In the old app we truncated strings, but that's such a workaround.

What if we made use of the expanded state a list item could man have instead? I’m not sure we need all the info all the time on the screen, and we could actually tuck more borrowing from the details page.

Do we already not show the description in the list view?
Yeah, I was going to suggest we use the expandable list view but not sure if that's what you mean by 2 rows?
The other thing is we can probably update the "configuration required" to just the warning icon with a tooltip.

an expandable list view would be kinda cool, definitely starting with putting the description in there.
What I mean by 2 rows is along the lines of what you see here

Would setting the stacked property to true in the ListView.Item work?


Get currently active node addresses.

How to get connected peers' IP addresses?

There's a function for this feature at casper-client.


Add "Obsidian" extension

Description
Add "Obsidian" extension.
This extension lets users interact with notes written in the knowledge management and note taking app Obsidian.
You can search, view, open, edit, copy and paste notes.
Type of change

New extension

Screencast
https://user-images.githubusercontent.com/67844154/145678597-cd652295-1f95-4d59-ac0a-80bb561857d8.mp4
Checklist

[x] I read the extension guidelines
[x] I read the documentation about publishing

Hi @pernielsentikaer,
thank you for pointing out this issue, I just committed a fix and accepted the change for the commands title and subtitle.

Thank you for the feedback.
I removed the "Empty state flicker" as described in the article.
"Hide links" will remove links in the Wikilink format: [[Link to another note]] -> Link to another note. This link format is used by nearly everyone in the Obsidian community to connect notes. When copying or viewing notes with the raycast extension these links might be distracting and as they don't include any extra information like a URL they can safely be hidden.
I won't remove normal markdown links as they do contain information like a URL. Additionally these markdown links will display just fine in "Quick Look".

Hi @pernielsentikaer,
thank you for the great review 😊

Thank you for reviewing and approving the extension.
To update the extension I will have to create new forks with pull requests right?

Yes, @marcjulianschwarz, you're right 👍


feat: support for extra pre-stop hook command

Codecov Report
Attention: Patch coverage is 71.42857% with 2 lines in your changes are missing coverage. Please review.

Project coverage is 28.83%. Comparing base (e235df4) to head (4582aeb).

Files
Patch %
Lines

pkg/config/setting.go
0.00%
1 Missing and 1 partial :warning:

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #909      +/-   ##
==========================================
+ Coverage   28.81%   28.83%   +0.02%
==========================================
  Files          47       47
  Lines        5959     5964       +5
==========================================
+ Hits         1717     1720       +3
- Misses       4077     4078       +1
- Partials      165      166       +1

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Monitoring Improvements

A more detailed dashboard for each of the wubloader components
A dashboard for each of the nodes
Turn off flamegraph to improve reliability of prometheus.

I'd like to do some of this as an exercise in learning Grafana and Prometheus.


The guidance on this page for upgrading will not work

This script and guidance worked a month ago. But today when I tried this, I encountered an error claiming "Public IP in use" and couldnt be used.  I'm not sure if there was internal upgrade for public IP in the past month. But currently the script couldnt work. It will create a new standard load balancer without frontend IP and no configurations or backend pools. This is really frustrating.
PS C:> AzurePublicLBUpgrade.ps1 -oldRgName "HJAAZ1RGFEPRD01" -oldLBName "HJAAZ1PSFTPLB03" -newLbName "HJAAZ1PRDSFTPLB03"
Static
Static
Set-AzPublicIpAddress : Public IP address /subscriptions/144d729c-123e-4499-9bb2-08c73da4313a/resourceGroups/HJAAZ1RGFEPRD01/providers/Microsoft.Network/publicIPAddresses/HJAAZ1PSFTP03-PIP is in use by ipconfig
/subscriptions/144d729c-123e-4499-9bb2-08c73da4313a/resourceGroups/HJAAZ1RGFEPRD01/providers/Microsoft.Network/loadBalancers/HJAAZ1PSFTPLB03/frontendIPConfigurations/LoadBalancerFrontEnd and cannot be updated.
StatusCode: 400
ReasonPhrase: Bad Request
ErrorCode: PublicIPAddressInUseCannotUpdate
ErrorMessage: Public IP address /subscriptions/144d729c-123e-4499-9bb2-08c73da4313a/resourceGroups/HJAAZ1RGFEPRD01/providers/Microsoft.Network/publicIPAddresses/HJAAZ1PSFTP03-PIP is in use by ipconfig
/subscriptions/144d729c-123e-4499-9bb2-08c73da4313a/resourceGroups/HJAAZ1RGFEPRD01/providers/Microsoft.Network/loadBalancers/HJAAZ1PSFTPLB03/frontendIPConfigurations/LoadBalancerFrontEnd and cannot be updated.
OperationID : 018c25f0-0a50-403d-a5f6-09f2eaefef7c
At C:\Program Files\WindowsPowerShell\Scripts\AzurePublicLBUpgrade.ps1:125 char:5

Set-AzPublicIpAddress -PublicIpAddress $pip

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

CategoryInfo          : CloseError: (:) [Set-AzPublicIpAddress], NetworkCloudException
FullyQualifiedErrorId : Microsoft.Azure.Commands.Network.SetAzurePublicIpAddressCommand

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: ab6eefa3-52f8-4172-46d3-c5a6c1f95370
Version Independent ID: 8a5653c5-158d-8154-451b-2840cf77b4d0
Content: Upgrade from Basic Public to Standard Public - Azure Load Balancer
Content Source: articles/load-balancer/upgrade-basic-standard.md
Service: load-balancer
GitHub Login: @irenehua
Microsoft Alias: irenehua

@AgnesLi666 , thank you for your feedback. We'll review this and get back to you shortly!

@AgnesLi666 , I have tried this script in my lab and received the same error and also as you mentioned correctly, it creates a new Standard Public load balancer without any frontend IP. Hence, we have assigned this issue to the respective product owner for review.

@irenehua , request you to look into the script and get it fixed as it is no longer working.

A new iteration of the PowerShell script has been published. Let me know if the issue still persists.
Thanks,
Irene

#please-close


Aws sam cli bounds

Motivation for this change
aws-sam-cli fails to build.
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[X] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[X] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[ ] Fits CONTRIBUTING.md.

Done.

Thanks @jonringer . Should I do a ZHF PR too? It looks like my last round of SAM CLI fixes landed on 20.09, but the breakage that was on master never came across.

I think this was only necessary because of a recent aws bump on master, 20.09 isn't affected https://github.com/NixOS/nixpkgs/pull/99276


Prefix match for routes

Hi,
I want to match the prefix :  /myapp/v5/resource or  /myapp/v6/resource
I tried below based on the documentation "https://www.envoyproxy.io/docs/envoy/latest/api-v2/api/v2/route/route.proto#envoy-api-field-route-routematch-regex" and it's not working.
"match": {
"prefix": "/myapp/v\d/resource"
}
Any suggestions??
Regards,

Even tried with the regex as below
"match": {
"regex": "/myapp/v\d/resource"
}

@Kishn0211 you definitely want regex rather than prefix match here. Please follow the guidance at https://www.envoyproxy.io/docs/envoy/latest/api-v2/api/v2/route/route.proto#envoy-api-field-route-routematch-safe-regex for recommended regex use. I would also suggest turning up logging to trace level to understand what is going on.

@htuch
I tried something like below and was working
"match": {
"regex": "/myapp/v[6-7]/resource"
}
Thanks for the support

Mypath has matrix param. So it was not detected.
Url: host/myapp/v[6-7]/resource;myname="ktr";lastname="min"
Is there a way to write regex for this

Documentation says:

The rule will not match if only a subsequence of the :path header matches the regex

Your regex must match the entire URL.
Suggestion:
"match": {
  "regex": "/myapp/v[6-7]/resource.*?"
}


Specific information to recreate leaderboard results?

Hello, I have been struggling to recreate the results shown on the detection leaderboard. Can someone provide me with the specific information to recreate?

Hi! The input resolution is 640 x 1600, the backbone we use is ConvNext, the bev resolution is 256 x 256, we also use train and val split for training.

Hi, thanks for the info! I'll make sure to give it a try!

Hello, can you tell me which ConvNeXt architecture you guys use? Tiny, small base or large? Thanks!

Hi, @yinchimaoliang, I wonder if the data augmentation in your submission is the same one used in the r50 config? So the only differences are (I) image resolution (2) bev resolution (3) image backbone (4) trainval for test, right?

Hi! When the resolution of the input image is changed, the resize_lim and final_dim of ida should be changed accordingly.

Hi, @Bluffish. BEVDepth-pure extends the BEVDepth by using ConvNeXt-base as backbone.

Hi, @yinchimaoliang thanks for your information. I am trying to use ConvNeXt but encountered OOM problem. I wonder if you adopt torch.utils.checkpoint to save your memory in the ConvNeXt backbone? If so, do you use it in the stage conv or also in the downsample layers ?Thanks!

Hi, @zehuichen123 , we didn't adopt torch.utils.checkpoint, we used V100 with a memory of 32GB, you can try it if you are using machines with smaller memory size.

Thanks! I am using V100 machine too. BTW, how did you set your batch size for each GPU as well as the learning rate? (since a batch size of 8 is impossible in this case)

Thanks! I am using V100 machine too. BTW, how did you set your batch size for each GPU as well as the learning rate? (since a batch size of 8 is impossible in this case)

The batch size for each card is set to 2, and we use 4 machines. Learning rate is 2e-4.

Got it! Thanks! 😊

Hi @yinchimaoliang , I am trying with ConvNeXt-base on the A100(40G) but I found that even with batch size 1 I still encounter OOM error. The code can be successfully run with 256x708 scale, but failed with 640x1600. :(

@yinchimaoliang Do you have any plan to release the config file and model shown on the detection leaderboard?

Hi, @zehuichen123, we also encountered OOM problem. Did you solve this problem？

Thanks! I am using V100 machine too. BTW, how did you set your batch size for each GPU as well as the learning rate? (since a batch size of 8 is impossible in this case)

The batch size for each card is set to 2, and we use 4 machines. Learning rate is 2e-4.

@yinchimaoliang Did you use base lr = 2e-4 or final lr = 2e-4? I see in the code the basic learning rate for 64 samples per batch is set to 2e-4. I wonder if it is the same for 2 (batch size) * 4 (machine) * 8(gpu per machine)?

Also, I see from here that you uses dcn head in the submitted version to the test set leaderboard, which is not mentioned in this issue. I wonder if there is anything else we should pay attention to to reproduce the leaderboard results (other than (I) image resolution (2) bev resolution (3) image backbone (4) use both train/val set for training)?

Also, I have noticed ConvNeXT uses different drop_path_rate for different architectures. What value did you use for drop_path_rate?

Why does the author not answer these questions （by zeyuwang615）？？ They are quite important T T


[BNP] Model & metric for test

We need a test so as to make sure that samples are distributed according the right distribution for the BNP project.

One way could be a correctness test such as Geweke.
A maybe better way, is a toy model (small mixture model ?) for which we can compute the posterior (analytically or with SMC with lots of particle)

I agree with the idea of computing analytically the posterior probabilities for a small case by combinatorial enumeration. Then, we need to compare these probabilities to the ones obtained with the SMC output using a large number of particles to asses correctness of the algorithm. See the analytic expressions for a PY mixture model in my thesis, Section 2.7.4, page 46.

@mlomeli1 that's seems like a good idea. If I understand correctly, the idea is to enumerate all clustering possible (for a fix -- and small -- number of observations), then for each clustering compute the posterior probability of observations conditional on that clustering, finally sum the posterior joint over all combination. Then we compare the obtain quantity which is the marginal likelihood, with the unbiased estimator returned by SMC ?
Yet, I don't understand how this method can be used to assess the validity of an other sampler than SMC.

There is a sample Geweke test code here https://github.com/yebai/Turing.jl/blob/master/benchmarks/gdemo-geweke.run.jl. You can probably reuse it by changing the sampling algorithm in Line 34 and 36.

@mlomeli1 otherwise we could use Bayes rule to compute p(Pi_n = pi | {Y_j}_{j=1,..,n}) with the analytic expression of your thesis that you referred, and then compute empirical clustering from posterior sample for a model with 4 or 5 observations (ordered Bell numbers of 13 or 75).

Yes, @emilemathieu, that's a good plan. I think I have some code in R for the analytic expression, can translate it to Julia.
@xukai92 , Geweke's getting it right test only applies for MCMC schemes.

@mlomeli1 I've just finished the test ! :) I've done what I proposed in my previous comment and then used a Chi-square test. It can be found here: https://github.com/yebai/Turing.jl/blob/project-bnp/test/rpm.jl/pym_posterior.jl

Well done, @emilemathieu !

@yebai I've previously used x = Array{Real}(N) for the infinite mixture model I had weird errors because some variables were not deep-copied (when particles are forked) I think.
It worked by using tzeros. Is that why you introduced it ?
It was a silent bug so it's pretty bad...

@yebai I've previously used x = Array{Real}(N) for the infinite mixture model I had weird errors because some variables were not deep-copied (when particles are forked) I think.
It worked by using tzeros. Is that why you introduced it ?
It was a silent bug so it's pretty bad...

yes, I created TArray for variables that have to be deep copied when copying particles. Sorry I didn't tell you about this earlier... I'm glad that you figured this out - perhaps we should document this on the wiki.


Cannot pass func call result to macro.

Codecov Report

Merging #26 (ffd9930) into main (200e1cb) will increase coverage by 0.23%.
The diff coverage is 42.85%.

@@            Coverage Diff             @@
##             main      #26      +/-   ##
==========================================
+ Coverage   54.28%   54.51%   +0.23%
==========================================
  Files           5        5
  Lines         665      664       -1
  Branches      337      337
==========================================
+ Hits          361      362       +1
+ Misses         26       25       -1
+ Partials      278      277       -1

Impacted Files
Coverage Δ

include/lisp/parser.h
43.12% <0.00%> (+0.58%)
:arrow_up:

src/evaluator.cpp
50.90% <35.71%> (-5.19%)
:arrow_down:

include/lisp/evaluator.h
57.97% <66.66%> (+0.32%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 200e1cb...ffd9930. Read the comment docs.


GUI control

I really like the ability to edit config on Control D's website and have ctrld pull the config with a restart.
Would be cool if the ctrld daemon polled the Control D's endpoint so that other things could be possible.  In addition to having ability to update config file by polling every 5-10 mins, maybe poll it for an option to upgrade, restart, etc.  Would require collaboration with Control D to develop the GUI.

You can change the config polling interval using a config param: https://github.com/Control-D-Inc/ctrld/blob/main/docs/config.md#refetch_time
The config is also auto-fetched if you visit the status page, this triggers an immediate config re-fetch.
The GUI app bundles the CLI under the hood, so there is nothing that needs to be added to it.

Sorry, I didn't even know that was in current builds.
So if I update the config file, I can just wait the default 3600 seconds and it will automatically pull the new config and implement it?  (I'm assuming this is 3600/on by default?)

Sorry, I didn't even know that was in current builds.
So if I update the config file, I can just wait the default 3600 seconds and it will automatically pull the new config and implement it?  (I'm assuming this is 3600/on by default?)

Yes, but only if you started ctrld with "--cd" flag.


Upgrade SignalR dependencies

Microsoft.Azure.WebJobs.Extensions.SignalRService has direct dependencies on System.IdentityModel.Tokens.Jwt.
Previously, the package is referenced indirectly from the direct reference Microsoft.Azure.SignalR.* packages. However, the latest versions of Microsoft.Azure.SignalR.* don't depend on the package anymore, and   Microsoft.Azure.WebJobs.Extensions.SignalRService needs to explicitly refer to the package if we want to upgrade the versions of "Microsoft.Azure.SignalR.*" packages.

API change check
APIView has identified API level changes in this PR and created following API reviews.
Microsoft.Azure.WebJobs.Extensions.SignalRService


How to update _id once the entry is updated to the server?

I have a localDb and entries will be added from the browser. At some point all these entries will be synced to the server and stored on a mongodb database. My issue is that when I create a local entry NEdb generated an id. However once it is synced to the server I need to use the Id given by mongodb. My Options are either re-write the _id locally(which is not possible I guess), create a new entry with the server provided Id or use another which I can rename as _id for synced entries(something on these lines).. Please let me know how can this situation be handled ?

the best solution is to ignore the "_id" field completely and create your own "id" field, that you control.
when syncing between nedb and mongodb, do not pass the _id field between them.

+1 to @kaizhu256's proposal, as manipulating MongoDB's and NeDB's _id fields is bound to be messy.


Add support for CDATA blocks

Hope this is good enough.
I have a xml (a rss feed) that use CDATA blocks to pass information. This seems to fix the parsing.
Let me know if there are any concerns.

Thanks!  Have made a new release with that as v0.0.3


feat(config)!: switch to user config override

Linked Issues

Closes #788

Notes to Reviewer

As a reviewer, please pay particular attention to the following areas when reviewing this PR and tick the above boxes after you have completed the steps.
Config Changes

[ ] Ensure proper order in which CLI and config arguments are applied.
[ ] Ensure that config changes work with individual build features by running cargo ci-check-features.

Test cases

[ ] Review and run tests that were added/changed.
[ ] Suggest places that may benefit from test cases.

Hate-Closed :trollface:


./.osx issue

Is there any way I can go back to my previous setting?

Not really, other than restoring from a backup or changing them back individually one by one.

and how do  i know the default setting for OSX ...

@mayank-ism see #58 :)

Oh it looks like we need to delete everything and reinstall osx to get rid of this?


Form elements validation

Moved the serialization of the elements from to the form to the elements
Moved the validation that used to be in the serialization functions in the validation functions in the elements
Created a base static class from which the other classes to use shared methods
Created a Validation class which wraps the native elements in order to be able to call element.validation, element.validation1 etc..
Created a Validation class for custom elements which extends HTMLElement
Added tests
Added a willSerialize check which is used to check if an element should be validated

Update for the added dot at the end of the error message - https://github.com/CoherentLabs/GameUIComponents/blob/form-elements-validation/tools/tests/form-control/validation.test.js#L13
Update for the name of the error to notAFormElement - https://github.com/CoherentLabs/GameUIComponents/blob/form-elements-validation/tools/tests/form-control/validation.test.js#L7


Update logic-apps-using-sap-connector.md

@ecfan

@divyaswarnkar : Thanks for your contribution! The author(s) have been notified to review your proposed change.

#sign-off

@divyaswarnkar: I'm sorry - only the author of this article, @ecfan, can sign off on your changes. But we do have an exception process - if you are on the Microsoft content or product team for this product area, you can ask the PR review team to review and merge it by sending mail to the techdocprs alias.

#sign-off


process is not defined

I am trying to use this component via global script, and have added it to my html as a simple script tag
<html>
    <body>
       <script src="node_modules/react/dist/react.js"></script>
       <script src="node_modules/react-dom/dist/react-dom.js"></script>
       <script src="node_modules/react-dropzone-component/dist/dropzone.js"></script>
</body></html>

However, the dropzone component is throwing an uncaught exception under ReactPerf saying 'process is not defined':
measureMethods: function (object, objectName, methodNames) {
    if (process.env.NODE_ENV !== 'production') {

Am I not able to use this component without adding a module loader and using require/define?  The description implies that I should be able to add this using a script tag.

Thanks, that's good feedback. I'll take a quick look this morning, this is probably fairly easy to fix.

Hey, this is now fixed in 0.9.0!


Completion Items with Command Should Always Be Marked As MakesTextEdit

From #25959
Repo

Create a completion item provider that returns a completion item with the insert text abc and a command that edits the document at the completion position
Trigger the completion item by typing abc in a document

Bug
Even though the insert text will not change the document, the command will. Because VSCode thinks the item will not change the current document, we do not capture enter which results in an extra new line being inserted into the document
Fix
Treat any completion item with a command as MakesTextEdit

@mjbvz Isn't this just a dupe of https://github.com/Microsoft/vscode/issues/25959?

Thanks for the fix. Yes, same root cause for both issues but I created this one to track the root problem on its own
Fixed by 	f8ede24

Great. Stupid me, it took two commits to fix this properly: https://github.com/Microsoft/vscode/commit/d1768a78bce98a5ce06373a67631fab49d326955 and https://github.com/Microsoft/vscode/commit/f8ede24fd90af759ee23de89f2baf8b4c5032011. Let's not forget to cherry-pick both if we decide to rebuild

I re-open this to be more prominent on our radar

pushed changed to release/1.12

Verified with a sample extension:
export function activate(context: vscode.ExtensionContext) {
    context.subscriptions.push(vscode.commands.registerCommand('extension.sayHello', () => {
        vscode.window.showInformationMessage('Hello World!');
    }));

    context.subscriptions.push(vscode.languages.registerCompletionItemProvider('java', {
        provideCompletionItems: (doc, pos) => {
            let item = new vscode.CompletionItem('abc');
            item.command = { title: 'hello', command: 'extension.sayHello' };
            return [ item ];
        }
    }));
}


Facing a 406 error on localhost API

I installed grodid & grobid quantities locally. It seems to be a great tool. Thanks for your work !
It works fine through the web interface and through curl command like this
curl -X POST -F "text=I've lost two minutes." localhost:8060/service/processQuantityText
But I did not succeed to make a simple example through a python API client.
Here is my code :
import requests
URL = 'http://localhost:8060/service/processQuantityText'
data = {"text" : "I have two minutes"}

headers={'Content-type':'application/ld+json',
         'Accept':'application/ld+json'}

try:
    r = requests.post(url = URL, data = data, headers=headers)
    # If the response was successful, no Exception will be raised
    r.raise_for_status()

    # extracting response text
    pastebin_url = r.text
    print("The pastebin URL is:%s"%pastebin_url)

    print(r.status_code)
    print(r.headers)
    print(r.encoding)
    print(r.json)

except Exception as err:
    print(f'Other error occurred: {err}')  # Python 3.6

I tried different values for Content-type and Accept in the header but nothing works.
I faced "415 Client Error: Unsupported Media" error (without header for instance) or '406 Client Error: Not Acceptable' error.
Can you help me ?

Hello @letailli !
Ah yes this text service should be updated, it's a “multipart/form-data” which is crap (thus the -F in the curl query).
Basically with python, the following should work:
r = requests.post(URL, files=dict(text='I have two minutes'))

@letailli you might want to try the python client https://github.com/lfoppiano/grobid-quantities-python-client (it's available also via pip)

We should put a link to this client in grobid-quantities readme !

@kermitt2 right! I will do it!

Both work perfectly !
Thanks a lot.

I will close the issue when the references to the python client are in the documentation 😅

I added the reference to the python client with commit d6347f8f584c8c0944af795929e04ca95c8410a1


closes #136 logo and user guides

Closes #136 .
Thank you for your Pull Request! We have developed this task checklist from the Development Process Guide to help with the final steps of the process. Completing the below tasks helps to ensure our reviewers can maximize their time on your code as well as making sure the admiral codebase remains robust and consistent.
Please check off each taskbox as an acknowledgment that you completed the task or check off that it is not relevant to your Pull Request. This checklist is part of the Github Action workflows and the Pull Request will not be merged into the devel branch until you have checked off each task.

[x] Code is formatted according to the tidyverse style guide
[x] Updated relevant unit tests or have written new unit tests - See Unit Test Guide
[x] If you removed/replaced any function and/or function parameters, did you fully follow the deprecation guidance?
[x] Update to all relevant roxygen headers and examples
[x] Run devtools::document() so all .Rd files in the man folder and the NAMESPACE file in the project root are updated appropriately
[x] Address any updates needed for vignettes and/or templates
[x] Update NEWS.md
[x] Build admiral site pkgdown::build_site() and check that all affected examples are displayed correctly and that all new functions occur on the "Reference" page.
[x] Address or fix all lintr warnings and errors - lintr::lint_package()
[x] Run R CMD check locally and address all errors and warnings - devtools::check()
[x] Link the issue so that it closes after successful merging.
[x] Address all merge conflicts and resolve appropriately
[x] Pat yourself on the back for a job well done!  Much love to your accomplishment!

@bms63 did i get the favicon bits right as you were explaining at the call earlier? i ran build_favicons() from pkgdown. i didn't find anywhere i needed to add anything more in _pkgdown.yml for the logo bit though. if i missed anything please comment and maybe @bundfussr could fix whilst i'm out of office this week.

This renders for me as the onco hex image.  I found that after the favicons update, I had to clear browser and restart the session to get admiraldev hex to appear.   It all looks good to me.

Did you guys mean to have both of those hex stickers on the same page?

@bms63 i don't think that was intentional no - if it's an easy fix let us know!

These are comparing the READMEs for admiral and admiralonco.  You just need to move the img stuff up to the next line.


Ability to provide alternate code formatters

I'm one of the contributors to CSharpier, an opinionated code formatter for C# (inspired by prettier for Js and black for python). I would like to be able configure omnisharp to use that to format code instead of the default roslyn code format.
Firstly, I would like to know if you folks are open to that idea and if such a pull request would be accepted before I start doing any serious work.
Assuming it's a yes, I would like to discuss what the best way of going about that is. It would be ideal if omnisharp could load the format provider from a nuget package referenced by the target project like it does rosyln code analyzers, but I don't understand the internal structure of omnisharp well enough to comment on its feasibility yet.

Thanks a lot for opening the issue and your interest in the project.
It is by design that we keep Omnisharp closely aligned with Roslyn and therefore would like to use its formatting capabilities only. We do not plan to take on any other custom external dependencies in that area.
However, most of the editors, including including VS Code allow contributing code formatters separately which I think could be something of interest to you.

Thanks for your response. What I wanted this functionality most in was omnisharp-vim actually.
But I understand your reasoning for staying aligned with Roslyn. I will try my luck in getting a new type of action provider into Rosyln itself, like the code analyzers. :crossed_fingers:


Tsbbloc 301 autogenerate documentation from solidity natspec

Description

Checklist:

[ ] Pull Request references Jira issue
[ ] Pull Request applies to a single purpose
[ ] I've added comments to my code where needed
[ ] I've updated any relevant docs
[ ] I've added tests to show that my changes achieve the desired results
[ ] I've reviewed my code
[ ] I've followed established naming conventions and formatting
[ ] I've generated a coverage report and included a screenshot
[ ] All tests are passing locally

readthedocs handles multibranch/tags compilations, i added this branch on it for example:
https://the-sandbox-game.readthedocs.io/en/tsbbloc-301-autogenerate-documentation-from-solidity-natspec/
is it doable with githubactions?

And the link to API docs is broken (obviously) on readthedocs