Why inputs for TX are treated as Set?

Summary
If inputs are set it is impossible to rely on order of inputs.
this forces us to do redundant filter-like searches instead of just pattern matching on list of inputs.
In  the case of outputs it's convemient that outputs are list and we can do
fooOut : barOut : bazOut : _ = txInfoOutputs info

Instead of more resource heavy filtering:
fooOut = head $ filter  fooPred (txInfoOutputs info)
barOut = head $ filter  barPred (txInfoOutputs info)
bazOut = head $ filter  bazPred (txInfoOutputs info)

but because inputs form not a list but set it can not be done for inputs.
This leads to redundant filtering.
Steps to reproduce the behavior

Actual Result
I'd like to be able to pattern match on list of inputs inside validator and rely on order.
It's true that spend inputs are unpredictable but we can move all arbitrary user inputs to the end of the list
and have easy to use access to inputs that spend scripts and which are useful for checks inside validator.
Expected Result

Describe the approach you would take to fix this
No response
System info

I don't know what you're referring to. Inputs are a list in TxInfo. If you're referring to the fact that inputs in transactions themselves are a set, then you need to open an issue with the ledger, which I think is highly unlikely to change.

Yeah, they are list in the TxInfo but they go over internal set...

I think what you are saying is that you cannot predict the ordering of the list, because it's based on the set in the real transaction.

That's right, this leads to some redundant checks or filter searches. That we could avoid if we were sure that order will be the same in validator as in the TX.

I think what you are saying is that you cannot predict the ordering of the list, because it's based on the set in the real transaction.

So the inputs aren't sorted or anything like that? The order just matches the order in the transaction, which is determined by the submitter I assume?

No, it does not match the order in the original serialization, but rather the internal sort order used by the ledger.

Do you have any tips for where to look for this sort order? is it defined in https://github.com/input-output-hk/cardano-ledger or do I have to look through the node implementation to figure it out?

@L-as the ordering is the lexicographic ordering on pairs (TransactionID, TransactionIndex). @michaelpj do you know if there is a place in our documentation where it would be helpful to include this detail?

No, it does not match the order in the original serialization, but rather the internal sort order used by the ledger.

Sounds like this is the expected behavior of Plutus then.
@michaelpj however given the confusion, maybe it's worth clarifying the alignment in Plutus docs too? Or was it already done at some point?

I think this should be in the part of the ledger spec that explains how the script context is converted into Data, which doesn't currently exist.

I've opened a ticket on the ledger side and I'm closing this one.

If we specify (as is currently true) that the transaction inputs in the script context are in lex order, then we are essentially promising that plutus scripts can depend on this. ie that we will never change it. with the advantage of make scripts more efficient. And a bug in the ledger code (breaking this promises, accidentally) would lead to problems in Plutus scripts.
If we make it clear that we make no promises, then we give ourselves flexibility. at the cost of this filtering.
In general, it's unclear to me where to draw the line on what ledger internals we should let Plutus scripts depend on.

Well, we should specify what happens, because it's part of the observable behaviour of the chain. You could not write a competing node without reproducing this behaviour, therefore it should be specified. I don't think we have the liberty to declare that it's undefined behaviour.

In this instance, I think it is fair, but in general, are we promising that everything that the ledger does for the script context is a promise forever? Ie the code is the true spec? I guess if, say, another implementation sent the inputs in a different order, we can never control what assumptions scripts might make, and such a divergence would lead to problems (even if the spec itself made it clear that it was undefined...).

FWIW scripts rely on this behaviour to optimise operations on maps.

In this instance, I think it is fair, but in general, are we promising that everything that the ledger does for the script context is a promise forever? Ie the code is the true spec?

Yes, we are. The script context object is observable by scripts, and so any changes to it can affect at least e.g. the budget and therefore whether or not scripts validate. This is why it's such a problem that we haven't specified it properly.

Anytime we've discussed a change and noticed that it would be observable by scripts, we've gone out of our way to do something different which was not observable, even when is seemed extremely unlikely to ever manifest, so I'm absolutely sympathetic to the concern.
But promising that we will never make a change observable by scripts is very daunting. It is daunting for two reasons:

The ledger is comprised of many lines of code, what human could say with the utmost confidence that they could list out every observable fact about the script context? How would anyone ever know if they've exhausted the list? I do think I've got most (nearly all?) of it in my head, Peter and I came up with a pretty good list, but there will always be a lingering doubt that we've missed something.
We are in the Rumsfeld "unknown unknowns" territory, making promises about a future that is hard to predict while ruling out future possibilities. (I do realize that we can have new assumptions in new versions of Plutus, but it can still be extremely difficult to maintain invariants for the existing languages that have no expiration).

The alternative that I see is this: write down an explicit list of assumptions that script authors can assume about the script context, write property tests for this list, and declare everything not on the list as undefined behavior that should not be relied upon.

The alternative that I see is this: write down an explicit list of assumptions that script authors can assume about the script context, write property tests for this list, and declare everything not on the list as undefined behavior that should not be relied upon.

I think the alternative is to fully specify the creation of the script context. Note that doesn't mean specifying everything about the ledger, just how the script context is created. Since it mostly re-presents information from the transaction, I don't expect that to be so difficult, apart from the odd thing like ordering.
I also just cannot see how we can ever say anything is "undefined behaviour". If behaviour is observable by scripts, and there are scripts in the history of the chain that depend on that to validate, then if we change that then we break history. It's just like everything else.

I think the alternative is to fully specify the creation of the script context. Note that doesn't mean specifying everything about the ledger, just how the script context is created. Since it mostly re-presents information from the transaction, I don't expect that to be so difficult, apart from the odd thing like ordering.

Maybe it's not so bad, and my time is better spent attempting the task than chatting :laughing:

I also just cannot see how we can ever say anything is "undefined behaviour". If behaviour is observable by scripts, and there are scripts in the history of the chain that depend on that to validate, then if we change that then we break history. It's just like everything else.

My concern is not about breaking history, indeed we cannot do that. I am worried about what we can and cannot change with a hard fork.

Maybe it's not so bad, and my time is better spent attempting the task than chatting

FWIW Matthias is already annoyed that it's not specified, and I basically promised that I'd specify it for the new SOP version: https://github.com/cardano-foundation/CIPs/pull/455#issuecomment-1411727761
Let's talk about how to do the work best, but I think the Plutus team can chip in here.

I am worried about what we can and cannot change with a hard fork.

Assuming we keep the old behaviour for old scripts then we can change whatever we like! This is covered by the ledger-script interface part of CIP-35, and I think it's fine to do in a new Plutus ledger language. We could do it on a protocol version change, but given that we need to retain the old behaviour anyway, I think we should generally do the usual thing and do it with a new Plutus ledger language.

Assuming we keep the old behaviour for old scripts then we can change whatever we like!

you make it sound so easy :lolsob:

Assuming we keep the old behaviour for old scripts then we can change whatever we like!

Imagine we have features a, b, and c in ledger that make their way into the script context. there are all kinds of properties P(a, b, c) that might hold for Plutus VX scripts. Imagine now adding feature d to the ledger. even if d is not used anywhere in some Plutus VX script, P(a, b, c) could fail in the presence of d in the ledger if we do not go out of our way to preserve it.
If we promise to maintain all the current observable facts, then we are promising: "for any idea d that we every come up with in the future, and any property P(a, b, c) of the current ledger, P(a, b, c) will still hold with the introduction of d. (I should probably use some notation like a(d) or something that makes a now depend on d or something...).
Of course I'm happy to make promises about all the Ps that I can think of (assuming they are not too ridiculous), but promises about the ones I've yet to think of are worrisome.
On the one hand, I can hear:

yeesh, Jared, you are making this way more complicated than it needs to be, it's just a simple translation.

But on the other hand, it sounds like exactly the kind of thing that usually bites us.
And I think what you are proposing is something like a "reference implementation" of the script context creation, so that all the Ps become implicit (reducing the infinite to the finite). But even then I worry that this could be too much of a simplification of the real story, and that tendrils of the provenance will find their way all the way back to the wire...

I think we should get together and figure out what we can and can't promise for the script context. Here are some examples of Ps where weird things happen:

We want to remove pointer addresses by turning them into enterprise addresses, but we still have to support them because some script might rely on them. But now we've changed semantics: Let's say a script allows minting some tokens once per epoch proportional to the amount of Ada delegated to some pool. If a pointer is allowed here, this change has now changed the semantics: you'll get the tokens, but the pool owner doesn't get your stake.

Let's say we didn't need to keep track of registered reward accounts anymore, and we decide to get rid of the certificates. Well, now some script contexts can't be constructed anymore, so maybe we can't do that and still have to be able to make them, but just as dummies. But now there are some Plutus scripts that might have more steps that they could take, since they can track parts the state of this registry internally. So, if we want to preserve the semantics, we have to keep the current system in place forever.

I think the introduction of new features might be less problematic (we can always disallow old Plutus versions in scripts with new features), but I think that if we wanted to make the strongest possible promise here large parts of the ledger would be completely unchangeable forever.

Ah, I think I see. Let me make it concrete: let's talk about the sorting order of the inputs. Suppose that they are initially in using ordering O1, and we want to make a change to O2.
I think:

We cannot say "haha, the ordering is unspecified, we're just going to change it silently and break literally everything"
We can say "we're going to change the ordering at a particular point using our usual mechanism for backwards compatibility"
We don't have to say "we'll maintain that particular ordering forever"

Let's say a script allows minting some tokens once per epoch proportional to the amount of Ada delegated to some pool

Scripts can't tell how much Ada is delegated to a pool, but I take your point. I didn't quite follow the other example but I agree it would be good to get together and think it through.

Let me make it concrete: let's talk about the sorting order of the inputs. Suppose that they are initially in using ordering O1, and we want to make a change to O2.

I definitely appreciate you making a concrete example. but just to be clear to everyone following along, I do think that the ordering of the transaction inputs in the script context is a property that we could make about the V1 and V2 context for eternity. Unfortunately I don't have a better example in mind atm (hence the unknown unknowns).

Was it an intentional decision somewhere to sort transactions in this way?  I.e. is there a good reason why this was done, or is it just an accident of the datastructures used?
Last I dug into it, there was an explicit step when building the script context to sort these inputs (though I'm no good at navigating haskell codebases, so I can't find it again), so it didn't feel like it was an artifact of "this is how the ledger stores things internally".
For reference, this sort order has caused some consternation for at least two of the major DEX's on Cardano (SundaeSwap / MinSwap), and is limiting throughput.  If you try to batch transactions, and do so in a way that preserves the on-chain order, this (quite surprising) sorting destroys that.
The only solution is to add an extra field in the redeemer and sort them in the script (costly), or to segment the batching into monotonic subsequences, which hurts protocol throughput.
I understand that the ship has sailed for PlutusV1/V2, likely (as, beyond even just execution units, the actual outcome of the transaction depends on this sort order now); but it'd be good to document somewhere not just that this is the behavior, but why: i.e. an accident of history, or xyz good reason.
That way when designing future Plutus versions, we can either revisit that decision, or continue to intentionally benefit from xyz good reason.

This is mostly an accident caused by a design decision made for Shelley (order didn't matter, so we put things in a set instead of a list). However, I still maintain that it's actually better for the ecosystem this way. There has been a bunch of discussion here: https://github.com/cardano-foundation/CIPs/pull/231

Last I dug into it, there was an explicit step when building the script context to sort these inputs (though I'm no good at navigating haskell codebases, so I can't find it again), so it didn't feel like it was an artifact of "this is how the ledger stores things internally".

It actually is how the ledger stores things internally.
There are a few things going on here, I will try to explain.
On the wire, using CBOR, the transaction inputs are just in a list:

https://github.com/input-output-hk/cardano-ledger/blob/ac405a977557a7c58ce1cf69d3c2a0bf148cf19f/eras/babbage/test-suite/cddl-files/babbage.cddl#L54
https://github.com/input-output-hk/cardano-ledger/blob/ac405a977557a7c58ce1cf69d3c2a0bf148cf19f/eras/babbage/test-suite/cddl-files/babbage_extra.cddl#L4

(Arguably, we should have used the CBOR set tag, #258, but we didn't.)
This list does not have to be sorted, but some services (like the CLI) might sort them anyway.
Now, inside the transaction body, we store the transaction inputs as a Set. We think of them semantically as sets, and we also use Haskell's Data.Set. I (we?) don't think of sets as inherently ordered, so in order to provide an index for the transaction input, we needed something, and lex ordering seemed a fine choice (though with hindsight 20/20 we should have at least more carefully considered the serialization ordering. or more like hindsight 2018 :laughing: ). The code for placing new outputs in the UTxO has been basically unchanged since Shelley. Moreover, this is the same order that they are now passed to the script context. And, very importantly, this is the order that the redeemers need to use.

If you try to batch transactions, and do so in a way that preserves the on-chain order, this (quite surprising) sorting destroys that.

Do you have a minimal example of this? I feel like I might have read a post about this a while back... :thinking:

relevant: #4753

@JaredCorduan
It actually is how the ledger stores things internally.

Right; but there was also an explicit sort call when building the ScriptContext, IIRC.

Do you have a minimal example of this? I feel like I might have read a post about this a while back... thinking

Consider SundaeSwap:

User A pays funds to swap 100 ADA to SUNDAE into our "escrow" contract at TxOut 9bd...#0
User B pays funds to swap 100 ADA to SUNDAE into our "escrow" contract at TxOut 1ef...#0

Our pool contract then enforces that, when transactions are batched together, the appropriate amounts are paid out to the user; since the price shifts in between each execution, the order that these are executed matters: the user who executes second will get slightly less SUNDAE, because the purchase of SUNDAE has pushed the price up.
Additionally, for fairness it's important that these be executed in the order they settled on-chain.
So, if we build a transaction that has the inputs:
 - `9bd...#0`
 - `1ef...#0`

Which then get sorted, and executed out-of-order, essentially letting 1ef...#0 skip the line.  This opens the protocol up to a sort of "proof of work" attack to enable frontrunning.
To solve this, scoopers break the stream of transactions into monotonically increasing subsequences: grab as many transactions as you can in their on-chain order, until one txID is less than the previous one.  This results in less-than-optimal batching (on average splitting the batch size in half);
This was such a subtle and nonobvious behavior that both MinSwap and us didn't notice this until after we had launched, when it was impossible to replace the contracts; WingRiders documented a solution where the order is provided in the redeemer, but this increases execution unit costs as you have to perform an additional sort over the inputs to "undo" the sorting that the node did when building the script context.
In my opinion, though it seems the ship has already sailed on this, we should move this sorting behavior, if desirable, to the person building the transaction; If the protocol doesn't care about order, they can sort them by TxHash for the map-building effiency gains mentioned above;  The argument that "scripts have to check every input anyway" doesn't hold much water for me, as it ignores the fact that the ordering contains useful information that is otherwise impossible to access on-chain; and it doesn't materially stop people from doing stupid things: someone can just as easily find the first input with a token as they could depend on the first input.
Perhaps as execution unit budgets increase and other languages with more efficient compilers come online, this will become a moot point as sorting a few inputs by redeemer will become cheap, but it very much is adding this annoying parabolic arc where many contracts will have to undo work that the node is doing.

Right; but there was also an explicit sort call when building the ScriptContext, IIRC.

Do you consider calling fromList on Set an explicit sort?
https://github.com/input-output-hk/cardano-ledger/blob/ac405a977557a7c58ce1cf69d3c2a0bf148cf19f/eras/babbage/impl/src/Cardano/Ledger/Babbage/TxInfo.hs#L203
Note that by this point in the ledger, the user's wire order is long gone (of course we could have chosen to memoize it, but we don't, we supply the ordering give by Set, which is a part of the internal structure. we don't go out of our way to sort it).

WingRiders documented a solution where the order is provided in the redeemer, but this increases execution unit costs as you have to perform an additional sort over the inputs to "undo" the sorting that the node did when building the script context.

Thank you for the example! Indeed I just assumed that the logic would be present in the redeemer. But I myself have never had to worry about writing optimized Plutus contracts and am probably ignorant of a lot of the struggles.

In my opinion, though it seems the ship has already sailed on this, we should move this sorting behavior, if desirable, to the person building the transaction;

It's not insurmountable, of course, but it would be a much more invasive change than perhaps folks realize. (and indeed a lot interesting discussion is in the issues that @WhatisRT mentioned).

Perhaps as execution unit budgets increase and other languages with more efficient compilers come online, this will become a moot point as sorting a few inputs by redeemer will become cheap, but it very much is adding this annoying parabolic arc where many contracts will have to undo work that the node is doing.

@JaredCorduan no; I guess I'm mistaken. I'm not a Haskell developer so maybe I misread the code, or was looking in some other place. It's probably not relevant now.
Re: invasive change; for sure, that's fair. All I can do is provide my limited perspective :) and from the outside, it seems like a violation of the general philosophy that's been taken elsewhere to bend over backwards to make sure the script context accurately represents the transaction on chain; for example, the incompatibility of PlutusV1 with even the mere presence of a reference input for the purpose of reference scripts, which you and I have talked about before 😅
So, not necessarily pushing strongly for a change, just putting our experience out there so it can feed into the complex decision making process you have to be responsible for ;)

Re: invasive change; for sure, that's fair. All I can do is provide my limited perspective :) and from the outside, it seems like a violation of the general philosophy that's been taken elsewhere to bend over backwards to make sure the script context accurately represents the transaction on chain; for example, the incompatibility of PlutusV1 with even the mere presence of a reference input for the purpose of reference scripts, which you and I have talked about before

I can see how it could feel that way! We never considered the ordering of the transaction inputs on the wire to be semantically meaningful, which is why it was placed into a set right upon deserialization. it was defined as a  set even before the first line of Haskell was ever written. but obviously the ordering is meaningful to folks building on Cardano and I wish we could have taken that into account.
As for the script contexts, we felt the decision was about things that really did matter a lot. In the circumstances where you have a square peg and an intentionally round hole, you have to pick between getting out the saw and rounding off the edges or giving up. With V1 we did the former and later regretted it.

In my two year journey of building on Cardano using plutus and the ledger - writing scripts that are sensitive to sequence of inputs, modifying transactions and repairing redeemer pointers through these modifications - I often felt that the choice for Set was with the right intention, but with too big of consequences.
IMO what actually matters is that inputs are unique in a transaction, but their order should not matter for the EUTxO ledger semantics. And with the example of @Quantumplation the order is even important in some cases.
But, constraints liberate! Today no script or users in general can rely on the order of the inputs being like originally constructed because they might have been lex ordered by the ledger. Doesn't that mean that if we remove the ordering constraint, existing usages will be fine?
EDIT: When typing out the last paragraph I realize - no. Users could rely on the fact that inputs are lex ordered of course. But, it would be less breakage as changing it in the other direction :shrug:

it seems like a violation of the general philosophy that's been taken elsewhere to bend over backwards to make sure the script context accurately represents the transaction on chain

Just to comment on this: the script context is supposed to accurately represent transactions, not the serialized form of transactions. Transaction inputs are a set. They're serialized in an order because they have to be, but that's an artifact of serialization. I think we are being consistent here, just perhaps not in the way you would like :)

Transaction inputs are a set.

That's true. But who said that the order of that set needs to be lex order and not a transaction-creator defined one?
The order of transaction input has meaning if an application developer wants (or needs) to give it meaning.

that's an artifact of serialization

To me, that we re-order inputs from what we are given (from the wire) seems to be the artifact here. Something implicitly happening because we used Set TxIn instead of a nubbed [TxIn].

Transaction inputs are a set.

That's true. But who said that the order of that set needs to be lex order and not a transaction-creator defined one?

Sets don't have an order, that's what distinguishes them from unique lists. So from the perspective of the ledger there's no re-ordering going on: we have a set of TxIn, and we need to give it to Plutus via (effectively) some serialization that's inherently ordered, so we need to pick one.
Note that there's also a difference between an actual Set and a Haskell Set (which is inherently ordered since Haskell doesn't have quotients), and when we're talking about Sets we mean the regular kind that you'd encounter in math. Everything else is just implementation details.

Sets don't have an order, that's what distinguishes them from unique lists. So from the perspective of the ledger there's no re-ordering going on...

I agree on that a set in math does not have an order. But I disagree that the ledger is not affected by the order.

In fact, the ledger spec defines "the set of inputs is an ordered set". Only that way it can indexof the powerset of tx inputs to determine / resolve redeemer pointers. See for example subsection 4.1 in https://github.com/input-output-hk/cardano-ledger/releases/download/cardano-ledger-spec-2023-01-18/alonzo-ledger.pdf

However, this issue is about users of the EUTxO model and its implementation on Cardano voicing a (IMO legitimate) request for preserving order of inputs as it conveys meaning.
The question I care about is.. why are inputs not a unique list in the spec and implementation as @anton-k originally asked?
This seems to have been discussed in https://github.com/cardano-foundation/CIPs/pull/231, but to no conclusive outcome or is it just discarded because nobody wants to change it (it's likely quite involved)?
In general, this makes me a sad user.

In fact, the ledger spec defines "the set of inputs is an ordered set".

I'd call this a misleading explanation. The intent here wasn't to suggest that there is some extra structure on this set, it's just to explain how this function works. Maybe I should go and fix that language.
The reason for this is historic. For accounting you really only need a set of inputs, and that's still true today. From that point of view, an ordering on the inputs really is just metadata about the inputs, and it makes sense that such extra information should be provided by the redeemers. In fact, if you imagine two scripts in a single transaction that would require an order on the inputs, why should those orders be forced to be equal? If you want to support a use-case like that, you'd then have to re-implement a permutation in the redeemer anyways, but now the scripts have to fight on which one gets preferential treatment.
From my perspective, it's really about complexity. There are many ways in which we can make writing scripts easier that add complexity, and almost all ledger features that increase complexity look like mistakes to me today. It's either a few script authors needing ordered inputs that have to pay the price, or ledger which would have to support this forever and now everyone has to pay the unknown price of a locked-in feature. To me this seems short-sighted, but I'm not a script author.

Are transaction inputs provided to the script context as a set type, or as a list type? If as a list, maybe that's where the breakdown (such as it was, I'm not trying to be overly critical or annoying, just trying to increase empathy where I can) was, and why the behavior was so surprising to us and others.

In fact, if you imagine two scripts in a single transaction that would require an order on the inputs, why should those orders be forced to be equal?

Scripts don't get to decide. They only verify whether the transaction is as they expect or not. The transaction creator (should) get to decide on the order and needs to do in a way to satisfy the scripts they want to redeem. If two scripts require ordered inputs, one ascending lex the other descending lex .. then obviously they are not compatible and will never be spent in the same transaction. This is does not work today, nor will it ever work - but it does not need to. What we (or me at least) care about is that the order of inputs is an interface between the transaction author and the script verifying the transaction.
Note, that the ledger is completely out of the picture here. It would just represent the transaction as it was created, validating it by doing it's own checks and also calling the scripts. And as you say @WhatisRT, the ledger rules do not need ordered inputs, they merely need to be unique to - for example, to check they sum up to a balanced transaction. That's great! So the ledger would not care if we just keep the order as it was in the transaction?
@Quantumplation To my understanding:

Transaction inputs are a list on the wire (in CBOR) according to the babbage cddl (where set = [* a]),
a Haskell Set TxIn in the ledger tx body (accessed via this lens), and
a Haskell/PlutusTx [TxInInfo] (resolved inputs) in the script context.

Are transaction inputs provided to the script context as a set type, or as a list type?

We don't have a native set type - in that respect it's similar to the transaction wire format, where the set of inputs is represented as a list (which is the source of all this confusion!).

Note, that the ledger is completely out of the picture here. It would just represent the transaction as it was created, validating it by doing it's own checks and also calling the scripts.

Just to try and convey the conceptual difference that I think I see here.
My interpretation of Sebastian's position:

A transaction is the wire format.
The author of a transaction is the creator of the wire format, and all the details represent their intention.
Therefore, all the details that were specified in the wire format should be preserved, as they represent the intention of the transaction author.

My interpretation of my own/Jared/Andre's position:

A transaction is the structure used by the ledger. The wire format is a serialization format for transactions, and has no special status other than that. Other serialization formats could be used and would be equally valid.
The author of a transaction can/should construct a value that corresponds to a ledger transaction, e.g. with a set of inputs. It is not their concern how it is serialised, that should be done by a library, and the details of serialization do not represent their intention. Authors who are constructing the wire format themselves are weird.
Therefore, details that are specified only in the wire format are incidental and can (should) be discarded, since they're just an artifact of serialization.

I think this is the crux: otherwise we just keep talking past each other saying "why are you discarding features of 'the transaction'" and "those are not features of 'the transaction'".

Great summary @michaelpj
To add my own position, which is somewhere between the two:

I think that it's fair to draw the distinction between wire format and conceptual data structures, as you do
However, I'd emphasize that the on-chain format is the primary interface for external parties like dApp developers
those external users are rarely engaging directly with the exact specification or data structures used in the ledger code
Thus, any differences between the wire format and the conceptual format can be extra surprising
some acknowledgement of / empathy for that fact, and extra care in identifying those differences in the future and emphasizing them in ways that are discoverable to those external consumers, would go a long way
otherwise those external parties can come away feeling a little gaslit 😅
additionally and separately, I think Sebastian and I are both arguing that it would be a useful change to the specification to document that, after some point in time, the order of the inputs become one of the explicit properties of the ledger state; it might not be useful for checking the accounting of the ledger, but I've brought you at least one example where it's useful for checking the overall transaction validity

some acknowledgement of / empathy for that fact, and extra care in identifying those differences in the future and emphasizing them in ways that are discoverable to those external consumers, would go a long way

for sure. above in this thread, you will see that @effectfully created an issue in the ledger repo for exactly this (and more!). the hard part for me is deciding a clean place to draw the line around what promises we can make about the script context. notice that txInfoInputs is not the only thing that has a CBOR ordering which is not preserved: there is also txInfoReferenceInputs and  txInfoSignatories ("well, if the Principle Investigator signed off on this, that's all I care about", etc :grin: ) and all the maps (which are association lists in both CBOR and in plutus, but are partial functions in the spec). Notice also that the validity window is actually two separate fields on the wire, the ttl and the start time, so in theory (where I am putting on my mathematician hat that cares about strict definitions, not likelihood) someone could hope to derive meaning from which field is placed first in the transaction body. etc. Of course we are not trying to be obnoxious here, but clean guiding principles are our survival strategy against death by complexity. If we did want to move the inputs to a unique list, we'd need a clear migration strategy for the redeemer pointers, so that we are not silently changing the semantics of them (certainly doable, this is just a small example of the complexity creep).
and all of these wire concerns pale in comparison to the semantic ones, such as @WhatisRT mentioned above...

@Quantumplation I think you make some good points about documenting things more cleanly. I don't actually know where script authors get their documentation from and I know that the specs can be quite dense. One thing that I'd like to have one day is a 'simplified spec' that only contains things script authors would care about, together with a proof that scripts that don't abuse certain features behave exactly the same on the real ledger as on the simplified ledger. That should make it easier to understand the ledger and to reason about correctness of scripts. Do you think that would help? (Only issue is that this is really a stretch goal and I'm currently pretty busy with all the Voltaire stuff, so it's unlikely to be started any time soon)
@JaredCorduan I really like your point that even the fields in the transaction itself could be reordered to intend some meaning, I didn't even think about that.
There are just so many places where the serialization can be changed without doing anything and we have to decide what to show to Plutus and what not. And once we show something to Plutus we're basically committed for doing that forever and we have to be incredibly careful with changing its semantics (since a change in semantics means possibly breaking assumptions a script made). The less we show to scripts, the freer we are to make changes in the future.

@WhatisRT thank you! And, actually, I chose my wording very carefully: emphasizing them in ways that are discoverable to. Because the big myth that everyone falls for about documentation is thinking you can just write the be-all-end-all documentation and solve people's confusion... But then at the end you just have yet another thing for people to read that's slightly different from the rest. It's like that xkcd comic about standards. That's not to say that both efforts you describe wouldn't be Good Things :tm:.  But I'm suggesting there are ways to be more subtly pervasive than that too: integrate it into our APIs / SDKs, our tooling, etc. If plutus V1 had passed it to us as a set; if our testing framework randomized the order or sorted them like the node does; if we had tools to vizualise the transaction that somehow emphasized this, etc.
Another example of this kind of surprising behavior is the mint-0-ada phenomenon 😅
Sundae was cursed to be early, so it's 1 Billion percent understandable that those things weren't in place back then with everything IOG had going on, and there are definitely ways we could have stumbled into this on our own. I'm not trying to play the blame game, just trying to emphasize how surprise can be bad, and help find ways to stop it from cropping up in future work (especially as we're about to make things way more complicated with input endorsers heh)

I absolutely agree that discoverability is important, but I just make the rules and write them down, I have no idea what happens afterwards 😅 In fact, at some point in time I expected people to ask me whether some documentation they wrote is correct, but that never happened and at some point that expectation went away. So my approach to this is really to try and always make things as simple as possible (for many reasons, but including fewer misunderstandings). But even that isn't the answer and I've been horrified a bunch of times over the years by issues like the 0-ada thing you mentioned.
I think this is particularly hard because the ledger isn't what people directly write their code against. The more layers you pile on top (or next to it), the harder it is to be sure that everything is right. So maybe the goal is to make the source of truth as attractive as possible, so people will want to sit close to it?


answer seclection

Regarding the Graftnet paper, a binary classification has been used to select the answer and according to its source code, it does not seems to be a cnn classification. While PullNet paper has mentioned that it is using the same CNN architecture used by GraftNet for the classification.
Could u please explain the classification algorithms which have been used in each work?

I am not sure what you mean by CNN. GRAFTNet is a variant of GCN https://arxiv.org/pdf/1609.02907.pdf https://arxiv.org/pdf/1609.02907.pdf. PullNet use GRAFTNet as the underlying classifier.
Please let me know if you have more questions.
Thanks,
Haitian

On Mar 3, 2021, at 4:04 AM, dinani65<EMAIL_ADDRESS>wrote:
Regarding the Graftnet paper, a binary classification has been used to select the answer and according to its source code does nor seems to be a cnn classification while PullNet paper has mentioned that it is using the same CNN architecture used by GraftNet for the classification.
Could u please explain the classification algorithms which have been used in each work?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub https://github.com/OceanskySun/GraftNet/issues/20, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADE5XL63JOPA6JFEQQBZHOTTBX3R7ANCNFSM4YQ2KNPQ.


Unclear of how to use parameters

I was trying this $fields = $ac->api("list/field/view", array("ids" => "all"); and it gave me no result at all. Then 40 minutes later in a moment of inspiration I tried this $fields = $ac->api("list/field/view?ids=all"); which worked. Please document the behaviour!
And I would much prefer the first way of doing things because it feels a lot more natural.
(ping @ac-pevans)

Thanks @jonathan-s! I've added some documentation here: (https://github.com/ActiveCampaign/activecampaign-api-php/pull/60). Could you let me know what you think?

👍
On Tue, 4 Oct 2016, 17:47 Femi Agbabiaka<EMAIL_ADDRESS>wrote:

Thanks @jonathan-s https://github.com/jonathan-s! I've added some
documentation here: (#60
https://github.com/ActiveCampaign/activecampaign-api-php/pull/60).
Could you let me know what you think?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/ActiveCampaign/activecampaign-api-php/issues/51#issuecomment-251444650,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACBsEvp9V4i8lqRJngPyr798YZNLLzTDks5qwoMNgaJpZM4JjOx-
.

For anyone else who stumbles on this when trying to get a list of all lists this works for that too,
ex. $this->ac->api('list/list?ids=all');

Addressed in #60


Issues with SAP GUI 7.70 Edge (based on Chromium) Browser Control

Today I've received the SAP GUI 7.70 Beta 2 shipment and I've installed this version:

with this GUI Version it is possible to select  Edge (based on Chromium) as the Browser Control:

Unfortunately when doing so the ZABAPGIT transaction looks like this:

I'm using the latest version from the master branch:

What is the best way to start correcting this issue?

can 7.70 be downloaded from the marketplace? any SAP note references?

I've applied to the beta program to be able to test abapGit. I will ping my SAP Contacts how they can support here.

as a first step, try verifying that SAP demo report SAPHTML_EVENTS_DEMO works

Looks OK and the Events seem to work:

So it might be only a CSS issue.

setKeyBinding is a JS function. So I'd say download the html + css (common + default theme) + js file and try openning it offline.
Additional info:

https://docs.abapgit.org/development/developing-ui-css.html ("Recommended ..." at the bottom)
also comment bundle css at the start of the real html and uncomment refs to original css files ("CSS variables support" in the same guide)

P.S. wow, edge in sap gui, is it 2020 ? Maybe I'll move from GUI 7.40 after all ... (sarcasm detected ... just ignore :)) )

In the Edge developer tools I see the CSS and JS files only as empty files in the sources:

In the console I get this warnings:

HTML000001.htm:7 Resource interpreted as Stylesheet but transferred with MIME type text/plain: "file:///SAPHTMLP://htmlviewer.sap.com/0lkeJ2dG7jgegQQRC5KBiG/css/common.css".
HTML000001.htm:8 Resource interpreted as Stylesheet but transferred with MIME type text/plain: "file:///SAPHTMLP://htmlviewer.sap.com/0lkeJ2dG7jgegQQRC5KBiG/css/ag-icons.css".
HTML000001.htm:12 Resource interpreted as Stylesheet but transferred with MIME type text/plain: "file:///SAPHTMLP://htmlviewer.sap.com/0lkeJ2dG7jgegQQRC5KBiG/css/bundle.css".

Also here I've created an incident in the SAP Marketplace: 366939 / 2020

Joining the 7.70 party so I can help out with testing.

file:///SAPHTMLP://htmlviewer.sap.com/0lkeJ2dG7jgegQQRC5KBiG/css/common.css
Hmm, by the way, does previous SAP GUI add the hash string into the path ? I'm not sure it is taken into account within html declaration. Maybe this is the issue ?

Today I've got a response in the Incident 366939 / 2020:

The issue here was the CSS which was in the relative path to the main HTML page.
The issue has been fixed and it will be  available in one of the subsequent patch of the SAP GUI 770.

I've just tested a patched version of SAPChromiumEdge.dll. You can now use DevTools:

Did you find any bugs in AG related to GUI 7.70 or is it all working now?

I'll go ahead and close this, open a new issue if there are more issues

Just for reference:
SAP Note 2901278 - SAP GUI HTML Control based on Chromium Edge: Legacy HTML does not work (correctly) / present limitations
https://launchpad.support.sap.com/#/notes/0002901278

Just for reference:
SAP Note 2901278 - SAP GUI HTML Control based on Chromium Edge: Legacy HTML does not work (correctly) / present limitations
https://launchpad.support.sap.com/#/notes/0002901278

Today I've received the SAP GUI 7.70 Beta 2 shipment and I've installed this version:

with this GUI Version it is possible to select Edge (based on Chromium) as the Browser Control:

Unfortunately when doing so the ZABAPGIT transaction looks like this:

I'm using the latest version from the master branch:

What is the best way to start correcting this issue?

i'm sorry for the quest, but how you debug in SAP GUI using edge?

@B0yEditi0n check debugging in our docs
PS: Please create a new issue next time


[question] Can this be used on real hardware?

I have an Intel Atom embedded board (D945GSEJT) which I was using as a home NAS for many years, and I thought about reusing it for some retro DOS/9x era gaming.
There's unfortunately no Win9x drivers for the GMA9xx series Intel IGP on the board, so no HW acceleration.
I wonder if this driver can be used with this hardware to do software 3D rendering?
There's VBEMP which adds a generic VESA based driver into Win9x and it also says it's compatible with Mesa3D which your project is also using.
Could this work, or it's just not worth trying it?

Thanks.


Fix issue #11400.  Fixes issue with arrays and dictionary acting as static objects between different instances of objects.

#11400 reports that arrays (and dictionaries) appear to share themselves across multiple instances of the same class.  These changes fixes that problem.

I tested this further this morning, putting the changes through some further scrutiny.  Here is what I did:

Modified my test script so that I created an array passing in a  PoolIntArray into the constructor.
"This worked in prior versions before the fix, and it continued to work after my fix."

Ensured that you still cannot create an Array as a static, as I was concerned that if this was possible then the change I made not work consistently.  Confirmed that you still cannot create an array as static.

Created an array and passed it around as a variable to other functions to ensure that my modification did not cause the code to act in an unexpected way.  This executed successfully with my changes.

So, I have a high level of confidence that this change is appropriate and will not cause any need to break compatibility as may have been previously suggested.

CC @reduz @bojidar-bg

Wouldn't this fail for scripts such as:
const A = [1,2,3]
var x = A

With the current code, x would be [1, 2, 3], while with the PR applied x would be [] (I assume).

I just tested your example.  Var x does properly populate with const A contents.  So, it isn't just an empty array.  That's good.
var x does continue to operate like a constant though, which is what the Array was doing before the fix, so it is sharing the same array across multiple instances of classes because of the 'const' keyword.
This might be an acceptable thing, as it might be desired to do just that.  At least the 'const' keyword is more explicit than the 'var x=Array()' and 'var x=[]' situation.

@remorse107 Ok, last question, does var x = Array([1,2,3]) differ between the two versions?

Just tested your example against the fix, and the version without the fix.  They both acted as they should.  The version before the fix didn't have a problem with an Array constructed with arguments.  It was only when it was an empty constructor.
So far, everything looks good.

var x does continue to operate like a constant though, which is what the Array was doing before the fix, so it is sharing the same array across multiple instances of classes because of the 'const' keyword.

That's a breaking change though. AFAIK currently x would be mutable in the above code, and it's actually a quite typical construct to initialize a mutable variable with the value of a constant. I wouldn't expect var x = A to make x a const.
Or do I misunderstand?

Just tested again.  I wanted to see if x was indeed mutable, and it is.  Test still looks good.
I may have just hit a very sweet spot in the code.  I understand that this is a breaking change, so if someone else wants to run some tests that might be good.

I tested the master branch version without the fix applied, and it appears that const variables are mutable. I know they shouldn't be. So this may have broke somewhere in the past.

Yeah that's #6221, I had forgotten about that one.

I took the change out of reduce_expression and focused on parse_expression instead.  Test still looks good, and I even tried cont a=Array().
I've tried:
var a = Array()              <- Test's great
var a = Array([1,2,3])    <-Test's great
var t = [1,2,3]
var a = Array(t)             <- Test's great
var a=Array(Array([1,2,3]))   <- Test's great
const a=Array()            <- Acts like a static.  So I haven't removed it's ability to detect a const.  So it is not negatively impacting the singleton set in place.
That made the Array act like it did before the proposed fix.

Does anyone have a test Godot project that would satisfy any concerns
regarding my changes?
I would be happy to run the project to save anyone time.
On Nov 13, 2017 3:25 PM, "Juan Linietsky"<EMAIL_ADDRESS>wrote:

@reduz commented on this pull request.
In modules/gdscript/gd_parser.cpp
https://github.com/godotengine/godot/pull/12845#discussion_r150669243:

@@ -1515,9 +1515,17 @@ GDParser::Node *GDParser::_reduce_expression(Node *p_node, bool p_to_const) {
return p_node;
}

			ConstantNode *cn = alloc_node<ConstantNode>();

			cn->value = v;

			return cn;

			if (v.get_type() == Variant::ARRAY) {

I'm not completely sure about this. If parsing a constant, you will
probably get an actual array (the array node will not work).
Also there may be a chance of the array having something in such case? In
any case, I would put this code in the variable parser, not sure if here.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/godotengine/godot/pull/12845#pullrequestreview-76254317,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AHHgUbjE1GV2FWPGi6T0P_pZfy1PIVASks5s2LOwgaJpZM4Qao2l
.

Sorry for the trouble, I did a small refactoring of the gdscript module in #12969 so you'll need to rebase. The filename changed and all GD* classes are now prefixed with GDScript.
The simplest way to rebase might be to reset your branch on the current master, then redo your changes manually, taking care to use GDScriptTokenizer instead of GDTokenizer. Then you can use git push --force to update the PR.

Once rebased I think we should merge it asap so that we can get it tested for potential regression before we start making 3.0 beta binaries next week.

I am at such odds.
Given the example below:
const A = [1,2,3]
var B = Array(A)
Part of me feels that B should be populated by an array of mutable elements.  But another part of me wants to say that these are referenced elements, and that being part of a constant the elements should not be able to be changed.
So, currently I have only fixed the case where Array and Dictionary have been constructed using a non parameterized constructor.  In this case the Array, and the Dictionary will not act as a constant as they did in the past and this solves some of the reported issues.
This is good thing I believe.
Now, since the const keyword does populate the array with constant nodes elements, it would seem that the next fix is to correct the problem with allowing constants to be mutable.
So in my example above, Attempting B[0] = 5 should not be allowed, because it is being initialized via the constant array A.  Therefore, some of the problems go away with coding misuse.


Support for Dawn Pro

Hi, I would like to ask if supporting MOONDROP Dawn Pro would be possible?
I have the device so I can help if you needed some info (Wireshark data etc.). Thanks!

Sure, i would need some more details.
Namely:

USB model name
USB product id
Preferably an overview of which parameters can be controlled via moondrop link (assume it offers the same controls as the og dawn)

You can retrieve the model name/product id on linux with lsusb.
E.g. for the dawn 4.4mm the output would be
lsusb
...
Bus 007 Device 005: ID 2fc6:f067 Comtrue Inc. Nano audio 1
...

$ lsusb
...
Bus 001 Device 002: ID 2fc6:f06a Comtrue Inc. MOONDROP Dawn Pro
...


Residuals and expression intensity

Is there a way to show the residuals for a given gene and at the same time show the level of exppression in each cell? Maybe by multiplying the residuals by the level on unspliced reads?

I am not sure if you are looking for a pre build function or just the data to plot.The residuals are stored in vlm.velocityAnd expression in vlm.Sx_sz and vlm.Ux_szPlotting functions that plot expression and residuals on the embedding are: vlm.plot_expression_as_color vlm.plot_velocity_as_colorSent from my mobile - excuse the typos


:wrench: [Workaround] Fixed navigation to work properly on both iOS and Android.

Issue

close #953

Overview (Required)

I combined the workarounds listed in the link to make the navigation work properly.

Links

https://github.com/DroidKaigi/conference-app-2024/discussions/882

Movie (Optional)

iOS Before
iOS After

The content of this PR is a workaround, and it cannot be merged as is.
We need to find a non-hacky solution by September 10th.
https://github.com/DroidKaigi/conference-app-2024/discussions/882#discussioncomment-10580276


Write to parquet

Closes #72
I'm pretty sure this is not what you had in mind, but wanted to put this up as at least an option. Even if we scrap the implementation, the test should stay the same.
I didn't include remote writing since it seemed like that would be something that we'd want to implement for all the file outputs not just for one.

I just stole the easiest test off that PR for roundtripping. See what you think

This looks good to me! Let's get it in and iterate if necessary!
Thanks @jsignell !


Make it work with hyperdb

Search-index in hyperdb on top of indexeddb. Then it will become a true service and we'll still not store anything
https://github.com/fergiemcdowall/search-index/issues/409

Nah, try to use hyperdb on top of indexeddb with ramdom-access-idb and access it with hyperdown.

hyperdb is deprecated, and similar stuff won't fully happen in a while. But maybe use something DAT to distribute JSON-files that can be indexed? Test out unwalled gardens!


Inbound rule of private subnet wrong

The private subnet has set inbound port to 'none' which won't allow any connection from the internet, but later the validation steps has RDP connection to the VM in private subnet, which is impossible I think.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: cfd9533e-c174-cd48-0f72-a58457efa958
Version Independent ID: b7a77b77-61c2-5e89-cb4b-0b9a502fce71
Content: Restrict access to PaaS resources - tutorial - Azure portal
Content Source: articles/virtual-network/tutorial-restrict-network-access-to-resources.md
Service: virtual-network
GitHub Login: @KumudD
Microsoft Alias: kumud

Thanks for the feedback! We are currently investigating and will update you shortly.

@eugene-xie The document has steps for the below NSG rules:
Outbound security rules:

Create a rule that allows outbound communication to the Azure Storage service
Create another outbound security rule that denies communication to the internet. This rule overrides a default rule in all network security groups that allows outbound internet communication.

Inbound security rules:

Create an inbound security rule that allows Remote Desktop Protocol (RDP) traffic to the subnet from anywhere. The rule overrides a default security rule that denies all inbound traffic from the internet.

This sequence of rules is not setting inbound port to none but it is adding a RDP 3389 allow rule with a priority of 120 which will override the existing default deny all inbound with a priority of 65500. Please let me know if you have any further questions or if I have not understood your question well. Thank you!

Since we have not heard back from you we will now proceed to close this thread. If there are further questions regarding this matter, please tag me in your reply. We will gladly reopen the issue and continue the discussion.

Since we have not heard back from you we will now proceed to close this thread. If there are further questions regarding this matter, please tag me in your reply. We will gladly reopen the issue and continue the discussion.


Suggestion: Better Bonemeal

Make it possible to bonemeal cactus and sugar cane.
And make coral blocks renewable by making coral plants grow coral structures when you bonemeal them underwater. (Maybe only in warm oceans maybe not only there idk).

This is something a small plugin would be better at handling, tbh.


Energy landscape: find path with lowest highest barrier

Investigate possibility to find path with lowest highest energy. Is this possible via networkx.

Connected to #191

Yes, the minmax-energy pathfinding algorithm solves the issue


linstor: Do not pretend handling disconnect paths that are non Linstor

Description
Excerpt PR from #8790, that only fixes Linstor adaptor pretending handling paths that don't belong to Linstor.

Types of changes

[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] New feature (non-breaking change which adds functionality)
[x] Bug fix (non-breaking change which fixes an issue)
[ ] Enhancement (improves an existing feature and functionality)
[ ] Cleanup (Code refactoring and cleanup, that may add test cases)
[ ] build/CI

Feature/Enhancement Scale or Bug Severity
Feature/Enhancement Scale

[ ] Major
[ ] Minor

Bug Severity

[ ] BLOCKER
[ ] Critical
[x] Major
[ ] Minor
[ ] Trivial

Screenshots (if appropriate):
How Has This Been Tested?
Manual with a Linstor cluster.
How did you try to break this feature and the system with this change?

@blueorangutan package

@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with  KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@blueorangutan test alma9 kvm-alma9

[SF] Trillian test result (tid-9785)
Environment: kvm-alma9 (x2), Advanced Networking with Mgmt server a9
Total time taken: 47593 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr8897-t9785-kvm-alma9.zip
Smoke tests completed. 109 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test
Result
Time (s)
Test File

test_01_migrate_VM_and_root_volume
Error
91.66
test_vm_life_cycle.py

test_02_migrate_VM_with_two_data_disks
Error
55.07
test_vm_life_cycle.py

test_08_migrate_vm
Error
0.07
test_vm_life_cycle.py

@DaanHoogland From what I've looked at the errors from https://github.com/apache/cloudstack/pull/8897#issuecomment-2048642812, them look like an environment instability; could we run the CI once more just to be safe?

@sureshanaparti @rohityadavcloud @shwstppr could we run the CI again?

[SF] Trillian test result (tid-9795)
Environment: kvm-rocky8 (x2), Advanced Networking with Mgmt server r8
Total time taken: 43358 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr8897-t9795-kvm-rocky8.zip
Smoke tests completed. 110 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test
Result
Time (s)
Test File

Rekicked CI by close/reopen.
@blueorangutan package

@rohityadavcloud a [SL] Jenkins job has been kicked to build packages. It will be bundled with  KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

Packaging result [SF]: ✔ el7 ✔ el8 ✔ el9 ✔ debian ✔ suse15. SL-JID 9231

@sureshanaparti @rohityadavcloud @shwstppr could we run the CI again?

@JoaoJandre I had already started it in the background and it seems succeful:

[SF] Trillian test result (tid-9795) Environment: kvm-rocky8 (x2), Advanced Networking with Mgmt server r8 Total time taken: 43358 seconds Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr8897-t9795-kvm-rocky8.zip Smoke tests completed. 110 look OK, 0 have errors, 0 did not run Only failed and skipped tests results shown below:
Test 	Result 	Time (s) 	Test File

sorry, for the lack of communication.

Merging based on reviews, CI result and manual tests

@DaanHoogland Who would merge this into 4.19 and main?

@rp- , RMs do so on a regular basis. A lot of merge commits can be found. On any branche not in freeze you/any of us can as well. There is a tool git-fwd-merge in ./tools/git that can do it and rolls back on conflicts. when on the path it can be called as git fwd-merge. A regular git merge works as well when conflicts occur.

@rp- I merged a bunch forward including this. There was a conflict and I think I resolved it but you might want to check the method disconnectPhysicalDiskByPath.

@rp- I merged a bunch forward including this. There was a conflict and I think I resolved it but you might want to check the method disconnectPhysicalDiskByPath.

Looks fine for me


Fix bug where formatted arguments are passed to self._warn, which only accepts a single string

Hey! I am hosting a new master branch for this repo since the old creator has been MIA for 2 years. Hoping to get this repo back on it's feet and in good standing. I've been working on updating tests and adding all new functionality and have merged some of the PR's that I thought were sufficient for merging. If your PR didn't make it, resubmit it please.
Thanks!
https://github.com/Harrison97/spotipy

Fixes https://github.com/plamere/spotipy/issues/367


Request: upload MDBM_File to cpan

Hi,
It's very nice if we can install the perl mdbm binding MDBM_File from cpan.
So could you upload MDBM_File to cpan?
I tried preparing a repository so that we can install MDBM_File via:
$ cpanm git://github.com/shoichikaji/perl-mdbm-install.git

Hi Shoichi-
I had started work on this, and even registered a PAUSE account.
However, I was trying to make sure that we didn't trample on
MDBM libraries that were installed via other means (yum/apt/etc).
I'm currently looking at "alien" perl packages (one of the alternatives
recommended on perl IRC), but haven't had time to get it wrapped
up yet.
Thanks!
.timrc

I'm very pleased to hear that you had started working on this!
Oh, you intend to prepare Alien::MDBM, and MDBM_File depends on it, right?
That sounds great.
I'm looking forward to MDBM_File being installable from cpan!


meshConfig.accessLogFormat customisation  is not working

Bug Description
Access logs customization not working as mentioned in the documentation
Installed istio via helm
helm install --namespace istio-system istio-base  manifests/charts/base
helm install --namespace istio-system istiod  manifests/charts/istio-control/istio-discovery
helm install --namespace istio-system istio-ingress  manifests/charts/gateways/istio-ingress
Under istio-discovery values.yaml if i update the accessLogFormat with following format it is not working. It just uses the default access log format and customization is not getting picked up
meshConfig:
accessLogFile: /dev/stdout
accessLogEncoding: JSON
accessLogFormat: |
{
systemTime: '%START_TIME%' ,
bytesReceived: '%BYTES_RECEIVED%'
}
If i apply the config with escaped json string it works fine
meshConfig:
accessLogFile: /dev/stdout
accessLogEncoding: JSON
accessLogFormat: |
{
"accessLogFormat": "{"protocol": "%PROTOCOL%","upstream_service_time": "%REQ(x-envoy-upstream-service-time)%","upstream_local_address": "%UPSTREAM_LOCAL_ADDRESS%"}"
}
Even though with escaped config it works the access logs generated seems escaped too which seems not correct
{"accessLogFormat":"{"protocol": "HTTP/1.1","upstream_service_time": "-","upstream_local_address": "<IP_ADDRESS>:42091"}"}
All Istio reference documentation is  referring multi line  format but it seems not working when applied via helem. I tried to manually update the istio configmap under istio-system that too does not help
https://github.com/istio/istio/issues/24318
https://github.com/istio/istio/issues/35812
I see two issues previously raised around the same issue but the issue is marked closed but I still see the issue exists with istio 1.17.1
Version
client version: 1.16.2
control plane version: 1.17.1
data plane version: 1.17.1 (2 proxies)

Client Version: v1.26.1
Kustomize Version: v4.5.7
Server Version: v1.25.3

Additional Information
No response

Able to solve the issue with the approach mentioned here -> https://github.com/istio/istio/issues/40823


Update Composer dependencies (2020-04-11-00-10)

Visual regression test passed!
View the visual regression test report


lerna run test was failing

Description
"lerna run test" command was failing.
Specific Changes
removed broken test script from package.json

Pull Request Test Coverage Report for Build 74690

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 88.289%

Totals

Change from base Build 72883:
0.0%

Covered Lines:
3535

Relevant Lines:
3846

💛  - Coveralls


'javascript:;' in anchor links breaks Content-Security-Policy

There are several anchor links like:
<a href="javascript:;">

Which breaks content security policy.

What's your suggestion?

Oh, I see your PR now

The use of .stop confused me, could you explain a lot more?


Add custom hooks for all the nosto elements so that they can placed anywhere at will

Whats the goal with this? It's just as easy to add he reco divs directly to the layout files instead.
I propose we close this issue.

Closing, not needed


Clinical Test Observations not correctly identified in Bulk Data tests

When using US Core 5, Clinical Test Observations are not identified. See https://chat.fhir.org/#narrow/stream/179309-inferno/topic/Multi-Patient.20Authorization.20and.20API.20STU2.20.288.2E3.2E16.29

Is the fix pushed to Inferno

It should be available on inferno-dev.healthit.gov, but it won't be on the main site until the official release next week.


Project structure contains field with wrong type

Describe the bug
Some field of the Project struct have changed type on deps.dev service and are not updated in this library.
Fields are:

OpenIssuesCount
StarsCount
ForksCount

They all should be integer instead of string.
To Reproduce
Steps to reproduce the behavior:

Check response from API call of getProject endpoint:

https://api.deps.dev/v3/projects/github.com%2Ffacebook%2Freact
https://api.deps.dev/v3/projects/github.com%2Fedoardottt%2Fdepsdev

Expected behavior
This library should update the type of the field to string so it can properly decode the json sent by deps.dev API.

I can open a PR to solve the issue.
And I think it would be good to also add a workflow that run periodically to ensure this kind of problem is catch as early as possible. I can also draft a GitHub Action workflow that does that if you want.

Thanks @zaibon .
Seems they are in a moment when things change frequently https://github.com/google/deps.dev/issues/47.

And I think it would be good to also add a workflow that run periodically to ensure this kind of problem is catch as early as possible. I can also draft a GitHub Action workflow that does that if you want.

How? There isn't an openapi file as far as i know. I've setup a change monitor instance for this and I was aware of this change. The main problem is that I'm working full time and I don't have too much time to spend on OS projects for now :(
Also, as you can see in the issues, I'm trying to align the project with the current implementation.

package v3 for the v3 API (stable, https://docs.deps.dev/api/v3/)
package v3alpha for the v3alpha API (unstable and frequent changes, https://docs.deps.dev/api/v3alpha/)

How? There isn't an openapi file as far as i know. I've setup a change monitor instance for this and I was aware of this change. The main problem is that I'm working full time and I don't have too much time to spend on OS projects for now :(

My idea is to define a bunch of tests in the code that does actual call to the API and verify the output corresponds to what we expect. We can run this for each PR and also like once a week automatically.
This would catch any unmarshal error very early and help to keep in sync with the dev.deps API.

We can run this for each PR and also like once a week automatically.

I see that on the devel branch, you already started creating these tests in https://github.com/edoardottt/depsdev/blob/devel/pkg/depsdev/v3/api_test.go
So the only thing left to do, would be to define a schedule in the Go workflow.

TBH I don't think this is reliable.
Consider that those tests use my own projects and not all the APIs are tested. If as example tomorrow I'll drop a new release of defangjs, the action will fail (https://github.com/edoardottt/depsdev/blob/devel/pkg/depsdev/v3/api_test.go#L31).
You've used defangjs for TestGetProject too, what if tomorrow someone stars the repo? The stars gonna be 23 and not 22 and the test (so the action) will fail...

That's right, checking the actual value won't be deterministic. But we can probably skip that, since the goal was really just to ensure the unmarshaling process is valid.

That's right, checking the actual value won't be deterministic. But we can probably skip that, since the goal was really just to ensure the unmarshaling process is valid.

makes sense

Thanks for you contribution @zaibon !


cube: Use volk to load functions

cube: Use volk to load functions
Add volk as a repo dependency to replace the dependency on the Vulkan-Loader.
This makes it easier to build the repo as the vulkan-loader has a much more
complex build than volk does. It also allows vkcube to launch without vulkan
being installed on the system, allowing for better error handling.
Because volk can be used on all platforms, there is no longer a need for
android_wrapper. Thus it has been removed.
cubepp: Use dynamic dispatcher
This removes vkcubepp's linking to the vulkan-loader in order to build. That way
the Vulkan-Loader isn't a dependency for vkcubepp.

CI Vulkan-Tools build queued with queue ID 79949.

CI Vulkan-Tools build # 1275 running.

CI Vulkan-Tools build # 1275 passed.

Mac build is having issues.
FAILED: cube/CMakeFiles/vkcube.dir/macOS/cube/DemoViewController.m.o
/usr/local/opt/ccache/libexec/cc -DVK_USE_PLATFORM_METAL_EXT -I/Users/runner/work/Vulkan-Tools/Vulkan-Tools/cube -I/Users/runner/work/Vulkan-Tools/Vulkan-Tools/external/Release/64/MoltenVK/MoltenVK/include -I/Users/runner/work/Vulkan-Tools/Vulkan-Tools/cube/. -isystem /Users/runner/work/Vulkan-Tools/Vulkan-Tools/external/Release/64/volk/build/install/include -isystem /Users/runner/work/Vulkan-Tools/Vulkan-Tools/external/Release/64/Vulkan-Headers/build/install/include -O3 -DNDEBUG -std=c99 -isysroot /Applications/Xcode_14.2.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.1.sdk -mmacosx-version-min=10.15 -fPIE -fvisibility=hidden -fsanitize=address -Werror -Wall -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -fno-strict-aliasing -fno-builtin-memcmp -Wno-sign-conversion -Wno-shorten-64-to-32 -Wno-string-conversion -Wno-implicit-int-conversion -Wno-enum-enum-conversion -Wno-deprecated-declarations -MD -MT cube/CMakeFiles/vkcube.dir/macOS/cube/DemoViewController.m.o -MF cube/CMakeFiles/vkcube.dir/macOS/cube/DemoViewController.m.o.d -o cube/CMakeFiles/vkcube.dir/macOS/cube/DemoViewController.m.o -c /Users/runner/work/Vulkan-Tools/Vulkan-Tools/cube/macOS/cube/DemoViewController.m
In file included from /Users/runner/work/Vulkan-Tools/Vulkan-Tools/cube/macOS/cube/DemoViewController.m:24:
In file included from /Users/runner/work/Vulkan-Tools/Vulkan-Tools/cube/cube.c:55:
/Users/runner/work/Vulkan-Tools/Vulkan-Tools/external/Release/64/volk/build/install/include/volk.h:940:37: error: redefinition of 'vkAllocateCommandBuffers' as different kind of symbol
extern PFN_vkAllocateCommandBuffers vkAllocateCommandBuffers;
                                    ^
/Users/runner/work/Vulkan-Tools/Vulkan-Tools/external/Release/64/MoltenVK/MoltenVK/include/vulkan/vulkan_core.h:4599:32: note: previous definition is here
VKAPI_ATTR VkResult VKAPI_CALL vkAllocateCommandBuffers(

CI Vulkan-Tools build queued with queue ID 80918.

CI Vulkan-Tools build # 1279 running.

CI Vulkan-Tools build queued with queue ID 80926.

CI Vulkan-Tools build # 1280 running.

CI Vulkan-Tools build # 1280 passed.

mac CI still failing

CI Vulkan-Tools build queued with queue ID 81012.

CI Vulkan-Tools build # 1282 running.

CI Vulkan-Tools build queued with queue ID 81105.

CI Vulkan-Tools build # 1283 running.

CI Vulkan-Tools build # 1283 passed.

The reason the Mac build is failing is due to how vkcube is able to 'find' moltenVK. When linked to the Vulkan-Loader, CMake sets up the RPATH so that it can link to libvulkan.dylib. When using volk, that link doesn't happen. Rather, because there is no linking to the loader, no RPATH is being set up (thus volk fails to find libvulkan.dylib)
One solution is to just make this PR apply to only windows & linux, since those are platforms where the loader can be assured to be in system paths. Whereas on apple, the loader would be in the bundle (as would moltenVK), yet there doesn't seem to be an easy way to tell dlopen to look inside the bundle.

Welp, vkcube doesn't work on linux with this PR either. Saying it can't find the loader, which is unfortunate.
My local setup had a LD_PRELOAD=/path/to/libvulkan.so which skewed my own local testing :(

CI Vulkan-Tools build queued with queue ID 85214.

The comment I just made and subsequently deleted was due to a bug where the result value of volkInitialize() wasn't being checked, and vkcube ALWAYS calling ERR_EXIT() afterwards. A very silly mistake on my part, and has now been fixed.
There is still the macOS issue of not being able to find vkcube, but that doesn't have as simple of a fix.

CI Vulkan-Tools build # 1289 running.

CI Vulkan-Tools build # 1289 passed.

I'm using Volk on macOS now successfully, so this strikes me as odd. You are still packaging libMoltenVK.dylib in the /Frameworks folder, correct?

Yes, libMoltenVK.dylib is in the Frameworks folder. When I run otool -L with this PR, there is no RPATH setup, whereas on main, RPATH points to the loader. Does dlopen not know to look in the Frameworks folder by default on macOS?

CI Vulkan-Tools build queued with queue ID 85409.

CI Vulkan-Tools build # 1290 running.

CI Vulkan-Tools build # 1290 passed.

CI Vulkan-Tools build queued with queue ID 85469.

CI Vulkan-Tools build # 1291 running.

CI Vulkan-Tools build # 1291 passed.


[Discussion] Improve search experience in the documentation

Hi :wave:,
I am member of the .NET foundation and I am working at Algolia on a project called DocSearch which goal is to enhance documentation websites with exhaustive, fast and relevant search. You might have seen DocSearch live already on websites like brew, Bootstrap, React or jQuery. We are using Json.NET in our .NET library so we thought it could be nice to add DocSearch to your documentation!
I have created a preview of the search experience and how DocSearch could look like on the Json.NET website. Feel free to try it and let us know what you think. Please not the learn-as-you-type experience and the typo tolerance:

The way DocSearch works is by crawling your content, pushing the results into an Algolia index, and then requesting this index directly from the website front-end through JavaScript.
We'll take care of crawling your website and populating the Algolia index with the latest changes every 24h for you. You don't need to change anything to your deployment process. The only thing you need to add are the following CSS and JS snippets that will bind the dropdown to your searchbox.
We built DocSearch with the idea of giving back to the Open-Source community. This is why your crawling configuration is available on GitHub if you want to change it. We also have our own documentation to help you tweak the dropdown to your needs. You'll also have access to analytics on the most searched terms or those with no results. All of this is of course entirely free.
We would be very happy to set it up and PR it!

I can do a PR to integrate it if needed. Such as https://github.com/dotnet/BenchmarkDotNet/pull/1148


[FLINK-1906] [docs] Add tip to work around plain Tuple return type of project operator

Add a tip about project transformation with type hinting in documentation. Related JIRA is here.

Good addition, +1 to merge

Thanks for the PR!


Process is not terminated remains null after bar is finished and program terminated.

After terminating the process with Ctrl-C it does not actually close and timers continue to run in the background before erroring when trying to write to console.
This behaviour is not seen when disabling the progress bars and only requires a new instance of the MultiProgressBars class to be created. Its not affected by calling .close or .cleanup.
You can checkout the codebase I'm trying to use it in here: https://github.com/Inrixia/Floatplane-Downloader/tree/5dae0aa57be48cbb87bebc0bfa4f2c3fcad18eef

I'll take a look. Seems I might have forgotten to call the cleanup code in the virtual-console from the multi-progress-bars class.

I thought so too, but debugging it it seems to still be called which was very confusing. From the looks of things everything works fine dealing with redirecting the console after close and its a issue with SIGINT from crl-c. As for why the process continues in the background but with console as null I'm lost.

Hey, I saw this github issue: https://github.com/nodejs/node/issues/9050
At first, I thought it was because of a loop, but actually, I think it's because I need to actually call process.exit() in cleanup().
I'm going to build and push a new version out, let me know if it solves the problem.

Published as<EMAIL_ADDRESS>

Looks like that fixed it! Thanks <3


Math support request (question)

Does CW have any support for MathJax or other such related things ? If they do can you inform me (or give a link) explaining how it works. If not can we add it to the ( already extremely long and boring) list of stuff that CW can add in the (near or far) future. I'd help a lot since creating Math related Kata's becomes a pain.

I don't think there is. @jhoffner correct me if I'm wrong.
I'll close this and label #1527 as feature request.


Add Upgrade Recommendations

Could be able to check if there are any recommendations for Masonite. See an example in the docs here.
This can be used when you hit a bug on Masonite 2.2.23 but the latest is version 2.2.31 for example

If this could be added to each package which shows available upgrades that would be nice

Can be added as an integration


Fixing build failures after Flavor Serialization backport

Signed-off-by: Andriy Redko<EMAIL_ADDRESS>Description
Fixing build failures after Flavor Serialization backport (https://github.com/opensearch-project/OpenSearch/pull/1757)
Issues Resolved
Multiple builds are failing:

https://ci.opensearch.org/logs/ci/workflow/OpenSearch_CI/PR_Checks/Gradle_Check/gradle_check_1804.log
...

Check List

[ ] New functionality includes testing.

[ ] All tests pass

[ ] New functionality has been documented.

[ ] New functionality has javadoc added

[ ] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@nknize @dblock please :-)

start gradle check


CuPy v11 Release Plan

Planned Features

Explore possibility of #3972
#6244
#6180
#4897
Make downstream packages easier to use CuPy

Provide an universal wheel package for downstream libraries (ongoing in #6012)
Document guideline for downstream libraries (c.f., NumPy: For downstream package authors

Increase NumPy/SciPy API Coverage
Deprecation (under discussion, may subject to change)

CUDA: Keep CUDA 10.2 for Jetson. Drop some of the 11.x variants.
ROCm: Drop 4.2 or earlier. Keep supporting ROCm 4.3 as it is the last version that runs on gfx80x.
Python: No deprecation (support 3.7+).
NumPy: Drop NumPy 1.19.
SciPy: Drop SciPy 1.4.

Schedule

[ ] v11.0.0a1 (v10.1.0) (2022-01-20)
[ ] v11.0.0a2 (v10.2.0) (2022-02-25)
[ ] v11.0.0b1 (v10.3.0) (2022-03-31)
[ ] v11.0.0b2 (v10.4.0) (2022-04-28)
[ ] v11.0.0b3 (v10.5.0) (2022-05-26)
[ ] v11.0.0rc1 (v10.6.0) (2022-06-30)
[ ] v11.0.0 (2022-07-28)

Notes:

The schedule and listed features are tentative and may be subject to change depending on the progress.
The code will be frozen around 15:00 on 2 days before the release date for QA. Dates are in JST.
v10.x.x releases will only contain backported pull-requests (mainly bug-fixes) from the v11 (master) branch.

Past Release Plans

v9: https://github.com/cupy/cupy/issues/3891
v10: https://github.com/cupy/cupy/issues/5049

Is there a typo in date? (2022-*)

@yixunzhang Yes, thanks for your reporting ;)

I provided my own on my website https://www.longervision.us:
Welcome to have a try...
cupy-11.0.0a2-cp38-cp38-linux_x86_64.whl

@jiapei100 We have wheels published officially. Are there any difficulties using them?
https://docs.cupy.dev/en/latest/install.html#installing-cupy
https://pip.cupy.dev/pre/

@kmaehashi
I got some issue when using XGBoost, please refer to https://stackoverflow.com/questions/71378334/attributeerror-module-cupy-has-no-attribute-ndarray.

v11.0.0rc1 (v10.6.0) (2022-06-30)
Revisit https://github.com/cupy/cupy/pull/6460

uarray PRs are not landed in SciPy (https://github.com/scipy/scipy/pull/15610), but I think it's ok to leave #6460 in CuPy as-is, as this is an unbreaking change. We can follow the changes in SciPy side and backport to v11 when needed.

Released v11.0.0! https://github.com/cupy/cupy/releases/tag/v11.0.0


Scala Common Enrich: add POST support for /i as well as com.snowplowanalytics.snowplow/tp2

In case there are ad blockers blocking POSTs to URIs with the word analytics in them.
Are there any ad blockers like this?
There would also need to be a flag in the Snowplow JS Tracker (or new default?) to use this.

Migrated to https://github.com/snowplow/enrich/issues/110


Minor additions to labelling steps

Small additions to the labelling steps instructions after a first run

add a pair of useful shortcuts
add a note about the project name shown when a project is re-loaded

👍


Tron Lighting does not work on Alienware x16.

Only 1 small section on the middle is highlighted when trying to set up lighting.

Also I have set up the lighting on the x16 keyboard. Hope this helps.
x16.csv

Thanks for CSV.
About tron - just check all IDs, sometime it can be located at something like 100-130.


Network details

I see that the network is dependent on actor details json as input for predicting the velocity. Am i correct?
Can i skip using the actor details for predicting the velocity.

Hi!

We use the velocity from actor details json file as the ground truth to supervise the network
No, the data from actor details json is needed to be used as GT.

Hi,
Appreciate you for taking your time to reply.
I have few more clarifications, can you please help me with them

What is the significance of using waypoints.npy file, as you are already taking the future waypoints in the measurements json.
what are the commands like x_command, y _command, command, gt_command that are in the measurements file, especially I see you are using the command key from the file, doing one hot encoding and adding speed to it . what is the significance of it?
what is the the third value in future waypoints? I see the third value to be 4 mostly.

Thanks in advance

Hi, thanks for your interest in our project!

Actually, intertuser doesn't use this file. It collect the future waypoints of the ego car in one file to speed up the io speed. But in our project, we only use the future waypoints that the navigation algorithm provided, not the ego car produced.
About these four value, you can refer to https://github.com/opendilab/InterFuser/blob/e0682c350892a243cf40bf448622743f4b26d0f3/leaderboard/team_code/base_agent.py#L431. The first two are the coordinate of the target points, the last two are two commands (ex. turn left, right, ...) from two different plannner. Using the key and speed in the network is to provide more information for the ego car.
Sorry i can't get what you mean, can you provide more information? may be z-axis or commands ?
target[4] includes the velocity ground truth.

Hi,
Thanks for your reply.

My third point I mentioned in the above comment is about 3rd value in the future waypoints list which are available in the measurements json file. I figured out that it represents the commands anyway.
Can you help me with the rough picture of network if u have. I saw a figure in the paper but I am not able to understand that. It would be helpful if u provide network architecture depiction. I want to understand how the data is flowing especially measurements and actors data.
What is the significance of variable end2end in InterFuser class?

Thanks in advance for your reply. I will come up with more doubts :)

Hi,

It denotes command, 4 means "Go straight", so is the most.
You may combine the paper and code (https://github.com/opendilab/InterFuser/blob/e0682c350892a243cf40bf448622743f4b26d0f3/interfuser/timm/models/interfuser.py#LL980C9-L980C16) to help you understand. Here, self.direct_concat = False, self.end2end = False, self.waypoints_pred_head = "gru".
If we set end2end as True, the model will directly output the waypoints, without other heatmap, traffic sign, etc.

thanks


DB2 Date Arithmetic

Actual Behavior
execute the following SQL caused exception below
SELECT DOB + 1 YEARS as NewDOB FROM TABLE
Caused by:
net.sf.jsqlparser.parser.ParseException: Encountered " "AS" "AS "" at line 1, column 36.
Was expecting one of: ....
it is unable to use db2 Date arithmetic
Expected Behavior
Steps to Reproduce the Problem

Specifications

Version:
Platform:
Subsystem:

Hello,
I am running into this issue (or at least something related to it) as well.  Date math seems to work in JSQLParser for simple cases, but there are some more which do not.
See https://www.ibm.com/support/knowledgecenter/en/SSEPEK_10.0.0/sqlref/src/tpc/db2z_datearithmetic.html for some documentation on this.
Some simplified examples that we have from our existing set of SQL reports, which fail in JSQLParser but work in DB2, are as follows:

Duration in brackets:
SELECT CURRENT_DATE + (1 DAY) FROM SYSIBM.SYSDUMMY1
Naming an output column (as reported above):
SELECT CURRENT_DATE + 1 DAY AS NEXT_DATE FROM SYSIBM.SYSDUMMY1.
This also fails if you leave out the 'AS':
SELECT CURRENT_DATE + 1 DAY NEXT_DATE FROM SYSIBM.SYSDUMMY1
Multiple operations on a single field:
SELECT CURRENT_DATE - 1 DAY + 1 YEAR - 1 MONTH FROM SYSIBM.SYSDUMMY1
Not sure exactly how to classify this one... This fails:
SELECT CASE WHEN CURRENT_DATE BETWEEN (CURRENT_DATE - 1 DAY) AND ('2019-01-01') THEN 1 ELSE 0 END FROM SYSIBM.SYSDUMMY1
but if you replace (CURRENT_DATE - 1 DAY) with ('2018-01-01') it works.

Please let me know if you need any other samples or information.
Thank you very much for your support!
Cheers

There is a JSqlParser misparsing between date literals (month, year) and aliases. This would be very difficult to implement (I think).
select 5 years or select 5 as years is for many dbms the same and years is the alias.

Hmm, good point.  According to https://www.petefreitag.com/tools/sql_reserved_words_checker/, YEAR, MONTH, DAY, etc (singular) are reserved words in many DBs including DB2, Postgres, ODBC, and future versions of SQL Server.  The plural forms YEARS, MONTHS, DAYS, etc are only reserved on DB2.  Would it be easier to implement this feature using the singular form as a reserved word?
This is a show stopper for any DB2 queries involving date math; speaking at least for our reports, this is a very common use case.  Furthermore, from what I have found there is no workaround that works in JSqpParser.  If the parser supported date math using the singular forms, that could be a way to make it work without some of the difficulties that you point out above.
I realize that adding reserved words this late in the game is dangerous, and may affect other users' queries.  If the changes cannot be put into your branch, I may look at forking a copy and doing modifications like this for better DB2 support.  Would doing it this way be an easy fix?  I am researching EBNF grammars, but do not yet understood the .jjt file.  While I would obviously prefer using the official version, this parser is so close to doing exactly what we need that I don't want to move to a different product if possible.
Thanks!

Adding keywords could be done without fork. There are so many tests, that one should be warned quite fast.

Would adding those keywords YEAR, MONTH, DAY, etc make parsing easier / possible?

It would be a first step. One should decide how to handle those alias conflicts. Maybe integrate some semantic lookaheads to make it configurable.

It's been a while... I have been working on other areas of the product not directly related to SQL Parsing.
I am now looking back at this.  I have been unable to figure out how to modify the code to support the date math.  Unfortunately this is a major holdup for our software, as we use date math frequently, and without either support for the date math or a workaround, JSQLParser would not work for us.
Are there any plans to implement this?  It looks like we are not the only ones encountering the issue, see for instance https://github.com/JSQLParser/JSqlParser/issues/666 .
Alternatively, if you would be able to give me some pointers on what would need to be done to implement this, I can try again.  I have not worked on parser code before, and especially not JavaCC, and so am not understanding what needs to go in the .jjt file and what should go into the classes.  I thought I could figure it out by jumping in, but the learning curve is too steep for me.
Thanks!

There is a good start at the IntervalExpression. This is what you want. But JSqlParser only supports it with a leading "INTERVAL" keyword.
Look e.g. at SelectTest.testInterval4.
The interval keyword has to be optional, therefore the production should look something like:
IntervalExpression IntervalExpression() : {
    IntervalExpression interval = new IntervalExpression();
    Token token;
    boolean signed = false;
}
{
    [ <K_INTERVAL> ] ["-" {signed=true;}] (token=<S_LONG> | token=<S_DOUBLE> | token=<S_CHAR_LITERAL>)
    { interval.setParameter((signed?"-":"") + token.image); }
    [ LOOKAHEAD(2) token = <S_IDENTIFIER> { interval.setIntervalType(token.image); } ]
    {
        return interval;
    }
}

Now comes the hard part. I guess there are multiple problems in the grammar. JavaCC cannot decide, which production to go. JavaCC gives you, while building the parser, warnings where a collision appears.
Hope that helps for a first start.

OK thanks, I will take a look and let you know if (or more likely when ;-) ) I have more questions.

Looking good, thanks very much!  There are some places in our set of SQL statements that are still not quite working, as my simplified test cases above didn't explicitly address them.

Date offset coming from an expression instead of a integer constant:
select offset, CURRENT_DATE + offset MONTH as new_date from mytable
Any integer expression should be fine here: constant, function, case statement, etc.  It looks like integer constants are working but the others are not.  Other examples:
select CURRENT_DATE + (RAND() * 12 MONTH) as new_date from mytable
select CURRENT_DATE + CASE WHEN CAST(RAND() * 3 AS INTEGER) = 1 THEN 100 ELSE 0 END DAY AS NEW_DATE from mytable
select CURRENT_DATE + (dayofweek(MY_DUE_DATE) + 5) DAY FROM mytable
select CASE WHEN CAST(RAND() * 3 AS INTEGER) = 1 THEN NULL ELSE CURRENT_DATE + (month_offset MONTH) END FROM mytable  (Within a case statement; it looks like this should Just Work when the first example is working, as if I change month_offset to 1 it works fine.)

Note that these will generally parse (incorrectly) if you don't supply the alias, but they are parsing as the expression being X + Y with an alias of MONTH, not X + (Y MONTH).

The spec also allows for time intervals such as HOUR, MINUTE, SECOND.    For instance:

select CURRENT_DATE - 1 HOUR AS FOO from SYSIBM.SYSDUMMY1
select CURRENT_DATE - 2 MINUTE AS FOO from SYSIBM.SYSDUMMY1
select CURRENT_DATE - 3 SECOND AS FOO from SYSIBM.SYSDUMMY1
This is easily changed by modifying line 143 of JSQLParserCC.jit to be:
|   <K_DATE_LITERAL : ( "YEAR" | "MONTH" | "DAY" | "HOUR" | "MINUTE" | "SECOND" ) >
All tests still pass, after compiling this works for the above test case.  I can supply a pull request if you want, but I imagine it would be simpler to just copy the above line.

Doing some more investigation into point 1...
If you add <S_IDENTIFIER> to the line ( token=<S_LONG> | token=<S_DOUBLE> | token=<S_CHAR_LITERAL> | token = <S_IDENTIFIER>) it fixes many of the issues we are seeing (most of the time we use a single identifier as the offset value; it is a less frequent case that we use an expression).
A test case for this scenario is:
select current_date + foo day as next_day from foo
It still does not work for expressions.  I tried adding support for an expression directly there, but it gives a left parse error when reading the .jj file.  I tried wrapping the expression in brackets (accepted but not required in DB2), and everything compiled but tests started to fail - it looks like the parser was getting confused.
Anyway, the S_IDENTIFIER change works for most of our reports.  There are now only 11 out of 1553 reports which fail to parse.
I have created a pull request https://github.com/JSQLParser/JSqlParser/pull/756 which adds the identifier, as well as the hour / minute / second date literals from point 2 above, as well as some new test cases which test these scenarios.
Cheers


WIP deps(upgrade): upgrade deps

Closing PR in favor of other fixes that accomplishes the same task.


Warn when using unsupported python kernel versions

For #7960

Codecov Report

Merging #8214 (b70f79b) into main (57dcd19) will increase coverage by 0%.
The diff coverage is 66%.

:exclamation: Current head b70f79b differs from pull request most recent head 3d180c6. Consider uploading reports for the commit 3d180c6 to get more accurate results

@@          Coverage Diff          @@
##            main   #8214   +/-   ##
=====================================
  Coverage     71%     71%
=====================================
  Files        368     368
  Lines      22591   22606   +15
  Branches    3418    3422    +4
=====================================
+ Hits       16212   16229   +17
+ Misses      4984    4982    -2
  Partials    1395    1395

Impacted Files
Coverage Δ

...t/datascience/notebook/vscodeNotebookController.ts
75% <60%> (-2%)
:arrow_down:

src/client/common/utils/localize.ts
95% <100%> (+<1%)
:arrow_up:

.../datascience/notebook/notebookControllerManager.ts
73% <100%> (+<1%)
:arrow_up:

...lient/datascience/jupyter/kernels/cellExecution.ts
74% <0%> (-1%)
:arrow_down:

src/client/debugger/jupyter/kernelDebugAdapter.ts
82% <0%> (-1%)
:arrow_down:

src/client/datascience/telemetry/telemetry.ts
76% <0%> (+<1%)
:arrow_up:

...ent/datascience/jupyter/kernels/kernelExecution.ts
69% <0%> (+1%)
:arrow_up:

...tascience/jupyter/kernels/kernelCommandListener.ts
70% <0%> (+3%)
:arrow_up:

...notebook/intellisense/jupyterCompletionProvider.ts
68% <0%> (+5%)
:arrow_up:


Settings are not editable on reload

It appears one save and reload of the page, the editor doesn't allow you to edit the values, but instead when you select 'background' again, it clears them:
E.g. Setting the background

t background
On Save and Reload - you can't edit the settings:

Any workaround on this?

+1 - any updates?

+1 - any updates?

I tested now, and i can't reproduce, may it have has been corrected

I have the same issue. When components is set by editor.setComponent and the components is string html.

I'd expect it to be fixed with the new release of the plugin