Draft: Apply Typestate pattern to Holder

TODO...
To determine:

Approach for legacy Holder support? - should this replace the existing or be a new construct?
How to handle state transitions which previous result in two different state?.. e.g. Holder<OfferReceived>::send_request could progress into Failed or RequestSent according to the old API

Codecov Report

Merging #768 (004afdc) into main (7a0d3ee) will decrease coverage by 0.38%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##             main     #768      +/-   ##
==========================================
- Coverage   54.60%   54.22%   -0.38%
==========================================
  Files         380      388       +8
  Lines       36740    36987     +247
  Branches     8082     8107      +25
==========================================
- Hits        20062    20058       -4
- Misses      10712    10958     +246
- Partials     5966     5971       +5

Flag
Coverage Δ

unittests-aries-vcx
54.12% <0.00%> (-0.38%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

 Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

Thanks for this work! Amazing to see the upgrade on other protocols.

I think it depends, we can keep your current approach having this as completely new code. If we by the end of this work find out there's not breaking changes in data structures for important states (Finished for holder, CredentialSent and Finished for issuer), it could be fairly smooth to replace old code by new one.
But if we find out we want to actually take the new implementation to make bunch of changes as well, then we can just deprecate old code and endorse the new state type pattern implementation). Personally I encourage you to fix any weirdness you run into and not hold yourself back too much for sake of compatibility.

Not sure about this. I guess that's a case we didn't run into with connection protocol. First thought which comes to my mind is that we'll just have to sometimes return some enum of 2 state and then the user would have to match the variant.
Perhaps smthing like:

enum SendRequestResult {
   Failed(FailedState),
   RequestSent(RequestSentState)
}

and then have Holder<SendRequestResult>::send_request?
Alternatively it can be worth looking from completely different angle, could we somehow linearize the state machine so that every method would call would either fail or progress (and return) new state?
I feel like this is actually the right way. Either pass to next state, or return error, and let the wrapping code - either retry or manually move the state machine to failed state. That would mean that transitions like send_request would no longer try to send problem report automatically.

Just to make sure i am coming clear with point 2b, simple picture to further elaborate on the suggestion

I second the idea of striving to deprecate the old implementation in favor of the typestate pattern. Ideally, the new impl would replace the old one wherever it was used.
Regarding error handling, I'll mention the ideas proposed in the community call:

Error states such as Holder<Failed> could implement Error and the result from an operation would be Result<Holder<NextState>, Holder<Failed>>. This would be ideal because a state machine ending in that state is practically unusable and should only be used for logging/error handling purposes, not reused in some way.
I haven't done this in the Connection state machine (yet), but the way I've seen it being used makes me believe we could return things like Result<Connection<NextState>,(Connection<OldState>, Error>). Essentially, this would allow us to re-use the state machine in case of an error unrelated to the protocol progress (network issue, wallet issue, etc.). This would allow us to stop cloning a state machine whenever we wanna use it and instead be able to re-use it in case of a non-protocol failure.

Thanks for the feedback @Patrik-Stas &  @bobozaur.
Something I've been struggling with finding a solution for, is the question around; when is it aries-vcx's job to send a problem report and consider a protocol as 'failed'? Should it ever be aries-vcx's job to do that automatically, or should we just offer the consumer an easy way for them to enter this state themselves?
Consider send_request; in the original handler, if it fails to _make_credential_request, it automatically sends a problem report and transitions into the terminal failed state.
But this could fail for numerous reasons;

a network anomaly occurs when communicating with the ledger
the wrong profile is used by the consumer - e.g. wrong wallet, or wrong genesis file being used for ledger
the wallet isn't opened
etc

A lot of these cases are not "fatal" failures, and they could potentially be recovered by the consumer in a way that lets them continue with the protocol.
So I wonder if instead, all aries-vcx should do, is attempt to progress in the protocol, and if it fails, we return that error back to the consumer without transitioning to 'fatal' or sending a problem report on their behalf. And then, if the consumer can process the error (with whatever application logic they have), and if they decide it is recoverable, then they can try again. And if they decide it is a fatal error, then they can call on some send_problem_report API which sends the report and transitions them into the Holder<Failed> state.

Another benefit of this approach, is that it allows consumers to control what the problem_report message payload is. Currently, the handlers just convert the VcxError to a string and send that as the problem report payload.
The consumer may not desire that behavior, and it may even potentially leak details they did not wish to share with the peer they are connected with.

I do agree with @bobozaur that since these type state APIs consume the current object, it would be nice if (in the case of an error result) we return back the original object in some way. As this could be more efficient than requiring them to clone the entire Holder before they partake in any consuming APIs.

@gmulhearn I agree with you that we should be as minimal as possible in terms of what we do. Things like sending a ProblemReport should be the user's decision. We perhaps only have the responsibility of generating it (when appropriate).
In fact, I do want to explore a sans IO approach to the protocols, but it would be quite the research. The idea would essentially be to take all IO out of our protocol implementations and leave it to the user.
A lot of errors that would occur because of this would simply not be our responsibility, and while that sounds like we're avoiding dealing with them, we in fact let users deal with them however they want, without sacrificing the state machines and stuff like that.
For instance, I want to explore the idea of having something like a Guard ZST bound to the state machine in such a way that if you don't take the appropriate (possibly external) action, you don't get an instance of the Guard  back that you would use to progress the state machine.
A very raw draft would be something like this:
pub struct Guard {
    users_cant_build_me: () // or PhantomData, or whatever
}

fn main() {
    let holder: Holder<OfferReceived> = gimme_holder();
    let guard: Guard = holder.send_request().unwrap() // non-consuming
    let holder: Holder<RequestSent> = holder.progress(guard);
}

This might require additional states in the state machine, beside the ones provided in the protocol (e.g: OfferReceived -> RequestCreated -> RequestSent). I personally don't see this as a problem, as I think the ones specified in the protocols are basically flow-related and not absolute. Also, the benefits of something like this are, in my opinion, huge.
The end goal of an approach like this would be to make all state machines operations either non-consuming or infallible.

hi @gmulhearn how do you see this coming along? I understand that you've mostly been waiting for moving parts around the messages to settle. I find it quite fortunate as in the meantime, we had extra time to come up with the implementation guidelines https://github.com/hyperledger/aries-vcx/blob/main/aries_vcx/docs/guidelines.md
Perhaps before you restore the work on this, how about you sketch out design for the states and transitions - we can get that approved here so there won't be any surprises upon the review

hey @Patrik-Stas , we can discuss this further tomorrow, but here's what I'm thinking so far:
Draft of new APIs:
impl Holder<ProposalPrepared> {
    fn create_from_proposal(proposal_data) -> Self
    fn get_proposal_message(&self) -> CredentialProposalMessage

    // can only fail if invalid thread_id
    fn receive_offer(&self, offer: CredOfferMsg) -> Result<Holder<OfferReceived>, AriesVcxError>
}

impl Holder<OfferReceived> {
    fn create_from_offer(offer) -> Self

    fn prepare_proposal(self, proposal_data) -> Holder<ProposalPrepared>

    // can fail if: offer attachment could not be parsed, create-cred-request fails (ledger or anoncreds failure)
    fn prepare_request(&self, ...) -> Result<Holder<RequestPrepared>, AriesVcxError>

    fn decline_offer(self) -> Failed
}

impl Holder<RequestPrepared> {
    fn get_request_message(&self) -> CredReqMessage

    // can fail if: cred attachment could not be parsed, processing/storing cred fails (ledger or anoncreds failure)
    fn receive_credential(&self, cred: CredMsg) -> Result<Holder<Finished>, AriesVcxError>
}

impl Holder<Finished> {
    fn get_ack_message(&self) -> Option<AckMessage>
}

impl Holder<Failed> {
    fn get_problem_report(&self) -> ProblemReport
}

// impl for all states
impl Holder {
    // could rename this to just `fail`?
    fn transition_to_failed(self) -> Holder<Failed>

    // ....
}

Note that I'm just focusing on the new transition based APIs, there will also be some utility APIs around getting data at different states, but that should be fairly straight forward
Things I'm unsure of:
Error Transitions
When transitions have the possibility of throwing an Error (e.g. prepare_request) I'm having the APIs take in &self as the param rather than self. This is one approach to satisfying this statement in the guidelines:

Transition should either fail and leave the current state unmodified, or transition to predetermined new state.

If the transition results in an error, the consumer will still have access to self. I know we discussed fancier approaches, (such as (self,..) -> Result<Holder<NewState>, (Holder<CurrentState>,AriesVcxError)>), however I don't see the benefit over just taking &self - which is a simpler and easier approach. Maybe someone could shed some light of why the fancier approach is better?
Transition in Failure
I have it such that transition into Failed are manually chosen by the consumer. This is done by calling the transition_to_failed. Is this what we intended.
ACK handling in the Finished state...
Holder<Finished>::get_ack_message returns optional AckMessage bcus the current implementation of the Holder handler ONLY sends the ack message if the credential message has the please_ack flag enabled. So the idea is that if the please_ack flag was true, this method would return Some, else None. But I'm not 100% sure. IMO it's always good to ACK.
And further, should the ACK maybe be an intermediate step before Finished??

@bobozaur @Patrik-Stas
keeping everyone in the loop; A few questions that were raised in the meeting today;
Result types:
should our result types be;

Result<Holder<NewState>, AriesVcxError>, or
Result<Holder<NewState>, Holder<Failed>>

Consumption of self;
for APIs which transition states AND have the chance of failing (error result), we want the consumer to not lose the original Holder state in the event that an error result occurs. So should the signature be;

take self as a reference: fn xyz(&self,..) -> Result<Holder<NewState>, Error>
consume self, but error type contains a tuple of the OG state: fn xyz(self,..) -> Result<Holder<NewState>, (Holder<CurrentState>, Error)>

IMO option 1 is much simpler for consumers BUT allows the consumers to easily have the old and new state simultaneously.
Option 2 is more complicated for consumers,  BUT makes the APIs consistent and makes it harder for consumers to have the old and new state simultaneously.
How to handle Ack...
The 0036 RFC has me believe that sending an ACK is an optional step for Holders:

If the issuer wants an acknowledgement that the issued credential was accepted, this message must be decorated with ~please-ack, and it is then best practice for the new Holder to respond with an explicit ack message as described in 0317: Please ACK Decorator.

Given this, do you think it is ok to have the ACK message be apart of the Finished state? i.e.
impl Holder<Finished> {
    fn get_ack_message(&self) -> Option<AckMessage>
}

where it is None if please_ack was not decorated.

Result types:
should our result types be;
* `Result<Holder<NewState>, AriesVcxError>`, or

* `Result<Holder<NewState>, Holder<Failed>>`

2nd option.

Consumption of self;
for APIs which transition states AND have the chance of failing (error result), we want the consumer to not lose the original Holder state in the event that an error result occurs. So should the signature be;
* take self as a reference: `fn xyz(&self,..) -> Result<Holder<NewState>, Error>`

* consume self, but error type contains a tuple of the OG state: `fn xyz(self,..) -> Result<Holder<NewState>, (Holder<CurrentState>, Error)>`

IMO option 1 is much simpler for consumers BUT allows the consumers to easily have the old and new state simultaneously.
Option 2 is more complicated for consumers, BUT makes the APIs consistent and makes it harder for consumers to have the old and new state simultaneously.

This is also a matter of performance. Taking selfby reference means you have to clone it to transition, since you need to create a new type when you change the state.
Option 2 is the way to go, in case the transition can indeed fail.

How to handle Ack...
The 0036 RFC has me believe that sending an ACK is an optional step for Holders:

If the issuer wants an acknowledgement that the issued credential was accepted, this message must be decorated with ~please-ack, and it is then best practice for the new Holder to respond with an explicit ack message as described in 0317: Please ACK Decorator.

Given this, do you think it is ok to have the ACK message be apart of the Finished state? i.e.
impl Holder<Finished> {
    fn get_ack_message(&self) -> Option<AckMessage>
}

where it is None if please_ack was not decorated.

I think that the transition to Finished should give the consumer all they need to proceed, which meams either providing the message directly or embedding it into the state.
Embedding it follows the "message generated state" philosophy we want to adopt, so I'd go with that.

George:
Result types:
should our result types be;
* `Result<Holder<NewState>, AriesVcxError>`, or

* `Result<Holder<NewState>, Holder<Failed>>`

Bogdan:
2nd option.

I was considering 1st option, as that would let the consumer "retry", but assuming we are minimizing IO (at least not sending external messages, although there might be IO with wallet), then consumers shouldn't retry to yield different results and there would be little value doing so.
So I agree with 2nd option as well. That means we'll need to somewhat update this paragraph
https://github.com/hyperledger/aries-vcx/blob/main/aries_vcx/docs/guidelines.md#deterministic-transitions
While still deterministic, the paragraph stated that we either move to NEXT state or keep the old state - eg. assumed the option 1 Result<Holder<NewState>, AriesVcxError>

George:
Consumption of self;
for APIs which transition states AND have the chance of failing (error result), we want the consumer to not lose the original Holder state in the event that an error result occurs. So should the signature be;
* take self as a reference: `fn xyz(&self,..) -> Result<Holder<NewState>, Error>`

* consume self, but error type contains a tuple of the OG state: `fn xyz(self,..) -> Result<Holder<NewState>, (Holder<CurrentState>, Error)>`

IMO option 1 is much simpler for consumers BUT allows the consumers to easily have the old and new state simultaneously.
Option 2 is more complicated for consumers, BUT makes the APIs consistent and makes it harder for consumers to have the old and new state simultaneously.
Bogdan:
This is also a matter of performance. Taking selfby reference means you have to clone it to transition, since you need to create a new type when you change the state.

Option 2 is the way to go, in case the transition can indeed fail.
I suppose you could potentially be consistent with both approaches, but with approach 1 you always end up with 2 state machines. With approach 2, you by default only have 1 - making maintaining pre-transition version opt-in by coping it prior to executing the transaction. So I'd go with

fn xyz(self,..) -> Result<Holder<NewState>, (Holder<CurrentState>, Error)>

or even just
fn xyz(self,..) -> Result<Holder<NewState>, Holder<Failed>>

George:
How to handle Ack...
The 0036 RFC has me believe that sending an ACK is an optional step for Holders:

If the issuer wants an acknowledgement that the issued credential was accepted, this message must be decorated with ~please-ack, and it is then best practice for the new Holder to respond with an explicit ack message as described in 0317: Please ACK Decorator.

Given this, do you think it is ok to have the ACK message be apart of the Finished state? i.e.
impl Holder<Finished> {
    fn get_ack_message(&self) -> Option<AckMessage>
}

where it is None if please_ack was not decorated.
Bogdan:

I think that the transition to Finished should give the consumer all they need to proceed, which meams either providing the message directly or embedding it into the state.
Embedding it follows the "message generated state" philosophy we want to adopt, so I'd go with that.

Makes me think that there can be more state outcomes than just "SuccessState" and "FailState" for some transition.
As I was thinking here, at first I drafted this:
enum HolderFinishedState {
   Finished(Finished)
   FinishedNotAcked(FinishedNotAcked)
}

impl Holder<RequestPrepared> {
    fn receive_credential(&self, cred: CredMsg) -> Result<Holder<HolderFinishedState>, Holder<Failed>>
}

impl Holder<FinishedNotAcked> {
    fn mark_acked(&self, cred: CredMsg) -> Holder<Finished>
}

So receive_credential would returns new state conditionally based on content of the received message. If the credential message contains please-ack request, then we would return Holder<FinishedNotAcked>. If not, we would return Holder<Finished>.
But this is kind of complicating things as now  the T in Holder<T> is some enum, which wraps 2 states - so why not just flatten in out to something like "HolderResultReceiveCredential" aka our "context specific" custom Result type.
impl Holder<RequestPrepared> {
    fn receive_credential(self, cred: CredMsg) -> HolderReceiveCredentialResult<Holder<Finished>, Holder<FinishedNotAcked>, Holder<Failed>>
}

So then the consuming code would have to much all of the N variants
let new_sm: Holder<Finished> = match sm.receive_credential(cred_msg) {
  HolderReceiveCredentialResult::Finished(sm_finished) -> {
     info("nice, we didn't even need to try to ack!");
     sm_finished
   },
  HolderReceiveCredentialResult::FinishedNotAcked(sm_not_acked) -> {
    let ack_msg = sm_not_acked.get_ack_message()
    if let Err(error) = send_msg(ack_msg) {
       error!("Failed to ack and we are giving up ¯\_(ツ)_/¯")
    }
   let sm_finished: Holder<Finished> = sm_not_acked.mark_acked(&self);
   sm_finished
  },
  HolderReceiveCredentialResult::Failed(sm) -> {
    unwrap!("omg!")
  }
}

Obviously the disadvantage is that we lose ability to use ? to propagate up the error result variant wrapping Holder<Failed>
So this is actually broader suggestion about how we could express varios states a transition can lead to. Not sure if this is good idea, just brainstorming :-)
Also this is even more in violation of forementioned guideline we agreed on https://github.com/hyperledger/aries-vcx/blob/main/aries_vcx/docs/guidelines.md#deterministic-transitions but given the user would be nudged to handle all of the possible conditions, maybe it's not bad?

Hmm. Yeah, it's quite awkward to deal with this final ACK state..
Just brainstorming and building on Patrik's suggestions;
what about this:
enum HolderReceiveCredentialNextState {
     Finished(Holder<Finished>),
     AckPrepared(Holder<AckPrepared>)
}

impl Holder<RequestPrepared> {
     fn receive_credential(self, cred) -> Result<HolderReceiveCredentialNextState, (Holder<CurrentState>, Holder<Failed>)>
}

impl Holder<AckPrepared> {
    fn get_ack_message(&self) -> AckMessage

    // consumer manually calls this after they are confident they have transmitted the ack message? - a bit awkward.
    fn transition_to_finished(self) -> Holder<Finished>
}

E.g. the flow might look as followings:
// API signature
async fn xyz(self,..) -> Result<Holder<NewState>, (Holder<CurrentState>, Holder<FailedState>)>

// consumer calls:
let res = holder.xyz(..).await;

if let Err((holder, failed_holder)) = res {
    if failed_holder.get_error() == AriesVcxError::LedgerTimeout {
         // consumer considers ledger timeouts as a soft error and wants to retry
         holder.xyz(...).await;
         // ... etc
    }
}

Why would this ever happen? Maybe I'm missing something but I don't understand why Holder<NewState> ,Holder<CurrentState> and Holder<Failed> would all be possible in the same method. If the operation is meant to be retried, then don't return Holder<Failed> but rather some Error.
Holder<Failed> is meant to be an unrecoverable final state. If something returns it, it's not meant to be retried.

Right, my bad, I misunderstood your original push for using Holder instead of AriesVcxError. I understand the motive now.
In that case, all the error results that can happen in this Holder typestate pattern (prepare request, and receive credential) are not HARD errors.
As they both could be a result of a ledger timeout or network error when fetching the cred def or rev reg states. Which could be considered a soft error in some use cases.
So they should effectively be;
fn xyz(self,..) -> Result<Holder<NewState>, (Holder<currentState>, AriesVcxError)>

@gmulhearn Did you merge main into this branch? Or did you rebase? I'm asking because if you merged that messes up history and it would be best to rebase instead.
I had a very hard time rebasing a while ago after aries_vcx_core crate was split from aries_vcx and you did that regression fix. Idk if you merged back then too, but it would make sense as the problems I had were coming from the commits being different (because of the merge). Ignore this comment if it's unrelated 😅

@bobozaur i generally merge main into branches rather than rebase. But I don't see how that can cause issues since the PR should squash the branch when merging into master (unless we aren't squashing PRs?)
I generally prefer merge main into branch rather than rebase, as I'm not the biggest fan on re-writing history of a branch. But I know that's a bit of a political debate...
If it is an issue to merge instead of rebase on PR branches, could you explain why pls

Yeah, after a closer look it seems to be completely unrelated. I don't really care that much, honestly, tho I personally prefer rebasing. My main concern was whether that caused the issue I had then, but it doesn't look like it.
I remember I rebased from the branch in #798 as I was integrating the messages crate and once that got merged into main and I wanted to rebase again all hell broke loose. Just noticed that you merged main in here and wondered if this was the issue if that's what you usually do.
But, like you said, you didn't even rebase/merge then. And as I'm writing this I realized what the issue was: I rebased from that PR branch, then the commits got squashed when merged into main, so the rebase was painful because of that. Should've reset to main and cherry pick, but oh well... The more you know 😅 .
In conclusion, do whatever you're comfortable with and ignore these comments 😄 .

Ahh yep, thanks for explaining, makes sense. Another personal reason I don't rebase is because I never seem to get it right... haha. Maybe I should learn how to do it properly.


✨ Provide older versions (3.9) of Python for VSCode on AP

Describe the feature request.
The AP currently has Python 3.12 installed by default on the Visual Studio Code 1.3.0 install. There are many changes between this version of Python to the versions of Python on the JupyterLab installs (3.9) which means working between JupyerLabs and VS Code becomes a bit of a headache because of the severe changes to things like dsutils in Python 3.12.
Whenever I try and QA code using VS Code, I am battling version issues if the other person has written the code in JupyterLab for example.
I would be able to install Python 3.9 on VS Code using pyenv, but an easier solution to provide interoperability and encourage changeover from JupyerLabs to VS Code would be to provide the same Python version on each. For now, that would make sense to put a Python 3.9 version on VS Code I think.
Describe the context.
No response
Value / Purpose
Prevents wasted time in installing through pyenv and breakages between using JupyterLab and VS Code.
User Types
Analysts

deadsnakes PPA does not provide 3.9 for Ubuntu 24.04

Ubuntu 24.04 (noble) Python3.7 - Python3.11, Python3.13
...
Why some packages aren't built:

Note: for focal, older python versions require libssl<1.1 so they are not currently built
Note: for jammy and noble, older python versions requre libssl<3 so they are not currently built
If you need these, reach out to asottile to set up a private ppa

Source: https://launchpad.net/~deadsnakes/+archive/ubuntu/ppa

@mshodge I was able to install and use Python 3.9 with Conda on the latest (1.4.0) release of VSCode on the Analytical Platform
$ conda create -n py39 python=3.9

$ conda init

Open a new shell and then
$ conda activate py39

$ python --version
Python 3.9.19

Source: https://conda.io/projects/conda/en/latest/user-guide/tasks/manage-python.html#installing-a-different-version-of-python

deadsnakes PPA does not provide 3.9 for Ubuntu 24.04

Ubuntu 24.04 (noble) Python3.7 - Python3.11, Python3.13
...
Why some packages aren't built:

Note: for focal, older python versions require libssl<1.1 so they are not currently built
Note: for jammy and noble, older python versions requre libssl<3 so they are not currently built
If you need these, reach out to asottile to set up a private ppa

Source: https://launchpad.net/~deadsnakes/+archive/ubuntu/ppa
However, noble is listed here https://github.com/deadsnakes/python3.9

Does that not mean they do provide 3.9 though and - Note: ... Python3.12 (noble) are not provided by deadsnakes as upstream ubuntu provides those packages.? See screenshot below too.

@mshodge I was able to install and use Python 3.9 with Conda on the latest (1.4.0) release of VSCode on the Analytical Platform
$ conda create -n py39 python=3.9

$ conda init

Open a new shell and then
$ conda activate py39

$ python --version
Python 3.9.19

Source: https://conda.io/projects/conda/en/latest/user-guide/tasks/manage-python.html#installing-a-different-version-of-python

I'll give this a try, thanks. But many of the Data Scientists tend not to use conda environments from what I see and instead use venv or poetry. I'll see how the two interact with one another.

Looks promising so far. But led me down a rabbit hole of possibly tidying up the User Guidance and adding some sections: https://asdslack.slack.com/archives/G2T9SMUVD/p1717749226385309

Does that not mean they do provide 3.9 though and - Note: ... Python3.12 (noble) are not provided by deadsnakes as upstream ubuntu provides those packages.? See screenshot below too.

@mshodge they do, just doesn't look like its reflected in the PPA site, i've got a working PoC here https://github.com/ministryofjustice/analytical-platform-visual-studio-code/pull/85

Looks promising so far. But led me down a rabbit hole of possibly tidying up the User Guidance and adding some sections: https://asdslack.slack.com/archives/G2T9SMUVD/p1717749226385309

I can't open that link, but let me know how you get on, if it doesn't work, I can publish a release candidate of my PoC next week and you can test that. There is a lot going on with pyenv and I'd need to test how it would function on our cloud development environment platform.

Thanks @jacobwoffenden. It's a slack post to the DEDs channel, copied here for ease:

Hi all. I thought I’d like to get your thoughts on some potential rearranging and adding of new sections of the AP User Guide now that Visual Studio Code is on the AP. Lots of the topics under JupyterLab are relevant to Visual Studio Code as well, so it seems weird to have them under JupyterLab only. Also have included a placeholder for Poetry as a Python package management system, and also a page on Moving from Visual Studio Code from JupyterLab which will have some of the benefits of VSCode but also some of the things to be aware (such as each having different versions of Python and how to resolve this).
Whilst we are tidying it up. What are the things missing from the AP Guidance under R/Python/RStudio/JupyterLab/Visual Studio Code? (edited)

I'll keep testing the conda approach, it worked with a simple venv so I will check with poetry too today. If it works, it might just be simpler to promote that as the solution to cut down on the VSCode versions.

@mshodge has confirmed that we can close this FR, as the built in Conda can provide this functionality


Performance issues on Ember 1.13.0

I've run into performance issues with credit card & phone number components under Ember 1.13.0; they are simply to slow to even be usable. All other ember-inputmask components work fine.
{{credit-card-input unmaskedValue=foo1 cardType=bar1 separator=" "}}
{{phone-number-input unmaskedValue=foo2 extensions=false}}

Thinking out loud, there might be an architectural problem on how data gets passed around, an observer going rogue or some sort of memory leak. I'm going to take a look at it. In the meantime I'm leaving this issue to keep track of things.

I'm experiencing the performance issue as well with the {{phone-number-input}}

I'm having performance issues also with phone-number-input and on ie 11 it moves focus to the end making it unusable

Hey, sorry it's taken me so long to get back to this.
To be honest I've dropped this addon from our stack specifically because of issues like this. Even on it's own, jquery.inputmask is a very large plugin and it's pretty slow.
If anyone wants to take a stab at improving that perf, by all means. In the meantime, I'm going to be looking for a more general and high performance masking library that we can build a better set of masking tools in general on top of. I'll continue to merge PRs, and if anyone wants to take over this project I'll gladly transfer ownership, but I'm not going to be actively developing it any longer.

@pzuraq is there another addon that you're using in place of this one?

I haven't actually had the need to use input masking in an app for some time, but if I needed it again I'd probably go with a thin wrapper around @estelle's excellent input masking library library. The approach is much more minimal, essentially pure javascript which makes it very small and much more performant.

Thanks @pzuraq! That does look really nice. By thin wrapper do you mean something like a component that wraps ember's input helper in it's template and instantiates input masking's InputMask object in it's js via the didInsertElement() method?

Yep, exactly. This way the component can focus on the UI part and InputMask can focus on the masking logic independently


Crash when opening Full PDF Viewer in Example Project

Hi! I've just build and run your Example Program with Xcode 8.0. It's OK for the first time after I click 'Show Full PDF Viewer', but the program crashed in a second when I back to ExampleViewController and click it again. The problem wasn't gone until I deleted the program in Simulator. And the problem comes again after I run the Example and click 'Show Full PDF Viewer'. Can you help to have a look please?
Very thanks!
'annotations = aDecoder.decodeObject(forKey: "annotations") as! [PDFAnnotation]' in PDFAnnotationStore

@kfindfly please take a look at the latest build. I was not able to reproduce your issue, but I'm assuming that the bug is caused by the inference that the decoded objected will contain annotations. It has been changed to an optional parameter. If the issue is still present, please give me more details as to the problem.

Closing this issue, reopen if it is still present.


fix(conditions) prints validation error when found

This PR is a fix for a bug I found while working on #289 .
It ensures that the validation error messages are printed to the end user when a condition failed to be validated.
Test
With a manifest having the following condition (missing spec.file)
# ...
conditions:
  checkUsername:
    kind: yaml
    spec:
      key: github.username

With updatecli 0.16.0, you got:
# ...
CONDITIONS:
===========

checkUsername
-------------
✗ condition not met, skipping pipeline

=> result is expected but no message to help the end user.
With this PR, you got:
$ go build -o ./dist/updatecli && ./dist/updatecli diff
# ...
CONDITIONS:
===========

checkUsername
-------------
ERROR: Validation error: the provided manifest configuration had the following validation errors:
Invalid spec for yaml resource: 'file' is empty.

✗ condition not met, skipping pipeline

=> which is clearly an improvement
Additional Information
Tradeoff

Potential improvement

@olblak I've added this to the 0.16.1 as it's clearly worth a patch: sounds good for you?


ML - Verifying ODBC Driver

Verifying ODBC driver 17 before package management. Showing an error message to user with a "learn more" link so user would get more info about how to install ODBC.

Coverage increased (+2.6%) to 40.28% when pulling 549cd7b13ca5a109923c2391ad3ba55bf5be3eb1 on llali/ml_verify_odbc into ee540ac89c9efedddb86310d003d4ed72ee995d4 on main.

looks good to me. @alanrenmsft tagging you also


Build fails for the custom environment with conda

Module: Work with environments in Azure Machine Learning
Lab/Demo: 04
Task: Create and use a custom environment in Notebook
Step: Cell 10 and 11
Description of issue:
Building the custom image with conda takes takes forever and eventually times out with following build_log:
Collecting package metadata (repodata.json): ...working...
done
Solving environment: ...working...

Updating the Python version from 3.7 to 3.11 in step 10 (conda-env.yml) helps.
%%writefile src/conda-env.yml
name: basic-env-cpu
channels:
  - conda-forge
dependencies:
  - python=3.11
  - scikit-learn
  - pandas
  - numpy
  - matplotlib

Another observation is that jobs on compute clusters are being shown as "preparing" for little more than 10 minutes, which is kind of slow. I am not sure if this is normal in Azure.

Yes Custom Environment creation is going on more than 45 minutes still running....

Job is also in Preparing Status since more than 45 minutes


Update jsoniter-scala-core, ... to 2.13.22

Updates

com.github.plokhotnyuk.jsoniter-scala:jsoniter-scala-core
com.github.plokhotnyuk.jsoniter-scala:jsoniter-scala-macros

from 2.13.10 to 2.13.22.
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Adjust future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "com.github.plokhotnyuk.jsoniter-scala" } ]

Or, add this to slow down future updates of this dependency:
dependencyOverrides = [{
  pullRequest = { frequency = "@monthly" },
  dependency = { groupId = "com.github.plokhotnyuk.jsoniter-scala" }
}]

labels: library-update, early-semver-patch, semver-spec-patch, commit-count:1

Superseded by #429.


CIA, 3DS builds

These didn't work properly before so they were removed. If anyone knows how to work with CIA and 3DS building, help would be very appreciated.

Now looking at how other CIA-compatible apps are built it doesn't seem so hard, but what would be a good way to handle the "game" directory?

Yeah, I'm not sure how to handle it.
How lpp-3ds does it is embed's index.lua into lpp-3ds itself and loads it from there.
However, this is only useful for 1-script projects, not a project with a ton of images/sounds/scripts etc.

I was thinking that it could load the game folder from another folder in the root of the SD card that matches the application's name.

Maybe can use the lua syntax "dofile"? I didn't try it but maybe can work.

Is it possible to implement the use of .love files? So the cia could load the .love file.

Added in 605502889f2d2cf883b435df10f106569db993dc to much drama, but I'll leave this issue open till I do a proper write-up about it in the readme.


No instructions for Pop_OS (Ubuntu Impish/Hirsuite/Deb-11)

Expected Behavior
To be able to install "Cloudflare-warp" either as deb package for Deb-11 or an Ubuntu one
Actual Behavior
Says it installs but can't find nor launch application like on Windows.
Section that requires update
N/A
Additional information
Perhaps a method for installing latest Debian version on Ubuntu could help those using current impish version? and those using Pop_OS

https://github.com/cloudflare/cloudflare-docs/blob/production/products/<IP_ADDRESS>/src/content/other-ways-to-use-<IP_ADDRESS>/index.md
Could be a suitable place?

You can download the package directly from https://pkg.cloudflareclient.com/packages/cloudflare-warp for now. Note that we will be moving to a new distribution mechanism in the future and will plan to properly fix this then.

I know it's closed now but could make one version that works on both of Ubuntu's yearly releases (Since there are only two) and then call it Ubuntu-Based installer to avoid confusion?
Although it doesn't seem perfectly compatible with 21.10 after using Warp+DoH with Malware protection as of writing :/


Crash on 10k subscription

good morning, is it expected to have vernemq to crash if I subscribe with a single client to 10k topics where we have retained messages? Running a single instance for testing. on AWS EC2 m4.4xlarge (16 cores, 64GB RAM)
We pre-published 10k retained messages. Each message is having 2KB size.
From 1 client we're trying to subscribe to all topics one-by-one and get back the retained message on that topic.
Messages on console are:
14:41:16.948 [info] monitor large_heap <0.418.0> [{name,vmq_reg_trie},{initial_call,{vmq_reg_trie,init,1}},{almost_current_function,{gen_server2,drain,1}},{message_queue_len,0}] [{old_heap_block_size,0},{heap_block_size,708538797},{mbuf_size,0},{stack_size,9},{old_heap_size,0},{heap_size,575993646}]
14:41:24.481 [info] monitor large_heap <0.418.0> [{name,vmq_reg_trie},{initial_call,{vmq_reg_trie,init,1}},{almost_current_function,{gen_server2,drain,1}},{message_queue_len,1}] [{old_heap_block_size,0},{heap_block_size,850246556},{mbuf_size,0},{stack_size,9},{old_heap_size,0},{heap_size,691201326}]
14:41:33.385 [info] monitor large_heap <0.418.0> [{name,vmq_reg_trie},{initial_call,{vmq_reg_trie,init,1}},{almost_current_function,{erlang,'--',2}},{message_queue_len,0}] [{old_heap_block_size,0},{heap_block_size,1020295867},{mbuf_size,0},{stack_size,9},{old_heap_size,0},{heap_size,833788078}]
14:41:49.164 [info] alarm_handler: {set,{process_memory_high_watermark,<0.418.0>}}
14:42:00.447 [info] monitor large_heap <0.418.0> [{name,vmq_reg_trie},{initial_call,{vmq_reg_trie,init,1}},{almost_current_function,{gen_server2,in,3}},{message_queue_len,103}] [{old_heap_block_size,0},{heap_block_size,1224355040},{mbuf_size,0},{stack_size,9},{old_heap_size,0},{heap_size,997140718}]
14:42:12.848 [info] monitor large_heap <0.418.0> [{name,vmq_reg_trie},{initial_call,{vmq_reg_trie,init,1}},{almost_current_function,{gen_server2,in,3}},{message_queue_len,42}] [{old_heap_block_size,0},{heap_block_size,1469226048},{mbuf_size,0},{stack_size,9},{old_heap_size,0},{heap_size,1268465440}]
14:42:12.920 [info] monitor large_heap <0.418.0> [{name,vmq_reg_trie},{initial_call,{vmq_reg_trie,init,1}},{almost_current_function,{gen_server2,drain,1}},{message_queue_len,0}] [{old_heap_block_size,0},{heap_block_size,1763071257},{mbuf_size,0},{stack_size,9},{old_heap_size,0},{heap_size,1488689562}]
14:42:29.393 [info] monitor large_heap <0.418.0> [{name,vmq_reg_trie},{initial_call,{vmq_reg_trie,init,1}},{almost_current_function,{gen_server2,drain,1}},{message_queue_len,1}] [{old_heap_block_size,0},{heap_block_size,2115685508},{mbuf_size,0},{stack_size,9},{old_heap_size,0},{heap_size,1747049966}]
14:42:51.305 [info] monitor large_heap <0.418.0> [{name,vmq_reg_trie},{initial_call,{vmq_reg_trie,init,1}},{almost_current_function,{gen_server2,drain,1}},{message_queue_len,1}] [{old_heap_block_size,0},{heap_block_size,2538822609},{mbuf_size,0},{stack_size,9},{old_heap_size,0},{heap_size,2078701614}]
14:43:19.133 [info] monitor large_heap <0.418.0> [{name,vmq_reg_trie},{initial_call,{vmq_reg_trie,init,1}},{almost_current_function,{gen_server2,drain,1}},{message_queue_len,0}] [{old_heap_block_size,0},{heap_block_size,3046587130},{mbuf_size,0},{stack_size,9},{old_heap_size,0},{heap_size,2483521262}]
14:43:51.709 [info] monitor large_heap <0.418.0> [{name,vmq_reg_trie},{initial_call,{vmq_reg_trie,init,1}},{almost_current_function,{gen_server2,drain,1}},{message_queue_len,1}] [{old_heap_block_size,0},{heap_block_size,3655904556},{mbuf_size,0},{stack_size,9},{old_heap_size,0},{heap_size,2989064430}]
14:44:28.264 [info] monitor large_heap <0.418.0> [{name,vmq_reg_trie},{initial_call,{vmq_reg_trie,init,1}},{almost_current_function,{gen_server2,drain,1}},{message_queue_len,1}] [{old_heap_block_size,0},{heap_block_size,4387085467},{mbuf_size,0},{stack_size,9},{old_heap_size,0},{heap_size,3606739886}]
[os_mon] cpu supervisor port (cpu_sup): Erlang has closed
[os_mon] memory supervisor port (memsup): Erlang has closed
/usr/sbin/start_vernemq: line 194:    86 Killed                  /vernemq/bin/vernemq console -noshell -noinput $@

Environment

VerneMQ Version: 1.12.3
OS: alpine3.14 (docker)
Erlang/OTP version (if building from source):  <IP_ADDRESS>-alpine
VerneMQ configuration (vernemq.conf) or the changes from the default

max_inflight_messages = 0
max_online_messages = -1
max_offline_messages = -1
max_message_size = 0
listener.max_connections = infinity
listener.nr_of_acceptors = 10

Cluster size/standalone: 1

Expected behaviour

get back all the messages

Actual behaviour

out of 10k messages we get back approx 100/200 (1/2 hundred) the rest is never arriving and vernemq crashes.

publish and subscribe are two separate things. First we publish as retain and leave there. So now we have vernemq with 10k retained messages.
[vernemq]/vernemq# bin/vmq-admin metrics show | grep retain
gauge.retain_memory = 5292992
gauge.retain_messages = 10002

Then I start the 'stress test' to have 1 connection and subscribe to the (known) 10k topic where I have the retained messages.
Tool is written in golang, it's starting 10k goroutines which does the subscribe near same time.
I checked both crash.log and error.log nothing is there.
where should erl_crash.dump be? I can't find such a file, but maybe that's just because correct directory is not bind mounted in the container.

We share full Verne code too... :)
Anyway, I cannot promise more than a quick look at script. Might see something, or not.

:point_right: Thank you for supporting VerneMQ: https://github.com/sponsors/vernemq
:point_right: Using the binary VerneMQ packages commercially (.deb/.rpm/Docker) requires a paid subscription.


added Anilist and OP.GG

I added Anilist which is a popular tracking site for animes similar to MyAnimeList and OP.GG which is the most used stats tracking site for League of Legends.
Although im Unsure about the OP.GG logo since it appears to be somewhat small and not really readable.

Sorry for the late reply on this, I've been out of town for the US holiday weekend! I think the Anilist one is fine, but I would make a few changes to the OP.GG for readability.
I see one of the primary colors they're using throughout the core op.gg site is #5383E8. So you could make that the background color for the button. Also looking at their Twitter and mobile apps for iOS and Android, it looks like they use a stacked version of the logo for better readability for iconography. So it could look more like this:

I created the OP.GG SVG in a stacked format for you based on their other implementations.

No worries, I don't mind at all! Hope you had an enjoyable weekend 😊
Thank you for providing the logo! I was looking on their desktop site for one like that, but did not find a white one.
I was already using the #5383E8 but for the font, I swapped it now to be the background colour and included the updated logo you provided.

Thank you! 🙂


Implementation_of_Error_Metric_for_Rest_Actions

Description
Complete implementation of issue: #4401
Issues Resolved
#4401
Check List

[x] New functionality includes testing.

[x] All tests pass

[x] New functionality has been documented.

[x] New functionality has javadoc added

[x] Commits are signed per the DCO using --signoff
[ ] Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Could you help with the sample request and response structure for this?

Closing this PR , new PR raised by Prabhat #5334


businessDiff does not consider holidays

Hello everyone, I tried to use the businessDiff function but it seems that it's not considering holidays that have been specified in the locale... has anyone faced the same issue?

@mnapoli76 me too! I copied-and-pasted the example exactly but no dice; same result as the normal businessDiff calculation

@mnapoli76 after digging in some more, there's nothing in the businessDiff method that considers holidays. I'm working on a PR right now to (hopefully) solve that

My last PR fixed that thing. Merged by now.

Sorry for the delay, it is solved in the new version, update the package please.


[BUG] State of Storage Class doesn't update when Disk(s) that represent disk selector are removed from nodes

Describe the bug
State of "Active" on a custom storage class never changes, when there are no disks that match host & disk selector combination.
To Reproduce
Steps to reproduce the behavior:

Go to add disk to node
Tag disk and host on node
Create custom storage class
Wait for storage class to become active
Remove disk from node
Run lsblk on node to double verify - ensure not mounted
Check Custom Storage Class state, will show "Active"

Expected behavior
Long polling/receiving a message from the host, like listening to ds/harvester-node-disk-manager when:
time="2023-07-05T21:23:22Z" level=info msg="Prepare to stop provisioning device 989754e4e66edadfd3390974a1aba3f8 to node harvester-node-0"
time="2023-07-05T21:23:22Z" level=info msg="Update existing block device status 989754e4e66edadfd3390974a1aba3f8"

Takes place, audit all existing "Active" storage classes, and update state of storage class to shift from Active -> InActive?
Maybe it would change from "Active" to "Not Ready", like Volumes do, if for instance:

Create custom storage class
Remove disk that matches disk selector
Create Volume, the volume will stick in "Not Ready" - but the State of the Storage class always stays in "Active"

Environment

Harvester ISO version: v1.2-head ->  Harvester Version: v1.2-7a8ac2c2-head
Underlying Infrastructure: qemu/kvm

Hi @irishgordo.
StorageClass should not have any status; it is more like the volume description.
You could refer to https://kubernetes.io/docs/concepts/storage/storage-classes/
The UI status depends on our framework (maybe wrangler, not sure).
The situation you mentioned might be another point we need to consider to remind users, we could discuss how to do it.
cc @guangbochen, @bk201

@irishgordo We have some offline discussion; here is a summary:

The active state means nothing; it doesn't represent whether we can create a volume with it. It's pure Kubernetes design.
@Vicente-Cheng will help add a doc to address this limitation. One suggestion is on the storage class page (https://docs.harvesterhci.io/dev/advanced/storageclass).

We might need storage-specific support (from Longhorn) and custom APIs to achieve what the issue description wants. Please help create another enhancement ticket to track it. Thank you.

@bk201  thanks for that info :smile:  - I'll go ahead and close this out and I've opened up:
https://github.com/harvester/harvester/issues/4320
To track that enhancement :+1: :smile:


feat: add optionalDependencies into consideration via flag

add support for field optionalDependencies of package.json

hey @jase88 ... thank you so much for the PR.
I'm actually refactoring the entire lib and I'll add this option as well.
Probably tomorrow I'll release the v2. You can follow it here: https://github.com/raulfdm/pin-dependencies-checker/pull/9


[FIX] This closes #13

Builds should now work with space in path.

Thank you, I will merge tmr

@nopslide what happened to merging tmr?

sorry for late merge, just tested for making sure it works


Defer to (good) compiler the task of computing ln10

xsimd is C++11 and hexadecimal floating point are C++17

See https://godbolt.org/z/5sxq7Ye5s

this works for alpine linux as well with gcc 12 https://gitlab.alpinelinux.org/a16bitsysop/aports/-/jobs/1007269 (includes rebuild of apache-arrow)

thanks @a16bitsysop for the review!


Ci/release binary action

1. Does this PR affect any open issues?(Y/N) and add issue references (e.g. "fix #123", "re #123".):

[ ] N
[X] Y
related to #34

2. What is the scope of this PR (e.g. component or file name):

3. Provide a description of the PR(e.g. more details, effects, motivations or doc link):

[ ] Affects user behaviors
[ ] Contains syntax changes
[ ] Contains variable changes
[ ] Contains experimental features
[ ] Performance regression: Consumes more CPU
[ ] Performance regression: Consumes more Memory
[ ] Other

4. Are there any breaking changes?(Y/N) and describe the breaking changes(e.g. more details, motivations or doc link):

[ ] N
[ ] Y

5. Are there test cases for these changes?(Y/N) select and add more details, references or doc links:

[ ] Unit test
[ ] Integration test
[ ] Benchmark (add benchmark stats below)
[ ] Manual test (add detailed scripts or steps below)
[ ] Other

6. Release note

Please refer to Release Notes Language Style Guide to write a quality release note.
None

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 66.782%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
1938

Relevant Lines:
2902

💛  - Coveralls


Investigate ell

Looks really nice: https://docs.ell.so/core_concepts/ell_complex.html
Not sure if a good fit for gptme though, but looks very similar in approach.

More intrigued by https://github.com/ErikBjare/gptme/issues/233

For what it's worth, I quite like ell, as it stores version history of your prompts so you can later review and replay them. Can help with prompt optimisation. Is multi modal too.

Might try it later, but no point keeping this open.
@lingster Thanks for sharing your experience with it :)


assertions not supported for parameter cdf in constriction.stream.model.CustomModel

I tried to implement a custom entropy model that is only defined on positive numbers.
To make sure only positive numbers are encoded I used an assert statement to check for non-negativity.
To make my point I use a lognormal distribution in the following snippet.
def cdf(x, mu, sigma):
     assert x >= 0
     return stats.lognorm.cdf(x, mu, sigma)

def inverse_cdf(q, mu, sigma):
     return stats.lognorm.ppf(q, mu, sigma)

mus = np.random.randn(100)
sigmas = np.random.randn(100)**2 +1
dummy_entropy_model = constriction.stream.model.CustomModel(cdf ,inverse_cdf, -10, 10)

message = (np.random.randn(100)**2).round().astype(np.int32)

coder = constriction.stream.stack.AnsCoder()
coder.encode_reverse(message, dummy_entropy_model, mus, sigmas)
decode = coder.decode(dummy_entropy_model, mus, sigmas)

thread '<unnamed>' panicked at 'TODO: PyErr { type: <class 'AssertionError'>, value: AssertionError(), traceback: Some(<traceback object at 0x7f22f0efac80>) }', src/pybindings/stream/model/internals.rs:380:14
PanicException: TODO: PyErr { type: , value: AssertionError(), traceback: Some() }
This problem is of course solved by deleting assertions from cdf.
A more permanent solution might be a change to the documentation or the error message.

Thanks for reporting! Please note that there's a bug in the following line:
dummy_entropy_model = constriction.stream.model.CustomModel(cdf ,inverse_cdf, -10, 10)

The argument -10 states that the CustomModel should support symbols from the integer range {-10, -9, ..., 10}. Therefore, constriction correctly assumes that it should be allowed to evaluate the cdf at negative values.
Changing the argument from -10 to 0 fixes the encoding part, but now decoding fails. This is still technically correct behavior as CustomModel does not guarantee to only probe the cdf within the range of supported symbols (being allowed to probe outside the range of supported symbols slightly simplifies the code that calculates the exact inverse cdf). But looking at this example, I now agree that this can cause confusion, and I'll look into fixing it. Also, I agree that the error messages should be better.
Side Node
Assertions generally can't be used to tell an interpreter or compiler not to do something. They are a tool for (i) documenting invariants for human readers of the source code and for (ii) bailing in case a program or library contains a logic error or is used incorrectly.
This is nothing specific about constriction (or even about python). For example, the code below throws an AssertionError because it calls divide with argument b=0 at some point even though the assert statement inside divide states otherwise. This is because an assert statement never enforces the asserted condition. It just checks if the condition is met, and interrupts control flow if it isn't.
def divide(a, b):
    assert b != 0
    return a / b

print([divide(1, x) for x in range(-10, 11)])

Preventing callers from calling a function unless certain conditions are met cannot be achieved with assertions inside the function body. It would instead require either a more expressive type system that makes invalid function arguments inexpressible, or a more elaborate mechanism such as design by contract.


Cypress/E2E: Fix autocomplete for users both DB only and with Elasticsearch

Summary

created common tests for Elasticsearch and DB, and soon for Bleve.
fix how the users for search are generated. Instead of suffixing with reverse timestamp which were still causing flakiness in test, use random letter prefix so that the user/s returned from search will always be unique for particular test.
Created 12 new test cases and added test keys for Autocomplete/Database: MM-T4071 to MM-T4082

Note: Once this is merged, I will submit a follow up PR for other autocomplete tests.
Ticket Link
failed on daily Cypress run
Screenshots

Release Note
NONE

Will try to auto merge this PR once all tests and checks are passing. This might take up to an hour.


Cannot install PyACVD for python3

Here is what I tried:
pip3 install PyACVD
Here is what I get:
Collecting PyACVD
  Using cached PyACVD-0.1.1.tar.gz
Building wheels for collected packages: PyACVD
  Running setup.py bdist_wheel for PyACVD ... error
  Complete output from command /Library/Frameworks/Python.framework/Versions/3.5/bin/python3.5 -u -c "import setuptools, tokenize;__file__='/private/var/folders/v8/vmb7nyj95slb3f__hc5r9x5h0000gn/T/pip-build-3a3roe5x/PyACVD/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" bdist_wheel -d /var/folders/v8/vmb7nyj95slb3f__hc5r9x5h0000gn/T/tmplbwuglhfpip-wheel- --python-tag cp35:
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.macosx-10.6-intel-3.5
  creating build/lib.macosx-10.6-intel-3.5/PyACVD
  copying PyACVD/__init__.py -> build/lib.macosx-10.6-intel-3.5/PyACVD
  copying PyACVD/Clustering.py -> build/lib.macosx-10.6-intel-3.5/PyACVD
  copying PyACVD/VTK_Plotting.py -> build/lib.macosx-10.6-intel-3.5/PyACVD
  creating build/lib.macosx-10.6-intel-3.5/PyACVD/Tests
  copying PyACVD/Tests/__init__.py -> build/lib.macosx-10.6-intel-3.5/PyACVD/Tests
  copying PyACVD/Tests/Remesh.py -> build/lib.macosx-10.6-intel-3.5/PyACVD/Tests
  copying PyACVD/Tests/StanfordBunny.ply -> build/lib.macosx-10.6-intel-3.5/PyACVD/Tests
  running build_ext
  building 'PyACVD.Clustering_Cython' extension
  creating build/temp.macosx-10.6-intel-3.5
  creating build/temp.macosx-10.6-intel-3.5/PyACVD
  creating build/temp.macosx-10.6-intel-3.5/PyACVD/cython
  ccache clang -Qunused-arguments -fcolor-diagnostics -fno-strict-aliasing -Wsign-compare -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -arch i386 -arch x86_64 -g -I/Library/Frameworks/Python.framework/Versions/3.5/include/python3.5m -I/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/numpy/core/include -c PyACVD/cython/Clustering_Cython.cpp -o build/temp.macosx-10.6-intel-3.5/PyACVD/cython/Clustering_Cython.o
  clang: error: no such file or directory: 'PyACVD/cython/Clustering_Cython.cpp'
  clang: error: no input files
  error: command 'ccache' failed with exit status 1

  ----------------------------------------
  Failed building wheel for PyACVD
  Running setup.py clean for PyACVD
Failed to build PyACVD
Installing collected packages: PyACVD
  Running setup.py install for PyACVD ... error
    Complete output from command /Library/Frameworks/Python.framework/Versions/3.5/bin/python3.5 -u -c "import setuptools, tokenize;__file__='/private/var/folders/v8/vmb7nyj95slb3f__hc5r9x5h0000gn/T/pip-build-3a3roe5x/PyACVD/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/v8/vmb7nyj95slb3f__hc5r9x5h0000gn/T/pip-7jmf0xkh-record/install-record.txt --single-version-externally-managed --compile:
    running install
    running build
    running build_py
    creating build
    creating build/lib.macosx-10.6-intel-3.5
    creating build/lib.macosx-10.6-intel-3.5/PyACVD
    copying PyACVD/__init__.py -> build/lib.macosx-10.6-intel-3.5/PyACVD
    copying PyACVD/Clustering.py -> build/lib.macosx-10.6-intel-3.5/PyACVD
    copying PyACVD/VTK_Plotting.py -> build/lib.macosx-10.6-intel-3.5/PyACVD
    creating build/lib.macosx-10.6-intel-3.5/PyACVD/Tests
    copying PyACVD/Tests/__init__.py -> build/lib.macosx-10.6-intel-3.5/PyACVD/Tests
    copying PyACVD/Tests/Remesh.py -> build/lib.macosx-10.6-intel-3.5/PyACVD/Tests
    copying PyACVD/Tests/StanfordBunny.ply -> build/lib.macosx-10.6-intel-3.5/PyACVD/Tests
    running build_ext
    building 'PyACVD.Clustering_Cython' extension
    creating build/temp.macosx-10.6-intel-3.5
    creating build/temp.macosx-10.6-intel-3.5/PyACVD
    creating build/temp.macosx-10.6-intel-3.5/PyACVD/cython
    ccache clang -Qunused-arguments -fcolor-diagnostics -fno-strict-aliasing -Wsign-compare -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -arch i386 -arch x86_64 -g -I/Library/Frameworks/Python.framework/Versions/3.5/include/python3.5m -I/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/numpy/core/include -c PyACVD/cython/Clustering_Cython.cpp -o build/temp.macosx-10.6-intel-3.5/PyACVD/cython/Clustering_Cython.o
    clang: error: no such file or directory: 'PyACVD/cython/Clustering_Cython.cpp'
    clang: error: no input files
    error: command 'ccache' failed with exit status 1

    ----------------------------------------
Command "/Library/Frameworks/Python.framework/Versions/3.5/bin/python3.5 -u -c "import setuptools, tokenize;__file__='/private/var/folders/v8/vmb7nyj95slb3f__hc5r9x5h0000gn/T/pip-build-3a3roe5x/PyACVD/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/v8/vmb7nyj95slb3f__hc5r9x5h0000gn/T/pip-7jmf0xkh-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /private/var/folders/v8/vmb7nyj95slb3f__hc5r9x5h0000gn/T/pip-build-3a3roe5x/PyACVD/

I am running python 3.5.2, and pip 8.1.2, on MacOS 10.11.6

Module builds fine on my end using Python 3.5.2, 9.0.1, and Ubuntu 16.04 and for Windows 7.  Could you try cloning the module and then running
pip install -e . --user

Let me know if Clustering_Cython.cpp is generated in the cython folder.  I'll have to get a local copy of MacOS to try to debug this if we can't figure it out.

Here is what I did:
git clone https://github.com/akaszynski/PyACVD.git
cd PyACVD
pip3 install Cython # I was missing this module
pip3 install -e . --user

The file Clustering_Cython.cpp is generated, and the installation succeeds.
Obtaining file:///Users/user/workspace/dev/misc/PyACVD
Installing collected packages: PyACVD
  Running setup.py develop for PyACVD
Successfully installed PyACVD-0.1.1

However, if I try to import the module in ipython3, I get a similar error as reported in #2 :
In [1]: import PyACVD
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-1-39a5f59cbb52> in <module>()
----> 1 import PyACVD

/Users/user/workspace/dev/misc/PyACVD/PyACVD/__init__.py in <module>()
----> 1 from . import Clustering_Cython
      2 from PyACVD import Clustering

ImportError: dlopen(/Users/user/workspace/dev/misc/PyACVD/PyACVD/Clustering_Cython.cpython-35m-darwin.so, 2): Symbol not found: __ZTINSt8ios_base7failureE
  Referenced from: /Users/user/workspace/dev/misc/PyACVD/PyACVD/Clustering_Cython.cpython-35m-darwin.so
  Expected in: flat namespace
 in /Users/user/workspace/dev/misc/PyACVD/PyACVD/Clustering_Cython.cpython-35m-darwin.so

The last problem can be solved as described in issue #2. Compilation works, loading the module I could not verify because vtk for python3 is not available on my machine.

May I ask, how you got VTK running for python3? pip3 install vtk does not work for me on MacOS. Did you build vtk from source?


Password field returned despite being omitted in$formatJson()

Hi,
I'm having a problem omitting the password field of my Member model. I use the bellow method (from the docs) to omit the password field, whenever the model get's queried, but sometimes it still returns it:
$formatJson(json: any) {
  json = super.$formatJson(json);
  delete json.password;
  //test
  console.log({json});
  return json;
}

for example, here I'm asking for one user, here the password is removed just fine from the response:
export const one = async (req, res, next) => {
  try {
    const member = await Member.query().findById(req.body.id);
    res.status(200).send(member);
  } catch (e) {
    next(e);
  }
};

but here, in my email.ctrl where I'm sending a notification about new registrations, the password gets included into the response:
const notifyAdmin = async (id: number) => {
  const memberTest = await Member.query().findById(id);
  console.log({memberTest}) // <--- has the field: `password`:(

  new Email().send({msg: 'hello...'})
}

It's the same model, the same query, same process and there's not even a concurrency issue, what could cause the password field to be present despite omitting it in $formatJson() ?
An even simpler reproduction of the issue is querying the model right in my server.ts - file, as soon as the app starts via:
...
server.listen(process.env.PORT || 3000, () => {
  console.log(`Server listening on port ${process.env.PORT || 3000}`);
});

(async function() {
  const memberTest = await Member.query().findById(1);
  console.log({memberTest})
})();

Log output:
{
  memberTest: Member {
    id: 1,
    full_name: 'xxx xxx',
    email<EMAIL_ADDRESS>    password: '$2a$10$ci2lQqsvhNi3kWbxvlo42u4Mc/r86AdMJUal48Kxxxxxxxxx',
    role_id: 1,
    created_at: 2019-01-15T10:42:52.000Z,
    updated_at: 2022-01-14T10:12:03.923Z
  }
}

You need to override the $parseDatabaseJson method. $formatJson is called when the model instance is serialized using JSON.strigify (when it's sent to the fronted).

@koskimas Thanks! I got it now. Haven't noticed the difference between the two, sorry!


Integrated editor converts line endings to LF

Gitea version (or commit ref): 1.10.0
Git version: ?
Operating system: Windows
Database (use [x]):

[ ] PostgreSQL
[ ] MySQL
[X] MSSQL
[ ] SQLite

Can you reproduce the bug at https://try.gitea.io:

[X] Yes (provide example URL)
[ ] No
[ ] Not relevant

Log gist:

Description
We are using * -text in .gitattributes to avoid any changes when checking in or out from git. This works very well and we do not intend to change this behavior.
We thus have many files with Window line endings committed. When we now use the integrated Gitea editor, all line endings will be replaced with LF.
See this commit:
https://try.gitea.io/Luke/Repo2/commit/a3ef603d463101239513288eb026ed55869e2490
Expected behavior

Do not touch lines that were not changed.
and use same line ending for new line as in rest of file.

Im having the same problem. I implemented a statistics screen on my local server and this problem keeps making my data unreliable.

After 1 minute of digging, I found this part of the code on "routers\repo\editor.go"
.... 	if _, err := repofiles.CreateOrUpdateRepoFile(ctx.Repo.Repository, ctx.User, &repofiles.UpdateRepoFileOptions{ 		LastCommitID: form.LastCommit, 		OldBranch:    ctx.Repo.BranchName, 		NewBranch:    branchName, 		FromTreePath: ctx.Repo.TreePath, 		TreePath:     form.TreePath, 		Message:      message, 		Content:      strings.ReplaceAll(form.Content, "\r", ""), 		IsNewFile:    isNewFile, 	}); err != nil { 		...
if I replace that strings.ReplaceAll(form.Content, "\r", "") with just form.Content, the line-endings don't change.
However, I dont know for what propose that line was added, so proceed with caution.

You are not supposed to have files in git repositories with non Unix file endings. So with this code we are essentially enforcing the git attribute:
* text=auto

Presumably you have set core.autocrlf=false for this project?

You are not supposed to have files in git repositories with non Unix file endings.

Where does this requirement come from? IMHO, a version control system should process files as they are, no matter what kind of line endings or encoding is used. And actually is this what git and Gitea do just fine. It seems that it is just the editor which rewrites the line endings.
Are there any unwanted side effects by the change that @emngaiden proposed? Can the change be applied?

The web editor should follow settings on .gitattributes about the line endings when insert a new line.

@lunny:

The web editor should follow settings on .gitattributes about the line endings when insert a new line.

That would be a potentially useful improvement however, I suspect that this might actually be a problem due to the use of core.autocrlf=false, core.eol=native in the local git config without a .gitattributes.
@LukeOwlclaw:

Where does this requirement come from?

It's not a requirement but the recommendation comes from Git documentation itself. I'm not going to waste a lot of time looking for this but a text file is essentially defined as being LF ended. Lots of hacks have been added to handle Windows' particular proclivities on this matter, and similarly in a SVN and CVS.
In Git's case, some of complexities go back to the early days of GitForWindows and its decision to introduce the core.autocrlf .
CRLF and LF conversion is extremely complex and every decision is wrong.

IMHO, a version control system should process files as they are, no matter what kind of line endings or encoding is used. And actually is this what git and Gitea do just fine. It seems that it is just the editor which rewrites the line endings.

That's fine for binary files however, if you want a file to be interpreted as text you need really need to think carefully about how you store encodings and line-endings. Git made a decision that it much prefers its text files to be UTF-8 with LF endings and a lot of git code makes that presumption.
SVN had a much more complicated system which was better in regards to this kind of thing but it was actually equally infuriating to use. Git's mechanisms are a lot more slapdash and poorly handled. Gitea doesn't even attempt to handle them as they're so rarely used that it's not even worth attempting to do it, we just attempt to detect the encodings ourselves.

Are there any unwanted side effects by the change that @emngaiden proposed? Can the change be applied?

Yes there are multiple side-effects due to this proposed change. There is no simple solution regarding file encodings or line endings - except everyone should use LF ending and UTF-8 encoding.

Amyway, I was trying to explain why we did the line-ending normalization in the first place.
It would be helpful to note that this CRLF issues are likely responsible for the weird conflict issues that some people suffer on merge. We have to do a line-ending normalization because editing on a windows browser will change the line endings to \r\n and editing on linux the other way round.
I'm not suggesting that we can't change this, but the question is going to be how to determine what the line-ending should be and whether anyone is going to be interested doing it.
It would be useful to know what the values of the config settings are locally and on the server:

core.autocrlf
core.eol

and if you have a .gitattributes.

The .gitattributes in the repository I created for this issue contains: * -text
There are no local settings as only Gitea's editor is being used.
An compromise might be to change editor.go so that it detects the type of line endings. Then it can enforce the same type of line endings for the whole file that was sent by the browser. Should be fairly easy to implement, shouldn't it?


Feature Request:  Change Language for Auto Commit Message

Currently, the language for auto Commit message is controllered by github.copilot.chat.localeOverride(v.1.86), But sometimes, users want to use language A for a project and use another language B for another project.
So, I think there could be something be done just like that in markdown:

For example, when users want the Commit message be in another language, this can be done by a similar trailing button like that in the image.

Couple ideas based on what's already in place that you could use:

github.copilot.chat.localeOverride does allow for workspace specific overrides. For that 1 or few repos where you want it to respond in language B, set language B in those repos
If you have many repos where you want A and many repos where you want B, use VS Code's Profile support to set which language override you want to use. Use Profile A with override A and Provide B with override B.

Both of these options give you the control you're looking for.

Thanks for your reply.  But after testing, I found that github.copilot.chat.localeOverride would be changed to be consistent across different workspaces.

@dbsxdbsx you can configure this setting for a specific project by using workspace settings https://code.visualstudio.com/docs/editor/workspaces#_singlefolder-workspace-settings

Thanks.


Experiment on Web Audio API and libraries

Added the minimal functionality needed. The code has been tested separately but haven't been integrated yet.


added Taiwan phone number pattern

added Taiwan phone number pattern and work perfectly

@TroyCode Nice! Can you add tests for this regex? Thanks

done!

@TroyCode Thanks!


Student App Recipes

I think the last recipe we are waiting on for 1.0 is student app. I seem to recall @michaelwebbjisc saying that this was already done and just needed adding to the 1.0 branch?

Just bumping this and hoping it lands in the right persons inbox, last call micheal/lee promised to upload them to 1.0 branch?

Bump received! will pickup next week

I've created a new feature branch feature-study-goal and added:
some generic mobile app templates (eg viewing something on the screen is pretty much the same as viewing a vle page)
https://github.com/jiscdev/xapi/blob/feature-study-goal/recipes/mobile-app.md
some template items for a tracking app (which is what study goal is)
https://github.com/jiscdev/xapi/blob/feature-study-goal/recipes/tracker-app.md
There are a bit rough at the moment but should show what I mean.

I think the second one is a bit more different from the first one than you've indicated at the moment. The tracker one seems to me to be 'setting a target', and so on, rather than 'viewing a tracker app'. It's an active process rather than simply viewing, because the Actor is changing the data not just looking at a resource.
I suspect that a profitable approach might be to look at a 'targets' recipe, which might apply to a mobile app or VLE (cf. Bedford's Moodle stuff has this, as does the tracker app). This could handle all the student-related aspects of targets - setting the target, updating progress vs target, achieving it, modifying it, achieving late, abandoning it, failing to reach it, and so on. There would be good coherence here in that the Actor could always be the student (though I guess there might be a wrinkle of staff setting a target?), there would be a limited range of 'target-related' verbs (abandoned, commented, completed, created (not sure where this comes from), failed, progressed, viewed), context would be in relation to the particular VLE or other relevant platform, and Object would be the specifics of the target.
Does this help?

These have been ssplit but are becoming setting a target app to be introduced in 1.0.2


Include relative path for candidate class files

In order to derive the fully-qualified names of test classes, we now
include the relative path of each class file within its output
directory.

OK, I've already triggered ReadyForMerge build for you.

@bot-gradle test and merge


Different results between int and String

If I call md5 with an integer parameter it return a different value from md5 called with a string parameter.
e.g.
md5(1003) = 64e0407ffc70c04a366526a7065cbd05
md5('1003') = aa68c75c4a77c87f97fb686b2f068676.
Is it a normal behavior?

Yes, this is "normal". Don't use this library. It will happily help you shoot yourself in the foot.
MD5 is an algorithm that is defined on binary input data. This library just does whatever your runtime will do to make your input binary. So, if your runtime will serialize strings in UTF-16 rather than UTF-8, you'd get different results for the same input string.


[SPARK-37488][CORE] When TaskLocation is HDFSCacheTaskLocation or HostTaskLocation, check if executor is alive on the host

// The online environment is actually hive partition data imported to tidb, the code logic can be simplified as follows
SparkSession testApp = SparkSession.builder()
    .master("local[*]")
    .appName("test app")
    .enableHiveSupport()
    .getOrCreate();
Dataset<Row> dataset = testApp.sql("select * from default.test where dt = '20211129'");
dataset.persist(StorageLevel.MEMORY_AND_DISK());
dataset.count();

I have observed that tasks are permanently pending and reruns can always be reproduced.
Since it is only reproducible online, I use the arthas runtime to see the status of the function entries and returns within the TaskSetManager.
I replaced the real host to avoid revealing company information
https://gist.github.com/guiyanakuang/431584f191645513552a937d16ae8fbd
NODE_LOCAL level, because the persist function is called, the pendingTasks.forHost has a collection of pending tasks, but it points to the machine where the block of partitioned data is located, and since the only resource spark gets is the driver.
Here is the forHost information I got through the arthas hook
    forHost=@HashMap[
        serialVersionUID=@Long[1],
        _loadFactor=@Integer[750],
        table=@HashEntry[][
            @DefaultEntry[(kv: hdfs-loc-1, ArrayBuffer(2, 1))],
            null,
            null,
            null,
            null,
            null,
            null,
            null,
            null,
            @DefaultEntry[(kv: driver-host, ArrayBuffer())],
            null,
            null,
            null,
            null,
            @DefaultEntry[(kv: hdfs-loc-2, ArrayBuffer(2, 1))],
            @DefaultEntry[(kv: hdfs-loc-3, ArrayBuffer(2, 1))],
        ],

When we can only provide driver resources, getAllowedLocalityLevel limits the attempt level to NODE_LOCAL. But in reality, the task cannot run at this level, and the attempt level cannot reach ANY
What changes were proposed in this pull request?
To solve the above mentioned problem, this pr modifies the initialization logic of forHost.
When TaskLocation is HDFSCacheTaskLocation or HostTaskLocation, check if executor is alive on the host.
Why are the changes needed?

In my online environment it may cause the task to pending permanently, this pr aims to fix this issue.
Does this PR introduce any user-facing change?

No
How was this patch tested?

Add unit test

Can one of the admins verify this patch?

Gentle ping @dongjoon-hyun, Can you help review this pr ?    : )

Codecov Report

Merging #34743 (030de1d) into master (0c3c4e2) will decrease coverage by 16.25%.
The diff coverage is 66.66%.

:exclamation: Current head 030de1d differs from pull request most recent head 158fbd6. Consider uploading reports for the commit 158fbd6 to get more accurate results

@@             Coverage Diff             @@
##           master   #34743       +/-   ##
===========================================
- Coverage   90.11%   73.85%   -16.26%
===========================================
  Files         292      127      -165
  Lines       62785    24065    -38720
  Branches     9902     3625     -6277
===========================================
- Hits        56580    17774    -38806
- Misses       4825     5666      +841
+ Partials     1380      625      -755

Flag
Coverage Δ

unittests
73.85% <66.66%> (-16.24%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

python/pyspark/sql/pandas/utils.py
6.89% <0.00%> (-55.18%)
:arrow_down:

python/pyspark/sql/session.py
60.24% <75.00%> (-22.44%)
:arrow_down:

python/pyspark/ml/tests/test_algorithms.py
98.01% <100.00%> (ø)

python/pyspark/sql/pandas/conversion.py
8.54% <0.00%> (-83.63%)
:arrow_down:

python/pyspark/sql/pandas/typehints.py
17.50% <0.00%> (-77.50%)
:arrow_down:

python/pyspark/sql/pandas/functions.py
27.77% <0.00%> (-72.23%)
:arrow_down:

python/pyspark/sql/pandas/types.py
9.67% <0.00%> (-69.04%)
:arrow_down:

python/pyspark/sql/observation.py
26.66% <0.00%> (-68.89%)
:arrow_down:

python/pyspark/sql/pandas/map_ops.py
21.62% <0.00%> (-64.87%)
:arrow_down:

python/pyspark/sql/pandas/group_ops.py
29.31% <0.00%> (-62.07%)
:arrow_down:

... and 196 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0f631b1...158fbd6. Read the comment docs.

Thank you for pinging me, @guiyanakuang .

I'll close this pr because this implementation doesn't solve the issue perfectly, and I don't have a better idea right now. Thanks to @mridulm and @dongjoon-hyun for the review

Thank you for closing, @guiyanakuang .


baobab: Fix missing icons on KDE

Motivation for this change
Missing baobab icons  on KDE.
Things done
Fixed this, generated icons with inkscape from the .svg file in the source.

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

Does KDE not support SVG icons?

From the few KDE apps I've packaged, they all use svgz icons and png for several sizes.

But is there any issue with regular SVGs? I do not like introducing more complexity to the expression unless strictly required.

Also, in case we really need the bitmap icons, they should be generated during build (using either Inkscape, or rsvg-convert), not included into the Nixpkgs repository.

Fixed https://gitlab.gnome.org/GNOME/baobab/-/commit/1ad24c8a3ac853a354e407650d3c5c6056988184


replace settings variables in debugger config #8042

Provide the ability to expose (share) settings from the settings.json file to the launch.json file, using the dollar notation. #8042
settings.json:
"python.pythonPath": "/Library/Frameworks/Python.framework/Versions/3.5/bin/python3"

launch.json:
"pythonPath": "${settings.python.pythonPath}",

@DonJayamanne PR looks good overall - I added comments in the commits directly. Also:

Did you test this both on the folder and no-folder workspace?
Can you please squash all the commits as the first 4 commits in this PR should be removed?
It would be great if this also worked for tasks.json
Is it possible that SettingsVariables just extends from SystemVariables - using that approach we would not need both SettingsVariables and SystemVariables in DebugConfigurationManager and we would get this feature for free in the tasks world?

@isidorn, @weinand

Did you test this both on the folder and no-folder workspace?

I have only tested this with a folder workspace.
Didn't know we can have VS Code with a no-folder workspace. How does that work?

Can you please squash all the commits as the first 4 commits in this PR should be removed?

Will do.

It would be great if this also worked for tasks.json
Is it possible that SettingsVariables just extends from SystemVariables - using that approach we would not need both SettingsVariables and SystemVariables in DebugConfigurationManager and we would get this feature for free in the tasks world?

Great idea. However, wouldn't this render SystemVariables obsolete? From what I can see this is only used in two places, running tasks and debug adapters.
Also it could be a little misleading, if we used this new class SettingsVariables where ever SystemVariables was being used, then one would be mistaken into assuming that only Settings (config) variables were being replaced.
I think this all boils down to the fact that this SettingsVariables (having extended SystemVariables) class will be doing two things:

Replacing System variables (base class functionality)
Replacing Settings (config) variables (overridden method functionality).

If you agree with the above then, how about we proceed with the following approach:

SettingsVaraibles extends SystemVariables (as per the suggestion)
Replace references to SystemVaraibles and SettingsVariables with ISystemVariables
Get ISystemVariables passed in as a constructor argument resolved via DI (resolved using the serviceCollection - no sure what the right terminology is here)

We should not call this 'setting.' but 'config.' to align with other places where we have this

Agreed, for now I'll continue using the word "Setting" and SettingsVariables so there's no confusion.

@DonJayamanne no folder workspace is the purple vscode - the window which you get when you say 'new window'. In that scenario we do not support debugging but just open that window and check for exceptions - since a lot of things are missing in that scenario.
Your proposed approach sounds good to me, can you code it up and then we can provide more comments?
Thanks!

@DonJayamanne pr looks great! There are only two things which should be updated - I commented in the code directly!

@isidorn , please check again, Thanks

@DonJayamanne looks great! Sorry for the slow response, last week I was on vacation -> merging

@DonJayamanne please create a test plan item for this so we can test it next week in the endgame - thanks!

@isidorn, done, please check the test plan (#9559) and let me know how it looks.

@DonJayamanne looks great! I just assigned it to the current milestone (july) so it is easier for the endgame master to find it. I would also mention to try it for tasks.json as well.
My personal preference is to write more concise test plan items so the tester is given more freedom as to how to exactly test the new feature. But that is up to you


How to terminate Native  Call Ui  When Create Make Call

when i  create a make call using this :-
await TwilioVoice.instance.call.place(
to: "+91xxxxxxxx", from: "+91xxxxxxxxxx",);
                              it  show native ui of calling . i want to  navigate own create screen , how i do please tell

Good day @lavishmanghnani2,
In version 0.0.9, this feature was available in Android. The rationale behind using the native system calling UIs is due to the limitations of iOS.
iOS requires any app to utilize their CallKit for any telephonic features, thus integrating into the native "callkit" of Android platforms to ensure no custom dependency and irregularity between platforms.
Feel free to open a feature request, to see the interest in custom Android call screens. With sufficient demand, I'd reintroduce this feature.


Staked FraxBP

Adds support for staked/locked FraxBP
Enables categorisation of Curve staked/locked LPs

Change in values is to be expected, as the base doesn't recognise staked or locked FraxBP, and this branch does.
Tested. Deployed to production.


Non-Highlighted Code + Weird OnClick Positioning

Running Chrome Version 40.0.2214.115 m
Window 7 - 64 Bit
Using Book III and I am clicking on the items.
Please ignore the code, I was changing from coffee to python because it was bugging.

Note, this occurs on all zoom levels, and on different resolutions.

Hmm, can't reproduce; let me know if you see it again, and if you do, check the JS console for any fun error messages in there. Thanks!


Rerun the harmonisation pipeline for studies killed by exitcode 137/130/140

There are 3097 studies that were killed because of the memory/wall time setting. We need to assign a larger memory and longer wall time to run it. I especially need to pay attention to the quality control step.

I generate a list containing all GCSTs that were killed by the error 130 (Command terminated by user),137 (container's memory exceeds the memory limit provided in the pod) or 140 (not given enough memory or time).

In the basic_config, provide more memory, longer time and more times of retry (3 to 5).

only 487/3097 were harmonised successfully and it shows an requirement and we need an integrated table to trace their harmonisation status, when they started harmonisation and the latest failed reason, and achieve a daily update.


Allow update with join

Hi,
Can you do something to allow this kind of requests :
UPDATE TABLEA a JOIN TABLEB b ON a.foo = b.bar SET a.foobar = 'barfoo'
Thanks

This is already possible by overriding the update() query builder:
let qry = squel.update({}, [
        new squel.cls.StringBlock(options, 'UPDATE'),
        new squel.cls.UpdateTableBlock(options),
        new squel.cls.JoinBlock(options),
        new squel.cls.SetFieldBlock(options),
        new squel.cls.WhereBlock(options),
        new squel.cls.OrderByBlock(options),
        new squel.cls.LimitBlock(options),
])

That's perfect, thank you !


Adding Technical PM section to staff page

@mvoong18 did you mean to include all these changes to the deliverable page as well?

Oops sorry! I did not mean to do that. It should be fixed now


[BUG] Delete one memo will delete all memos

Describe the bug
Call DELETE ${endpoint}/api/v1/memo/${id} API will delete all memos.
Database driver: PostgreSQL
Memos version: 0.18.0 via Docker
Steps to reproduce

Create a lot of memos from WebUI;
Query database to ensure they exist;
Delete one memo from WebUI;
All memos are deleted.

Screenshots or additional context
No response

The vacuumMemo was generated SQL DELETE FROM memo WHERE creator_id NOT IN (SELECT id FROM user), which should be DELETE FROM memo WHERE creator_id NOT IN (SELECT id FROM "user").
https://github.com/usememos/memos/blob/ec7992553f6b33227058c7bde59d329f9cfe24de/store/db/postgres/memo.go#L227-L246


boot-server.tsx no longer compiles with TypeScript 2.4 (React + Redux template)

Title
boot-server.tsx no longer compiles with TypeScript 2.4 (React + Redux template)
Functional impact
When webpacking in prod env mode, projects will fail TypeScript compilation. I'm not sure if any of the other templates are affected, but I expect they would be.
Minimal repro steps

dotnet new reactredux
Update the version of typescript to "2.4.0" in package.json
npm install
node node_modules/webpack/bin/webpack.js --env.prod

Expected result
Successful compilation.
Actual result
TS2345: Argument of type '(params: BootFuncParams) => Promise' is not assignable to parameter of type 'BootFunc'.
Type 'Promise' is not assignable to type 'Promise'.
Type 'RenderResult' is not assignable to type 'RenderToStringResult'.
Type 'RedirectResult' is not assignable to type 'RenderToStringResult'.
Property 'html' is missing in type 'RedirectResult'.
Further technical details
I think this can be fixed by updating the BootFunc interface like so:
interface BootFunc {
    (params: BootFuncParams): Promise<RenderToStringResult | RedirectResult>;
}

If you typecast the function passed to createServerRenderer as BootFunc you should compile:
export default createServerRenderer((params => {

    return new Promise<RenderResult>((resolve, reject) => {
...
...
    });
}) as BootFunc);

Yep, that works, thanks. I still hope someone will fix the d.ts, though.

Might give this a looksy 😜 - I don't know how it is for everyone else, but TS 2.4 has made a lot of previously good code bad. Good thing it's meant to 😂

Should probably wait though till 2.4 is out of rc

@DanielRosenwasser and @mhegazy, is this expected to break?

Thanks for reporting this! It's fixed in aspnet-prerendering 2.0.6 which is now published.
Also note that to make React typings work with TypeScript 2.4.0+, you must also update your copy of @types/react to 15.0.29 or later.


Iron Browser

Iron - https://github.com/WhichBrowser/Parser-PHP/issues/159

Coverage remained the same at 100.0% when pulling b60ad2ddb23c358d5e78a06c9ef7144893e6c83d on ayumi-cloud:iron into 4899110cd2f87b01e04ced62dbb9dec541031dee on WhichBrowser:master.

Also needs a closer look. There already appears to be a browser named Iron in the list.

Deleting this pr. Iron already added to this repo and this UA (without a version number) seems incorrect.


Invalid HTTP upgrade 403

I am getting 403 response from backend, using SpringBoot websocket with stomp.

@milanovici did you got any answer for this??


Footer bar update

'bar' has been deprecated; use colors.footer_bar instead.

Just did the update to my own alacritty.yml, moving to footer_bar, and it fixed the deprecation warning.

Duplicate of https://github.com/nordtheme/alacritty/pull/27


Mention todoman as a supported client.

This is a bit of a shameless plug, but I did design todoman with vdirsyncer support in mind, and there doesn't seem to by anything else listed to cover these requirements. 😄
Thanks for vdirsyncer, BTW, it did make it possible for me to finally organize my list of todos.

Neat, thanks!
I'm currently using watdo for this and like it, but others say that it's interface is kinky, so I'd always hoped that somebody would step up and write an alternative (I probably should mention watdo in the docs too).
I've found a minor issue with your current code, it's not writing the .ics-files atomically. I've mentioned this in the docs, but don't really do a good job explaining it ATM. Take a look in vdirsyncer's code to see how I implemented it.

Never came across watdo before. It's definitely worth mentioning as well (people might be interested). Though I think event-oriented CalDav clients should be separated from todo-oriented clients if we start having too many.
Atomicity when writing slipped my mind, but I'll put it high up in my priority list.
Hope you enjoy the interface though. :)

Huh, nice. I was already expecting to have to create a tmp-file, hardlink to it, and then clean up.
Looks like atomic_writer does all the work for me, thanks!

I've now restructured the docs.
I'm currently trying to get todoman running and have run into a few issues. Should I report those on the GitHub project or on your own one?

Opening an issue on the issue tracker is fine. I'll look into them ASAP.


RBF follow-ups

Here are a couple things which didn't warrant holding up the PR but would be nice to have:

Expand the explicit-RBF-signaling sanity check to all conflicting transactions (https://github.com/wizardsardine/liana/pull/816#discussion_r1417618042)
The "feerate too low" error could include the minimum feerate to be more helpful (https://github.com/wizardsardine/liana/pull/816#discussion_r1416268844)

@jp1ac4 feel free to suggest more.

We can also check if any directly conflicting transactions are spending unconfirmed change and include these unconfirmed coins as candidates for the replacement transaction (https://github.com/wizardsardine/liana/issues/826#issuecomment-1853058734).

We should add ancestor info to coin selection candidates when creating a replacement to ensure we pay extra for any parent transactions that have a lower feerate (https://github.com/wizardsardine/liana/pull/873#discussion_r1518611295).

We should support RBF on a recovery transaction. Currently, the RBF candidate coins all have their sequence set to None.

I updated the issue description to include Michael's comments. I also divided it in different task and marked the second one as completed as per @aaroncarlucci's PR #1186 which was merged (thanks!)

We should add ancestor info to coin selection candidates when creating a replacement to ensure we pay extra for any parent transactions that have a lower feerate (#873 (comment)).

See https://github.com/wizardsardine/liana/issues/1255.


Product taxonomies and attributes handling

This release fixes a number of issues around handling of attributes and translations.
In particular:

New translations can now use auto-copy of source language, to help save time translating.
In future a machine translation will be added.
Copy option covers Product Title, Short Description and Long Description.
Also when creating a new product, any missing Product Categories, Tags and Attributes are copied,
to avoid unexpected problems which occur if a translation is saved with missing term translations.

it is now possible to set up the system to allow different types of product attributes
to be synchronised, translated, or independent in each language. The default options will be:

Translation and Synchronization Enabled for Product Attributes
Synchronization off for Custom Product Attributes

In this case choose how to set up your product attributes as follows:

Translated Attribute?  Add in Products\Attributes and turn on Translation in Polylang at:
Languages\Settings\Custom Taxonomies
Synchronised Attribute? [eg same value in all languages, eg product code, numeric properties]
Add in Products\Attributes and leave Translation turned off in Polyang.
Different value in each language? add directly to Product as a Custom Product Attribute

Enh: synchronisation for Custom Product Attributes and Global Product Attributes can now be
turned on and off independently in
Settings\WooPoly, Metas List, Attributes Metas, Custom Product Attributes.
The fields locker is unlocked for the attribute types which are not synchronized.
Fix: Global Product Attributes can now be individually configured in Polylang:
When Settings\WooPoly Translation attributes is checked then attributes appear in Polylang:
Languages\Settings\Custom Taxonomies lists the individual taxonomies
Previously all attributes translation were forced on: now they can be selectively turned on and off.  This means that there is no longer any need to create dummy translations for untranslateable values such as reference codes and numeric fields.  Fixes #127.

CHANGE: new Product Attributes are no longer automatically enabled for translation,
After creating new Attribute, enable Translation in Polylang if needed by checking:
Languages\Settings\Custom Taxonomies

Enh: Missing Term Translations are now added by default. Fixes #72
Applies to Products\Categories, Products\Tags, Products\Attributes
Previously missing term translations caused
Fixes: #148 WooCommerce3 product_visibility is now a taxonomy not a meta item
Fixes: #153 Fields locker doesn't correctly lock Product Attributes of type Select
Fixes: #147 When adding new variations, tool should also add the new variation to other languages

@Jon007 Thanks for huge effort, amazing work, clean code  🍺