Active learning refactor

Looks to solve many of the current open issues:
Major package refactor for #19, adds structs for readability of data. Sets up Energy, Force, LocalDescriptor, and ForceDescriptor structs so that we can make use of multiple dispatch on methods appropriate for each type of data. Comes up with Configuration level data struct (to hold information for a particular configuraiton of atoms) and a DataSet type (which is a vector of Configurations). Most of the methods in PotentialLearning operate on the DataSet type. This allows users to load all of their data into a DataSet, separate training and testing datasets, and either do dimension reduction, dpp active sampling, or fitting based on the DataSet struct.  Ultimately this makes the code more reusable and general.
Adds documentation for #8 - each of the new methods has appropriate documentation. Unit testing is also in progress for each of the new methods.
Closes #23 - we have moved NNBasisPotential to InteratomicBasisPotentials.jl. There is some basic support for fitting neural network (and general loss functions) within PotentialLearning. More testing is needed.
Closes #24 - With the movement of NNBasisPotential to InteratomicBasisPotentials.jl, I have fixed the issue with gradients of gradients.
Closes #22 and #21  - Have added DPP subset selector (as well as random subset selector). Unit testing is sufficient.

Thank you very much, I will review it carefully early next week.

Busy week. I will look at it next week.

A key aspect of neural ace training is to be able to use BFGS. Flux does not provide this optimizer. To combine the NN model provided by Flux with BFGS provided by Optimization.jl, I used Flux.destructure. This function extracts the parameters of the NN model so they can interact nicely with the Optimization.jl interface. I am not sure how to train a Flux model with BFGS with this new code. In case it is not possible, we would need to re-incorporate this feature, I can address this after the merge. Perhaps another approach is to use FluxOptTools.jl.

I would strongly recommend against using BFGS for neural ace training, let alone have it be a built in feature into PotentialLearning.jl. We can provide further scaffolding to allow for users to define their own optimizer, particularly in learn.jl

Regarding the LearningProblem, should the datasets and/or loss functions (e.g. a sum of the squared differences of non-linear functions, its derivatives, etc) commonly used in NN potentials be transformed in any way to define the LearningProblem under the BI approach? E.g. logprob = something that depends on the loss

The data is assumed to be in the form of a DataSet struct and logprob is assumed to be a function that takes params::Vector{T} as input arguments and optimizes over those arguments. I think this is a fairly flexible approach to be taken that allows the users to transform their problem as needed.

In the LearningProblem function, the gradient is computed based on f. Should this function be logprob?

Yes, will be fixed in the next push.

Closes #24 - With the movement of NNBasisPotential to InteratomicBasisPotentials.jl, I have fixed the issue with gradients of gradients.

What would you modify in the following snippet to make it work using reverse mode over reverse mode and the latest version of Flux?

using Flux
# Range
xrange = 0:π/99:π
xs = [ Float32.([x1, x2]) for x1 in xrange for x2 in xrange]
# NN model: multi-layer perceptron (MLP)
mlp = Chain(Dense(2,4, Flux.σ),Dense(4,1))
ps_mlp = Flux.params(mlp)
E_mlp(x) = sum(mlp(x))
dE_mlp(E_mlp, x) = sum(gradient(E_mlp, x))
# Computing "nested" gradient
gradient(()->sum(sum.(dE_mlp.([E_mlp], xs))), ps_mlp)

I suppose something like this works:
using Flux
# Range
xrange = 0:π/99:π
xs = [ Float32.([x1, x2]) for x1 in xrange for x2 in xrange]
# NN model: multi-layer perceptron (MLP)
mlp = Chain(Dense(2,4, Flux.σ),Dense(4,1))
ps_mlp = Flux.params(mlp)
E(x) = sum(mlp(x))
dE(m) = x -> gradient(x -> sum(m(x)), x)[1]
gradient(()->sum(dE(mlp)(xs[1])), ps_mlp)

I would not close Accelerate learning by automatically reducing the size of the training dataset. #21, the idea here is to show how much training can be accelerated for different potentials and atomic systems using subsampling. The code will be based on an improved version of this approach: https://github.com/argonne-lcf/active-learning-md/blob/master/workflow/activesample.py  There is already a student assigned to this project.

The linked code using HDBSCAN, which itself has some limitations vis-a-vis DPP subsampling. In any case, an implementation of HDBSCAN is already present in PotentialLearning.jl. Perhaps this can be made more robust, but that could be another issue.

This PR merged the code of Aug 8 version of main, there are new commits that are not currently merged in this version, in particular related to load_datasets function and metrics.

The load_datasets function should be deprecated in favor of the new methods for dataset input and construction. The structures in this PR are better structured and more robust.

Should the workflows you uploaded here(https://github.com/cesmix-mit/AtomisticComposableWorkflows/tree/main/workflows) be moved to PotentialLearning.jl examples folder? (After this merge I want to move the stuff about nace/ace/snap, hyper-par opt, AL, and docs to this repo. Before that I need to adapt the code to the new interfaces.)

Not directly, I am adding all of the workflows to the /example folder, but in such a way that it is adapted to the current PR.

A key aspect of neural ace training is to be able to use BFGS. Flux does not provide this optimizer. To combine the NN model provided by Flux with BFGS provided by Optimization.jl, I used Flux.destructure. This function extracts the parameters of the NN model so they can interact nicely with the Optimization.jl interface. I am not sure how to train a Flux model with BFGS with this new code. In case it is not possible, we would need to re-incorporate this feature, I can address this after the merge. Perhaps another approach is to use FluxOptTools.jl.

I would strongly recommend against using BFGS for neural ace training, let alone have it be a built in feature into PotentialLearning.jl. We can provide further scaffolding to allow for users to define their own optimizer, particularly in learn.jl

I am open to better options that can exceed the accuracy and performance of linear ACE. I think PotentialLearning.jl should provide built in features that allow users to leverage neural ACE (or non-linear ACE) advantages over linear ACE. Why BFGS here? Although (at least in my experience) this solver could not handle large NN models (several hundred parameters) and large datasets, these limitations can be addressed by using small NNs, subsampling of the original dataset, and minibatches during training. BFGS showed very fast convergence for small MLP ACE models (e.g. 225 parameters: a couple of dense layers with 8 nodes per layer) trained using medium size datasets (e.g. 57800 data points: energies and force components), something I could not get using other Flux solvers, like ADAM, which showed a slow convergence rate (and did not achieve the accuracy I need in the time I am willing to wait). Furthermore, larger NN models can be addressed by using parallel ensembles. It is this rapid convergence that allows for better accuracy (in different training and testing metrics) compared to linear ACE. BFGS is also used in NeuralPDE 1, 2, alone or combined with ADAM (btw, they are using Lux now) to train relatively small MLPs. You can find the codes I have been working on here and here. I can provide more details if you need them. Suggestions are welcome.

Regarding the LearningProblem, should the datasets and/or loss functions (e.g. a sum of the squared differences of non-linear functions, its derivatives, etc) commonly used in NN potentials be transformed in any way to define the LearningProblem under the BI approach? E.g. logprob = something that depends on the loss

The data is assumed to be in the form of a DataSet struct and logprob is assumed to be a function that takes params::Vector{T} as input arguments and optimizes over those arguments. I think this is a fairly flexible approach to be taken that allows the users to transform their problem as needed.

What I mean is whether mathematical transformations based on the Bayesian approach need to be performed on the data or loss functions before training. In other words, can any loss function commonly used to train NN potentials be considered a logprob?

In the LearningProblem function, the gradient is computed based on f. Should this function be logprob?

Yes, will be fixed in the next push.

👍

I suppose something like this works:
...dE(m) = x -> gradient(x -> sum(m(x)), x)[1]
gradient(()->sum(dE(mlp)(xs[1])), ps_mlp)

Good, based on these lines I put together an example closer to actual neural potential training. This code in particular consumes a substantial amount of memory, so I would like to give it another shot. Months ago something similar happened with the reverse mode over forward mode approach, it worked but very slowly. Suggestions are welcome.
using Flux

# Range
xrange = 0:π/99:π
xs = [ Float32.([x1, x2]) for x1 in xrange for x2 in xrange]

# Target function: E
E_analytic(x) = sin(x[1]) * cos(x[2])

# Analytical gradient of E
dE_analytic(x) = [cos(x[1]) * cos(x[2]), -sin(x[1]) * sin(x[2])]

# NN model
mlp = Chain(Dense(2,4, Flux.σ),Dense(4,1))
ps_mlp = Flux.params(mlp)
E(x) = sum(mlp(x))
dE(mlp, x) = sum(gradient(x -> sum(mlp(x)), x))

# Loss
loss(x, y) = Flux.Losses.mse(x, y)

# Training
epochs = 10; opt = Flux.Adam(0.1)
for _ in 1:epochs
    g = gradient(()->loss(reduce(vcat, dE.([mlp],xs)),
                          reduce(vcat, dE_analytic.(xs))), ps_mlp)
    Flux.Optimise.update!(opt, ps_mlp, g)
    l = loss(reduce(vcat, dE.([mlp],xs)),
             reduce(vcat, dE_analytic.(xs)))
    println("loss:", l)
    # GC.gc()
end

I would not close Accelerate learning by automatically reducing the size of the training dataset. #21, the idea here is to show how much training can be accelerated for different potentials and atomic systems using subsampling. The code will be based on an improved version of this approach: https://github.com/argonne-lcf/active-learning-md/blob/master/workflow/activesample.py  There is already a student assigned to this project.

The linked code using HDBSCAN, which itself has some limitations vis-a-vis DPP subsampling. In any case, an implementation of HDBSCAN is already present in PotentialLearning.jl. Perhaps this can be made more robust, but that could be another issue.

I think the information about the progress related to HDBSCAN should be reported in #21. Then the issue could be closed if another issue with a follow-up is open. The new issue could have a more specific title that better represents the original description. A comparison between the two methods could be a nice publication. First the parallel Julia version of the method based on HDBSCAN (and/or other clustering algorithms) has to be implemented.

This PR merged the code of Aug 8 version of main, there are new commits that are not currently merged in this version, in particular related to load_datasets function and metrics.

The load_datasets function should be deprecated in favor of the new methods for dataset input and construction. The structures in this PR are better structured and more robust.

I think for the moment it is ok to deprecate load_datasets since it is not integrated to the new interfaces.

Should the workflows you uploaded here(https://github.com/cesmix-mit/AtomisticComposableWorkflows/tree/main/workflows) be moved to PotentialLearning.jl examples folder? (After this merge I want to move the stuff about nace/ace/snap, hyper-par opt, AL, and docs to this repo. Before that I need to adapt the code to the new interfaces.)

Not directly, I am adding all of the workflows to the /example folder, but in such a way that it is adapted to the current PR.

I agree. Just to make sure we are not duplicating efforts, I am currently working with the integration of the NACE, ACE, and SNAP folders in AtomisticComposableWorflows repo to the new interfaces.
New observations:

In fit_sodium_dft.jl,
     - "examples/Sodium/data/liquify_sodium_dftk_calculations.jld" seems to be missing
     - Should YAML(u"eV", u"Å") be something like YAML(:Na, u"eV", u"Å")
In the LinearProblem
    -   Should this line: true,  compute_features(ds, GlobalSum()), get_values.(get_energy.(ds))
        be replaced by something like this: true, compute_feature.(get_local_descriptors.(ds_train_e), [GlobalSum()]), get_values.

I am open to better options that can exceed the accuracy and performance of linear ACE. I think PotentialLearning.jl should provide built in features that allow users to leverage neural ACE (or non-linear ACE) advantages over linear ACE.

To what extent? Right now PotentialLearning.jl implements a very simple training routine that is based on Flux (we can completely divest from Flux but I think that would be a separate issue). If the user wants a more bespoke routine, like I imagine they would most of the time, then they can implement the training themselves (especially if there is batch training and so forth). I believe that PotentialLearning.jl should mostly help with providing structure in the data and model.
The debate about BFGS is moot and PotentialLearning.jl already provides enough flexibility for the user to tailor their own training method to their own needs.

What I mean is whether mathematical transformations based on the Bayesian approach need to be performed on the data or loss functions before training. In other words, can any loss function commonly used to train NN potentials be considered a logprob?

This is somewhat philosophical, but essentially yes.
I don't believe the rest of the discussion needs to hold up this pull request. Any changes to HDBSCAN can be made in future PRs and edits to the structure of NNBasis functions can occur in InteratomicBasisPotentials.jl .

New observations:

In fit_sodium_dft.jl,
     - "examples/Sodium/data/liquify_sodium_dftk_calculations.jld" seems to be missing
     - Should YAML(u"eV", u"Å") be something like YAML(:Na, u"eV", u"Å")

Yes, this error is fixed in the upcoming push.

In the LinearProblem
    -   Should this line: true,  compute_features(ds, GlobalSum()), get_values.(get_energy.(ds))
        be replaced by something like this: true, compute_feature.(get_local_descriptors.(ds_train_e), [GlobalSum()]), get_values.

If you look at the  definition of the function, you will find that it wraps the relevant code:
function compute_features(ds::DataSet, f::Feature; dt = LocalDescriptors)
    compute_feature.(ds, (f,); dt = dt)
end

I am open to better options that can exceed the accuracy and performance of linear ACE. I think PotentialLearning.jl should provide built in features that allow users to leverage neural ACE (or non-linear ACE) advantages over linear ACE.

To what extent? Right now PotentialLearning.jl implements a very simple training routine that is based on Flux (we can completely divest from Flux but I think that would be a separate issue). If the user wants a more bespoke routine, like I imagine they would most of the time, then they can implement the training themselves (especially if there is batch training and so forth). I believe that PotentialLearning.jl should mostly help with providing structure in the data and model.

I agree that structuring the data and model is a fundamental part of this package. I also agree with the approach of providing basic training and leaving the door open to extending this functionality. At some point we will have to decide what extensions will be added to the package. I believe that new methods and Julia implementations born in CESMIX, such as new subsampling algorithms or new ML potentials, should live in the packages developed in this project even if it requires adding some extra code. In other words, I would like the Julia packages developed for CESMIX to shine for their interfaces and the workflows they enable but also for the novel methods implemented in them. However, I think the most practical thing to do now is to merge this PR and address this point in the future.

The debate about BFGS is moot and PotentialLearning.jl already provides enough flexibility for the user to tailor their own training method to their own needs.

Addressed above. Please understand that I started writing about the use of this optimizer to address the comment about its integration into this package.

What I mean is whether mathematical transformations based on the Bayesian approach need to be performed on the data or loss functions before training. In other words, can any loss function commonly used to train NN potentials be considered a logprob?

This is somewhat philosophical, but essentially yes.

Okay. Actually my concern was practical rather than philosophical: whether it was necessary to transform the training data or loss function before using the new interfaces to comply with the Bayesian approach.

I don't believe the rest of the discussion needs to hold up this pull request. Any changes to HDBSCAN can be made in future PRs and edits to the structure of NNBasis functions can occur in InteratomicBasisPotentials.jl .

Good.

New observations:
In fit_sodium_dft.jl,
- "examples/Sodium/data/liquify_sodium_dftk_calculations.jld" seems to be missing

Should YAML(u"eV", u"Å") be something like YAML(:Na, u"eV", u"Å")

Yes, this error is fixed in the upcoming push.

Good.

In the LinearProblem

Should this line: true,  compute_features(ds, GlobalSum()), get_values.(get_energy.(ds))  be replaced by something like this: true, compute_feature.(get_local_descriptors.(ds_train_e), [GlobalSum()]), get_values.

If you look at the definition of the function, you will find that it wraps the relevant code:
function compute_features(ds::DataSet, f::Feature; dt = LocalDescriptors)
compute_feature.(ds, (f,); dt = dt)
end

I will check this again after the merge.

@dallasfoster if it's ok with you, I can merge the pull request.

You have to finish your review first.


Add Contributin Guidelines

Add Contributin Guidelines that describes aspects like:

[x]  Using Issue Tracker
[x] Bug Reports
[x] Contributing
[x] License

#26


Specify subscription API

Most EL clients implement the subscription API documented here: https://geth.ethereum.org/docs/interacting-with-geth/rpc/pubsub. These are widely adopted but not yet included in the specs. It's easy to specify eth_subscribe and eth_unsubscribe. I'm unsure how to specify the concrete methods (newHeads, newPendingTransactions, etc.) as they are passed as parameters to eth_subscribe. Plus when we push data, we're effectively sending an eth_subscription request from server to the client, and the data for that request depends on the concrete method.
After the current state is done I'd like to propose the following modifications:

Adding an optional parameter to newPendingTransactions named fullTx. It defaults to false. If true the server returns the full tx object instead of the hash. This is already merged in geth.
Adding an optional parameter to newHeads named verbosity. Default value of 0 returns only the hash. If value is 1, we return the complete head object. If value is 2, we return the complete block object. If value is 3 we return the block object + receipts.
Add optional parameter to newPendingTransactions named filter. The server uses the filter to narrow down the list of txes to return.

Open-RPC cannot express the semantics of the eth_subscribe. It is one method that can have a different set of params based on the event type. E.g.:

If type is logs next param can be a filter { address: .., topics: ..}
The other types don't have at the moment further parameters.

The same thing happens in the notification requests, i.e. eth_subscription. Each event returns a different object. Because the parameters of this request are addressed by key, we could define a schema with keys for all possible event types. But that's less than ideal.
There exists an issue https://github.com/open-rpc/spec/issues/381 which if fixed would unblock this item.


How to get available buttons for the current turn?

Hi, I am implementing this package to a web game and having an issue with available button for the current turn.
Is there any way to get the available buttons (actions) after use whoTurnsIsIt method?
Or must I predict myself based on the round and game condition?
Thank you in advance.

On the round, you should be able to fetch these from src/Game/Round.php and use ->actions(). You can then do a filter based on the user

Thanks!


Failing test: ln_collab_close

Probably because of our tendency to skip CI due to #355, we have introduced a regression which causes the ln_collab_close test to fail, as reported by @klochowicz in #715.
I was able to reproduce this by running
cargo test -p ln-dlc-node -- tests::just_in_time_channel::channel_close::ln_collab_close --exact --ignored

which runs into
thread 'tests::just_in_time_channel::channel_close::ln_collab_close' panicked at 'assertion failed: `(left == right)`
  left: `1000`,
 right: `0`', crates/ln-dlc-node/src/tests/just_in_time_channel/channel_close.rs:77:5

This happens because we are mining an unexpected number of blocks. The test has always only mined:

1 block to fund the coordinator's on-chain wallet.
1 block to confirm the close transaction.

Somehow I get 4 extra blocks some time after mining the first one.
The logs only show one instance of
2023-05-31T06:20:43.353410Z DEBUG ln_dlc_node::tests::bitcoind: Mining n_blocks=1

which appears to match the explicit call to bitcoind::mine when funding the coordinator.

@luckysori I've opted for the simple "fix" right now, as it's more important to have the test running and checking than fixing the test code.
It would be good to understand why we mine additional blocks though.

I believe that I found the problem, whilst hunting down other things: #722


Entitiy is nil when working with NSManagedObject subclass in Swift

Thank you for your amazing library, I could get CoreSpotlight running in 3mins! The only thing I noticed is that objectFromUserActivity: crashed, because
[NSEntityDescription entityForName:NSStringFromClass(mapping.objectClass)                                                               inManagedObjectContext:self.managedObjectContext];
returns nil. After short Investigation I found out that NSStringFromClass returns ModuleName.ClassName and, therefore, I don't a valid Entity. I couldn't find a nice solution so I just fixed it in my code.

Hi,
Sorry for the delay, just had some vacations ;)
What do you mean with ModuleName ?


cylinder example: gmsh2sod2d.py not working in converting msh to h5

Hi, Bernat:
It seems  gmsh2sod2d.py  contained in  sod2d_gitlab master branch  commit 433356db  does not work for the cylinder example.
gmsh version 4.12.1
python3 /home/mchao/code/sod2d_gitlab/utils/gmsh2sod2d/gmsh2sod2d.py cyl_p4_psenv -p 1
The error is
File "/home/mchao/code/sod2d_gitlab/utils/gmsh2sod2d/gmsh2sod2d.py", line 313, in
raiseError('Error! len(conec)!=npbou in boundary element!!')
Could you please resolve the issue? I think the cylinder case will be a very good simple example to test the coupling of the framework with SOD2D.

Is your SOD2D up to date? The error suggests a problem on the setup  of the boundary conditions in GMsh, but I assume you have used the example .geo file. @RicardMontala is the .geo up to date with the latest SOD2D version?
Also, maybe is better to directly open issues in SOD2D for this sort of trouble. This library (SmartSOD2D) is for the coupling, not the solver itself.

The issue has been resolved. Thank you!


Parser options

I've created configurable parser (I need it to push more language extensions from JSCS).

Babel parse method is https://github.com/babel/babel/blob/71031be044d783704c356e4cbeeb54d5a37a223d/packages/babel/src/api/node.js#L81-L106

:+1:


Match tag styles on meeting and orga level

Currently:
The appearance from the tags on the Orga-level and the meeting-level differ:
Tags-list on Orga-level:

Orga Tags on something (here a committee):

Tags-list on Meeting-level:

Meeting Tags on something (here a motion):

Wanted:
The Tags on the Orga level and the meeting level should be displayed in the same manner
Backend:

https://github.com/OpenSlides/openslides-backend/issues/2381
first/waiting for: changes in backend: additional field colour, and the ability to set it using tag.update and -create
for already created tags I would recommend setting the colour to the accent colour

Client:

[ ]  The client needs to allow to create/update tags with colour
[ ] The client needs to display the tags with the same outline and according colour as the orga-tags

As I also said in the backend PR This would cause the two models to be functionally the same in meeting and orga, with the exception of the meeting_id/organization_id properties, what do we think about unifying them into one model, paving the way for new features like globally available tags and giving the developers the ability to add new features to both at the same time?


Provide an offline engine API

Motivation

This PR is to support "Add APIs for using the inference engine in a single script without launching a separate server" in https://github.com/sgl-project/sglang/issues/1487. This is a simplified version of https://github.com/sgl-project/sglang/pull/1127 @JianyuZhan  where I reuse most of the existing code.
Modifications

Context
The current SRT server consists of an HTTP server and the SRT engine.

HTTP server: A FastAPI server that routes requests to the engine.
SRT engine:

Tokenizer Manager: Tokenizes the requests and sends them to the controller.
Controller (subprocess): Receives requests from the Tokenizer Manager, schedules batches, forwards them, and sends the output tokens to the Detokenizer Manager.
Detokenizer Manager (subprocess): Detokenizes the output tokens and sends the result back to the Tokenizer.

HTTP server and Tokenizer Manager are both running in the main process, but there is no way to decouple them and only instantiate Tokenizer Manager.
Decouple SRT engine and HTTP server
This PR introduces SRT engine  by decoupling launch_server to launch_server and launch_engine.
launch_server: launch_engine + HTTP server creation, used by SRT Runtime and standalone server.
launch_engine: SRT Engine creation, used by SRT engine.
New public API: Engine
Uplift Engine to the top level, so users can easily call with sgl.Engine
Engine Usage Example
Same settings as vllm but use SRT Engine.
import sglang as sgl

# Sample prompts.
prompts = [
    "Hello, my name is",
    "The president of the United States is",
    "The capital of France is",
    "The future of AI is",
]
# Create a sampling params object.
sampling_params = {"temperature": 0.8, "top_p": 0.95}

# Create an LLM.
llm = sgl.Engine(model_path="meta-llama/Meta-Llama-3.1-8B-Instruct")

outputs = llm.generate(prompts, sampling_params)
# Print the outputs.
for prompt, output in zip(prompts, outputs):
    print("===============================")
    print(f"Prompt: {prompt}\nGenerated text: {output['text']}")

===============================
Prompt: Hello, my name is
Generated text:  Alistair. I am an independent game developer.

I am currently working on a virtual reality game that focuses on exploring the world and discovering new things. The game has a unique art style that combines elements of both photorealism and cartoonish art. The game will have no story, but rather focus on the exploration of the world and the various things that can be found within.

I am looking for a 3D artist who can help me bring my vision to life. The ideal candidate will have experience in creating environments, characters, and objects in Unity. They should also have a strong
===============================
Prompt: The president of the United States is
Generated text:  Donald Trump. The president of India is Narendra Modi. The president of China is Xi Jinping. The president of Russia is Vladimir Putin. The president of France is Emmanuel Macron. The president of the United Kingdom is Boris Johnson. The president of Canada is Justin Trudeau. The president of Australia is Scott Morrison. The president of Japan is Shinzo Abe. The president of South Korea is Moon Jae-in. The president of Brazil is Jair Bolsonaro. The president of Colombia is Ivan Duque. The president of Argentina is Mauricio Macri
===============================
Prompt: The capital of France is
Generated text:  Paris. Paris is one of the world’s leading cities and is known for its history, art, culture, architecture, and gastronomy. Paris is located in the north-central region of France and is surrounded by the Seine River, which flows through the city. The city is divided into twenty districts or arrondissements, each with its own unique character and charm. Some of the most famous landmarks in Paris include the Eiffel Tower, the Louvre Museum, Notre-Dame Cathedral, the Arc de Triomphe, and the Champs-Élysées. Paris is also known
===============================
Prompt: The future of AI is
Generated text:  bright, and its potential to revolutionize various industries is immense. Here are some ways in which AI can impact the future of healthcare:

1. Personalized Medicine: AI can help in developing personalized treatment plans for patients based on their genetic makeup, medical history, and lifestyle factors. This can lead to more effective treatments and better outcomes for patients.
2. Diagnosis and Treatment: AI-powered tools can help healthcare providers diagnose diseases more accurately and quickly. They can also help in selecting the most appropriate treatment options based on the patient's condition.
3. Drug Development: AI can help in acceler

Discussion
One caveat is that we construct ServerArgs, but the HTTP server related args will not be used. I think this is ok because  ServerArgs is the superset of Engine Args, so it can cover everything.
Testing
Add test_srt_engine.py, which runs batch inference and assert the answer.
TODO

Add async generate
Add encode

Checklist

[x] Format your code according to the Contributor Guide.
[x] Add unit tests as outlined in the Contributor Guide.
[x] Update documentation as needed, including docstrings or example tutorials.

I think async generate is needed. QQ What does add decode in todo 2 refer to?

@zhyncs oh i mean encode like the one in Runtime. It was a typo

@zhyncs i can do the async gen in the next PR

Consistency test is failing on H100 in CI, but passing on my A100

hey guys i'm getting the AttributeError: module 'sglang' has no attribute 'Engine' on sgl '0.3.2' still not in prod ?

@imadoualid the changes should be in the main HEAD if you can build from source.


Add oauth2 script to blog (merge after CORS blog fix)

Links to https://gist.github.com/msavy/9390bfcd497f3c683a1c

Please merge this after the CORS blog fix, then it'll merge without any conflicts

The cord fix is merged but this still has conflicts

Because i force pushed it won't let me reopen so i'll make a new PR - it claims it merges cleanly


Documentation Typo Fixes for Motoko Base Library Design Guide

Overview
This pull request addresses several typographical errors in the Motoko Base Library's design documentation. Ensuring accuracy and clarity in our documentation is vital for effective communication among developers and for maintaining the high standard of our project.
Changes Made

Corrected 'encapsultion' to 'encapsulation' in the Classes section. This change clarifies the intended meaning, emphasizing the importance of encapsulation in abstract data types.
Updated 'valus' to 'values' in the Containers section under the description of the keys iterator. This fix ensures that the documentation accurately represents the functionality of the iterator.
Fixed 'distinghuish' to 'distinguish' in the Containers section. This modification aids in presenting a clearer rationale for the choice of the put method name over set, especially in the context of sets and other containers.

Impact
These changes, while minor, contribute to the overall readability and professionalism of our documentation. They do not affect the actual functionality of the codebase but are essential for consistent and error-free documentation.
Additional Notes
While reviewing the documentation, I've noticed a few areas where further clarification might be beneficial. I plan to propose additional updates in subsequent pull requests to continuously improve the documentation.

Dear @joaolago1113,
In order to potentially merge your code in this open-source repository and therefore proceed with your contribution, we need to have your approval on DFINITY's CLA.
If you decide to agree with it, please visit this issue and read the instructions there. Once you have signed it, re-trigger the workflow on this PR to see if your code can be merged.
— The DFINITY Foundation

Obrigado pelas correções! Infelizmente estamos péssimos com a linguagem natural 😜

@joaolago1113 Colega já fechou este, se vc. achar mais, seja convidado de abrir um outro. Obrigado de novo!


Space Management Permissions

Adds new space management permissions

:boom: Acceptance test localApiTests-apiSpaces-ocis failed. Further test are cancelled...


[BUG] Home screen frozen with blogs activated

Describe the bug
If blogs are enabled, you can't navigate to any of the tabs (trades/waivers, open league info drop down, etc.) on the home screen. The tabs show up and you can click on them, but the pages will not load. There is a workaround to where if you click on a tab (i.e. matchups) and refresh the page it'll load and the drop downs will work properly, but you can't physically get to any of the pages from the home screen.
Your League ID
784484879164362752
To Reproduce
Steps to reproduce the behavior:

Go to home page (draftdaydynasty.vercel.app)
Attempt to click on any of the above tabs (matchups, trades & waivers, league info, etc.)
See error

Expected behavior
When I click on the matchups tab from home page, I expect the matchups page to load. If I click on league info from home page, I expect a drop-down menu.
Screenshots
If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):

OS: MacBook Pro
Browser Chrome
Version 96.0.4664.131

Smartphone (please complete the following information):

Device: iPhone 13 Pro Max
OS: iOS 15.1

Make sure that all fields are filled out for the blog post (title, author, type, body). Omitting one will cause errors. If that doesn't resolve the issue, let me know.

Just to follow up, there was a bug that was trying to display draft blogs. I fixed that in a separate pr. Let me know if that might have been the cause, and I'll close this out!

I'm closing this for now. If neither of the above help, feel free to reopen and I'll investigate further.

Ahh, my mistake for just now getting back around to this. Work got hectic this week, but finally have some down time.
The issue is still persisting for me (I have 2 different pages for my leagues and I opened up the issue on the wrong one... this is the page I'm having the issue on). Just double-checked Contentful, and there are no drafts and everything is assigned an author, type, title and body.

I'll re-open this then. Would you mind posting screenshots of the your contentful setup? The error on that page is Post-afe91257.js:1 Uncaught (in promise) TypeError: Cannot read properties of undefined (reading 'title'), so it's having some kind of an issue with the title of one of your posts. It's hard to debug this since I'm not hooked up to your contentful, but I might figure it out with some screenshots.
In particular a copy of paste of the JSON from your post content model:
https://app.contentful.com/spaces/<your_space>/content_types/blog_post/preview
i.e:
{
  "name": "Blog Post",
  "description": "",
  "displayField": "title",
  "fields": [
    {
      "id": "title",
      "name": "Title",
      "type": "Symbol",
      "localized": false,
      "required": true,
      "validations": [],
      "disabled": false,
      "omitted": false
    },
    {
      "id": "type",
      "name": "Type",
      "type": "Symbol",
      "localized": false,
      "required": true,
      "validations": [
        {
          "in": [
            "Weekly Preview",
            "Weekly Recap",
            "PSA",
            "General",
            "League Change Proposal",
            "Trash Talk"
          ]
        }
      ],
      "disabled": false,
      "omitted": false
    },
    {
      "id": "featured",
      "name": "Featured",
      "type": "Boolean",
      "localized": false,
      "required": false,
      "validations": [],
      "defaultValue": {
        "en-US": true
      },
      "disabled": false,
      "omitted": false
    },
    {
      "id": "body",
      "name": "Body",
      "type": "RichText",
      "localized": false,
      "required": true,
      "validations": [
        {
          "nodes": {}
        }
      ],
      "disabled": false,
      "omitted": false
    },
    {
      "id": "author",
      "name": "Author",
      "type": "Symbol",
      "localized": false,
      "required": true,
      "validations": [],
      "disabled": false,
      "omitted": false
    }
  ],
  "sys": {
    "space": {
      "sys": {
        "type": "Link",
        "linkType": "Space",
        "id": "zzzzzzzzz"
      }
    },
    "id": "blog_post",
    "type": "ContentType",
    "createdAt": "2021-09-08T23:24:28.476Z",
    "updatedAt": "2021-09-10T20:22:29.983Z",
    "environment": {
      "sys": {
        "id": "master",
        "type": "Link",
        "linkType": "Environment"
      }
    },
    "publishedVersion": 5,
    "publishedAt": "2021-09-10T20:22:29.983Z",
    "firstPublishedAt": "2021-09-08T23:24:28.735Z",
    "createdBy": {
      "sys": {
        "type": "Link",
        "linkType": "User",
        "id": "zzzzzzz"
      }
    },
    "updatedBy": {
      "sys": {
        "type": "Link",
        "linkType": "User",
        "id": "zzzzzzz"
      }
    },
    "publishedCounter": 3,
    "version": 6,
    "publishedBy": {
      "sys": {
        "type": "Link",
        "linkType": "User",
        "id": "yyyyyyy"
      }
    }
  }
}

As well as a screenshot of the posts you've made?

Made some changes to Contentful settings but still nothing yet. I can tell some of my settings differ from the one you sent, so I think that might have something to do with it. I've attached the 3 blog posts I've made (yes, the last one I've made was in week 6, 2022 goal is to be more consistent with those posts haha)

{
"name": "Blog",
"description": "",
"displayField": "title",
"fields": [
{
"id": "title",
"name": "Title",
"type": "Symbol",
"localized": false,
"required": true,
"validations": [],
"disabled": false,
"omitted": false
},
{
"id": "type",
"name": "Type",
"type": "Symbol",
"localized": false,
"required": true,
"validations": [
{
"in": [
"Weekly Preview",
"Weekly Recap",
"PSA",
"General",
"League Change Proposal",
"Trash Talk"
]
}
],
"disabled": false,
"omitted": false
},
{
"id": "featured",
"name": "Featured",
"type": "Boolean",
"localized": false,
"required": false,
"validations": [],
"disabled": false,
"omitted": false
},
{
"id": "body",
"name": "Body",
"type": "RichText",
"localized": false,
"required": true,
"validations": [
{
"enabledMarks": [
"bold",
"italic",
"underline",
"code"
],
"message": "Only bold, italic, underline, and code marks are allowed"
},
{
"enabledNodeTypes": [
"heading-1",
"heading-2",
"heading-3",
"heading-4",
"heading-5",
"heading-6",
"ordered-list",
"unordered-list",
"hr",
"blockquote",
"embedded-entry-block",
"embedded-asset-block",
"hyperlink",
"entry-hyperlink",
"asset-hyperlink",
"embedded-entry-inline"
],
"message": "Only heading 1, heading 2, heading 3, heading 4, heading 5, heading 6, ordered list, unordered list, horizontal rule, quote, block entry, asset, link to Url, link to entry, link to asset, and inline entry nodes are allowed"
},
{
"nodes": {}
}
],
"disabled": false,
"omitted": false
},
{
"id": "author",
"name": "Author",
"type": "Symbol",
"localized": false,
"required": true,
"validations": [],
"disabled": false,
"omitted": false
}
],
"sys": {
"space": {
"sys": {
"type": "Link",
"linkType": "Space",
"id": "b2ldzzl4lvug"
}
},
"id": "blog_post",
"type": "ContentType",
"createdAt": "2021-09-28T04:32:51.311Z",
"updatedAt": "2022-01-13T21:44:49.144Z",
"environment": {
"sys": {
"id": "master",
"type": "Link",
"linkType": "Environment"
}
},
"publishedVersion": 5,
"publishedAt": "2022-01-13T21:44:49.144Z",
"firstPublishedAt": "2021-09-28T04:32:51.697Z",
"createdBy": {
"sys": {
"type": "Link",
"linkType": "User",
"id": "5UYiZVG3lSpxwKxNcF1PJV"
}
},
"updatedBy": {
"sys": {
"type": "Link",
"linkType": "User",
"id": "5UYiZVG3lSpxwKxNcF1PJV"
}
},
"publishedCounter": 3,
"version": 6,
"publishedBy": {
"sys": {
"type": "Link",
"linkType": "User",
"id": "5UYiZVG3lSpxwKxNcF1PJV"
}
}
}
}

I haven't been able to reproduce this locally yet. I'm going to convert this to a discussion for the time being


colorChange event does not run when you change the color in the picker

Details
What browser were you using: Chrome
What OS were you using: Windows
Project version you have: Latest (cdn)
Describe the bug
colorChange event does not run when you change the color in the picker, only when you exit it.
To Reproduce
Steps to reproduce the behavior:

Open the picker
Change the color
See error, it doesn't run the event

If this isn't a bug and it's meant to do this, would you be able to make a separate event for when you change the color on the picker.

Could you send me the error you were getting please, I don't seem to be getting the error on my side when following your reproduce bug instructions.
A guess as to why you're getting an error may be because you call the CDN link in a script tag above all your other HTML, if so try moving the script tag to the very bottom of the  tag. Otherwise just send me the error message and I'll take a look.
Thanks

@lewis45454

It works if you click out of the picker, just not while you are choosing the color.

Oh, so you mean the color only updates your elements when you leave the picker and not while dragging around the color box and slider?

@lewis45454 If so then this supposed to be the case. The pickers is in the center of the screen with a darkened background I cant see much use in making the picker update while changing color.
When i next work on this I plan on making a widget that appears next to the button used to open it, I can see how it would be useful to update color event while changing color when in this case

That's correct, it's fine.


Card not showing multiple image attachments

Example Tweet: https://twitter.com/sullycasts/status/1562195981067620352?t=Ba95-03OLrcAoOfqhuT67g&s=19
Expected behaviour(s):
Card should at least indicates multiple images were published.

Could you assign me to do this task?


KnightCrawler | ElfHosted

Addon Manifest URL
https://torrentio.elfhosted.com/manifest.json
Addon Description
This is an ElfHosted instance of Knight Crawler, a fork of torrentio.strem.fm, which provides torrent streams from torrent providers' RSS feeds, and DebridMediaManager hashes.
(ElfHosted is a open-source PaaS built and run by geeks)
This public instance (http://torrentio.elfhosted.com) is provided free for the Stremio community - Enjoy! (reasonable rate-limits apply, appropriate for casual individual streaming use, but not automation)
Hosted instances with rate-limits appropriate for automation are available for $0.15/day, with $10 free credit for new accounts.
An internal, un-rate-limited instance is provided free, with all ElfHosted apps, for automation.
Why use ElfHosted Stremio addons?

✅ Highly-available, open-source infrastructure /
✅ Active contributors to r/StremioAddons subreddit and Discord
✅ Geek out over public performance metrics

Language of Content
None
Choose Labels

[ ] anime
[ ] asian drama
[ ] bollywood
[ ] cartoons
[ ] debrid support
[ ] http streams
[ ] live tv
[ ] metadata
[ ] misc
[X] movies
[ ] music
[ ] nsfw
[ ] podcasts
[ ] radios
[ ] subtitles
[X] torrents
[X] tv shows
[ ] video games

Great work,thanks

Love your work mate

@danamag I don't mean to be a PITA re the title, I was making edits in the UI and GitHub was pushing them as updates. Happy for you to reset the title, I just thought it should be something different to "Public", now that there are 5-6 ElfHosted addons!

Exclude Filter & size limit not working

Exclude Filter & size limit not working

As far as size limit be sure to put "GB" after the number, with no spaces. Ex: 20gb
I tried a few different ways and that seems to be the only way it works for  me

Works, but not on everything. Might be my filters (10gb size limit) but a lot of movies do not generate links (new and old) even though they do with other similar addons. Good backup I guess

Does it work,is not showing any links

Thanks for all your hard work

not showing any links

not showing any links

https://stremio-addons-guide.elfhosted.com/
Says it is deprecated.... I also read somewhere that their addons (some or all I'm not sure) moved away from real debrid and require torbox or something else. I have not researched this yet myself as torrentio still works most of the time.


[#16935] Workspace creation with Policy Draft info

Details

Created 2 new keys to save Policy and PolicyMembers information, PolicyDraft and PolicyMembersDraft. Those keys are filled with basic Policy information just for fast navigation to the Workspace Init Page, where those drafts are removed and the Policy data are saved and a request is made.
Fixed Issues

$ https://github.com/Expensify/App/issues/16935
PROPOSAL: https://github.com/Expensify/App/issues/16935#issuecomment-1752799106
Tests

[ ] Verify that no errors appear in the JS console

Click on the User Avatar
Go to Workspace
Click on New Workspace Button

Offline tests

Click on the User Avatar
Go to Workspace
Click on New Workspace Button
When is back to online, the workspace should work normally

QA Steps

[ ] Verify that no errors appear in the JS console

PR Author Checklist

[x] I linked the correct issue in the ### Fixed Issues section above
[x] I wrote clear testing steps that cover the changes made in this PR

[x] I added steps for local testing in the Tests section
[x] I added steps for the expected offline behavior in the Offline steps section
[x] I added steps for Staging and/or Production testing in the QA steps section
[x] I added steps to cover failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[x] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)
[x] I tested this PR with a High Traffic account against the staging or production API to ensure there are no regressions (e.g. long loading states that impact usability).

[x] I included screenshots or videos for tests on all platforms
[x] I ran the tests on all platforms & verified they passed on:

[x] Android: Native
[x] Android: mWeb Chrome
[x] iOS: Native
[x] iOS: mWeb Safari
[x] MacOS: Chrome / Safari
[x] MacOS: Desktop

[x] I verified there are no console errors (if there's a console error not related to the PR, report it or open an issue for it to be fixed)
[x] I followed proper code patterns (see Reviewing the code)

[x] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick)
[x] I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
[x] I verified that comments were added to code that is not self explanatory
[x] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[x] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method

[x] If any non-english text was added/modified, I verified the translation was requested/reviewed in #expensify-open-source and it was approved by an internal Expensify engineer. Link to Slack message:

[x] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[x] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is approved by marketing by adding the Waiting for Copy label for a copy review on the original GH to get the correct copy.
[x] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[x] I verified the JSDocs style guidelines (in STYLE.md) were followed

[x] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[x] I followed the guidelines as stated in the Review Guidelines
[x] I tested other components that can be impacted by my changes (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar are working as expected)
[x] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[x] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[x] I verified that if a function's arguments changed that all usages have also been updated correctly
[x] If a new component is created I verified that:

[x] A similar component doesn't exist in the codebase
[x] All props are defined accurately and each prop has a /** comment above it */
[x] The file is named correctly
[x] The component has a clear name that is non-ambiguous and the purpose of the component can be inferred from the name alone
[x] The only data being stored in the state is data necessary for rendering and nothing else
[x] If we are not using the full Onyx data that we loaded, I've added the proper selector in order to ensure the component only re-renders when the data it is using changes
[x] For Class Components, any internal methods passed to components event handlers are bound to this properly so there are no scoping issues (i.e. for onClick={this.submit} the method this.submit should be bound to this in the constructor)
[x] Any internal methods bound to this are necessary to be bound (i.e. avoid this.submit = this.submit.bind(this); if this.submit is never passed to a component event handler like onClick)
[x] All JSX used for rendering exists in the render method
[x] The component has the minimum amount of code necessary for its purpose, and it is broken down into smaller components in order to separate concerns and functions

[x] If any new file was added I verified that:

[x] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[x] If a new CSS style is added I verified that:

[x] A similar style doesn't already exist
[x] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(themeColors.componentBG))

[x] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[x] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[x] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[x] If the PR modifies a component or page that can be accessed by a direct deeplink, I verified that the code functions as expected when the deeplink is used - from a logged in and logged out account.
[x] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[x] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.
[x] I have checked off every checkbox in the PR author checklist, including those that don't apply to this PR.

Screenshots/Videos

Android: Native

https://github.com/Expensify/App/assets/1676818/2ddef97c-ffbf-454b-9793-a4b20efdabf6

Android: mWeb Chrome

https://github.com/Expensify/App/assets/1676818/c8fe075a-b2d1-4b8f-ba17-6b4566669bda

iOS: Native

https://github.com/Expensify/App/assets/1676818/baaa39f4-3869-4f6f-a3b6-91c80960dffa

iOS: mWeb Safari

https://github.com/Expensify/App/assets/1676818/20f4c263-0141-4e8c-aa08-f6227c8b7611

MacOS: Chrome / Safari

https://github.com/Expensify/App/assets/1676818/99c48397-8852-4900-9006-d67d9e629e6d
https://github.com/Expensify/App/assets/1676818/132741b5-af68-4d61-909c-e34683261f52

MacOS: Desktop

https://github.com/Expensify/App/assets/1676818/d39503f1-abdf-4292-962b-c09ea34c50a7

This is causing a crash if we try to access the admin/announce room offline
https://github.com/Expensify/App/assets/16493223/06865f46-29f2-4a1d-b470-d28855842944

This is causing a crash if we try to access the admin/announce room offline

@s77rt can you send me the steps, I'm not able to reproduce it.
I just pushed, there was a typo, I wasn't deleting the POLICY_MEMBERS_DRAFTS

I can't reproduce the bug now. Seems working fine. Probably related to the typo

Reviewer Checklist

[X] I have verified the author checklist is complete (all boxes are checked off).
[X] I verified the correct issue is linked in the ### Fixed Issues section above
[X] I verified testing steps are clear and they cover the changes made in this PR

[X] I verified the steps for local testing are in the Tests section
[X] I verified the steps for Staging and/or Production testing are in the QA steps section
[X] I verified the steps cover any possible failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[X] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)

[X] I checked that screenshots or videos are included for tests on all platforms
[X] I included screenshots or videos for tests on all platforms
[X] I verified tests pass on all platforms & I tested again on:

[X] Android: Native
[X] Android: mWeb Chrome
[X] iOS: Native
[X] iOS: mWeb Safari
[X] MacOS: Chrome / Safari
[X] MacOS: Desktop

[X] If there are any errors in the console that are unrelated to this PR, I either fixed them (preferred) or linked to where I reported them in Slack
[X] I verified proper code patterns were followed (see Reviewing the code)

[X] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick).
[X] I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
[X] I verified that comments were added to code that is not self explanatory
[X] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[X] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method
[X] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[X] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is approved by marketing by adding the Waiting for Copy label for a copy review on the original GH to get the correct copy.
[X] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[X] I verified the JSDocs style guidelines (in STYLE.md) were followed

[X] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[X] I verified that this PR follows the guidelines as stated in the Review Guidelines
[X] I verified other components that can be impacted by these changes have been tested, and I retested again (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar have been tested & I retested again)
[X] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[X] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[X] If a new component is created I verified that:

[X] A similar component doesn't exist in the codebase
[X] All props are defined accurately and each prop has a /** comment above it */
[X] The file is named correctly
[X] The component has a clear name that is non-ambiguous and the purpose of the component can be inferred from the name alone
[X] The only data being stored in the state is data necessary for rendering and nothing else
[X] For Class Components, any internal methods passed to components event handlers are bound to this properly so there are no scoping issues (i.e. for onClick={this.submit} the method this.submit should be bound to this in the constructor)
[X] Any internal methods bound to this are necessary to be bound (i.e. avoid this.submit = this.submit.bind(this); if this.submit is never passed to a component event handler like onClick)
[X] All JSX used for rendering exists in the render method
[X] The component has the minimum amount of code necessary for its purpose, and it is broken down into smaller components in order to separate concerns and functions

[X] If any new file was added I verified that:

[X] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[X] If a new CSS style is added I verified that:

[X] A similar style doesn't already exist
[X] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(themeColors.componentBG)

[X] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[X] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[X] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[X] If the PR modifies a component or page that can be accessed by a direct deeplink, I verified that the code functions as expected when the deeplink is used - from a logged in and logged out account.
[X] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[X] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.
[X] I have checked off every checkbox in the PR reviewer checklist, including those that don't apply to this PR.

Screenshots/Videos

Web

https://github.com/Expensify/App/assets/16493223/81a85772-9d4d-4805-9dae-2168188ef865

Mobile Web - Chrome

https://github.com/Expensify/App/assets/16493223/4012a2fe-d191-4d0e-aedb-4d5d8b6dec27

Mobile Web - Safari

https://github.com/Expensify/App/assets/16493223/4f64c3bf-5871-4882-a27c-5a3aabc8b396

Desktop

https://github.com/Expensify/App/assets/16493223/90857f39-adf1-448d-a9d2-4d916a53bfd3

iOS

https://github.com/Expensify/App/assets/16493223/afc850ee-5165-4036-b69d-6aa6bad6d55a

Android

https://github.com/Expensify/App/assets/16493223/750c63fd-f98d-4a82-9eea-d5579184a1a0

@gedu Great work! Day & night UX difference. Thanks!

:hand: This PR was not deployed to staging yet because QA is ongoing. It will be automatically deployed to staging after the next production release.

🚀 Deployed to staging by https://github.com/marcochavezf in version: 1.3.85-0 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

🚀 Deployed to production by https://github.com/francoisl in version: 1.3.85-4 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
failure ❌

🕸 web 🕸
success ✅

🚀 Deployed to staging by https://github.com/marcochavezf in version: 1.3.86-0 🚀

platform
result

🤖 android 🤖
failure ❌

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

🚀 Deployed to production by https://github.com/francoisl in version: 1.3.86-5 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
failure ❌

🕸 web 🕸
success ✅


npm package is not compatible on angular2 beta.9

runtime error: EXCEPTION: No Directive annotation found __
reference clone source code works fine.

seting mangle:flase
should fix this issue


Fix typos in automerge and finishReleaseCycle workflows

Details
Fixes typos in workflow that have caused our automerge PRs to fail, for example: https://github.com/Expensify/Expensify.cash/runs/2143260738?check_suite_focus=true

@AndrewGable looks like this was merged without passing tests. Please add a note explaining why this was done or remove the Emergency label if this is not an emergency.

Merging to fix deploys!

First preDeploy: https://github.com/Expensify/Expensify.cash/actions/runs/665933255
PR against master: https://github.com/Expensify/Expensify.cash/pull/1867
Automerge master: https://github.com/Expensify/Expensify.cash/actions/runs/665935855
Second preDeploy: https://github.com/Expensify/Expensify.cash/actions/runs/665938192
PR against staging: Not created b/c one was already open

There were other problems, but fortunately this PR served it's purpose in repairing the automerge workflow


KeyError: 0 when running main.py

When running python . in the main directory, I get this error:
WARNING:root:Limited tf.compat.v2.summary API due to missing TensorBoard installation.
WARNING:root:Limited tf.compat.v2.summary API due to missing TensorBoard installation.
WARNING:root:Limited tf.compat.v2.summary API due to missing TensorBoard installation.
WARNING:root:Limited tf.summary API due to missing TensorBoard installation.
Traceback (most recent call last):
  File "/Users/martinkondor/miniforge3/envs/tf24/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/Users/martinkondor/miniforge3/envs/tf24/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "./__main__.py", line 43, in <module>
    predictor = MBType()
  File "./__main__.py", line 13, in __init__
    self.vectorizer = load_prep('trained/vectorizer.pkl')
  File "/Users/martinkondor/miniforge3/envs/tf24/lib/python3.8/site-packages/joblib/numpy_pickle.py", line 585, in load
    obj = _unpickle(fobj, filename, mmap_mode)
  File "/Users/martinkondor/miniforge3/envs/tf24/lib/python3.8/site-packages/joblib/numpy_pickle.py", line 504, in _unpickle
    obj = unpickler.load()
  File "/Users/martinkondor/miniforge3/envs/tf24/lib/python3.8/pickle.py", line 1210, in load
    dispatch[key[0]](self)
KeyError: 0
Loading the model...

Possibly because the model was saved with a previous installation of sklearn. A new model release might resolve this issue.


Adding first/second-seed classes for inputs

If 2 inputs are checked (1 first-seed and 1 second-seed), if I uncheck the input with first-seed while the second-seed is still checked, the first-seed class will append to the second input.
The second input (thats now the only one checked), will have both first-seed and second-seed classes on it.
Now if I want to recheck an input it will still check as second-seed, resulting in 2 second-seed inputs and not being able to check an input as first-seed.
Basicly I can't apopend a first-seed class to an input unless its checked first.
Only way to fix this for now is to uncheck all input that have a second-class and start checking witht the input we want as first-seed.
  // add classes for first and second checked element
  $("input[name='team']").each(function(e) {
    if ($(this)[0].checked) {
      if($(this).siblings('.first-seed').length <= 0
        && !$(this).hasClass('.second-seed')) {
        $(this).addClass("first-seed");
      } else if ($(this).siblings('.first-seed').length === 1) {
        $(this).addClass("second-seed");
      } else if ($(this).siblings('.second-seed').length === 1) {
        $(this).addClass("first-seed");
      }
    } else {
      $(this).removeClass();
    }
  });``

Fixed by pulling the code out of the unnecessary each() loop


Change the mandated parameter name

Changed the mandated parameter name from Certificate (
which I can not find anywhere in the docs other than this page) CertificatePolicy

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs-conceptual/azurermps-6.4.0/migration-guide.6.0.0.md
:white_check_mark:Succeeded
View (azurermps-6.4.0)

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

Thank you for your contribution!


Configuration loading

In https://github.com/killercup/quicli/issues/19#issuecomment-359879839, @moosingin3space suggested adding configure.

While most CLI applications don't load their configuration from environment variables, configuration files are pretty common.
I think it would be cool to be able to load most of the configuration from a file (additionally, there should be a -c flag to change the default path to the configuration file) and set/override the rest using args.
An application could load a config file:
host = "<IP_ADDRESS>"
port = 8080

and passing --host or --port should overwrite the value in the config file.
Using feature flags, it should be pretty trivial to support different configuration formats using serde.
A second step would be a feature to dump a config file using the default values in the default config path on first run (e.g. if the config file does not exist)


[BUG] ...when i click toolkiy reports i get a message saying there is a bugf and to "click here" to report it

Describe the bugi did not report 3 hrs ago  i reported JUSTNOW
A clear and concise description of what the bug is.
To Reproduce
Steps to reproduce the behavior:

Go to '...'
Click on '....'
Scroll down to '....'
See error

Expected behavior
A clear and concise description of what you expected to happen.
Screenshots
If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):

OS: [e.g. OSX, Windows]
Browser [e.g. chrome, firefox]
Toolkit Version [e.g. 2.22.0]

Setting Export
Please replace the text below with your exported Toolkit settings. You can export these by going to the Toolkit Options page, click the Import/Export Settings button at the bottom left and copying the text in the modal which appears.
REPLACE_ME_WITH_SETTINGS

This has already been reported and a PR has been submitted for it. Closing this as a duplicate.


Write a wiki article about using electron

Describe how electron brings web technologies to desktop apps
Give some tips to web developers who are starting to work on electron

like not having to worry about making the app cross browser

Examples for things that only a desktop application can do. (Can't do in a website)

like accessing the local files of the user without a hassle

Article should go here: https://github.com/roonyh/fossasia-gci-companion/wiki

I would like to work on this Issue, it might help me to understand electron more clearly and it will help others too.

@shashank-sharma great :+1:  thanks for picking this up.

I have made an edit. Please take a look at wiki page.

@shashank-sharma looks good on a quick look. I will go through it again in a few minutes. In the mean time provide links to any resources giving credit that you referred.

@roonyh  I have added the links. I hope its fine now.

@roonyh there are some changes to be made i'll review it :)

@shashank-sharma Looks good enough for me. @abdulhannanali sure. go ahead and make your changes.
Make this a seperate wiki page. Don't edit the Home page.
Add a link to this from the README.md.
In README where it says
" It's developed using the electron platform. [Learn more] " <-- This should link to this article.

@roonyh Ok, I have made separate page for that and link has been added in the PR.

merged. thanks!


Simple release

Analgous to wbls and wbbt @chris-grove

I've confirmed that the file converts to our ACE format without issue. Now just waiting to make sure it reads into our ontology browser software.


ENVOY-S Standard running D7.6.175 is not recognized to use api/v1/production

Describe the bug
Pyenphase (0.9.0 used in Home assistant core-2023.9.0.dev0) with ENVOY-S-EU Standard (800-00656-r06) running D7.6.175 generates an error:
  File "/workspaces/ha-core-dev/homeassistant/components/enphase_envoy/coordinator.py", line 88, in _async_update_data
    return (await envoy.update()).raw
            ^^^^^^^^^^^^^^^^^^^^
  File "/home/vscode/.local/lib/python3.11/site-packages/pyenphase/envoy.py", line 252, in update
    system_production = EnvoySystemProduction.from_production(production_data)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vscode/.local/lib/python3.11/site-packages/pyenphase/models/system_production.py", line 34, in from_production
    inverters = all_production[1]
                ~~~~~~~~~~~~~~^^^
IndexError: list index out of range

Envoy.probe function decides to use URL_PRODUCTION rather then URL_PRODUCTION_V1. When forcing it to use URL_PRODUCTION_V1 it works fine with this Envoy model.
When disabling or indent line 'self._production_endpoint = URL_PRODUCTION' all runs well and uses correct data.
Possible solutions:

indent line more, only if consumption is there use production.json
use: 'if production and consumption:' to use production.json.

   async def probe(self) -> None:
        """Probe for model and supported features."""
        try:
            production_json: dict[str, Any] = await self.request(URL_PRODUCTION)
        except (json.JSONDecodeError, httpx.HTTPError) as e:
            _LOGGER.debug("Production endpoint not found at %s: %s", URL_PRODUCTION, e)
        else:
            production: list[dict[str, str | float | int]] | None = production_json.get(
                "production"
            )
            if production:
                for type_ in production:
                    if type_["type"] == "eim" and type_["activeCount"]:
                        self._supported_features |= SupportedFeatures.METERING
                        break
            consumption: list[
                dict[str, str | float | int]
            ] | None = production_json.get("consumption")
            if consumption:
                for meter in consumption:
                    meter_type = meter["measurementType"]
                    if meter_type == "total-consumption" and meter["activeCount"]:
                        self._supported_features |= SupportedFeatures.TOTAL_CONSUMPTION
                    elif meter_type == "net-consumption" and meter["activeCount"]:
                        self._supported_features |= SupportedFeatures.NET_CONSUMPTION
            self._production_endpoint = URL_PRODUCTION

        if not self._production_endpoint:
            for endpoint in (URL_PRODUCTION_V1, URL_PRODUCTION_JSON):
                try:

To Reproduce
Running Home assistant devcontainer and add enphase envoy. After entering credentials Envoy is added but starts registering the error in the log file.
Additional context
Add any other context about the problem here.
Production.json for ENVOY-S Standard D7.6.175. Numbers in production segment are wrong. Should use numbersr from V1/production
Production.json
{
  "production": [
    {
      "type": "inverters",
      "activeCount": 24,
      "readingTime":<PHONE_NUMBER>,
      "wNow": 2317,
      "whLifetime": 118599
    }
  ],
  "storage": [
    {
      "type": "acb",
      "activeCount": 0,
      "readingTime": 0,
      "wNow": 0,
      "whNow": 0,
      "state": "idle"
    }
  ]
}

api/v1/production.json, these numbers are correct
{
  "wattHoursToday": 8353,
  "wattHoursSevenDays": 193658,
  "wattHoursLifetime": 6083914,
  "wattsNow": 2154
}

This is a case we didn't consider where we need to get data from two end points for production and consumption.
We can't just switch the end point because it will break compatibility with envoy_readers and everyone's data would change in ha
We need to write tests for all the systems we have before we could add support for this type of bifurcated set up.
Might be a month or two before we can get to the point of supporting that.

Actually this will probably break production systems as well
I think we should consider reverting the ha changes back to the old version for now and wait to roll out the new pyenphase lib until we can support this case
If I had free to to finish it this week I would consider otherwise but I don't

This is a case we didn't consider where we need to get data from two end points for production and consumption.

The ENVOY-S Standard has no consumption data at all. It resembles a legacy one with the inverters added.

This is a case we didn't consider where we need to get data from two end points for production and consumption.

The ENVOY-S Standard has no consumption data at all. It resembles a legacy one with the inverters added.

I think fixing your specific case it's pretty simple.  It's the bifurcated case I'm worried about

Agree, there's more issue's we know I haven't even looking into in the new one.
What I would do (did) is build a custom integration with it and invite the testers in HA Core issue 88983  to test with it or share their data. I realize it's not common practice, but in this case many of the Envoy users are surviving like this since it broke with new firmware.

I forgot I wrote all the tests for envoy reader. We can pull them in here as it will get things moving faster and hopefully avoid the need to revert
https://github.com/jesserizzo/envoy_reader/tree/main/tests/fixtures

Working on a test set for this now
Can you post your inverters endpoint as well?

I will. I have a bunch data laying around from all testing we did last weeks.let me dig it up for your cases. might not be formatted as you need it.

Great.  I'll try to get the directories set up in the next hour or so. Than you can PR it instead as it might be a bit easier

@catsmanac I wrote the tests and made the dirs for the fixtures. https://github.com/pyenphase/pyenphase/tree/main/tests/fixtures
It would be great if you could add your data.  Even better if you can add tests for it, but happy to do that if you just want to add the fixtures

I'll have a stab at it, not too much experience with these test, but willing to learn. Will let you know if I get stuck.
We will need to introduce envoy type in addition to firmware version in some way. Envoy-s standard, metered and metered without connected CT clamps all run same version but return different results.
And legacy  < 3.9 returns an html info with lifetime production in there, not an xml info, nor a production page. Yes I had a tester for it who had a legacy and a new Envoy just installed and needed the new stuff.

I'll have a stab at it, not too much experience with these test, but willing to learn. Will let you know if I get stuck.
We will need to introduce envoy type in addition to firmware version in some way. Envoy-s standard, metered and metered without connected CT clamps all run same version but return different results.

That should be covered by this check https://github.com/pyenphase/pyenphase/blob/6d01261dc3d711f8b3907b9113a3c2ad8de546be/src/pyenphase/envoy.py#L189

And legacy  < 3.9 returns an html info with lifetime production in there, not an xml info, nor a production page. Yes I had a tester for it who had a legacy and a new Envoy just installed and needed the new stuff.

Home assistant doesn't allow scraping html so we likely won't add support for that

@catsmanac
bdraco@MacBook-Pro-9 pyenphase % ENVOY_HOST="ip"<EMAIL_ADDRESS>ENVOY_PASSWORD="pass" python3 devtools/fixture_collector.py
Fixtures written to enphase-7.3.130
Zip file written to enphase-7.3.130.zip

@joostlek  just added a fixture collector that might make it a lot easier

That should be covered by this check

For the test data I meant, you would need 3 different fixtures with the same version label.
Envoy-s metered without CT clamps only returns 0 values in v1/production. Needs to use data from first production segment, but that only has wnow and lifetime, not today and last seven days. Real neat feature of last firmware

For the test data I meant, you would need 3 different fixtures with the same version label.

The dir names are arbitrary so we can do
X.Y.Z_no_ct_clamps
X.Y.Z_ct_clamps
etc

Home assistant doesn't allow scraping html so we likely won't add support for that

Well current version of envoy_reader on pypi does it using regex. If you don't include it's going to be a breaking change for the old ones.

That worked just fine, this is for ENVOY-S EU Standard on 7.6.175. ENVOY-S metered en metered withouth clamps have some different ones I'll add later.
enphase-7.6.175.zip

I marked the PR to switch libs as a breaking change https://github.com/home-assistant/core/pull/97862

That worked just fine, this is for ENVOY-S EU Standard on 7.6.175. ENVOY-S metered en metered withouth clamps have some different ones I'll add later.
enphase-7.6.175.zip

Thanks. They were added in https://github.com/pyenphase/pyenphase/pull/35

I marked the PR to switch libs as a breaking change home-assistant/core#97862

Looking at 3.9.36 fixture I see you have an info.xml file there. How did you obtain that one?
In my testing with <3.9 and >= 3.9  I got html back in both cases for info.xml. If you know how to get xml for 3.9 then that may work for <3.9 as well!?
<3.9
info.xml reports error html page with serial no in there
2023-07-26 12:15:03.067 DEBUG (MainThread) [custom_components.enphase_envoy.envoy_reader] HTTP GET Attempt #1: http://<IP_ADDRESS>/info.xml: False: Header:None
2023-07-26 12:15:03.091 DEBUG (MainThread) [custom_components.enphase_envoy.envoy_reader] Checking Token value:
2023-07-26 12:15:03.091 DEBUG (MainThread) [custom_components.enphase_envoy.envoy_reader] Found empty token:
2023-07-26 12:15:07.650 DEBUG (MainThread) [custom_components.enphase_envoy.envoy_reader] Fetched from http://<IP_ADDRESS>/info.xml: <Response [200 OK]>:     <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN">

      <title>error</title>
      <meta http-equiv="Content-Type" content="text/html; charset=utf-8">

.
.
          Envoy Serial Number:<PHONE_NUMBER>12
.

and production returns html with production numbers
2023-07-21 12:41:38.290 DEBUG (MainThread) [custom_components.enphase_envoy.envoy_reader] HTTP GET Attempt #1: http://<IP_ADDRESS>/production: Header:None
2023-07-21 12:41:38.494 DEBUG (MainThread) [custom_components.enphase_envoy.envoy_reader] Fetched from http://<IP_ADDRESS>/production: <Response [200 OK]>:     <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN">

      <title>production</title>
      <meta http-equiv="Content-Type" content="text/html; charset=utf-8">

.
.

System has been live since

Thu Oct 18, 2012 03:20 PM PDT

Currently	        0 W

Today	      5.36 kWh

Past Week	      362 kWh

Since Installation	      172 MWh

.

and v1/production
2023-07-21 12:41:37.906 DEBUG (MainThread) [custom_components.enphase_envoy.envoy_reader] HTTP GET Attempt #1: http://<IP_ADDRESS>/api/v1/production: Header:None
2023-07-21 12:41:38.287 DEBUG (MainThread) [custom_components.enphase_envoy.envoy_reader] Fetched from http://<IP_ADDRESS>/api/v1/production: <Response [404 Not Found]>: {
  "status": 404,
  "error": "",
  "info": "Resource /api/v1/production not found",
  "moreInfo": ""
}

>=3.9
info.xml
2023-07-25 07:35:50.393 DEBUG (MainThread) [custom_components.enphase_envoy.envoy_reader] HTTP GET Attempt #1: http://<IP_ADDRESS>/info.xml: Header:None
2023-07-25 07:35:50.480 DEBUG (MainThread) [custom_components.enphase_envoy.envoy_reader] Fetched from http://<IP_ADDRESS>/info.xml: <Response [200 OK]>:     <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN">

      <title>error</title>
      <meta http-equiv="Content-Type" content="text/html; charset=utf-8">

.
.
          Envoy Serial Number:<PHONE_NUMBER>07
.

and v1/production
2023-07-25 07:35:50.499 DEBUG (MainThread) [custom_components.enphase_envoy.envoy_reader] HTTP GET Attempt #1: http://<IP_ADDRESS>/api/v1/production: Header:None
2023-07-25 07:35:50.507 DEBUG (MainThread) [custom_components.enphase_envoy.envoy_reader] Fetched from http://<IP_ADDRESS>/api/v1/production: <Response [200 OK]>: {
  "wattHoursToday": 290,
  "wattHoursSevenDays": 103157,
  "wattHoursLifetime": 35724674,
  "wattsNow": 522
}

and production.json
2023-07-26 12:15:10.143 DEBUG (MainThread) [custom_components.enphase_envoy.envoy_reader] Fetched from http://<IP_ADDRESS>/home.json: <Response [301 Moved Permanently]>:

Oh you're right, I probably did something wrong when I tried to use git to check the differences of your files vs the files we already have

The old fixture collector script converted the files into json, and since XML isn't json it fails. I made an update for it that also collects headers and status codes, so we can also create a valid test response when the endpoint isn't available. Checkout the file in https://github.com/pyenphase/pyenphase/pull/43

Here's an updated 7.6.175 For ENVOY-S EU Standard.
enphase-7.6.175-standard.zip

<3.9
info.xml reports error html page with serial no in there

envoy_reader scrapes out the serial via a regex https://github.com/jesserizzo/envoy_reader/blob/0d3fb696519d487d14b0fff7def7d77ab2173cff/envoy_reader/envoy_reader.py#L402
With ADR004 we aren't allowed to do that for a core integration.  I realize that was the status quo for a long time, but it should not have been allowed.

If we wanted to add support for < 3.9, we would need to either get an exception to ADR004 which is unlikely, or get ADR004 changed to allow scraping when older firmware requires it but newer firmware provides non-scraping methods.  That seems more likely to be accepted if its constrained in that manner.
If we want to go that route an https://github.com/home-assistant/architecture discussion needs to be started and it needs a champion

Also the serial number fetch is coming from /info and /info.xml is not used so maybe that works on older firmware.
We need someone with the ancient firmware to run the new fixture collector to figure out the best way forward on that.

We need someone with the ancient firmware to run the new fixture collector to figure out the best way forward on that.

The ones I know don't do any python stuff afaik. Best they can give you is browser access I expect or maybe a curl command.

If we want to go that route an https://github.com/home-assistant/architecture discussion needs to be started and it needs a champion

Just gave it a try in 952

It looks like the answer for older version is to go the custom component route.

You may want to check where the cutoff point is and with which firmware the info.xml is available. Would a browser test to http://envoy.local/info.xml returning xml be sufficient proof?

I think so.
I have 4 envoys and all of them respond with the xml at http://envoy.local/info.xml and http://envoy.local/info

[x] R3.9.36 (a1a93f) <IP_ADDRESS> (89 nem)
[x] D3.17.3 (0e8c7a) <IP_ADDRESS> (81 nem)
[x] D5.0.62 (5e57a9) <IP_ADDRESS> (12 nem)
[x] D7.3.130 (91f206) <IP_ADDRESS> (89 nem plus)

Got feedback from testers, R3.11.30 gave 404 on /info.xml, /info gave xml and api/v1/production returns the well known data.
<envoy_info>
<time>1691436287</time>
<device>
<sn>121445034107</sn>
<pn>800-00069-r05</pn>
<software>R3.11.30</software>

R3.7.31 gave 404 on /info.xml, /info returns xml with version, /api/v1/production returns 404. /production works but it's html

He's been running this for 11 years. On the positive side he told he'll get an RMA for his legacy swapped for an ENVOY-s so Enphase can push new firmware to the envoy connected...

Just to let you know I synced ha core dev again and tested the new enphase_envoy. It contained pyenphase 0.14.1 and worked fine with my ENVOY-S standard 7.6.175  :smile:.

So it seems like 3.7 won't work but 3.9 will
I guess we will find out if 3.8 works next month via reports
We can update the docs once we have better idea of what is going to be too old to work

We can update the docs once we have better idea of what is going to be too old to work

Before I learned about this effort I worked on updated docs as part of the discussion/activities in 88983 going on to promote a custom integration to new core version. But that's HA doc, no programmers  description as is probably needed here. Though HA could use an update as well.

I have the html example that comes back for that R3.7 production, the xml for info and the json 404 for api/v1/production laying around (somewhere) would that help?
HA itself has http and scrape integrations. Could they be used from HA Envoy side when needed?

I have the html example that comes back for that R3.7 production, the xml for info and the json 404 for api/v1/production laying around (somewhere) would that help?

Yes

HA itself has http and scrape integrations. Could they be used from HA Envoy side when needed?

Someone could probably make a scrape sensor using the scrape integration without too much work

This is what I have: 3.7.31-fixtures.zip

As for scraping, as pyencharge is a public library, add the functionality to read the html using the regex. This does not require any additional packages.  Add a parameter to the envoy class signalling to not use webscraping (or other way around) or an additional method. Then in HA Core use that option to prevent webscraping. Someone (maybe me) then can clone the new HA Core enphase Envoy as custom integration with the scrape option enabled. Think that will meet ADR004.

It would be better if we had a registration interface for another library to register a new endpoint for production and provide all the scraping logic. Than there is no scraping logic in this lib.
That could mean that a custom component would only need register a new endpoint for the production feature and provide the parser for it

Working on the 7.6.175_noct fixture I wonder if there's an issue with this one.
production reports eim activecount 0 (numbers there are wrong), but first segment has only wNow and whLifetime, no today nor 7day values.
Current logic in probe would recognize the activecount 0 and switch to api/v1/production but that is only reporting zero's.
In this case it should use wNow and whLifetime from first segment and set today and sevenday values to None. (Yes feature of this fw...)
{
	"production": [
		{
			"type": "inverters",
			"activeCount": 12,
			"readingTime":<PHONE_NUMBER>,
			"wNow": 3429,
			"whLifetime": 634218
		},
		{
			"type": "eim",
			"activeCount": 0,
			"measurementType": "production",
			"readingTime":<PHONE_NUMBER>,
			"wNow": -8.579,
			"whLifetime": 0.006,
			"varhLeadLifetime": 35.489,
			"varhLagLifetime": 46.254,
			"vahLifetime": 4781.293,
			"rmsCurrent": 0.192,
			"rmsVoltage": 709.67,
			"reactPwr": -0.0,
			"apprntPwr": 45.683,
			"pwrFactor": -0.28,
			"whToday": 0.006,
			"whLastSevenDays": 0.006,
			"vahToday": 4781.293,
			"varhLeadToday": 35.489,
			"varhLagToday": 46.254
		}
	],

and api/v1/production
{
	"wattHoursToday": 0,
	"wattHoursSevenDays": 0,
	"wattHoursLifetime": 0,
	"wattsNow": 0
}

the firmware inconsistency for these devices is pretty bad

not to mention how they document their api. Just sharing our findings while we where improving the custom integration and got to the point were we covered them all (except the next new fw probably).
And other item is multi-phase support, there will be up to 3 lines sections in production and consumption. One for each phase measured where phase is like 3 phase or 1 phase with ct behind different breakers.
			"lines": [
				{
					"wNow": 296.316,
					"whLifetime": 1028298.771,
					"varhLeadLifetime": 2628.982,

production.txt

https://github.com/pyenphase/pyenphase/pull/54/files#diff-b5dbe106863e55297cbef9b3b3108e0cfdbe4e64ddb5f00cc51f04ed4237c07eR24
I added a hook to register a new updater module.
The updater needs to implement probe and update
With probe, you return which features the updater will provide
With update, it should update the data for those features and set it on the EnvoyData dataclass
This should make it very easy to add new updaters and if you want register one that does scraping for the legacy devices

I also added some tests for the registration interface an example on how one might make a scraper updater
https://github.com/pyenphase/pyenphase/blob/82efcec228dbe263c3f1c39e6ded3e9283fbfac2/tests/test_endpoints.py#L412

Thanks! It would need to clone the new HA Core and add it there and then offer as custom integration as many of these legacy users run newer ones as well and they probably want to continue the domain naming scheme for their long term history continuation.

Thanks! It would need to clone the new HA Core and add it there and then offer as custom integration as many of these legacy users run newer ones as well and they probably want to continue the domain naming scheme for their long term history continuation.

It could be as simple as a custom integration that requires core enphase_envoy as a dependency and that registers a newer updater as part of the code being loaded. It wouldn't have any other functionality other than patching in support for the old firmware to the library

That one is a bit more difficult to fix

The method we used was both production and consumption segments being present and eim activecount is zero. The first item can set feature metered and the second can drive feature cts or something similar.
Info has  but I'm not sure how that's driven, could very well be true as this is ENVOY-S Metered.

Once we get fixtures in place for this version we can come up with a solution. Probably no rush as it wasn't supported before so no breaking change to not have it out of the gate

Original isse was resolved long time ago. Other suggestions noted. Thanks.


Revert "Explicitly declare sudo for Travis"

This reverts commit f0b1603663f1e61085f27cbdd4c24fdfec88d728.
It was a workaround a temporary Travis issue. See https://github.com/travis-ci/travis-ci/issues/9875#issuecomment-408867228.
Signed-off-by: Valentin Deniaud<EMAIL_ADDRESS>

Merging this now might break #272
https://github.com/travis-ci/travis-ci/issues/9069#issuecomment-401828871

@Croydon rebased on master and it looks like all is fine :)


Add creydr to OWNERS

As per eventing onboarding: Adding myself to the OWNERS file

/cherry-pick release-v1.6

/cherry-pick release-v1.5

/cherry-pick release-v1.4


rpy2-3.5.9 dropped broken Converter objects as context managers.

The method Converter.context should be used instead.

PR merged.


persp-mode does not get activated during startup

I have an issue getting persp-mode running during startup. The following is my setup, but the persp-mode variable is always nil  after startup and perspective is not available anymore.

 (use-package perspective
    :ensure t
    :bind
    ("C-x k" . persp-kill-buffer*)
    ("C-x C-b" . persp-list-buffers)   ; or use a nicer switcher, see below
    :config
    (setq persp-state-default-file "~/my-emacs-stuff/perspectives")
    (setq persp-sort 'access)
    (persp-mode)
    :hook
    ((kill-emacs-hook . persp-state-save)))

I have to explicitly execute M-x persp-mode twice to get it started.
Initially, I had a emacs-startup-hook setup to also have pers-state-load to be called on emacs startup to load the perspective file.

Any ideas why that happens?

Are there any errors when you run M-x persp-mode the first time? Does the *Messages* buffer show anything interesting?
What happens if you change :config to :init in your use-package form?

I think changing the :config to :init did the trick for now. I have a few perspectives open and I do not want to restart my emacs right now, but I will try again and report back if it really works.

Please reopen if this is still a problem.


chore(pevm): Post-processing lazy evaluation: assert that the evaluated nonce matches the transaction's

This PR addresses https://github.com/risechain/pevm/issues/310

Hey @hai-rise, I've tried to have a crack at this one. And I'm not sure if I've identified a bug or not (thinking possibly yes as the other MemoryValue::Basic checks pass).
The mainnet_blocks_from_disk is failing. I'd love to try and debug this more to better understand the stack, but I may need some pointers if that's okay 🙏 .
Just for my understanding of what's going on here: this is essentially validating the incarnation (tx) that resulted in the particular state mutation of an account, which is encoded in the write_history.  Where the write_history holds all particular state updates for a given memory location from the resulting incarnations?
Here is the particular tx_env where this check fails:
TX TxEnv {
    caller: 0xa9cefd4474086c52f15abc0e604284440768da57,
    gas_limit: 23100,
    gas_price:<PHONE_NUMBER>5,
    transact_to: Call(
        0xcff0b829fd9bd7a58ca8d62eef8105506f859592,
    ),
    value:<PHONE_NUMBER>00000,
    data: 0x,
    nonce: Some(
        18,
    ),
    chain_id: Some(
        1,
    ),
    access_list: [],
    gas_priority_fee: Some(
<PHONE_NUMBER>,
    ),
    blob_hashes: [],
    max_fee_per_blob_gas: None,
    authorization_list: None,
}

Tyvm.

Thank you for the feedback @hai-rise. That makes sense. Was way off the mark with that bug... Thank you for the learning here.


dpdk: refactor to allow building extapps

Motivation for this change
These patches enable building external DPDK apps like pktgen. The primary output can now be used as RTE_SDK root.
Kernel modules are split into "kmod".
All apps and examples are bundled together and copied to bin thus ending up on PATH.
After adding the missing dependencies vm_power_mgr is now built into "examples" output.
Things done

[ ] Tested using sandboxing
(nix.useSandbox on NixOS,
or option build-use-sandbox in nix.conf
on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] OS X
[ ] Linux

[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Fits CONTRIBUTING.md.

I had to massage my previous release-16.03 patches a bit to apply them cleanly on top of master.

Looking good. Can you just rename the last commit to pktgen: init at 3.0.00 to follow the CONTRIBUTING guideline ?

Sure. Will send another PR shortly.

No need to make a new PR, you can just amend the last commit and force-push

Oops, too late, already sent a new PR. I also rebased on top of master for a smooth fast-forward (in case I beat other 278 PRs in the queue ;)


Customizing theme colors

Basic branding for the new edX based training website.

updated colors with the ones in Enthought's color palette
restyled footer to look like EToD footer

@agrawalprash Would you have time to review and test this out on your end?

Thanks @kamalx . I will probably test this only tomorrow - could you please add screenshots with this PR? Thanks.

A screenshot of the page below:

LGTM. Thanks.


local scope,Simplify the template, and Reduce function run that some  function with args

What problem does this feature solve?

        {{func(a.b).field1}}
        {{func(a.b).field2}}
        {{func(a.b).field3}}
        {{func(a.b).field4}}

What does the proposed API look like?

          {{const tmp = func(a.b) //such a way like this}}
        {{tmp.field1}}
        {{tmp.field2}}
        {{tmp.field3}}

Prior art: https://github.com/vuejs/vue/issues/6913
This should go through an RFC process. It would probably be useful to have in-template computed properties as well. I created this for vue 2 https://github.com/posva/vue-local-scope

now i get a hack very useful
like :justAHack="(SUDUKO = gene([S, I]),0)"

This is something that we'd like to find a good solution for but it should probably go through the RFC process instead of being discussed here.