[WIP] Pseudo-random sampling

Fixes https://github.com/scikit-optimize/scikit-optimize/issues/193

I was looking at Spearmint implementation again. They are actually using this strategy when optimizing the aquisition function:

Initialize a grid of points with a sobol sequence (https://github.com/HIPS/Spearmint/blob/e719c56cee9eefda5df6af40f834c9fa310e23a5/spearmint/choosers/default_chooser.py#L259)
Extend this list with random points close from the current best point (https://github.com/HIPS/Spearmint/blob/e719c56cee9eefda5df6af40f834c9fa310e23a5/spearmint/choosers/default_chooser.py#L339)
For all these points (i.e, many of them!!), evaluate the acquisition function EI  (https://github.com/HIPS/Spearmint/blob/e719c56cee9eefda5df6af40f834c9fa310e23a5/spearmint/choosers/default_chooser.py#L343)
Run the actual optimization of the acquisition using the 20 best points as initial points (https://github.com/HIPS/Spearmint/blob/e719c56cee9eefda5df6af40f834c9fa310e23a5/spearmint/choosers/default_chooser.py#L375)

We should certainly do the same. I presume this coarse brute-force like optimization is helping a lot to locate a good enough location from which the optimizer should be started.

Also instead of adding and maintaining code for sobol sequences, we could maybe consider using instead a Python module that already implements it. I found https://github.com/naught101/sobol_seq, which is available in pip as sobol_seq.

Do we need a Sobol sequence? My understanding of them is that they solve the problem of "real" random numbers clumping together. They are a low discrepancy sequence. You can draw a new sample from them and it will be "as far away" as possible from the existing ones. So people use them for MC integration were it is not known upfront how many samples will be drawn and you want to cover the space evenly. If you decide/know up front how many samples you will draw, you can make a regular grid and be done with it.
Unless we really need it I'd be -1 on adding the code or a new dependency. There might be something I am missing about the spearmint strategy or we might want to adopt a different strategy so please point that out to me :)

Yes, for the coarse scan, a regular grid should be enough, though this sobol sequence does that nicely for us.

(in particular, defining that regular grid for n points in R^d is not that immediate)

I'm also -1 for adding an additional dependency, the code from https://github.com/naught101/sobol_seq is also a copy.

I am -1 for copy pasting code we dont own and dont understand. The less code we have, the less we have to maintain.

OK, let us debate the merits and demerits of doing that, after we convince ourselves that it is useful through experiments.

Here are my benchmarks. Doesn't look too promising:
On Branin:

Acquisition Function
Minimum
Best minimum
Mean f_calls to min
Std f_calls to min
Fastest f_calls to min
Time

lbfgs
0.401 +/- 0.004
0.398
60.7
40.591
24
28.95

pseudosampling + lbfgs
0.401 +/- 0.004
0.398
60.6
43.107
32
29

spearmint (w/o pseduosampling)
0.398 +/ 0.000
0.398
71.2
22.203
32
188.98

spearmint (with pseduosampling)
0.398 +/ 0.000
0.398
118.7
23.199
88
190

On Hart6:

Acquisition Function
Minimum
Best minimum
Mean f_calls to min
Std f_calls to min
Fastest f_calls to min
Time

lbfgs
-3.095 +/- 0.222
-3.311
73.3
50.465
31
322

pseudosampling + lbfgs
-3.146 +/- 0.110
-3.298
99.2
42.107
34
379.21

spearmint (w/o pseduosampling)
-3.230 +/ 0.057
-3.321
84.5
54.57
31
1145.35

spearmint (with pseduosampling)
-3.112 +/ 0.050
-3.192
90.9
46.580
23
2362.57

Could you write a little interpretation of what the results mean to you?
I'm looking at the spread (for hart6) on the minimum we find and comparing it to spearmint -> our spread is 0.222 compared to their 0.05 -> we seem to sometimes get it right and sometimes we are miles off -> we are just getting lucky?
pseudosampling + lbsgs has a smaller spread -> we are relying less on luck/find the minimum more consistently -> so even though we don't find the minimum in less steps it is overall better?

Do you use the same value of n_restarts_optimizers in all cases? (we should if we want to make a fair comparison and determine whether this strategy is better)

Could you also add a comparison for the case where the optimizer is "sampling" with a value of n_points identical to what you use in the case of spearmint? Unless there is a bug somewhere we should at least see comparable runtimes.

Could you also share your benchmark script please?

Could you also share your benchmark script please?

Should be in benchmarks/ no?

One other thing, it seems spearmint sets xi=0.0 when evaluating the EI.

Do you have an idea why the spearmint strategy appears so much slower than lbfgs? For both, LBFGS is run the same number of times. Then the additional computations come from generating the sobol grid and evaluating EI through the GP. I guess the slow down comes from this step, as it basically require computing a cholesky decomposition for a large matrix (because of n_points=10000).

Just had a look to gpr.py. Why dont we precompute at fit these values? (L291+)
                L_inv = solve_triangular(self.L_.T, np.eye(self.L_.shape[0]))
                K_inv = L_inv.dot(L_inv.T)

As far as I understand, they dont depend on the test data, and therefore could be compruted. But that is maybe not the meat of the computation here...

One more thing. I just did some debugging and looked at values. In many cases, it seems we have again this issue of completely flat GP. Setting n_restarts_optimizer > 0 in GPRegressor partially solves this issue.
Even if this slows things down, I would really be in favour of increasing that value. Given that in real cases, evaluating the function should takes minutes-hours, I dont mind spending some more second in properly fitting the GP.

I'll address your comments soon. I was completely convinced that it will not work but seems like my obvious mistakes does not help :P

Any progress on this Manoj? Very curious to see the benchmarks with all these little things updated :)

I guess the slow down comes from this step, as it basically requires computing a cholesky decomposition for a large matrix (because of n_points=10000)

Yes, I agree. I have noticed in tests that most of the times just the "sampling" strategy is slower than the "lbfgs"!

(Average of branin across 5 runs)
|Acquisition Function| Minimum | Best minimum | Mean calls to min | Fastest calls to min | Time |
------------------|------------|-----------|---------------------|--------------------|-----------------------|-----|
lbfgs | 0.398 +/- 0.000 | 0.398 | 81.2 +/ 24.72| 42 | 76.61
sampling | 0.398 +/- 0.000 | 0.398 | 56.2 +/  23.02 | 38 | 167.55
pseudosampling | 0.398 +/- 0.000 | 0.398 | 82.2 +/  34.30 | 45 | 172.19
spearmint (lbfgs + sampling) | 0.398 +/ 0.000 | 0.398 | 53.8 +/ 4.72 | 47 | 191.18
spearmint (lbfgs + pseudosampling) | 0.398 +/ 0.000 | 0.398 | 114.4 +/ 47.99 | 58 | 203.
(I reran the benchmarks for lbfgs + sampling again just to be sure)

The benchmarks can be reproduced by setting rvs as uniform / pseudo_uniform and acq_optimzer as sampling, lbfgs, spearmint.

Future steps involve seeing if the superior performance of lbfgs + sampling is because of the better restarts or the addition of spray points.

When you do spearmint (lbfgs + sampling), the only difference with doing instead pseudosampling is that you draw the points uniformly at random instead of using the sobol sequence, right?
Also, could you run your benchmark on a more difficult problem?

From the results, spearmint (lbfgs + sampling) looks very good in my opinion. It yields the lowest mean number of calls and the lowest standard deviation (by quite a margin). This is very promising, but would be good to evaluate on a more difficult problem to see if that holds.

When you do spearmint (lbfgs + sampling), the only difference with doing instead pseudosampling is that you draw the points uniformly at random instead of using the sobol sequence, right?

Yes.

Benchmarks on brain (average across 5 runs). I removed the time since I was doing a lot of data-intensive operations in the side.
|Acquisition Function| Minimum | Best minimum | Mean calls to min | Fastest calls to min | Time |
------------------|------------|-----------|---------------------|--------------------|-----------------------|-----|
lbfgs | -3.207 +/ 0.075 | -3.281 | 90.8 +/ 64.22 | 30 |
sampling | -3.207 +/ 0.047 | -3.295 | 127.4 +/ 61.242 | 25 |
pseudo_sampling | -2.998 +/ 0.088 | -3.158 | 146.2 +/ 30.55 | 96 |
spearmint (lbfgs + uniform) | -3.251 +/ 0.058 | -3.224 | 148.0 +/ 51.712 | 73 |
spearmint (lbfgs + pseudo_uniform) | -3.299 +/ 0.047 | -3.224 | 134.0 +/ 30.47 | 100 |
Not sure how to interpret these numbers except that the spearmint strategy returns a better minimum. Maybe it makes sense to measure the (mean number of iterations / time) to reach a fixed number since the minimum clearly varies? Also I"m searching for other good benchmark problems.

For branin, the minima are fixed and dont vary. We should in priority measure the distance to these known minima. I would say here that spearmint (lbfgs + pseudo_uniform) wins, followed by spearmint (lbfgs + uniform).
Also I am not quite sure the "mean calls to min" is really a good metric. We should rather see the problem as having a budget of number of calls and then evaluate what is the minimum attained within that budget. It is not that important whether it is reached soon or late.
What do you think?

Also I think you are evaluating on hart6, not branin :)

Hmm, something I dont quite get. What is "best minimum"? and how this value could be worse than "minimum" in the cases of spearmint?

Yes, I agree that in case we should fix the budget as the number of calls and measure the minimum obtained within this budget. But if we see that the minimum is reached earlier we can always reduce the budget right? (i.e change the default number of n_calls, nothing comes free)
Also should we have more hyper-parameter sensitive benchmarks and not rely just on these two benchmarks for making decisions from now on?

There are a number of benchmarks in the original paper (https://arxiv.org/pdf/1206.2944.pdf) under section 4. (Empirical Analyses). I'll try to have a look over the next few days.

Training a clf on make_classification? Might be harder in the sense that you have mixed dimensions but also takes a while to evaluate.
Right now we do measure how close to the minimum each optimizer gets no? Just that the budget is very generous compared to the hardness of the problem. We should measure the spread on the minimum found to help us judge how much the algorithm relies on luck or maybe how "consistent" it is.
Can we add a purely random strategy to the comparison as well? This would give us a benchmark for the benchmarks: how much better than random is the strategy?

I was thinking of hacking this to make it work (https://github.com/automl/HPOlib-logreg). It seems to also be a standard benchmark for Bayesian-optimisation problems.
I agree that 200 calls is a bit too many, I stuck with that because that was the one described in the original paper. Maybe we should cut it down to say 50 and allow 1 random start?

I wouldnt change n_random_starts. But I am fine with reducing the budget to 50 or 100 iterations.

Here are the benchmarks when branin and hart6 with n_calls=50, averaged across 10 runs.
Branin:

Acquisition Function
Minimum
Best minimum
Time

lbfgs
0.403 +/ 0.005
0.398
33.885

uniform + sampling
0.400 +/ 0.003
0.398
49.747

pseudo_uniform + sampling
0.402 +/ 0.009
0.399
57.640

uniform + spearmint
0.400 +/ 0.004
0.398
47.894

pseudo_uniform + spearmint
0.400 +/ 0.004
0.398
63.014

Hart6

Acquisition Function
Minimum
Best minimum
Time

lbfgs
-3.048 +/ 0.177
-3.297
15.227

uniform + sampling
-3.138 +/ 0.171
-3.322
16.210

pseudo_uniform + sampling
-3.012 +/ 0.325
-3.322
63.307

uniform + spearmint
-3.090 +/ 0.215
-3.322
23.503

pseudo_uniform + spearmint
-3.138 +/ 0.194
-3.322
77.941

From just these two benchmarks, maybe we can infer that just doing a pseudo-uniform sampling and lbfgs are not great strategies?

From just these two benchmarks, maybe we can infer that just doing a pseudo-uniform sampling and lbfgs are not great strategies?

Agree. Also we should expect that just doing sampling cannot be the best strategy. Therefore, I would advocate we switch to a spearmint-like procedure for optimizing the acquisition function. From hart6, it seems pseudo-random numbers work better, which is also consistent with the actual spearmint implementation. What do you think?

The only reason that I'm slightly reluctant is that I would like to test with at least a couple of more benchmarks to make sure.
The logistic regression benchmark that I've described above uses the number of epochs and batch size as hyperparameters as well. Since lbfgs acts on the one-hot encoded transformed space, it might not be fair to make a comparison for that right now.
We should fix https://github.com/scikit-optimize/scikit-optimize/pull/62 first?

+1 for trying on a third benchmark problem. Not sure going all the way up to properly support categorical variables is necessary here (that is, for the purpose of determining whether this optimization of the acquisition is indeed better). I would suggest trying on a problem with non-categorical variables only.

I wrote a wrapper around 2-layer MLP in tensorflow, the script can be seen here (https://github.com/MechCoder/Notebooks/blob/master/MLP_tensorflow_skopt_on_mnist.ipynb). I am running it on the clusters and will update once I get the results.
I just tune the regularization, initialisation scale and the learning rate. Are there any other interesting continuous parameters?

I just tune the regularization, initialisation scale and the learning rate. Are there any other interesting continuous parameters?

Sounds good

My job stopped just before it was about to end because of insufficient memory :/ Restarted it after requesting much more memory.

When calling lbfgs, shall we remove maxiter=20? (it is not set in spearmint).
Shall we also set n_restarts_optimizer=20 and n_spray=10 to have the same settings as they have?

MNIST seems to be a really easy problem. All of them reach a minimum of 1.0 within 50 calls. I shall try with a different dataset (CIFAR 10)
I don't think we should increase the maxiter parameter. It seems to be an overkill to me especially since we have set n_restarts_optimizer as well.

Fair enough. Can we still switch to 20 for n_restarts_optimizers and 10 for n_spray? We shoud really make everything necessary to not waste calls to the objective.

I ran a benchmark for the CIFAR dataset using keras before I realized that (https://github.com/scikit-optimize/scikit-optimize/pull/279) might be creating a non-trivial impact because of the different scales of the Integer and Real dimensions.
Also could someone do a sanity check of the code before I run any more benchmarks?

Sorry, I cannot run any GPU based benchmarks till the start of next week since there seems to be a limit for all users and I require it for my projects. After Monday, I can run the benchmarks in parallel for all the (acq_optimizer and rvs) combinations to decide the default values.
Also there seems to be a bug in the keras cnn script here (https://github.com/fchollet/keras/issues/4587) which makes the default SGD optimizer perform bad. I can replace it with the Adam Optimizer in the next set of benchmarks which is likely to give better results along with the bug-fix in the normalization of Integer dimensions.

I am rerunning the benchmarks for "pseudorandom" and "spearmint" after rebasing on master and using the adam optimizer. Hopefully if all is well, will post the results in 24 hours and rerun the other combinations in parallel.

The results don't look great. I used the cross-entropy on the validation dataset as the minimization objective. You can have a look at the script here (https://github.com/MechCoder/Notebooks/blob/master/bench_keras_cifar10.py) which is mainly a modification of the original keras cnn-model here (https://github.com/fchollet/keras/blob/master/examples/cifar10_cnn.py)
Here (https://gist.github.com/MechCoder/bd8c1cf5b2681b06569cfe4773265fb2) are the objectives at each of the 50th call., the first ten are at random. The minimum is obtained at the 7th call, which is at random.

Here are plots of some of the integer hyperparameters against the function calls.

It seems that all the optimum points for these Integer hyperparameters are either at the upper bound and lower bound. Anything obviously wrong with how we handle integer dimensions?

Hmmm, could be many things...

an issue due to hyper-parameters that are mostly irrelevant, hence making the optimization more difficult.
an issue with the gp
an issue due to scaling integers: for this can you try instead using only Real bounds and then casting back to int within the objective function? or can you evaluate your benchmarks only on the continuous hyper-parameters? (dropout rates, learning rate, and you can add epsilon in Adam)

In any case, I am not sure what you observe here is due to the sampling strategy of the starting points... I would be glad if we could settle this issue once and for all and move on.

Here are the results for varying the three dropouts, learning rate and epsilon. (5 hyperparameters) I shall update the remaining two after the runs are finished. (~12 hours from now). The value here is the mean validation cross-entropy loss on the cross-validation data. For the benchmarks, it seems sampling is the best strategy ?!

Acquisition Function
Minimum
Time

lbfgs

uniform + sampling
0.589
38402.7

pseudo_uniform + sampling

uniform + spearmint
0.7359
40740.49

pseudo_uniform + spearmint
0.639
43089.46

Seems to concur with my benchmarks here (https://github.com/scikit-optimize/scikit-optimize/pull/262#issuecomment-260835534)? Also should we change the default from lbfgs to sampling? I'll wait till the other benchmarks finish though.

(updated)

The benchmarks seem to suggest that doing anything fancier than random sampling does not perform better (at least on these two problems).
I would like https://github.com/scikit-optimize/scikit-optimize/pull/281 to be merged and this to be kept open for people to try it out if possible. I would also like to move on given the amount of time spent on this pull request and the not so obvious benefits.
WDYT?

Geh. I really dont understand. How could spearmint be worse than random sampling?
What you call "uniform + sampling" is simply brute forcing the acquisition function right? Why would it be worse if you then run lbfgs from these top points? I should only improve.
For how many runs do you perform your last benchmarks? Reading the discussion again, it seems that for branin and hart6, combining pseudo-uniform sampling of the starting points + lbfgs was doing quite well...

The benchmarks seem to suggest that doing anything fancier than random sampling does not perform better (at least on these two problems).
I would like #281 to be merged and this to be kept open for people to try it out if possible. I would also like to move on given the amount of time spent on this pull request and the not so obvious benefits.
WDYT?

I disagree. Your results from Nov 14 show otherwise.

An opinion on this @betatim ? Would be good to decide something and move.
I am personnally in favour of something close to uniform sampling of random initial points (many), evaluating the acquisition functions there, then starting lbfgs at the top 20 of them.

Which results are you referring to? This (https://github.com/scikit-optimize/scikit-optimize/pull/262#issuecomment-260252061)? The best minimun obtained is the same no?

Yes, but the mean number of calls is smaller, and variance is much smaller
too. It is indeed a very good thing to not have to rely on luck to hope
getting the best results.
(sent from my phone)
On 29 Dec 2016 10:35 p.m., "Manoj Kumar"<EMAIL_ADDRESS>wrote:

Which results are you referring to? This (#262 (comment)
https://github.com/scikit-optimize/scikit-optimize/pull/262#issuecomment-260252061)?
The best minimun obtained is the same no?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/scikit-optimize/scikit-optimize/pull/262#issuecomment-269697129,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAdKS-V16Y6SUjZ8Mfi42gIouiWw_gM1ks5rNCekgaJpZM4Knd8I
.

But please have a look at the benchmarks on hart6 here (https://github.com/scikit-optimize/scikit-optimize/pull/262#issuecomment-260835534), both the mean and standard deviation of the minimum obtained.

I cant believe running lbfgs on the top points can make things worse.

I don't know as well but it could be a number of things, such as non-smoothness of the acquistion function or the acquisition function not modeling the original objective function.

Btw, I thought we agreed here (https://github.com/scikit-optimize/scikit-optimize/pull/262#issuecomment-260445579) that the mean calls to the minimum is not a great metric since we fix the budget beforehand.

Running lbfgs can only help us find a better minimum of the acquisition
function. And we do want to optimize the acquisition function correctly. If
we don't trust it and instead rely on luck, then there is no point in
building it in the first place. It could be that the acquisition function
is bad, but this is separate issue from optimizing it. What do you think ?
(sent from my phone)
On 29 Dec 2016 22:52, "Manoj Kumar"<EMAIL_ADDRESS>wrote:

I cant believe running lbfgs on the top points can make things worse.
I don't know as well but it could be a number of things, such as
non-smoothness of the acquistion function or the acquisition function not
modeling the original objective function.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/scikit-optimize/scikit-optimize/pull/262#issuecomment-269699307,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAdKS-wZVA6Q6tEYVTOqpFGzfQjg4_aIks5rNCuxgaJpZM4Knd8I
.

Yes, I'll verify on how the acquisition function is optimized and I'll reserve my comments for after that.

Superseded by #284.


How sass files generated?

A simple question? how all those sass files generated?  Are they wrote manually or generated by maxas.pl ?
if they are generated by maxas.pl, any based source files? Thanks

Some kind of superhuman intelligence may have been involved.

The sass files are hand-written, but also contain blocks of perl code used to generate specific parts of the kernels (such as the main loop of GEMM). Some information on maxas and writing kernels for maxas can be found here: https://github.com/NervanaSystems/maxas. The instructions are in SASS which is a level below PTX, but many of the instructions have similar functionality to PTX so you can start there if you're interested in what the kernels are doing.


Minor fixes

This PR adds a /product/random endpoint, with tests
The /product/random endpoint redirect to a random product.  This could
be useful in production if we want to have a random product feature.
More importantly right now, it's helpful for manual testing to be able
to see many product pages quickly.
The PR also changes the title (displayed in the browser) from "React Starter" to "Trailblazers Reviews"

Looks good, I like your usage of the random product for tests


[FALSE-POSITIVE]

Domains or links
Please list any domains and links listed here which you believe are a false positive.
micrsoftalerts.com
More Information
How did you discover your web site or domain was listed here?

Incorrectly marked as Phishing on Phishtank or OpenPhish?

I work for DNSFilter and we have confirmed this site is used for testing responses to Phishing.

Have you requested removal from other sources?
None
Additional context
We use your threat feed as part of DNSFilter Third Party threat vendors

I work for DNSFilter

How can this postulate be verified?

No website found


[FEAT] Try unauthenticated request when Github token is invalid

Is your feature request related to a problem? Please describe.

Currently, we can specify the Github token to allow authenticated requests to the Github API when pulling questionnaires and protocols.
However, if this token is invalid, it still attempts to make the request with the incorrect token.

Describe the solution you'd like

Try unauthenticated request (this is allowed but lower request limits) when the Github token is invalid

Priority
4
Difficulty
4
Additional context
Add any other context or screenshots about the feature request here.

Fixed in #455


pre-compiled binarys

Hi,
The README file says that pre-compiled banaries are available.
Where are they located ?
Thanks,
Alex

http://cole-trapnell-lab.github.io/cufflinks/install/


Bug: data modified fails

After Obsidian is upgraded, the text is formatted using linter. The time cannot be modified, and it will not be automatically added after deletion.
This is the timestamp of the formatted text before the upgrade.

Hey @zhl111 , could you provide your data.json, a sample file to reproduce the issue with, and a list of steps to follow to see the issue?
It looks like there may be issues with using the Linter and properties plugins together.

Could you try what you were doing on Obsidian v1.4.12? It seems there was a bug prior to that that made edits in source mode not show in properties view.

I'm seeing the same issue. I'm using properties as well, but didn't see the issue before today. I updated both Linter and Obsidian today.

They just released a patch for what seems to be this issue @ishirbhan  and @zhl111 . Please verify that you are on Obsidian v1.4.12 or higher.

@ishirbhan  and @zhl111 , unless I hear that the latest public version of Obsidian is experiencing this issue, I plan to close this issue Saturday since Obsidian announced that they fixed an issue where changes to the fronmatter were not being picked up by properties view in v1.4.11.

This appears to be resolved for me

Alrighty. I will go ahead and close this. It can be reopened if for some reason it is still happening.


4 variables without unit

The variables Gender Inequality Index, Political institutions|Equality before law and individual liberty, Ocean|Carbonate Saturation|Aragonite and Ocean|Carbonate Saturation|Calcite  are missing a unit.
Thank you.

These are unitless variables, see this discussion https://github.com/IAMconsortium/common-definitions/pull/73#issuecomment-2012002837


Day 1 link leads to Day 0

On this page:
https://scalacenter.github.io/scala-advent-of-code/2024/
If you click Day 1, it leads you to
https://scalacenter.github.io/scala-advent-of-code/2024/puzzles/day0
Instead of
https://scalacenter.github.io/scala-advent-of-code/2024/puzzles/day01
It also contains a link to Day 2, which does not exist yet (clicking that leads to Day 1)
And the "Next" link at the end of Day 1 leads to Day 1 2023
I would therefore advise to rename Day 0 to "missing day" with as body something along the lines of "Day X will be available soon/ in Y hours"
The next day which would lead to non-existant days would then point to it
(I tried to do so myself, but couldn't figure it out)

I believe we should just delete day0

fixed by #559


Artboard snapping

Closes #1662
Note that this is currently based on the vector-modify branch because of several essential bug fixes and changes.

I'll finish this MR.

!build


set latestTime when new relative filter is created

Description
set latestTime when new relative filter is created
Related Issue :

How Has This Been Tested?

Need additional checks?
Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

[ ] I have read the CONTRIBUTING document.
[ ] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
[ ] I have added tests to cover my changes.

Additional Context

deploy to 3

deploy to 3


Use chunk_size in openai embeddings

Currently the embed_documents (and async version) ignore the chunk_size param.
Update passes it along to the relevant function.

Apologies for the slow review! Pr has some merge conflicts, happy to re-review if you'd like to resolve

closing due to inactivity and merge conflicts


Enhance vaccination certificate acceptance for outside of Europe

At the moment (July 2021) the vaccination certificate is convenient for traveling in the region of Europe (e.g. within the countries accepting the European vaccination certificate).
But the global travel behaviour also has the need to certify the vaccination status, when traveling as European citizen to e.g. Asia, Africa or America. Or a outer european citrizen, traveling through Europe probably wants to certify his/her vacc status to the european authorities using an accepted European smartphone app, e.g. CWA.
So this wish is to enhance the possibility to scan vaccination certificates from outside of Europe for EU-acceptance.
(see: https://www.japantimes.co.jp/news/2021/05/24/national/ana-iata-coronavirus-app-trial/)

There is such a standard called SMART Health Cards: https://smarthealth.cards
Starting in iOS 15 every iOS device can just scan the QR Code and validate the status. Android also supports this, altho being it with special Apps i think.

This standard can do less in comparison to the EU certificates? Who made this a standard? The W3C? (I couldn't find something on their site)

True, its not a standard, but an already existing solution that works on iOS and Android.

Yeah, but it shouldn't be a problem for apple/Google to integrate the eu certificates. Maybe the eu can talk to the W3C to make this a standard. Cause I think there should be one, and the EU Certificate is the most spread one and as much as I understand the best standard.

Is there some technical documentation for the EU Certificates? I don't know where exactly the difference is, i just know that they currently are incompatible.

@idoodler

Is there some technical documentation for the EU Certificates?

https://www.coronawarn.app/en/faq/#eu_dcc =>
https://ec.europa.eu/info/live-work-travel-eu/coronavirus-response/safe-covid-19-vaccines-europeans/eu-digital-covid-certificate_en =>
https://ec.europa.eu/health/ehealth/covid-19_en including "Trust framework and detailed technical specifications"

as far as I understand there is also standardizing activity at the WHO, only the European health certificate activities didn't want to wait for them.
But maybe W3C should align with WHO before creating yet another standard?


Should selectCompositionsByNamebe called for every matching composition?

The comment about nexrender function selectCompositionsByName says that the function should be called for every composition matching the name but the code only matches the first.
If every composition should be matched, the return needs to be moved to outside the for loop.

Indeed, going to push a fix.

Thanks!

Pushed update with v1.8.2


Update API to work with addresses other than localhost

Currently the API will only work if the server is running on localhost. Need to fix this.

see https://github.com/SimonBiggs/scriptedforms/issues/138


How to Reproduce
Used:

"@asyncapi/generator": "^1.8.14"
"@asyncapi/html-template": "^0.23.10"

Entity example:
...
Human:
    title: Human
    description: Human.
    allOf:
        - $ref: '#/components/schemas/Entity'
        - type: object
          properties:
            'parent':
                allOf:
                    - $ref: '#/components/schemas/Human'
            name:
              type: string
              minLength: 1
              maxLength: 16
              readOnly: true
          required:
            - name
...

When I tried to generate the html template I got an exception:
RangeError: Maximum call stack size exceeded
    at PROJECT_PATH\node_modules\@asyncapi\parser\lib\models\schema.js:151:38
    at Array.map (<anonymous>)
    at Schema.allOf (PROJECT_PATH\node_modules\@asyncapi\parser\lib\models\schema.js:151:29)
    at traverseCombinedSchemas (PROJECT_PATH\node_modules\@asyncapi\parser\lib\iterators.js:96:34)
    at traverseSchema (PROJECT_PATH\node_modules\@asyncapi\parser\lib\iterators.js:75:5)
    at PROJECT_PATH\node_modules\@asyncapi\parser\lib\iterators.js:92:7
    at Array.forEach (<anonymous>)
    at checkCombiningSchemas (PROJECT_PATH\node_modules\@asyncapi\parser\lib\iterators.js:91:26)
    at traverseCombinedSchemas (PROJECT_PATH\node_modules\@asyncapi\parser\lib\iterators.js:96:5)
    at traverseSchema (PROJECT_PATH\node_modules\@asyncapi\parser\lib\iterators.js:75:5)
    at recursiveSchemaObject (PROJECT_PATH\node_modules\@asyncapi\parser\lib\iterators.js:195:7)
    at traverseSchema (PROJECT_PATH\node_modules\@asyncapi\parser\lib\iterators.js:68:7)
    at PROJECT_PATH\node_modules\@asyncapi\parser\lib\iterators.js:92:7
    at Array.forEach (<anonymous>)
    at checkCombiningSchemas (PROJECT_PATH\node_modules\@asyncapi\parser\lib\iterators.js:91:26)
    at traverseCombinedSchemas (PROJECT_PATH\node_modules\@asyncapi\parser\lib\iterators.js:96:5)

The code that was used to generate:
const Generator = require('@asyncapi/generator');
Object.keys(asyncApisFiles).forEach(key => {
    let targetDir = `${outputDir}/${asyncApisFiles[key].path}/generated/${key}`;
    const generator = new Generator('@asyncapi/html-template', targetDir, {forceWrite: true});
    generator.generateFromFile(`${outputDir}/${asyncApisFiles[key].path}/${asyncApisFiles[key].hashedFileName}`).then(() => {
        console.log('Html for ' + key + ' schema is generated! Done! ✨');
        let htmlAsString = fs.readFileSync(targetDir + "/index.html", "utf8").toString();
        let chars = {'<img': '<img', '"': '"', '>': '>'};
        htmlAsString = htmlAsString.replace(/(<img)|(")|(>)/g, m => chars[m]);
        fs.writeFileSync(targetDir + "/index.html", htmlAsString, {encoding: "utf8"});
    }).catch(console.error)
});

Unfortunately I cannot provide the whole async-Api file due to NDA.
Expected behavior
Successful build.

@oBJIADo Hi! Currently, we don't support the circular references in some cases, like yours. We only handle circular refs in items and properties - more info is in this issue -> https://github.com/asyncapi/parser-js/issues/293
Did this problem exist before or did it only appear in 0.23.10 version of html-template? I'm asking to make sure if now the html-template version didn't mess something up (and it shouldn't).
Also, why do you make some strange logic for img tags?

@magicmatatjahu Hi!
About "some strange logic for img tags?": this is deprecated logic and should be removed. Never mind this.
The "AllOf" tag was recently added and I don't know if it works before 0.23.

@oBJIADo

The "AllOf" tag was recently added and I don't know if it works before 0.23.

So then it shouldn't work also for previous versions. As I wrote to you, we know about the problem, I even tried to fix it myself, but it's not that easy, because circular references can be everywhere, which complicates things, even when creating a generic solution.

@oBJIADo The only solution I can see now is to change the schema shape to not have circular references for allOf, but I guess in your case this will be very difficult.

@oBJIADo Hello! I have a good news :) We merged new version of generator and new version of html-template that support circular refs :) You can try it with latest generator 1.8.27 and latest html-template 0.24.0. Let us know about any bugs/problems! I remember that we test solution with your example and it handles the circular refs in your case :) I will close it, please reopen if you want.

I finally have time to try it and it works fine. Thanks a lot!


Create a readme file to help new users get going

create a basic readme file that will allow users to setup and run projread

created


GitHub flavored Markdown

Thank you @codemonkey800!

@codemonkey800 Can I close this since we likely don't need it for search?

@codemonkey800 Can I close this since we likely don't need it for search?

sounds good, I had a similar hunch but wanted to see if you could find a use case 😄


Fix CAST transform function for chained transforms

Description
Currently the CAST transform function relies on the caller to call the correct API to perform the cast. When the CAST is chained with other transform functions such as ADD, the next transform function might ignore the output type, and ignore the CAST. E.g. CAST(doubleCol AS INT) + 5 will return the same result as doubleCol + 5, which is unexpected.
This PR fixes the bug by checking the result type and always cast the values.
Also fix a bug for TIMESTAMP cast (missing break;)

Codecov Report

Merging #6941 (88cf497) into master (9becc57) will decrease coverage by 8.01%.
The diff coverage is 41.75%.

@@             Coverage Diff              @@
##             master    #6941      +/-   ##
============================================
- Coverage     73.41%   65.40%   -8.02%
  Complexity       12       12
============================================
  Files          1431     1431
  Lines         70689    70765      +76
  Branches      10232    10252      +20
============================================
- Hits          51897    46282    -5615
- Misses        15344    21157    +5813
+ Partials       3448     3326     -122

Flag
Coverage Δ
Complexity Δ

integration
?
?

unittests
65.40% <41.75%> (-0.04%)
12.00 <0.00> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9becc57...88cf497. Read the comment docs.

@yupeng9 Added more tests and find the issue where cast double -> long -> int is not the same as cast double -> int when the value is out of the int range. Fixed it by always cast first regardless of the type.


Crash in the networkRequestDidComplete method

Sometimes I’m getting crash in the networkRequestDidComplete method, screenshot and backtrace attached. Would you advice me what can be the reason?
* thread #50, queue = 'NSOperationQueue 0x17483d7a0 :: NSOperation 0x171c5b930 (QOS: DEFAULT)', stop reason = EXC_BAD_ACCESS (code=1, address=0xd2800014b40000f8)
    frame #0: 0x000000018896815c libobjc.A.dylib`objc_msgSend + 28
    frame #1: 0x00000001062fe3f8 libswiftCore.dylib`swift::metadataimpl::ValueWitnesses<swift::metadataimpl::ObjCRetainableBox>::initializeWithCopy(swift::OpaqueValue*, swift::OpaqueValue*, swift::TargetMetadata<swift::InProcess> const*) + 24
    frame #2: 0x0000000106193650 libswiftCore.dylib`Swift._VariantDictionaryBuffer.nativeDelete (Swift._NativeDictionaryBuffer<A, B>, idealBucket : Swift.Int, offset : Swift.Int) -> () + 828
    frame #3: 0x0000000106193b90 libswiftCore.dylib`Swift._VariantDictionaryBuffer.nativeRemoveObject (forKey : A) -> Swift.Optional<B> + 872
    frame #4: 0x00000001061807b4 libswiftCore.dylib`Swift._VariantDictionaryBuffer.removeValue (forKey : A) -> Swift.Optional<B> + 148
    frame #5: 0x000000010615ccb8 libswiftCore.dylib`Swift.Dictionary.subscript.setter : (A) -> Swift.Optional<B> + 352
  * frame #6: 0x0000000102de4c9c AlamofireNetworkActivityLogger`NetworkActivityLogger.networkRequestDidComplete(notification=Foundation.Notification @ 0x000000016ef15a80, self=0x000000017025fd70) -> () at NetworkActivityLogger.swift:162
    frame #7: 0x0000000102de6294 AlamofireNetworkActivityLogger`@objc NetworkActivityLogger.networkRequestDidComplete(notification : Notification) -> () at NetworkActivityLogger.swift:0
    frame #8: 0x0000000189e8d5f4 CoreFoundation`__CFNOTIFICATIONCENTER_IS_CALLING_OUT_TO_AN_OBSERVER__ + 20
    frame #9: 0x0000000189e8cd08 CoreFoundation`_CFXRegistrationPost + 400
    frame #10: 0x0000000189e8ca84 CoreFoundation`___CFXNotificationPost_block_invoke + 60
    frame #11: 0x0000000189efb7a8 CoreFoundation`-[_CFXNotificationRegistrar find:object:observer:enumerator:] + 1412
    frame #12: 0x0000000189dd095c CoreFoundation`_CFXNotificationPost + 376
    frame #13: 0x000000018a8e2930 Foundation`-[NSNotificationCenter postNotificationName:object:userInfo:] + 68
    frame #14: 0x0000000102bc557c Alamofire`SessionDelegate.(session=0x00000001743cde30, task=0x000000011c6137a0, error=nil, self=0x0000000110a41810) -> ()).(closure #1) at SessionDelegate.swift:449
    frame #15: 0x0000000102bc50a0 Alamofire`SessionDelegate.urlSession(session=0x00000001743cde30, task=0x000000011c6137a0, error=nil, self=0x0000000110a41810) -> () at SessionDelegate.swift:489
    frame #16: 0x0000000102bc5fdc Alamofire`@objc SessionDelegate.urlSession(URLSession, task : URLSessionTask, didCompleteWithError : Error?) -> () at SessionDelegate.swift:0
    frame #17: 0x000000018a6caad4 CFNetwork`__51-[NSURLSession delegate_task:didCompleteWithError:]_block_invoke.185 + 72
    frame #18: 0x000000018a9c1814 Foundation`__NSBLOCKOPERATION_IS_CALLING_OUT_TO_A_BLOCK__ + 16
    frame #19: 0x000000018a906770 Foundation`-[NSBlockOperation main] + 96
    frame #20: 0x000000018a8f6b28 Foundation`-[__NSOperationInternal _start:] + 612
    frame #21: 0x000000018a9c3bb0 Foundation`__NSOQSchedule_f + 228
    frame #22: 0x0000000106a29a10 libdispatch.dylib`_dispatch_client_callout + 16
    frame #23: 0x0000000106a372e8 libdispatch.dylib`_dispatch_queue_serial_drain + 1140
    frame #24: 0x0000000106a2d634 libdispatch.dylib`_dispatch_queue_invoke + 852
    frame #25: 0x0000000106a39630 libdispatch.dylib`_dispatch_root_queue_drain + 552
    frame #26: 0x0000000106a3939c libdispatch.dylib`_dispatch_worker_thread3 + 140
    frame #27: 0x0000000188fb3100 libsystem_pthread.dylib`_pthread_wqthread + 1096
    frame #28: 0x0000000188fb2cac libsystem_pthread.dylib`start_wqthread + 4

Details:
AlamofireNetworkActivityLogger v2.2
iOS 10.3.2,
Xcode 8.3.2
Swift 3.1
Thanks.

## I'm also facing the same crash issue
AlamofireNetworkActivityLogger
_T030AlamofireNetworkActivityLogger0bcD0C25networkRequestDidComplete33_C4DAE19DB50F7F6C893FBE4CEB3851D9LLy10Foundation12NotificationV12no...

Also seeing this crash.

Also seeing this crash.
iOS 12.2
Swift 4
Xcode 10.2

Also seeing this crash.
iOS 11.4.1 and 12.2
Swift 4 and 5
XCode 10.2

Also seeing this crash.
iOS 12.2
Swift 4 and 5
XCode 10.2

Also seeing this crash.

I'm seeing this as well.

I am also facing crash as well,
Xcode 10.2.1
swift 5

i am not sure but below changes works for me,
startDates = [task: Date()]

@chetandobariya Can you provide some context for that fix? I'm getting Use of unresolved identifier 'task'

@chaseacton  The task is already declared as constant in above guard statement
` @objc
private func networkRequestDidStart(notification: Notification) {
guard let userInfo = notification.userInfo,
let task = userInfo[Notification.Key.Task] as? URLSessionTask,
let request = task.originalRequest,
let httpMethod = request.httpMethod,
let requestURL = request.url
else {
return
}
    if let filterPredicate = filterPredicate, filterPredicate.evaluate(with: request) {
        return
    }

    startDates = [task: Date()]

`
it should not throw error about unresolved identifier.
Insight about fix:
StartDates declared in init() method could take type of   [URLSessionTask: Date] so apparently you should assign the relavant type to the variable.

This change seems to be working for me (NetworkActivityLogger, networkRequestDidComplete(notification:), line ~160):
        if let startDate = startDates[task] {
            elapsedTime = Date().timeIntervalSince(startDate)
            startDates[task] = nil
        }

->
        if let startDate = startDates.removeValue(forKey: task) {
            elapsedTime = Date().timeIntervalSince(startDate)
        }

Also seeing this crash.

Also seeing this crash.

+1 on this

Sames issue arround "startDates = [task: Date()]"
Xcode 10.2.1
swift 5

@konkab Any chance we can get this fixed?

Sorry guys, will try to fix the crash shortly.

Any update on this?

Same here, many crashes

Fixed in latest 3.0.0 release.


Please make new version of .NET Core not bound to linux version

Hello! When I install current version of dotnet to, e.g. OpenSUSE 42.1, it doesn't work. The almost same history is with Linux Mint 18. Dotnet's CLI tools work, but debugging in VSCode doesn't work and appears message: "unsupported linux version".
I like .NET Core very much but I cannot use it with more fresh linuces, then specified in tools and SDK. Please make new version of .NET Core not bound to linux version (and specific versions of libraries) like java and other universal tools/sdks/environments.
Thank you!

Yes. We would like this, too. We haven't decided on a particular solution for this yet.
/cc @blackdwarf

@janvorli , Can you make sure we have an issue tracking this in the coreclr repo. @rv82 , we're actively investigating this.


fix driver crash when change param start_angle while runtime

@JnsHndr, you should set the parameter to double on runtime so you will not get this crashing.  Maybe a more appropriate fix would be to catch the exception on the wrong parameter type and send this to the user or try to handle it by reading it as int instead of forcing people to use it as the start angle.

Thank you @destogl for your suggestion. At the moment I only want to fix the driver crash and do not want to open a major construction site. With this one line fix I handle all parameters in the same way. But yes a better way to handle this kind of crashes and inform the user would be nice in the future. Please feel free to make a change suggestion in/with source code.


Ubuntu 18.04

@Virtuald Can you please let me know the steps you to solve on Ubuntu 18.04. I am facing issue, kindly help out in this concern

Please see https://code.visualstudio.com/updates/v1_86#_linux-minimum-requirements-update

I rolled back to 1.85 for now. I'm sure they'll have a workaround soon.

See https://code.visualstudio.com/updates/v1_86#_linux-minimum-requirements-update to find compatible version for Ubuntu 18.04.


Turtle Addition

Describe your idea
okay 3 ideas:

turtle upgrade, it can hold more then just 2 peripherals
saddle peripheral - you equip a saddle to a turtle and then you can write code so that the turtle mounts a nearby entity
furnace peripheral - since you can already make it a peripheral normally, why not be able to attach it to a turtle?

Describe alternatives you've considered if you've any

alternatively, be able to remove the "inventory" peripheral (the thing on the back of the turtle, that has a chest) and then the turtle wont have a inventory, but will open up another peripheral slot
be able to mount turtles without the peripheral
n/a

Additional context
No response
Linked Issues
No response

tnt peripheral to explode the turtle :trollface:

The first thing is something I'll maybe do.
The second thing isn't something I can do easily. Adding mounting functionality to a block from another mod isn't something I can do
I'll think about the third thing

The first thing is something I'll maybe do.

recently i had the need for more than two upgrade too, till I've realised that this can easily be done programmatically by just how-swap them at need.


feat: Allow passing custom message to literal

Fixes #2974
Uses a message key instead of "invalid_xx" key - because those are only used for required and invalid type using a message key is more in line with other checks

Thanks for this! Error customization is going to be a lot better in Zod 4. I'll merge this but it'll likely undergo some changes before being released.


Add "inf" as a valid float tag

Currently PafPy breaks with a valid "inf" value for float values. This PR corrects the regex in this sense.

Thanks @lucventurini !


Add ELMo BiLM

Description
This PR contains the implementation of ELMo proposed in Peters, Matthew E., et al. "Deep contextualized word representations." NAACL (2018):

add ELMo model: including ELMoBiLM, ELMoCharacterEncoder, ELMoCharacterVocab
add pretrained ELMo models: including all the four pretrained models from allennlp via elmo.get_model API
add ELMo usage notebook to show how to generate sentence representation from raw text using ELMo representation
add some general useful modules in sequence modeling: BiLMEncoder, LSTMPCellWithClip

Checklist
Essentials

[] Changes are complete (i.e. I finished coding on this PR)
[] All changes have test coverage
[] Code is well-documented

Changes

[ ] Feature1, tests, (and when applicable, API doc)
[ ] Feature2, tests, (and when applicable, API doc)

Comments

If this change is a backward incompatible change, why must this change be made.
Interesting edge cases to note here

Job PR-227/5 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-227/5/index.html

@sxjscience @eric-haibin-lin @szha

Job PR-227/54 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-227/54/index.html

Job PR-227/55 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-227/55/index.html

Job PR-227/56 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-227/56/index.html


Update Ravencoin Dev Kit

Replace Ravencore with a new DevKit stack and API
Supports all current Ravencoin features including: (both main and testnet)
ASSETS
TAGS
RESTRICTED ASSETS
MESSAGES
Ravencore
https://github.com/RavenDevKit/ravencore
Bitcore
[GitHub - bitpay/bitcore: A full stack for bitcoin and blockchain-based applications](https://github.com/bitpay/bitcore#bitcore)
Blockbook
https://github.com/trezor/blockbook


Popup hover: target popup disappears when mouseout on parent element

Current Behavior
Popup disappears when hovering on it. I think there maybe an option to showing target popup when hovering over it. As an example implementation: change mouseout to mouseleave and hide only after some delay (300-500 ms should be fine I think)
Reproduction / Steps To Reproduce
No response
Anything else?
No response

This would fall under a feature request more than a bug ticket @rskvazh. The behavior your described was not not the intention of how the current event handlers were implemented, therefore the popup is currently working as intended in this regard.
That said, I understand your use case. We have some updates planned for the popup based on various feature requests and feedback. But not ETA for when this will occur. For now I'll relabel this ticket and keep this for consideration when we reach that point. Given we have a number of these requests I may opt to consolidate these into a singular ticket in the new future.
Either way, keep an eye out here for progress on this going forward.


Ver 2.0.0 - BigUpdate

ApTester 2.0
久しぶりのアップデート(^^♪
Tasks

[x] #10
[ ] #9
[ ] #3
[ ] #7
[ ] #6
[ ] #5
[ ] #4
[ ] #8
[ ] #11

https://stackoverflow.com/questions/938733/total-memory-used-by-python-process

https://docs.python.org/ja/3/library/subprocess.html

https://stackoverflow.com/questions/7989922/opening-a-process-with-popen-and-getting-the-pid

https://stackoverflow.com/questions/31833897/python-read-from-subprocess-stdout-and-stderr-separately-while-preserving-order

https://qiita.com/mokemokechicken/items/a84b0aa96b94d1931f08

https://stackoverflow.com/questions/11937028/poll-method-of-subprocess-popen-returning-none-for-long-process


typeof breaks syntax

VS Code Version: 1.53.2
OS Version: macOS 11.2.1

Steps to Reproduce:

Open new TypeScript file
Paste this code:

function Baz() {}

class Foo {
    bar(): typeof Baz {
        if (true) {
            console.log('Hello World')
        }
        return
    }
}

With typeof:

Without typeof:

As you can see somehow typeof return type in bar function breaks syntax coloring. If you cannot see difference just look at the if and return keywords and compare them by looking at the screenshots provided above. Also, I suggest to copy-paste given code to VS Code and you will see the problem immediately.
/cc @mjbvz

/needsMoreInfo
From report it's not clear what is exact problem here and what is expected behavior? Please provide more details about it.
If it's error on Baz - You cannot use function in type position.
If it's color - type or interface is expected in type position which has own color and in case of typeof it inherits color from variable/function etc. which is used in value postion.

Look at the screenshots I provided or try to copy-paste given code to VS Code and you will see the problem immediately :)
Anyways, I updated description. I expected screenshots provide way more details about it, but seems I was wrong.

Is it supposed to work in Untitled files in Latest VS Code insiders?
I was able to reproduce it in (lnsiders with typescript@next but issue doesn't depend on semantic syntax highlight)
Version: 1.54.0-insider (user setup)
Commit: e590188f17162393f50feec19263398e6fe02d13
Date: 2021-02-26T20:39:10.253Z
OS: Windows_NT x64 10.0.19041
and in master as well (screenshot below).


add chapter 1, chapter 2 with no input, ask_for_title = false,

local function add_chapter_noinput()
    local time_pos = mp.get_property_number("time-pos")
    local chapter_list = mp.get_property_native("chapter-list")
    local chapter_index = (mp.get_property_number("chapter") or -1) + 2

    mp.osd_message(mp.get_property_osd("time-pos/full"), 1)

    table.insert(chapter_list, chapter_index, {title = "", time = time_pos})

    msg.debug("inserting new chapter at ", chapter_index, " chapter_", " time: ", time_pos)

    mp.set_property_native("chapter-list", chapter_list)
    chapters_modified = true

        edited_chapter = chapter_index

        if options.pause_on_input then
            mp.set_property_bool("pause", true)
        end
end

mp.add_key_binding(nil, "add_chapter_noinput", add_chapter_noinput)
;  script-binding chapters/add_chapter  #! [History Chapters Bookmarks] > [Edit Chapters] > Add a Chapter
"  script-binding chapters/add_chapter_noinput #! [History Chapters Bookmarks] > [Edit Chapters] > Add Chapter - Chapter_Number"
I realize there is ask_for_title = true,
one could just set it for ask_for_title = false,
and achieve desired result.  However, one can't do both simultaneously.
Idea being one wants to just quickly add chapter markers with no input and then later possibly edit those chapter titles with just Chapter (Number).

nevermind just always asking for input then hitting ENTER ain't a big deal whatsoever. ...much better than even another key binding..hehe


feature(sheet): options to have "fit" mode with a max-height

The fit mode was great, I was doing my own layout calculation before to feed Sheet with the proper height to it so I can get that behaviour, so it's awesome to have it built in.
However, I'd also think that it would be fitting to have this option to work in tandem with a percentage or height, which would be the biggest point instead of the fit to be the biggest snap points.
One example is that for a list within a sheet. For a short list, I'd like to open up to the content size. And when the list is bigger than certain breakpoint, I'd like the sheet to max to that size, instead of overflowing bigger than the screen size.
Right now I have done some workaround -- I'm using a mixed mode, and put the snap points into a state. I've also put a layout event on my element just inside the Sheet.Frame and calculate its layout and if its more than the percentage that I'd like, I'd set the snap points to percentage eg 80%, else I'd set it to fit. This seems to work decent, but I'm not sure if its performant, and I do sometimes get some error thrown by Animated eg Excessive number of pending callbacks which doesn't happen before adding the layout event.
Would be great if there's a prop for this perhaps somewhere on the layout event within the Sheet, that could do such thing.

Right now there's a hard coded maximum for fit mode that caps out at 100% of the screen height (or parent container height in non-modal mode)
Until there's a way to configure/control that behavior, if you want a simpler workaround you can set a maximum height on the Sheet.Frame in pixels (percentage heights don't work because of the additional wrapper elements) like this:
const { height } = useWindowDimensions();

<Sheet snapPointsMode="fit">
  <Sheet.Frame maxHeight={height * .8}> // constrain height to maximum of 80% screen height
    {/* ... */}
  </Sheet.Frame>
</Sheet>

@nderscore did your latest change address this or no? Converting to a discussion for now.


Introduce a Protobuf message for the query response

In this PR we:

bump module version to 0.10.67-SNAPSHOT;
introduce new FirebaseQueryResponse message;
redocument WebQuery to be for more general HTTP clients, not Spine JS client;
fix unit tests accordingly;
remove unnecessary finals;
replaces all Builders with VBuilders.

@armiol @mdrachuk PTAL


Set up server

[x] Create server
[x] Grant access to Ilya and Jump Servers
[x] Build ceno-client
[x] Create CENO injector
[x] Add Webrecorder (deets? subtask?)

Sandbox server up and running with Ouinet injector, client, and Browsertrix crawler. Initial website scrape on May 5th was successful!


Classic subscription administrator roles (is a misnomer)

'Classic' is a term that is most often used with Azure Service Manager (ex: Classic portal etc).
In this context it seems to be used in another context.  But it is not very clear.    By using the term 'classic' are you meaning to say this only exists with earlier versions?  Or else why use the term.  Why not call it simply 'subscription administrator roles'.  I think clarification would be very helpful to a reader of this article.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 11cda2e8-81c4-ab6a-371e-73506948488f
Version Independent ID: f18c9bf9-a0e7-ea07-144b-c2c7cedc8b6b
Content: Classic subscription administrator roles, Azure RBAC roles, and Azure AD roles
Content Source: articles/role-based-access-control/rbac-and-directory-admin-roles.md
Service: role-based-access-control
GitHub Login: @rolyon
Microsoft Alias: rolyon

@D-Moonesinghe  Thank you for your feedback, i have assigned this issue to our document author @rolyon to further evaluate the feedback and share his insights.

Hi @D-Moonesinghe
Thanks for the feedback. In this context, classic administrators refers to the roles that were part of the initial release of Azure. The Service Administrator and Co-Administrator roles appear on the Classic administrators tab in the Azure portal. You can still use these roles, but a newer authorization system called Azure RBAC is now available. We recommend using these new roles instead. Classic administrators are only needed if you are still using Azure classic deployments.

Hi @D-Moonesinghe
Let us know if you have additional feedback. You can re-open this issue.
thanks

#please-close


Immanuel

Had the scoreboard appear when the number of laps (4) have been run. The board returns dummy data for now but has a functioning restart and exit button.

Looks good, there seems to be a lot of deleted lines of code but I have jumped in your branch and had a look at the project and looks like it is just a lot of bin files and other stuff. Will merge


CaseOfCase UPLC transformation tests.

[x] Chores: extract caseOfCase tests from Simplifier module into a dedicated test module.
[x] Chores: a bit of re-formatting (1st commit)
[x] A test case that demonstrates how CaseOfCase UPLC transformation changes program evaluation result in a presence of Error. (2nd commit)

Untyped Plutus Core
  CaseOfCase
    CaseOfCase/1:                                  OK
    CaseOfCase/2:                                  OK
    CaseOfCase/3:                                  OK
    Transformation doesn't evaluate error eagerly: FAIL
      untyped-plutus-core/test/Transform/CaseOfCase/Test.hs:92:
      expected: EvaluationSuccess (Constant () (Some (ValueOf DefaultUniUnit ())))
       but got: EvaluationFailure
      Use -p '/CaseOfCase/&&/Transformation doesn'\''t evaluate error eagerly/' to rerun this test only.

[x] Fix for the CaseOfCase UPLC transformation to make the test above pass.

Closes #5958

BTW, if anybody is confused why we even bother optimizing ifThenElse b (constr <...>) (constr <...>), then here's where we produce such expressions:

[x] Rebased on master.
[x] Addressed review comments.
[x] Isolated caseOfCase transformation in tests.

BTW, if anybody is confused why we even bother optimizing ifThenElse b (constr <...>) (constr <...>), then here's where we produce such expressions:

@effectfully why are those functions defined the way they are?

@ana-pantilie

Nvm, I see it's because you need to convert from bools to builtin bools. This builtin machinery is so awkward in so many places...

This is very much by design, originally the builtins machinery would return a regular Plutus bool (Scott-encoded back then), but folks wanted me to remove that feature and return built-in bools instead, so I did that in like 2019.
Although now that you mention it, we'll hopefully soon be able to return SOPs from builtins, sounds like we need to come full circle and add builtins returning SOP bools directly.
In any case, for you it's obvious that we should return SOPs and not doing so seems awkward. But for, say, Aiken people it's obvious that we should return constants instead, because they prefer to pretend that SOPs don't exist. It's all subjective.

I've reviewed this once again, I don't think any of my comments are blockers, so let's merge it and do follow-ups:

why did only one budget test changed? That's quite weird
should we avoid generating force-delay when branches are pure?

@zliu41 please still review this PR.

Oh, and great work, thank you @Unisay!

@Unisay just made an issue for you, not urgent. I decided not to dive into it for now, so hopefully you could.


Ensure that tf.exp() uses non-int32 tensors.

Fixes https://github.com/tensorflow/tfjs/issues/445#event-1687297929

This change is

Can we actually solve this by relaxing the requirement in node by casting to a float32 if it's an integer?

We could - but then we differentiate from the TF api:
https://www.tensorflow.org/api_docs/python/tf/exp
On Thu, Jun 28, 2018 at 9:43 AM Nikhil Thorat<EMAIL_ADDRESS>wrote:

Can we actually solve this by relaxing the requirement in node by casting
to a float32 if it's an integer?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/tensorflow/tfjs-core/pull/1129#issuecomment-401098904,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AASsZHJRbHqY-rMsgRay5CnRDRDvDCE9ks5uBQeYgaJpZM4U7tcH
.

Yep, I know, but it's a bit of a strange requirement to have tf.exp not allow integers.

Based on a few reports on github and the mailing list, there is a need to relax tfjs API to allow mixing ints and floats (e.g. allowing mul(int, float), add(int, float) etc). This makes the API different from TF, but not in a way that will hurt the user, yet improve usability of the library.

Closing and fixing in https://github.com/tensorflow/tfjs-node/pull/107


Update app-service-mobile-offline-data-sync.md

Hi @migig, I'm your friendly neighborhood Azure Pull Request Bot (You can call me AZPRBOT). Thanks for your contribution!

This seems like a small (but important) contribution, so no contribution license agreement is required at this point. Real humans will now evaluate your PR.

TTYL, AZPRBOT;

@migig : Thanks for your contribution to the Azure documentation! The author, @wesmc7777, has been notified to review your proposed change.

Unfortunately, this edit can't be accepted for this article, because this query name restriction only applies on the .NET managed client. (For instance, on native iOS, any string length that is supported by Core Data is allowed.) To document query name in this article, we'd either have to enumerate all of the query ID restrictions for all clients, or better yet, include this content in the client how-to documentation.
For the .NET client SDK, it doesn't look like there is a section that goes into the specifics of offline sync currently (https://azure.microsoft.com/en-us/documentation/articles/app-service-mobile-dotnet-how-to-use-client-library/). You could either contribute some content there, or we will just make an note to add documentation on query names once there is specific offline sync content in that article.

#please-close

Okay makes sense!


Is Integrated Security supported ??

I was able to successfully test connection using below connection string
Data Source=<some_server>;Initial Catalog=<some_db>;Integrated Security=True;Trust Server Certificate=True;

But fails during scaffolding with error
One or more errors occurred. (No scaffolded files found in output dir.)

Note - I am able to make a connection to some other sql server using Basic Auth

It should work yes. Unfortunately the error "No scaffolded files found in output dir" isn't very informative, and hides the true issue. The upcoming update will show the actual error/issue the app faced during scaffolding.
I'll try to test this and see what's going on.

Thanks for the update.

@fasilmarshooq can you try the new version and let me know if the issue still persists for you.

This issue should be fixed. Please feel free to reopen if the problem persists with the new version.

@tareqimbasher   i get this scaffolding issue
Output: To protect potentially sensitive information in your connection string, you should move it out of source code. You can avoid scaffolding the connection string by using the Name= syntax to read it from configuration - see https://go.microsoft.com/fwlink/?linkid=2131148. For more guidance on storing connection strings, see https://go.microsoft.com/fwlink/?LinkId=723263.
Skipping foreign key 'FK_auth_role_assigned_roles_auth_roles__RoleId' on table 'dbo.auth_role_assigned_roles' since it is a duplicate of 'FK_auth_role_assigned_roles_auth_roles__AssignedRoleId'.
Skipping foreign key 'FK_auth_user_outofoffice_mapping_auth_users_id' on table 'dbo.auth_user_outofoffice_mapping' since it is a duplicate of 'FK_auth_user_outofoffice_mapping_auth_users_assignedid'.
The column 'dbo.configurable_landing_page_happyhour_setting.is_city_accepted' would normally be mapped to a non-nullable bool property, but it has a default constraint. Such a column is mapped to a nullable bool property to allow a difference between setting the property to false and invoking the default constraint. See https://go.microsoft.com/fwlink/?linkid=851278 for details.
The column 'dbo.configurable_landing_page_hotel_setting.is_hotel_enabled' would normally be mapped to a non-nullable bool property, but it has a default constraint. Such a column is mapped to a nullable bool property to allow a difference between setting the property to false and invoking the default constraint. See https://go.microsoft.com/fwlink/?linkid=851278 for details.
Could not find type mapping for column 'dbo.experiences.coordinate' with data type 'geography'. Skipping column.
Could not find type mapping for column 'dbo.geo_informations.geo_info' with data type 'geography'. Skipping column.
Unable to scaffold the index 'six01_geography_informations'. The following columns could not be scaffolded: geo_info.
Could not find type mapping for column 'dbo.geo2_area.area_center_coordinates' with data type 'geography'. Skipping column.
Could not find type mapping for column 'dbo.geo2_area.area_boundary' with data type 'geography'. Skipping column.
Could not find type mapping for column 'dbo.geo2_area.map_center_coordinates' with data type 'geography'. Skipping column.
Unable to scaffold the index 'SIndx_geo2_area(area_center_coordinates)'. The following columns could not be scaffolded: area_center_coordinates.
Unable to scaffold the index 'SIndx_geo2_area(map_center_coordinates)'. The following columns could not be scaffolded: map_center_coordinates.
Could not find type mapping for column 'dbo.geo2_city.city_center_coordinates' with data type 'geography'. Skipping column.
Could not find type mapping for column 'dbo.geo2_city.map_center_coordinates' with data type 'geography'. Skipping column.
Could not find type mapping for column 'dbo.geo2_city.city_boundary' with data type 'geography'. Skipping column.
Unable to scaffold the index 'IX01_geo2_city(city_center_coordinates)_Spatial'. The following columns could not be scaffolded: city_center_coordinates.
Unable to scaffold the index 'SIndx_geo2_city(map_center_coordinates)'. The following columns could not be scaffolded: map_center_coordinates.
Could not find type mapping for column 'dbo.geo2_continent.continent_boundary' with data type 'geography'. Skipping column.
Could not find type mapping for column 'dbo.geo2_continent.map_center_coordinates' with data type 'geography'. Skipping column.
Could not find type mapping for column 'dbo.geo2_country.country_center_coordinates' with data type 'geography'. Skipping column.
Could not find type mapping for column 'dbo.geo2_country.country_boundary' with data type 'geography'. Skipping column.
Could not find type mapping for column 'dbo.geo2_country.map_center_coordinates' with data type 'geography'. Skipping column.
Unable to scaffold the index 'SIndx_geo2_country(country_center_coordinates)'. The following columns could not be scaffolded: country_center_coordinates.
Unable to scaffold the index 'SIndx_geo2_country(map_center_coordinates)'. The following columns could not be scaffolded: map_center_coordinates.
Could not find type mapping for column 'dbo.geo2_county.county_center_coordinates' with data type 'geography'. Skipping column.
Could not find type mapping for column 'dbo.geo2_county.county_boundary' with data type 'geography'. Skipping column.
Could not find type mapping for column 'dbo.geo2_county.map_center_coordinates' with data type 'geography'. Skipping column.
Unable to scaffold the index 'SIndx_geo2_county(county_boundary)'. The following columns could not be scaffolded: county_boundary.
Unable to scaffold the index 'SIndx_geo2_county(county_center_coordinates)'. The following columns could not be scaffolded: county_center_coordinates.
Unable to scaffold the index 'SIndx_geo2_county(map_center_coordinates)'. The following columns could not be scaffolded: map_center_coordinates.
Could not find type mapping for column 'dbo.geo2_landmark.landmark_center_coordinates' with data type 'geography'. Skipping column.
Could not find type mapping for column 'dbo.geo2_landmark.landmark_boundary' with data type 'geography'. Skipping column.
Unable to scaffold the index 'SIndx01_geo2_landmark(landmark_center_coordinates)'. The following columns could not be scaffolded: landmark_center_coordinates.
Could not find type mapping for column 'dbo.geo2_local_essential.local_essential_center_coordinates' with data type 'geography'. Skipping column.
Could not find type mapping for column 'dbo.geo2_region.region_center_coordinates' with data type 'geography'. Skipping column.
Could not find type mapping for column 'dbo.geo2_region.region_boundary' with data type 'geography'. Skipping column.
Could not find type mapping for column 'dbo.geo2_region.map_center_coordinates' with data type 'geography'. Skipping column.
Unable to scaffold the index 'SIndx_geo2_region(map_center_coordinates)'. The following columns could not be scaffolded: map_center_coordinates.
Unable to scaffold the index 'SIndx_geo2_region(region_boundary)'. The following columns could not be scaffolded: region_boundary.
Unable to scaffold the index 'SIndx_geo2_region(region_center_coordinates)'. The following columns could not be scaffolded: region_center_coordinates.
Could not find type mapping for column 'dbo.geo2_state.state_center_coordinates' with data type 'geography'. Skipping column.
Could not find type mapping for column 'dbo.geo2_state.state_boundary' with data type 'geography'. Skipping column.
Could not find type mapping for column 'dbo.geo2_state.map_center_coordinates' with data type 'geography'. Skipping column.
Unable to scaffold the index 'SIndx_geo2_state(map_center_coordinates)'. The following columns could not be scaffolded: map_center_coordinates.
Unable to scaffold the index 'SIndx_geo2_state(state_center_coordinates)'. The following columns could not be scaffolded: state_center_coordinates.
Could not find type mapping for column 'dbo.geo2_tourist_area.tourist_area_center_coordinates' with data type 'geography'. Skipping column.
Could not find type mapping for column 'dbo.geo2_tourist_area.tourist_area_boundary' with data type 'geography'. Skipping column.
Could not find type mapping for column 'dbo.geo2_tourist_area.map_center_coordinates' with data type 'geography'. Skipping column.
Could not find type mapping for column 'dbo.product_hotels.geo_info' with data type 'geography'. Skipping column.
Unable to scaffold the index 'SIndx_product_hotels(geo_info)'. The following columns could not be scaffolded: geo_info.
Could not find type mapping for column 'dbo.sanctioned_territory.sanctioned_territory_boundary_geography' with data type 'geography'. Skipping column.
Could not find type mapping for column 'dbo.sanctioned_territory.sanctioned_territory_boundary_geography_orientation' with data type 'geography'. Skipping column.
Could not find type mapping for column 'dbo.sanctioned_territory.sanctioned_territory_boundary_geometry' with data type 'geometry'. Skipping column.
Unable to scaffold the index 'IX01_SPT_sanctioned_territory__sanctioned_territory_boundary_geography_orientation'. The following columns could not be scaffolded: sanctioned_territory_boundary_geography_orientation.
Could not find type mapping for column 'dbo.tourist_region.tourist_region_center_coordinates' with data type 'geography'. Skipping column.
Could not find type mapping for column 'dbo.tourist_region.tourist_region_boundary' with data type 'geography'. Skipping column.
Could not find type mapping for column 'dbo.tourist_region.map_center_coordinates' with data type 'geography'. Skipping column.
System.InvalidOperationException: The key {'MailId'} cannot be added to keyless type 'EmailMailQueue (Dictionary<string, object>)'.
   at Microsoft.EntityFrameworkCore.Metadata.Internal.EntityType.AddKey(IReadOnlyList`1 properties, ConfigurationSource configurationSource)
   at Microsoft.EntityFrameworkCore.Metadata.Internal.EntityType.Microsoft.EntityFrameworkCore.Metadata.IMutableEntityType.AddKey(IReadOnlyList`1 properties)
   at Microsoft.EntityFrameworkCore.Scaffolding.Internal.RelationalScaffoldingModelFactory.VisitForeignKey(ModelBuilder modelBuilder, DatabaseForeignKey foreignKey)
   at Microsoft.EntityFrameworkCore.Scaffolding.Internal.RelationalScaffoldingModelFactory.VisitForeignKeys(ModelBuilder modelBuilder, IList`1 foreignKeys)
   at Microsoft.EntityFrameworkCore.Scaffolding.Internal.RelationalScaffoldingModelFactory.VisitDatabaseModel(ModelBuilder modelBuilder, DatabaseModel databaseModel)
   at Microsoft.EntityFrameworkCore.Scaffolding.Internal.RelationalScaffoldingModelFactory.Create(DatabaseModel databaseModel, ModelReverseEngineerOptions options)
   at Microsoft.EntityFrameworkCore.Scaffolding.Internal.ReverseEngineerScaffolder.ScaffoldModel(String connectionString, DatabaseModelFactoryOptions databaseOptions, ModelReverseEngineerOptions modelOptions, ModelCodeGenerationOptions codeOptions)
   at Microsoft.EntityFrameworkCore.Design.Internal.DatabaseOperations.ScaffoldContext(String provider, String connectionString, String outputDir, String outputContextDir, String dbContextClassName, IEnumerable`1 schemas, IEnumerable`1 tables, String modelNamespace, String contextNamespace, Boolean useDataAnnotations, Boolean overwriteFiles, Boolean useDatabaseNames, Boolean suppressOnConfiguring, Boolean noPluralize)
   at Microsoft.EntityFrameworkCore.Design.OperationExecutor.ScaffoldContextImpl(String provider, String connectionString, String outputDir, String outputDbContextDir, String dbContextClassName, IEnumerable`1 schemaFilters, IEnumerable`1 tableFilters, String modelNamespace, String contextNamespace, Boolean useDataAnnotations, Boolean overwriteFiles, Boolean useDatabaseNames, Boolean suppressOnConfiguring, Boolean noPluralize)
   at Microsoft.EntityFrameworkCore.Design.OperationExecutor.ScaffoldContext.<>c__DisplayClass0_0.<.ctor>b__0()
   at Microsoft.EntityFrameworkCore.Design.OperationExecutor.OperationBase.<>c__DisplayClass3_0`1.<Execute>b__0()
   at Microsoft.EntityFrameworkCore.Design.OperationExecutor.OperationBase.Execute(Action action)
The key {'MailId'} cannot be added to keyless type 'EmailMailQueue (Dictionary<string, object>)'.
Error: ```

Also I got Ef core error initially `The Entity Framework tools version '8.0.4' is older than that of the runtime '8.0.8'` would be great if its installed as part of app installation.

unfortunately i cant reopen the issue https://github.com/orgs/community/discussions/41926#discussioncomment-6975082

FYI, windows auth is working , the issues is when the db has geo related columns

Thanks @fasilmarshooq for confirming windows security works, I'll create a new issue for the spatial data scaffolding problem. I'll test it out and see why EF is having trouble scaffolding it. Thanks for reporting!

Also, can we have windows auth in the drop-down instead of the user manually fiddling with the connection string ??

Yep, will do.


Feature blog for KEP-3333 Retroactive default StorageClass assignment

KEP: https://github.com/kubernetes/enhancements/issues/3333
/sig storage

Hello. As discussed in Slack, this Feature Blog wasn't being tracked because the PR was not known to Comms when the deadline arrived.
Since this has been advancing in terms of review, I've added it to the Feature blog series, with a publication date of 5-1-2023 (January 5).

/assign @divya-mohan0209
Divya, please review this and ideally get it to the point that it's just waiting for a /approve.
/hold
We should not merge any post-release announcements until the release happens.

@RomanBednar if you could go through the above suggestions it would be great, with the release date being tomorrow we're trying to have everything ready to publish, with incremental improvements still being possible. Thanks!

Thanks @RomanBednar !

@divya-mohan0209 @fsmunoz I've addressed the remaining comments, hope it's ok now, thank you for the reviews!

@divya-mohan0209 @fsmunoz sorry for the delay, some of the reviews got hidden and I haven't noticed them until now :/ I hope I addressed all of them, PTAL if the current version is ok.

/hold
Two reasons:

the article date is wrong
Kubernetes v1.26 is not yet released

Even when the article date is set to 2023-01-05, we should maintain the hold unless v1.26 has already been released.

Also, if you're happy with how this looks once you've edited it, please squash down to 1 commit.

@sftim @divya-mohan0209 please review again.

Thanks
/lgtm
/approve

v1.26 released, everything else in place, removing hold.
/unhold


Integrate Jest

integrate jest as testing library
write first tests
integrate into CI
integrate with sonar coverage

closed with #17 and #19


Fix bug of file_upload

Adjust the endpoint according to Hub API Endpoints (I think there is a typo in the document because {type} needs a suffix "s" to be a valid endpoint.)
Change repo_file_upload to file_upload

Looks good, thanks!


[Bug]: shopify app deploy automatically selects client-id, if not provided

Please confirm that you have:

[x] Searched existing issues to see if your issue is a duplicate. (If you’ve found a duplicate issue, feel free to add additional information in a comment on it.)
[x] Reproduced the issue in the latest CLI version.

In which of these areas are you experiencing a problem?
App
Expected behavior
shopify app deploy should deploy the config specified via shopify app config use or --config=<path to file>
Actual behavior
I have two app .toml files for dev and prod. I previously was swapping between them with shopify app config use and was able to deploy both apps separately just with shopify app deploy. Currently, shopify app dev and shopify app deploy seem "locked" to the dev version.
Both dev and deploy always use the api key from the dev version and overwrite .../shopify-cli-app-nodejs/config.json with the incorrect app id, even if I set it again with shopify app config use.

dev shows the warning (The flag api-keyhas been deprecated in favor ofclient-id), deploy` does not
explicitly setting --config does not work
explicitly setting --client-id does work
changing .env files did not work
clearing all shopify npm cache directories did not work

These commands work fine:

shopify app versions list
shopify app info

I'm assuming it's a local setup thing, but as far as I know, I didn't change anything except upgrading the cli, so others might run into this.
Verbose output
Sorry, I don't have time to redact the output right now! All I've got is:

FindApp runs with the wrong apiKey value.

Reproduction steps
I'll add exact steps later if I can but so far all I know is:

try to run shopify app deploy in a project with multiple deployed .toml configurations, switching between them with shopify app config use

Operating System
Mac 14.6 (23G80)
Shopify CLI version (shopify --version)
@shopify/cli/3.71.4 darwin-arm64 node-v20.16.0
Shell
zsh
Node version (run node -v if you're not sure)
20.16.0
What language and version are you using in your application?
node v20.16.0

Hi @Goondrious!
So this is likely because you have some api key defined in your ENV, this line is the hint:

The flag api-key has been deprecated in favor of client-id

Look for SHOPIFY_API_KEY in your terminal env (echo $SHOPIFY_API_KEY) or in any process that might be setting this value (SHOPIFY_API_KEY). This ENV value takes precedence over any other defined via config use.
Let me know if this works!

With all the random testing I did, I somehow did not think of directly checking system ENV. I have no idea how it go there. Thank you!!!


Better page breaking - particularly for the first section

There's quite a bit of pre-ample in the Gutenberg books - I think it's possible to skip this - just not sure how at the moment.

I think in addition to this each page change should do a complete refresh otherwise the last page merges with the new one like you can see in this picture of the Rabbits book

[ ] Add also page indicator below:  1 / total pages
[ ] Save in NVS the last read page in a key/value table

Key: name of EPub  Value: last page open
so you can open the book and continue reading after resetting he device.

That's interesting - how did you make it do that? Was it after the device had gone to sleep and woken up?
It's supposed to save the current state of the display so when it wakes up it knows how to update the screen - I wonder if that's not working when using spiffs.

That's done simply loading the book with Spiffs and just going to next page till that one. But you can also experience it when going backwards.
If the new page are is smaller than the last page, then that part that is not drawn stays and blends with the new one.
I believe an epd_fullclear is needed between pages.

Very weird managed to recreate it after going into deep sleep with SPIFFS. Makes me think that the EPD hydrate and dehydrate is not working properly.
There's a clear screen at the start of the page render, but if the back buffer has not been restored properly it won't know to update the area of the screen that has old content in it.

For some reason the code in EpdRenderer.h -  virtual void dehydrate() isn't working... when it tries to save the back buffer it doesn't write any bytes.
I (36947) main: Saving state
I (36947) EPD: Dehydrating EPD
I (37477) EPD: Front buffer compressed size: 35875
I (38167) EPD: Front buffer saved 35875
I (38697) EPD: Back buffer compressed size: 35875
I (40557) EPD: Back buffer saved 0
I (40557) EPD: Dehydrated EPD

This may help - b6a0b685943ecb4fa68f0305964c2334fb55abbc
I suspect that a move to LittleFS would improve things considerably. From what I can see SPIFFS is struggling to find free space for the file so although it opens ok, it doesn't actually write any data.
I'll add more graceful handling when it does fail so that it gets back into a sensible state.

I've added this issue to move over to LittleFS - https://github.com/atomic14/diy-esp32-epub-reader/issues/13

Great. When I find some time I will fork this and try to add touch.
Only show stopper is that touch consumes aprox. 1.5 mA/Hour so it kind of drains your battery. But it will be nice to have it, since with that we can have more input options, like for example a slider from 1 to last page, so you can jump to a page.
Also like I commented up there, to save last page read, will be a great usability help so you can keep on reading after deepsleep without need to navigate again to that page.

Also like I commented up there, to save last page read, will be a great usability help so you can keep on reading after deepsleep without need to navigate again to that page.

I'll take a look at this as it should be working.

Additionally and not strictly related to this issue, there are some book cover images that are not being loaded (Maybe it's because I'm on SPIFFS?)
For example one of my favorite poetry books:
https://sync.luckycloud.de/d/bf3a5b1d14f949b89955/?p=%2FePub&mode=list
poesia_aPizz.epub
Other than that the book is being opening correctly and I can read it.

I've added an issue for the missing cover image - https://github.com/atomic14/diy-esp32-epub-reader/issues/14
It may be that the image is not compatible - progressive jpegs are not supported. But I'll investigate tonight.

Fixed

About #14 please don't close so fast before an additional test
It's not working for me and caught in a restart loop again.

Updated a comment on issue #14 - the book has some invalid utf-8 sequences which was crashing the font code in epdiy - that's now fixed upstream so should work if the git submodules update properly (having some issues with my understanding of git submodules...)

Ah now. Great you are doing it good, you just have to update it locally and then push the branch like you did.
The detail is that to update from my side I just need to do:
$ git submodule update
From https://github.com/atomic14/epdiy

branch            07925dc10e8bdf3c537270605bbe50705c9fdff2 -> FETCH_HEAD
Submodule path 'lib/epdiy/epdiy': checked out '07925dc10e8bdf3c537270605bbe50705c9fdff2'

Not: git submodule update --remote (That won't update anything)


[backport] Skip some LOBPCG tests failing with SciPy 1.11

Backport of #7924 by @asi1024

/test mini,head

Test failure seems unrelated.


Lazy instantiation of a function object's 'length' and 'prototype' properties

Memory consumption on IoT.js
test_cwd.js

Heap stats:

Peak allocated chunks count: 1105 -> 1040

Peak allocated: 70184 bytes -> 66024 bytes

Peak waste: 865 bytes -> 865 bytes

Pools stats:

Peak pools: 441 -> 342
Peak allocated chunks: 3525 -> 2734

Freya's report:

mem_pools_alloc: 27.5 kb -> 21.3 kb

mem_heap_alloc_block_try_give_memory_back: 52.9 kb -> 52.9 kb

total peak: 247.8 kb -> 243.7 kb

test_httpserver.js:

Heap stats:

Peak allocated chunks count: 2478 -> 2216

Peak allocated: 157288 bytes -> 140892 bytes

Peak waste: 1740 bytes -> 1740 bytes

Pools stats:

Peak pools: 1441 -> 1188
Peak allocated chunks: 11527 -> 9503

Freya's report:

mem_pools_alloc: 90.0 kb -> 74.2 kb

mem_heap_alloc_block_try_give_memory_back: 89.6 kb -> 89.6 kb

total peak: 391.4 kb -> 375.5 kb

                           Benchmark |         RSS
(+ is better) |        Perf
(+ is better) |
                           --------- |                          --- |                         ---- |
                          3d-cube.js |          132->   132 (0.000) |       3.7824->3.6936 (2.348) |
              access-binary-trees.js |           88->    88 (0.000) |       2.8656->2.7392 (4.411) |
                  access-fannkuch.js |           40->    40 (0.000) |      10.1256->9.8208 (3.010) |
                     access-nbody.js |           64->    64 (0.000) |        4.764->4.6408 (2.586) |
         bitops-3bit-bits-in-byte.js |           36->    36 (0.000) |      3.3416->3.3464 (-0.144) |
              bitops-bits-in-byte.js |           36->    36 (0.000) |      4.4608->4.5408 (-1.793) |
               bitops-bitwise-and.js |           32->    32 (0.000) |       4.3128->4.1784 (3.116) |
            controlflow-recursive.js |          220->   220 (0.000) |       3.3624->3.2304 (3.926) |
                date-format-xparb.js |          100->    92 (8.000) |      3.0136->3.1936 (-5.973) |
                      math-cordic.js |           48->    48 (0.000) |        4.976->4.9176 (1.174) |
                math-partial-sums.js |           40->    40 (0.000) |        2.712->2.6408 (2.625) |
               math-spectral-norm.js |           52->    52 (0.000) |       3.3096->3.2032 (3.215) |
                     string-fasta.js |           52->    52 (0.000) |      27.4592-> 27.18 (1.017) |
                     Geometric mean: |       RSS reduction: 0.6393% |            Speed up: 1.5375% |

Good to me

LGTM


[assets-controllers] Fetch token rates in batches to avoid hitting limits

We currently fetch the token rates for all tokens (for a given network) in a single call. However, the CoinGecko API we're currently using supports a maximum of 10 token addresses in a single request. Beyond that, it throws an error. The price API that we're considering migrating to has a similar limit of 15 addresses.
We should update the TokenRatesController to batch requests by a configurable amount (10 for now) to avoid hitting this error.

Also, for the Coingecko it's a deliberate choice to limit to 10, we receive a clear error message :
{"timestamp":"2023-11-23T14:08:29.074+00:00","error_code":10012,"status":{"error_message":"Number of contract addresses in the request exceeds the allowed limit (10 addresses). Please note that to ensure fair usage and prevent abuse, the allowed limit may be adjusted periodically. To enjoy higher allowed limit, please subscribe to Demo plan or a paid plan at https://www.coingecko.com/en/api/pricing"}}

For the Codefi Price API, the limit of 15 was found by trial and error as the result is gateway timeout. So we can't tell if it's a deliberate limit or one that's caused by a memory limit or anything else including a bug.
So, on the Codefi API, instead of just sending the requests by chunks, we also have to investigate the root cause for the api non responding in some cases. It may reveal an issue that has to be fixed.
If we realise that this issue can't easily be fixed but has a deterministic way to happen, like always over 15 tokens) then we should return an error message like Coingecko, clearly indicating the limit.

Note that I think Coingecko implemented that limit recently and have not documented it yet. I can't find anything about it.

Mobile team last update:
We change from the coin gecko endpoint to price api endpoint as we can verify here, and it's working as expected

Updates:
Validated again from my side, and it seems like the issue has been solved
https://api.coingecko.com/api/v3/simple/token_price/ethereum?contract_addresses=0x0104f019c8889BdabC62aE6C7b84F6f2ed351133,0x767FE9EDC9E0dF98E07454847909b5E959D7ca0E,0x6123B0049F904d730dB3C36a31167D9d4121fA6B,0x1f9840a85d5aF5bf1D1762F925BDADdC4201F984,0xCc802c45B55581713cEcd1Eb17BE9Ab7fcCb0844,0x75231F58b43240C9718Dd58B4967c5114342a86c,0xae7ab96520DE3A18E5e111B5EaAb095312D7fE84,0x95aD61b0a150d79219dCF64E1E6Cc01f0B64C4cE,0x2b591e99afE9f32eAA6214f7B7629768c40Eeb39,0x0001A500A6B18995B03f44bb040A5fFc28E45CB0,0x9f8F72aA9304c8B593d555F12eF6589cC3A579A2&vs_currencies=eth
<img width="1035" alt="Screenshot 2023-11-30 at 15 47 48" src="https://github.com/MetaMask/core/assets/12678455/50033fcc-f223-4430-9074-b2097b7048ab">

Rate limit occur error after making five consecutive calls. The error response is as follows:
{   status: {     error_code: 429,     error_message: "You've exceeded the Rate Limit. Please visit https://www.coingecko.com/en/api/pricing to subscribe to our API plans for higher rate limits."   } }


Request: setup/compilation walkthrough

Lars, I found this project through your NIK 2018 paper while looking for information about GPU affinity in Vulkan.  It looks very interesting!  You have clearly put in a lot of work on this project.
Context: I am hoping to adapt Bonzomatic to use Vulkan (instead of GLFW+OpenGL) to render a fragment shader in a full-screen quad, locked to one of two GPUs in the test system (for a Shader Showdown at the Demosplash demoparty).  I think bp+vmgpu can get me most of the way there.
Request: Would you be willing to expand the README to include a walkthrough of how you set up your system and compiled bp and vmgpu?  For example, does the find_package(Vulkan) in the CMake file refer to the LunarG SDK or some other Vulkan package?  What OS and compiler did you use?  Do bp and vmgpu require C++11 or another specific C++ language version?  What extra libraries did you need to install (I see the dependency on shaderc).  What graphics cards did you use, and with which drivers?
Thank you for considering this request!

Do you use windows or linux? I used to compile this either on linux or with mingw-w64 with MSYS2. MSYS2 have a package manager (pacman) and packages for boost, qt, etc, so it is easy to set up.
I have spent a few hours now trying to build with visual studio. Gotten to the point where I successfully built bp. I added glm and shaderc as submodules of bp so it is easier to compile it (see https://github.com/larso0/bp). I have issues with boost and visual studio so haven't been able to compile vmgpu yet.
I'll probably sit down and update the README of both bp and vmgpu the coming weekend. But I'll probably leave compiling boost with visual studio as an exercise to the user.
My master thesis (pdf) describes bp and vmgpu in more detail (with code examples) than the NIK paper, if you're interrested.

@larso0 Thank you!  I am currently on Windows 7 x64 and have both Cygwin and mingw-w64/msys2 available.  I am currently compiling Bonzomatic on mingw-w64's mingw32 (i686, non-MSYS), so I am glad to hear that you have also used mingw-w64.  At Demosplash, we will be using a Windows 10 x64 box.  I am perfectly comfortable not touching Visual Studio :D , so no worries there.
Thanks for the work; I look forward to the README updates!  I will check out the thesis and examples.  I also spent some quality time with OGLDEV's Vulkan tutorials 50--52 yesterday and it is ... quite an API!

I have updated the readme now. Though I have an issue causing black screen. I probably broke something when I added shaderc etc as submodules. But it is probably not shaderc that is the problem, as the shaders are compiled with glslangvalidator and loaded from file. Maybe just some difference in the math library versions (glm) causing the geometry to become not visible. I don't have the motivation to sit down and fix this for now.

Okay I got some motivation after a good night sleep. The GLM version was the problem. I changed the glm submodule of bp to point to GLM version 0.9.8 specifically which works as intended. For future reference, the version of boost: 1.66 and qt: 5.10.1.

Lars, thank you!  I will give it a try.


ROX-11537 - Fix test falke TestExecIntoPodNameEventPolicy

Description
The test TestExecIntoPodNameEventPolicy failed sometimes in openshift-ci.
I tested the test locally and run the complete go-unit-test suite but never saw the failure. It is likely that the timeout of 1 second was too short in a resource limited environment.
Also it was reported that openshift-ci is slower compared to our previous CI solution.
If the test still fails the timeout must be increased even further or more investigations are necessary.
Checklist

[x] Investigated and inspected CI test results
[ ] Unit test and regression tests added
[ ] Evaluated and added CHANGELOG entry if required
[ ] Determined and documented upgrade steps
[ ] Documented user facing changes (create PR based on openshift/openshift-docs and merge into rhacs-docs)

If any of these don't apply, please comment below.
Testing Performed

Run test multiple times locally
Run CI pipeline

@SimonBaeumer

The test TestExecIntoPodNameEventPolicy failed sometimes in openshift-ci. I tested the test locally and run the complete go-unit-test suite but never saw the failure.

did you see this warning when you tested locally? I wonder whether this is a hint or a red herring?
admission-control/manager: 2022/06/27 16:47:07.371882 evaluate_runtime.go:124: Warn: Deployment for stackrox/sensor-74f6965874-qckz6 not found. Policies with deploy-time fields for kubernetes event CREATE:namespace/stackrox/pods/sensor-74f6965874-qckz6/exec will be detected in background


fix: MD039/no-space-in-links

Spaces inside link text

@nschonni : Thanks for your contribution! The author, @jeevansd, has been notified to review your proposed change.

We're reviewing this PR for impact.

Thank for this update @nschonni ! Much appreciated.


Add address, organization and person schemas to common folder

Add address, organization and person schemas to common folder from platform XDM schemas.

@jwen-adobe is this now complete, can we close it?

@cdegroot-adobe closed


✨ Uplift sigs.k8s.io/cluster-api/test to v1.0.1

Couldn't uplift capi/test to v1.0.1 in #381 since CAPI folks missed test/v1.0.1 tag in the release somehow and now they have added it.

/test-v1b1-integration

/test-v1b1-integration

/test-v1b1-e2e

/test-v1b1-e2e

/test-v1b1-e2e


Fix/improved test for overriding css style

Improved tests for Challenge Waypoint: Override Styles in Subsequent CSS

Created a new test to check if both classes belong to the same element

closes #5054

@KrishnaYeolekar Can you squash your commits?
This works well, we just want to keep our commit history clean.

Also, you may need to rebase your local fork.

@SaintPeter squashed them and rebased the local fork.

Looks good, thanks! :+1:


Upgrade swagger 2.0 to openapi 3.0

Summary
Openapi 3.0 is the updated version of swagger 2.0.  It has several nice features to make api documentation easier so we should investigate how much work it will take to migrate and whether it is possible with cosmos still using swagger 2.0.
Also should take note of: https://github.com/cosmos/cosmos-sdk/issues/7620
Here is an article explaining some of the differences between swagger 2.0 and openapi 3.0: https://medium.com/@tgtshanika/open-api-3-0-vs-swagger-2-0-94a80f121022
Problem Definition

Proposal
Investigate the feasibility of upgrading swagger 2.0 to openapi 3.0 and make the migration if benefitial.

For Admin Use

[ ] Not duplicate issue
[ ] Appropriate labels applied
[ ] Appropriate contributors tagged
[ ] Contributor assigned/self-assigned

This appears to now be a copy of #2063.
Closing this one in favor of that one.