RFC: TemplateExpression with parameters

@gm89uk @Andrea-gm here's an attempt at https://github.com/MilesCranmer/PySR/discussions/787. If you get a chance to try, I'd be very interested to hear what you think.
The API is like this (v2). We request an expression f and parameter vectors p1, p2, and p3, via the type parameter {(:f,), (:p1, :p2, :p3)}. We then write out how a function specifying how these are combined, and how the expression is evaluated. We then specify the length of each parameter vector (which here is the number of unique parameter classes):
structure = TemplateStructure{(:f,), (:p1, :p2, :p3)}(
    function ((; f), (; p1, p2, p3), (x, y, class))
        p1[class] * x^2 + f(x, y, p2[class]) - p3[1]
    end;
    num_parameters=(; p1=10, p2=10, p3=1)
)

This is equivalent to (for p1 = $\alpha$, p2 = $\beta$, p3 = $\gamma$):
$$
\alpha_{\text{class};i} x^2 + f(x_i, y_i, \beta_{\text{class};i} ) - \gamma
$$
for each datapoint $i$ with features $x_i$ and $y_i$. There are 10 classes which each have parameters $\alpha$ and $\beta$. There is also $\gamma$ which is simply a manually-specified constant.

Original API

The API is like this:
structure = TemplateStructure{(:f,)}(
    ((; f), (x, y, i), p) -> f(x) + p[i] * y; num_parameters=2
)

You need to specify how many total parameters you ask for in advance. Usually this is just length(unique(i)) for some vector i that contains the categorization of each row. But you can have multiple categorizations too – just pass those as additional columns to the dataset. (They will get converted back to integers when indexing p here)
Those parameters can be used in any way you want. You can pass them to one of the subexpressions:
structure = TemplateStructure{(:f,)}(
    ((; f), (x, y, i), p) -> f(x, p[i]) + y; num_parameters=2
)

which basically let's you do what ParametricExpression does, but in a fixed functional form way.
You can also get parameters manually, like this:
structure = TemplateStructure{(:f,)}(
    ((; f), (x, y), p) -> f(x) + p[1] * y; num_parameters=1
)

Where you can see we just ask for a single parameter in the structure.
Full example:
using SymbolicRegression
using Random: MersenneTwister
using MLJBase: machine, fit!, report, matrix

# structure => f(x) + p[1] * y, single param
struct_search = TemplateStructure{(:f,)}(
    ((; f), (x, y, i), p) -> f(x) + p[i] * y; num_parameters=2
)

rng = MersenneTwister(0)

# We'll create 2 features, and 1 categorical variable (with 2 categories total)
X = (; x=rand(rng, 32), y=rand(rng, 32), i=rand(rng, 1:2, 32))
true_params = [0.5, -3.0]
true_f(x) = 0.5 * x * x

y = [true_f(X.x[i]) + true_params[X.i[i]] * X.y[i] for i in 1:32]

model = SRRegressor(;
    niterations=20,
    binary_operators=(+, *, -),
    unary_operators=(),
    expression_type=TemplateExpression,
    expression_options=(; structure=struct_search),
    early_stop_condition=(l, c) -> l < 1e-6 && c == 5,
)

mach = machine(model, X, y)
fit!(mach)

r = report(mach)
best_expr_idx = findfirst(
    i -> r.losses[i] < 1e-6 && r.complexities[i] == 5, 1:length(r.equations)
)
best_expr = r.equations[best_expr_idx]

# We can check that we recovered the `true_params`,
# since this problem fully constrains them:
params = get_metadata(best_expr).parameters
@test isapprox(params, true_params; atol=1e-3)

Pull Request Test Coverage Report for Build<PHONE_NUMBER>6
Details

92 of 127 (72.44%)  changed or added relevant lines in 2 files are covered.
2 unchanged lines in 1 file lost coverage.
Overall coverage decreased (-0.9%) to 93.832%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

src/TemplateExpression.jl
88
123
71.54%

Files with Coverage Reduction
New Missed Lines
%

src/ComposableExpression.jl
2
97.56%

Totals

Change from base Build<PHONE_NUMBER>2:
-0.9%

Covered Lines:
3210

Relevant Lines:
3421

💛  - Coveralls

Hi Miles. Apologies for the later reply, I am also on and off these days. The API looks great to me, thanks so much for it!
Is it already possible to test this from Python? I am unfortunately not familiar with julia

Hi Miles,
Really like the syntax although I misunderstood that parameters = length(unique(i))
For full clarity:
I interpreted (and with ParametricExpression) that parameters were the number of parameters to optimise for each categorial variable. i.e. if you had 1 categorial variable with 4 categories (1,2,3,4), and you set number of parameters to 1, then it would optimise p1 (or whatever) to 4 different values within a function, one for each category; but actually you mean in this instance we should have number of parameters to be 4?
If batching is running, then length(unique(lens)) may be variable with each run.
So lets say i has 4 categories, 1,2,3,4. To find the form p*sin(x) = y, I should set number of parameters to be 4 or 1?
Nevertheless, a little stuck, thank you again for your patience.
I removed and ran ] add SymbolicRegression#master
structure = TemplateStructure{(:f,)}(
  ((; f), (x1, x2, x3, x4, x5, x6, y, cat), c) -> begin
    o = f(x1, x2, x3, x4, x5, x6, c[cat])
    if !o.valid
        return ValidVector(Vector{Float32}(undef, length(o.x)), false)
    end
    #Compute gradients for all variables at once
    for varidx in 1:3
    	grad_column = D(f, varidx)(x1, x2, x3, x4, x5, x6, c[cat]).x
    	if !(all(grad_column .>= 0) || all(grad_column .<= 0))
        	return ValidVector(fill(Float32(1e9), length(o.x)), true)
    	end
    end
    return o
  end;
  num_parameters = length(unique(cat)) #I understand that cat is not defined here, but how do I pass a variable number of parameters so that this code works with batching?
)
model = SRRegressor(
    niterations=1000000,
    binary_operators=[+,-,*,/],
    maxsize=50,
    bumper=true,
    turbo=true,
    populations=18,
    expression_type = TemplateExpression,
    expression_options = (; structure),
    population_size=100
    parsimony = 0.01,
    batching=true,
)
mach = machine(model, X, y)
fit!(mach)

So lets say i has 4 categories, 1,2,3,4. To find the form p*sin(x) = y, I should set number of parameters to be 4 or 1?

And if i had 4 categories, and you were searching the equation p1 * sin(x) - p2 = y, then you would set the number of parameters to 8.
In other words, num_parameters is literally the length of the vector c in your example. You can use that vector in any way you want – which makes it super flexible.
Note that confusingly, num_parameters here (still just an idea, nothing is fixed) is different than ParametricExpression's max_parameters, which in this case, would be 1 and 2 respectively, rather than 4 and 8. This is because the ParametricExpression automatically infers the length of c based on the number of categories. Whereas here, we are just setting the length explicitly.
The one downside is you do need to do a bit more work to set up a problem. But I feel like it's so explicit and flexible that it's worth it, and maybe even easier to understand?

This also means you could have two category types: i1 and i2. If i1 had 4 categories, and corresponded to the parameter p1, and i2 had 2 categories for parameter p2, then you would set num_parameters to 6.

Do you think num_parameters is the right word? Or is there a better term here that might differentiate it with respect to the ParametricExpression max_parameters?

What about num_parameter_values instead of num_parameters? Then it contrasts a bit with ParametricExpression's max_parameters which is more like the number of parameter variables, rather than the number of actual scalar values.

Oh and to try this PR, you need to do
]add SymbolicRegression#template-parametric-expression

Thank you Miles, it works well now, I was half asleep from a very delayed flight and didn't realise it wasn't merged yet!

structure = TemplateStructure{(:f,)}(
  ((; f), (x1, x2, x3, x4, x5, x6, y, cat), c) -> begin
    o = f(x1, x2, x3, x4, x5, x6, c[cat])
    ...
    return o
  end;
  num_parameters = 12
)

Here if I set num_parameters to 12 (which it is for the whole dataset, but batching is true and num_parameters is only 1 or 2 for that particular batch which calls upon structure, does it matter if num_parameters > length(unique(cat)) in that particular batch?
The code seems to be running ok otherwise.

I think the current naming is a little confusing. I think the word parameter is overloaded here.
"In other words, num_parameters is literally the length of the vector c in your example. You can use that vector in any way you want – which makes it super flexible."

Would it better to name it exactly what it is total_num_unique_cat although a bit unwieldy?
There is no way to have this automatically determined based on feeding in a tuple of categorial variables initially to removal any confusion?

In the event you have >1 category across multiple functions, f and g, would you still total the number of params as before? So:

structure = TemplateStructure{(:f,:g)}(
  ((; f, g), (x1, x2, x3, x4, x5, x6, y, i1, i2), c) -> begin
    o = f(x1, x2, x3, x4, x5, x6, c[i1]) + g(x1,c[i2])
    ...
    return o
  end;
  num_parameters = 6
)

if i1 had 4 unique categories and i2 had 2 unique categories?

Agreed on parameter being an overloaded word here.

total_num_unique_cat

Well technically you could use the same category but have it correspond to multiple parameters. Like cat in 1 to 4 could be used for parameters p1, p2, p3 -> which means 12 parameters even though there are only 4 unique categories.
The way you would set it up for this would be:
((; f), (x1, x2, cat), p) -> let
    p1 = p[cat]
    p2 = p[cat + 4]
    p3 = p[cat + 8]

    # for example:
    f(x1) + p1 * x2 + p2 * x2^2 + p3 * x2^3
end

So the number of parameter scalar values is 12, even though there are 4 categories in the dataset.
The other tricky thing is that this p vector doesn’t need to be used for this type of pattern. You could actually just use the p vector to learn a single constant for some sort of functional form — no categories needed. (As an alternative to giving a f() - which is less efficient). So I’m imagining someone could use this to learn parameters for a complex function that they are calling within the template structure.

In your example it would need
- o = f(x1, x2, x3, x4, x5, x6, c[i1]) + g(x1,c[i2])
+ o = f(x1, x2, x3, x4, x5, x6, c[i1]) + g(x1,c[i2+4])
```

I can’t think of another API here that is as flexible and generic. But clearly this API is a bit confusing for this common type of pattern. So maybe try to brainstorm some alternatives if you have any ideas.

Hi Miles,
I have to say (with a large disclaimer) that the last suggestion makes much more intuitive sense for me. The disclaimer is this is not my profession and I just do this stuff in my spare time when I get the chance as it greatly adds to my research output.
What might not be intuitive for me might make perfect sense to other users.
((; f), (x1, x2, cat), p) -> let
    p1 = p[cat] # gets value 1 to 4
    p2 = p[cat + 4] # gets value 5 to 8
    p3 = p[cat + 8] # gets value 9 to 12

    # for example:
    f(x1) + p1 * x2 + p2 * x2^2 + p3 * x2^3
end

In this example, cat presumably will be a vector of ints ranging from 1 to 4, one category with 4 members of that category. If I am understanding right p[cat] would mean 4 parameters for p1 (one for each member of cat), and by setting 5 to 8, it is finding an additional 4 parameters for each member of cat, hence p2.
I would have thought it makes more sense to have p take a tuple p[cat,<parameter_num>]. So that the input would be:
((; f), (x1, x2, cat), p) -> let
    p1 = p[cat,1] # gets value 1 to 4
    p2 = p[cat,2] # gets value 5 to 8
    p3 = p[cat,3] # gets value 9 to 12

    # for example:
    f(x1) + p1 * x2 + p2 * x2^2 + p3 * x2^3
end

Then the constructor would use a mod and length(unique(cat)) to convert it, something like (although I know it would need more tweaking).
p1 = p[mod(<parameter_num>,length(unique(cat)) * length(unique(cat))] #which should return 4
p2 = p[mod(<parameter_num>,length(unique(cat)) * length(unique(cat))] #which should return 8

Hopefully that would give you the same functionality but without the user having to keep track of the number of unique categories within each categorial variable, but only have to keep track of how many parameters they have called on for each categorial variable?

Thanks for the comments, it is much appreciated. Just as a user it is super helpful for me to know what is intuitive.
For this syntax, what would happen if you have a cat and a cat2 here, with different number of unique members in each?

Just thinking about the constant optimisation, so as it currently stands this is how I would do it?
structure = TemplateStructure{(:f,)}(
  ((; f), (x1, x2, x3, x4, x5, x6, y, cat), c) -> begin
   p1 = c[1]
    o = f(x1, x2, x3, x4, x5, x6, c[cat+1])
    ...
    return o
  end;
  num_parameters = 13 #12 members of categorical variable cat PLUS 1 constant optimisation
)

Yeah exactly!  Assuming cat has 12 categories.

@atharvas would be interested in your thoughts on this API as well, I think this will be useful for our stuff!

I've got it printing parameters now too which is nice:
https://github.com/user-attachments/assets/ce99dab9-2e9c-4530-9076-2d64c42ea5b7

Great progress! I was actually about to ask, how do you acquire the parameters in a very long run where you set the niterations to a very high number.

Thanks for the comments, it is much appreciated. Just as a user it is super helpful for me to know what is intuitive.
For this syntax, what would happen if you have a cat and a cat2 here, with different number of unique integers in each?

I've been thinking about this with an example that captures the complexities that can arise:
i1 is categorial variable with 4 members, i2 is a categorial variable with 2 members, we aim to have 2 parameters for i1 (so num_parameters 8), 1 parameter for i2 (so num_paramters 10), and a single parameter for constant optimisation (so num_parameters 11).
Current syntax:
((; f), (x1, x2, i1,i2), p) -> let
    p1 = p[i1]  #1st parameter for i1, 1st parameter overall
    p2 = p[i2+4]  #1st parameter for i2, 2nd parameter overall
    p3 = p[i1]  #reusing first parameter
    p4 = p[i1+6] #2nd parameter for i1, 3rd parameter overall
    p5 = p[10] #1st parameter constant optimisation, 4th parameter overall
   #used however you want between functions or within
end;
num_paramters = 11

Honestly, I find in complex workflows this will very prone to errors, especially if you get beyond 2,3 parameters.
Proposed synxtax:
((; f), (x1, x2, i1,i2), p) -> let
    p1 = p[i1,1]  #1st parameter for i1, 1st parameter overall
    p2 = p[i2,2]  #1st parameter for i2, 2nd parameter overall
    p3 = p[i1,1]  #reusing first parameter
    p4 = p[i1,3] #2nd parameter for i1, 3rd parameter overall
    p5 = p[,4] #1st parameter constant optimisation, 4th parameter overall
   #used however you want between functions or within
end

Then behind the hood, have an initiator:
parameters_indices = [1, 2, 1, 3, 4]  # Example data that is picked up from a parser
parameters_indices_variables = [:i1, :i2, :i1, :i1, missing]  # Example variables that is picked up from the parser, maybe going through structure and searching for p[*]
combined_matrix = hcat(parameters_indices, parameters_indices_variables)'

# Unique parameter indices
unique_parameters_indices = unique(parameters_indices)
no_of_unique_parameters = maximum(unique_parameters_indices)

num_parameters = 0
num_parameters_indices = zeros(Int, no_of_unique_parameters)

for i in 1:no_of_unique_parameters
    matching_rows = combined_matrix[:, combined_matrix[1, :] .== i]
    unique_variable_count = length(unique(matching_rows[2, :]))
    num_parameters += unique_variable_count
    num_parameters_indices[i] = num_parameters
end

num_parameter_indices2 = similar(parameters_indices)

for i in 1:length(parameters_indices)
    parameter_index = parameters_indices[i]
    matching_rows = combined_matrix[:, combined_matrix[1, :] .== parameter_index]
    unique_variable_count = length(unique(matching_rows[2, :]))
    num_parameter_indices2[i] = num_parameters_indices[parameter_index] - unique_variable_count
end

if I have written it write, then the output should be:
num_parameters_indices: [4, 6, 10, 11]
num_parameters_indices2: [0, 4, 0, 6, 10]
That should match with the original syntax:
p[parameters_indices_variables[1] + num_parameters_indices2[1]] = p[i1 + 0]
p[parameters_indices_variables[2] + num_parameters_indices2[2]] = p[i2 + 4]
p[parameters_indices_variables[3] + num_parameters_indices2[3]] = p[i1 + 0]
p[parameters_indices_variables[4] + num_parameters_indices2[4]] = p[i1 + 6]
p[parameters_indices_variables[5] + num_parameters_indices2[5]] = p[missing + 10]
Let me know if that makes any sense!

With regards to API v2:
structure = TemplateStructure{(:f,), (:p1, :p2)}(
  function ((; f), (; p1, p2), (x1, x2, x3, x4, x5, i1, y, i2))
    o = f(x1, x2, x3, x4, x5, p1[i1], p2[i2])
    if !o.valid
        return o
    end
    #Compute gradients for all variables at once
    for varidx in 1:3
    	grad_column = D(f, varidx)(x1, x2, x3, x4, x5, p1[i1], p2[i2]).x
    	# Check if all nonnegative or all nonpositive
	if !(all(g -> g >= 0, grad_column) || all(g -> g <= 0, grad_column))
		return ValidVector(fill(1f09, length(o.x)), true)
    	end
    end
    return o
  end;
  num_parameters = (; p1=2, p2=12)
)

i1 is a binary categorial variable of 0,1s
i2 is has 12 unique members
ERROR: BoundsError: attempt to access 2-element SymbolicRegression.TemplateExpressionModule.ParamVector{Float32} at index [Float32[1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0  …  0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 1.0, 1.0, 1.0]]
Can't quite see my mistake!

Quick reply – since Julia is one-indexed, you will need to offset i1 by +1.
(I think I may make a 0-indexed array for the PySR interface so its easier for Python users)

Thank you, that worked. Theoretically, I suppose it should be more efficient to pass a categorial binary variable as a parametric variable than as a 0,1 as a normal variable?

Maybe but note that everything gets converted to Float64 anyways (and then back to Int64 when indexing). But these sorts of memory considerations are <1% compared to all of the allocations that happen within the evaluation and optimization loops


Implement automatic slowmode management into HalalBot.

assuming this is Abu Nur and assuming you already tested it so ima merge & put it up
جزاك الله خيرا
might need you to help me refactor it to be more self explanatory later though because the code is kind of confusing atm lol


Change license to MIT

Change license to MIT

[x] @0Lucifer0
[ ] @BlowaXD
[ ] @Elendan
[ ] @ApourtArtt
[ ] @Adanlink
[ ] @senterol
[ ] @Zagith
[x] @Bari77

Can you please tell me if you agree with License moving to MIT
Will wait till 1 of july and move to MIT except if somebody is against that

I don't see any reason to move it to MIT honestly, I decline unless there is a good reason.

That's not a problem for me.

@Adanlink Because that would allow people with private projects to use it legally. I don’t want to enforce GPL but I don’t like to see everybody breaking this license policies

Moreover it often helps to get free license for tools like resharper

Yeah, i agree too,

Fine for me ^^

@Adanlink Because that would allow people with private projects to use it legally. I don’t want to enforce GPL but I don’t like to see everybody breaking this license policies

Using NosCore.Packet's source code for doing your own version is legal with GPL3, what is not allowed (without going too deep) is to distribute it without the same license or in other words, the source code (with distribute it can mean selling/sharing/publishing).
Maybe MIT would make more attractive to contribute to the project, but with the cancerous scene of NosTale I doubt it will help, it could even make it worst.
Also, I don't understand why it would help to get tools, GPL3 is valid here https://opensource.org/docs/osd as MIT is too.

Tbh, I don't see the point, but I don't see the problem either. Enforcing the GPL in a waste of time, the NosTale community doesn't give a shoot about the licence type as long as they can leech.
Also, there is probably no server owner around that is able to pull a nuget package to use the NosCore.Packets library so I don't see why the type licence matters so much

Sure, go for it !

Last time I asked for free license I’ve seen in case of MIT and Apache everything is automatic. In case of GPL they manually review it and it need multiple emails. Most of the free licenses are only valid for 1 year and I would like to refresh this as easily as I can

Last time I asked for free license I’ve seen in case of MIT and Apache everything is automatic. In case of GPL they manually review it and it need multiple emails. Most of the free licenses are only valid for 1 year and I would like to refresh this as easily as I can

@Adanlink after checks it seems like it’s actually Apache not MIT For jetbrains
However that doesn’t really change the fact that in the end the GPL is not respected at all and that nobody will enforce it. So letting people do whatever they want as long as authors are still defined seems better to me

Maybe I'm wrong, but what does people not enforce? The majority of people (to not say all of which I have heared) that distributes (principally sells) "reflavored" versions of Opennos (for example) gives it with the source code (as established with the GPL2 license) and not a compiled version or worst, an obfuscated one.
I really think that this change of license is not positive as it would only worsen the possible future contributions to it (because as it is known, people leaches a lot, and it would even worsen if they could sell their private versions of NosCore without the source code, as it would allure to the doing of those private versions).

I’ve seen multiple release done as non GPL, leak of real developers work by leechers and some people even changed the license. With mit the few real dev of the community can sell their change under a private copyrighted license if they want to do it and they can sue anybody who would release it without consent.
Won’t change much technically moreover noscore itself will be difficult to change as it has lot more contributors (even if most of them are for translations and those are going to disappear)

@Adanlink the main goal is to let real developers have the power over what they are creating.
there are better example out there with people who didnt even have coded anything but for example : https://www.elitepvpers.com/forum/nostale/4645443-release-opennos-sources.html
Its OpenNos so i have the right to publish this shitty Sources without that i should care if someone wanna it or not.
this is the kind of behavior i'm trying to prevent here

i want real dev to have the choice of their licenses

@Adanlink the main goal is to let real developers have the power over what they are creating.
there are better example out there with people who didnt even have coded anything but for example : https://www.elitepvpers.com/forum/nostale/4645443-release-opennos-sources.html
Its OpenNos so i have the right to publish this shitty Sources without that i should care if someone wanna it or not.
this is the kind of behavior i'm trying to prevent here

Let's be realistic, no single dev in this community will ever sue anyone for leaking its work.
Especially that, given NosCore's design, instead of making a community-centric project, you are dividing people and expect them to make their own fork of NosCore mostlikely not interoperable between every versions (like already done on OpenNos, many forks and some piece of code are not interoperable with the different forks online), which fucks up the idea of an opensource community driven project.


Platformer fixes and tuning

Also includes Switch Gamepad touchpad buttons fix
List of fixes (including previous pull request fixes since I'm too lazy to separate them):

[x] beating game in debug mode should not set any achievements
[x] Can’t die by falling off screen in second half of 4-2
[x] Scroll stop bug in second half of 4-2
[x] High Score achievement conditions must be evaluated at the ending screen after extra lives are added to score, not when switching to the ending screen.
[x] No enemies in 4-1
[x] No enemies at the beginning of 4-2
[x] Unreachable warp in 2-1
[x] Player can jump while on ladder
[x] 100 coin 1UP is actually 101 coins
[x] Dying while on ladder sends dead player into the sky
[x] Coin sound gets higher priority than 1UP sound when getting 100 coins
[x] Scrunchy tiles at bottom of screen
[x] “Time Up” message is misaligned
[x] Level Select skipped in debug mode menu if no levels have been unlocked
[x] Level Select vs Start Game ambiguity
[] Enemies/Powerup should turn around when touching each other… (not fixing!!!)
[x] 1-2 needs backgrounds
[x] 1-2 “the jump” is too hard
[x] 1-2 too hard to reach star goal block with a combo
[x] 1-2 10 coin blocks should just act like 1 coin blocks
[x] 1-2 entrance and exit need backgrounds
[x] 1-2 need a powerup after checkpoint
[x] 1-4 bees at end are meant to be run through; impossible without taking damage?
[x] 2-1 First powerup block should be a container block, not a brick block
[x] 2-1 star goal block too hard to reach?
[x] 2-2 needs more coins to keep combo
[x] 2-2 needs background
[x] 2-2 needs 1up?
[x] 2-3 needs background
[x] 2-3 needs 1up?

[x] 3-1 1up impossible to get
[x] 3-1 star goal block is too hard to reach
[x] 3-2 should be a sunset level just to mix things up
[x] 3-2 needs backgrounds
[x] 3-2 platform adjustments
[x] 3-3 maybe increase time limit?
[x] 3-3 crash when jumping off screen above bounce blocks on very top row of level
[x] 3-4 too hard to keep combo at end
[x] 3-4 needs extra powerup
[x] 4-1 checkpoint area does not scroll lock until player moves
[x] 4-1 definitely need to increase timer
[x] 4-1 hard to keep combos since there are ladders everywhere
[x] 4-1 camera bug when starting at checkpoint!!!

[x] 4-1 retrying ladder to bounce block challenge is too annoying!!!

[x] 4-3 crash entering 4-4
[x] 4-3 need an A at 3rd bounce block jump
[x] 4-4 impossilbe to reach star goal block
[x] 4-4 bee before spiral is too hard

Tested switch controller, now the touchpad maps to YL_RX with the middle pad doing nothing. Is that correct?


Enable posting Runnables to the render thread

This is pretty useful for any render-loop-based application, but in particular this will help with instances where a client application needs to make multiple synchronous changes to the map view. For example, this code might cause stuttering in the view:
mapController.setMapPosition(a.longitude, a.latitude);
// Render might start here...
mapController.setMapRotation(a.bearing);
// ... then render again after producing one frame with the wrong rotation!

But this code has no such risk:
mapController.queueEvent(new Runnable() {
    public void run() {
        // This runs on the render thread!
        mapController.setMapPosition(a.longitude, a.latitude);
        mapController.setMapRotation(a.bearing);
    }
});

@hjanetzek Does this look alright to you?

Nice. Its exactly http://developer.android.com/reference/android/view/View.html#post(java.lang.Runnable) right? Can the View::post not be used from the app?
This is also one of the suggestions here: http://stackoverflow.com/a/8976579

In theory we could use View.post() but then the client application needs to have a reference to the MapView as well as the MapController, the latter of which handles all other interaction. Using our own method also allows us to ensure that requestRender() is called so that the Runnable is actually run.

For posting a Runnable on the render thread an trigger a redraw there is a GLSurfaceView.queueEvent method already http://grepcode.com/file/repository.grepcode.com/java/ext/com.google.android/android/5.1.1_r1/android/opengl/GLSurfaceView.java#GLSurfaceView.GLThread.queueEvent(java.lang.Runnable)
Since the native methods of MapController are synchronized on the class itself, I think one could also use:
synchronized(MapController.class) {
    mapController.setMapPosition(a.longitude, a.latitude);
    mapController.setMapRotation(a.bearing);
}

True, it seems there's no need to store our own ArrayList of Runnables - we could post directly to that of the GLSurfaceView. It is still a useful convenience to be able to do this through a MapController and not only a MapView.
I'll check whether synchronization achieves the same result in Eraser Map, but I'm still figuring out how concurrency works in Kotlin :\

Ok, so a synchronized block doesn't quite do the trick. The render call itself (onDrawFrame) isn't synchronized so it can still occasionally produce incorrect frames.
I've edited the changes such that runnables are sent directly to the queue in our GLSurface view, and it seems to work nicely!

Any further feedback before merging?

This looks good as per my limited Java knowledge.
I would like @ecgreb to also have a look at this to make sure the user of our API acknowledges this change.

Yep, I'll be demonstrating this update with a suggested change to eraser-maps.


Adds support for connector node worker api

Issues:
Fixes #1101
Problem:
The nodes-worker api was not part of the sdk.
Analysis:
this adds it
Tests:
functional

@wojtek0806 @jasonrahm

lgtm


Implement Assemble trait

Implement an Assemble trait with an assemble method that accepts a str and returns a Vec<u8>
depends on #53

This was already added via #56 but should probably be extended to include contexts from the Preparser that will be fleshed out in #53


Use CacheFirstDataSource in ExchangeApi

Closes #314

ExchangeApi now uses CacheFirstDataSource in order to avoid calling external fiat rate exchange endpoints more often than required

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

12 of 12 (100.0%)  changed or added relevant lines in 3 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.02%) to 90.208%

Totals

Change from base Build<PHONE_NUMBER>:
0.02%

Covered Lines:
3732

Relevant Lines:
4085

💛  - Coveralls


pythonPackages.unidecode: 1.1.1 -> 1.2.0

Motivation for this change
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

running nixpkgs-review, this breaks swaglyrics. scikit-learn also doesn't build, but I can't tell if that's related.

@yorickvP Oh, yeah, I thought this was a small package without many dependencies, the 101-500 comes as a surprise to me

@Synthetica9 please resolve the merge conflict.


Resources in this stack

I think for a start this can be a standard cluster:

[ ] 1 node. I think we may separate LLM service into a different node later on
[ ] 1 postgresql database. I suppose RDS. Start with may db.t4g.small @leothomas is that too small?
[ ] Setup cluster level logging. @sunu what's our preferred approach now? Prometheus/graphana?

cc @yellowcap @srmsoumya

Also noting that @srm had mentioned langfuse for logging the agent interactions/processes. There's a langfuse-k8s framework that we might make use of

Start with may db.t4g.small @leothomas is that too small?

Because PostgreSQL is backed by RDS the storage can essentially grow infinitely, so the instance size mainly affects the volume/speed of data ingestion and index building. The IVF Index build time grows with data volume, but doesn't occur in memory, so a small database will just take longer.  HNSW indexes, on the other hand, take orders of magnitude longer to build if the vectors cannot all be held in memory. One way to get around this limitation is to bump the instance to a larger type (eg: t4g.2xlarge) during the insert and then bump back down afterwards.
Given that this is for a RAG pipeline (as opposed to something like an image similarity search) I don't think the index recall will be the biggest bottleneck in the output accuracy/quality, since text similarity is inherently "fuzzy" and we'll be collecting > 10 results per query. This means an IVF Index will very likely be more than enough
All this to say, t4g.small should be fine for now.


renogy dcc50s bt-2  modbus exception

Hi
Iwas testing the renogy Bluetooth module bt-2
having a requestion,
once connected the BT-2  to the renogy I had an issue with other serial readings
I do not get any serial data when bt was disconnected again?
Modbus exception error. no answer
have you experienced something like this?
Thanks T

It is likely that the bt module changed the rovers modbus address. It commonly changes it to 16, so in your base.json, try setting modbus to 16. If that doesn't work, I can run the rover-setup program and run scan 1 20. It will take a while, but it should show you what address it has.
You should also be able to change the address back to 1 in the app somehow.
I assume your error is a modbus timeout exception? If it's not, then posting the error would be helpful.

thanks for the hint
will check for new adress with the scan.
i thought something like this an tried address 2 already . will check for 16 and scan.
Perfect thanks

It just occurred to me that the scan feature is not a part of SolarThing 2021.3.0, so you can try to find the address manually, compile SolarThing yourself and use scan, or wait for me to update it.

SolarThing 2021.3.1 has the scan feature. You can update if you'd like, but there aren't any new features I'm aware of that will benefit you.
Close this if you get it working.

Hi
thanks just updated
but how to run scan 1 20"?

Hi Joshua,
found the scan feature. but where do i get the result from?
`
Scan finished!
scan 1 20
Starting scan. (Will tell you when scan has stopped)
}`
thanks

Hi Joshua,
I found new slave address at hex 0x60/dec 96  confirmed by renogy support
given by BT-2 connection
Thanks for your support

Huh. That's really weird that it decided it wanted an address of 96. If I have to recommend that people scan from 1 to 247, that's gonna take a long time.

Scan finished!

If there was nothing between this message and the "scan started" message, it means it didn't find a rover, which makes sense because it was on an address outside the range you were scanning.
I probably need to figure out a way to do a quicker scan. 2 seconds per address isn't really acceptable if it's address can be in a large range of addresses.
I guess I'll probably recommend try to find their address through their BT module from now on lol.


Move fast-forward links into sidebar

What's new
Some research participants have got confused by the fast-forward links, thinking they’re part of the journey. Move them to the sidebar, colour them consistently, put "User research example:" in front of them to make it clear why they’re there.
Screenshot

 Improve styling of fast-forward links


Add ts-loader and fix webpack configuration to support next version

Questions
Answers

Description?
Actual develop branch of prestashop migrated to typescript and as we rely on a core file, we need to add the ts-loader in order to compile ts files, I also added a PSJs path in order to avoid backslash hell

Type?
bug fix

BC breaks?
no

Deprecations?
no

How to test?
Pull this branch, npm i && npm run build the module and check if everything is fine

aproved and tested on develop 👍

thanx @NeOMakinG !


Build shared library?

How to build a shared library?
cmake -DBUILD_SHARED_LIBS=ON does not create a shared library.

There is no way to create a shared library at the moment.  There is no technical barrier to it, we just haven't had the need for it.  PRs are welcome! 🙂

I have some answers, but I do not know how to use cmake.
Change lsquic/src/liblsquic/CMakeLists.txt to SHARED:
# add_library(lsquic STATIC ${lsquic_STAT_SRCS} )
add_library(lsquic SHARED ${lsquic_STAT_SRCS} )

or

# add_library(lsquic STATIC ${lsquic_STAT_SRCS} )
IF (SHARED EQUAL 1)
	ADD_LIBRARY(lsquic SHARED ${lsquic_STAT_SRCS})
ELSE()
  ADD_LIBRARY(ls-hpack STATIC ${lsquic_STAT_SRCS})
ENDIF()

Build:
cmake -DBORINGSSL_DIR=/Users/xxx/installed/C/tls/boringssl -DBORINGSSL_LIB=/Users/xxx/installed/C/tls/boringssl/build -DCMAKE_BUILD_TYPE=Release -GNinja cmake ..
ninja

Errors is OSX:

```bash
Undefined symbols for architecture x86_64:
  "_ASN1_INTEGER_to_BN", referenced from:
      _lsquic_enc_session_handle_chlo in lsquic_handshake.c.o
  "_BIO_free", referenced from:
      _lsquic_bio_to_crt in lsquic_crypto.c.o

I tried to google those errors, but I could not find the answer.

I was able to generate dylib from static lib in OSX:
gcc -fpic -shared /usr/lib/libz.dylib /Users/xxx/installed/C/tls/boringssl/libcrypto.dylib /Users/xxx/installed/C/tls/boringssl/libssl.dylib /Users/xxx/installed/C/tls/boringssl/libdecrepit.dylib -Wl,-force_load, src/liblsquic/liblsquic.a -o liblsquic.dylib

When I tried to build shared with -DSHARED=1  in Ubuntu 20 I got this error:
[6/111] Linking C executable echo_client
FAILED: echo_client
: && /usr/bin/cc -DLSQUIC_DEBUG_NEXT_ADV_TICK=1 -Wall -Wextra -Wno-unused-parameter -fno-omit-frame-pointer -Wno-implicit-fallthrough -Wno-missing-field-initializers -O3 -g0  -O3 -DNDEBUG  -rdynamic CMakeFiles/echo_client.dir/bin/echo_client.c.o CMakeFiles/echo_client.dir/bin/prog.c.o CMakeFiles/echo_client.dir/bin/test_common.c.o CMakeFiles/echo_client.dir/bin/test_cert.c.o  -o echo_client  -llsquic  -Wl,-Bstatic  -levent  -Wl,-Bdynamic  /home/xxx/Documents/C/boringssl/build/ssl/libssl.a  /home/xxx/Documents/C/boringssl/build/crypto/libcrypto.a  -Wl,-Bstatic  -lz  -Wl,-Bdynamic  -lpthread  -lm && :
/usr/bin/ld: cannot find -llsquic
collect2: error: ld returned 1 exit status
[8/111] Building C object CMakeFiles/echo_server.dir/bin/test_common.c.o
ninja: build stopped: subcommand failed.

@stuta have the same use case to generate a shared library as you, did you have any luck on Linux? I ended up with the same sk_deep_copy symbol problem. I managed to generate a shared library of sorts, but it was missing some things that should have been statically linked.

@stuta managed to build a really janky version of the shared library by compiling boringssl with position independent code and then adding a library target to CMakeLists inside liblsquic.Lists inside liblsquic.
add_library(lsquic2 SHARED ${lsquic_STAT_SRCS})
TARGET_LINK_LIBRARIES(lsquic2 PRIVATE ${BORINGSSL_LIB_ssl} ${BORINGSSL_LIB_crypto} /usr/lib/x86_64-linux-gnu/libz.so)

To compile boringssl so it can be statically linked into the dynamic library (not applicable to you).
cmake -DCMAKE_POSITION_INDEPENDENT_CODE=true .

it would be very useful to have a supported way to do this as it greatly improves being able to use lsquic in FFI contexts.

We'd take a PR...

So I've made some changes locally that are based on what else is in this thread, can just send that up and see what you think.

Sure, we'll check it out!

Fixed via PR #152 -- closing.


Terminal: command history will not be saved to $HISTFILE when close vscode

Does this issue occur when all extensions are disabled?: Yes

VS Code Version: 1.77.3
OS Version: macOS Ventura 13.3.1

Problem:

After update to vscode 1.77, the command history will not be saved when closing vscode or kill terminal.
If you type 'exit' to close the terminal, the command history will be saved.
this problem are not existed in the previous vscode version such as 1.75.

https://user-images.githubusercontent.com/53963800/233556401-7db58b50-bc76-4f0a-8f18-0d5c5b57615e.mov
For example, in this case, the cmd history, ls, clear were saved successfully when close terminal by typing exit, but the cmd cat ./.zsh_history was not saved when I just close the VScode (the same problem will be occurred when I kill the terminal).
By the ways, this problem also existed when I working in Ubuntu 20.04 by using a remote SSH.
This problem has been bothering me for a while, which causing me to use an older version VScode.

The same thing is happening to me, I'm using MacBook PRO M2.
VSCODE:
Version: 1.77.3
Commit: 704ed70d4fd1c6bd6342c436f1ede30d1cff4710
Date: 2023-04-12T09:41:48.638Z (1 wk ago)
Electron: 19.1.11
Chromium: 102.0.5005.196
Node.js: 16.14.2
V8: <IP_ADDRESS>-electron.0
OS: Darwin arm64 22.4.0
Sandboxed: Yes

Same issue on Ubuntu 22.04
VSCode installed as "code" from Snap store
I verified that typing exit in the integrated terminal saves history.

Same for me on Win10 with Remote SSH into Debian 11.
Closing with exit saves history.

Same issue here using WSL on Windows 11

/duplicate https://github.com/microsoft/vscode/issues/181508


Column name changes causing code 106 error

Parse Server version: 2.1.4
During development stage, I changed the column name of a parse class and my Query.find() is returning the following error
{"code":106,"message":"key objectId not present"}
The parse help says, I am tinkering with the internals of parse. Certainly no.... Is this a known issue? Do i need to clear the schema cache?

// This is ParseCloud Code
console.log("merchant id console check..." + MerchantId);
var Merchant = Parse.Object.extend("Merchant");
var MQuery = new Parse.Query("Merchant");
MQuery.equalTo("RollNo",10000);

The following error message {"code":106,"message":"key objectId not present"}
Please help.

Can you include a list of steps to reproduce the issue? For example, how did you change the name of a key?

Closing due to lack of activity, please update to latest parse-server version and reopen if the issue persist.
Don't forget to include your current:

node version
npm version
parse-server version
any relevant logs (VERBOSE=1 will enable verbose logging)


wxwidgets 3.1.0 -> 3.1.2

Motivation for this change
Need this dependency for another package I'm working on. :smile:
Things done

[X] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[X] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Assured whether relevant documentation is up to date
[ ] Fits CONTRIBUTING.md.

It might make sense to just get rid of the 3.0 package and put everything in the 3.1 directory. We'd need to verify everything still works though.

@grahamcofborg build wxGTK30 wxGTK31 pwsafe

Looks like it's going to break things, unfortunately. I had to work around the fact that there was no --enable-compat26 flag in 3.1.2. Also looks like there is some error with --enable-universal-binaries missing on x86_64-darwin.

Resolved by 63423


The game freezes for a long time when the world resets

This is an issue found in #2

This is fixed. The game will continue to run behind the cutscene without locking up the window.


BUG/API with filtering

@jdammers just demonstrated to me how to silently produce nans with our band pass filter.
https://gist.github.com/dengemann/7580547
Or old non-Python routines default to order 16 butterworth band pass filters
(FFT implementation) ...
If you go for lower orders, in certain application evil filter artifacts would occur.
I think we should

add a clear RuntimeError if NaNs are returned by scipy.signal.filtfilt

@jdammers suggested to port his IDL fft code, currently we don't support order type construction for fft filters.

WDYT?

We have a check for poles outside the unit circle already, closing this one


Add support for Cloudflare Bot Management

Bot management is a service provided by CloudFlare that detects bots and exposes new variables you can use in custom firewall rules. For more information, see here: https://www.cloudflare.com/products/bot-management/
It is enabled/disabled in the Settings menu of a zone, but it is not exposed via the settings API so it can not be set using this library at the moment. It is exposed via the following URL: https://api.cloudflare.com/client/v4/zones/$ZONE_ID/bot_management
This endpoint allows setting of the enabled flag, which enables bot management, and the fight_mode flag which will automatically challenge probable bots.
example GET response:
{
  "result": {
    "fight_mode": false,
    "enabled": true
  },
  "success": true,
  "errors": [],
  "messages": []
}

and via PUT you can set values:
{
  "enabled": true,
  "fight_mode": true
}

Would be great if this feature was supported so it then can also be used in the Cloudflare Terraform provider.

There isn’t public API documentation for this yet so it won’t be added as we are only building on the stable exposed endpoints.
Are you able to open a support ticket to get this documented?

They said they won't document it until the API is final, so we'll have to wait for a bit here.

Thanks for the update! Let's leave this as is for now then and once the team releases a stable API, we can add the functionality here.

@jacobbednarz @LukasdeBoer any update on this?
Would be awesome to see this feature integrated into applications like terraform

Hi @dc232, It's good that they didn't declare the API final, because it looks like Cloudflare has completely overhauled this setup and now has Bot Management enabled by default, so this issue can be closed.

For those wondering how to make this available as a header just use -> https://developers.cloudflare.com/terraform/additional-configurations/transform-rules/#create-an-http-request-header-modification-rule and then grab the value request.cf.botManagement.score


WARN logs printed at startup

Below WARN logs can be seen at startup,
[2019-05-20 09:37:48,884]  WARN {org.wso2.carbon.idp.mgt.IdentityProviderManager} -  SAMLMetadataValidityPeriod in identity.xml should be a numeric value
[2019-05-20 09:37:48,885]  WARN {org.wso2.carbon.idp.mgt.IdentityProviderManager} -  SAMLMetadataSigningEnabled in identity.xml should be a boolean value

Most probably due to a missing default configuration in the identity.xml

This issue is being closed due to extended inactivity. Please feel free to reopen it if further attention is needed. Thank you for helping us keep the issue list relevant and focused!


Size exceeds the maximum allowed file size

This issue tracker is only for issues related to Frappe-Manager. Please use https://github.com/rtcamp/frappe-manager/discussions for support questions.
Describe the bug
When I try to upload an image that exceeds 1MB in any DocType, the error "Size exceeds the maximum allowed file size" is displayed.
To Reproduce
Steps to reproduce the behavior:

Edit Item in Item List
Add an image larger thatn 1MB
The error is displayed.

Expected behavior
Upload the image.
System information (please complete the following information):
Attach log
Additional context
I think the problem is related to the nginx container fm_global-nginx-proxy configuration. I can't find the client_max_body_size configuration directive in tha nginx config file for this containter.

Please refer to this FAQ section here to increase the upload limit.
Let me know if you face any issues


Remove manual nonce interaction

Per discussion with Martin  https://github.com/o1-labs/zkapp-cli/blob/main/src/lib/deploy.js#L278

Resolved by Martin's PR https://github.com/o1-labs/zkapp-cli/pull/197


SQL window

🤔 Tell Us About the Feature
Provide an SQL access for @mryan4 in the Focus BMDB interface.
🎇 What Solution You'd Like
Maybe a way to execute SQL (but read only!)
〽 Alternative Ideas
@dcrichto suggests Google Sheets.

@mryan4 do you have JPL VPN? The system admins may wish to seal this particular feature off to those only with VPN.

Yes, I do. Phew!

On Apr 20, 2021, at 5:16 PM, Sean Kelly @.***> wrote:

@mryan4 do you have JPL VPN? The system admins may wish to seal this particular feature off to those only with VPN.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.

@mryan4 okay, I've configured and deployed "phpMyAdmin" and it's ready to use for the Biomarker Database! Rather than home-brew a SQL entry-window solution, this one has all the bells and whistles that should make exploring the data, paging through tables, creating complex queries, exporting to spreadsheets, and even more possible. You can give BMDB the "third degree" with this! 👮
To access it, you will need:

JPL VPN
The database username biobrowse¹ and password²
The URL to access it: https://bmdb.jpl.nasa.gov/3rd-degree/

If you want to schedule a time to go over the features, let me know. I'm happy to help explore it.
¹The biobrowse username has read-only access, so don't worry about messing anything up³
²The password will come under a plain brown wrapper
³If you do want to be able to do mass updates and want a read+write account, let me know

Hi Sean,
Thank you!  I’ll take a look at it tomorrow. This will be great.
I’m sure I will definitely need some help with queries, so I will accept your offer to go over it.  I’m open pretty much any time Thursday or Friday.  Let me know what works for you.
Maureen
Maureen Ryan
Division of Gastroenterology & Hepatology
Department of Medicine
UNC at Chapel Hill
On Apr 21, 2021, at 3:17 PM, Sean Kelly @.***> wrote:
@mryan4 https://github.com/mryan4 okay, I've configured and deployed "phpMyAdmin" and it's ready to use for the Biomarker Database! Rather than home-brew a SQL entry-window solution, this one has all the bells and whistles that should make exploring the data, paging through tables, creating complex queries, exporting to spreadsheets, and even more possible. You can give BMDB the "third degree" with this! 👮
To access it, you will need:
JPL VPN
The database username biobrowse¹ and password²
The URL to access it: https://bmdb.jpl.nasa.gov/3rd-degree/ https://bmdb.jpl.nasa.gov/3rd-degree/
If you want to schedule a time to go over the features, let me know. I'm happy to help explore it.
¹The biobrowse username has read-only access, so don't worry about messing anything up³
²The password will come under a plain brown wrapper
³If you do want to be able to do mass updates and want a read+write account, let me know
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/EDRN/biomarker-database/issues/41#issuecomment-824296569, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQLPXMKR2MCUTWM6DGH2XZ3TJ4QENANCNFSM4237CBMA.


cvxopt rank ValueError

Creating this ticket for our records:
cvxopt raises a ValueError due to matrix rank on the following example:
from gpkit import Variable, GP

x = Variable('x')
y = Variable('y')

gp = GP(y*x,
        [y*x >= 12])
sol = gp.solve(solver='cvxopt')

Mosek solves this with no problems.

Not a gpkit issue -- opened an issue for cvxopt developers: https://github.com/cvxopt/cvxopt/issues/36


JK - Test Technique 28/09/2023 Joan KUCUKOGLU

Exercice réalisé sans test (toujours lié aux soucis de SSL). Vous pouvez ainsi étudier ma façon d'aborder l'exercice.
Bonne journée à vous.
Joan.

@JoanKUCUKOGLU thanks for the sharing
This is a good work. Congrats.
I have put a few comments on your files.
I'd like also to get to the bottom of your ssl config. Can you try to install curl and run the following command to display the ssl certificate.
curl -v  https://cacib-saturn-test.francecentral.cloudapp.azure.com

It works well on my side and it displays the following result
*   Trying <IP_ADDRESS>:443...
* Connected to cacib-saturn-test.francecentral.cloudapp.azure.com (<IP_ADDRESS>) port 443 (#0)
* ALPN: offers h2,http/1.1
* (304) (OUT), TLS handshake, Client hello (1):
*  CAfile: /etc/ssl/cert.pem
*  CApath: none
* (304) (IN), TLS handshake, Server hello (2):
* (304) (IN), TLS handshake, Unknown (8):
* (304) (IN), TLS handshake, Certificate (11):
* (304) (IN), TLS handshake, CERT verify (15):
* (304) (IN), TLS handshake, Finished (20):
* (304) (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / AEAD-AES256-GCM-SHA384
* ALPN: server accepted http/1.1
* Server certificate:
*  subject: CN=cacib-saturn-test.francecentral.cloudapp.azure.com
*  start date: Aug 16 01:39:20 2023 GMT
*  expire date: Nov 14 01:39:19 2023 GMT
*  subjectAltName: host "cacib-saturn-test.francecentral.cloudapp.azure.com" matched cert's "cacib-saturn-test.francecentral.cloudapp.azure.com"
*  issuer: C=US; O=Let's Encrypt; CN=R3
*  SSL certificate verify ok.
* using HTTP/1.1
> GET / HTTP/1.1
> Host: cacib-saturn-test.francecentral.cloudapp.azure.com
> User-Agent: curl/8.1.2
> Accept: */*
>
< HTTP/1.1 200 OK
< Server: nginx/1.18.0 (Ubuntu)
< Date: Fri, 29 Sep 2023 09:43:49 GMT
< Content-Length: 0
< Connection: keep-alive
< Vary: Origin
<
* Connection #0 to host cacib-saturn-test.francecentral.cloudapp.azure.com left intact

Also, try running the backend inside a docker instance (ubuntu for instance) as I feel that your PC is lacking the root certificate for the issuer R3 of let's encrypt

@guenoledc i just tried the command that you told me to try and and got this result :
COMMENTAIRES : GET https://cacib-saturn-test.francecentral.cloudapp.azure.com/ with 0-byte payload
COMMENTAIRES : received 0-byte response of content type

StatusCode        : 200
StatusDescription : OK
Content           : {}
RawContent        : HTTP/1.1 200 OK
                    Connection: keep-alive
                    Vary: Origin
                    Content-Length: 0
                    Date: Fri, 29 Sep 2023 13:56:21 GMT
                    Server: nginx/1.18.0 (Ubuntu)

Headers           : {[Connection, keep-alive], [Vary, Origin], [Content-Length, 0], [Date, Fri, 29 Sep 2023 13:56:21 GMT]...}
RawContentLength  : 0

So it did not fail with curl.
However it does not display the validation (or not) of the server certificate.
Try running in docker !

Thanks for the work done. We will not merge this PR so let's close it.
You can continue to update it if needed


The song for my mod keeps playing the goddamn test music

Describe your problem here.
when i tried playing my mod the song would just play the main menu theme but when i changed the mod redirector it worked but when i played through the free play menu it happened again
Are you modding a build from source or with Lua?
Source
What is your build target?
Windows x64
Did you edit anything in this build? If so, mention or summarize your changes.
no

Commonly for songs added through Lua mods you need to make sure there is an associated folder for the song in song (Specifically Mods/Mod-Name/Songs/Song-Name/Song-Files) The Folder containing the Song Files must contain an Inst audio file (The Voices audio file is optional) and the file extensions must be .ogg files otherwise the engine can't locate them... It plays the Test music by default when making one from scratch... if you know you put the song files in just change the name in the category where you can save the chart file to the one that matches the name of the folder (It is not Case-Sensitive) and press reload audio... then boom it's done... But for Source code it's too complicated for me to figure out... Sorry...


Control buttons size

This is an feature request :
It would be really nice to be able to control the size of the buttons
Ideally we could control the size of each ActionButton.Item and the ActionButton independently

We can! It's somehow not documented in the Readme but you can use the prop size to change the size of each button independently.

OK thanks !
I thought it was strange to have such a developed library without this option !


Odd flag/language behaviour when updating from beta to v0.1.0

Hi All!
Great work on this! Loving it.  The update to v0.1.0 went really smooth with one small hiccup.
Here is what my translations selector looked like before during beta versions:

And here is what it looks like now with v0.1.0:

I prefer the 2nd version, with the caveat that it does not seem to be properly recognizing the ids and languages for the flags and I could see this leading to some confusion for editors.
Here is what my supportLanguages.js config file looks like:
export const supportedLanguages = [
  { id: "en", title: "English", isDefault: true },
  { id: "fr", title: "French" },
  { id: "en_ca", title: "English (CA)" },
  { id: "fr_ca", title: "French (CA)" },
  { id: "fr_fr", title: "French (FR)" },
];

export const baseLanguage = supportedLanguages.find((l) => l.isDefault);

EDIT:
I am going to leave this issue up but the more I think about it, maybe it is better off for my content editors to not have flags at all.  It is a bit confusing, as flags represent countries and not languages. I can see in the docs that I can workaround and export my own flag component and just return null so the flag area is always blank, which I think makes more sense. It doesn't seem to work right now as I think the part that needs to implemented has changed with the new language selector.

@ehowey This is now resolved in 0.1.1! I also re-enabled the custom flag implementation - you were right this got lost in the new UI

Wow - that was fast. Amazing!


Update flyway-core to 7.11.4

Updates org.flywaydb:flyway-core from 7.9.2 to 7.11.4.
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "org.flywaydb", artifactId = "flyway-core" } ]

labels: library-update, semver-minor

Superseded by #307.


Scaling control actions for BRAX environments

This PR is an attempt to fix https://github.com/google/brax/issues/472. All environments assume the control action is restricted to [-1,1].
For the few environments that have an action space that is not [-1,1] (humanoid, humanoidstandup, pusher, inverted_pendulum), the input action is linearly scaled to the limits of the action space inside the environment's step() function.
Please let me know if this is an appropriate solution, I'd be happy to iterate it. Note also that scaling the actions will mean that the "best" hyperparameters for these environments will likely change. Does this need to be addressed anywhere?

@btaba just a heads up I haven't forgotten about this and will run ablation studies soon! All my GPUs are currently in use for some other research.

@btaba I've just run ablation studies on the humanoid and pendulum environments.
Results are attached. Action scaling doesn't seem to harm performance in either case.
humanoid_results_scaled_v2.pdf
humanoid_results_v2.pdf
inverted_pendulum_results_scaled_v2.pdf
inverted_pendulum_results_v2.pdf
Would you like me to run them for humanoidstandup and pusher too?
For reference, here's the code I used to generate these results. It's basically exactly the same as the Brax training notebook but with some additional data logging/plotting.
import functools
import matplotlib.pyplot as plt
import numpy as np

from brax import envs
from brax.io import model
from brax.training.agents.ppo import train as ppo
from datetime import datetime
from pathlib import Path

# suffix = ""
suffix = "_scaled"
env_names = ["humanoid"] # ["inverted_pendulum", "humanoid", "pusher", "humanoidstandup"]
seeds = list(range(3))

def _get_fname(env_name, seed):
    dirpath = Path(__file__).resolve().parent
    fname = dirpath / f"{env_name}_results{suffix}_v{seed}"
    return fname

def train_model(env_name, seed, backend='positional'):

    # Environments and training functions from brax tutorial
    env = envs.get_environment(env_name=env_name, backend=backend)
    train_fn = {
    'inverted_pendulum': functools.partial(ppo.train, num_timesteps=2_000_000, num_evals=20, reward_scaling=10, episode_length=1000, normalize_observations=True, action_repeat=1, unroll_length=5, num_minibatches=32, num_updates_per_batch=4, discounting=0.97, learning_rate=3e-4, entropy_cost=1e-2, num_envs=2048, batch_size=1024, seed=seed),
    'humanoid': functools.partial(ppo.train,  num_timesteps=50_000_000, num_evals=10, reward_scaling=0.1, episode_length=1000, normalize_observations=True, action_repeat=1, unroll_length=10, num_minibatches=32, num_updates_per_batch=8, discounting=0.97, learning_rate=3e-4, entropy_cost=1e-3, num_envs=2048, batch_size=1024, seed=seed),
    'humanoidstandup': functools.partial(ppo.train, num_timesteps=100_000_000, num_evals=20, reward_scaling=0.1, episode_length=1000, normalize_observations=True, action_repeat=1, unroll_length=15, num_minibatches=32, num_updates_per_batch=8, discounting=0.97, learning_rate=6e-4, entropy_cost=1e-2, num_envs=2048, batch_size=1024, seed=seed),
    'pusher': functools.partial(ppo.train, num_timesteps=50_000_000, num_evals=20, reward_scaling=5, episode_length=1000, normalize_observations=True, action_repeat=1, unroll_length=30, num_minibatches=16, num_updates_per_batch=8, discounting=0.95, learning_rate=3e-4,entropy_cost=1e-2, num_envs=2048, batch_size=512, seed=seed),
    }[env_name]

    # Log rewards and print if required
    results = {"rewards": [], "stdev": [], "steps": [], "times": []}
    results["times"].append(datetime.now())

    def progress(num_steps, metrics):
        results["times"].append(datetime.now())
        results["steps"].append(num_steps)
        results["rewards"].append(metrics["eval/episode_reward"])
        results["stdev"].append(metrics["eval/episode_reward_std"])

        print("step: {} \t reward: {:.2f} \t stdev: {:.2f} \t time: {}".format(
            num_steps,
            metrics["eval/episode_reward"],
            metrics["eval/episode_reward_std"],
            results["times"][-1],))

    # Train
    _, params, _ = train_fn(environment=env, progress_fn=progress)
    times = results["times"]
    print(f'time to jit: {times[1] - times[0]}')
    print(f'time to train: {times[-1] - times[1]}')

    # Save params and metrics
    data = (*params, results)
    fname = _get_fname(env_name, seed)
    model.save_params(fname, data)

def plot_rewards(env_name, seeds):

    # Read in results
    xs, ys = [], []
    for seed in seeds:
        fname = _get_fname(env_name, seed)
        results = model.load_params(fname)[2]

        xs.append(np.array(results["steps"]))
        ys.append(np.array(results["rewards"]))
    x = np.vstack(xs).T
    y = np.vstack(ys).T

    # Plot formatting
    max_y = {'inverted_pendulum': 1100, 'humanoid': 13000, 'humanoidstandup': 75_000, 'pusher': 0}[env_name]
    min_y = {'reacher': -100, 'pusher': -150}.get(env_name, 0)

    # Make the plot and save it
    _, ax = plt.subplots()
    ax.plot(x, y)
    ax.set_ylim(min_y, max_y)
    ax.set_xlabel("Environment steps")
    ax.set_ylabel("Reward")
    ax.set_title(f"{env_name}{suffix} ({len(seeds)} random seeds)")
    plt.tight_layout()
    plt.savefig(f"{fname}.pdf")
    plt.close()

for env_name in env_names:
    for seed in seeds:
        train_model(env_name, seed)
    plot_rewards(env_name, seeds)

Hey @nic-barbara thanks for checking on those two envs, scaled plots LGTM!


Ronin accepts & normalizes Eth address as well

Description
Fixes #2236
How to test

Types of changes

Checklist

[ ] Create pull request as draft initially, unless its complete.
[ ] Add tests to cover changes as needed.
[ ] Update documentation as needed.
[ ] If there is a related Issue, mention it in the description.

If you're adding a new blockchain

[ ] I have read the guidelines for adding a new blockchain.

@catenocrypt https://github.com/trustwallet/wallet-core/blob/master/src/Ethereum/Signer.cpp#L115 this can be removed too

@catenocrypt https://github.com/trustwallet/wallet-core/blob/master/src/Ethereum/Signer.cpp#L115 this can be removed too

I don't think so, because there Ethereum::Address is used, which is not modified, so it will not accept ronin: address.

I am still hesitant to merge this, want to understand the root cause better, because with this change we just give up the point of a prefix, to reduce the risk of mixing up addresses.  To be able to normalize, the non-prefix version also has to be accepted as valid.

Let's see if we can fix assets repo instead

I attempted to change the token address in the assets repo, but it turns out that only Linux supports file names that contain ':', which may result in some breaking changes to the backend / assets api.


Workaround for pydantic 2.0 incompatibility, new AWS glue job runner, and new makefile targets.

This pull request:

Refactors requirements.txt, setup.py, and dev-requirements.txt to get make test-integration working again after PyIceberg's migration to use Pydantic v2 (see https://github.com/apache/iceberg/pull/7782 and https://github.com/ray-project/ray/issues/37019 for details).
Introduces new make targets to separate build artifact creation/maintenance from venv creation/maintenance. Also introduces appropriate dependencies between targets so that you don't have to know the correct order to run targets in anymore.
Introduces glue_runner.py to provide rapid feedback for cyclical distributed development workflows that follow steps like: (a) run $ python glue_runner.py deltacat/examples/hello_world.py --deploy-local-deltacat (b) {change deltacat/examples/hello_world.py and any other DeltaCAT files} (c) return to step [a]
Has a lot of additional lines of code and files changed due to also including a rebase of the iceberg branch on top of deltacat main branch. =(
We may want to break these out into separate PRs but, for now, the actual changed files to pay attention to are requirements.txt, dev-requirements.txt, setup.py, glue_runner.py, Makefile, s3-build-and-deploy.sh, deltacat/examples/hello_world.py, deltacat/examples/logging.py.

Tests run:

$ make test-integration (passing with latest Iceberg master commit and ray[data] == 2.6)
$ python glue_runner.py deltacat/examples/hello_world.py
$ python glue_runner.py deltacat/examples/logging.py

Glue runner tests were run with various permutations of supported arguments against local and PyPi builds of DeltaCAT.
Follow-up Issues to create:

Makefile Developer Documentation: Update README-development.md to include instructions on how to best use the new make targets and resolve common problems (e.g. make clean -> make build | make test-integration solves most dirty workspace problems).
Revert workarounds for PyIceberg+Ray+Pydantic v2 Incompatibilities: Once Ray 2.8 is released (or at least a nightly wheel is published that resolves the Pydantic v2 conflict) we need to (1) remove pyiceberg, pydantic, and ray[data] overrides from dev-requirements.txt and (2) update the corresponding lines for each of these dependencies in requirements.txt and setup.py to settle on either a single version or set of backwards-compatible versions.
DeltaCAT+Iceberg+Glue Read & Transform Example: Introduce a self-contained AWS Glue example script to setup an Iceberg catalog, read an Iceberg table, and run some simple distributed transforms on its data.
AWS Glue Runner User Guide: Introduce dedicated documentation guiding users through various AWS Glue Runner usage patterns.
DeltaCAT+Daft+Iceberg+Glue Read & Transform Support + Example: Refactor DeltaCAT's DistributedDataset to refer to either a Ray Dataset or a Daft Dataframe, add the ability to read Iceberg tables into Daft dataframes natively (no intermediate conversion through a Ray Dataset), and run some simple distributed transform on its data within an AWS Glue example script.
CI/CD Workflows: Introduce CI/CD workflows that run distributed integration tests to protect against regressions between DeltaCAT+RayData+AWS Glue+Daft+Iceberg integration points.
Support running latest PyIceberg Commit on AWS Glue for Ray: Investigate best ways to install latest PyIceberg from Apache Iceberg master branch inside of an AWS Glue job run. One possible method may involve using a command like pip install "https://github.com/apache/iceberg/archive/refs/heads/master.zip#subdirectory=python&egg=pyiceberg[s3fs]" once python is included in the master.zip artifact or extracting this zip within an Glue Job then configuring the Ray Runtime Environment to use the python subdirectory as the working_dir.
Merge Iceberg Branch into Main Branch: After we take care of PydanticV2 compatibility problems, we should consider merging out work here back into the main branch of DeltaCAT to prevent ongoing merges/rebases between the two (with merges effectively required once other developer dependencies are taken on the iceberg branch).

Some things I'm looking for feedback on:

AWS Glue Ray runtime environment installation for DeltaCAT takes longer than expected... a simple end-to-end job run that just prints a few lines to the console and exits now takes ~1.5-2 minutes here. Any suggestions, ideas to reduce this are welcome.
A command like aws logs tail /aws-glue/ray/jobs/script-log --follow prints a lot of duplicate log lines on my machine (2021 Apple M1 Pro with AWS CLI version: aws-cli/2.13.3 Python/3.11.4 Darwin/22.5.0 source/arm64) - curious if others see the same thing.
One-time IAM configuration for running AWS Glue jobs can be a bit tricky/finicky to configure correctly... in my case, I had to add full CloudWatch permissions to resolve a persistent internal server error in my AWS account here, so again curious if others see similar problems.

@jaychia @samster25 FYI! Feel free to review & try out the new AWS Glue Runner here. :)
@chappidim @jjacobj84 Please review and let me know what you think about the Glue integration. Specifically, looking for feedback about how to get the log lines printed by logging_worker printed out to the /aws-glue/ray/jobs/ray-worker-out-logs log group inside of deltacat/examples/logging.py since this hasn't been working so far (only plain print statements show up in the worker standard out logs).

@jaychia @samster25 FYI! Feel free to review & try out the new AWS Glue Runner here. :) @chappidim @jjacobj84 Please review and let me know what you think about the Glue integration. Specifically, looking for feedback about how to get the log lines printed by logging_worker printed out to the /aws-glue/ray/jobs/ray-worker-out-logs log group inside of deltacat/examples/logging.py since this hasn't been working so far (only plain print statements show up in the worker standard out logs).

@pdames can you point me to a JobRun ID mainly to understand 1) higher install time of delta cat 2) repeated log line printed issue. Now wrt log lines from deltacat/examples/logging.py missing in /aws-glue/ray/jobs/ray-worker-out-logs  is because the CW configuration that we have in Glue Ray for log group ray-worker-out-logs reads from log files at  /tmp/ray/session_latest/logs/worker-**.out. The logger used by deltacat by default writes to /tmp/deltacat/var/output/logs/. Would it be possible to configure the logger to write to /tmp/ray/session_latest/logs/worker-**.out

@jaychia @samster25 FYI! Feel free to review & try out the new AWS Glue Runner here. :) @chappidim @jjacobj84 Please review and let me know what you think about the Glue integration. Specifically, looking for feedback about how to get the log lines printed by logging_worker printed out to the /aws-glue/ray/jobs/ray-worker-out-logs log group inside of deltacat/examples/logging.py since this hasn't been working so far (only plain print statements show up in the worker standard out logs).

@pdames can you point me to a JobRun ID mainly to understand 1) higher install time of delta cat 2) repeated log line printed issue. Now wrt log lines from deltacat/examples/logging.py missing in /aws-glue/ray/jobs/ray-worker-out-logs is because the CW configuration that we have in Glue Ray for log group ray-worker-out-logs reads from log files at /tmp/ray/session_latest/logs/worker-**.out. The logger used by deltacat by default writes to /tmp/deltacat/var/output/logs/. Would it be possible to configure the logger to write to /tmp/ray/session_latest/logs/worker-**.out

@jjacobj84 Thanks for the info about the worker log file path - I've confirmed that both the driver and workers will pick up logs written to this path in the /aws-glue/ray/jobs/ray-worker-out-logs CloudWatch log group. However, I noticed that I also need to initialize a dedicated logger inside of each worker process for this to work correctly, which remains a minor con.
Job runs are still taking quite a while to install the runtime environment, here are the logs from my last job run (including embedded job run ID) which shows the time taken to complete the runtime environment setup as ~1m43s:
2023-09-01T04:10:17.155000+00:00 deltacat-example-runner-pdames-jr_82256a2a4fde372dc76572026451ae0166cf488587e35859405c20b1bcc0cbd3 2023-09-01 04:10:12,374 - __main__ - INFO - Setting up ray runtime environment to install custom lib
2023-09-01T04:11:55.429000+00:00 deltacat-example-runner-pdames-jr_82256a2a4fde372dc76572026451ae0166cf488587e35859405c20b1bcc0cbd3 2023-09-01 04:11:55,102 - __main__ - INFO - Successfully set up ray runtime environment installing custom lib

Looks good overall. Is the runtime 1.5 - 2mins with pip installing deltacat version from PyPI or from custom S3 URL?

Doesn't seem to make much difference (they're both slow). The log lines I gave to @jjacobj84 were from a custom S3 URL, and these are from an install from PyPi (0.1.18b14) which took 2m45s:
2023-09-01T05:17:24.428000+00:00 deltacat-example-runner-pdames-jr_2cff5bf30b7e0489a03714fcfca0de3c4135707c061b31d7df1d460b6376c76c 2023-09-01 05:17:19,604 - __main__ - INFO - Setting up ray runtime environment to install custom lib
2023-09-01T05:20:09.840000+00:00 deltacat-example-runner-pdames-jr_2cff5bf30b7e0489a03714fcfca0de3c4135707c061b31d7df1d460b6376c76c 2023-09-01 05:20:09,348 - __main__ - INFO - Successfully set up ray runtime environment installing custom lib

Alright, I think all existing questions/comments/issues are resolved by the latest push. @raghumdani @jjacobj84 @chappidim  Any additional questions or concerns before merging back to the iceberg branch?


fix: dual webkit and competing constants

split data result queue into one for detail view and one for sublist. Add synchronized block to all persistent content to ensure that the threads from the two webkits don't interleave when accessing the content. Clear the action and dispatch strings before signaling the webkit that a doAction result is available. This avoids a potential error if the webkit calls back into the Java layer before these fields can be cleared. Share these important string constants, don't re-declare them.
This is dependent upon #https://github.com/opendatakit/androidcommon/pull/17

runtests


Would it be feasible to use only vram as extension of main gpu?

Where gpu0 would do all the performance, but if gpu0 vram is full, it will offload anything above that to gpu1. Not to use gpu2 to sample any images, but only purpose being to hold the data, same way shared memory, aka ram already does, but gpu1 would process that data. Or would that be even slower process than making system ram holding it due to say 8GB's transfer rate over pcie gen 4 x4 lane bifurcation?

Hmm, interesting idea but I'm unsure how you'd force comfy to manage that internally. Like, you can just set where the model goes when it's not on the main GPU via the "offload_device" param of the model patcher, but that only allows you to specify one device. For this you'd have to make the actual backend multi-GPU aware so it doesn't try to unload the entire model onto your second GPU (which would just OOM unless it can fit the entire model). Maybe if you mess with the load function but yeah, no clue.
I guess if you have 2x3090s you could try it by editing the end of the UnetLoaderGGUF class in nodes.py to look like this:
        model = GGUFModelPatcher.clone(model)
        model.offload_device = torch.device("cuda:1")
        model.model.to(model.offload_device)
        model.patch_on_device = patch_on_device
        return (model,)

My second GPU is a P40 connected via PCIe 3.0 x1 so as expected it's awful lol. It'd be faster if I was reading from my m.2 drive.
CPU offload: 2.43s/it
CUDA1 offloat: 8.32s/it


Add extra class for card body

Question
Answer

Issue or Enhancement
Issue

License
MIT

This is a pull request for help to add an extra class for the card body.  There is a problem before when using a responsive data table inside the card. There is no attribute to pass extra class for card body.
Before
<x-adminlte-card title="Card Title"></x-adminlte-card>

After
<x-adminlte-card title="Card Title" body-class="table-responsive"></x-adminlte-card>

Checklist

[x] I tested these changes.
[x] I have linked the related issues.

Thanks, I will review this soon...

@shankhadevpadam I have reviewed these changes and made some small fixes. The extra classes for the card body could be helpful in some particular situations. However, and just to note, in your particular case all the problem may be fixed if you define a wrapper div with the table-responsive class, for example:
<x-adminlte-card title="Card Title">

       {{-- Here goes the datatable markup... --}}

</x-adminlte-card>

We now wait for @resslinger approval before merging this one...


MB-13329 As a Customer I will see a page where I can enter my pro gear details

Jira Ticket: MB-13329
Summary
This PR adds a page for the Pro-gear form, where weight tickets will be able to be added for pro-gear, and the first field for this form, including the Storybook component Customer Components > PPM Closeout > Pro Gear > Default, at this link.
Note that this page appears both when adding and editing a Pro-gear ticket, and while editing is not yet hooked up on the front-end, the route is in place.
Setup to Run Your Code

💻 You will need to use three separate terminals to test this locally.
Terminal 1
Start the Storybook locally.
make storybook

Terminal 2
Start the UI locally.
make client_run

Terminal 3
Start the Go server locally.
make server_run

Additional steps

Access the customer site
Login as a<EMAIL_ADDRESS>Click the "Upload PPM Documents" button
Click the "Add Pro-Gear Weight" button

Verification Steps for Author
These are to be checked by the author.

[ ] Tested in the Experimental environment (for changes to containers, app startup, or connection to data stores)
[ ] Request review from a member of a different team.
[ ] Have the Jira acceptance criteria been met for this change?

Verification Steps for Reviewers
These are to be checked by a reviewer.

Frontend

[ ] User facing changes have been reviewed by design.
[ ] There are no aXe warnings for UI.
[ ] This works in Supported Browsers and their phone views (Chrome, Firefox, IE, Edge).
[ ] There are no new console errors in the browser devtools
[ ] There are no new console errors in the test output
[ ] If this PR adds a new component to Storybook, it ensures the component is fully responsive, OR if it is intentionally not, a wrapping div using the officeApp class or custom min-width styling is used to hide any states the would not be visible to the user.

Warnings

:warning:

Files located in legacy directories (src/shared or src/scenes) have
been edited. Are you sure you don’t want to also relocate them to the new file structure?
View the frontend file org ADR for more information

:warning:
Please add the JIRA issue key to the PR title (e.g. MB-123)

  Generated by :no_entry_sign: dangerJS against 28c72a042740280812cf7e58c9bff44126513c4f

@ronaktruss
I pushed updates from your suggestions.
I went ahead with & over & since the encoded character isn't used anywhere in the codebase.
I'm also targeting the label on the radio buttons since Cypress complains that the actual inputs are offscreen, just FYI.

@ronaktruss I pushed updates from your suggestions. I went ahead with & over & since the encoded character isn't used anywhere in the codebase. I'm also targeting the label on the radio buttons since Cypress complains that the actual inputs are offscreen, just FYI.

Thanks for that!
Apologies for not making this clearer, but I actually think the way you had originally implemented the save and continue button was more correct and that we should potentially look into changing the previous versions that are just using Save & Continue directly to something more similar to what you had (though not necessarily in this PR). That said I don't think this has much of a functional impact since most browsers properly translate the & to what we expect. Happy to discuss this in more detail if you're interested.
The current cypress test targeting the labels are great! Just FYI I think you might be able to target the inputs doing something like cy.get('input[name="selfProGear"]');. This should return both radio options. If you want to select a specific one you can do something like cy.get('input[name="selfProGear"][value="true"]') where you replace the value with the respective value that is defined in ProGearForm.jsx. To then trigger the "checking" of that radio button you could do something like cy.get('input[name="selfProGear"][value="false"]').check({ force: true });


remove Promise package init.  using common logic for both MJS and CJS

The recent fix to handle both MJS and CJS compiling,  included a change in the implementation;   The package is currently loaded asynced (Promise),  which causes it to not be ready when calling it. (async function loadNativeBinding()).
Example where this issue impacts:
import { RedisClient } from "@aws/glide-for-redis";
const addresses = [
        {
             host: "localhost",
             port: 6379,
        },
];
const client = await RedisClient.createClient({
     addresses,
});

The code above crashes,  since RedisClient is not exported yet,  since it's an async initialization.
Description of changes:

Removed the async initialization,  making it sync.
Since the goal is to have support for both MJS and CJS,  the code was refactored a bit:

build will compile separate modules for MJS and CJS.

using common logic to avoid duplication between the modules.

Tested locally with both CJS and MJS.
Let me know what you think and if I can assist further 🙏
Thanks!
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

The recent fix to handle both MJS and CJS compiling, included a change in the implementation; The package is currently loaded asynced (Promise), which causes it to not be ready when calling it. (async function loadNativeBinding()).
Example where this issue impacts:
import { RedisClient } from "@aws/glide-for-redis";
const addresses = [
        {
             host: "localhost",
             port: 6379,
        },
];
const client = await RedisClient.createClient({
     addresses,
});

The code above crashes, since RedisClient is not exported yet, since it's an async initialization.
Description of changes:

Removed the async initialization,  making it sync.
Since the goal is to have support for both MJS and CJS,  the code was refactored a bit:

build will compile separate modules for MJS and CJS.

using common logic to avoid duplication between the modules.

Tested locally with both CJS and MJS.
Let me know what you think and if I can assist further 🙏 Thanks!
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Hi @ofirsnb,
Its been fixed, can you check that the version you use is 0.3.3?

Hey @avifenesh ! Thanks, I just saw that, tested few minutes ago. Using CommonJS seems to work great. Do you need me to test ESM too?
Anyway, if you need my further assistant, feel free to reach!! Feel free to close this PR, or take from it whatever you need for future reference:)

@ofirsnb I appreciate your efforts and willingness!
I've tested ECMA as well so you don't need to bother :)
If you want to join the effort we will really appreciate any feature ideas, improvements and manly any contributes.
You can have a look at the open issues if you find yourself looking for something that is currently known as needed.
We would love to see our comunity grows :)
And feel free to reach out here or in private for anything!


[MM-29500] Migrate changeCSS() to CSS variable in utils/utils.jsx

Summary
Migrate changeCss('.app*_body #channel*view.channel-view', `background: ${theme.centerChannelBg}`) in line 620 to CSS variables. Css applied in sass/base/_structure.scss.
Ticket Link
Fixes https://github.com/mattermost/mattermost-server/issues/15887
Jira: https://mattermost.atlassian.net/browse/MM-29500.
If I have to make changes in the PR, please do tell me. Thanks

Hello @oliverJurgen,
Thanks for your pull request! A Core Committer will review your pull request soon. For code contributions, you can learn more about the review process here.

I applied the changes

@calebroseland Hello, any more changes I should do?

@oliverJurgen I think it would be a good chance to simplify and put in just .channel-view down by :80. If you wouldn't mind, that be great :)

sounds good, will do that :)

please check my latest change @calebroseland  :)

This PR has been automatically labelled "stale" because it hasn't had recent activity.
A core team member will check in on the status of the PR to help with questions.
Thank you for your contribution!
/cc @jasonblais @jfrerich @emilyacook

@calebroseland @asaadmahmood is this ready to merge?

@jasonblais yes, my bad.


How to retrieve list of matches that are 1) pub matches and 2) parsed

I would like to do some analyses on match data using machine learning algorithms. This should be based on matches that have the following characteristics:

only pub matches (i.e. no pro matches, no matches from leagues)
only parsed matches (to be able to use for example gold and xp values for each player over the course of a match)

How can I find and download such matches using the API?
I (unsuccessfully) tried the following approach:

Use a query to retrieve a list of parsed pub-matches
Download detailed match data for a sample of that list

Using the API, 2) seems to be easy.
However, I couldn't achieve 1), probably because I don't fully understand the db scheme of the project. I tried to use the tables player_matches, matches and public_matches.
This seems to return only parsed matches (good!), but only in leagues (bad):
select matches.match_id, matches.leagueid
from matches
where version > 0
limit 100

Is it correct that matches only contains league matches? And in consequence, player_matches also only contains league matches?
public_matches seems to contain some matches that are not from a league, but parsed (for example, match id<PHONE_NUMBER>). Player-level gold charts over time are shown here: https://www.opendota.com/matches/3925110408/graphs
However, these queries return empty results:
select *
from match_logs
where match_id =<PHONE_NUMBER>

select *
from player_matches
where match_id =<PHONE_NUMBER>

Where do the data for the player-level gold charts over time come from?
Thanks in advance for any help! :)

You can use GET /publicMatches ( https://docs.opendota.com/#tag/public-matches ) to get a sample of recent public matches. The explorer will only return matches in a league like you noticed.
If you want to get more data about a specific match you'll need to use the GET /matches/{match_id} endpoint ( https://docs.opendota.com/#tag/matches%2Fpaths%2F~1matches~1{match_id}%2Fget )

You can use GET /publicMatches ( https://docs.opendota.com/#tag/public-matches ) to get a sample of recent public matches.

I was aiming to use data from at least 20.000 matches or so. GET /publicMatches doesn't really help much, because it only returns 100 matches.
Do you know if these 100 matches are parsed?

The explorer will only return matches in a league like you noticed.

Is this really the case?
SELECT COUNT(match_id)
FROM public_matches

returns 41.324.737 matches. Are these all league matches?

If you want to get more data about a specific match you'll need to use the GET /matches/{match_id} endpoint ( https://docs.opendota.com/#tag/matches%2Fpaths%2F~1matches~1{match_id}%2Fget )

Yes, my initial idea was to use GET /matches/{match_id} for every match from a list of 20.000 matches that are 1) parsed and 2) not from a league. However, I don't know how to create that list.
A possible workaround might be to create a large list of public matches (let's say 100.000), without knowing which matches on this list are parsed and which are not. I could than download data for each of these matches using GET /matches/{match_id}. After downloading all data for the 100.000, I could detect and remove unparsed matches. However, this seems very inefficient; I would very much prefer to only download data for matches that have been parsed.

You are correct, public_matches is an approximately 15% sample of all matches. There is no parsed data on these rows, so you'll have to take the approach you described where you get a sample and then check for parsed matches. Alternatively, you could sample and then request parses through the API for your sample if the matches aren't too old.

@howardchung, Thanks for this info! I'll look into the two alternatives. :)

You are correct, public_matches is an approximately 15% sample of all matches. There is no parsed data on these rows, so you'll have to take the approach you described where you get a sample and then check for parsed matches. Alternatively, you could sample and then request parses through the API for your sample if the matches aren't too old.

public_matches is a true random sample ?

No, it's not. We pick from matches that have at least 3 players with rank data.

No, it's not. We pick from matches that have at least 3 players with rank data.

You mean num_rank_tier >= 3? But I find some matches in public_matches which num_rank_tier is 2.
How can I get true random sampling from OpenDota api?
Most new players don't have rank, If it is random sampling, we can do a lot of interesting analysis.

@howardchung

You can try fetching the raw match IDs from the Steam WebAPI and then making OpenDota request for a sample of those IDs.


issues of running python main_fed.py --dataset mnist --num_channels 1 --model cnn --epochs 50 --gpu 0

when I tried to run python main_fed.py --dataset mnist --num_channels 1 --model cnn --epochs 50 --gpu 0, then it shows me a problem.
Jians-Air:FedAvg jiansun$ python main_fed.py --dataset mnist --num_channels 1 --model cnn --epochs 50 --gpu 0
Traceback (most recent call last):
File "main_fed.py", line 11, in
from torchvision import datasets, transforms
File "/Library/Python/2.7/site-packages/torchvision/init.py", line 1, in
from torchvision import models
￼

Sorry, I can't understand what's your problem. Did you install torchvision?


cannot start wagtail in an alpine Linux container

Found a bug? Please fill out the sections below. 👍
Issue Summary
manage.py runserver coredumps when I try to start wagtail:
# python3 manage.py runserver
Performing system checks...

Segmentation fault (core dumped)

dmesg reports:
python3[4738]: segfault at 7fbc7e14fff8 ip 00007fbc7ed934da sp 00007fbc7e14fff0 error 6 in libpython3.5m.so.1.0[7fbc7ec8e000+3d6000]
Any  idea what's wrong ? How do I know what's in the coredump ?  I think I followed closely the installation guide.
Steps to Reproduce
I did start an empty alpine linux container and then I ran the following commands in the same order:
apk add python3-dev g++ gcc musl-dev wget make cmake zlib-dev libjpeg-dev libwebp-dev libpng-dev tiff-dev jasper-dev linux-headers postgresql-dev
apk add python3 zlib libjpeg libwebp libpng tiff libjasper
ln -s /usr/include/locale.h /usr/include/xlocale.h
pip3 install cython numpy
wget --no-check-certificate https://github.com/Itseez/opencv/archive//master.zip -O /tmp/opencv.zip
cd /tmp && unzip /tmp/opencv.zip
cd /tmp/opencv-master
mkdir build && cd build
cmake -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/local -D INSTALL_C_EXAMPLES=OFF -D INSTALL_PYTHON_EXAMPLES=OFF -D BUILD_EXAMPLES=OFF  -D WITH_FFMPEG=NO -D WITH_IPP=NO -D WITH_OPENEXR=NO ..
make && make install
pip3 install wagtail django-redis psycopg2 wand
mkdir /wagtail && cd /wagtail
wagtail start mysite
cd mysite
pip3 install -r requirements.txt
python3 manage.py migrate
python3 manage.py createsuperuser
python3 manage.py runserver

Technical details

Python version: 3.5.2
Django version: 1.10.6.
Wagtail version: 1.9
Browser version: not working up to there.

Hi @digrouz,
I can confirm I'm seeing this when using vagrant with https://atlas.hashicorp.com/maier/boxes/alpine-3.4-x86_64 as the base image - trying to pin down exactly what's triggering it now. This is very unlikely to be a Wagtail-specific issue, since Wagtail is pure Python code - a segfault would have to originate from (Alpine's build of) the Python interpreter, or one of Wagtail's native dependencies such as Pillow.

From further experimenting, and reading related issues such as https://github.com/python-pillow/Pillow/issues/1935 and https://github.com/lovell/sharp/issues/242, it appears that Alpine Linux (and specifically the musl-libc library it uses) has a very limited stack size by default, and Wagtail is hitting that limit. I've replicated the issue on a plain Django project by adding from import1 import foo to urls.py, where import1.py is a module consisting of from import2 import foo and so on up to import30.
It looks like you can raise the limit by adding a line such as the following to manage.py, just after the from __future__ import line:
import threading; threading.stack_size(100000)

...but personally I'd recommend switching to a more tried-and-tested Linux distro such as Ubuntu or Debian. If Alpine needs a workaround for something as routine as this, I suspect there'll be more problems further down the line..

ok, thanks for figuring out what was the issue, I'll give a try to ubuntu then.


Issue gitignore

Resolves #536
A line is added to gitignore to ignore the files in examples/*/*.pdf files.

@jaclark5 Thanks for this!


[SPARK] CREATE EXTERNAL TABLE LOCATION

Description of changes

Support Spark CREATE EXTERNAL TABLE LOCATION command

@cloud-fan

We need to wait Spark publish a RC including https://github.com/apache/spark/pull/47592 so that we can test this PR.
This is because Spark only support built-in file formats in spark_catalog where there is a bug that a custom session catalog does not work functionally before the fix.


MGNAPIDeploy fails to deploy

Describe the bug
Cannot deploy the CFN stack due to MGNAPIDeploy failed to deploy.
To Reproduce
Download v3.3.5 CloudFormation template and deploy with these parameters:

DeploymentType: Private
CognitoFederation: false
Tracker: true
ReplatformEC2: true
Others left with sensible values

Expected behavior
A clear and concise description of what you expected to happen.
Please complete the following information about the solution:

[x] Version: v3.3.5

To get the version of the solution, you can look at the description of the created CloudFormation stack. For example, "(SO0097) - AWS CloudEndure Migration Factory Solution. Version v1.1.0".

[x] Region: ap-southeast-1
[x] Was the solution modified from the version published on this repository? NO
[x] If the answer to the previous question was yes, are the changes available on GitHub?
[x] Have you checked your service quotas for the sevices this solution uses? - YES, and this is a brand new account
[x] Were there any errors in the CloudWatch Logs? - No

Screenshots

Additional context

I tried to diagnose the problem and figured out that the AppMigService might need to have some resources of ToolsAPI to be ready, adding the following DependsOn will help:
AppMigService:
    Type: AWS::CloudFormation::Stack
    DependsOn:
    - ToolsAPIDeploy

I plan to create a PR to accommodate this fix.

Hi @sleepwithcoffee, thanks for raising this issue, the PR you have provided will be pulled ASAP.

As of v4.0.0 your update has been included. Thanks again for the contribution.


WebP support?

I was wondering if binwalk already supports finding webp files in a dump/bin and if not, if that would be something easy to be implemented :)

Type “webp” into the github search box for this project, that will probably answer your first question
I can’t speak to the second, but generally speaking if webp files have a “magic” and/or well-known header, then no, it’s not terribly difficult. Though you may have noticed this repository is no longer maintained so you’re on your own (see Binwalk pro)

Binwalk v3 now supports WebP image detection and extraction:


Largest Arithmetic Subarray

Description 📜
Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context.
Fixes #2305
Longest Arithmetic Subarray

Type of change 📝

[ ] Au#dio (Should be in mp3 format Includes speech clarity, Concise ,Low distortion)
[ ] Vi#deo (Animations, screen-recordings, presentations and regular explanatory films are all possibilties etc)
[x] Documentation (Content Creation in the form of codes or tutorials)
[ ] Other (If you choose other, Please mention changes below)

Domain of Contribution 📊

[ ] Android Dev #(Flutter)
[ ] Android Dev #(Java)
[ ] Android Dev #(Kotlin)
[ ] Backend Dev #(Java)
[ ] Backend Dev #(.NET)
[ ] Backend Dev #(PHP)
[ ] Backend Dev #(Python)
[ ] C/#CPP
[ ] Competitive #Programming
[ ] Cyber #Security
[x] DSA
[ ] Data#base
[ ] Datascience with #Python
[ ] Datascience with #R
[ ] Frontend Dev #HTML/CSS/JS
[ ] Frontend Dev #React/Angular/Vue
[ ] Go#lang
[ ] Interview #Prep
[ ] Java_#Domain
[ ] Java#Script
[ ] ME#RN
[ ] Machine #Learning
[ ] Open #Source
[ ] Python_#Domain
[ ] Ru#st
[ ] Statis#tics
[ ] UI/#UX

Checklist ✅

[x] I follow Contributing Guidelines & Code of conduct of this project.
[x] I have performed a self-review of my own code or work.
[x] I have commented my code, particularly in hard-to-understand areas.
[x] My changes generates no new warnings.
[x] I'm GWOC'21 contributor

@nimisha-yadav can you please review my work once

/assign documenation

Please review @harshsri28 @sneha-nitdgp

Also @priyanka001tech, do send in your full name and DSA Batch number

Discuss about time complexity and space complexity

Okk sir I'll do that.

Also @priyanka001tech, do send in your full name and DSA Batch number

Yes ma'am
Name: Priyanka Kothari
DSA Batch no: 1

Good.

LGTM

When the pull request is going to merge??

LGTM. Just do the following changes and you're good to go. Update fork Put your file in Standard Problems folder in Array Rename it 1.1.5_Largest_Arithmetic_Subarray.md

Ok ma'am I'll do it

@nimisha-yadav ma'am please review this