Reinstate SSL STs in semaphore with saved cfssl binaries

Looks like this is failing due to a ST change when the SSL STs were disabled . Looking into it.

This is passing now. @tomdee, mind giving it another look?

LGTM


Example of a simple (very routine) problem in economics

Hi & thank you for this package!
I'd like to use it to solve a very simple (routine) problem in economics.

This is a consumption-saving problem.
The household lives during time: t ∈ [0, 10]
The household's wealth @ time t is: B(t)
The household's choice is consumption: c(t)
The household is endowed w/ $100: B(0) = 100
The household cannot die in debt: B(10) = 0
I have the closed form solution for comparison.
Is it possible to solve this problem w/ your package?

Hi there, thanks for reaching out. The principle limitation of InfiniteOpt (in its current form) is support for non-quadratic nonlinear functions of decision variables (e.g., log(c(t))). This is due to our dependence on JuMP which does allow general nonlinear functions/expressions when modeling directly in JuMP but this nonlinear interface is unextendible for packages like ours. This limitation will be lifted once https://github.com/jump-dev/MathOptInterface.jl/issues/846 is addressed.
Until then, our currently accepted expression types are described here.
There are plans to make a temporary patch to address this problem (#16). I hope to address this sometime this summer, but we'll see what my schedule allows.
It might be possible to use a change of variables to adapt your formulation. Otherwise, you can transcribe the problem manually into a traditional NLP form that can be used by JuMP.

something like:
using JuMP, Ipopt
Δt = 0.01
T = 10
time = 0.0:Δt:T
model = Model(Ipopt.Optimizer)
@variable(model, c[time] >= 0.0001)  # To avoid log(0).
@variable(model, B[time])
fix(B[0.0], 100.0)
fix(B[T], 0.0)
@NLobjective(model, Max, sum(exp(-0.01 * t) * log(c[t]) for t in time))
@constraint(
    model,
    [i = 2:length(T)],
    B[t] == B[t - 1] + Δt * (0.05 * B[t - 1] - c[t]),
)
optimize!(model)

@pulsipher
I repeated it w/ quadratic utility (-1)*(c - 100)^2 and it works beautifully except for at the very corners:
using InfiniteOpt, JuMP, Ipopt, Plots
m = InfiniteModel(Ipopt.Optimizer)
@infinite_parameter(m, t in [0, 10],num_supports = 10_001)
@infinite_variable(m, B(t))
@infinite_variable(m, 0 <= c(t) <= 99)  #m, c(t) >= 0
@objective(m, Max, integral( (-1)*(c - 100)^2, t) )
@BDconstraint(m, (t == 0), B == 100)      # B[t=0] = 100
@BDconstraint(m, (t == 10), B == 0)
@constraint(m, c1, @deriv(B, t) == 0.01*B - c)

m
optimize!(m)
termination_status(m)
opt_obj = objective_value(m) #
#c, B, value.(c), value.(B)
c_opt, B_opt = value.(c), value.(B)
u_ts = supports.(c)
u_ts =[u_ts[j][1] for j in eachindex(u_ts)]

plot(u_ts,   B_opt, lab = "B: wealth balance")
plot!(u_ts,  c_opt, lab = "c: consumption")

Here are the control/state vars from your package:

From Mathematica:

This can be a nice example of a (very) simple econ problem for your package!
Is it yet possible to allow exponentially discounted objective? exp(-0.01*t)*(-1)*(c(t) - 100)^2

@odow thanks!
I tried your method applied to the quadratic example (above) & it doesn't seem to work:
using JuMP, Ipopt, Plots
Δt = 0.01; T = 10.0; time = 0.0:Δt:T;
m = Model(Ipopt.Optimizer)
@variable(m, 0.0001 <= c[time] <= 99.0)
@variable(m, B[time])
fix(B[0.0], 100.0)
fix(B[T], 0.0)
@NLobjective(m, Max, sum((-1.0)*(c[t] - 100.0)^2.0 for t in time) )
@constraint(m,
    [i = 2:length(T)],
    B[t] == B[t - 1] + Δt * (0.01 * B[t - 1] - c[t])
    )
optimize!(m)
objective_value(m)
c_opt, B_opt = value.(c), value.(B)
plot(time,   B_opt.data, lab = "B: wealth balance")
plot!(time,  c_opt.data, lab = "c: consumption")

Do you have any idea what it might be?

The perils of writing code without checking that it runs:
Δt = 0.01; T = 10.0; time = 0.0:Δt:T;
m = Model(Ipopt.Optimizer)
@variable(m, 0.0001 <= c[time] <= 99.0)
@variable(m, B[time])
fix(B[0.0], 100.0)
fix(B[T], 0.0)
@NLobjective(m, Max, sum((-1.0)*(c[t] - 100.0)^2.0 for t in time) )
for i in 2:length(time)
    t_1, t = time[i - 1], time[i]
    @constraint(m, B[t] == B[t_1] + Δt * (0.01 * B[t_1] - c[t]))
end
optimize!(m)

@odow works beautifully. Thank you!
Do you know why there is an initial jump in the control variable @ first index:

@azev77 That's a cool example, nice job setting it up.
Yes you can add in the discount factor since it is only a function of the infinite parameter t which is not a decision variable. There are 2 ways to accomplish this.
Option 1 use a parameter function and embed it in the expression:
using InfiniteOpt, JuMP, Ipopt, Plots

discount(t) = exp(-0.01 * t) # this will work as a function for our parameter function

m = InfiniteModel(Ipopt.Optimizer)
@infinite_parameter(m, t in [0, 10], num_supports = 100)
@infinite_variable(m, B(t))
@infinite_variable(m, 0 <= c(t) <= 99)
pfunc = parameter_function(discount, t)
@objective(m, Min, integral(pfunc * (c - 100)^2, t))
@BDconstraint(m, (t == 0), B == 100)
@BDconstraint(m, (t == 10), B == 0)
@constraint(m, c1, deriv(B, t) == 0.01*B - c)
optimize!(m)
termination_status(m)
opt_obj = objective_value(m) #
c_opt, B_opt = value(c), value(B)
ts = supports(t)
plot(ts,   B_opt, lab = "B: wealth balance")
plot!(ts,  c_opt, lab = "c: consumption")

Option 2 use the weight_func keyword in integral:
using InfiniteOpt, JuMP, Ipopt, Plots

discount(t) = exp(-0.01 * t) # this will work as a weight function in the integral
m = InfiniteModel(Ipopt.Optimizer)
@infinite_parameter(m, t in [0, 10], num_supports = 100)
@infinite_variable(m, B(t))
@infinite_variable(m, 0 <= c(t) <= 99)
@objective(m, Min, integral((c - 100)^2, t, weight_func = discount))
@BDconstraint(m, (t == 0), B == 100)
@BDconstraint(m, (t == 10), B == 0)
@constraint(m, c1, deriv(B, t) == 0.01*B - c)
optimize!(m)
termination_status(m)
opt_obj = objective_value(m) #
c_opt, B_opt = value(c), value(B)
ts = supports(t)
plot(ts,   B_opt, lab = "B: wealth balance")
plot!(ts,  c_opt, lab = "c: consumption")

weird error
julia> @objective(m, Min, integral(pfunc * (c - 100)^2, t))
ERROR: *(::GeneralVariableRef,::GenericQuadExpr{Float64, GeneralVariableRef}) is not defined. Are you trying to build a nonlinear problem? Make sure you use @NLconstraint/@NLobjective.
Stacktrace:
 [1] error(s::String)
   @ Base .\error.jl:33
 [2] *(#unused#::GeneralVariableRef, #unused#::GenericQuadExpr{Float64, GeneralVariableRef})
   @ JuMP C:\Users\azevelev\.julia\packages\JuMP\MIPb6\src\operators.jl:409
 [3] macro expansion
   @ C:\Users\azevelev\.julia\packages\MutableArithmetics\DcLoq\src\rewrite.jl:279 [inlined]
 [4] macro expansion
   @ C:\Users\azevelev\.julia\packages\JuMP\MIPb6\src\macros.jl:1150 [inlined]
 [5] top-level scope
   @ REPL[26]:1

julia>

Oh, my bad that temporarily introduces an expression with 3 variable reference multiplied together. This can be alleviated by introducing a placeholder variable:
using InfiniteOpt, JuMP, Ipopt, Plots

discount(t) = exp(-0.01 * t) # this will work as a function for our parameter function

m = InfiniteModel(Ipopt.Optimizer)
@infinite_parameter(m, t in [0, 10], num_supports = 100)
@infinite_variable(m, B(t))
@infinite_variable(m, 0 <= c(t) <= 99)
@infinite_variable(m, placeholder(t))
pfunc = parameter_function(discount, t)
@constraint(m, placeholder == (c - 100)^2)
@objective(m, Min, integral(pfunc * placeholder, t))
@BDconstraint(m, (t == 0), B == 100)
@BDconstraint(m, (t == 10), B == 0)
@constraint(m, c1, deriv(B, t) == 0.01*B - c)
optimize!(m)
termination_status(m)
opt_obj = objective_value(m) #
c_opt, B_opt = value(c), value(B)
ts = supports(t)
plot(ts,   B_opt, lab = "B: wealth balance")
plot!(ts,  c_opt, lab = "c: consumption")

Alternatively, option 2 provides a more compact avenue.

@odow is there a way to do this in JuMP.jl with:

derivative in the constraint instead of @constraint(m, B[t] == B[t_1] + Δt * (0.01 * B[t_1] - c[t]))
integral in the objective instead of @NLobjective(m, Max, sum((-1.0)*(c[t] - 100.0)^2.0 for t in time) )

Do you know why there is an initial jump in the control variable @ first index:

c[0.0] isn't used, so it just picks a random value. You could fix it to zero or just plot from the second element onwards.

is there a way to do this in JuMP.jl

No. @pulsipher has done an excellent job adding these extensions to InfiniteOpt instead...

@pulsipher can I help you add this econ example to the docs?

@azev77 Ya that would be great. I think this would make a nice addition to the Examples page in the docs. Please refer the contribution guide (https://github.com/pulsipher/InfiniteOpt.jl/blob/master/CONTRIBUTING.md) for info on how to implement this and make a pull request. You can follow-up here with any questions you have.

Unbounded infinite domains can be defined directly @infinite_parameter(model, t in [0, Inf]) however this syntax has not well been explored.
In this case, the only integral evaluation method we support is Gauss Laguerre quadrature (see the developmental docs https://pulsipher.github.io/InfiniteOpt.jl/dev/guide/measure/#Evaluation-Methods). However, this does not work properly on the current release, but this will be fixed with the next release coming in about 2 weeks.
I am not quite sure how to enforce the limit constraint, perhaps we could try it via a bounded constraint with a sufficiently large t. My experience is limited with infinite time horizon problems, but I look further in how to properly model it when I have some free time later this week.

@azev77 Sorry that it's been a little while, but I finally finished releasing the new version and now am taking a look back at this.
First, the new version has some breaking syntax changes for better long term stability, so here is the updated finite horizon case:
using InfiniteOpt, Ipopt, Plots

discount(t) = exp(-0.01 * t) # this will work as an additional weight function in the integral

m = InfiniteModel(Ipopt.Optimizer)
@infinite_parameter(m, t in [0, 10], num_supports = 100)
@variable(m, B, Infinite(t))
@variable(m, 0 <= c <= 99, Infinite(t))
@objective(m, Min, integral((c - 100)^2, t, weight_func = discount))
@constraint(m, B == 100, DomainRestrictions(t => 0))
@constraint(m, B == 0, DomainRestrictions(t => 10))
@constraint(m, c1, deriv(B, t) == 0.01*B - c)
optimize!(m)
termination_status(m)
opt_obj = objective_value(m) #
c_opt, B_opt = value(c), value(B)
ts = supports(t)
plot(ts,   B_opt, lab = "B: wealth balance")
plot!(ts,  c_opt, lab = "c: consumption")

Now we can try to implement the infinite horizon case:
using InfiniteOpt, Ipopt, Plots

int_discount(t) = exp(0.99 * t) # additional weight function for the integral (will be multiplied by e^-t by Gauss Laguerre)
discount(t) = exp(-0.01 * t) # weight function for parameter function in the limit constraint

m = InfiniteModel(Ipopt.Optimizer)
@infinite_parameter(m, t in [0, Inf])
@variable(m, B, Infinite(t))
@variable(m, 0 <= c <= 99, Infinite(t))
@parameter_function(m, dis == discount(t))
@objective(m, Min, integral((c - 100)^2, t, eval_method = GaussLaguerre(), num_nodes = 100, weight_func = int_discount))
@constraint(m, B == 100, DomainRestrictions(t => 0))
@constraint(m, dis * B == 0, DomainRestrictions(t => supports(t)[end])) # not sure what u'(c_t) is...
@constraint(m, c1, deriv(B, t) == 0.01*B - c)
optimize!(m)
termination_status(m)
opt_obj = objective_value(m) #
c_opt, B_opt = value(c), value(B)
ts = supports(t)
plot(ts,   B_opt, lab = "B: wealth balance")
plot!(ts,  c_opt, lab = "c: consumption")

Let me know if you have any further questions.
As for adding an example to the docs, the documentation now details how to do this here.

Hi:

I created a PR: https://github.com/pulsipher/InfiniteOpt.jl/pull/141
I tried hard to follow your example: hovercraft.jl
This ended up taking me a lot of time. Sorry if I missed some part of your format etc

Here is the first figure (created in my example):

Here is the second figure (where I compare your solution w/ the closed form solution):

Two issues:

Writing the objective -(c - k)^2.0 gives an error, while -(c - k)^2 works (this feels like a bug)
I can get the value function at the initial point objective_value(m), how do I get the entire value function v(B,t)?

Hi @azev77, thank you for your contribution please refer to #141 to finish that up.

Writing the objective -(c - k)^2.0 gives an error, while -(c - k)^2 works (this feels like a bug)

This is because the JuMP.@objective and JuMP.@constraint macros only support linear and quadratic expressions, hence only integer powers of 2 are allowed. However, perhaps the JuMP developers could change it to allow 2.0 as well.

I can get the value function at the initial point objective_value(m), how do I get the entire value function v(B,t)?

I am not exactly sure where v(B,t) shows up in your problem, but I can explain how to get the values of measured (e.g., integrated functions). The objective value of a integrated function will give you the optimized value of the integral of so just a single number. To get the optimal values of the integrand function, we can do the following (just a general example):
m = InfiniteModel(Ipopt.Optimizer)
@infinite_parameter(m, t in [0, 1])
@variable(m, y >= 0, Infinite(t))

@expression(m, integrand, (1 - y)^2) # make the integrand function
@objective(m, Min, integral(integrand, t)) # make the objective function
optimize!(m)

opt_obj = objective_value(m) # the optimized scalar value of the integral
opt_integrand = value(integrand) # the values of the integrand function

Note that in this case you won't want to use the weight_func keyword argument, but rather use a parameter function.

Solve an infinite horizon version of the problem (w/ appropriate TVC)

This would be interesting, I don't have a lot of experience with the solution methods for these types of problems and I am sure we can make additions to enhance InfiniteOpt's capabilities as we better learn what else would be helpful.

Solve the problem w/ non-linear objective once jump-dev/MathOptInterface.jl#846 is resolved

Yep, I look forward to that day!

Add inequality constraints on the state-variables (borrowing constraints etc)

That should be straightforward to do.

Solve a stochastic version of the problem (if possible w/ InfiniteOpt.jl)

Yes this should be possible depending on exactly how you want to model the stochastic elements. Stochastic optimization is my primary expertise.

A quick side note about the PR, I'd rather discuss here.
The most generic simple OC problem economists solve can be written:

The way I coded the current PR, with a generic u and BC, so a user can only tweak the (infinite-dimensional) parameter to change the utility function or the constraint without having to change any of the code, bc the objective has u(c)...
If you prefer I will be happy to put the expression directly inside, but I think the current form is potentially more appealing to economists...

I see, that makes sense. Thank you for the clarification.  That should be fine as it is then.

I look forward to working on the four todos this summer

@pulsipher the table below summarizes the 4 main approaches to modeling in economics:
(to be clear, there are far more than 4 approaches to solving these...)

We now have a simple example in the docs for solving continuous time-deterministic problems (top right quadrant).
I can't find a deterministic discrete time example in the docs (top left quadrant).
Here is my attempt:
using InfiniteOpt, Ipopt, Plots
ρ = 0.020  # discount rate
k = 90.0  # utility bliss point
T = 10.0   # life horizon
r = 0.05   # interest rate
B0 = 100.0 # endowment
u(c; k=k) = -(c - k)^2       # utility function
#discount(t; ρ=ρ) = exp(-ρ*t) # discount function

opt = Ipopt.Optimizer   # desired solver

m = InfiniteModel(opt)
@variable(m, B[0.0:1.0:11.0]) ## state variables   B[0.0] B[1.0] ⋯ B[11.0]
@variable(m, c[0.0:1.0:10.0]) ## control variables c[0.0] c[1.0] ⋯ c[10.0]

@objective(m, Max, sum(u(c[i]) for i ∈ 0.0:1.0:10.0) )
@constraint(m, [i in 0.0:1.0:T], B[i+1] == B[i] + r*B[i] - c[i] )
@constraint(m, B[0.0] == B0)
@constraint(m, B[11.0] == 0)

julia> optimize!(m)
ERROR: ArgumentError: reducing over an empty collection is not allowed
Stacktrace:
  [1] _empty_reduce_error()
    @ Base .\reduce.jl:299
  [2] mapreduce_empty(f::Function, op::Base.BottomRF{typeof(Base.mul_prod)}, T::Type)
    @ Base .\reduce.jl:342
  [3] reduce_empty(op::Base.MappingRF{typeof(length), Base.BottomRF{typeof(Base.mul_prod)}}, #unused#::Type{Union{}})
    @ Base .\reduce.jl:329
  [4] reduce_empty_iter
    @ .\reduce.jl:355 [inlined]
  [5] reduce_empty_iter
    @ .\reduce.jl:354 [inlined]
  [6] foldl_impl(op::Base.MappingRF{typeof(length), Base.BottomRF{typeof(Base.mul_prod)}}, nt::Base._InitialValue, itr::Tuple{})
    @ Base .\reduce.jl:49
  [7] mapfoldl_impl(f::typeof(length), op::typeof(Base.mul_prod), nt::Base._InitialValue, itr::Tuple{})
    @ Base .\reduce.jl:44
  [8] mapfoldl(f::Function, op::Function, itr::Tuple{}; init::Base._InitialValue)
    @ Base .\reduce.jl:160
  [9] mapfoldl
    @ .\reduce.jl:160 [inlined]
 [10] #mapreduce#218
    @ .\reduce.jl:287 [inlined]
 [11] mapreduce
    @ .\reduce.jl:287 [inlined]
 [12] #prod#224
    @ .\reduce.jl:555 [inlined]
 [13] prod(f::Function, a::Tuple{})
    @ Base .\reduce.jl:555
 [14] build_transcription_model!(trans_model::Model, inf_model::InfiniteModel; check_support_dims::Bool)
    @ InfiniteOpt.TranscriptionOpt C:\Users\azevelev\.julia\packages\InfiniteOpt\8U5IK\src\TranscriptionOpt\transcribe.jl:875
 [15] #build_optimizer_model!#95
    @ C:\Users\azevelev\.julia\packages\InfiniteOpt\8U5IK\src\TranscriptionOpt\optimize.jl:34 [inlined]
 [16] build_optimizer_model!
    @ C:\Users\azevelev\.julia\packages\InfiniteOpt\8U5IK\src\TranscriptionOpt\optimize.jl:32 [inlined]
 [17] build_optimizer_model!(model::InfiniteModel; kwargs::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ InfiniteOpt C:\Users\azevelev\.julia\packages\InfiniteOpt\8U5IK\src\optimize.jl:525
 [18] build_optimizer_model!

InfiniteOpt.jl is intended to model infinite-dimensional optimization problems such that we decouple the infinite (e.g., continuous) model from the discrete model. Hence, InfiniteOpt is not intended for modeling finite models (e.g., discrete time models directly), these should instead be modeled in JuMP as you did above which will use the same syntax since InfiniteOpt is a JuMP extension (basically if you don't use @infinite_parameter then all you have to do is change InfiniteModel to Model). I have added #151 to fix the erroneous error you get above and instead get a nice warning along with the solution (this warning is triggered when no infinite parameters are added to the model).
Note that in the JuMP above, registering d is unnecessary since it doesn't use decision variables. Also, the performance will be enhanced by using a power of integer 2 in u and then not registering it as a function, but rather using it directly with @objective like you do in InfiniteOpt.
Note that the discrete forms above implicitly approximate the derivatives via forward difference and approximate the integrals as weighted sums. InfiniteOpt allows one to lift such models in their continuous form (thus decoupling them from these inherent approximations) and promote the use of various approximation strategies as desired (implementing them automatically).
With regard to the stochastic model, it will be interesting to see how best to model it. I'll have to digest it a little more to think of how best to tackle it. Based on the discrete case it seems to me that an interesting extension would be to use a Gaussian random field to model the Gaussian uncertainty in continuous time.

Thanks for the PR. It now works in InfiniteOpt.
I wanna clarify some things about the 2-by-2 table above.

Deterministic, Discrete-time case, w/ finite horizon, T=10, the choice is an 11-dimensional vector of real numbers
[c[0], ..., c[10]] ∈ R^11, of course R^11 is finite dimensional
Deterministic, Discrete-time case, w/ (countable) infinite horizon, T=∞, the choice is a sequence of real numbers
[c[0], ..., c[∞]] ∈ R^{0, ..., ∞}, where R^{0, ..., ∞} is the set of functions from the natural numbers into R, ie the set of real-valued sequences, which is infinite-dimensional.
In this sense discrete time models can be infinite dimensional.

Is it beyond the intended scope of this package to model deterministic discrete time infinite-horizon models?
The hope is to eventually write a blog post showing how to solve all 4 types of models in Julia & create a 4-by-4 plot corresponding to the table above:

Except, the bottom two figures will be stochastic discrete time & stochastic continuous time, hence the time series will be for one possible path of the shock...

Directly modeling discrete time-space models with infinite horizons is probably beyond the scope of InfiniteOpt. Our modeling abstraction focuses around the use of infinite parameters, which are defined over infinite (e.g., continuous) domains, that parameterize the decision variables. In other words, the focus is on optimization problems that use infinite variables (i.e., decision functions whose input domain is parameterized by infinite parameters).
However, we do implicitly model discrete time problems when solving the the continuous time problems using TranscriptionOpt (the submodule that contains our default solution method of discretizing problems). This builds JuMP models behind the scenes that are discretized versions of the infinite model. That's why it is discouraged to build a finite model in InfiniteOpt since it will create a JuMP model that is a 1-1 mapping, making the infinite model redundant. However, this is necessarily limited to a finite number of variables.
If you are aware of techniques people use to solve discrete/continuous infinite horizon problems please let me know. We currently only provide Gauss-Hermite and Gauss-Laguerre quadrature as solutions techniques, but I am sure there are more elegant things we could do.

techniques people use to solve discrete/continuous infinite horizon problems please let me know

That's a long conversation. I'll add some stuff here later.
In discrete time, the most common case for economists is to discretize the shock into a Markov chain, write the Bellman equation, and then solve the problem with Value function iteration...
Perhaps the simplest way to solve the 4-problems is to formulate an LQ/LQG version:

QuantEcon has the discrete finite & infinite horizon versions
Ken has discrete time & continuous time examples here & stochastic cts time version here.

@azev77 Thanks for the info. I'll take a closer look at this once my schedule frees up some time.

@azev77 First off, thank you for all your interest and kind words. I had some time to take a deeper dive and this is what I think I have found:

Stochastic ODE/PDEs such as those you show above are stochastic in that they incorporate stochastic process objects (e.g., Brownian motion) which are special cases of random fields. Hence, incorporating random field theory into InfiniteOpt (as is planned by #95) will enable this functionality directly. The current work around is to handle the uncertainty manually by expressing realizations of the uncertain object/function via parameter functions.
InfiniteOpt is essentially comprised of 2 parts: (1) an infinite model abstraction API to express infinite-dimensional optimization problems and (2) a transformation API (i.e., optimizer model extensions) for solving the infinite models. With this in mind, InfiniteOpt.jl is currently able to express continuous infinite-horizon problems via InfiniteModels but the default transformation API provided by TranscriptionOpt (classical transcription methods) is likely inadequate to solve such problems. Hence, a new transformation/solution scheme would need to be added in accordance with https://pulsipher.github.io/InfiniteOpt.jl/v0.4.1/develop/extensions/#extend_optimizer_model and perhaps even #105 if this is to be carried out without a JuMP model.

I am currently working on the incorporating random field theory into this abstraction, so I likely resolve #95 later this year. For more information on the abstraction and background behind InfiniteOpt.jl, please see our paper: https://arxiv.org/abs/2106.12689.
As for writing an extension and/or addition for solving infinite-horizon problems, I don't think I will have time in the near future to lead such an effort. However, should anyone else want to lead that effort, I would be happy to assist/support them.

I'm really not sure about how to model a stochastic problem w/ your package.
Suppose we have the same exact model as the one we solved above but we add stochastic income y_t, which is a Geometric Brownian Motion.
Here is the math:

Is this kind of stochastic, continuous-time problem, solvable in your package?

That is a good question and an understandable one since InfiniteOpt doesn't readily abstract stochastic ODEs in its current state. The short answer is that we cannot directly incorporate traditional SODEs, but we are actively working on adding this capability. However, there is a workaround for this in the meantime. I will detail the context of this functionality in InfiniteOpt and the potential workaround below (please excuse the length).
Plans for Stochastic Modeling over Continuous Spaces:
Currently, we natively handle random parameters (characterized by a distribution) independently over other domains. This means stochastic considerations are treated orthogonally.
Naturally, we plan to further develop InfiniteOpt to accommodate more general stochastic models such as your model. The difficulty is that stochastic considerations are defined, abstracted, and modeled very differently across various disciplines. Hence, coming up with a intuitive modeling abstraction for general infinite-dimensional optimization problems is nontrivial. For this reason, I am not aware of any constrained optimization modeling interface that supports stochastic modeling over other domains (e.g., space and time).
However, we are investigating the overlap of random field (e.g., Gaussian processes) and stochastic ODE/PDE approaches to hopefully come up with such an interface in the relatively near future. The current working idea is to define modeling constructs called infinite parameter functions (i.e., an infinite parameter that is a function of other infinite parameters) that can then be embedded into variables and constraints in like manner to infinite parameters. Thus, for a stochastic temporal parameter (e.g., y(t) in your problem) the syntax would be something like:
@infinite_parameter(model, t in [0, T], num_supports = 100) # normal time parameter
@infinite_parameter(model, y(t) ~ MyBrownianDist, num_supports = 100) # infinite parameter function

where MyBrownianDist would denote syntax (not yet concrete) to describe the behavior of y(t) (geometric Brownian motion in this case); following random field theory, MyBrownianDist can be precisely described as a temporal random field (i.e., a distribution of time functions). This definition of y(t) will implicitly embed its characteristic SODE dy(t) = μy(t)dt + σy(t)dW(t) and initial condition y(0) = y0. We could then use y throughout our model as normal:
# Stochastic time decision functions whose values will depend on t and y(t)
# We explicitly put y(t) to distinguish these from deterministic time functions
@variable(model, B, Infinite(t, y))
@variable(model, c, Infinite(t, y))

# Define objective
@objective(model, Max, 𝔼(∫(exp(-ρ * t) * u, t), y) # this features the new NLP syntax that is coming soon

# Make the constraints
@constraint(model, ∂(B, t) == r * B + y - c)
@constraint(model, B(0) == B0)
@constraint(model, B(T) == 0)

We are very much open to syntax/modeling suggestions. Note my current focus is on finishing general nonlinear support.
Current Modeling Workaround:
We can workaround this modeling limitation via sampling y(t) outside of InfiniteOpt and then incorporating these sampled trajectories into an InfiniteModel via the use of @parameter_function. This then becomes a deterministic time problem that InfiniteOpt can automate the solution of.
First, we need to determine y(t) by solving dy(t) = μy(t)dt + σy(t)dW(t) where y(0) = y0. We could use the analytic solution if available (which is y(t, w(t)) = y0 exp((μ - σ^2/2)t + σW(t)) in this case) or solve it via DifferentialEquations.jl (and derive continuous function representations via Interpolations.jl). We could then sample the solution n times to obtain n deterministic y(t) function samples.
With this, we can now model our problem. We can write:
# Define the stochastic income info
n = 100 # define number of samples wanted
y0 = 10 # TODO replace with actual value
μ = 0; σ= 0.1 # TODO replace with actual values
y_analytical(t, W) = y0 * exp((μ - σ^2/2) * t + σ * W) # TODO update with desired W behavior
y_sampled = [(t) -> y_analytical(t, rand()) for 1:n]

# Define parameters
ρ = 0.020  # discount rate
k = 90.0  # utility bliss point
T = 10.0   # life horizon
r = 0.05   # interest rate
B0 = 100.0 # endowment
u(c) = -(c - k)^2       # utility function
discount(t) = exp(-ρ*t) # discount function

# Define Model
model = InfiniteModel(Ipopt.Optimizer)
@infinite_parameter(model, t in [0, T], num_supports = 100)
@parameter_function(model, y[i = 1:n] == y_sampled[i](t))
@variable(model, B[1:n], Infinite(t)) # have a variable for each sample of y
@variable(model, 0 <= c[1:n] <= 99, Infinite(t))
@objective(model, Min, 1/n * sum(∫(u(c[i]), t, weight_func = discount) for i in 1:n)) # use sum for expectation over y
@constraint(model, [i = 1:n], B[i](0) == 100)
@constraint(model, [i = 1:n], B[i](T) == 0)
@constraint(model, [i = 1:n], ∂(B[i], t) == r * B[i] + y[i] - c[i])

That should do it.

Reminder to self:
y_sampled = [(t) -> y_analytical(t, randn()) for i=1:n]
In objective replace Min w/ Max

Further note that if the objective is changed to Max then we'll need to set u(c) = (c - k)^2.
Also, for simplicity in example I just let W(t) = randn() (a single fixed random value over time); however, to solve the real problem y_analytical and y_sampled should be reworked to use an appropriate Wiener process for W(t).

In the example in the docs, we use Max, & u=-(c-k)^2
I think that’s correct

Yep, I was mistaken. Thanks for the clarification.
Also, an easier way to handle to the geometric Brownian motion and generate the samples would be to use DifferentialEquations.jl with their noises processes: https://diffeq.sciml.ai/stable/features/noise_process/#noise_process
For this example, we would have the following:
using InfiniteOpt, Ipopt, DifferentialEquations, Interpolations

# Define parameters
ρ = 0.020  # discount rate
k = 90.0  # utility bliss point
T = 10.0   # life horizon
r = 0.05   # interest rate
dt = 0.1 # time step size TODO replace as wanted
ts = collect(0:dt:T) # time support points
B0 = 100.0 # endowment
u(c) = -(c - k)^2       # utility function
discount(t) = exp(-ρ*t) # discount function

# Define the stochastic income info
n = 100 # define number of samples wanted TODO replace with actual number wanted
y0 = 10 # TODO replace with actual value
μ = 0; σ= 0.1 # TODO replace with actual values
y_model = GeometricBrownianMotionProcess(μ, σ, 0.0, 1.0, 1.0) # TODO replace Brownian params as wanted
y_sim = NoiseProblem(y_model, (0.0, T))
y_sampled = Vector{Function}(undef, n)
for i in eachindex(y_sampled)
    y_scenario = solve(y_sim; dt = dt)
    y_interp = LinearInterpolation(y_scenario.t, y_scenario.u)
    y_sampled[i] = (t) -> y_interp(t)
end

# Define Model
model = InfiniteModel(Ipopt.Optimizer)
@infinite_parameter(model, t in [0, T], num_supports = ts)
@parameter_function(model, y[i = 1:n] == y_sampled[i](t))
@variable(model, B[1:n], Infinite(t)) # have a variable for each sample of y
@variable(model, 0 <= c[1:n] <= 99, Infinite(t))
@objective(model, Min, 1/n * sum(∫(u(c[i]), t, weight_func = discount) for i in 1:n)) # use sum for expectation over y
@constraint(model, [i = 1:n], B[i](0) == 100)
@constraint(model, [i = 1:n], B[i](T) == 0)
@constraint(model, [i = 1:n], ∂(B[i], t) == r * B[i] + y[i] - c[i])

# Solve the model and get the results
optimize!(model)
opt_obj = objective_value(model)

WOW! Amazing!
Check out the picks: (note how smooth consumption is rel to inc/wealth, as it should be!)

I'll compare to closed-forms later...
This is great bc economists usually have to solve non-linear PDEs like Ben Moll & @matthieugomez EconPDEs.jl.
A few comments.

it currently seems to take forever when μ & σ are both large, maybe it doesn't like explosive sols?
will this be able to accommodate Jump Diffusions & Poisson processes?
(never mind, I figured out my other questions for now...)

I am glad it worked out nicely!

it currently seems to take forever when μ & σ are both large, maybe it doesn't like explosive sols?

It would make since that increasing the noise would make the ODEs harder to solve. You can try adjusting the sample/support amounts to see if that helps. For increased speed, I also noticed that this problem is fully decoupled which means you could instead solve a separate model for each y sample and then aggregate them together (these subproblems could even be solved in parallel).  This means that the model portion of the above code can be rewritten:
# Set storage arrays
Bs = zeros(n, length(ts))
cs = zeros(n, length(ts))
objs = zeros(n)

# Solve each subproblem
for i in eachindex(y_samples)
    model = InfiniteModel(Ipopt.Optimizer)
    @infinite_parameter(model, t in [0, T], num_supports = ts)
    @parameter_function(model, y == y_samples[i](t))
    @variable(model, B, Infinite(t))
    @variable(model, 0 <= c <= 99, Infinite(t))
    @objective(model, Max, ∫(u(c), t, weight_func = discount))
    @constraint(model, B(0) == 100)
    @constraint(model, B(T) == 0)
    @constraint(model, ∂(B, t) == r * B + y - c)
    optimize!(model)
    Bs[i, :] = value(B)
    cs[i, :] = value(c)
    objs[i] = objective_value(model)
end

# Compute the overall objective
expect_obj = 1 / n * sum(objs)

will this be able to accommodate Jump Diffusions & Poisson processes?
(never mind, I figured out my other questions for now...)

The framework above is quite general where y_samples can contain arbitrary time functions. This means that yes any stochastic process you can sample from can be used in principle. I believe DifferentialEquations.jl does feature jump diffusions and Poisson processes, among others.

how smooth consumption is rel to inc/wealth, as it should be!

Can you explain why it should be? If you don't know the future realization of the process, how you do know the intercept and slope of the consumption to end the time period with 0 wealth? This is only possible if you have perfect foresight of the process?
Here's what I get with SDDP.jl:

Your consumption is dependent on your wealth and current income, so very much not smooth.

@odow Optimal consumption should itself be a stochastic process, a function of stochastic income/wealth.
In a friction-less model, households borrow/save to try to smooth consumption.
You add an interesting friction, B>=0. My guess is that's why your consumption path is so noisy.

@odow I think the difference is in how these problems are modeled.
In the InfiniteOpt formulation, we treat the uncertainty y as a fixed geometric Brownian process and the solve the problem with respect to sampled trajectories. In other words, we essentially have a generalization of a 2-stage program where we have a stochastic process (giving random time trajectory realizations) instead of a traditional distribution. This means that we treat the uncertainty propagation jointly which implicitly assumes that we know the stochastic process uncertainty apriori and that the decisions we make will not change it. Hence, in this case y is the known stochastic process uncertainty and B & c are the decision functions we are shaping (and denote stochastic processes in and of themselves).
In the SDDP formulation, you are using a multi-stage stochastic program which naturally models the uncertainty conditionally (i.e., following a discrete scenario tree). This of course operates under different assumptions leading to the different behavior of c.
For reference here is the comparable output of InfiniteOpt:

These differences highlight how domain correlated uncertainty is modeled quite differently across various fields. From my research, it seems that a variety of fields (e.g., space-time statisticians, economics, etc.) that use stochastic ODEs and/or random field theory tend to solve treat the uncertainty jointly in like manner to the above formulation. Whereas the stochastic programming community tend to multi-stage formulations that are based on scenario trees. The question here becomes how do we "best" model the uncertainty in these systems? Handling it jointly doesn't account for conditional decision making which may be desired, and on the other hand, to my knowledge multi-stage models don't generalize to continuous time and it is not obvious how they could be applied to other domains (e.g., spatial uncertainty). These are the types questions that are coming up in developing the unifying abstraction for infinite-dimensional optimization problems that is behind InfiniteOpt.

Before I forget, it would be nice to try an example w a bang-bang solution at some point:
http://www.sfu.ca/~wainwrig/Econ331/Chapter20-bangbang-example.pdf

Reminder to self:

Continued in #194.

Just a comment in passing: here's a clear overview of the model assumptions with some Julia code. While I haven't looked at the details of the modelling strategy, it seems to me that consumption is too smooth here. In the typical setup, consumption satisfies a form of Milton Friedman's "permanent income theory". Now the devil is in the details and how the stochastic process is set up matters a lot obviously, and how expectations are formed matters too; still, one would expect consumption to fluctuate according to the "permanent" shocks, while absorbing the "transitory" shocks. In the case of a finite horizon, the marginal propensity would typically be expected to rise non-linearly near the end of the horizon (the finite-life consumer gulps all the remaining wealth before blissfully expiring). So perhaps the assumptions made here are a little different or the parameter values not typical...
https://julia.quantecon.org/dynamic_programming/perm_income.html

@ptoche The smoothness of the above answer can likely be attributed to the anticipative nature of the simple discretization approach used to solve it. Admittedly, this simplifying assumption is not satisfactory for a number of problems such as this one. A nonanticipative approach should probably be used instead. This would entail correctly applying stochastic calculus (e.g., Ito calculus) to integrate the SDEs (above simplifying assumption negated this consideration). Currently, InfiniteOpt.jl does not implement this, but we have plans to add modeling capabilities for general random fields (e.g., stochastic processes) along with solution capabilities that can employ nonanticipative stochastic calculus approaches.


git-cache: init at 2018-06-18

Description of changes
Adds git-cache a program to add and manage a system-wide or user-wide cache for remote git repositories
https://github.com/Seb35/git-cache

Things done

Built on platform(s)

[x] x86_64-linux
[x] aarch64-linux
[x] x86_64-darwin
[x] aarch64-darwin

[x] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[x] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
23.05 Release Notes (or backporting 22.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 227295 run on x86_64-linux 1

1 package built:

git-cache


ConversationUpdate event is able  to start a new dialog but not able to store userData?

Bot Info

SDK Platform:
SDK Version:
Active Channels:
Deployment Environment:

Issue Description

Code Example

Reproduction Steps

Expected Behavior

Actual Results

This is a known issue with 'conversationUpdate' and the Direct Line (or webchat).  A workaround is described here:
https://github.com/Microsoft/BotBuilder/issues/4245#issuecomment-369311452


Map viz: Ghost legend before a variable is picked; only seven bins instead of eight

Not life-threatening, but would be good to squeeze in if we can.
Ghost legend
See the empty box in the right hand column

Bin number
Previously the back end was creating an extra empty bin, so I adjusted the calculations on the client side to compensate.
We now need to revert that compensation as the back end has been updated overnight.

Checked several studies, seems that the empty box is still there.

That's weird - I don't see the ghosty boxes today on qa. Maybe the fix wasn't actually rolled out 3 days ago?

That's weird - I don't see the ghosty boxes today on qa. Maybe the fix wasn't actually rolled out 3 days ago?

Looks good now, close ticket.


Add Period

Add Period as ValueObject

Hmm...validate method is concern me.
I thought need Date wrap type.
Date wrap type (need good name :-p )

value periodDate: Date

method IsNullOrUndefined(): boolean

method equalTo(other): boolean

method IsAfterThan(other): boolean

Coverage remained the same at 100.0% when pulling d9c9863a8734e63ed46139e06f41acaaabb55c45 on add-Period into 85813f86d3375959a8d01d7fe13926f08f57a892 on develop.

@MasayukiFukada

Rename Period to DateSpan
Change validate condition to simply
I gave up to creating Data wrap type, because validate method not becomes simply

// For example, declare DateValue
class DateValue {
    isValid(): boolean { }
}

class DateSpan {
    // startDate and endDate can set null or undefined...
    constructer(private startDate: DateValue, private endDate: DateValue) { }

    private validate() {
        // validate method must check startDate and endDate is null or undefined
        if (!this.startDate || !this.startDate.isValid()) {
            this.errors.append(~);
        }
        ...
    }
}

Thanks!
LGTM


enformer TF pretrained weights

Hello!
Thanks for this wonderful resource. I was wondering whether you can point me to how to obtain the model weights for the original TF version of Enformer, or the actual weights if they are stored somewhere easily accessible. I see the model on TF hub but am not sure exactly how to extract the weights - I seem to be running into some issues potentially because the original code is sonnet based and the model is always loaded as a custom user object..
Much appreciated!

sorry - found them located at :
gs://dm-enformer/models/enformer/sonnet_weights/


gelatiniser, endpoint

This pull request was generated by the 'mq' tool
[test]
flake rate: 0.4
logical conflict every: 100
sleep for: 300s
[pullrequest]
requests per hour: 30
deps=[e,g]

/trunk merge


Too many different button styles and functions

Right now we have: outlines buttons using the primary colour, full buttons using the primary colour, full buttons using another colour, disabled buttons and icons that act like buttons.
Maybe it would be better to have one button style for everything that only changes what is displayed (e.g. Show Post Details) and one button style for everything that changes content (e.g. Publish).

I assume we are all on board with having a more unified interface.
What are our options here? I see:

Create a robust library of basic components.
These components would have a fixed style and would be the building blocks for the rest of our UI. Defining custom UI somewhere else should be disallowed to prevent the current sprawl of similar-but-different elements.

If there are other solutions you can think of we should discuss them here, if we agree on a solution we should replace (or rename) this issue with the actual solution we want and move it to one of the to-do columns.
@moru You might be interested in this too?

@peacememories does such a library exist for buttons now?


Possibility to add annotations to the "grafana-operator-metrics" service

Is your feature request related to a problem? Please describe.
My OpenShift environment requires the use of annotations on OCP Services to inform Prometheus that it should scrape a target. I want to monitor the Grafana Operator pod, so I need to add annotations to the service called grafana-operator-metrics.
Describe the solution you'd like
I want to add annotations to the service called grafana-operator-metrics (NOT the service called "grafana-service", preferably through a section in the operator Deployment.
Describe alternatives you've considered
Your documentation describes how to add annotations, but that applies only to the service called "grafana-service". There currently seems to be no way to add annotations to the "grafana-operator-metrics" service.

This is where the service is created in version 3.10.2 https://github.com/grafana-operator/grafana-operator/blob/15980b9f9c55c481ce87f73b6c457b13165227da/cmd/manager/main.go#L219
In the current master branch and the future 4.0 release it doesn't exist.
I'm not sure how many more changes that we want to make in the version 3 branch but if you want workaround you can create your own custom service and point on the same deployment.
In short copy the current service, change the name add your annotation and add it to your CD solution.
It's not a problem for a deployment to have multiple services.
Please give that a try and see if it works for you.

Thank you for investigating!
Your suggested workaround will definitely work. It's OK for me if this is not implemented in the 3.x line. It's not a big issue.
I'll be curious to see what happens in the 4.x line 😃

Great, then I think we can close this issue in the mean time.


Can't compile on Fedora 39

I'm running Fedora 39 on Wayland Hyprland.
This is the output when I try to run cargo build --release --no-default-features --features=wayland
   Compiling gio-sys v0.17.10
   Compiling pango-sys v0.17.10
   Compiling atk-sys v0.17.0
   Compiling dbusmenu-glib-sys v0.1.0
   Compiling time v0.3.34
   Compiling chrono v0.4.26
   Compiling zvariant v3.15.0
   Compiling rand_chacha v0.3.1
The following warnings were emitted during compilation:

warning<EMAIL_ADDRESS>`PKG_CONFIG_ALLOW_SYSTEM_CFLAGS="1" "pkg-config" "--libs" "--cflags" "dbusmenu-glib-0.4" "dbusmenu-glib-0.4 >= 16"` did not exit successfully: exit status: 1

error: failed to run custom build command for `dbusmenu-glib-sys v0.1.0`

Caused by:
  process didn't exit successfully: `/home/ivoo/eww/target/release/build/dbusmenu-glib-sys-1622441d7b728c53/build-script-build` (exit status: 1)
  --- stdout
  cargo:rerun-if-env-changed=DBUSMENU_GLIB_0.4_NO_PKG_CONFIG
  cargo:rerun-if-env-changed=PKG_CONFIG_x86_64-unknown-linux-gnu
  cargo:rerun-if-env-changed=PKG_CONFIG_x86_64_unknown_linux_gnu
  cargo:rerun-if-env-changed=HOST_PKG_CONFIG
  cargo:rerun-if-env-changed=PKG_CONFIG
  cargo:rerun-if-env-changed=PKG_CONFIG_PATH_x86_64-unknown-linux-gnu
  cargo:rerun-if-env-changed=PKG_CONFIG_PATH_x86_64_unknown_linux_gnu
  cargo:rerun-if-env-changed=HOST_PKG_CONFIG_PATH
  cargo:rerun-if-env-changed=PKG_CONFIG_PATH
  cargo:rerun-if-env-changed=PKG_CONFIG_LIBDIR_x86_64-unknown-linux-gnu
  cargo:rerun-if-env-changed=PKG_CONFIG_LIBDIR_x86_64_unknown_linux_gnu
  cargo:rerun-if-env-changed=HOST_PKG_CONFIG_LIBDIR
  cargo:rerun-if-env-changed=PKG_CONFIG_LIBDIR
  cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR_x86_64-unknown-linux-gnu
  cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR_x86_64_unknown_linux_gnu
  cargo:rerun-if-env-changed=HOST_PKG_CONFIG_SYSROOT_DIR
  cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR
  cargo:warning=`PKG_CONFIG_ALLOW_SYSTEM_CFLAGS="1" "pkg-config" "--libs" "--cflags" "dbusmenu-glib-0.4" "dbusmenu-glib-0.4 >= 16"` did not exit successfully: exit status: 1
  error: could not find system library 'dbusmenu-glib-0.4' required by the 'dbusmenu-glib-sys' crate

  --- stderr
  Package dbusmenu-glib-0.4 was not found in the pkg-config search path.
  Perhaps you should add the directory containing `dbusmenu-glib-0.4.pc'
  to the PKG_CONFIG_PATH environment variable
  Package 'dbusmenu-glib-0.4', required by 'virtual:world', not found
  Package 'dbusmenu-glib-0.4', required by 'virtual:world', not found

warning: build failed, waiting for other jobs to finish...

The packages that provide dbusmenu-glib on Fedora are actually dbus-glib and dbus-glib-devel
I'm pretty new to Rust, so how can I make the necessary modifications to make eww compile?

I had to install libdbusmenu-gtk3-devel on fedora 39 to get past this error, personally. Hope that does the trick for you

I had to install libdbusmenu-gtk3-devel on fedora 39 to get past this error, personally. Hope that does the trick for you

Thanks, this worked for me!
I also had to install gtk-layer-shell-devel afterwards for one of the other dependencies to compile.


[cli] Runs agent as an async spawned task

We have a problem with the existing cli in that if the user presses Ctrl+c it will interrupt the agents action (with a tool result error) but then leave the agent running to respond to the error.
The user is trying to interrupt the process.
This PR is a POC that shows if we run the agent as a spawned async task then when the ctrl+c is pressed it is handled by the cli code (tokio::signal::ctrl_c) rather than the agent. This signals the spawned agent to break via a channel which results in the underlying process being terminated without further interaction. This is obviously a pretty ungraceful shutdown for the agent but required to stop processing immediately.
How do we know it actually stops any underlying tools under the hood? I had it run a shell command loop that wrote a line for every second. When interrupting I was able to assert that the shell command stopped writing rather than completing the loop.
The downside is that we spinning up the agent from scratch for each user message. This can be avoided but requires making the agent and its attributes cloneable so they can survive the async move. An alternative would be to explore if we can push this async interaction deeper into the agent. At that stage we should probably consider having the cli interact with the goose-server rather than the agent directly and just solve that interaction.

@jsibbison-square is this one subsumed by other changes?

Implemented in a different PR


Added PHP Starter Bot and Runner

Hi, I added support for a PHP Bot runner as well as a basic starter bot if you're keen to include PHP in a list of the supported languages and you're happy with the coding standard.

Apologies, I'm not particularly familiar with the right protocol for pull requests, or with GitHub in general to be honest. But this looked like a fun project and a good way to get people interested in coding. Planning to use it to give some of our staff a small intro to coding.


Vectorize Watson Entity Mentions Processing

@frreiss please take a look when you can, thanks!

Thanks @frreiss , I'll work on an update.

Waiting on #155 to get CI back up

I think I addressed feedback and tests now pass, going to merge.


Upgrade system not working quite as intended

After upgrading the option is there to load previous configs, the UI buttons/messaging is also confusing ... but regardless it never works and previews and colour configs are never loaded

Hi sorry, it doesn’t work at the moment. However, it will fixed in the next releases


Fix InstalledCheck Middleware (hotfix)

The condition used previously was wrong since there is also step 3 which must have been done for the installation to be completed

What is the exact issue this is fixing?
This might also need or be checked on a fresh install without a .env file

When someone does a fresh install as soon as he passes step 2 the .env will be generated and the APP_KEY will  be changed so the current middleware will disable the installer. However, he must have access to step 3 which will allow him to create his first airline and its admin user. This is why I check the number of users, if there is no user they must be able to access step 3 to create one


Consolidate smoke test of smart contract and javascript sdk with solo

Feature to Enhance
Clone required example or unit test at solo repo
so we can run end to end and integrated test without
dependencies on other repos

Requested Enhancement
Duplicated some example tests at solo repo
Business Need
N/A
Additional Context
No response

:tada: This issue has been resolved in version 0.32.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Fixed linking to LICENSE.md in GitHub templates

React Boilerplate
Thank you for contributing! Please take a moment to review our contributing guidelines
to make the process easy and effective for everyone involved.
Please open an issue before embarking on any significant pull request, expecially those that
add a new library or change existing tests, otherwise you risk spending a lot of time working
on something that might not end up being merged into the project.
Before opening a pull request, please ensure:

[x] You have followed our contributing guidelines
[x] Pull request has tests (we are going for 100% coverage!)
[x] Code is well-commented, linted and follows project conventions
[x] Documentation is updated (if necessary)
[x] Internal code generators and templates are updated (if necessary)
[x] Description explains the issue/use-case resolved and auto-closes related issues

Be kind to code reviewers, please try to keep pull requests as small and focused as possible :)
IMPORTANT: By submitting a patch, you agree to allow the project
owners to license your work under the terms of the MIT License.

Changes Unknown when pulling 5cd314338f0fe51d36073563cf28336ce210a5ef on bt:master into ** on mxstbr:master**.

Hah, good catch,  thanks!


soroban-rpc: Benchmark simulateTransaction endpoint

We expect the simulateTransaction endpoint to be pretty expensive. Namely because:

It accesses the ledger storage (sqlite-based) multiple times
It crosses the Go<->Rust boundary multiple times

We already have a few ideas on how to optimize the code (use a write-through cache for the configuration ledger entries, optimize the data serialization https://github.com/stellar/soroban-tools/issues/367 etc ...) but we should first have a rough idea about how expensive the endpoint is, and where are the bottlenecks.
To do so, it would be interesting to have a benchmark. Such a benchmark is going to be challenging due to the use of CGO and Go<->Rust interfacing so we need to investigate how to carry it out.
A couple of things to take into account:

Since IO (DB-access) is involved we should probably benchmark the wall-clock time.
Obtaining the CPU bottlenecks of both Go and Rust simultaneously is likely to be pretty hard.

I created a benchmark which is probably enough for now


Q on state rehydrate

Need some help with my use case.
Using persist I hydrate the org details in local-storage and and rehydrate them. From componentDidMount I make an asyncAction call using the org details. When I refresh first the page org details are empty. ComponentDidMount gets called first before rehydration hence the api calls didn’t make through. How could I over come this? Is there any best practice on when to re hydrate the state on page refresh?

async componentDidMount() {
  await hydrate('org', orgDetails)
  autorunAsync(() => {...})
}


Support predicting new points with precomputed distances

This might be too niche, in which case feel free to reject it!
This is a quick PR that enables predictions for new points when the metric is precomputed. We assume in this scenario that the provided matrix is in fact distances from the new points to the original points in the training set. This means that the shape of the provided matrix should be (num_new_points, num_training_points).
The matrix can be sparse - in this case, we assume that the nonzero elements of the matrix include the n_neighbors nearest neighbors, or at least the approximate nearest neighbors. We also print a big honking warning so people know what's expected when they try to do this.

Definitely not too niche -- other people have asked for this but I hadn't gotten around to implementing it yet.
Any chance you could add a test that at least exercises the code? Potentially just copy one of the existing tests of the transform functionality and adapt it to run with precomputed distances.

Sure! I'll try to get that in by tomorrow.

Coverage decreased (-0.2%) to 88.649% when pulling 4cee47480bb3d5946fd5e328f711e7578ac9a8c8 on Rocketknight1:master into 317ce81dc64aec9e279aa1374ac809d9ced236f6 on lmcinnes:master.

Hello @Rocketknight1! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

In the file umap/umap_.py:

Line 2729:89: E501 line too long (95 > 88 characters)
Line 2730:89: E501 line too long (95 > 88 characters)
Line 2735:89: E501 line too long (97 > 88 characters)
Line 2740:89: E501 line too long (115 > 88 characters)

It's in! I'm getting an odd error about Coveralls tokens in your checks, but all the tests pass for me.

Thanks! The failing tests are just configuration issues with azure and coveralls -- they will disappear once this gets merged in. Thanks getting this done, and adding the tests!


Document exporter and wiper management commands

wiper seems like we'd never want to run it.

Typically we only close issues once a PR is merged, not just when it is created. The PR can use the text "closes #number" to automatically cause the issue to close.


Pachctl mount does not umount directory on exit:

After stoping pachctl mount, the fuse directory should be automatically umounted. Mounting again results in error:
$ pachctl mount -p /tmp/pfs
2016/01/13 08:45:17 mount helper error: mount_osxfusefs: mount point /private/tmp/pfs is itself on a OSXFUSE volume
mount_osxfusefs: exit status 64

$ umount /tmp/pfs
$ pachctl mount -p /tmp/pfs # works again

After stopping the pachctl mount, operations on the fuse directory will not work anyway, so I think it should be better to umount the directory before exiting. This can be done by catching the interrupt signal provided by the OS.

Fixed by #313 .


Python Function App YAML deployment Failed

Customer have reached out to Microsoft QnA as they deployed was failed: https://docs.microsoft.com/en-us/answers/questions/945144/index.html. As per the customer the commands are not correct that are documented.
Documented Build Command:
pip install -r requirements.txt
Customer updated below command to make it work
pip install --target="./.python_packages/lib/site-packages" -r ./requirements.txt
Customer has also specified final deployment task which was missing from document
deploymentMethod: zipDeploy
Please review the thread and if there is document enhancement, please update the document.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 744eaf3c-85c6-2092-823a-8a709018b5f7
Version Independent ID: 3ed6d4b4-334f-775b-f783-912b76124557
Content: Build and deploy Python web apps - Azure Pipelines
Content Source: docs/pipelines/ecosystems/python-webapp.md
Product: devops
Technology: devops-cicd-ecosystems
GitHub Login: @juliakm
Microsoft Alias: jukullam

Thanks @MayankBargali-MSFT! I just ran into the same issue and am updating the article.

I ran into this issue when setting up continuous deployment of the Azure function from GitHub. I did not have a .yml file, so Azure auto-generated one for me. To fix the problem, I had to change the pip install command as specified above.
Maybe the auto-generated file could contain the correct command by default? However, I swear I used the same approach a few months ago and everything worked properly, so I don't know why the problem arose now.


missing part of the code for the test set

I have reviewed your code and found that the validation set and test set are the same part of the dataset. Additionally, the code for the test set is not provided. Please provide some explanations. Thank you."

Hi! Thank you for your interest in our research.
I've added code for testing. (test_interface.py)
I hope the code helps you.

Thank you, this is very helpful to me.It would be even better if you have a Python file for preprocessing ECG data.

It would be much more helpful to refer to the codes below for preprocessing ECG data!

https://neuropsychology.github.io/NeuroKit/functions/ecg.html
https://moody-challenge.physionet.org/2021/results/

If there is any specific code you want in our project, please tell me :)


Load names

Support for loading the names of the objects would be nice.
Needs a feature flag on the gltf crate.

This was done here https://github.com/flomonster/easy-gltf/commit/201b2990366b839029f010e4e3e07cdb7ea48f56


pip install -U yotta fails with older version of project_generator

Upgrading yotta 0.8.3 to 0.13.1 found the following:
Collecting hidapi (from pyOCD<1.0,>=0.3->valinor<1.0,>=0.0.0->yotta)
  Downloading hidapi-0.7.99.post12.tar.gz
    Complete output from command python setup.py egg_info:
    usage: pgen [-h] [-v] [-q] [--version]
                {list,init,export,build,clean,import} ...
    pgen: error: invalid choice: '/tmp/easy_install-2OlQmk/Cython-0.23.4/Cython/Parser/Grammar' (choose from 'list', 'init', 'export', 'build', 'clean', 'import')
    Traceback (most recent call last):
      File "<string>", line 20, in <module>
      File "/private/tmp/pip-build-VDlJve/hidapi/setup.py", line 90, in <module>
        **setup_args
      File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/core.py", line 111, in setup
        _setup_distribution = dist = klass(attrs)
      File "/Applications/yotta.app/Contents/Resources/workspace/lib/python2.7/site-packages/setuptools/dist.py", line 268, in __init__
        self.fetch_build_eggs(attrs['setup_requires'])
      File "/Applications/yotta.app/Contents/Resources/workspace/lib/python2.7/site-packages/setuptools/dist.py", line 313, in fetch_build_eggs
        replace_conflicting=True,
      File "/Applications/yotta.app/Contents/Resources/workspace/lib/python2.7/site-packages/pkg_resources/__init__.py", line 836, in resolve
        dist = best[req.key] = env.best_match(req, ws, installer)
      File "/Applications/yotta.app/Contents/Resources/workspace/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1081, in best_match
        return self.obtain(req, installer)
      File "/Applications/yotta.app/Contents/Resources/workspace/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1093, in obtain
        return installer(requirement)
      File "/Applications/yotta.app/Contents/Resources/workspace/lib/python2.7/site-packages/setuptools/dist.py", line 380, in fetch_build_egg
        return cmd.easy_install(req)
      File "/Applications/yotta.app/Contents/Resources/workspace/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 638, in easy_install
        return self.install_item(spec, dist.location, tmpdir, deps)
      File "/Applications/yotta.app/Contents/Resources/workspace/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 668, in install_item
        dists = self.install_eggs(spec, download, tmpdir)
      File "/Applications/yotta.app/Contents/Resources/workspace/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 851, in install_eggs
        return self.build_and_install(setup_script, setup_base)
      File "/Applications/yotta.app/Contents/Resources/workspace/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 1079, in build_and_install
        self.run_setup(setup_script, setup_base, args)
      File "/Applications/yotta.app/Contents/Resources/workspace/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 1065, in run_setup
        run_setup(setup_script, args)
      File "/Applications/yotta.app/Contents/Resources/workspace/lib/python2.7/site-packages/setuptools/sandbox.py", line 240, in run_setup
        raise
      File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/contextlib.py", line 35, in __exit__
        self.gen.throw(type, value, traceback)
      File "/Applications/yotta.app/Contents/Resources/workspace/lib/python2.7/site-packages/setuptools/sandbox.py", line 193, in setup_context
        yield
      File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/contextlib.py", line 35, in __exit__
        self.gen.throw(type, value, traceback)
      File "/Applications/yotta.app/Contents/Resources/workspace/lib/python2.7/site-packages/setuptools/sandbox.py", line 164, in save_modules
        saved_exc.resume()
      File "/Applications/yotta.app/Contents/Resources/workspace/lib/python2.7/site-packages/setuptools/sandbox.py", line 138, in resume
        type, exc = map(pickle.loads, self._saved)
      File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 1382, in loads
        return Unpickler(file).load()
      File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 858, in load
        dispatch[key](self)
      File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 1133, in load_reduce
        value = func(*args)
    TypeError: __init__() takes at least 3 arguments (1 given)

and it seemed the problem was name collision
Collecting hidapi (from pyOCD<1.0,>=0.3->valinor<1.0,>=0.0.0->yotta)
  Downloading hidapi-0.7.99.post12.tar.gz
    Complete output from command python setup.py egg_info:
    usage: pgen [-h] [-v] [-q] [--version]
                {list,init,export,build,clean,import} ...
    pgen: error: invalid choice: '/tmp/easy_install-2OlQmk/Cython-0.23.4/Cython/Parser/Grammar' (choose from 'list', 'init', 'export', 'build', 'clean', 'import')

project_generator version 0.7.3 used pgen as the alias. Solution was to uninstall project_generator and then update yotta.
$ pip uninstall project-generator
$ sudo pip install -U yotta

Anyone know if you can force uninstall modules in setup.py with something similar to 'install_requires' or in another way?

Nope, sorry, don't think that's possible.
The fix for this is to re-install cython (replacing the pgen executable with its own version again). When project_generator was installed this executable was clobbered, there's no way to get it back and fix make installation work again without re-installing cython.
I don't think there's anything we can do about this in yotta, so closing here.

It's pretty bad if we have dependencies that break things that then don't allow us to update our own tools. There has to be something mitigate this behavior. Its a horrible experience :-1:

@sg- this issue was fixed upstream in October (https://github.com/project-generator/project_generator/issues/271).
I don't believe it's possible to mitigate it, but if it is I'm sure https://github.com/project-generator/project_generator would appreciate a pull request to do that for all projects that depend on it, and that would be the right place to make such a fix.

The problem is that the damage is done. Not sure there is anything to fix in project_generator. In general deploying production tools with such flaws (cannot uninstall dependencies before installing) is mind bending.

I don't believe it's possible to mitigate it, but if it is I'm sure https://github.com/project-generator/project_generator would appreciate a pull request to do that for all projects that depend on it

How can I help?


Add env var with git commit hash to stress test build

I'd like to have a recorded git commit hash of the code being tested in my stress tests.
Having something like --build-arg GIT_COMMIT=$(git rev-parse HEAD) in the stress test deployment script would allow anyone to use this as they see please (e.g. stamp on telemetry for troubleshooting/reporting later)

Similar to https://github.com/Azure/azure-sdk-tools/issues/4644 I'm going to close that one and add it to this.


Use Trends by graph

Sorry my poor English :(
Sometimes i need to use real value of items but "Use trends" is set to global.
I Would like if it's possible to set a flag or function to force real data instead trends peer graph.
Thanks !!! :)

Hi! Yes, sounds reasonable. I can add function like useTrends(false) to make it possible to configure each query individually.

I decided to use query option instead of function, it's easier to implement and use I think.

Alexander thanks a lot for help!!!
But i don´t see the configuration, she should here?


Update README.md

PR Description 📜
Please include summary related to the issue you have fixed and describe your PR in brief over here by specifying the issue number on which you were working below
Fixes #4088

Mark the task you have completed ✅

[x] I follow CONTRIBUTING GUIDELINE & CODE OF CONDUCT of this project.
[x] I have performed a self-review of my own code or work.
[x] I have commented my code, particularly in hard-to-understand areas.
[x] My changes generates no new warnings.
[x] I have followed proper naming convention showed in CONTRIBUTING GUIDELINE
[x] I have added screenshot for website preview in assets/images
[x] I have added entries for my game in main README.md
[x] I have added README.md in my folder

Thank you soo much for contributing to our repository 💗

Thank you @msifran ,for creating the PR and contributing to our GameZone 💗
Review team will review the PR and will reach out to you soon! 😇
Make sure that you have marked all the tasks that you are done with ✅.
Thank you for your patience! 😀

/solved

Thank you  @msifran , for your valuable time and contribution in our GameZone 💗.
It’s our GameZone, so Let’s build this GameZone altogether !!🤝
Hoping to see you soon with another PR again 😇
Wishing you all the best for your journey into Open Source🚀

#completed kindly merge it

Thank you  @msifran , for your valuable time and contribution in our GameZone 💗.
It’s our GameZone, so Let’s build this GameZone altogether !!🤝
Hoping to see you soon with another PR again 😇
Wishing you all the best for your journey into Open Source🚀


Draft - Proposal for code linting

Best to check out this branch and run the commands for yourself.
Didn't commit all the file changes that would happen because of this (over 300).
Have a branch  where I ran this command but haven't fixed all the code smells/warnings yet (there's a lot). If there's interest I can go through all the files and fixed them in one fell swoop.

Added linting commands
Added prettier & stylelint
Extended eslint configuration
Removed .vscode folder from .gitignore so project settings are picked up when the project is cloned
Added codeActionsOnSave command for relevant files
Added .vscode/extensions.json to notify developers for suggested extensions when working on this project

commands

npm run format will auto fix and format css (scss) and js (ts) files
npm run lint will lint all js and css files
npm run lint:{js/css} will lint all js or css files
npm run lint:{js/css}:fix will fix all js or css files based on linting rules
npm run prettier will check all js, json and css files for formatting
npm run prettier:fix will fix formatting for all js and css files

Hey @kayla-glick, Yea sure can do!
I would update this branch with the latest changes then. However once I run the linting there will be A LOT of changes (basically all the frontend). That's why I would prefer to first introduce the linting changes, and then you can validate this PR by running the lint commands yourself?

@kayla-glick I updated it with the latest master builds. Feel free to play around with it. I haven't changed any imports / warnings yet since it'll be over 200+ files.

Sweet, nice to hear!
Just prepare for the 200+ files of changes when you run the linx fiz command 👌


oldindex retains original index after sortupdate with multiple moves.

Greetings,
e.detail.oldindex seems to retain the original index of the item that was moved if an item is moved more than once within a list.  Note that e.detail.oldElementIndex does appear to be updated with each move.
I am working with items in a single list.  Observed this behavior in Firefox and Chrome, have not tried others.
Not sure if this is by design as I could see it being useful referring back to what the lists original order was but it does not sound like it from the documentation which simply states that it contains the old index of the dragged element (considering only list items).
If people do want the original order retained, perhaps an originalIndex property?

Hey, thanks. This sounds like a bug to me. Could you send a PR to fix this?


[All Hosts] (manifest) copyedit changes that were too late to make th…

…e last PR for these files
Since these are, in effect, changes from a review of an earlier PR, they don't need a review themselves.

Docs Build status updates of commit fdfd4a5:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs/reference/manifest/versionoverrides-1-0-content.md
:white_check_mark:Succeeded
View

docs/reference/manifest/versionoverrides-1-0-mail.md
:white_check_mark:Succeeded
View

docs/reference/manifest/versionoverrides-1-0-taskpane.md
:white_check_mark:Succeeded
View

docs/reference/manifest/versionoverrides-1-1-mail.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel


work in progress fix for  #7230

Some experiments toward making Grpc.Core nuget work with dotnet cli project targeting net45.
-- I don't like adding Microsoft.NETCore.Platforms dependency, but it doesn't work without it.
-- we can get rid of duplicate of native grpc_csharp_ext library by changing more of NativeExtension.cs code.
One problem is that on macOS X, it seems that the  /runtimes  nuget functionality doesn't work very well because:
On my machine, despite having a 32-bit version of mono, dotnet build copies a 64bit version of libgrpc_csharp_ext.dylib to the output directory  and then mono is used to run it -  which results in error loading the native library (can't load 64bit library into 32bit process).
[ERROR] FATAL UNHANDLED EXCEPTION: System.IO.IOException: Error loading native library "/Users/jtattermusch/experiments/grpc_dotnet_cli/bin/Release/net45/osx.10.11-x64/libgrpc_csharp_ext.dylib"
  at Grpc.Core.Internal.UnmanagedLibrary..ctor (System.String[] libraryPathAlternatives) <0x7809f0 + 0x00103> in <filename unknown>:0

(also, this attempts to fix the nugets generated by "dotnet pack" - which is not yet the official way of generating nugets for gRPC. Analogous steps are needed to fix the old .nuspec based nugets).

Pushed 2 more commits with a fix for the mono 32bit vs 64bit version problem described above.

Not quite sure how this PR relates to my one at #7282. That has a significant set of changes for getting it working on Windows which I don't believe are in this PR.
I don't think Grpc.Core itself should have the Microsoft.NETCore.Platforms dependency, because we don't want it pulled in for users using csproj files. It's just a final application-user dependency. Admittedly if we could make it so the end user didn't need to know about it, that would be better... we should see what that dependency does on non-dotnet-cli environments.
I suspect your issue running on OSX between 32-bit and 64-bit is related to the one I've been encountering trying to run the 32-bit CLR on Windows; I believe the tooling for dotnet run just doesn't take note of the architecture properly. If you run dotnet build -a x86 does that help? I'd expect that to copy the right file into the output directory. (That much works for me on Windows, but running it doesn't - it still uses the x64 CLR.)

Retiring this PR in favor of #7282.

On Fri, Jul 8, 2016 at 11:43 PM, Jon Skeet<EMAIL_ADDRESS>wrote:

Not quite sure how this PR relates to my one at #7282
https://github.com/grpc/grpc/pull/7282. That has a significant set of
changes for getting it working on Windows which I don't believe are in this
PR.

yes, windows tests are actually broken in my PR. Your fix seems reasonable.

I don't think Grpc.Core itself should have the
Microsoft.NETCore.Platforms dependency, because we don't want it pulled
in for users using csproj files. It's just a final application-user
dependency. Admittedly if we could make it so the end user didn't need to
know about it, that would be better... we should see what that dependency
does on non-dotnet-cli environments.

I added dependency on Microsoft.NETCore.Platforms, because that's what
"libuv" nuget package does. Also, the neat trick they are using it that the
Microsoft.NETCore.Platforms dependency is specified for targetPlatform
"netstandard1.0" so if you have older version of nuget, the dependency is
just ignored. If you have a net45 project depending on libuv, the
dependency is resolved correctly. Unfortunately, because we target net45
and netstandard1.5 separately, that trick probably won't work for us.

I suspect your issue running on OSX between 32-bit and 64-bit is related
to the one I've been encountering trying to run the 32-bit CLR on Windows;
I believe the tooling for dotnet run just doesn't take note of the
architecture properly. If you run dotnet build -a x86 does that help? I'd
expect that to copy the right file into the output directory. (That
much works for me on Windows, but running it doesn't - it still uses the
x64 CLR.)

A quick fix is I make Microsoft.NETCore.Platforms copy both x86 and x64
libraries by adding them under /runtimes/osx  (and the right variant is
chosen by gRPC loader code at runtime). Making that change to your PR would
be good I think.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/grpc/grpc/pull/7295#issuecomment-231519092, or mute
the thread
https://github.com/notifications/unsubscribe/AJeq5Mj2JY30jZiecjw6lyp_79rtGSrkks5qT0MYgaJpZM4JIhuB
.

I added dependency on Microsoft.NETCore.Platforms, because that's what
"libuv" nuget package does.

Mmm. Microsoft.Dia.SymReader.Native doesn't though. I shall see if I can get a question into the community standup.

A quick fix is I make Microsoft.NETCore.Platforms copy both x86 and x64
libraries by adding them under /runtimes/osx

Yes, that would be simple. I'll do that across the board for the moment.


unpin flake8<3dev

pytest-flake8 0.6 has been released, fixing the bug that prompted the pin.
The pin currently makes setuptools master fail to install because pytest-flake8 requires flake8 >= 3, while setuptools requires flake8 < 3.

Not everyone can upgrade to flake8 v3 due to many bugs, and flake8 v3 explicitly breaks on py26.
IMO pytest-flake8 needs to support flake8 v2 also, otherwise it isnt usable for any projects that continue to support py26.

Gotcha, in that case this would need to pin pytest-flake8 to < 0.6 as well as flake8 to < 3. However, it's odd that this runs successfully with flake8 3 on Python 2.6 on Travis, isn't it?

It is quite odd.  Maybe this project is avoiding the single function which uses them flake8.utils.parameters_for, which is about plugins.
flake8 $ git grep OrderedDict
src/flake8/utils.py:        parameters = collections.OrderedDict([
src/flake8/utils.py:        parameters = collections.OrderedDict([

Sorry I missed this before committing my own fix at 93c69277ef35.


CLOUDSTACK-9654 Missing hypervisor mapping of various SUSE Linux gues…

Jira
CLOUDSTACK-9654 Missing hypervisor mapping of various SUSE Linux guest os versions on VMware 6.0
Issue:
Currently many versions of SUSE Linux does not have any hypervisor mapping entry in guest_os_hypervisor table in cloud database for VMware 6.0. Also observed that the guest_os_name field is incorrect for some SUSE Linux variants, which results in deployed instance (with SUSE Linux) set to guest OS type as "Other (64-bit)" on vCenter, which would not represent the guest OS accurately on hypervisor.
Fix:
Add the missing hypervisor mappings
Signed-off-by: Sateesh Chodapuneedi<EMAIL_ADDRESS>The current (4.9) list of SUSE Linux guest os in database looks as below,
> mysql> select id,display_name from guest_os where display_name like '%suse%';
 +-----+----------------------------------------------+
 | id  | display_name                                 |
 +-----+----------------------------------------------+
 |  40 | SUSE Linux Enterprise Server 9 SP4 (32-bit)  |
 |  41 | SUSE Linux Enterprise Server 10 SP1 (32-bit) |
 |  42 | SUSE Linux Enterprise Server 10 SP1 (64-bit) |
 |  43 | SUSE Linux Enterprise Server 10 SP2 (32-bit) |
 |  44 | SUSE Linux Enterprise Server 10 SP2 (64-bit) |
 |  45 | SUSE Linux Enterprise Server 10 SP3 (64-bit) |
 |  46 | SUSE Linux Enterprise Server 11 (32-bit)     |
 |  47 | SUSE Linux Enterprise Server 11 (64-bit)     |
 |  96 | SUSE Linux Enterprise 8(32-bit)              |
 |  97 | SUSE Linux Enterprise 8(64-bit)              |
 | 107 | SUSE Linux Enterprise 9(32-bit)              |
 | 108 | SUSE Linux Enterprise 9(64-bit)              |
 | 109 | SUSE Linux Enterprise 10(32-bit)             |
 | 110 | SUSE Linux Enterprise 10(64-bit)             |
 | 151 | SUSE Linux Enterprise Server 10 SP3 (32-bit) |
 | 152 | SUSE Linux Enterprise Server 10 SP4 (64-bit) |
 | 153 | SUSE Linux Enterprise Server 10 SP4 (32-bit) |
 | 154 | SUSE Linux Enterprise Server 11 SP1 (64-bit) |
 | 155 | SUSE Linux Enterprise Server 11 SP1 (32-bit) |
 | 185 | SUSE Linux Enterprise Server 11 SP2 (64-bit) |
 | 186 | SUSE Linux Enterprise Server 11 SP2 (32-bit) |
 | 187 | SUSE Linux Enterprise Server 11 SP3 (64-bit) |
 | 188 | SUSE Linux Enterprise Server 11 SP3 (32-bit) |
 | 202 | Other SUSE Linux(32-bit)                     |
 | 203 | Other SUSE Linux(64-bit)                     |
 | 244 | SUSE Linux Enterprise Server 12 (64-bit)     |
 +-----+----------------------------------------------+
 26 rows in set (0.00 sec)

The current (4.9) hypervisor mappings for SUSE Linux guest os over VMware 6.0 in database looks as below. We can observe in the below query result, which lists all hypervisor mappings for SUSE Linux guest OS over VMware 6.0, many guest os listed in above query result are missing their mappings for VMware 6.0. Hence the need to add the missing hypervisor mappings.
mysql> select o.id,o.display_name, h.guest_os_name, h.hypervisor_version from guest_os as o, guest_os_hypervisor as h where o.id=h.guest_os_id and h.hypervisor_version='6.0' and h.hypervisor_type='vmware' and o.display_name like '%SUSE%';
+-----+----------------------------------+---------------+--------------------+
| id  | display_name                     | guest_os_name | hypervisor_version |
+-----+----------------------------------+---------------+--------------------+
|  96 | SUSE Linux Enterprise 8(32-bit)  | suseGuest     | 6.0                |
|  97 | SUSE Linux Enterprise 8(64-bit)  | suse64Guest   | 6.0                |
| 107 | SUSE Linux Enterprise 9(32-bit)  | suseGuest     | 6.0                |
| 108 | SUSE Linux Enterprise 9(64-bit)  | suse64Guest   | 6.0                |
| 109 | SUSE Linux Enterprise 10(32-bit) | suseGuest     | 6.0                |
| 110 | SUSE Linux Enterprise 10(64-bit) | suse64Guest   | 6.0                |
| 202 | Other SUSE Linux(32-bit)         | suseGuest     | 6.0                |
| 203 | Other SUSE Linux(64-bit)         | suse64Guest   | 6.0                |
+-----+----------------------------------+---------------+--------------------+
8 rows in set (0.00 sec)

Changed base branch to 4.9 from master.

LGTM. Travis failure is due to timeout on one of the jobs, @sateesh-chodapuneedi can you push -f to rekick Travis.
@blueorangutan package

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

@rhtyd Yes, rebased with latest 4.9, and did a force push to kick Travis checks.

Packaging result: ✔centos6 ✔centos7 ✔debian. JID-349

@rhtyd I see one travis script failed due to timeout. Remaining are all successful.
/home/travis/build.sh: line 151: 10405 Terminated              travis_jigger $! $timeout $cmd
The command "travis_wait 40 ./tools/travis/script.sh ${TESTS}" exited with 1.

Thanks @sateesh-chodapuneedi
@blueorangutan package

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

@sateesh-chodapuneedi the issue is not timeout, but a failing component test in components/test_volumes. On local env, this issue is not reproducible. With Travis, this is happening intermittently but not all the time. I'll investigate this today/tomorrow.

Packaging result: ✔centos6 ✔centos7 ✔debian. JID-350

@blueorangutan package

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: ✔centos6 ✔centos7 ✖debian. JID-358

LGTM, this is db related change. Travis has validated this (the failure is due to one intermittent issue, recently fixed). Merging this based on manual testing against my 4.9 based db.

Thanks @rhtyd for merging this.


Abstract out host validation as an on-demand playbook

Description
Making host validation optional which can be run as an on-demand playbook. Customers can run this as per their need before a vanilla installation or an upgrade maybe.
Fixes # (ANSIENG-1884)
Type of change

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[x] This change requires a documentation update

How Has This Been Tested?
Locally
Test Configuration:
Checklist:

[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[ ] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] Any dependent changes have been merged and published in downstream modules
[ ] Any variable changes have been validated to be backwards compatible

This is similar to how we use fetch_logs.yml or health_checks.yml present in playbook dir.
I found this to be more modular. But, we can totally do what you're suggesting. There aren't any serious pros/cons of either approach.

Closing in favour of #1300


Upgrade electron to v12

Closes https://github.com/sqlectron/sqlectron-gui/issues/550
Now that we are not using remote anymore for sqlectron-db-core calls, and instead we are using IPC calls and running that in the browser process, which is a node process, there is no need anymore to build native dependencies to Electron anymore. It allow us to upgrade to latest version of Electron, ~which is actually required to fix https://github.com/sqlectron/sqlectron-gui/issues/471 because a new version of ssh2 that supports new auth types depends on Node12.~ (just noticed Electron v8 is already on Node12).
I tested it running from source and a compiled version for Mac, and both worked fine with MySQL and Sqlite databases.
PS: It is pointing to ipc branch because that has not been merged yet, I am going to point it to master once that has been merged.

Yeah, don't think we're affected by any breaking changes? Would be good to confirm stuff like menus and opening/saving queries also works under electron 12 along with the database connector stuff.

@MasterOdin I have tested everything I could think of, including those you mentioned and couldn't find any issues so far.


This dependency was not found:  * mssql/package.json in ./node_modules/knex/lib/dialects/mssql/index.js

Tried installing your project but it stops my project from compiling with this error
ERROR  Failed to compile with 1 errors                                                                                4:38:44 PM
This dependency was not found:

mssql/package.json in ./node_modules/knex/lib/dialects/mssql/index.js

To install it, you can run: npm install --save mssql/package.json
Is this a bug in your project or is this related to one of your dependencies?

This was user error. Closing...


[traffic_light_classifier] support autoware auto msgs

Description
port traffic light classifier

Review Procedure

Remarks

Pre-Review Checklist for the PR Author
PR Author should check the checkboxes below when creating the PR.

[x] Read pull request guidelines
[x] Code follows coding guidelines
[x] Assign PR to reviewer

Checklist for the PR Reviewer
Reviewers should check the checkboxes below before approval.

[ ] Commits are properly organized and messages are according to the guideline
[ ] Code follows coding guidelines
[ ] (Optional) Unit tests have been written for new behavior
[ ] PR title describes the changes

Post-Review Checklist for the PR Author
PR Author should check the checkboxes below before merging.

[x] All open points are addressed and tracked via issues or tickets
[ ] Write release notes

CI Checks

Build and test for PR / build-and-test-pr: Required to pass before the merge.
Build and test for PR / clang-tidy-pr: NOT required to pass before the merge. It is up to the reviewer(s). Found false positives? See the guidelines.
Check spelling: NOT required to pass before the merge. It is up to the reviewer(s). See here if you want to add some words to the spell check dictionary.

@kmiya Approveお願いできないでしょうか？


Not support React Native 0.6 or higher.

React Native 0.6 has been changed to CocoaPods integration by default.
react-native-unity-view 1.3.3 not supports that. (lots of errors.)
Do you have an update plan?

We've been using this in a cocoapods project with react native 0.8+. If you provide more specifics about your issue we might be able to help out.

We've been using this in a cocoapods project with react native 0.8+. If you provide more specifics about your issue we might be able to help out.

Description
I have clean react-native: 0.60.4 project where I'm trying to add this library.
I've manually merged project pods with UnityConfig, i.e passed this step successfully.
However, I'm experiencing following error in Xcode:

Steps to reproduce

react-native init
cd ios/ && pod install
npm i react-native-unity-view && react-native-link react-native-unity-view
cd ios/ && pod install - issues.
Merge manually pods into UnityConfig
See screenshot.

Next issue is related with UnityInterface.h
And I can't see UnityExport folder nowhere in the project in rn-0.60.5

See https://github.com/f111fei/react-native-unity-view/issues/136 if you're interested in forking this library and making it CocoaPods compatible.

Save yourself the hassle by using https://www.npmjs.com/package/react-native-unity-view-reinvented


Add JSON tagging and comments to persistedState fields

Signed-off-by: Justin Terry (VM<EMAIL_ADDRESS>

Cool, thanks!


Integrate agent's "proxy" binding into PeerOS

Subutai Agent has CLI binding named proxy. We need to make PeerOS to use that command.

How to test:

Create https://domain:443 port mapping via Bazaar and select redirect option in UI
Navigate to http://domain (port 80)
Result: it should redirect to https://domain (port 443)

Followed the steps described above^^
After navigating to http://test1.idea-x.org, domain was redirected to https


Differences between local query and website query

I downloaded the metasra JSON file you provide ( ), and tried to replicate the results I get via your service online. I used the following snippet to search for all samples that match a DOID term:
import json

example_ontology = 'DOID:9119'
samples = []

with open('../datasets/metasra.v1-5.json') as json_file:
    data = json.load(json_file)

for sample in data:
    if example_ontology in data[sample]['mapped ontology terms']:
        samples.append(sample)

print(len(samples))

For the DOID term 9119, I get 756 samples, but if I use the online service, it returns 903 samples. Do  you know why I'm seeing this discrepancy, and how to address it?

It appears your backend also looks for child DOID terms for a given DOID term, which increases the sample counts. Is there an automated way to grab all these child terms?

Hi, yes the backend should return not only samples labelled with the query ontology term, but also samples labelled with any descendent of the query ontology term.  As far as an automated way to query the ontology for all descendants, there is not a great way to do that at the moment via the MetaSRA.
If it's any help at all, I can point you to the MetaSRA code that manages the ontology (I tried to decouple it from the MetaSRA pipeline itself and placed it into its own repo): https://github.com/mbernste/onto_lib.  Unfortunately, it is not currently well-documented.

@mbernste awesome, thanks! For now, it appears that we have to download the entire ontology JSON, and search for edges of a given terms, and collect the DOID terms that are children of a given term. Thanks!


Get mailing list cleanup routine from news.php.net

Currently doing a half-assed cleanup job with regexes, better to just base my cleanup on theirs to see how they dealt with encoding issues and stuff.
Also move logic to the NNTP BODY command instead of in the synchronizer.

Done partly in bb55c72f36b43177ce2d33a245c17df10c7e3b6a


Unclear error message on password reset

If the new password does not follow the password policy a BadRequest error with the right error message is raised from the backend but the message displayed in the frontend is a generic error message that does not help the user to understand what the problem is.

It appears that on the test/prod infrastructure the new message does not appear as a syntax error seems to be raised instead.


AttachmentList option "includePDF" removes other attachments from AttachmentList.dataTypeIds.

Description of the bug
When using AttachmentList-component the option "includePDF", doesnt seem to work as expected.
If the value "true" is added to the option "includePDF" it will ignore any other attachments from the AttachmentList.dataTypeIds.
Example: AttachmentList with dataTypeIds "datatype1, datatype2", will show a list of attachments with one attachment beeing datatype1 and one attachment is datatype2. If you add the option "includePDF: true" add "ref-data-as-pdf" to the dataTypeIds list, then it will ignore datatype1 and datatype2 from the attachmentlist view.
Steps To Reproduce

Have your applicationmetadata.json include the standard "ref-data-as-pdf" with "enablePdfCreation": true.
Add a component AttachmentList to a JSON Layout in ui/layouts
Add an attachment ID to the AttachmentList (another then "ref-data-as-pdf").
Add "ref-data-as-pdf" ID to the AttachmentList.
Check result view and see that only the attachment that is not "ref-data-as-pdf" is shown in the list.
Add the option includePDF:true to the AttachmentList.
Check result view and see that only the attachment that is "ref-data-as-pdf" is shown in the list.

From my view you can not get both the attachment and the generated PDF to show up in the same list.
Additional Information
No response

Result:

Based on the AltinnCDN I would expect the attachment list to show both the generated PDF and the attachments:

Related:

https://altinn.slack.com/archives/C02EVE4EXQA/p1688370813311539
#339 (fixing this may be breaking, and/or we should probably clean this up properly with simpler config)


feat(function) Implement host function with an environment

Fix #183.
WIP
Need to double-check everything.

cc @AdamSLevy you might be interested by this PR :-)

bors try

bors r+


Feature: improved schema proposition error handling

Adds custom errors for:

ColumnMismatchError (number of columns in header != number of columns in data)
EmptyDatasetError

Merge order:

CLI- Frontend
Backend

I like the principle of giving more specific error messages in general, though I wouldn't work too hard to support particularly badly formatted datasets like empty ones.
This change seems fine overall, aside from some minor details.

Agreed on not wanting to support egregiously badly formatted datasets. This change is just to allow us to catch and handle those errors properly (not resulting in unhandled exceptions in the backend)


Archive this feedstock

Comment:
The need for this has been removed from the upstream, as it would have required building a lot of wheels in the future.

https://github.com/ipython/ipython/pull/13880

Once https://github.com/conda-forge/ipython-feedstock/pull/183 lands, will ask for this to be archived.

@conda-forge/core please archive this feedstock :boom:
Is there a way this issue can be pinned as well?


full text search on specific fields

I would like to use fulltext search but limited to one ore more properties:
GET /post?q=bar&qFields=title,author
// something like SELECT * FROM post WHERE (title LIKE '%ba%r') OR (author LIKE '%bar%')

makes sense?
thank you

see #558

see #558

@masciugo the problem is that the repo seems to be abandoned by its maintainer because this is a quite old PR

@masciugo the problem is that the repo seems to be abandoned by its maintainer because this is a quite old PR

@gsouf do you know any good alternative or mantained fork?

@gsouf do you know any good alternative or mantained fork?

@masciugo nothing that I know that would be as easy as json-server to setup and that would store json files. I'm myself using json-server and I just accepts its limitation as it's nothing I use in production.
If you're looking for a rest api that stores the data on the file system then I believe that this loopback framework has a connector for this: https://loopback.io/doc/en/lb3/Storage-connector.html however it's definitely more setup than json-server.

@masciugo nothing that I know that would be as easy as json-server to setup and that would store json files. I'm myself using json-server and I just accepts its limitation as it's nothing I use in production.
If you're looking for a rest api that stores the data on the file system then I believe that this loopback framework has a connector for this: https://loopback.io/doc/en/lb3/Storage-connector.html however it's definitely more setup than json-server.

to search by title for example : YOUR_URL/post?title_like=YOUR_SEARCH_TEXT
that should filter in your json server db by title so it YOUR_FIELD_NAME followed by _like
this article explain a lot of feature of json_server : https://blog.eleven-labs.com/en/json-server/
I don't know why they cannot  put this all on their documentation ...
hope it help people and let me know if it