Gurobi integration

With the following code:
       Variable[] objective = new Variable[]{
                new Variable("X1").weight(0.8),
                new Variable("X2").weight(0.2),
                new Variable("X3").weight(0.7),
                new Variable("X4").weight(0.3),
                new Variable("X5").weight(0.6),
                new Variable("X6").weight(0.4)};

        ExpressionsBasedModel.addIntegration(SolverGurobi.INTEGRATION);
        ExpressionsBasedModel model = new ExpressionsBasedModel(objective);

        model.addExpression("C1").set(0, 1).set(2, 1).set(4, 1).level(23);
        model.addExpression("C2").set(1, 1).set(3, 1).set(5, 1).level(23);
        model.addExpression("C3").set(0, 1).lower(10);
        model.addExpression("C4").set(2, 1).lower(8);
        model.addExpression("C5").set(4, 1).lower(5);

        Result result = model.maximise();

        // A valid solution of 25.8 can be produced with:
        //     X1=10, X2=0, X3=8, X4=0, X5=5, X6=23
        Assert.assertEquals(25.8, result.getValue(), 0.001);

I am getting: gurobi.GRBException: Unable to retrieve attribute 'ObjVal'.
The code works without Gurobi integration. And Gurobi 7.5 by itself works as well. Tested with Java 8 on OS X.

I never actually tested the Gurobi integration. It was written at the same time as the MOSEK and CPLEX integrations, but was never actually used.
If you want to try and fix it I'll help, but I don't have Gurobi license.
It's probably a good idea to look at how the MOSEK and CPLEX integrations are done.

I made some changes to SolverGurobi - made it more similar to SolverCPLEX and fixed some problems. (Probably didn't fix your issue.)
I asked Gurobi for a temporary license so I could finish this, but I haven't heard back from them. With a license I have no way to test. Someone else (someone with a license) needs to do that.

I tried the updated class with 44.0.1-SNAPSHOT of ojAlgo. The error is the same, but maybe a complete stack trace could help:
Optimize a model with 2 rows, 6 columns and 6 nonzeros
Coefficient statistics:
  Matrix range     [1e+00, 1e+00]
  Objective range  [2e-01, 8e-01]
  Bounds range     [5e+00, 1e+01]
  RHS range        [2e+01, 2e+01]
Presolve removed 1 rows and 1 columns
Presolve time: 0.00s

Solved in 0 iterations and 0.00 seconds
Infeasible or unbounded model
gurobi.GRBException: Unable to retrieve attribute 'ObjVal'
	at gurobi.GRBModel.get(GRBModel.java:3551)
	at org.ojalgo.optimisation.external.SolverGurobi.solve(SolverGurobi.java:242)
	at org.ojalgo.optimisation.ExpressionsBasedModel.solve(ExpressionsBasedModel.java:764)
	at org.ojalgo.optimisation.ExpressionsBasedModel.maximise(ExpressionsBasedModel.java:626)
	at SolverTest.testMath286(SolverTest.java:35)

java.lang.NullPointerException
	at java.util.Objects.requireNonNull(Objects.java:203)
	at org.ojalgo.ProgrammingError.throwIfNull(ProgrammingError.java:94)
	at org.ojalgo.optimisation.Optimisation$Result.<init>(Optimisation.java:300)
	at org.ojalgo.optimisation.external.SolverGurobi.solve(SolverGurobi.java:255)
	at org.ojalgo.optimisation.ExpressionsBasedModel.solve(ExpressionsBasedModel.java:764)
	at org.ojalgo.optimisation.ExpressionsBasedModel.maximise(ExpressionsBasedModel.java:626)
	at SolverTest.testMath286(SolverTest.java:35)

But I do not fell to be competent to debug it.
If you are from an academia, then academic license of Gurobi is without waiting - you just have to activate the license from an academic network.

Think maybe I just forgot to update the GRBModel after adding the constraints. Try again.
(The academic license doesn't apply to me.)

Once I add constraints:
        model.addExpression("C6").set(1, 1).lower(0);   // X2 >= 0
        model.addExpression("C7").set(3, 1).lower(0);   // X4 >= 0
        model.addExpression("C8").set(5, 1).lower(0);   // X6 >= 0

It works. The problem was that during the definition of the variables the lower bounds for {X2, X4, X6} were set to Double.NEGATIVE_INFINITY. And once you permit negative values, the problem is Infeasible or unbounded model and ObjVal is not set.
It looks like both, ojAlgo and Gurobi, assume that variables are, if not told otherwise, non-negative. But why is then negative infinity passed as the lower bound?

Good to hear that something works, but ojAlgo makes no assumptions on restrictions (non-negative) of variables (and I'm pretty sure neither does Gurobi). That part sounds confusing.
Could you do a few more tests (be sure to use the latest source code) with some variations to check if you get the same results with or without the Gurobi integration.

http://www.gurobi.com/documentation/7.0/refman/variables.html

In mathematical programming, the convention is that variables are non-negative unless stated otherwise, so if you don't explicitly provide bounds for a variable, you should assume that the lower bound is 0 and the upper bound is infinite.

I will check the changes.

ojAlgo seems to make the same assumption. With the original formulation of the problem, ojAlgo returns 25.8. But if I add constraint:
model.addExpression("C6").set(1, 1).upper(-10);

ojAlgo returns 27.8.

ojAlgo does not make that assumption, I know that ;-) With the original problem formulation ojAlgo outputs this Optimisation.Result:
UNBOUNDED 25.8 @ [10.0, 0.0, 8.0, 0.0, 5.0, 23.0]

The fact that a problem/model is unbounded doesn't mean it's impossible to find a feasible solution, and that's what ojAlgo does.

I just noticed that as well :)

I executed some unit tests with Gurobi solver. LinearProblems finished without any issue. But there was a few for IntegerProblems:
In testP20130225:
Optimize a model with 57 rows, 569 columns and 1539 nonzeros
Coefficient statistics:
  Matrix range     [1e-01, 1e+01]
  Objective range  [1e-01, 5e+01]
  Bounds range     [1e+00, 1e+00]
  RHS range        [5e-01, 9e+00]
Presolve removed 0 rows and 28 columns
Presolve time: 0.00s
Presolved: 57 rows, 541 columns, 1511 nonzeros

Iteration    Objective       Primal Inf.    Dual Inf.      Time
       0    2.3350000e+02   1.132500e+01   0.000000e+00      0s
      28    2.5560000e+02   0.000000e+00   0.000000e+00      0s

Solved in 28 iterations and 0.01 seconds
Optimal objective  2.556000000e+02

junit.framework.AssertionFailedError: Solution To Relaxed Problem Not Valid!

In testP20140819:
Optimize a model with 8 rows, 16 columns and 56 nonzeros
Variable types: 0 continuous, 16 integer (0 binary)
Coefficient statistics:
  Matrix range     [4e-01, 7e+00]
  Objective range  [3e+00, 3e+00]
  Bounds range     [4e+02, 4e+02]
  RHS range        [4e+00, 2e+04]
Presolve time: 0.00s

Explored 0 nodes (0 simplex iterations) in 0.00 seconds
Thread count was 1 (of 4 available processors)

Solution count 0

Model is infeasible
Best objective -, best bound -, gap -

junit.framework.AssertionFailedError: INFEASIBLE 0.0 @ [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]

In testP20160701:
Optimize a model with 32 rows, 35 columns and 120 nonzeros
Variable types: 5 continuous, 30 integer (30 binary)
Coefficient statistics:
  Matrix range     [1e+00, 6e+00]
  Objective range  [1e+00, 4e+00]
  Bounds range     [1e+00, 1e+00]
  RHS range        [1e+00, 5e+00]
Presolve removed 0 rows and 1 columns
Presolve time: 0.01s

Explored 0 nodes (0 simplex iterations) in 0.01 seconds
Thread count was 1 (of 4 available processors)

Solution count 0

Model is infeasible
Best objective -, best bound -, gap -

junit.framework.AssertionFailedError: INFEASIBLE 0.0 @ [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]

In testP20150127full:
java.lang.NullPointerException
	at gurobi.GRBModel.setObjective(GRBModel.java:892)
	at org.ojalgo.optimisation.external.SolverGurobi$1.build(SolverGurobi.java:126)
	at org.ojalgo.optimisation.external.SolverGurobi$1.build(SolverGurobi.java:83)
	at org.ojalgo.optimisation.ExpressionsBasedModel.solve(ExpressionsBasedModel.java:762)
	at org.ojalgo.optimisation.ExpressionsBasedModel.minimise(ExpressionsBasedModel.java:635)

In testP20130409a:
Optimize a model with 3 rows, 3 columns and 8 nonzeros
Variable types: 1 continuous, 2 integer (0 binary)
Coefficient statistics:
  Matrix range     [1e-02, 5e+01]
  Objective range  [1e+00, 1e+00]
  Bounds range     [4e+03, 4e+03]
  RHS range        [2e+03, 2e+03]
Presolve removed 1 rows and 2 columns
Presolve time: 0.00s

Explored 0 nodes (0 simplex iterations) in 0.00 seconds
Thread count was 1 (of 4 available processors)

Solution count 0

Model is infeasible
Best objective -, best bound -, gap -

junit.framework.AssertionFailedError: Solution Not Correct, large norm differences 4994.066979927344 !<< 4994.066979927344

In testP20130409b:
Optimize a model with 5 rows, 4 columns and 17 nonzeros
Variable types: 1 continuous, 3 integer (0 binary)
Coefficient statistics:
  Matrix range     [1e-02, 5e+01]
  Objective range  [1e+00, 1e+00]
  Bounds range     [0e+00, 0e+00]
  RHS range        [2e+03, 2e+03]
Presolve removed 3 rows and 2 columns
Presolve time: 0.00s

Explored 0 nodes (0 simplex iterations) in 0.00 seconds
Thread count was 1 (of 4 available processors)

Solution count 0

Model is infeasible
Best objective -, best bound -, gap -

junit.framework.AssertionFailedError: Solution Not Correct, large norm differences 5328.333229067044 !<< 5328.333229067044

There are also some issues in ConvexProblems:
In testP20080117:
Optimize a model with 1 rows, 9 columns and 9 nonzeros
Model has 45 quadratic objective terms
Coefficient statistics:
  Matrix range     [1e+00, 1e+00]
  Objective range  [3e+00, 7e+01]
  QObjective range [1e-01, 1e+02]
  Bounds range     [2e-02, 8e-01]
  RHS range        [1e+00, 1e+00]
gurobi.GRBException: Objective Q not PSD (diagonal adjustment of 4.7e+00 would be required)
	at gurobi.GRBModel.optimize(GRBModel.java:1027)
	at org.ojalgo.optimisation.external.SolverGurobi.solve(SolverGurobi.java:241)
	at org.ojalgo.optimisation.ExpressionsBasedModel.solve(ExpressionsBasedModel.java:764)
	at org.ojalgo.optimisation.ExpressionsBasedModel.minimise(ExpressionsBasedModel.java:635)
	at org.ojalgo.optimisation.convex.ConvexProblems.doEarly2008(ConvexProblems.java:181)

In testP20111129:
Optimize a model with 4 rows, 4 columns and 12 nonzeros
Model has 3 quadratic objective terms
Coefficient statistics:
  Matrix range     [1e+00, 4e+00]
  Objective range  [1e+00, 4e+00]
  QObjective range [1e+00, 4e+00]
  Bounds range     [1e+00, 1e+02]
  RHS range        [1e+00, 6e+00]
Presolve removed 2 rows and 2 columns
Presolve time: 0.00s
Presolved: 2 rows, 2 columns, 4 nonzeros
Presolved model has 3 quadratic objective terms
Ordering time: 0.00s

Barrier statistics:
 Free vars  : 1
 AA' NZ     : 3.000e+00
 Factor NZ  : 6.000e+00
 Factor Ops : 1.400e+01 (less than 1 second per iteration)
 Threads    : 1

                  Objective                Residual
Iter       Primal          Dual         Primal    Dual     Compl     Time
   0  -2.79988454e+04 -2.56995000e+04  9.76e+02 3.48e-06  1.01e+06     0s
   1  -3.21723183e+03 -3.23125285e+04  1.98e+02 4.49e-07  1.36e+05     0s
   2   4.12241209e+03 -3.00623670e+04  9.68e-05 1.93e-12  6.84e+03     0s
   3   2.05113937e+03 -2.12814946e+03  6.32e-06 1.14e-13  8.36e+02     0s
   4   3.26397825e+02 -5.16259136e+02  6.32e-12 2.84e-14  1.69e+02     0s
   5   4.64409342e+01 -5.60640071e+01  2.89e-15 5.68e-14  2.05e+01     0s
   6   3.79971471e+00 -9.30747831e+00  2.00e-15 1.07e-14  2.62e+00     0s
   7   1.11327222e-02 -3.46272752e-01  1.33e-15 1.07e-14  7.15e-02     0s
   8   1.17249278e-05 -3.46644201e-04  3.77e-15 9.77e-15  7.17e-05     0s
   9   1.17365957e-08 -3.46989925e-07  2.66e-15 1.42e-14  7.17e-08     0s
  10   1.22284405e-11 -3.47309737e-10  2.66e-15 1.78e-14  7.18e-11     0s

Barrier solved model in 10 iterations and 0.00 seconds
Optimal objective 1.22284405e-11

junit.framework.AssertionFailedError: double != double
Expected :-5.281249989
Actual   :0.0

In testP20150720:
Optimize a model with 6 rows, 8 columns and 16 nonzeros
Model has 4 quadratic objective terms
Coefficient statistics:
  Matrix range     [3e-03, 1e+02]
  Objective range  [1e+00, 1e+00]
  QObjective range [2e+00, 2e+00]
  Bounds range     [4e-01, 4e+04]
  RHS range        [6e+01, 5e+04]
Presolve removed 5 rows and 8 columns
Presolve time: 0.00s

Barrier solved model in 0 iterations and 0.00 seconds
Model is infeasible
Optimize a model with 1 rows, 4 columns and 4 nonzeros
Model has 4 quadratic objective terms
Coefficient statistics:
  Matrix range     [1e+00, 1e+00]
  Objective range  [6e+03, 5e+04]
  QObjective range [2e+00, 2e+00]
  Bounds range     [5e+03, 4e+04]
  RHS range        [5e+04, 5e+04]
Presolve time: 0.00s
Presolved: 1 rows, 4 columns, 4 nonzeros
Presolved model has 4 quadratic objective terms
Ordering time: 0.00s

Barrier statistics:
 AA' NZ     : 0.000e+00
 Factor NZ  : 1.000e+00
 Factor Ops : 1.000e+00 (less than 1 second per iteration)
 Threads    : 1

                  Objective                Residual
Iter       Primal          Dual         Primal    Dual     Compl     Time
   0   4.15826932e+08  2.38234130e+09  1.81e+04 1.03e+04  5.63e+07     0s
   1   8.15132240e+08  8.73852070e+08  0.00e+00 1.03e-02  6.52e+06     0s
   2   8.16748617e+08  8.18608529e+08  0.00e+00 8.58e-05  2.07e+05     0s
   3   8.17121046e+08  8.17160757e+08  0.00e+00 8.55e-11  4.41e+03     0s
   4   8.17148822e+08  8.17148908e+08  0.00e+00 0.00e+00  9.56e+00     0s
   5   8.17148897e+08  8.17148897e+08  0.00e+00 0.00e+00  9.57e-03     0s
   6   8.17148897e+08  8.17148897e+08  0.00e+00 0.00e+00  9.58e-06     0s
   7   8.17148897e+08  8.17148897e+08  0.00e+00 0.00e+00  9.59e-09     0s

Barrier solved model in 7 iterations and 0.00 seconds
Optimal objective 8.17148897e+08

Optimize a model with 16 rows, 45 columns and 88 nonzeros
Model has 82 quadratic objective terms
Coefficient statistics:
  Matrix range     [1e+00, 1e+00]
  Objective range  [1e+11, 1e+11]
  QObjective range [4e-01, 2e+02]
  Bounds range     [3e+02, 6e+04]
  RHS range        [3e+02, 9e+05]
Warning: Model contains large objective coefficients
         Consider reformulating model or setting NumericFocus parameter
         to avoid numerical issues.
Presolve time: 0.00s

Barrier solved model in 0 iterations and 0.00 seconds
Model is infeasible or unbounded
Optimize a model with 6 rows, 8 columns and 16 nonzeros
Model has 4 quadratic objective terms
Coefficient statistics:
  Matrix range     [3e-03, 1e+02]
  Objective range  [1e+00, 1e+00]
  QObjective range [2e+00, 2e+00]
  Bounds range     [4e-01, 4e+04]
  RHS range        [6e+01, 5e+04]
Presolve removed 5 rows and 8 columns
Presolve time: 0.00s

Barrier solved model in 0 iterations and 0.00 seconds
Model is infeasible

junit.framework.AssertionFailedError: INFEASIBLE -0.3302946228035407 @ [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, -0.5747126436781609]

In testInfeasibleCase:
Optimize a model with 1 rows, 3 columns and 3 nonzeros
Model has 3 quadratic objective terms
Coefficient statistics:
  Matrix range     [1e+00, 1e+00]
  Objective range  [1e+00, 3e+00]
  QObjective range [2e+00, 2e+01]
  Bounds range     [1e+00, 2e+00]
  RHS range        [2e+00, 2e+00]
Presolve removed 1 rows and 3 columns
Presolve time: 0.00s
Presolve: All rows and columns removed

Barrier solved model in 0 iterations and 0.00 seconds
Optimal objective 7.10050288e+01

junit.framework.AssertionFailedError
	at junit.framework.Assert.fail(Assert.java:55)
	at junit.framework.Assert.assertTrue(Assert.java:22)
	at junit.framework.Assert.assertFalse(Assert.java:39)
	at junit.framework.Assert.assertFalse(Assert.java:47)
	at org.ojalgo.TestUtils.assertFalse(TestUtils.java:336)

I did not try other unit tests.

That's a lot of problems. What did you do? Did you run all ojAlgo unit test on Gurobi instead?
Some of these problems may just be rounding errors. Inspecting the solution you may find it acceptable.
Since I don't have a Gurobi license and can't run any of these tests, I can only help with small models where you have essentially found the problem. (I can help with the fix.)

What did you do? Did you run all ojAlgo unit test on Gurobi instead?

Yes, but I modified just 3 classes.

Some of these problems may just be rounding errors. Inspecting the solution you may find it acceptable.

I agree.

Since I don't have a Gurobi license and can't run any of these tests, I can only help with small models where you have essentially found the problem. (I can help with the fix.)

I understand. Since my resources are limited, I propose to close the issue as the original problem was solved. If I encounter some issue where I know where is the problem, I will create a new issue.
Thank you for your support.

Sounds good to me.

Just fixed a bug in the CPLEX integration, and the same problem (same code mistake) was also in the Gurobi integration. Maybe all those other tests work better now.

No improvement was observed. But now one more unit test, testP20081119 in ConvexProblems, is failing:
Optimize a model with 4 rows, 4 columns and 12 nonzeros
Model has 3 quadratic objective terms
Coefficient statistics:
  Matrix range     [1e+00, 4e+00]
  Objective range  [1e+00, 4e+00]
  QObjective range [1e+00, 4e+00]
  Bounds range     [1e+00, 1e+02]
  RHS range        [1e+00, 6e+00]
Presolve removed 2 rows and 2 columns
Presolve time: 0.01s
Presolved: 2 rows, 2 columns, 4 nonzeros
Presolved model has 3 quadratic objective terms
Ordering time: 0.00s

Barrier statistics:
 Free vars  : 1
 AA' NZ     : 3.000e+00
 Factor NZ  : 6.000e+00
 Factor Ops : 1.400e+01 (less than 1 second per iteration)
 Threads    : 1

                  Objective                Residual
Iter       Primal          Dual         Primal    Dual     Compl     Time
   0  -2.79988454e+04 -2.56995000e+04  9.76e+02 3.48e-06  1.01e+06     0s
   1  -3.21723183e+03 -3.23125285e+04  1.98e+02 4.49e-07  1.36e+05     0s
   2   4.12241209e+03 -3.00623670e+04  9.68e-05 1.93e-12  6.84e+03     0s
   3   2.05113937e+03 -2.12814946e+03  6.32e-06 1.14e-13  8.36e+02     0s
   4   3.26397825e+02 -5.16259136e+02  6.32e-12 2.84e-14  1.69e+02     0s
   5   4.64409342e+01 -5.60640071e+01  2.89e-15 5.68e-14  2.05e+01     0s
   6   3.79971471e+00 -9.30747831e+00  2.00e-15 1.07e-14  2.62e+00     0s
   7   1.11327222e-02 -3.46272752e-01  1.33e-15 1.07e-14  7.15e-02     0s
   8   1.17249278e-05 -3.46644201e-04  3.77e-15 9.77e-15  7.17e-05     0s
   9   1.17365957e-08 -3.46989925e-07  2.66e-15 1.42e-14  7.17e-08     0s
  10   1.22284405e-11 -3.47309737e-10  2.66e-15 1.78e-14  7.18e-11     0s

Barrier solved model in 10 iterations and 0.01 seconds
Optimal objective 1.22284405e-11

junit.framework.AssertionFailedError: double != double
Expected :-5.281249989
Actual   :0.0

The original message used to be:
Optimize a model with 4 rows, 4 columns and 12 nonzeros
Model has 3 quadratic objective terms
Coefficient statistics:
  Matrix range     [1e+00, 4e+00]
  Objective range  [1e+00, 4e+00]
  QObjective range [1e+00, 4e+00]
  Bounds range     [1e+00, 1e+02]
  RHS range        [1e+00, 6e+00]
Presolve removed 2 rows and 2 columns
Presolve time: 0.01s
Presolved: 2 rows, 2 columns, 4 nonzeros
Presolved model has 3 quadratic objective terms
Ordering time: 0.00s

Barrier statistics:
 Free vars  : 1
 AA' NZ     : 3.000e+00
 Factor NZ  : 6.000e+00
 Factor Ops : 1.400e+01 (less than 1 second per iteration)
 Threads    : 1

                  Objective                Residual
Iter       Primal          Dual         Primal    Dual     Compl     Time
   0  -2.79988454e+04 -2.56995000e+04  9.76e+02 3.48e-06  1.01e+06     0s
   1  -3.21723183e+03 -3.23125285e+04  1.98e+02 4.49e-07  1.36e+05     0s
   2   4.12241209e+03 -3.00623670e+04  9.68e-05 1.93e-12  6.84e+03     0s
   3   2.05113937e+03 -2.12814946e+03  6.32e-06 1.14e-13  8.36e+02     0s
   4   3.26397825e+02 -5.16259136e+02  6.32e-12 2.84e-14  1.69e+02     0s
   5   4.64409342e+01 -5.60640071e+01  2.89e-15 5.68e-14  2.05e+01     0s
   6   3.79971471e+00 -9.30747831e+00  2.00e-15 1.07e-14  2.62e+00     0s
   7   1.11327222e-02 -3.46272752e-01  1.33e-15 1.07e-14  7.15e-02     0s
   8   1.17249278e-05 -3.46644201e-04  3.77e-15 9.77e-15  7.17e-05     0s
   9   1.17365957e-08 -3.46989925e-07  2.66e-15 1.42e-14  7.17e-08     0s
  10   1.22284405e-11 -3.47309737e-10  2.66e-15 1.78e-14  7.18e-11     0s

Barrier solved model in 10 iterations and 0.01 seconds
Optimal objective 1.22284405e-11

junit.framework.AssertionFailedError: double != double
Expected :-5.281249989
Actual   :0.0


Return invoice on successful lnurl-withdraw

Include the LNInvoice as inner data in the Ok result of BreezServices::lnurl_withdraw.
Subtasks:

[x] uniffi
[x] flutter bridge
[x] React Native bindings

Fixes #427

@roeierez it's ready for review.
I will add the RN bindings as last step, once we agree on the general structure.

I will add the RN bindings as last step, once we agree on the general structure.

It could be the first use of the RN codegen 😎

The PR simply adds the invoice in the withdraw Ok response.
However, if the goal is to be able to link a payment with a LNURL-withdraw request, an alternative way to do this is to extend LnPaymentDetails, which is a field of Payment
https://github.com/breez/breez-sdk/blob/41b3a01321b79ffc576755fa5ec24c80b959d5a0/libs/sdk-core/src/models.rs#L661-L678
with an lnurl_withdraw_endpoint: Optional<String>, where we can store the LNURL-w endpoint associated with this Payment.
That may be preferable to the approach in this PR, because in the PR the LNURL handling is inconsistent:

lnurl_pay result has no reference to the invoice, but LnPaymentDetails has a few pay-related optional fields: https://github.com/breez/breez-sdk/blob/41b3a01321b79ffc576755fa5ec24c80b959d5a0/libs/sdk-core/src/models.rs#L669-L677
lnurl_withdraw has the reverse: invoice in the service call result, but no reference in the Payment

What do you think? Is the PR approach ok? Or better switch to the above? Or do both?

If possible, as a user, I would like to have both :) that way, I can avoid always searching for the correct payment after making an LNURL withdrawal.
Otherwise, the withdrawal flow is:

Start withdrawal
Search for the created payment associated with the withdrawal
Wait for BreezEvent::InvoicePaid that includes the hash of the withdrawal payment

If we have what the current version of this PR proposes, the middle step can be discarded.

@dangeross I took your RN generator for a test-drive and ran it on this PR.
Can you please doublecheck if all looks good? fe8884de178d4135b64f13f1a80c8224c079ccb5

ktlint --version

1.0.0

kotlin -version

Kotlin version 1.9.10-release-459 (JRE 20.0.2)

ktlint --version

1.0.0

Ok, seems to be a version difference. I've upgraded to 1.0.0

@ok300 the PR looks good to me. As for your comment above do you want to add the step of associated the withdraw endpoint with the payment in the storage or not?  Another option is in a separate PR.

If possible, as a user, I would like to have both :)

Then I'll add both in this PR.

@ok300 @roeierez
Shouldn't we apply these naming changes to LnUrlCallbackStatus used on lnurl_auth as well?

@erdemyerebasmaz how about LnUrlCallbackStatus => LnUrlCallbackResult ?
It's used not only in LNURL-auth, but also as intermediary step in LNURL-withdraw.

@erdemyerebasmaz how about LnUrlCallbackStatus => LnUrlCallbackResult ?
It's used not only in LNURL-auth, but also as intermediary step in LNURL-withdraw.

@ok300 I see, isn't it supposed to be LnUrlResult according to:

Perhaps if we are introducing new types here we can use better naming?
LnUrlWithdrawCallbackStatus => LnUrlWithdrawResult
LnUrlWithdrawOkData => LnurlWithdrawSuccess
LnUrlErrorData => LnurlError

Well, there are 2 naming conventions which don't overlap:

one is for the LNURL-withdraw specific case. Calling the struct LnUrlWithdrawResult makes sense.
one is for a generic case that affects several callback types. LnUrlResult gives the wrong impression that its what a LNURL-workflow is supposed to end in, but it's only the case for auth. It's an intermediary result in withdraw. IMO LnUrlCallbackResult better captures the nature if it.

Thanks for the clarification, second item was what made it confusing for me. If that's the case, I see no issue with keeping LnUrlCallbackStatus as it is.

Then @roeierez  this PR is ready for final review.
I also linked the c-breez PR by @erdemyerebasmaz in the description above.


[plugin-parser] Add plugin-parser to handle rpk packages

Remove pkg-mgr module from the daemon
Add libmlops-plugin-parser.so to handle package install / upgrade / uninstall for Tizen

Consider the cpp-linter is now broken https://github.com/cpp-linter/cpp-linter-action/issues/214

Ignore lint error, merge this PR.

@yeonykim2 please update json format on this parser.


childhood absence epilepsy

Mondo terms (ID and Label):
childhood absence epilepsy MONDO:0010826
epilepsy, childhood absence, susceptibility to, 1  MONDO:0020759
Xrefs that should be fixed (ID and label):
https://www.omim.org/phenotypicSeries/PS600131: Epilepsy, childhood absence
https://www.omim.org/entry/600131: EPILEPSY, CHILDHOOD ABSENCE, SUSCEPTIBILITY TO, 1; ECA1
Other comments:

HI @maglott I hope to get to this soon, we're planning to have a call to discuss how to address susceptibility terms soon. Thanks for reporting this

https://www.omim.org/entry/600131: EPILEPSY, CHILDHOOD ABSENCE, SUSCEPTIBILITY TO, 1; ECA1 is part of the phenotypic series in OMIM:
https://omim.org/phenotypicSeries/PS600131

Update:
The OMIMPS seems to combine a mix of diseases and susceptibility to diseases:

OMIM:611136  Epilepsy, childhood absence, susceptibility to, 13 (MONDO_0012627 'epilepsy, idiopathic generalized, susceptibility to, 13'). OMIM includes susceptibility to, 4 and 5 in this. These are synonyms for type 13 in Mondo.
OMIM:611136  Epilepsy, juvenile myoclonic, susceptibility to, 5 (see above)
OMIM:607681   Febrile seizures, familial, 8 (MONDO_0011891 'febrile seizures, familial, 8'). OMIM includes  susceptibility to, 2
OMIM:607681  Generalized epilepsy with febrile seizures plus, type 3 (included with Febrile seizures, familial, 8)
OMIM:600131 EPILEPSY, CHILDHOOD ABSENCE, SUSCEPTIBILITY TO, 1 (MONDO_0020759 'epilepsy, childhood absence, susceptibility to, 1')
OMIM:612269 Epilepsy, childhood absence, susceptibility to, 5 (MONDO_0012843 'epilepsy, childhood absence, susceptibility to, 5')
OMIM:611942 Epilepsy, childhood absence, susceptibility to, 6 (MONDO_0012763 'epilepsy, childhood absence, susceptibility to, 6')
OMIM:611942 Epilepsy, idiopathic generalized, susceptibility to, 6 (included with OMIM:611942)

In OMIM:


Fix xdist test error. Also make a small cleanup some codes

Description
Resolves #792
@blrchen Let's see if this resolves the errors in the github test pipelines.
How was this PR tested?
In a local machine, a test w/ the xdist option passed.
Does this PR introduce any user-facing changes?

[x] No. You can skip the rest of this section.
[ ] Yes. Make sure to clarify your proposed changes.

databricks:====== 12 failed, 72 passed, 7 skipped, 74 warnings in 873.66s (0:14:33) =======
synapse:====== 22 failed, 63 passed, 6 skipped, 16 warnings in 1478.44s (0:24:38) ======

This PR only resolves the issue that test not work with pytest-xdist. For the 12 databricks failures and 22 synapse, they are related to #756. The failed test case # before this PR is before about 5 or 7)
@loomlike @Yuqing-cat i think we should revert #756 from main now for 0.9.0 release. @loomlike can submit it again to next release after all CI test failures are resolved.

Sorry for the large file changes. It's due to reverting #798.
Additionally, this PR includes changes in #786


Camera display is not showing overlays (Iron and Rolling)

Camera display is not showing overlays (Iron and Rolling). In Humble this feature is working

This PR broke this feature https://github.com/ros2/rviz/commit/4e976dc444e7e59ec9cdef30f1f04e0beafcafc8


feat: Improve error reporting and logging

I think we might be over engineering the problem here.
Have you thought of just using and express logger? https://github.com/pinojs/express-pino-logger
I don't really like to see logger dependencies being explicitly injected all the way to the controller.

I think we might be over engineering the problem here.
Have you thought of just using and express logger? https://github.com/pinojs/express-pino-logger
I don't really like to see logger dependencies being explicitly injected all the way to the controller.

hmmm... why is it over-engineered? Haven't we been using debug logger all over the place? I'd argue that using logging library without DI lowers the quality of the code. I'd often like to assert that an error was logged for instance. But yeah we don't obviously need to inject loggerMock in our tests if we don't want to, we can just use winston logger and turn off logging in tests entirely, but I chose to inject mock for the reasons stated earlier.
I've looked into this pino thing as well as pino-express but I chose more lightweight and more popular library. I'd probably go for https://www.npmjs.com/package/express-winston but it doesn't seem to provide anything more than we already have with the lightweight implementation that I used. Plus we are still having to adjust the transports for lambda anyway.

I've given it another thought. The bit I was considering over-engineer is just what it is. If we want to have a request-contextualised logger in the controllers and utils, then we need to inject it as a dependency. 🤷🏽♂  I think having DI everywhere, for everything is not pretty, considering that some of this dependencies are very easy to mock/test, but that is a whole other debate.
Yet, I feel like we are still not addressing the problem the right way. We are using express, yet we are not taking advantage of any of its features. If you use the express version of Winston you don't need to directly inject the logger as a dependency to the error handler middleware, or other middleware we might use.
The other thing I'm not particularly keen on is Winston's defaults. It's a big lib, with a lot of features, most of which we don't use and therefore require us too make considerable boiler-plating just to get JSON logged to the console. Whereas Pino's default and interface is much much friendlier.

I've given it another thought. The bit I was considering over-engineer is just what it is. If we want to have a request-contextualised logger in the controllers and utils, then we need to inject it as a dependency. 🤷🏽♂ I think having DI everywhere, for everything is not pretty, considering that some of this dependencies are very easy to mock/test, but that is a whole other debate.
Yet, I feel like we are still not addressing the problem the right way. We are using express, yet we are not taking advantage of any of its features. If you use the express version of Winston you don't need to directly inject the logger as a dependency to the error handler middleware, or other middleware we might use.
The other thing I'm not particularly keen on is Winston's defaults. It's a big lib, with a lot of features, most of which we don't use and therefore require us too make considerable boiler-plating just to get JSON logged to the console. Whereas Pino's default and interface is much much friendlier.

ok, ill change to pino, you might be right it may not require this stupid console log formatter to work with lambda out of the box so its worth a try. With winston the problem was the missing request-id.
But we still need to pass the dependency everywhere the logger is used, no? I think that's what you said, that you agree with that bit.
Request contextualised - do you mean a logger instance per request? yeah im not actually doing it know, im only relying on lambdas to print the request ID next to the log line. Are you thinking we should inject the request ID somewhere?

Cool 🙌🏽 The initialisation of the logger is probably get simpler that way.
Regarding requestId, feel free to use whichever you prefer. Pino also generates reqIds and rotates them by default. Eventually, when we grab tracing we might want to revisit this and make traces use the same request ID as the logs, but for now either one will do.
I thought you were injecting the logger dependency because you need it to contain the requestID, so that the logs are request-contextualised. If that is not the case, then I don't see the reason to have it as dependency. We can just import it straight from the utils. And I'm not sure about this, but we might even get context into the logger if we use async-hooks https://itnext.io/request-id-tracing-in-node-js-applications-c517c7dab62d.

Cool 🙌🏽 The initialisation of the logger is probably get simpler that way.
Regarding requestId, feel free to use whichever you prefer. Pino also generates reqIds and rotates them by default. Eventually, when we grab tracing we might want to revisit this and make traces use the same request ID as the logs, but for now either one will do.
I thought you were injecting the logger dependency because you need it to contain the requestID, so that the logs are request-contextualised. If that is not the case, then I don't see the reason to have it as dependency. We can just import it straight from the utils. And I'm not sure about this, but we might even get context into the logger if we use async-hooks https://itnext.io/request-id-tracing-in-node-js-applications-c517c7dab62d.

We can't inject request ID like that because the instantiation of the app happens outside of the handler so it's the same logger instance per every request anyway. Like I said earlier we could create some sort of global variable which holds request Id (and, potentially, some metadata).
I would like to inject logger as dependency because it then makes it testable (i would like to assert that an error is logged in some of your handlers) but I see your point, we could just export an instance as a module and simply import that without having to use the DI at all.

Cool 🙌🏽 The initialisation of the logger is probably get simpler that way.
Regarding requestId, feel free to use whichever you prefer. Pino also generates reqIds and rotates them by default. Eventually, when we grab tracing we might want to revisit this and make traces use the same request ID as the logs, but for now either one will do.
I thought you were injecting the logger dependency because you need it to contain the requestID, so that the logs are request-contextualised. If that is not the case, then I don't see the reason to have it as dependency. We can just import it straight from the utils. And I'm not sure about this, but we might even get context into the logger if we use async-hooks https://itnext.io/request-id-tracing-in-node-js-applications-c517c7dab62d.

unfortunately the exact same thing happens with pino:

and the solution is just as ugly as with winston (having to swap stdout for regular console logs). But we can still stick to pino if you want, ill add the console bit as suggested here and see if it works.
Don't worry about async hooks for now - we don't need it in lambdas and as far as I know we are not getting off lambdas any time soon, are we?
Anyway would you like me to import logger as a module and use it without the DI? I think then I could just change the log level to silent so it does not log anything at all in tests. The only thing is - like I mentioned before I would like to assert that the error or warning is logged in some cases so I still might want to inject the logger at least into the event handlers. How do you feel about that?

Last catch up on this on Monday. We got a few loose ends here that we should address 😉

This is looking great, mate! Good job! 🙌🏽

From the ticket

Logs should be filterable by log level, or other parameter (i.e. userId)

Could you investigate how hard it would be for us to add the userId to the log message?

This is looking great, mate! Good job! 🙌🏽

From the ticket

Logs should be filterable by log level, or other parameter (i.e. userId)

Could you investigate how hard it would be for us to add the userId to the log message?

The code might look great but the output is far worse:

It doesn't prefix the log lines with request ID, instead the pino-lambda lib only adds the text message there (not sure what the benefit of that would be anyway).
We could try to add requestId and userId inside the log object with async hooks or something but i feel like it would be easier to just replace the console.stout thing with regular console.log just to get it out

It seems like we're not using pino-lambda properly. We're supposed to initialised the lib with the event & context in order to build proper logs.

Yet, doing just this won't solve all our issues. We still need to get request context to the controllers and so on. To prevent injecting this logger as a dependency we must use async_hooks. I was reading this blogpost on using async hooks for logging and I was mind blown.
We can catch up on this tomorrow again! 😉

It seems like we're not using pino-lambda properly. We're supposed to initialised the lib with the event & context in order to build proper logs.

Yet, doing just this won't solve all our issues. We still need to get request context to the controllers and so on. To prevent injecting this logger as a dependency we must use async_hooks. I was reading this blogpost on using async hooks for logging and I was mind blown.
We can catch up on this tomorrow again!

Yeah I missed that bit about actually tying it to the event and context.
I think setting up async hooks is not actually such a big deal in express cause there is many available tutorials, it's just the testing part that puts me off a little bit but I have tested it before so it shouldn't be such a big deal either. I think I am leaning more towards async hooks than storing per-request info in some global cache than I did before.
I think there are some performance implications to using async hooks but surely not in our scale.
BTW i dont think it has a lot to do with the DI, you know. You may have realised that we are only creating a single app instance once, outside of the handler, so either way we would have to use some sort of global state to hold the current requestId which btw wouldn't work if we moved away from lambdas. Async-hooks are definitely a way to go, DI or not.

actually mate I dont even think we need async hooks for now:

The request Id and all the request info are all inside the log object, we could try to pick the user info later if we want to but i think this is done

As of our conversation, please make sure you:

Hide sensitive information from the logs, such as authorization headers;
Test the error handler to make sure the logger is called, to make use of DI. Or, if you prefer not to, import the logger instance directly from utils and the Factory pattern;

Authorization header is now hidden:


Update fedora template to f41

CI is failing:
https://github.com/lima-vm/lima/actions/runs/11577435687/job/32232565275?pr=2821
TEST| [INFO] Testing home bind mount ("/home/runner/lima-container-engine-test-tmp")
+ limactl shell fedora nerdctl pull --quiet ghcr.io/containerd/alpine:3.14.0
time="2024-10-29T16:52:08Z" level=warning msg="treating lima version \"d08c206\" from \"/home/runner/.lima/fedora/lima-version\" as very latest release"
+ echo random-content-11144
++ cat /home/runner/lima-container-engine-test-tmp/random
+ expected=random-content-11144
++ limactl shell fedora nerdctl run --rm -v /home/runner/lima-container-engine-test-tmp/random:/mnt/foo ghcr.io/containerd/alpine:3.14.0 cat /mnt/foo
time="2024-10-29T16:52:12Z" level=warning msg="treating lima version \"d08c206\" from \"/home/runner/.lima/fedora/lima-version\" as very latest release"
time="2024-10-29T16:52:13Z" level=fatal msg="failed to get unprivileged mount flags for \"/home/runner/lima-container-engine-test-tmp/random\": stat /home/runner/lima-container-engine-test-tmp/random: no such file or directory"

Similar to:

https://github.com/lima-vm/lima/issues/2701

Quite weird
[suda@lima-fedora lima-container-engine-test-tmp]$ cat random
cat: random: No such file or directory
[suda@lima-fedora lima-container-engine-test-tmp]$ sudo cat random
random-content-19795
[suda@lima-fedora lima-container-engine-test-tmp]$ sudo mount -o remount /home/suda
[suda@lima-fedora lima-container-engine-test-tmp]$ cat random
random-content-19795

Fedora 41's kernel 6.11 causes QEMU to segfault?
cd $HOME
rm -rf foo bar
mkdir foo bar
touch foo/foo
limactl shell $INST cat $HOME/foo/foo
rm -rf foo
touch bar/bar
limactl shell $INST cat $HOME/bar/bar
# output: "cat: /home/suda/bar/bar: Is a directory"
limactl shell $INST ls $HOME/bar/bar
# output: (none, QEMU segfaults here)

(Tested on Ubuntu 24.10 x86_64, QEMU emulator version 9.0.2 (Debian 1:9.0.2+ds-4ubuntu5)

Looks like we are hitting the kernel bug introduced in v6.9-rc1, fixed in v6.12-rc5: Revert "fs/9p: simplify iget to remove unnecessary paths" https://github.com/torvalds/linux/commit/be2ca38

Looks like we are hitting the kernel bug introduced in v6.9-rc1, fixed in v6.12-rc5: Revert "fs/9p: simplify iget to remove unnecessary paths" torvalds/linux@be2ca38

Hm. This is interesting. Fedora tends not to respin their images after releasing. So that leaves us in a weird position. Can we mitigate this at all in the lima template? If not, we could consider requesting a new cloud image with an updated kernel (once the fix is released). I don't know whether this would be considered significant enough to get approved.


not able to pass custom headers in WebSocket link

tried to pass custom headers in web-socket link. i have tried passing headers in headers and also in initial Payload. i am receiving "Sec-WebSocket-Protocol" header but not the custom headers.below is my code. thanks in advance.
final WebSocketLink websocketLink = WebSocketLink(
url,
config: SocketClientConfig(
autoReconnect: true,
inactivityTimeout: Duration(seconds: 30),
initialPayload:()=>{
"headers": {
"Sec-WebSocket-Protocol": "graphql-ws",
"device_id":"PKQ1",
}
},
headers :{
"device_id":"PKQ1",
"Sec-WebSocket-Protocol": "graphql-ws",
}
),
);

You are using the header in the wrong way.
You may want take a look to this enum https://github.com/zino-hofmann/graphql-flutter/blob/main/packages/graphql/lib/src/links/websocket_link/websocket_client.dart#L162-L176 and Just grab the code on how it is used


Handle relationships

Since dynamodb doesn't have relationships, it may be beneficial to add an example or a default implementation of relationships using dynamodb.

Added relationships to both library and flask example.


Roots watch not noticing any changes

Running roots watch does not pick up any changes for me. I get no error message, it just doesn't see any changes that I make in my code. Any ideas?

Which system are you on? Does the server serve the correct files?

Yeah I can't replicate this issue. Need a lot more detail here.

I was on MacOS 10.9.5. Anyway, updating Node & NPM seems to have fixed this for me.

I'm getting this issue at the moment. Running roots watch compiles successfully, but does not detect changes after that (without cancelling the watcher and running again).
I'm using the latest version of Roots, but working on an old site that I built on version 2 or 3 I think. I got a few warnings when installing the package:
npm WARN deprecated<EMAIL_ADDRESS>Jade has been renamed to pug, please install the latest version of pug instead of jade
npm WARN deprecated<EMAIL_ADDRESS>Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
npm WARN deprecated<EMAIL_ADDRESS>Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
npm WARN deprecated<EMAIL_ADDRESS>Deprecated, use jstransformer
npm WARN deprecated<EMAIL_ADDRESS>Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
npm WARN prefer global<EMAIL_ADDRESS>should be installed with -g


Create sensors for Connections and Folders

Could be created through MQTT discovery (which I already know well, but would require config settings) or I could learn the HA API.

There is already an HA integration for this, however it requires exposing the syncthing UI port outside of ingress.  So I have to think about that.

A complicated option would be to allow proxying of the rest calls but not the GUI.  This is probably a dumb idea because 1) it is overly complicated and 2) it probably gives a false sense of security as the rest interface can likely do everything the GUI can.
So, I think the answer is to make exposing the GUI port optional.  Add some documentation to note the increase security risk and call it a day.

Closed by #9


Build can't be easily automated when a parser function is needed

Prerequisites

[X] Write a descriptive title.
[X] Make sure you are able to repro it on the latest released version
[X] Search the existing issues.

Steps to reproduce
Create a build script for a Crescendo configuration that requires an external script for parsing
. "$PSScriptRoot\helpers\parse.ps1"

Export-CrescendoModule -ConfigurationFile "$PSScriptRoot\crescendo\mycommand.json" -ModuleName MyCommand -Force

Expected behavior
The functions in parse.ps1 should be available for Export-CrescendoModule

(I'm actually not sure they will be available for any external scripts unless deployed from a console that has had the functions dot sourced.)

Actual behavior
Crescendo fails with error

Work around is to dot source the script by hand into the console that starts the build script

Error details
Exception: Cannot find output handler function 'XXX'.

Environment data
Name                           Value
----                           -----
PSVersion                      7.2.10
PSEdition                      Core
GitCommitId                    7.2.10
OS                             Microsoft Windows 10.0.19045
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           <IP_ADDRESS>
WSManStackVersion              3.0

Version
1.1.0
Visuals
No response

My workaround is to execute a separate "Build.ps1" script that dot sources the helper functions ps1 and then executes the ps1 that contains your Export-CrecendoModule command. In my example all 3 ps1 files are in the same directory.
This approach creates a fresh PowerShell session and ensures and eliminates any clobber issues with changing functions in the same session before re-running .
I keep a separate PowerShell session open and I re-run the ". build.ps1" anytime I make modifications to crescendo jsons or the helper functions. This resolved my issues with Crescendo not finding the helper functions or not using the latest version and allowed me to keep helper functions in separate files.
. Build.ps1
. "$PSScriptRoot\Build_Functions.ps1"    #Contains All helper functions
. "$PSScriptRoot\Build_Deploy.ps1"       #Contains Export-CrecendoModule


Incorrect maven purls for packages with different Scala targets

What happened:
I recently updated from syft v0.85.0 to syft v0.94.0 to get a maven purl fix introduced in v0.94.0.  I rescanned my container, and while many maven purls were fixed, I noticed that a subset of the maven packages were not displaying properly, having a number instead of the package name.  This also impacts the bom-ref component field.  This appears to be related to maven dependencies that can have different Scala targets, like akka-actor.  Specifically, when using a Scala version that has a dot in it, like 2.13.
What you expected to happen:
Expected purl<EMAIL_ADDRESS>Received purl<EMAIL_ADDRESS>Steps to reproduce the issue:
I built a basic maven project where I included the akka-actor_2.13 dependency in pom.xml:
    <!-- https://mvnrepository.com/artifact/com.typesafe.akka/akka-actor -->
    <dependency>
      <groupId>com.typesafe.akka</groupId>
      <artifactId>akka-actor_2.13</artifactId>
      <version>2.7.0</version>
    </dependency>

I also used maven-dependency-plugin with copy-dependencies to copy dependency jars over to my target directory, then built a Dockerfile containing the target directory.
Syft command is syft {IMAGE_NAME} --scope all-layers -o cyclonedx-json > bom.json
Anything else we need to know?:
Environment:

Output of syft version:

Application: syft
Version:    0.94.0
BuildDate:  2023-10-20T17:21:07Z
GitCommit:  8f6bdde6662aa8050a71eadbdb7bd5a3b079a56d
GitDescription: v0.94.0
Platform:   linux/amd64
GoVersion:  go1.21.3
Compiler:   gc

OS (e.g: cat /etc/os-release or similar):

NAME="Alpine Linux"
ID=alpine
VERSION_ID=3.18.4
PRETTY_NAME="Alpine Linux v3.18"
HOME_URL="https://alpinelinux.org/"
BUG_REPORT_URL="https://gitlab.alpinelinux.org/alpine/aports/-/issues"

Hi @jdweaver275, thanks for the report. I think we know what's going on here. We will put this in our backlog for a fix as soon as we are able.


TECH-1016: test review and data_agreement tests

Description

Smoke test is reviewed
Added example feature + implication test
Added tests for "/config/data-agreement/{dataAgreementId}"

Confluence tutorial:
https://govstack-global.atlassian.net/l/ce/yrmQGygR
Doc reference:
https://behave.readthedocs.io/en/latest/
Related Issue

Motivation and Context

How Has This Been Tested?

@lruzicki the PR is in draft mode, so I just wanna check in about the intentions? Are you adding more work?

@benjaoming Yes I plan to add more work into this. I will try to make one more test and organize my comments. Once it will be done I will set it to the "ready for review" status

PR is ready and can be merged

@lruzicki Thanks for the PR. I'll merge it.


fix(widget-module-flags): Prevent multiple flagging calls

SSDK-1775

Deploy preview for hopeful-hamilton-4090b9 ready!
Built with commit 8066cf975bf194a76b077f7c923109edc352e23b
https://deploy-preview-673--hopeful-hamilton-4090b9.netlify.com


Test Issue created at Fri, 28 Jan 2022 02:20:12 GMT

Test issue body  Fri, 28 Jan 2022 02:20:12 GMT

Comment on issue at Fri, 28 Jan 2022 02:20:14 GMT


External authorization using ext_authz for Redis AUTH

Hello,
I have set up an external authorization service to authorize Redis AUTH calls. But in my authorization service, I cannot see the original Redis AUTH command AUTH username password.
How can I parse the AUTH command? In the CheckRequest object, the Request attribute is coming null.
Envoy configuration
"filter_chains": [
                                {
                                    "filters": [
                                        {
                                            "name": "envoy.filters.network.ext_authz",
                                            "typed_config": {
                                                "@type": "type.googleapis.com/envoy.extensions.filters.network.ext_authz.v3.ExtAuthz",
                                                "stat_prefix": "ext_authz",
                                                "grpc_service": {
                                                    "envoy_grpc": {
                                                        "cluster_name": "ext-authz-cluster"
                                                    }
                                                },
                                                "transport_api_version": "V3"
                                            }
                                        },
                                        {
                                            "name": "envoy.redis_proxy",
                                            "typed_config": {
                                                "@type": "type.googleapis.com/envoy.extensions.filters.network.redis_proxy.v3.RedisProxy",
                                                "stat_prefix": "redis_stats",
                                                "settings": {
                                                    "op_timeout": "1s",
                                                    "enable_redirection": true,
                                                    "read_policy": "PREFER_MASTER",
                                                    "enable_command_stats": true
                                                },
                                                "prefix_routes": {
                                                    "catch_all_route": {
                                                        "cluster": "RedisCluster"
                                                    }
                                                }
                                            }
                                        }
                                    ]
                                }
                            ]

C# gRPC service
public override async Task<CheckResponse> Check(CheckRequest request, ServerCallContext context)
{
   request.Attributes.Request -> this is null
    var response = new CheckResponse
    {
        OkResponse = new OkHttpResponse(),
    };

    return await Task.FromResult(response);
}

Tagging based on another similar issue - https://github.com/envoyproxy/envoy/issues/21194
@esmet  @pradeepcrao  @ggreenway  @alyssawilk

I don't think there's currently any way to do this.
To write this feature, you'd need a filter that runs before the ext_authz filter that can extract the data you want to pass to auth, and set it as dynamic metadata. Then you could add an option to the network ext_authz filter to send specific metadata with an auth request.

I don't think either part can be done without changes to envoy.

should I submit a separate feature ask?

No need to file another issue

@ggreenway Can i take a shot at this? Do I need to send a proposal first ?


Heuristics on choosing a model structure

Hi,
I was wondering if there was any heuristics on choosing a model structure for different types / sizes of datasets. For instance, if I had a standard corporate dataset with 20,000 rows and 15 columns, are there any sure-fire methods / parameters I should be using? Are there any clear do's or dont's in certain situations?

Hi @AnotherSamWilson, thanks for your question. We think the default settings should be fine for your dataset, but we've actually been working on a paper that discusses various heuristics/diagnostics for choosing model parameters. We'd be happy to share it, if you give us your email address (mine is R.Lall@lse.ac.uk).


Adds a machine_type property to orchestra.remote.Remote

There is also some minor test clean up and I made the existing machine_type test use the new property.
Signed-off-by: Andrew Schoen<EMAIL_ADDRESS>

Tests failed for this pull request.
Refer to this link for build results: http://jenkins.ceph.com/job/teuthology-pull-requests/829/

test this please

Tests passed for this pull request.
Refer to this link for build results: http://jenkins.ceph.com/job/teuthology-pull-requests/830/

Tests passed for this pull request.
Refer to this link for build results: http://jenkins.ceph.com/job/teuthology-pull-requests/831/

Tests passed for this pull request.
Refer to this link for build results: http://jenkins.ceph.com/job/teuthology-pull-requests/832/


Reduce webview size by banner height on iOS

On iOS, the webview height currently doesn't take into account the height of the banner, which means you are unable to scroll all the way to the bottom of a page or access any links there.
This change should fix the issue such that the webview ends right where the location bar begins. Tested both on initial load and setting the banner via iab.setBannerMessage().
Will need to update the CMS contents to revise the button placement after this starts going out.

Probably doesn't matter for now, but does this still work when we switch the location of the browser controls, banner message, etc?

It should, since this is inside the if (_browserOptions.banner) condition. I will give it a test real quick


use StopAllIterationAndWatermark in geoip filter

Commit Message: use StopAllIterationAndWatermark in geoip filter
Additional Description:
Use StopAllIterationAndWatermark in geoip filter to stop both headers and data when performing async geoip lookup. This fixes the issue in #33021 where large POST request data got corrupted.
Risk Level: low
Testing:
Docs Changes:
Release Notes:
Platform Specific Features:
[Optional Runtime guard:]
Fixes #33021
[Optional Fixes commit #PR or SHA]
[Optional Deprecated:]
[Optional API Considerations:]

Thank you for neat contribution, PR lgtm modulo ci and changelog needs to be updated.


Add setup instructions into README file

In the README file should be setup instructions for newcomers.

Thanks!!
Completely forgot about that.
I have updated the README


Chore: Replace broken Codepen link - Show National Contiguity with a Force Directed Graph

Quite a shame about the dataset problem. At least there is a new Codepen for it.
Checklist:

[X] I have read freeCodeCamp's contribution guidelines.
[X] My pull request has a descriptive title (not a vague title like Update index.md)
[X] My pull request targets the main branch of freeCodeCamp.
[X] I have tested these changes either locally on my machine, or GitPod.

Closes #47289

Tom(?)'s project has replaced the link used for the challenge as it shows how it is supposed to look.

@moT01 do we want to move the Codepen link over to freeCodeCamp (can happen in a separate PR, if that happens)?


Unable to retrieve the verification code when trying to use an e-mail to get your token

Character.ai doesn't send you a verification code anymore. Instead, it sends you a link you have to click to verify your email (which obviously renders the e-mail login method useless)

You're right, In order to get the token you have to go to the older version of charecter ai old.character.ai, then inspect the web page, go to local in system and you should see char_token, take the value parameter from it and thats your token

@Nytro69
Yeah, I know that, but it's just that I'm coding on mobile right now (which is an absolute pain) because I'm on vacation and my parents didn't let me take my laptop with me.
It's whatever though, I'll be home in about a week from now anyways.


Don't check floating IP status DOWN after disassociation

Workaround until https://jira.onosproject.org/browse/ONOS-6454 is fixed.


what's the meaning of this code?

@abisee could you tell me the meaning of this code?

or anyone can tell me the meaning, thx.

and this code

Hi, I am also confused about this part, especially the shape of the inputs of the attention score. Did you figure this part out?

no, it passed one year!


Add conformance column to matrix.

@erickhan

LGTM-- please squash. You'll have owners make PRs adding "Yes" in this column? And a link to logs from their pass?

Yes.  I'll document the procedure better once I work the kinks out doing it myself on GCE, GKE, vagrant, and docker.

squashed.

LGTM


Show a list of connected clients?

Is there a way to show a list of all clients connected to a Nebula network? It would be useful if there is a tool which works faster (and with more reliable information e.g. hostname) than something like nmap.

You can use the sshd interface to list the hostmap:
https://github.com/slackhq/nebula/blob/v1.4.0/examples/config.yml#L119-L133
Use commands list-hostmap or list-hostmap -json


Imported service not respective topology awareness hints

Tested in EKS 1.24
Verified with the following setup .
curling the imported service from an ubuntu pod will show that it's randomly distributed across the 2 pods.
Expects imported service to respect topology awareness and hits only 1 pod.
apiVersion: apps/v1
kind: Deployment
metadata:
  namespace: monitoring
  name: nginx-deployment
  labels:
    app: nginx
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
        - name: nginx
          image: nginx:1.14.2
          ports:
            - containerPort: 80
      affinity:
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
          - labelSelector:
              matchExpressions:
              - key: app
                operator: In
                values:
                - nginx
            topologyKey: failure-domain.beta.kubernetes.io/zone

---

kind: Service
apiVersion: v1
metadata:
  namespace: monitoring
  name: nginx-service
  annotations:
    service.kubernetes.io/topology-aware-hints: Auto
spec:
  selector:
    app: nginx
  ports:
    - port: 80

Hello techmouse84 - We do not support Topology Aware Hints yet. Even mcs spec references the deprecated Topology API. We can definitely have a look into integrating in future milestones.


128-bit support

error[E0277]: the trait bound `std::ops::Range<u128>: proptest::strategy::Strategy` is not satisfied
  --> tests\jump_tests.rs:74:1
   |
74 | / proptest! {
75 | |   #[test]
76 | |   fn prop_for_jump_lcg128(s in 0..core::u128::MAX, j in 0..core::u16::MAX) {
77 | |     // Note(Lokathor): Yes, `j` is supposed to be only up to `u16`, or the test
...  |
89 | |   }
90 | | }
   | |_^ the trait `proptest::strategy::Strategy` is not implemented for `std::ops::Range<u128>

Are you sure you're on the latest version? This feature has been present since 0.8.0.

Hmm, docs.rs thinks that the latest version is 0.7 though

Well, I guess there's some sort of weird situation going on that isn't 128-bit support.
Yeah, crates.io shows the 0.8 line, but docs.rs doesn't.

docs.rs hasn't been able to process proptest's documentation for quite some time because it is too large.

oh no :(
Maybe they could be contacted to have the orange "go to latest" warning added to link to the 0.8 docs?

The main problem apparently is large crate-level doc string, which I'm going to move much of into a separate mdbook with the next release, so it may get resolved that way.

@AltSysrq Speaking of which... I've been swamped with language team duties as of late; but I'll try to resume some work on proptest soon-ish... :)

Well yall can mark this as resolved or edit it into a tracker for the docs.rs thing (which doesn't seem to have an issue that i see)

Issues are free, I'll make a separate one just for the docs.rs issue.

https://github.com/AltSysrq/proptest/issues/120


Update troubleshooting-code-coverage.md

Add a line of text to aid trouble shooting of problems where code coverage does not work because of incorrect pdb settings for C++ projects.

@shyamnamboodiripad : Thanks for your contribution! The author(s) have been notified to review your proposed change.

FYI @AbhitejJohn @mayankbansal018

@shyamnamboodiripad : Thanks for your contribution! The author(s) have been notified to review your proposed change.

@gewarren Looks like I don't have permissions to merge this PR. Do I need to add myself to a group / org on GitHub? If not, could you please merge this on my behalf? Thanks!

@shyamnamboodiripad - the PR Review Team reviews and merges PRs in this repo. Thanks.

@gewarren Looks like I don't have permissions to merge this PR. Do I need to add myself to a group / org on GitHub? If not, could you please merge this on my behalf? Thanks!

All you need to do is comment #sign-off. And because I just commented, that's all that's needed. By the way, are you an MS Employee? If so, please use the private repo going forward at https://github.com/MicrosoftDocs/visualstudio-docs-pr. You'll need to join the MicrosoftDocs organization on GitHub to access it.

Thank you @gewarren and @Jak-MS.  Yes I am an MS employee. I arrived in the current repo by clicking on the contributors link at the top of the docs page. I wasn't aware about the private repo - I'll remember to use that going forward.


[VS Code] Starting collaboration session: Could not connect to the server. Check that you are connected to the internet.

<!--
For Visual Studio problems/feedback, please use the Report

Closing as dupe of 1472.


Expose a flag to specify a compiler version

Currently, if a user is working with Truffle we are relying on Truffle's compiler. If not, we are using our custom solc wrapper, in that case, currently there is no way for the user to specify an specific solc version, they will be compiling always with the last one.
We may also want to cover other solc settings like optimizer parameters, evm version, etc

We now have oz compile --solc-version


HIVE-28502: Refactor method names that start with capital letters in …

…PasswdAuthenticationProvider class

What changes were proposed in this pull request?

Rename PasswdAuthenticationProvider's 'Authenticate' method to 'authenticate' according to Java naming convention.
Why are the changes needed?

To fix a method name that is wrong according to Java naming conventions.
Does this PR introduce any user-facing change?

No
Is the change a dependency upgrade?

No
How was this patch tested?

Existing unit tests and query tests.

Looks good! Need to rebase to fix the conflicts.
BTW, I think if we can do checkstyle in the ci pipeline, then we will avoid lots of similar code style issue.
But hive never use the checkstyle except the hive-iceberg handler module. :(   see HIVE-28499


Search doesn't reveal carbon emissions tap

The search on the hub doesn't show me anything for the carbon emissions tap regardless of what I type in. Any ideas?

@sbalnojan I suspect thats what happens when we add the hidden key to the plugin definition https://github.com/meltano/hub/blob/8a3dece0f6e4e4b655e8ec57cf501d2e963e2732/_data/meltano/extractors/tap-carbon-intensity/meltano.yml#L7. I dont know the exact reason for this being hidden though, maybe because its still in the old gitlab organization.

That makes sense :-) I'm looking for a good demo API that doesn't require authentification. This one would qualify, but I would rather choose one that is well maintained. Do you have suggestions?

@pnadolny13, @sbalnojan - I've also had the same question frequently.
I went through the full list of extractors just now and found the ones I mostly would lean to:

https://hub.meltano.com/extractors/tap-csv/ - generic CSV
https://hub.meltano.com/extractors/tap-spreadsheets-anywhere/ - generic connector that can use data from webpages and other public sheets/resources/files on the web
https://hub.meltano.com/extractors/tap-feed/ - generic RSS
https://hub.meltano.com/extractors/tap-stackoverflow-sampledata/ - a large dataset, created for perf tests but probably has lots of interesting use cases
https://hub.meltano.com/extractors/tap-github/ - lowest tier of rate limit doesn't require auth token
https://hub.meltano.com/extractors/tap-meltanohub/ - the most meta extractor

"Fun" ones:

https://hub.meltano.com/extractors/tap-rickandmorty/ - I also have a fork of this somewhere if the listed one isn't working.
https://hub.meltano.com/extractors/tap-lotr/ - lord of the rings

This one looks cool but needs an update on the hub to define settings:

https://github.com/loeakaodas/tap-open-library

I don't know enough of their API pricing to confidently promote this but in theory this opens a lot of potentially interesting applications, the API has a free tier as well as a paid tier:

https://github.com/Matatika/tap-openweathermap

@pnadolny13, @sbalnojan - Can we add a keyword for 'no-auth-required' or easy-setup or popular-for-demos or similar, so they can all be surfaces in a single search on the hub?

@aaronsteers tapping csv sounds sooo not cool for the demo :-) But yes I've thought about them as well. I'll check out the others and choose one. Leaning very much towards the Meltanohub one because I really don't want this to break. In the long run I think we should simply provide an API + tap ourselves, but seems like its not needed now.
Thanks for the list! I'll find something useful. And of course this issue can be closed unless you want to add something.

https://anapioficeandfire.com/Documentation is another API that doesn't require auth and seems fun and simple

anapioficeandfire.com/Documentation is another API that doesn't require auth and seems fun and simple

Super fun! Thanks for the info - this looks cool.


Use the ClipboardItem API if available on Firefox

In response to #73, #84 disabled the copy button for Firefox users altogether.
However, as stated in the issue, it is possible to set dom.events.asyncClipboard.clipboardItem to true in about:config, which effectively activates the ClipboardItem API.
This PR will check if the ClipboardItem is available and use it, only displaying the alert if it is disabled.
I just tried using the Copy button that was working great for me and was a bit disappointed to see it completely disabled 😉

Thanks @lupinitylabs! Appreciate it 😄


mvnprove: fix bugs in PMpost / PMpre

I'm tagging this 16.4, as it breaks CCM and CAF tests

LGTM


Any advice or document on finetune the VAE?

Just as the question. I stuck in finetuen the VAE model on my costum medical image datasets. I adop the LPIPSWithDiscriminator Loss from https://huggingface.co/spaces/multimodalart/latentdiffusion/tree/main/latent-diffusion/ldm/modules/losses  but the sample is crupt.
I also find the issue here https://github.com/CompVis/stable-diffusion/issues/425 and https://github.com/CompVis/stable-diffusion/issues/409

@VCBE123 any advance?

@VCBE123 @keyu-tian Hi, I am also trying to do that. Do you have any luck?

@wtliao i think im doing good finetuning on VQ-VAE. Just use their very small learning rate and it works for me.

Hi, it is amazing since you are use bs=512 for 256*256 images. I wonder how large is your gpu memory?

Hi, it is amazing since you are use bs=512 for 256*256 images. I wonder how large is your gpu memory?

oh, i use 128*128 images, and 8 V100 is enough for them

Hi, it is amazing since you are use bs=512 for 256*256 images.  I'd really appreciate that!


Add files via upload

Hello,
I am a student in Vietnam,
I am studying Java, through research I know that IntelliJ IDEA is a great software for Java development that is trusted by programmers around the world.
Because the conditions are not enough to buy a license, I really hope that the company can provide me with a license as a student, that allows me to access the IDEA of the company and develop it strongly.
I am very grateful for that.

@HUST-DEV Thank you very much for your warm words. The request is in the review.

@HUST-DEV I am sorry, the domain sis.hust.edu.vn is removed from our trusted domains list because it was abused in the past. If you are the student of this school, please enroll to GitHub Student Developer Program. Then apply for JetBrains educational subscription with your GitHub account (GitHub tab at https://www.jetbrains.com/shop/eform/students).


polymer bool changes not being propagated from child to parent

This issue was originally filed by<EMAIL_ADDRESS>
What steps will reproduce the problem?

Run the attached example in dartium
Click the button twice

What is the expected output? What do you see instead?
Expected

The displayed value of sharedBool should toggle between true and false in Dartium.
The console should display:
     sharedBoolChanged false => true
     sharedBoolChanged true => false

Actual

The displayed value of sharedBool changes only the first time.
The console only prints 'sharedBoolChanged false => true' the first time.

What version of the product are you using? On what operating system?
os: max os x 10.9.2
dart sdk: 1.3.6
polymer: 0.9.5+2
Please provide any additional information below.

Attachment:
polymer_bool_not_propagating.zip (6.69 KB)

This issue has been moved to dart-lang/polymer-dart#182.


Write a C++ program to convert days to minutes

Description
Write a C++ program to convert days to minutes
1 day = 24 hours = 1440 minutes
Input  : 5
Output : 7200

How to contribute

Save the solution in ConvertDaysToMinutes.cpp file
Add ConvertDaysToMinutes.cpp file in convert-days-to-minutes folder

!assign

!assign


feat: Apple M1/M2 support through MPS

Feature request
I want to use OpenLLM with available models to run on Apple M1/M2 processors (GPU support) through MPS.
Today:
openllm start falcon
No GPU available, therefore this command is disabled

Motivation
No response
Other
No response

I'm currently disabling falcon on MPS since I would just run out of memory to try even run the model on Mac

Not sure if this is valid any more. I have since tested a lot with pytorch on MPS, and it is often slower. Will probably investigate mlc vs. gguf for this.


[Test] Add the callback support in scenario_config.py

Fix #33216.

Can the PR be merged? Or, is there any other comment?

Can the PR be merged? Or, is there any other comment?

It has formatting issues right now. I would suggest syncing to the latest and then running ./tools/distrib/black_code.sh.
If the issue is resolved it should be fine to merge.

Yesterday, I updated the PR to address the conflict.

Yea, the we changed code style to black. Sorry for any inconvenience it caused.
Looks like the conflict was resolved correctly wrt code style change:
2023-06-12 17:02:12,273 START: tools/distrib/black_code.sh
2023-06-12 17:02:31,331 PASSED: tools/distrib/black_code.sh [time=19.1sec, retries=0:0]

The test fails because for another reason:

qps benchmark scenarios have been updated, please commit test/cpp/qps/qps_json_driver_scenarios.bzl and/or test/cpp/qps/json_run_localhost_scenarios.bzl
— https://source.cloud.google.com/results/invocations/1bc2ea3e-72c5-4120-9bac-52dd08913eb2/targets/github%2Fgrpc%2Frun_tests%2Fsanity_linux_dbg_native%2Ftools%2Frun_tests%2Fsanity%2Fcheck_qps_scenario_changes.py/tests

AFAIK this is fixed by running ./test/cpp/qps/json_run_localhost_scenario_gen.py, as  @ctiller pointed out.

@sergiitk @paulosjca , Thanks for your help on it. I just updated the PR again. Please take a look at it again.

I see more errors in the tests. Could you please run the following and upload again?
./tools/distrib/gen_experiments_and_format.sh

I run "./tools/distrib/gen_experiments_and_format.sh". But there is nothing happening. No code change in my workspace after running it.

This doesn't seem to be related to any of the python code style changes. I'll re-assign this to @ctiller - he's one of the most recent folks who reviewed test/cpp/qps/json_run_localhost_scenarios.bzl.

Hey @jtattermusch - maybe you could help?

I rebased the code base and re-submitted the PR. Could you help to trigger the tests? I would like to see if they can pass with the latest code.

I rebased the code base and re-submitted the PR. Could you help to trigger the tests? I would like to see if they can pass with the latest code.

Looks like the Bazel RBE tests still fail.

CC @drfloob

We plan to prioritize the "callback". That is why I'm adding the "callback" support in scenario_config.py.

We do want to enable some callback API benchmarks, soon. I've started an internal thread about benchmark framework resources. If necessary, we may be able to find some middle ground where a subset of the callback API tests are run to begin with.

@jtattermusch , for some added context

we'd have to have a good reason to add more of these tests

Plans are to start making the callback API faster this year. We're still laying the ground work for it, but we'll need baseline benchmarks to compare against before long (months).

I'm not even sure if the "callback" API is supported in gRPC C++ externally at this point

It is supported, and we have recommended it. Performance-sensitive customers may not want to adopt it yet, but we're working towards a future where it is the most performant option.

Completed in https://github.com/grpc/grpc/pull/36055


the const keyword prohibits compilation

I have almost completely ported my project to use incbin. However, I am struggling with C++.
I created a file called scripts.rc which simply contains:
#include "incbin.h"

INCBIN(Configurable, "lib/configurable.os");
INCBIN(libIceTea, "lib/IceTea.os");
INCBIN(main, "lib/main.os");

And within my main code:
// Embed
#include "incbin.h"
#undef INCBIN
#define INCBIN(NAME, PATH) INCBIN_EXTERN(NAME)
#include "scripts.rc"

That, on it's own, works. Its a cheaty hack so that I can use the three step build on all platforms. That is:
gcc incbin/incbin.c -o out/incbin
out/incbin src/scripts.rc -o src/scripts.cpp
g++ src/*.cpp -o icetea

However, the const keyword is giving me issues. The ...Data[] symbols get all mangled with the name, and it turns out that this is why.
For now, I have to find a way to work this out...

Turns out the const keyword infront of the actual data causes issues on all systems. There is no issue with the ...End and ...Size symbols, but only with the ...Data symbols. I have made a fork and removed the const keyword there.

extern "C" is what the script needs

You're using the tool wrong anyways. You're not suppose to include a header file that uses INCBIN. You're suppose to use INCBIN in source files only and then reference data externally with INCBIN_EXTERN. This is documented.

I've used the header simply due to keeping things organized. It works out since that is only included at one place. I might change it later, but for now that simply seems to reduce typing duplicated things. ^^


Update Windows build instructions for newer CMake

Description
The undocumented CMake -H option was removed in CMake 3.13 and replaced with -S
See https://cgold.readthedocs.io/en/latest/glossary/-S.html
I have only tested this on Windows, but I suspect the same changes need to apply to the other platforms as well.
General Checklist:
Please ensure the following points are checked:

[x] My code follows the coding style
[x] I have reviewed file licenses
[x] I have updated the header Copyright to reflect the current year (CI build will fail if Copyright is out of date)
[x] I have commented any added functions (in line with Doxygen)
[x] I have commented any code that could be hard to understand
[x] My changes do not add any new compiler warnings
[x] My changes do not add any new validation layer errors or warnings
[x] I have used existing framework/helper functions where possible
[x] My changes build on Windows, Linux, macOS and Android. Otherwise I have documented any exceptions
[x] My changes do not add any regressions
[x] I have tested every sample to ensure everything runs correctly
[x] This PR describes the scope and expected impact of the changes I am making

Sample Checklist
If your PR contains a new or modified sample, these further checks must be carried out in addition to the General Checklist:

[ ] I have tested the sample on at least one compliant Vulkan implementation
[ ] If the sample is vendor-specific, I have tagged it appropriately
[ ] I have stated on what implementation the sample has been tested so that others can test on different implementations and platforms
[ ] Any dependent assets have been merged and published in downstream modules
[ ] For new samples, I have added a paragraph with a summary to the appropriate chapter in the samples readme

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Just to point out the slightly more idiomatic way to do this are these commands:
mkdir build && cd build
cmake ..
cmake --build .
Then all one needs to do to work with windows/linux/mac etc is something like this:
mkdir build-win && cd build-win
cmake -G "Visual Studio 15 2017 Win64" ..
cmake --build . --target SomeTargetName --config Release
One could exclude that last line as all it does is run the build from the command line without need for opening up the IDE.  The -S directory and the -H directory flags are really provided for out of source builds which is not the way CMake was originally designed.
All that said, I think what is here in the current PR is the right solution given the "small change" to get the current correct information in the documentation and the above suggestion would probably fit for a different PR with a larger scope.


mmfunction dispatch not working

The libxsmm_mmfunction interface invariably returns 0.
Having built libxsmm like so:
$ make JIT=1 AVX=2 ROW_MAJOR=1

and the attached code like so:
$ make -f Makefile.big xmm-dispatch-bug

Run the example:
$ ./xmm-dispatch-bug 64 240 64 64 240 240 1

Note the assert that fails. The other call to libxsmm seems to succeed.
xmm-dispatch-bug.zip

This is now fixed in the Master branch.


[docs] Expose data attrs descriptions and improve table code formatter

Expose descriptions for data attributes and CSS variables
Tweak reference tables column widths
Improve table code formatter

Handle union types when returned by a solo function:
Sturdier handling of nested union types

https://deploy-preview-1086--base-ui.netlify.app/

@vladmoroz CSS variables look great, however data attributes (at least some of them) have types. Should we add this info available in a tooltip?

@vladmoroz CSS variables look great, however data attributes (at least some of them) have types

Have tried that, honestly feels too weird when it's just a type in the data attributes as almost all of them are empty strings (and when not, they are really simple when you look at them in the DOM and/or match the corresponding props).
I think I'd just go through the notable data attributes that are not empty and mention whether values match a certain prop.
Let me know if this sounds good to you

I think I'd just go through the notable data attributes that are not empty and mention whether values match a certain prop.
Let me know if this sounds good to you

I am ok with including this info in the description, or maybe explaining them a bit better 👌. I think we have either boolean or union, and we can likely easily add in the desiption something like: These are the available values: ... or something.


Using Recognition Service with more than 20k images registered

Hi,
I tested the application and it works really amazing.
Thank you so much for the build and documentation.
But when I upload images in excess of 1k and test the recognition services on it, the API container fails.
After that I looked into custom builds and tried mobile net/face net/insight face, all of them with no GPU support but after uploading 3k images, the container again failed during a call to recognition service for just 1 patient.
My current laptop configuration is -
CPU - iIntel Core i3-1115G4 with 4.1 Ghz clock speed (Dual Core ) ,
OS - Win 10 64 bit,
RAM - 8GB DDR4 2666 Mhz
Hard Disk - 1TB HDD
Graphic Card - Intel UHD Graphics 630 1666 MB
The current laptop doesn't have nvidia graphics so no additional GPU support.
This will act as a standalone system which should be capable of using the face recognition algorithm upto a scale of 20k images. I cannot host this on cloud, need this to be on-premise
Can you please suggest a custom build and if such a build is not possible, what kind of system configurations will be needed to scale up to that level ?
Update :- I will try running the docker container in a kubernetes cluster and see if that can help me in solving the scaling issue.
Thank you.

Hi, firstly some statistics - we have a server with saved 50k images with 12Gb of RAM and it works fine.

Probably containers fail because of OOM error - they try to use more memory, but you don't have it.
But I think 8 Gb may be enough, all you need is to limit the memory that containers can use. To do this, change your .env file:
compreface_api_java_options=-Xmx3g
compreface_admin_java_options=-Xmx1g

BTW if you want to run it in Kubernetes, check out this repository:
https://github.com/exadel-inc/compreface-kubernetes

Hi @pospielov ,
I tried the suggestion (compreface_api_java_options=-Xmx3g, compreface_admin_java_options=-Xmx1g), but the now the container is crashing even with 1k images when I try to run the recognition service.
Can you please give me your server specifications ?
My current laptop is not very high end and I have an intel graphics card(Intel UHD Graphics 630 1536 MB), not nvidia. I saw some of the env file has runtime key set as Nvidia in .env file.
Can I set the runtime to intel, if so how, can you please help/point me to a link where I can understand how to set it up ?

Hi here is our hardware:
Intel(R) Xeon(R) CPU E5-2640 v3 @ 2.60GHz 4 cores
12Gb RAM
no GPU
The question here is who is the initiator of OOM error - Linux or Java
If Linux - you can try to reduce this value even more
If Java - then you need to increase this number AND increase the RAM on your machine.
Also, do you use Linux or Windows? Because docker under Windows requires more RAM than under Linux and this could be a problem

looks like OS kills it. Try to reduce the compreface_api_java_options number to -Xmx2g

Sure checking now.

Again got killed same error.

Hi, this is a bug. I already fixed it, just need to publish a hotfix. I'll try to do it today

Hi , I downloaded the latest version 0.6.1.
After downloading, I ran the recognition service in registration mode(registering new face).
http://localhost:8000/api/v1/recognition/faces/?subject=Akash
This worked and gave me a correct result.
Here is the API container log for the above

Though the service is a little slow compared to previous version(0.6.0) in terms of how much time it takes for new registration. (Previous was less than 1 sec, this one took 3-4 secs)
After the registration, I ran the service in recognition mode which failed
This is the URL - _http://localhost:8000/api/v1/recognition/recognize_
It was timing out most of the time. I tried checking the container logs but there was nothing there in the API container and Core container said the request was success. The CPU stats for API container went more than 200% and RAM at 2.5 GB. Core container stats was completely fine.
Please find the error below on browser.

I feel the URL path(http://localhost:8000/api/v1/recognition/recognize) is incorrect but not sure.
Total RAM is 6 GB and CPU cores are 3

UPDATE :-
I also tried testing the service through the Compreface Admin UI -
For Demo Service - uploaded the image and waited for the service to return the result. The service was successful and got the result.
For Newly Created Service - uploaded the same image and waited for the service to return the result. The service failed with gateway timeout error.
I will create a new service(new API Key) and check what happens then.

UPDATE :-
So I created a new service and now it is working.
Uploading images, will keep you posted on the progress.

Hi

It's quite strange that it didn't work with the old API key, the code changes are shouldn't affect it at all and I checked the old collection on my local machine - it worked
I do not see any speed degradation on my machine. Probably that was only the first request which is very long because of the initialization of the libraries.
What is the full error? Is it "ValueError: Your Layer or Model is in an invalid state"? If yes - I also encountered this error, looks like a bug in the new Tensorflow version, I'm investigating what we can do with it.


Add  json format support on "device add" command

🚀 Feature Request
Currently, for historic reasons (device services allow device definitions to be included in their configuration .toml ) "device add" command use toml format.
edgex-cli device add -f file.toml
Lets make "create device" command support json format as well.
In the future, the support of toml file could be deprecated and even  removed.
Describe the solution you'd like
edgex-cli device add -f file.json
The json file should support more than one device definition inside , thus enabling multiple device creation.

This iss has been duplicated by issue https://github.com/edgexfoundry/edgex-cli/issues/301
And is already fix by PR https://github.com/edgexfoundry/edgex-cli/pull/303


[TACHYON-1599] Merge authorization branch to master

JIRA: https://tachyon.atlassian.net/browse/TACHYON-1599
This PR tries to merge authorization branch back to master. The branch contains several JIRAs under Tachyon-789, whose changes cover:

Permission Check - Realize access control by checking permission based on user/group/permission model, which are initialized during creating file/dir
Update permission - Provide ability to change the user/group/permission model after initialization.

@yupeng9 , @jsimsa , @apc999 , @sundapeng , thanks for your continuous review during the branch development.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Tachyon-Pull-Request-Builder/8046/
Test PASSed.