codegenerated action models + improvements in some crocoddyl code

This PR is a mix of some changes in crocoddyl to improve the efficiency, and the addition of code-generation.
In addition, I've added some benchmarks, and unittests to compare multi-threading and code-generated performances.
The efficiency part is not yet finished, and I'm working on improving the code still.
@wxmerkt : I've changed assert_pretty to be ineffective depending on NDEBUG. I have some benchmarks to support this choice: calculation time of calcdiff for pyrene-robot is increased by 20us because of the asserts. By comparison, the new contact-dynamics that we are planning to introduce in crocoddyl takes about 40us to calculate the derivatives. This change would concern you directly based on the discussion that we had on a previous PR.

This PR needs to be merged after https://github.com/stack-of-tasks/pinocchio/pull/1156 has been merged, and probably also released by @jcarpent

@proyan we will need a release too. I will review the code later. Just do not forget to add the new minimum version dependency.

@proyan Can you please add that benchmark to this PR? I'd like to have it as a test or similar

@wxmerkt I think the best solution might be to simply try the benchmark bipedal-timings with and without the commit 56346bc6ec539b594915f28ce7c35cdb7321a635.
 It is not that difficult to see the reason, since the assertions often compare a high dimensional matrix with another. That is quite costly.

Thanks for the details on how to reproduce. I can look into this on Monday.
Two things from a cursory look right now:

Maybe let's use a different define flag that isn't NDEBUG. NDEBUG is set by Release mode for instance, and I think this performance improvement can be differentiated. However, see (2) for why we can likely continue with NDEBUG as you suggest.
I looked into where assert_pretty is used as I was surprised when you said "it compares matrices". Then I realised there are two use cases: (a) to compare entire matrices, (b) to check matrix dimensions. I think you're spot on that we should have a way to disable (a) because that can slow things down and is unnecessary in Release mode. Where assert_pretty is used for (b) though, where (b) can be caused by user-input error, we should never disable it by a flag such as NDEBUG - perhaps in those very few cases move it to throw_pretty. Then, assert_pretty can be deactivated when NDEBUG is defined as suggested.

In summary: I don't have an objection to disable assert_pretty when in Release mode or when needing performance - this really is a "Debug"/development check. My worry in general is more to regards where we use throw_pretty: Those checks are very cheap and should not be turned off.
To do this we should fix the few cases where assert_pretty is used, when throw_pretty should have been used. From a quick look these are:

include/crocoddyl/core/numdiff/diff-action.hxx, L21
include/crocoddyl/multibody/contact-base.hxx, L30, L33
include/crocoddyl/multibody/impulse-base.hxx, L25

I think you are right, it would be useful to do runtime dimension checks, they are quite useful. Would you have a little time to simply push the required changes to this PR?
On April 18, 2020 10:06:39 PM GMT+02:00, Wolfgang Merkt<EMAIL_ADDRESS>wrote:

Thanks for the details on how to reproduce. I can look into this on
Monday.
Two things from a cursory look right now:

Maybe let's use a different define flag that isn't NDEBUG.
NDEBUG is set by Release mode for instance, and I think this
performance improvement can be differentiated. However, see (2) for why
we can likely continue with NDEBUG as you suggest.
I looked into where assert_pretty is used as I was surprised when
you said "it compares matrices". Then I realised there are two use
cases: (a) to compare entire matrices, (b) to check matrix dimensions.
I think you're spot on that we should have a way to disable (a) because
that can slow things down and is unnecessary in Release mode. Where
assert_pretty is used for (b) though, where (b) can be caused by
user-input error, we should never disable it by a flag such as NDEBUG

perhaps in those very few cases move it to throw_pretty. Then,
assert_pretty can be deactivated when NDEBUG is defined as
suggested.

In summary: I don't have an objection to disable assert_pretty when
in Release mode or when needing performance - this really is a
"Debug"/development check. My worry in general is more to regards where
we use throw_pretty: Those checks are very cheap and should not be
turned off.
To do this we should fix the few cases where assert_pretty is used,
when throw_pretty should have been used. From a quick look these are:

include/crocoddyl/core/numdiff/diff-action.hxx, L21
include/crocoddyl/multibody/contact-base.hxx, L30, L33
include/crocoddyl/multibody/impulse-base.hxx, L25

--
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
https://github.com/loco-3d/crocoddyl/pull/758#issuecomment-615937116

I think you are right, it would be useful to do runtime dimension checks, they are quite useful. Would you have a little time to simply push the required changes to this PR?

Sure, I can do this Monday late morning

I can do this Monday late morning
Great. Thanks Wolfgang!

@proyan as you might know from my comments, I have to do few things in this PR.
Are you planning to make other changes? I would like to work on it when you are done.

I am about to push the fixes to assert_pretty and caught a few errors where it wouldn't have worked before.

@proyan @cmastalli I am done with the required changes and pushed to this branch (but didnt update the copyrights). Note, there is also a case where the equal was wrong but it wasn't caught by the unit tests. This must be because we aren't testing expected-failure-cases or this class wasn't under tests?

@proyan did you finish with this work? Shall I work on the NTHREAD task?
Another thing, do you think we should add in the README file the information about CppAD and CppADCodegen dependency (as optional one and via Pinocchio)?

@cmastalli If you have time to contribute to this PR, perhaps you could please take care of the changes to python bindings with the new JintegrateTransport signature?

@cmastalli If you have time to contribute to this PR, perhaps you could please take care of the changes to python bindings with the new JintegrateTransport signature?

It sounds good, just let me know when you stop pushing the branch.

@cmastalli I've added a preliminary python bindings to make the examples work. However, there are issues with unittests/bindings.
The CI currently has pinocchio 2.3.1. And I think that is the only reason it is passing.
Since pinocchio 2.4.0, some change in pinocchio is making the unittests/bindings fail. I'm not sure which ones.
Specifically, the switch switchToNumpyMatrix doesn't seem to be working anymore. As a result, the python implementation inside the unittests/bindings and bindings/python/crocoddyl is complaining about array and matrix sizes/conversions. And the unittests/bindings is giving "cannot convert array to matrix" errors.
Do you have time to look at this issue in a different PR?

Specifically, the switch switchToNumpyMatrix doesn't seem to be working anymore. As a result, the python implementation inside the unittests/bindings and bindings/python/crocoddyl is complaining about array and matrix sizes/conversions. And the unittests/bindings is giving "cannot convert array to matrix" errors.

This is not handle by Crocoddyl. The issue should be somewhere else (in the bindings itself). I encountered myself these problems, but I have an idea which could be the issue.

Do you have time to look at this issue in a different PR?

I will fix the issues with unittest/bindings in this PR. It is not a good policy to rush and introduce problems in the library. Please be a bit patience, unfortunately. I have other commitments now.
I expect to fix this issue before Tuesday, I will try to handle it today (if I can).
If this prevents you to keep working on other topics, then just create a new branch from here and later you rebase it.
Thank you for this great effort. Soon I will finish the last piece :)

I will fix the issues with unittest/bindings in this PR

What I mean is, this is an issue which is unrelated to this PR, and can be handled separately. It is already too long, and it is hard to keep track of all the merge commits here. If you handle it separately, I can rebase on the crocoddyl/devel afterwards.

What I mean is, this is an issue which is unrelated to this PR, and can be handled separately. It is already too long, and it is hard to keep track of all the merge commits here. If you handle it separately, I can rebase on the crocoddyl/devel afterwards.

The issue is related with this PR, there were not problems before. I suspect it's related with the new functions in the state class.
I will check it myself, and I promise you that I will merge the PR if this issue doesn't belong to this PR.

I just to want to provide a development remark on this PR: I would not recommend mixing Python bindings development and core development in the same PR that implies very new features such as the current one.
Indeed, it slows down the testing of the new feature which does not require Python bindings and certainly, a new change of the core features will appear and will require a new pass on the bindings.

Thank you for your feedback.  Our bottleneck is the releasing of Pinocchio v2.4.3.
Note that it was unavoidable to work with the Python bindings since changes in the StateAbstractTpl.

@cmastalli Pinocchio 2.4.3 is released. You should the conda approach to test it, and you should not wait for robotpkg, which is not done for fast development.

@cmastalli Pinocchio 2.4.3 is released. You should the conda approach to test it, and you should not wait for robotpkg, which is not done for fast development.

I didn't know about it. Does conda install c++ files too?

Yes, this is the main advantage of conda with respect to Pip for instance.

Why the renaming of the existing benchmarks to -optctrl? I understand the suffix for the new codegen, but the optctrl suffix makes no difference for the existing files (it's more unclear than before?).

Why the renaming of the existing benchmarks to -optctrl? I understand the suffix for the new codegen, but the optctrl suffix makes no difference for the existing files (it's more unclear than before?).

Before this PR, all the benchmark were solving an optimal control problem (optctrl). However, @proyan has added benchmarks

purely regarding codegen, and they do not solve an optimal control (XXX-codegen), and
purely regarding single computations for an arm and biped problems (e.g. a quadratic activation model, and so on).

Okay, thanks for the explanation :-).

The CI fails because some example-robot-data meshes cannot be found
15/55 Test #15: pybinds-states ................................***Failed    0.19 sec

Traceback (most recent call last):

  File "/home/travis/build/loco-3d/crocoddyl/unittest/bindings/test_states.py", line 95, in <module>

    class StateMultibodyTalosArmTest(StateAbstractTestCase):

  File "/home/travis/build/loco-3d/crocoddyl/unittest/bindings/test_states.py", line 96, in StateMultibodyTalosArmTest

    MODEL = example_robot_data.loadTalosArm().model

  File "/opt/openrobots/lib/python2.7/site-packages/example_robot_data/robots_loader.py", line 81, in loadTalosArm

    robot = RobotWrapper.BuildFromURDF(modelPath + URDF_SUBPATH, [modelPath])

  File "/opt/openrobots/lib/python2.7/site-packages/pinocchio/robot_wrapper.py", line 17, in BuildFromURDF

    robot.initFromURDF(filename, package_dirs, root_joint, verbose, meshLoader)

  File "/opt/openrobots/lib/python2.7/site-packages/pinocchio/robot_wrapper.py", line 21, in initFromURDF

    model, collision_model, visual_model = buildModelsFromUrdf(filename, package_dirs, root_joint, verbose, meshLoader)

  File "/opt/openrobots/lib/python2.7/site-packages/pinocchio/shortcuts.py", line 45, in buildModelsFromUrdf

    geom_model = pin.buildGeomFromUrdf(model, filename, geometry_type, package_dirs)

ValueError: Mesh package://example-robot-data/robots/talos_data/meshes/arm/arm_1_collision.STL could not be found.

      Start 16: pybinds-actuations

16/55 Test #16: pybinds-actuations ............................***Failed    0.15 sec

Traceback (most recent call last):

  File "/home/travis/build/loco-3d/crocoddyl/unittest/bindings/test_actuations.py", line 41, in <module>

    class TalosArmFullActuationTest(ActuationModelAbstractTestCase):

  File "/home/travis/build/loco-3d/crocoddyl/unittest/bindings/test_actuations.py", line 42, in TalosArmFullActuationTest

    STATE = crocoddyl.StateMultibody(example_robot_data.loadTalosArm().model)

  File "/opt/openrobots/lib/python2.7/site-packages/example_robot_data/robots_loader.py", line 81, in loadTalosArm

    robot = RobotWrapper.BuildFromURDF(modelPath + URDF_SUBPATH, [modelPath])

  File "/opt/openrobots/lib/python2.7/site-packages/pinocchio/robot_wrapper.py", line 17, in BuildFromURDF

    robot.initFromURDF(filename, package_dirs, root_joint, verbose, meshLoader)

  File "/opt/openrobots/lib/python2.7/site-packages/pinocchio/robot_wrapper.py", line 21, in initFromURDF

    model, collision_model, visual_model = buildModelsFromUrdf(filename, package_dirs, root_joint, verbose, meshLoader)

  File "/opt/openrobots/lib/python2.7/site-packages/pinocchio/shortcuts.py", line 45, in buildModelsFromUrdf

    geom_model = pin.buildGeomFromUrdf(model, filename, geometry_type, package_dirs)

ValueError: Mesh package://example-robot-data/robots/talos_data/meshes/arm/arm_1_collision.STL could not be found.

      Start 17: pybinds-actions

17/55 Test #17: pybinds-actions ...............................***Failed    0.15 sec

Traceback (most recent call last):

  File "/home/travis/build/loco-3d/crocoddyl/unittest/bindings/test_actions.py", line 87, in <module>

    class TalosArmFreeFwdDynamicsTest(ActionModelAbstractTestCase):

  File "/home/travis/build/loco-3d/crocoddyl/unittest/bindings/test_actions.py", line 88, in TalosArmFreeFwdDynamicsTest

    ROBOT_MODEL = example_robot_data.loadTalosArm().model

  File "/opt/openrobots/lib/python2.7/site-packages/example_robot_data/robots_loader.py", line 81, in loadTalosArm

    robot = RobotWrapper.BuildFromURDF(modelPath + URDF_SUBPATH, [modelPath])

  File "/opt/openrobots/lib/python2.7/site-packages/pinocchio/robot_wrapper.py", line 17, in BuildFromURDF

    robot.initFromURDF(filename, package_dirs, root_joint, verbose, meshLoader)

  File "/opt/openrobots/lib/python2.7/site-packages/pinocchio/robot_wrapper.py", line 21, in initFromURDF

    model, collision_model, visual_model = buildModelsFromUrdf(filename, package_dirs, root_joint, verbose, meshLoader)

  File "/opt/openrobots/lib/python2.7/site-packages/pinocchio/shortcuts.py", line 45, in buildModelsFromUrdf

    geom_model = pin.buildGeomFromUrdf(model, filename, geometry_type, package_dirs)

ValueError: Mesh package://example-robot-data/robots/talos_data/meshes/arm/arm_1_collision.STL could not be found.

      Start 18: pybinds-shooting

18/55 Test #18: pybinds-shooting ..............................***Failed    0.15 sec

Traceback (most recent call last):

  File "/home/travis/build/loco-3d/crocoddyl/unittest/bindings/test_shooting.py", line 72, in <module>

    class TalosArmShootingTest(ShootingProblemTestCase):

  File "/home/travis/build/loco-3d/crocoddyl/unittest/bindings/test_shooting.py", line 73, in TalosArmShootingTest

    ROBOT_MODEL = example_robot_data.loadTalosArm().model

  File "/opt/openrobots/lib/python2.7/site-packages/example_robot_data/robots_loader.py", line 81, in loadTalosArm

    robot = RobotWrapper.BuildFromURDF(modelPath + URDF_SUBPATH, [modelPath])

  File "/opt/openrobots/lib/python2.7/site-packages/pinocchio/robot_wrapper.py", line 17, in BuildFromURDF

    robot.initFromURDF(filename, package_dirs, root_joint, verbose, meshLoader)

  File "/opt/openrobots/lib/python2.7/site-packages/pinocchio/robot_wrapper.py", line 21, in initFromURDF

    model, collision_model, visual_model = buildModelsFromUrdf(filename, package_dirs, root_joint, verbose, meshLoader)

  File "/opt/openrobots/lib/python2.7/site-packages/pinocchio/shortcuts.py", line 45, in buildModelsFromUrdf

    geom_model = pin.buildGeomFromUrdf(model, filename, geometry_type, package_dirs)

ValueError: Mesh package://example-robot-data/robots/talos_data/meshes/arm/arm_1_collision.STL could not be found.

      Start 19: pybinds-solvers

19/55 Test #19: pybinds-solvers ...............................***Failed    0.15 sec

Traceback (most recent call last):

  File "/home/travis/build/loco-3d/crocoddyl/unittest/bindings/test_solvers.py", line 113, in <module>

    class TalosArmDDPTest(SolverAbstractTestCase):

  File "/home/travis/build/loco-3d/crocoddyl/unittest/bindings/test_solvers.py", line 114, in TalosArmDDPTest

    ROBOT_MODEL = example_robot_data.loadTalosArm().model

  File "/opt/openrobots/lib/python2.7/site-packages/example_robot_data/robots_loader.py", line 81, in loadTalosArm

    robot = RobotWrapper.BuildFromURDF(modelPath + URDF_SUBPATH, [modelPath])

  File "/opt/openrobots/lib/python2.7/site-packages/pinocchio/robot_wrapper.py", line 17, in BuildFromURDF

    robot.initFromURDF(filename, package_dirs, root_joint, verbose, meshLoader)

  File "/opt/openrobots/lib/python2.7/site-packages/pinocchio/robot_wrapper.py", line 21, in initFromURDF

    model, collision_model, visual_model = buildModelsFromUrdf(filename, package_dirs, root_joint, verbose, meshLoader)

  File "/opt/openrobots/lib/python2.7/site-packages/pinocchio/shortcuts.py", line 45, in buildModelsFromUrdf

    geom_model = pin.buildGeomFromUrdf(model, filename, geometry_type, package_dirs)

ValueError: Mesh package://example-robot-data/robots/talos_data/meshes/arm/arm_1_collision.STL could not be found.

      Start 20: pybinds-costs

20/55 Test #20: pybinds-costs .................................***Failed    0.15 sec

Traceback (most recent call last):

  File "/home/travis/build/loco-3d/crocoddyl/unittest/bindings/test_costs.py", line 134, in <module>

    class StateCostTest(CostModelAbstractTestCase):

  File "/home/travis/build/loco-3d/crocoddyl/unittest/bindings/test_costs.py", line 135, in StateCostTest

    ROBOT_MODEL = example_robot_data.loadICub().model

  File "/opt/openrobots/lib/python2.7/site-packages/example_robot_data/robots_loader.py", line 248, in loadICub

    robot = RobotWrapper.BuildFromURDF(modelPath + URDF_SUBPATH, [modelPath], pinocchio.JointModelFreeFlyer())

  File "/opt/openrobots/lib/python2.7/site-packages/pinocchio/robot_wrapper.py", line 17, in BuildFromURDF

    robot.initFromURDF(filename, package_dirs, root_joint, verbose, meshLoader)

  File "/opt/openrobots/lib/python2.7/site-packages/pinocchio/robot_wrapper.py", line 21, in initFromURDF

    model, collision_model, visual_model = buildModelsFromUrdf(filename, package_dirs, root_joint, verbose, meshLoader)

  File "/opt/openrobots/lib/python2.7/site-packages/pinocchio/shortcuts.py", line 45, in buildModelsFromUrdf

    geom_model = pin.buildGeomFromUrdf(model, filename, geometry_type, package_dirs)

ValueError: Mesh package://example-robot-data/robots/icub_description/meshes/upmc/collision/icub_simple_collision_root_link.dae could not be found.

      Start 21: pybinds-contacts

21/55 Test #21: pybinds-contacts ..............................***Failed    0.15 sec

Traceback (most recent call last):

  File "/home/travis/build/loco-3d/crocoddyl/unittest/bindings/test_contacts.py", line 109, in <module>

    class Contact3DTest(ContactModelAbstractTestCase):

  File "/home/travis/build/loco-3d/crocoddyl/unittest/bindings/test_contacts.py", line 110, in Contact3DTest

    ROBOT_MODEL = example_robot_data.loadHyQ().model

  File "/opt/openrobots/lib/python2.7/site-packages/example_robot_data/robots_loader.py", line 173, in loadHyQ

    robot = RobotWrapper.BuildFromURDF(modelPath + URDF_SUBPATH, [modelPath], pinocchio.JointModelFreeFlyer())

  File "/opt/openrobots/lib/python2.7/site-packages/pinocchio/robot_wrapper.py", line 17, in BuildFromURDF

    robot.initFromURDF(filename, package_dirs, root_joint, verbose, meshLoader)

  File "/opt/openrobots/lib/python2.7/site-packages/pinocchio/robot_wrapper.py", line 21, in initFromURDF

    model, collision_model, visual_model = buildModelsFromUrdf(filename, package_dirs, root_joint, verbose, meshLoader)

  File "/opt/openrobots/lib/python2.7/site-packages/pinocchio/shortcuts.py", line 45, in buildModelsFromUrdf

    geom_model = pin.buildGeomFromUrdf(model, filename, geometry_type, package_dirs)

ValueError: Mesh package://example-robot-data/robots/hyq_description/meshes/trunk/trunk.dae could not be found.

      Start 22: pybinds-impulses

22/55 Test #22: pybinds-impulses ..............................***Failed    0.15 sec

Traceback (most recent call last):

  File "/home/travis/build/loco-3d/crocoddyl/unittest/bindings/test_impulses.py", line 107, in <module>

    class Impulse3DTest(ImpulseModelAbstractTestCase):

  File "/home/travis/build/loco-3d/crocoddyl/unittest/bindings/test_impulses.py", line 108, in Impulse3DTest

    ROBOT_MODEL = example_robot_data.loadHyQ().model

  File "/opt/openrobots/lib/python2.7/site-packages/example_robot_data/robots_loader.py", line 173, in loadHyQ

    robot = RobotWrapper.BuildFromURDF(modelPath + URDF_SUBPATH, [modelPath], pinocchio.JointModelFreeFlyer())

  File "/opt/openrobots/lib/python2.7/site-packages/pinocchio/robot_wrapper.py", line 17, in BuildFromURDF

    robot.initFromURDF(filename, package_dirs, root_joint, verbose, meshLoader)

  File "/opt/openrobots/lib/python2.7/site-packages/pinocchio/robot_wrapper.py", line 21, in initFromURDF

    model, collision_model, visual_model = buildModelsFromUrdf(filename, package_dirs, root_joint, verbose, meshLoader)

  File "/opt/openrobots/lib/python2.7/site-packages/pinocchio/shortcuts.py", line 45, in buildModelsFromUrdf

    geom_model = pin.buildGeomFromUrdf(model, filename, geometry_type, package_dirs)

ValueError: Mesh package://example-robot-data/robots/hyq_description/meshes/trunk/trunk.dae could not be found.

      Start 23: pybinds-force_cost

23/55 Test #23: pybinds-force_cost ............................***Failed    0.14 sec

Traceback (most recent call last):

  File "/home/travis/build/loco-3d/crocoddyl/unittest/bindings/test_force_cost.py", line 9, in <module>

    ROBOT_MODEL = example_robot_data.loadICub().model

  File "/opt/openrobots/lib/python2.7/site-packages/example_robot_data/robots_loader.py", line 248, in loadICub

    robot = RobotWrapper.BuildFromURDF(modelPath + URDF_SUBPATH, [modelPath], pinocchio.JointModelFreeFlyer())

  File "/opt/openrobots/lib/python2.7/site-packages/pinocchio/robot_wrapper.py", line 17, in BuildFromURDF

    robot.initFromURDF(filename, package_dirs, root_joint, verbose, meshLoader)

  File "/opt/openrobots/lib/python2.7/site-packages/pinocchio/robot_wrapper.py", line 21, in initFromURDF

    model, collision_model, visual_model = buildModelsFromUrdf(filename, package_dirs, root_joint, verbose, meshLoader)

  File "/opt/openrobots/lib/python2.7/site-packages/pinocchio/shortcuts.py", line 45, in buildModelsFromUrdf

    geom_model = pin.buildGeomFromUrdf(model, filename, geometry_type, package_dirs)

ValueError: Mesh package://example-robot-data/robots/icub_description/meshes/upmc/collision/icub_simple_collision_root_link.dae could not be found.

      Start 24: pybinds-friction_cone_cost

24/55 Test #24: pybinds-friction_cone_cost ....................***Failed    0.14 sec

Traceback (most recent call last):

  File "/home/travis/build/loco-3d/crocoddyl/unittest/bindings/test_friction_cone_cost.py", line 11, in <module>

    ROBOT_MODEL = example_robot_data.loadICub().model

  File "/opt/openrobots/lib/python2.7/site-packages/example_robot_data/robots_loader.py", line 248, in loadICub

    robot = RobotWrapper.BuildFromURDF(modelPath + URDF_SUBPATH, [modelPath], pinocchio.JointModelFreeFlyer())

  File "/opt/openrobots/lib/python2.7/site-packages/pinocchio/robot_wrapper.py", line 17, in BuildFromURDF

    robot.initFromURDF(filename, package_dirs, root_joint, verbose, meshLoader)

  File "/opt/openrobots/lib/python2.7/site-packages/pinocchio/robot_wrapper.py", line 21, in initFromURDF

    model, collision_model, visual_model = buildModelsFromUrdf(filename, package_dirs, root_joint, verbose, meshLoader)

  File "/opt/openrobots/lib/python2.7/site-packages/pinocchio/shortcuts.py", line 45, in buildModelsFromUrdf

    geom_model = pin.buildGeomFromUrdf(model, filename, geometry_type, package_dirs)

ValueError: Mesh package://example-robot-data/robots/icub_description/meshes/upmc/collision/icub_simple_collision_root_link.dae could not be found.

      Start 25: pybinds-momemtum_cost

25/55 Test #25: pybinds-momemtum_cost .........................***Failed    0.15 sec

Traceback (most recent call last):

  File "/home/travis/build/loco-3d/crocoddyl/unittest/bindings/test_momemtum_cost.py", line 18, in <module>

    robot = loadANYmal()

  File "/opt/openrobots/lib/python2.7/site-packages/example_robot_data/robots_loader.py", line 66, in loadANYmal

    robot = RobotWrapper.BuildFromURDF(modelPath + URDF_SUBPATH, [modelPath], pinocchio.JointModelFreeFlyer())

  File "/opt/openrobots/lib/python2.7/site-packages/pinocchio/robot_wrapper.py", line 17, in BuildFromURDF

    robot.initFromURDF(filename, package_dirs, root_joint, verbose, meshLoader)

  File "/opt/openrobots/lib/python2.7/site-packages/pinocchio/robot_wrapper.py", line 21, in initFromURDF

    model, collision_model, visual_model = buildModelsFromUrdf(filename, package_dirs, root_joint, verbose, meshLoader)

  File "/opt/openrobots/lib/python2.7/site-packages/pinocchio/shortcuts.py", line 45, in buildModelsFromUrdf

    geom_model = pin.buildGeomFromUrdf(model, filename, geometry_type, package_dirs)

ValueError: Mesh package://example-robot-data/robots/anymal_b_simple_description/meshes/base/anymal_base.dae could not be found.

      Start 26: pybinds-impulse_com_cost

26/55 Test #26: pybinds-impulse_com_cost ......................***Failed    0.14 sec

Traceback (most recent call last):

  File "/home/travis/build/loco-3d/crocoddyl/unittest/bindings/test_impulse_com_cost.py", line 10, in <module>

    ROBOT_MODEL = example_robot_data.loadICub().model

  File "/opt/openrobots/lib/python2.7/site-packages/example_robot_data/robots_loader.py", line 248, in loadICub

    robot = RobotWrapper.BuildFromURDF(modelPath + URDF_SUBPATH, [modelPath], pinocchio.JointModelFreeFlyer())

  File "/opt/openrobots/lib/python2.7/site-packages/pinocchio/robot_wrapper.py", line 17, in BuildFromURDF

    robot.initFromURDF(filename, package_dirs, root_joint, verbose, meshLoader)

  File "/opt/openrobots/lib/python2.7/site-packages/pinocchio/robot_wrapper.py", line 21, in initFromURDF

    model, collision_model, visual_model = buildModelsFromUrdf(filename, package_dirs, root_joint, verbose, meshLoader)

  File "/opt/openrobots/lib/python2.7/site-packages/pinocchio/shortcuts.py", line 45, in buildModelsFromUrdf

    geom_model = pin.buildGeomFromUrdf(model, filename, geometry_type, package_dirs)

ValueError: Mesh package://example-robot-data/robots/icub_description/meshes/upmc/collision/icub_simple_collision_root_link.dae could not be found.

@proyan I am very exciting with the great work that you bring with this PR.
Are you planning to codegen Fx^T Vx' Fx? or Box-QP? Is this make sense to do it?
We could split the effort :)

@proyan I think you may have to add export ROS_PACKAGE_PATH=/opt/openrobots/share to travis

The CI fails because some example-robot-data meshes cannot be found

There is a recent release in the example-robot-data, it seems this is the problem. In that release we change the path of the meshes to matches ROS standards. You need to define properly the ROS_PACKAGE_PATH as explained here: https://github.com/Gepetto/example-robot-data
Could you update Travis in this PR?

Thanks @cmastalli for the review


Update balena.yml to conform to current balena push requirement.

Following the discussion regarding the version key here and referring to the balena.yml from balena-dash, I've omitted the version key in the contract.
Testing done:

Verified the the build/release is reported successful on the terminal.
Verified for a Fin, that the version on the web dashboard matches the output of balena push <my-app>.

@tmigone, @phil-d-wilson Can one of you please take a look? TIA!

Thanks @tmigone!


fix: useMeasure top/left/y/x values were always zero

Description

Fix for this issue
x, y, top and left are always 0 inside contentRect
Replace the usage of ResizeObserverEntry.contentRect, and retrive values from ResizeObserverEntry.target.getBoundingClientRect()
https://developer.mozilla.org/en-US/docs/Web/API/ResizeObserverEntry
Type of change

[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as before)

Checklist

[x] Read the Contributing Guide
[x] Perform a code self-review
[ ] Comment the code, particularly in hard-to-understand areas
[ ] Add documentation
[ ] Add hook's story at Storybook
[x] Cover changes with tests
[x] Ensure the test suite passes (yarn test)
[x] Provide 100% tests coverage
[ ] Make sure code lints (yarn lint). Fix it with yarn lint:fix in case of failure.
[x] Make sure types are fine (yarn lint:types).

Any chance to get this merged? Facing the same issue

Any news yet on when this will be merged, please? It's been over a year since the PR was submitted.

@tomasdisk It doesn't look promising, however, it looks like there's been activity within the last month or so, if you look at the commit history.

Any news? I think this is more helpful than contentRect @streamich

I confirmed locally that the fix in this PR works. Could you consider merging it, @streamich? I’m willing to help where I can, if any help is welcomed.

Any news about this PR ?  I think it would be really helpful @streamich

@nleborgne you may be interested to take a look at the better-maintained (yet awkwardly-named) https://github.com/react-hookz/web library. This one seems to be ignored by its author.

Thanks, I gave a quick look at the repo, it seems like the useMeasure hook only returns the width and height in the Measure object. Is there any way to access properties like top and left like in the react-use useMeasure hook?


npm install wechaty-openapi<EMAIL_ADDRESS>

🐛 Bug Report
(A clear and concise description of what the bug is.)
npm WARN deprecated<EMAIL_ADDRESS>This library will not receive further updates other than security fixes. We recommend using @grpc/grpc-js instead.
(##################) ⠋ reify:grpc: WARN deprecated<EMAIL_ADDRESS>This library will not receive further updates other than security fixes. We recommend usi
nodejs版本：16.16.0
系统：mac, centos都存在该问题

Thanks for the reporting.
I'd love to accept a PR to fix it.
Or you can wait for me to continue working on it in the following months.
And, you are welcome to join our Discord for future OpenAPI discussion at https://discord.gg/UZNtDNsrKv
Happy New Year!


crazy shows raw values as hex

The raw values are shown in hex for the values (dec is more human readable ;-)
Ex. wear leveling count shows as 107 which is 263 decimal which is the value we are looking for.
crazy output on /dev/sdj:
                                                                                | Status  ID AttributeName                Current Worst Threshold   Raw Values |
                                                                                | Good    05 reallocated-sector-count         100   100        10<PHONE_NUMBER>00 |
                                                                                | Good    09 power-on-hours                    99    99         0 000000000E25 |
                                                                                | Good    0C power-cycle-count                 99    99         0<PHONE_NUMBER>08 |
                                                                                | Good    B1 wear-leveling-count               87    87         0<PHONE_NUMBER>07 |
                                                                                | Good    B3 used-reserved-blocks-total       100   100        10<PHONE_NUMBER>00 |
                                                                                | Good    B5 program-fail-count-total         100   100        10<PHONE_NUMBER>00 |
                                                                                | Good    B6 erase-fail-count-total           100   100        10<PHONE_NUMBER>00 |
                                                                                | Good    B7 runtime-bad-block-total          100   100        10<PHONE_NUMBER>00 |
                                                                                | Good    BB reported-uncorrect               100   100         0<PHONE_NUMBER>00 |
                                                                                | Good    BE airflow-temperature-celsius       68    57         0<PHONE_NUMBER>20 |
                                                                                | Good    C3 hardware-ecc-recovered           200   200         0<PHONE_NUMBER>00 |
                                                                                | Good    C7 udma-crc-error-count             100   100         0<PHONE_NUMBER>00 |
                                                                                | Good    EB good-block-rate                   99    99         0<PHONE_NUMBER>01 |
                                                                                | Good    F1 total-lbas-written                99    99         0 000034D4CFFD |
                                                                                +------------------------------------------------------------------------------+

# smartctl -A /dev/sdj
smartctl 6.2 2013-07-26 r3841 [x86_64-linux-3.13.0-96-generic] (local build)
Copyright (C) 2002-13, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF READ SMART DATA SECTION ===
SMART Attributes Data Structure revision number: 1
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  5 Reallocated_Sector_Ct   0x0033   100   100   010    Pre-fail  Always       -       0
  9 Power_On_Hours          0x0032   099   099   000    Old_age   Always       -       3621
 12 Power_Cycle_Count       0x0032   099   099   000    Old_age   Always       -       8
177 Wear_Leveling_Count     0x0013   087   087   000    Pre-fail  Always       -       263
179 Used_Rsvd_Blk_Cnt_Tot   0x0013   100   100   010    Pre-fail  Always       -       0
181 Program_Fail_Cnt_Total  0x0032   100   100   010    Old_age   Always       -       0
182 Erase_Fail_Count_Total  0x0032   100   100   010    Old_age   Always       -       0
183 Runtime_Bad_Block       0x0013   100   100   010    Pre-fail  Always       -       0
187 Reported_Uncorrect      0x0032   100   100   000    Old_age   Always       -       0
190 Airflow_Temperature_Cel 0x0032   068   057   000    Old_age   Always       -       32
195 Hardware_ECC_Recovered  0x001a   200   200   000    Old_age   Always       -       0
199 UDMA_CRC_Error_Count    0x003e   100   100   000    Old_age   Always       -       0
235 Unknown_Attribute       0x0012   099   099   000    Old_age   Always       -       1
241 Total_LBAs_Written      0x0032   099   099   000    Old_age   Always       -<PHONE_NUMBER>54

I made this change in my fork.
RAW_VALUES_DEC CMake option set to ON will build with Raw Value as Decimal

With packaging in Linux distros in mind, #19 should probably have been a runtime option rather than a compile switch.

I just learned that you can press key d to switch between decimal and hex display, see:
https://github.com/otakuto/crazydiskinfo/blob/803a58571b286f4ff8ce9d77cd5655a98ec0abaa/main.cpp#L556-L561
Can this bug be closed?


Remove unnecessary project files

We don't need these for the standalone

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (a0999e7) 93.69% compared to head (cdec430) 93.69%.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@            Coverage Diff            @@
##             master       #8   +/-   ##
=========================================
  Coverage     93.69%   93.69%
  Complexity       22       22
=========================================
  Files             5        5
  Lines           111      111
=========================================
  Hits            104      104
  Misses            7        7

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


53 | import annotations from future to tests pass under Python < 3.9

Using from __future__ import annotations so that type hints work under older versions of Python.

Codecov Report

Merging #72 (dccef71) into main (df3105c) will increase coverage by 0.03%.
The diff coverage is 100.00%.

:exclamation: Current head dccef71 differs from pull request most recent head 325d9ca. Consider uploading reports for the commit 325d9ca to get more accurate results

@@            Coverage Diff             @@
##             main      #72      +/-   ##
==========================================
+ Coverage   82.87%   82.91%   +0.03%
==========================================
  Files          30       30
  Lines        3358     3365       +7
==========================================
+ Hits         2783     2790       +7
  Misses        575      575

Impacted Files
Coverage Δ

clarity/enhancer/compressor.py
100.00% <100.00%> (ø)

clarity/data/HOA_tools_cec2.py
96.85% <0.00%> (+0.06%)
:arrow_up:

clarity/data/scene_renderer_cec2.py
91.61% <0.00%> (+0.16%)
:arrow_up:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

Very strange, I've now seen the following error message under Python 3.8, Python 3.9 and Python 3.10 but on other runs the tests have passed without any problem its one of the other versions that fails. I don't get this error locally on a fresh 3.10 or 3.7 install with all packages/dependencies installed anew.
tests/regression/test_predictors.py F.                                   [100%]

=================================== FAILURES ===================================
_____________________________ test_torch_msbg_stoi _____________________________

regression test output differences for tests/regression/test_predictors.py::test_torch_msbg_stoi:

>   --- current
>   +++ tobe
>   @@ -1,2 +1,2 @@
>   -Torch MSBG STOILoss -0.46198, ESTOILoss -0.32999
>   +Torch MSBG STOILoss -0.46198, ESTOILoss -0.33000
>

Its clearly a precision issue, and manually re-running the failed test it passed ok, but highlighting as it may crop up again in the future.


Be able to modify "stepSize" and "count" options for axis in green-chart

We found out that there is an issue with green-chart that we want to use. As you can see in pic below, to many ticks shows in y-axis for this diagram. That is why we think that a good solution to this problem is that we be able to set the y-axis stepsize of a chart in chartsettings.

Todos:
A suggestion of how solution looks like:
There are two options in billboardjs which is very good to have: "stepSize" and "count".

The "count" option will automatically choose which values needs to show on y-axis but the number of ticks
will be equal with "count" value.

The "stepSize" sets axis tick step(interval) size. NOTE: stepSize will be ignored if axis.y.tick.count is
option is set.
How do this in billboardjs:

axis: {
   y: {
     tick: {
    count: 5 //or stepSize:1000
     }
   }
 },

How green-charts ChartSettings may look like:
export interface ChartStyle {
    fitHeightToParent?: boolean;
    axis?: {
        show: {
            x?: boolean;
            y?: boolean;
            y2?: boolean;
        } | boolean;
    };
    point?: {
        show?: boolean | 'focus';
    };
**ticks?:{
            y?: {
             count?: number
             stepSize?:number
            };
            y2?: {
             count?: number
             stepSize?:number
            };
}**
}

Hello, I've added ticksCount and stepSize options to the ChartStyle interface in green-charts.
You can set it in the style options of the settings object, for example:
style: {
  axis: {
    y: {
      label: 'Axis label',
      stepSize: 50,
      (or ticksCount: xxx)
    }
  }
}

Will this work for your use case?
For more details, check out #584

Hi @splashdust!
Sorry for late response. Thank you so much, yes this will work for us. When do you release this? I can not see these changes yet.
BR
--Parham


tablechop deletes files which currently used by Cassandra

tablechop checks only the last modified date from the index_key file to decide if the backuped file will be deleted:
    for index_key in index_keys:
        if days_ago(index_key.last_modified) > args.age:
            break
        index_files_to_keep.add(index_key.name)

For my understanding, tablechop should only delete this files which no longer necessary for restoring the backup in the specified retention time. Depending from the table size and the compaction strategy some SSTables can live weeks or months without being compacted. These files are mandatory to restore the table, regardless of whether they have been in S3 for weeks or months. Currently active SSTables should not be deleted (unless uses a force parameter).
Meanwhile there can be exists small backuped SSTables, which have been compacted and are no longer required for restoring.
I'd be grateful if we can add could an additional verification whether the file is currently being used by cassandra/exists on the filesystem.

@Mortinke The index file (-listdir.json) contains the list of all sstable files that were present when that particular sstable was being uploaded. So the index file has a snapshot with the complete list of files to restore to a state when that sstable was created. Tablechop loops through the files in the index file(s), and ensures these are not deleted.
Check files_to_keep here : https://github.com/JeremyGrosser/tablesnap/blob/master/tablechop#L87
That said there is a race here since multiple sstables can be uploaded in parallel, so if the node fails some of these latest -listdir.json might not be valid, since the file may not have been uploaded. This is a problem as of now, @JeremyGrosser is this a known issue?


feat(caip): use nominal types for string literals

TL;DR: right now we use string-type aliases, e.g. export type AccountId = string, but they don't actually do anything as they are widened back to a string before type-checking by the compiler.
By nominally typing our string types we actually get the benefits of types.

A proof of concept demonstrating the use of nominal typing on AssetId, AccountId, and ChainId.
These are currently structurally typed as strings, which is not useful as they aren't checked by the compiler.
An example is outlined below:
interface Flavoring<FlavorT> {
  _type?: FlavorT
}
export type Nominal<T, FlavorT> = T & Flavoring<FlavorT>

type StructuralChainId = string
type NominalChainId = Nominal<string, 'chainId'>

const notAChainId = '1234'

const acceptsStructuralChainIdArg = (chainId: StructuralChainId) => chainId
const acceptsNominalChainIdArg = (chainId: NominalChainId) => chainId

acceptsStructuralChainIdArg(notAChainId) // 😞 No type error! We might miss this...
acceptsNominalChainIdArg(notAChainId) // 😁 Argument of type 'string' is not assignable to parameter of type 'NominalChainId'

A primer on the approach used, and why it's useful: https://spin.atomicobject.com/2018/01/15/typescript-flexible-nominal-typing/

BREAKING CHANGE: changes AssetId, AccountId, and ChainId types

Is this overkill on types?

I'm uncomfortable with the idea of strongly typing chainId. It means that adding any level of support for another chain can't just be done in a plug-in. The global types have to be updated first.

I don't think that's the case - a nominal type has no concept of which ChainIds are supported, only that the strings that are supported are ChainIds

Is this overkill on types?

I don't think so. We use types for a reason and currently, we type strings in a way that does not work, in that they are not enforced by the compiler.
This PR makes string-type aliases actually work instead of them being widened back to a string before type-checking.

closed by https://github.com/shapeshift/lib/pull/774


highlight test codelines in stacktraces

I'm not sure if this is hard or easy, but here goes:
It would be great if there was a way to get the standard reporter to highlight the lines in the stacktrace that match a simple regexp.
This would be a great tool to quickly find the offending line when looking through a stacktrace of 40 - 50 lines and looking for the one line where you invoke a function - the rest of the lines are often just noise..

I'm gonna close this as I think the end goal was achieved in https://github.com/mochajs/mocha/pull/1564
For example
$ cat example.js
it('test', function(done) {
  done(new Error('Something went wrong!'));
});

$ mocha example.js

  ․

  0 passing (5ms)
  1 failing

  1)  test:
     Error: Something went wrong!
      at Context.<anonymous> (example.js:2:8)

Neat, right :)


ENH: Multivariate t-distribution

Reference issue
Closes https://github.com/scipy/scipy/issues/10042. Duplicates https://github.com/scipy/scipy/pull/11413 because the old branch was out-of-date.
What does this implement/fix?
Adds support for the multivariate t-distribution, particularly pdf, logpdf, and rvs. See my blog post for a high-level discussion of my approach.
Additional information
Completed:

[x] No licensing issues.
[x] Unit tests comparing logpdf and pdf to MATLAB's mvtpdf function; unit tests for arguments, reproducibility (with random_state), and some testable mathematical properties.
[x] All tests pass locally.
[x] All public functions have docstrings and examples.
[x] Code style follows existing content in _multivariate.py.
[x] Docstrings with new functionality.
[x] Added myself to THANKS.txt
[ ] Added a documentation page via the docstring. The make html command no longer works for me. I'm happy to build and test docs if someone can advise. I get the error:

installed scipy 487487dd30 != current repo git version 'f22d40692d'
use "make dist" or "GITVER=487487dd30 make html ..."

Would appreciate feedback on:

How to unit test rvs?
Should we support cdf and logcdf given that the multivariate t-distribution does not have a closed-form solution for the CDF? While I have seen some implementations (see https://github.com/scipy/scipy/issues/10042#issuecomment-576806054 in particular), I don't yet fully understand how they work and would prefer adding these methods to be a separate PR.
Should we support entropy? I don't know anything about the differential entropy of the multivariate t-distribution, and would also prefer punting that to a future PR.

My tests pass locally, but the CI tests fail for a reason I don't understand, e.g.:
loc = rng.random(3)
E   AttributeError: 'mtrand.RandomState' object has no attribute 'random'

@gwgundersen Thanks for submitting this. Other opinions may differ, but I suspect it's OK to start without cdf and entropy in this PR and add later, since not all multivariate distributions have all the methods we might like.
I'm not sure if this will help, but try git submodule update --init before make html. You can also see the documentation built by CI: click "Details".

It looks like it's not showing up in the documentation. I think you need to add your distribution to scipy/stats/__init__.py:
Multivariate distributions
==========================
.. autosummary::
   :toctree: generated/
   multivariate_normal   -- Multivariate normal distribution
   matrix_normal         -- Matrix normal distribution
   dirichlet             -- Dirichlet
   wishart               -- Wishart
   invwishart            -- Inverse Wishart
   multinomial           -- Multinomial distribution
   special_ortho_group   -- SO(N) group
   ortho_group           -- O(N) group
   unitary_group         -- U(N) group
   random_correlation    -- random correlation matrices

I'm not sure about the failing test or how to test the RVS; perhaps @WarrenWeckesser has thoughts?

@mdhaber, thanks for that suggestion. I tried initializing and updating the git submodule, but it did not seem to work. Is it possible I need to rebuild Scipy from scratch to update the version number based on the commit? I've tried to do that and, oddly, am completely failing to do so—I was able to build Scipy previously, and I don't know if it's a change in the build procedure or something local to me. I wrote more details here.

@gwgundersen yes, it sounds to me like you need to build SciPy again, too. I'm not sure how to fix your build error, though. Sometimes my approach has been to nuke my environment and start from scratch, but I see that you've written up your issue, so hopefully you'll get a more helpful reply. I've also seen this issue for macs lately, but it looks different from yours.

@mdhaber and @gwgundersen I did some testing of this PR locally to see if I could get the docstrings to build.
I couldn't set my local changes to track this pr for some reason (not sure why) so I posted the changes here:
https://github.com/rlucas7/scipy/commit/56c9d2738b0038af6f4a9ab1d46976aa8fab574d
it is mostly spacing around sections of docstrings (e.g. before example line, etc).
The examples render locally for my html and latex doc builds.
There may be an extra newline character here or there but that should give you an idea of what's left for docstring changes.
Let me know if I can help further to get this PR to merging, seems close and just needs a few more small changes.

@rlucas7 Do you want me to add those to this PR?

@mdhaber and @rlucas7, thanks for keeping this fire alive. I should have time to look at this again later in the week. Also, I have access to a new machine, which may help with my issues building the docs. I hope that building the docs correctly and pushing will fix some of the failing tests.

@mdhaber wrote:

@rlucas7 Do you want me to add those to this PR?

If @gwgundersen is ok with that then yes.
Would be nice to get the doc build to green so we can focus on the scientific/probabilistic aspects of the PR and move this one forward.

@mdhaber and @rlucas7, thanks for keeping this fire alive. I should have time to look at this again later in the week. Also, I have access to a new machine, which may help with my issues building the docs. I hope that building the docs correctly and pushing will fix some of the failing tests.

No worries @gwgundersen please let us know if you run into issues w/dev setup.

OK I went ahead and pushed @rlucas7's changes to @gwgundersen's branch so we can see how they affect the doc rendering.
@gwgundersen you'll need to pull the changes e.g. git pull origin/10042-multivariate-t-again before continuing work.

I noticed something weird with the shapes, not sure the cause but behavior is inconsistent:
>>> import numpy as np
>>> import scipy.stats as ss
>>> loc = np.zeros(4)
>>> shape = np.eye(4)
>>> df = 1
>>> shape[2, 2] = 0
>>> ss.multivariate_t(loc, shape, df, allow_singular=True).pdf(np.zeros(4))
array([0.07599089])
>>> ss.multivariate_t(loc, shape, np.inf, allow_singular=True).pdf(np.zeros(4))
0.06349363593424098

Ah, good catch. When using np.inf, we delegate to multivariate_normal, which uses _squeeze_output, converting values like [[2]] to 2. I've added _squeeze_output to multivariate_t's _logpdf function and rvs functions. I've added a new unit test test_shape_correctness:
def test_shape_correctness(self):
    # pdf and logpdf should return scalar when the
    # number of samples in x is one.
    dim = 4
    loc = np.zeros(dim)
    shape = np.eye(dim)
    df = 1
    x = np.zeros(dim)
    res = multivariate_t(loc, shape, df).pdf(x)
    assert np.isscalar(res)
    res = multivariate_t(loc, shape, df).logpdf(x)
    assert np.isscalar(res)

    # pdf() and logpdf() should return probabilities of shape
    # (n_samples,) when x has n_samples.
    n_samples = 7
    x = np.random.random((n_samples, dim))
    res = multivariate_t(loc, shape, df).pdf(x)
    assert (res.shape == (n_samples,))
    res = multivariate_t(loc, shape, df).logpdf(x)
    assert (res.shape == (n_samples,))

    # rvs() should return scalar unless a size argument is applied.
    res = multivariate_t(np.zeros(1), np.eye(1), 1).rvs()
    assert np.isscalar(res)

    # rvs() should return vector of shape (size,) if size argument
    # is applied.
    size = 7
    res = multivariate_t(1, 1, 1).rvs(size=size)
    assert (res.shape == (size,))

@rlucas7 and @mdhaber, thanks for all your help and feedback. I think I've addressed all your comments, but please let me know if I missed anything.

@rlucas7, okay done. Thanks again.

@gwgundersen it looks like there are some changes with checks on an upgraded version of numpy to the failing CI builds. It looks like these are failing on other branches too and they don’t appear related to this PR. Let’s hold off for now and I’ll check back in a couple days (tomorrow is a busy work day).

@gwgundersen it looks like @WarrenWeckesser put a fix (now in master) that should make these failing builds pass:
https://github.com/scipy/scipy/pull/12902
do you feel up for a rebase of this branch on master?
If not I can do it over the weekend and ping you to confirm after I force push to the branch.
Either way is fine w/me but wanted to see which you prefer.

@rlucas7, I think it's safer if you do the rebase. I should learn, but maybe not here. Thanks for asking.

@rlucas7, I think it's safer if you do the rebase. I should learn, but maybe not here. Thanks for asking.

I am not able to push to the branch from the git terminal client because the PR is under your repository. The authentication mechanism is different for the git hub web browser than git itself. I should've known this, git is not github. Anyway, you'll need to perform the rebase.
If you don't have upstream setup on your local repo already, do this:
https://docs.github.com/en/free-pro-team@latest/github/collaborating-with-issues-and-pull-requests/configuring-a-remote-for-a-fork
then assuming that is setup on your local copy, try following these instructions:
https://www.digitalocean.com/community/tutorials/how-to-rebase-and-update-a-pull-request
note you will run into 2 rebase conflicts, both are because of THANKS.txt conflicts.
Three things worth noting:
(1) Be sure your master branch is up to date with the upstream master before doing the rebase.
(2) After you've completed the rebase you will see a message like:
(scipy-dev) Lucass-MacBook:scipy rlucas$ git status
On branch test
Your branch and 'origin/10042-multivariate-t-again' have diverged,
and have 195 and 39 different commits each, respectively.
  (use "git pull" to merge the remote branch into yours)

this is expected, and you do not want to perform a git pull. At that point confirm your local branch builds and passes tests by python runtests.py -v and assuming everything passes, then do a git push -f assuming you have the 10042-multivariate-t-again tracking the origin remote branch.
(3) Before you begin the rebase process create a local branch that is checked out on the HEAD of the existing state of 10042-multivariate-t-again this way you have a local copy if something breaks.

Also, @gwgundersen when I rebased your copy locally and reran the tests everything passes.

Thanks for the detailed instructions. Should I be rebasing from a specific stable commit on master? I ask because after following them, my tests do not all pass. Specifically, I see:
=================================================== short test summary info ====================================================
FAILED scipy/linalg/tests/test_decomp_cossin.py::test_cossin[True-40-12-20-float32] - AssertionError:
FAILED scipy/linalg/tests/test_decomp_cossin.py::test_cossin[True-100-50-50-float32] - AssertionError:
FAILED scipy/linalg/tests/test_decomp_cossin.py::test_cossin[False-40-12-20-float32] - AssertionError:
FAILED scipy/linalg/tests/test_decomp_cossin.py::test_cossin[False-100-50-50-float32] - AssertionError:
============== 4 failed, 29652 passed, 2030 skipped, 10990 deselected, 76 xfailed, 6 xpassed in 441.03s (0:07:21) ==============

I also confirmed that these tests fail on master. I even created a new directory from a fresh git clone ..., built Scipy, and ran tests; and those tests still fail.

Resolved two merge conflicts in THANKS.txt.

Re-built Scipy (not sure if this is necessary but seems like it should not hurt):

python3 setup.py build_ext --inplace

Ran all tests:

python runtests.py -v

IIUC you still need to complete the rebase by doing
git rebase -i master
On your branch.
The merge-base gives a MRCA if the two branches and rebase on that commit puts your commits on top of the MRCA commit.  Your branch commits are stacked on top of the tip (HEAD) commit from master (but on your local branch).
Try the command above and once done with rebase see if you still get the test fail. IIRC that test was failing but was fixed on master. So that’s why I think the commits aren’t all the way up to tip.

On Oct 4, 2020, at 10:26 PM, Gregory Gundersen<EMAIL_ADDRESS>wrote:

Thanks for the detailed instructions. Should I be rebasing from a specific stable commit on master? I ask because after following them, my tests do not all pass. Specifically, I see:
=================================================== short test summary info ====================================================
FAILED scipy/linalg/tests/test_decomp_cossin.py::test_cossin[True-40-12-20-float32] - AssertionError:
FAILED scipy/linalg/tests/test_decomp_cossin.py::test_cossin[True-100-50-50-float32] - AssertionError:
FAILED scipy/linalg/tests/test_decomp_cossin.py::test_cossin[False-40-12-20-float32] - AssertionError:
FAILED scipy/linalg/tests/test_decomp_cossin.py::test_cossin[False-100-50-50-float32] - AssertionError:
============== 4 failed, 29652 passed, 2030 skipped, 10990 deselected, 76 xfailed, 6 xpassed in 441.03s (0:07:21) ==============
I also confirmed that these tests fail on master. I even created a new directory from a fresh git clone ..., built Scipy, and ran tests; and those tests still fail.

Resolved two merge conflicts in THANKS.txt.
Re-built Scipy (not sure if this is necessary but seems like it should not hurt):
python3 setup.py build_ext --inplace
Ran all tests:
python runtests.py -v
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.

I think I understand.  That said, even after adding the second rebase (git rebase -i master), the unit tests do not all pass.
And again, if I re-clone the Scipy repo, build, and run tests on master, the same tests fail.

@rlucas, what if I just do a git push --force with my rebased branch, despite these failing tests? I suspect the issue has to do with my machine or OS (Ubuntu 20.04.1), since:

The unit tests pass for you.
The code and tests appear to rely on LAPACK.
The same unit tests fail when using a newly cloned version of master.
The same unit tests fail when using my current branch (not rebased).

Alternatively, I could ask the developers at https://github.com/scipy/scipy/pull/11524 for help, since that PR introduced the unit tests that are failing for me.

Also, the failing tests "smell" like they may be caused by differing mathematical or algorithmic conventions, e.g.:
>>> np.isclose(u1_2, u1)
array([[False,  True, False, ..., False, False, False],
       [False,  True, False, ..., False, False, False],
       [False,  True, False, ..., False, False, False],
       ...,
       [False, False, False, ..., False, False, False],
       [False,  True, False, ..., False, False, False],
       [False,  True, False, ..., False, False, False]])
>>> u1_2[0,0]
0.058479194
>>> u1[0,0]
-0.058478586
>>> u1_2[0,2]
0.040936727
>>> u1[0,2]
-0.040936757

Try
python runtests.py -v -s stats
That will run only the stats tests. If that passes for you locally then yeah I think a force push is fine.
Also, can you leave a comment on the other PR with the failing test stack trace?
If you’re seeing that fail others are or will too.
I run a Mac box so if this is specific to Ubuntu (Not sure if it is) I can’t help much.
-Lucas Roberts

On Oct 5, 2020, at 5:21 PM, Gregory Gundersen<EMAIL_ADDRESS>wrote:

Also, the failing tests "smell" like they may be caused by differing mathematical or algorithmic conventions, e.g.:

np.isclose(u1_2, u1)
array([[False,  True, False, ..., False, False, False],
[False,  True, False, ..., False, False, False],
[False,  True, False, ..., False, False, False],
...,
[False, False, False, ..., False, False, False],
[False,  True, False, ..., False, False, False],
[False,  True, False, ..., False, False, False]])
u1_2[0,0]
0.058479194
u1[0,0]
-0.058478586
u1_2[0,2]
0.040936727
u1[0,2]
-0.040936757
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.

Just to make sure that this is not a sign problem (since the results are unique up to columns signs), could you please check if this turns out all true
>>> np.allclose(np.abs(u1_2), np.abs(u1), rtol=0., atol=np.finfo(np.float32).eps*100)

@rlucas7, I rebased, confirmed the stats tests pass, and force pushed. All checks pass via GitHub, and I didn't bring in any additional commits.

@ilayn, that command returns False:
> /home/gwg/projects/scipy/build/testenv/lib/python3.7/site-packages/scipy/linalg/tests/test_decomp_cossin.py(151)test_cossin_separate()
-> assert_allclose(u1_2, u1, rtol=0., atol=10*np.finfo(dtype_).eps)
(Pdb) np.allclose(np.abs(u1_2), np.abs(u1), rtol=0., atol=np.finfo(np.float32).eps*100)
False

Also, despite my linalg tests failing locally, the CI tests all pass.

That can happen also due to an outdated blas/lapack distro (mkl or openblas etc.) Because apparently not only signs are wrong but the results are also different so it is an actual miscalculation.

That can happen also due to an outdated blas/lapack distro (mkl or openblas etc.) Because apparently not only signs are wrong but the results are also different so it is an actual miscalculation.

Thanks for commenting on this @ilayn I hadn't think about BLAS version issues.
So that we know in case of future questions, it would be great @gwgundersen if you could let us know which BLAS and version you have running.

@ilayn, I'm not sure which BLAS and LAPACK versions are being used. This is what I tried when trying to find out:
>>> import numpy.distutils.system_info as sysinfo

>>> sysinfo.get_info('lapack')
{'libraries': ['lapack', 'lapack'],
 'library_dirs': ['/usr/lib/x86_64-linux-gnu'],
 'language': 'f77'}

>>> sysinfo.get_info('blas')
{'libraries': ['blas', 'blas'],
 'library_dirs': ['/usr/lib/x86_64-linux-gnu'],
 'include_dirs': ['/usr/local/include',
  '/usr/include',
  '/home/gwg/miniconda3/envs/scipy/include'],
 'language': 'c',
 'define_macros': [('HAVE_CBLAS', None)]}

When I ls in those directories, I see multiple shared object files:
/usr/lib/x86_64-linux-gnu/lapack$ ls
liblapack.a  liblapack.so  liblapack.so.3  liblapack.so.3.9.0

/usr/lib/x86_64-linux-gnu/openblas-pthread$ ls
cmake       libblas.so.3  liblapack.so.3         libopenblasp-r0.3.8.so  pkgconfig
libblas.a   liblapack.a   libopenblas.a          libopenblas.so
libblas.so  liblapack.so  libopenblasp-r0.3.8.a  libopenblas.so.0

I also tried to re-build Scipy from scratch pointing to the latest object files:
export LAPACK=/usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.9.0
export BLAS=/usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.8.so
python setup.py install

But the tests still fail. Happy to try something else if you have advice, but I can't seem to find any info online as to the actual BLAS or LAPACK versions being used.

Squashed and merged, nice work @gwgundersen

Thanks @gwgundersen, @rlucas!

Hello @gwgundersen ,
I find the code in the following line has a problem with Lint check and cause failure in continuous-integration/travis-ci/pr
I guess it's because of space before "="
https://github.com/scipy/scipy/blob/882849f81e4eda89008261b691206c5ae7795d26/scipy/stats/_multivariate.py#L4083
it produces the following error
36.65s$ pycodestyle scipy benchmarks/benchmarks
scipy/stats/_multivariate.py:4083:12: E221 multiple spaces before operator
1       E221 multiple spaces before operator
The command "pycodestyle scipy benchmarks/benchmarks" exited with 1.

@hp5588 I'm seeing that now, I've got a PR to fix it.
ref:
https://github.com/scipy/scipy/pull/12938


Any way to force re-calculation of formulas?

Having an issue where formulas are not being automatically recalculated on some versions of Excel and LibreOffice. I cannot guarantee that clients will be using the correct version of an app and was hoping to get OpenXLSX to recalculate formulas that are saved from a template XLSX file.
I am using the library to open an existing XLSX Template file that has a number of formulas saved, reading data from different sheets. I use the library to write data to some sheets and save the output. When a client opens the resulting file, the data is not recalculated.
I've tried reapplying the formula after writing the sheets but that does not solve the issue. Is there a way I can get this done with the library?

Looks like this can be resolved by assigning an empty string value to the formula cells. Closing the issue.

If anyone reading this is encountering this issue, check this PR: https://github.com/troldal/OpenXLSX/pull/80


TextureRegistry.SurfaceProducer.Callback class Callback not found

Steps to reproduce
/Users/divyamverma/.pub-cache/hosted/pub.dev/camera_android_camerax-0.6.8+2/android/src/main/java/io/flutter/plugins/camerax/PreviewHostApiImpl.java:84: error: cannot find symbol
            new TextureRegistry.SurfaceProducer.Callback() {
                                               ^
  symbol:   class Callback
  location: interface SurfaceProducer
/Users/divyamverma/.pub-cache/hosted/pub.dev/camera_android_camerax-0.6.8+2/android/src/main/java/io/flutter/plugins/camerax/PreviewHostApiImpl.java:85: error: method does not override or implement a method from a supertype
              @Override
              ^
/Users/divyamverma/.pub-cache/hosted/pub.dev/camera_android_camerax-0.6.8+2/android/src/main/java/io/flutter/plugins/camerax/PreviewHostApiImpl.java:91: error: method does not override or implement a method from a supertype
              @Override
              ^
3 errors

FAILURE: Build failed with an exception.

Expected results
it should not give that issue
Actual results
apk should  built
Code sample
plugin has issue
Screenshots or Video
video not required
Logs
TextureRegistry.SurfaceProducer.Callback() {
                                               ^
  symbol:   class Callback
  location: interface SurfaceProducer

Flutter Doctor output
[!] Flutter (Channel [user-branch], 3.22.0, on macOS 14.5 23F79 darwin-arm64, locale en-IN)
    ! Flutter version 3.22.0 on channel [user-branch] at /Users/divyamverma/Documents/flutter
      Currently on an unknown channel. Run `flutter channel` to switch to an official channel.
      If that doesn't fix the issue, reinstall Flutter by following instructions at
      https://flutter.dev/docs/get-started/install.
    ! Upstream repository unknown source is not a standard remote.
      Set environment variable "FLUTTER_GIT_URL" to unknown source to dismiss this error.
    • Framework revision 5dcb86f68f (5 months ago), 2024-05-09 07:39:20 -0500
    • Engine revision f6344b75dc
    • Dart version 3.4.0
    • DevTools version 2.34.3
    • If those were intentional, you can disregard the above warnings; however it is recommended to use "git"
      directly to perform update checks and upgrades.

[!] Android toolchain - develop for Android devices (Android SDK version 35.0.0)
    • Android SDK at /Users/divyamverma/Library/Android/sdk
    ✗ cmdline-tools component is missing
      Run `path/to/sdkmanager --install "cmdline-tools;latest"`
      See https://developer.android.com/studio/command-line for more details.
    ✗ Android license status unknown.
      Run `flutter doctor --android-licenses` to accept the SDK licenses.
      See https://flutter.dev/docs/get-started/install/macos#android-setup for more details.

[✓] Xcode - develop for iOS and macOS (Xcode 15.4)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Build 15F31d
    • CocoaPods version 1.15.2

[✗] Chrome - develop for the web (Cannot find Chrome executable at /Applications/Google
    Chrome.app/Contents/MacOS/Google Chrome)
    ! Cannot find Chrome. Try setting CHROME_EXECUTABLE to a Chrome executable.

[✓] Android Studio (version 2024.1)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 17.0.11+0-17.0.11b1207.24-11852314)

[✓] VS Code (version 1.93.1)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.96.0

[✓] Connected device (3 available)
    • SM E346B (mobile)               • RZCW80JP8FA           • android-arm64 • Android 14 (API 34)
    • macOS (desktop)                 • macos                 • darwin-arm64  • macOS 14.5 23F79 darwin-arm64
    • Mac Designed for iPad (desktop) • mac-designed-for-ipad • darwin        • macOS 14.5 23F79 darwin-arm64
    ! Error: Browsing on the local area network for iPhone. Ensure the device is unlocked and attached with a cable
      or associated with the same local area network as this Mac.
      The device must be opted into Developer Mode to connect wirelessly. (code -27)

[✓] Network resources
    • All expected network resources are available.

! Doctor found issues in 3 categories.

This is similar to https://github.com/flutter/flutter/issues/154332, https://github.com/flutter/flutter/issues/154054 and it is said that it is fixed on 3.24.1, please upgrade your Flutter SDK and retry.
Closing this as a duplicate of https://github.com/flutter/flutter/issues/154332. If the issue still persists on the latest Flutter stable release, please write in comments and I will reopen this. Thanks!


Add top level spec field to traffic specs and targets

This PR adds a top level spec field to the specs and targets API in order to make them conformant with Kubernetes. The traffic target specs field has been renamed to rules. This is a breaking change!
Fix: #155

@michelleN can you please review this one more time? thanks


enh: update tmq api

Codecov Report

Merging #122 (1754b35) into main (61b9c7b) will decrease coverage by 1.09%.
The diff coverage is 58.11%.

@@            Coverage Diff             @@
##             main     #122      +/-   ##
==========================================
- Coverage   77.70%   76.60%   -1.10%
==========================================
  Files          19       19
  Lines        2184     2300     +116
==========================================
+ Hits         1697     1762      +65
- Misses        487      538      +51

Impacted Files
Coverage Δ

taos/cinterface.py
73.10% <ø> (-0.14%)
:arrow_down:

taos/tmq.py
70.92% <58.11%> (-13.76%)
:arrow_down:

taos/result.py
72.28% <0.00%> (-0.55%)
:arrow_down:

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


XFAIL failing tflite test

The datatset for some tests doesnt seem currently accessible. More details on
https://github.com/iree-org/iree/issues/18518

https://github.com/iree-org/iree/pull/19127 will supercede this (though I didn't run the integrations/tensorflow/test/iree_tfl_tests/update_tflite_model_documentation.py script)

#19127 will supersede this (though I didn't run the integrations/tensorflow/test/iree_tfl_tests/update_tflite_model_documentation.py script)

Sounds good. I can keep that part and refactor this PR tomorrow.

#19127 will supersede this (though I didn't run the integrations/tensorflow/test/iree_tfl_tests/update_tflite_model_documentation.py script)

Sounds good. I can keep that part and refactor this PR tomorrow.

That's fine, thanks. A better fix than just running the script as-is would be updating the tests to use files that are still available, so the test status is accurate (we should support mobilenet_v1 generally, even if our mirror of the .tflite file for mobilenet_v1 used in a test is inaccessible ;P). I have https://github.com/iree-org/iree-test-suites/issues/5 for the long term solution for such tests.

After speaking to Scott offline, I think the way the state of these tests is currently is what it should be until the long term fixes start landing so I am not going to make any changes.


New Bug, possibly

Any time myself or player use the sheet to roll we get "An Error occurred while rendering this roll template." but can still roll with /roll

Which sheet?

"Rogue Trader Improved", have informed the the author

not an isolated incident, it's system wide

All of roll20 or just that sheet

Here is a screenshot from a forum post on roll20


Position of close button on alert messages in Chrome

The cross on alert messages (e.g. validation results) overlaps the text in Chrome on Mac. Other platforms not tested.

Yeah, I noticed this as well, I wasn't sure what was causing it, the growl plugin example on their site works fine, I just wasn't willing to spend time on tracking it down yet.

Well, it looks like this just turned into a bigger update but perhaps for the best. It seems Bootstrap Growl has just had a new major release and renamed itself... this could solve this bug though http://bootstrap-growl.remabledesigns.com/