Type {LST, MST, SST} should be an attribute of the OpticsDescription, not the TelescopeDescription

Describe the (datamodel) bug
We store type at the TelescopeDescription although it is determined by mirror (area / diameter).

Fixed in #2019


Instacart 3M orders

https://tech.instacart.com/3-million-instacart-orders-open-sourced-d40d29ead6f2

This was closed with #27


Automated logviewer launch for 0.x

Notes

Automated logviewer launch on any worker hosts running Storm bits (0.x specific)
State of these logviewer daemons are tracked per host by stuffing state into ZooKeeper
Reconciliation is also implemented to ensure that logviewers are brought back alive if ever killed.

ZKClient.java

Wrapper around Apache Curator in order to interact with ZK
Can create, delete, and check for existence of nodes in ZK
Used to track which hosts are running logviewer daemons by tracking that state into ZK
Junit tests are written for this class

MesosNimbus.java

Added TimerTask to perform implicit reconciliation every 5 minutes

Implicit reconciliation reconciles that state of every running task between the framework scheduler and master (sends update of task status to statusUpdate method in NimbusMesosScheduler)

launchLogviewer method called in allSlotsAvailableForScheduling loops through the existingSupervisors and schedules the logviewer daemon as Mesos tasks based on the offers available on each host

Checks ZK if logviewer already running on the host which the supervisor is running on, as such, ensures that logviewer only runs on hosts with Storm bits

NimbusMesosScheduler.java

Updated statusUpdate method to look for logviewer tasks to check if they need to be relaunched
updateLogviewerState method updates the state of the logviewer task in ZK if needed

reviveOffers if logviewer needs to be relaunched

storm.yaml

Added new config called storm.logviewer.zookeeper.dir which allows you to configure under which "path" you want the logviewer state to be stored in ZK

@changreytang : thanks for updating this!
I'm confused by the presence of this commit in your PR: https://github.com/mesos/storm/pull/209/commits/7ae98cb358d73cd00489ae5f9b8c30aa76b1980f, can you explain why it's there or ideally get rid of it somehow?
The commit comment for it is from a different commit from awhile ago: https://github.com/mesos/storm/commit/4153391a0972918f1dbbb7a470d384968510aab2

@erikdw not 100% sure why it's like that, must've been something when I rebased, will look into it to see if I can get rid of it

This has been merged via #228, see that PR for details.


pybind / python unittests fail when building from source

Hello,
I am trying to install Crocoddyl from source on Ubuntu 18.04 with the multi-threading option. I cloned the devel branch and did
cmake .. -DBUILD_WITH_MULTITHREADS=ON
make
make install
make test

I get the following output
Running tests...
/usr/bin/ctest --force-new-ctest-process
Test project /home/sebastien/crocoddyl/cmake
      Start  1: test_states
 1/39 Test  #1: test_states ...................................   Passed    0.20 sec
      Start  2: test_actuation
 2/39 Test  #2: test_actuation ................................   Passed    0.04 sec
      Start  3: test_activations
 3/39 Test  #3: test_activations ..............................   Passed    0.02 sec
      Start  4: test_costs
 4/39 Test  #4: test_costs ....................................***Failed    2.28 sec
      Start  5: test_cost_sum
 5/39 Test  #5: test_cost_sum .................................   Passed    3.98 sec
      Start  6: test_contacts
 6/39 Test  #6: test_contacts .................................   Passed    0.14 sec
      Start  7: test_impulses
 7/39 Test  #7: test_impulses .................................   Passed    0.10 sec
      Start  8: test_multiple_contacts
 8/39 Test  #8: test_multiple_contacts ........................   Passed    1.23 sec
      Start  9: test_multiple_impulses
 9/39 Test  #9: test_multiple_impulses ........................   Passed    1.23 sec
      Start 10: test_actions
10/39 Test #10: test_actions ..................................   Passed    0.02 sec
      Start 11: test_diff_actions
11/39 Test #11: test_diff_actions .............................   Passed    0.39 sec
      Start 12: test_friction_cone
12/39 Test #12: test_friction_cone ............................   Passed    0.00 sec
      Start 13: test_boxqp
13/39 Test #13: test_boxqp ....................................   Passed    0.00 sec
      Start 14: test_solvers
14/39 Test #14: test_solvers ..................................   Passed    3.44 sec
      Start 15: pybinds-states
15/39 Test #15: pybinds-states ................................***Failed    0.49 sec
      Start 16: pybinds-actuations
16/39 Test #16: pybinds-actuations ............................***Failed    0.23 sec
      Start 17: pybinds-actions
17/39 Test #17: pybinds-actions ...............................***Failed    0.23 sec
      Start 18: pybinds-shooting
18/39 Test #18: pybinds-shooting ..............................***Failed    0.19 sec
      Start 19: pybinds-solvers
19/39 Test #19: pybinds-solvers ...............................***Failed    0.22 sec
      Start 20: pybinds-costs
20/39 Test #20: pybinds-costs .................................***Failed    0.24 sec
      Start 21: pybinds-contacts
21/39 Test #21: pybinds-contacts ..............................***Failed    0.23 sec
      Start 22: pybinds-impulses
22/39 Test #22: pybinds-impulses ..............................***Failed    0.23 sec
      Start 23: pybinds-force_cost
23/39 Test #23: pybinds-force_cost ............................***Failed    0.18 sec
      Start 24: pybinds-friction_cone_cost
24/39 Test #24: pybinds-friction_cone_cost ....................***Failed    0.18 sec
      Start 25: pybinds-momemtum_cost
25/39 Test #25: pybinds-momemtum_cost .........................***Failed    0.24 sec
      Start 26: pybinds-impulse_com_cost
26/39 Test #26: pybinds-impulse_com_cost ......................***Failed    0.19 sec
      Start 27: pybinds-squashing
27/39 Test #27: pybinds-squashing .............................   Passed    0.26 sec
      Start 28: example-python-double_pendulum
28/39 Test #28: example-python-double_pendulum ................***Failed    0.28 sec
      Start 29: example-python-boxfddp_vs_boxddp
29/39 Test #29: example-python-boxfddp_vs_boxddp ..............***Failed    0.32 sec
      Start 30: example-python-quadrotor
30/39 Test #30: example-python-quadrotor ......................***Failed    0.24 sec
      Start 31: example-python-quadrotor_ubound
31/39 Test #31: example-python-quadrotor_ubound ...............***Failed    0.23 sec
      Start 32: example-python-arm_manipulation
32/39 Test #32: example-python-arm_manipulation ...............***Failed    0.22 sec
      Start 33: example-python-quadrupedal_gaits
33/39 Test #33: example-python-quadrupedal_gaits ..............***Failed    0.31 sec
      Start 34: example-python-quadrupedal_walk_ubound
34/39 Test #34: example-python-quadrupedal_walk_ubound ........***Failed    0.31 sec
      Start 35: example-python-bipedal_walk
35/39 Test #35: example-python-bipedal_walk ...................***Failed    0.34 sec
      Start 36: example-python-bipedal_walk_ubound
36/39 Test #36: example-python-bipedal_walk_ubound ............***Failed    0.34 sec
      Start 37: example-python-humanoid_manipulation
37/39 Test #37: example-python-humanoid_manipulation ..........***Failed    0.35 sec
      Start 38: example-python-humanoid_manipulation_ubound
38/39 Test #38: example-python-humanoid_manipulation_ubound ...***Failed    0.37 sec
      Start 39: example-python-humanoid_taichi
39/39 Test #39: example-python-humanoid_taichi ................***Failed    0.38 sec

36% tests passed, 25 tests failed out of 39

Total Test time (real) =  19.91 sec

The following tests FAILED:
	  4 - test_costs (Failed)
	 15 - pybinds-states (Failed)
	 16 - pybinds-actuations (Failed)
	 17 - pybinds-actions (Failed)
	 18 - pybinds-shooting (Failed)
	 19 - pybinds-solvers (Failed)
	 20 - pybinds-costs (Failed)
	 21 - pybinds-contacts (Failed)
	 22 - pybinds-impulses (Failed)
	 23 - pybinds-force_cost (Failed)
	 24 - pybinds-friction_cone_cost (Failed)
	 25 - pybinds-momemtum_cost (Failed)
	 26 - pybinds-impulse_com_cost (Failed)
	 28 - example-python-double_pendulum (Failed)
	 29 - example-python-boxfddp_vs_boxddp (Failed)
	 30 - example-python-quadrotor (Failed)
	 31 - example-python-quadrotor_ubound (Failed)
	 32 - example-python-arm_manipulation (Failed)
	 33 - example-python-quadrupedal_gaits (Failed)
	 34 - example-python-quadrupedal_walk_ubound (Failed)
	 35 - example-python-bipedal_walk (Failed)
	 36 - example-python-bipedal_walk_ubound (Failed)
	 37 - example-python-humanoid_manipulation (Failed)
	 38 - example-python-humanoid_manipulation_ubound (Failed)
	 39 - example-python-humanoid_taichi (Failed)
Errors while running CTest
Makefile:109: recipe for target 'test' failed
make: *** [test] Error 8

Then I tried the same installation without the multi-threading option and it still fails. I also (naively) tried with an additional -DPYTHON_EXECUTABLE='/usr/bin/python2' to see if it would change anything, but it doesn't. I guess it's somehow related to my installation, but I'm not sure how to debug this. Do you have any idea where the problem may come from ? Should I use the master branch instead ?

All unittests are passing in non multithreading mode. Are you sure you don't have multiple installed copies of Crocoddyl.
For the multhreading mode, there is a known issue that needs to be tackle. The problem is that some of the python unittests creates internal data (not possible to avoid). However, it could be possible to create different copies of a Python model, which could be handle inside the code after creating objects that are copyable or pickleable.

I made sure to uninstall Crocoddyl before building from source and was able to reproduced the same output. In non multithreading mode I get the following
Running tests...
/usr/bin/ctest --force-new-ctest-process
Test project /home/sebastien/crocoddyl/cmake
      Start  1: test_states
 1/39 Test  #1: test_states ...................................   Passed    0.14 sec
      Start  2: test_actuation
 2/39 Test  #2: test_actuation ................................   Passed    0.04 sec
      Start  3: test_activations
 3/39 Test  #3: test_activations ..............................   Passed    0.01 sec
      Start  4: test_costs
 4/39 Test  #4: test_costs ....................................   Passed    2.21 sec
      Start  5: test_cost_sum
 5/39 Test  #5: test_cost_sum .................................   Passed    4.01 sec
      Start  6: test_contacts
 6/39 Test  #6: test_contacts .................................   Passed    0.14 sec
      Start  7: test_impulses
 7/39 Test  #7: test_impulses .................................   Passed    0.10 sec
      Start  8: test_multiple_contacts
 8/39 Test  #8: test_multiple_contacts ........................   Passed    1.23 sec
      Start  9: test_multiple_impulses
 9/39 Test  #9: test_multiple_impulses ........................   Passed    1.23 sec
      Start 10: test_actions
10/39 Test #10: test_actions ..................................   Passed    0.02 sec
      Start 11: test_diff_actions
11/39 Test #11: test_diff_actions .............................   Passed    0.39 sec
      Start 12: test_friction_cone
12/39 Test #12: test_friction_cone ............................   Passed    0.00 sec
      Start 13: test_boxqp
13/39 Test #13: test_boxqp ....................................   Passed    0.00 sec
      Start 14: test_solvers
14/39 Test #14: test_solvers ..................................   Passed    6.79 sec
      Start 15: pybinds-states
15/39 Test #15: pybinds-states ................................***Failed    0.24 sec
      Start 16: pybinds-actuations
16/39 Test #16: pybinds-actuations ............................***Failed    0.28 sec
      Start 17: pybinds-actions
17/39 Test #17: pybinds-actions ...............................***Failed    0.29 sec
      Start 18: pybinds-shooting
18/39 Test #18: pybinds-shooting ..............................***Failed    0.31 sec
      Start 19: pybinds-solvers
19/39 Test #19: pybinds-solvers ...............................***Failed    0.32 sec
      Start 20: pybinds-costs
20/39 Test #20: pybinds-costs .................................***Failed    0.34 sec
      Start 21: pybinds-contacts
21/39 Test #21: pybinds-contacts ..............................***Failed    0.32 sec
      Start 22: pybinds-impulses
22/39 Test #22: pybinds-impulses ..............................***Failed    0.32 sec
      Start 23: pybinds-force_cost
23/39 Test #23: pybinds-force_cost ............................***Failed    0.25 sec
      Start 24: pybinds-friction_cone_cost
24/39 Test #24: pybinds-friction_cone_cost ....................***Failed    0.24 sec
      Start 25: pybinds-momemtum_cost
25/39 Test #25: pybinds-momemtum_cost .........................***Failed    0.32 sec
      Start 26: pybinds-impulse_com_cost
26/39 Test #26: pybinds-impulse_com_cost ......................***Failed    0.24 sec
      Start 27: pybinds-squashing
27/39 Test #27: pybinds-squashing .............................   Passed    0.29 sec
      Start 28: example-python-double_pendulum
28/39 Test #28: example-python-double_pendulum ................***Failed    0.29 sec
      Start 29: example-python-boxfddp_vs_boxddp
29/39 Test #29: example-python-boxfddp_vs_boxddp ..............***Failed    0.33 sec
      Start 30: example-python-quadrotor
30/39 Test #30: example-python-quadrotor ......................***Failed    0.23 sec
      Start 31: example-python-quadrotor_ubound
31/39 Test #31: example-python-quadrotor_ubound ...............***Failed    0.23 sec
      Start 32: example-python-arm_manipulation
32/39 Test #32: example-python-arm_manipulation ...............***Failed    0.24 sec
      Start 33: example-python-quadrupedal_gaits
33/39 Test #33: example-python-quadrupedal_gaits ..............***Failed    0.33 sec
      Start 34: example-python-quadrupedal_walk_ubound
34/39 Test #34: example-python-quadrupedal_walk_ubound ........***Failed    0.34 sec
      Start 35: example-python-bipedal_walk
35/39 Test #35: example-python-bipedal_walk ...................***Failed    0.36 sec
      Start 36: example-python-bipedal_walk_ubound
36/39 Test #36: example-python-bipedal_walk_ubound ............***Failed    0.35 sec
      Start 37: example-python-humanoid_manipulation
37/39 Test #37: example-python-humanoid_manipulation ..........***Failed    0.35 sec
      Start 38: example-python-humanoid_manipulation_ubound
38/39 Test #38: example-python-humanoid_manipulation_ubound ...***Failed    0.36 sec
      Start 39: example-python-humanoid_taichi
39/39 Test #39: example-python-humanoid_taichi ................***Failed    0.35 sec

38% tests passed, 24 tests failed out of 39

Total Test time (real) =  23.85 sec

The following tests FAILED:
	 15 - pybinds-states (Failed)
	 16 - pybinds-actuations (Failed)
	 17 - pybinds-actions (Failed)
	 18 - pybinds-shooting (Failed)
	 19 - pybinds-solvers (Failed)
	 20 - pybinds-costs (Failed)
	 21 - pybinds-contacts (Failed)
	 22 - pybinds-impulses (Failed)
	 23 - pybinds-force_cost (Failed)
	 24 - pybinds-friction_cone_cost (Failed)
	 25 - pybinds-momemtum_cost (Failed)
	 26 - pybinds-impulse_com_cost (Failed)
	 28 - example-python-double_pendulum (Failed)
	 29 - example-python-boxfddp_vs_boxddp (Failed)
	 30 - example-python-quadrotor (Failed)
	 31 - example-python-quadrotor_ubound (Failed)
	 32 - example-python-arm_manipulation (Failed)
	 33 - example-python-quadrupedal_gaits (Failed)
	 34 - example-python-quadrupedal_walk_ubound (Failed)
	 35 - example-python-bipedal_walk (Failed)
	 36 - example-python-bipedal_walk_ubound (Failed)
	 37 - example-python-humanoid_manipulation (Failed)
	 38 - example-python-humanoid_manipulation_ubound (Failed)
	 39 - example-python-humanoid_taichi (Failed)
Errors while running CTest
Makefile:109: recipe for target 'test' failed
make: *** [test] Error 8

Concerning the multithreading mode, I tried to run a simple program despite the failure of the unitttests and I got memory allocations errors upon instantiation of ShootingProblem and Solver objects (both in python and C++). Is this related to the known issue you are mentioning ? In this case, could you be more specific about how to circumvent this issue ?
Besides, before opening this issue I also tried installing from the master branch (with and without multithreading). Python unittests failed similarly in both cases, however I was able to use in my simple program without any memory error...

It is hard to know what it's going on, you could run the test with in verbose mode: ctest -v. However, given the recent changes in example-robot-data, I would say that you need to check that it is updated and working the repository. For instance, could you run the Crocoddyl examples?

Concerning the multithreading mode, I tried to run a simple program despite the failure of the unitttests and I got memory allocations errors upon instantiation of ShootingProblem and Solver objects (both in python and C++). Is this related to the known issue you are mentioning ? In this case, could you be more specific about how to circumvent this issue ?

Again, it is hard to give you an answer without further details. Which program were you running? Which action model were you trying?
Let me try to explain you the issue. We cannot derived action data in an action model written in Python, and this is not possible to solver it. Therefore, in the action models written in Python, we need to allocate internal data. However, if you create one action model and use it for each node, then when we run a multithreading application, the data used in a given node is updated in parallel by another node. This results in a wrong computation.
So, you need, as an user, to create a copy of the action model written in Python per each node. To do it easily, we need to enable the copying of the action models. This is in my TODO list, but not priority.

Thank you for your detailed answer. So if I understand correctly, this memory allocation issue only affects action models created in Python, and the fix would be to enable copying these action models in order to avoid conflicts in the data when running the computations in parallel.

It is hard to know what it's going on, you could run the test with in verbose mode: ctest -v. However, given the recent changes in example-robot-data, I would say that you need to check that it is updated and working the repository. For instance, could you run the Crocoddyl examples?

Let me be try to be more specific : I made sure example-robot-data works fine and ran the tests with ctest -v (built from devel branch with multi-threading on)
Test project /home/sebastien/crocoddyl/cmake
      Start  1: test_states
 1/39 Test  #1: test_states ...................................   Passed    0.13 sec
      Start  2: test_actuation
 2/39 Test  #2: test_actuation ................................   Passed    0.04 sec
      Start  3: test_activations
 3/39 Test  #3: test_activations ..............................   Passed    0.01 sec
      Start  4: test_costs
 4/39 Test  #4: test_costs ....................................***Failed    2.18 sec
      Start  5: test_cost_sum
 5/39 Test  #5: test_cost_sum .................................   Passed    4.15 sec
      Start  6: test_contacts
 6/39 Test  #6: test_contacts .................................   Passed    0.14 sec
      Start  7: test_impulses
 7/39 Test  #7: test_impulses .................................   Passed    0.10 sec
      Start  8: test_multiple_contacts
 8/39 Test  #8: test_multiple_contacts ........................   Passed    1.23 sec
      Start  9: test_multiple_impulses
 9/39 Test  #9: test_multiple_impulses ........................   Passed    1.24 sec
      Start 10: test_actions
10/39 Test #10: test_actions ..................................   Passed    0.02 sec
      Start 11: test_diff_actions
11/39 Test #11: test_diff_actions .............................   Passed    0.42 sec
      Start 12: test_friction_cone
12/39 Test #12: test_friction_cone ............................   Passed    0.00 sec
      Start 13: test_boxqp
13/39 Test #13: test_boxqp ....................................   Passed    0.00 sec
      Start 14: test_solvers
14/39 Test #14: test_solvers ..................................   Passed    3.45 sec
      Start 15: pybinds-states
15/39 Test #15: pybinds-states ................................   Passed    1.20 sec
      Start 16: pybinds-actuations
16/39 Test #16: pybinds-actuations ............................***Exception: Child aborted  0.56 sec
      Start 17: pybinds-actions
17/39 Test #17: pybinds-actions ...............................***Exception: SegFault  0.60 sec
      Start 18: pybinds-shooting
18/39 Test #18: pybinds-shooting ..............................***Exception: SegFault  0.49 sec
      Start 19: pybinds-solvers
19/39 Test #19: pybinds-solvers ...............................***Exception: SegFault  0.60 sec
      Start 20: pybinds-costs
20/39 Test #20: pybinds-costs .................................***Exception: SegFault  4.50 sec
      Start 21: pybinds-contacts
21/39 Test #21: pybinds-contacts ..............................***Exception: SegFault  1.55 sec
      Start 22: pybinds-impulses
22/39 Test #22: pybinds-impulses ..............................***Exception: SegFault  1.44 sec
      Start 23: pybinds-force_cost
23/39 Test #23: pybinds-force_cost ............................***Exception: Child aborted  0.73 sec
      Start 24: pybinds-friction_cone_cost
24/39 Test #24: pybinds-friction_cone_cost ....................***Exception: Child aborted  0.71 sec
      Start 25: pybinds-momemtum_cost
25/39 Test #25: pybinds-momemtum_cost .........................   Passed    2.73 sec
      Start 26: pybinds-impulse_com_cost
26/39 Test #26: pybinds-impulse_com_cost ......................***Exception: Child aborted  0.64 sec
      Start 27: pybinds-squashing
27/39 Test #27: pybinds-squashing .............................   Passed    0.25 sec
      Start 28: example-python-double_pendulum
28/39 Test #28: example-python-double_pendulum ................***Exception: Child aborted  0.59 sec
      Start 29: example-python-boxfddp_vs_boxddp
29/39 Test #29: example-python-boxfddp_vs_boxddp ..............***Exception: Child aborted  2.88 sec
      Start 30: example-python-quadrotor
30/39 Test #30: example-python-quadrotor ......................   Passed    0.20 sec
      Start 31: example-python-quadrotor_ubound
31/39 Test #31: example-python-quadrotor_ubound ...............   Passed    0.30 sec
      Start 32: example-python-arm_manipulation
32/39 Test #32: example-python-arm_manipulation ...............   Passed    0.56 sec
      Start 33: example-python-quadrupedal_gaits
33/39 Test #33: example-python-quadrupedal_gaits ..............***Exception: Child aborted  3.02 sec
      Start 34: example-python-quadrupedal_walk_ubound
34/39 Test #34: example-python-quadrupedal_walk_ubound ........***Exception: SegFault  3.06 sec
      Start 35: example-python-bipedal_walk
35/39 Test #35: example-python-bipedal_walk ...................***Exception: Child aborted  1.31 sec
      Start 36: example-python-bipedal_walk_ubound
36/39 Test #36: example-python-bipedal_walk_ubound ............***Exception: Child aborted  1.56 sec
      Start 37: example-python-humanoid_manipulation
37/39 Test #37: example-python-humanoid_manipulation ..........***Exception: Child aborted  1.49 sec
      Start 38: example-python-humanoid_manipulation_ubound
38/39 Test #38: example-python-humanoid_manipulation_ubound ...***Exception: Child aborted  1.30 sec
      Start 39: example-python-humanoid_taichi
39/39 Test #39: example-python-humanoid_taichi ................***Exception: Child aborted  1.28 sec

49% tests passed, 20 tests failed out of 39

Total Test time (real) =  46.66 sec

The following tests FAILED:
	  4 - test_costs (Failed)
	 16 - pybinds-actuations (Child aborted)
	 17 - pybinds-actions (SEGFAULT)
	 18 - pybinds-shooting (SEGFAULT)
	 19 - pybinds-solvers (SEGFAULT)
	 20 - pybinds-costs (SEGFAULT)
	 21 - pybinds-contacts (SEGFAULT)
	 22 - pybinds-impulses (SEGFAULT)
	 23 - pybinds-force_cost (Child aborted)
	 24 - pybinds-friction_cone_cost (Child aborted)
	 26 - pybinds-impulse_com_cost (Child aborted)
	 28 - example-python-double_pendulum (Child aborted)
	 29 - example-python-boxfddp_vs_boxddp (Child aborted)
	 33 - example-python-quadrupedal_gaits (Child aborted)
	 34 - example-python-quadrupedal_walk_ubound (SEGFAULT)
	 35 - example-python-bipedal_walk (Child aborted)
	 36 - example-python-bipedal_walk_ubound (Child aborted)
	 37 - example-python-humanoid_manipulation (Child aborted)
	 38 - example-python-humanoid_manipulation_ubound (Child aborted)
	 39 - example-python-humanoid_taichi (Child aborted)
Errors while running CTest

The Crocoddyl examples arm_manipulation and quadrotor work fine but not the other ones. For instance when I run make examples-humanoid_taichi
cd /home/sebastien/crocoddyl/cmake && make -f CMakeFiles/Makefile2 examples/CMakeFiles/examples-humanoid_taichi.dir/rule
make[1]: Entering directory '/home/sebastien/crocoddyl/cmake'
/usr/bin/cmake -H/home/sebastien/crocoddyl -B/home/sebastien/crocoddyl/cmake --check-build-system CMakeFiles/Makefile.cmake 0
/usr/bin/cmake -E cmake_progress_start /home/sebastien/crocoddyl/cmake/CMakeFiles 0
make -f CMakeFiles/Makefile2 examples/CMakeFiles/examples-humanoid_taichi.dir/all
make[2]: Entering directory '/home/sebastien/crocoddyl/cmake'
make -f examples/CMakeFiles/examples-humanoid_taichi.dir/build.make examples/CMakeFiles/examples-humanoid_taichi.dir/depend
make[3]: Entering directory '/home/sebastien/crocoddyl/cmake'
cd /home/sebastien/crocoddyl/cmake && /usr/bin/cmake -E cmake_depends "Unix Makefiles" /home/sebastien/crocoddyl /home/sebastien/crocoddyl/examples /home/sebastien/crocoddyl/cmake /home/sebastien/crocoddyl/cmake/examples /home/sebastien/crocoddyl/cmake/examples/CMakeFiles/examples-humanoid_taichi.dir/DependInfo.cmake --color=
make[3]: Leaving directory '/home/sebastien/crocoddyl/cmake'
make -f examples/CMakeFiles/examples-humanoid_taichi.dir/build.make examples/CMakeFiles/examples-humanoid_taichi.dir/build
make[3]: Entering directory '/home/sebastien/crocoddyl/cmake'
cd /home/sebastien/crocoddyl/cmake/examples && /usr/bin/cmake -E env PYTHONPATH=/home/sebastien/crocoddyl/cmake/bindings/python:/opt/openrobots/lib/python2.7/site-packages:/usr/local/lib/python2.7/dist-packages:/opt/ros/melodic/lib/python2.7/dist-packages /usr/bin/python2 -c import\ humanoid_taichi
double free or corruption (out)
Child aborted
examples/CMakeFiles/examples-humanoid_taichi.dir/build.make:60: recipe for target 'examples/CMakeFiles/examples-humanoid_taichi' failed
make[3]: *** [examples/CMakeFiles/examples-humanoid_taichi] Error 1
make[3]: Leaving directory '/home/sebastien/crocoddyl/cmake'
CMakeFiles/Makefile2:4510: recipe for target 'examples/CMakeFiles/examples-humanoid_taichi.dir/all' failed
make[2]: *** [examples/CMakeFiles/examples-humanoid_taichi.dir/all] Error 2
make[2]: Leaving directory '/home/sebastien/crocoddyl/cmake'
CMakeFiles/Makefile2:4517: recipe for target 'examples/CMakeFiles/examples-humanoid_taichi.dir/rule' failed
make[1]: *** [examples/CMakeFiles/examples-humanoid_taichi.dir/rule] Error 2
make[1]: Leaving directory '/home/sebastien/crocoddyl/cmake'
Makefile:175: recipe for target 'examples/CMakeFiles/examples-humanoid_taichi.dir/rule' failed
make: *** [examples/CMakeFiles/examples-humanoid_taichi.dir/rule] Error 2

Which program were you running? Which action model were you trying?

I tried to run the following program
import numpy as np
import pinocchio as pin
from py_robot_properties_iiwa.robot import IiwaRobot
import crocoddyl

# Create a robot instance. This initializes the simulator as well.
robot = IiwaRobot()
id_endeff = robot.pin_robot.model.getFrameId('contact')
nq = robot.pin_robot.model.nq
nv = robot.pin_robot.model.nv

# Reset the robot to some initial state, get initial placement and define goal placement
q0 = pin.utils.zero(nq)
dq0 = pin.utils.zero(nv)
x0 = np.concatenate([q0, dq0])
robot.reset_state(q0, dq0)
M0 = robot.pin_robot.data.oMf[id_endeff]
M_des = pin.SE3(np.eye(3), np.matrix([.5, .5, .5]).T)

# Define cost models
Mref = crocoddyl.FramePlacement(id_endeff, M_des)
state = crocoddyl.StateMultibody(robot.pin_robot.model)
goalTrackingCost = crocoddyl.CostModelFramePlacement(state, Mref)
xRegCost = crocoddyl.CostModelState(state)
uRegCost = crocoddyl.CostModelControl(state)

# Create cost model per each action model
runningCostModel = crocoddyl.CostModelSum(state)
terminalCostModel = crocoddyl.CostModelSum(state)

# Running and terminal cost functions
cost_weights = [1, 1e-2, 1e-5, 1e4]
runningCostModel.addCost("endeff", goalTrackingCost, cost_weights[0]) # 5
runningCostModel.addCost("stateReg", xRegCost, cost_weights[1]) # 1e-2
runningCostModel.addCost("ctrlReg", uRegCost, cost_weights[2]) # 1e-4
terminalCostModel.addCost("endeff", goalTrackingCost, cost_weights[3]) # 1e3

# Action model
dt = 1e-2
actuation = crocoddyl.ActuationModelFull(state)
runningModel = crocoddyl.IntegratedActionModelEuler(crocoddyl.DifferentialActionModelFreeFwdDynamics(state, actuation, runningCostModel), dt)
terminalModel = crocoddyl.IntegratedActionModelEuler(crocoddyl.DifferentialActionModelFreeFwdDynamics(state, actuation, terminalCostModel))

# Shooting problem
N = 200
problem = crocoddyl.ShootingProblem(x0, [runningModel] * N, terminalModel)

# Solver
ddp = crocoddyl.SolverDDP(problem)

The last line caused Segmentation fault (core dumped) . In C++, I tried the following code
    // Desired end-effector frame placement and state
    boost::shared_ptr<crocoddyl::StateMultibody> state = boost::make_shared<crocoddyl::StateMultibody>(boost::make_shared<pinocchio::Model>(pin_model_));

    // Create running and terminal cost models
    boost::shared_ptr<crocoddyl::CostModelFramePlacement> goalTrackingCost = boost::make_shared<crocoddyl::CostModelFramePlacement>(state, ref_placement_);
    boost::shared_ptr<crocoddyl::CostModelState> xRegCost = boost::make_shared<crocoddyl::CostModelState>(state);
    boost::shared_ptr<crocoddyl::CostModelControl> uRegCost = boost::make_shared<crocoddyl::CostModelControl>(state);
    boost::shared_ptr<crocoddyl::CostModelSum> runningCostModel = boost::make_shared<crocoddyl::CostModelSum>(state);
    boost::shared_ptr<crocoddyl::CostModelSum> terminalCostModel = boost::make_shared<crocoddyl::CostModelSum>(state);
    runningCostModel.get()->addCost("endeff", goalTrackingCost, cost_weights(0));
    runningCostModel.get()->addCost("stateReg", xRegCost, cost_weights(1));
    runningCostModel.get()->addCost("ctrlReg", uRegCost, cost_weights(2));
    terminalCostModel.get()->addCost("endeff", goalTrackingCost, cost_weights(3));

    // Create action model
    boost::shared_ptr<crocoddyl::ActuationModelFull> actuation = boost::make_shared<crocoddyl::ActuationModelFull>(state);
    boost::shared_ptr<crocoddyl::ActionModelAbstract> runningModel =
        boost::make_shared<crocoddyl::IntegratedActionModelEuler>(boost::make_shared<crocoddyl::DifferentialActionModelFreeFwdDynamics>(state, actuation, runningCostModel), dt);
    boost::shared_ptr<crocoddyl::ActionModelAbstract> terminalModel =
        boost::make_shared<crocoddyl::IntegratedActionModelEuler>(boost::make_shared<crocoddyl::DifferentialActionModelFreeFwdDynamics>(state, actuation, terminalCostModel));

    // Stack running models
    std::vector< boost::shared_ptr<crocoddyl::ActionModelAbstract> > runningModels(N, runningModel);

    // Create the shooting problem
    problem_ = boost::make_shared<crocoddyl::ShootingProblem>(x0, runningModels, terminalModel);

    // Create the solver
    ddp_ = boost::make_shared<crocoddyl::SolverDDP>(problem_);

This code works fine, except when calling ddp_->solve(x_traj_, u_traj_, maxit_); which also causes a segmentation fault error.
It seems however that my problem is not related to multi-threading, since as I mentioned, I encounter exactly the same issues running the same programs when building without multi-threading mode. But these programs work fine when Crocoddyl is installed from robotpkg, so I guess it's related to my installation somehow... Is there additionnal tests / information I can provide to help spotting where the issue(s) may come from ?

Thank you for your detailed answer. So if I understand correctly, this memory allocation issue only affects action models created in Python, and the fix would be to enable copying these action models in order to avoid conflicts in the data when running the computations in parallel.

Yes, it is correct.

The Crocoddyl examples arm_manipulation and quadrotor work fine but not the other ones. For instance when I run make examples-humanoid_taichi

All the examples are passing, with exception of example-python-double-pendulum. To be precise, the unittests that should fail, only under multithreading compilation, are:

pybinds-shooting
example-python-double-pendulum

It seems however that my problem is not related to multi-threading, since as I mentioned, I encounter exactly the same issues running the same programs when building without multi-threading mode. But these programs work fine when Crocoddyl is installed from robotpkg, so I guess it's related to my installation somehow... Is there additionnal tests / information I can provide to help spotting where the issue(s) may come from ?

I think you should use gdb to find where your issue is happening. This might help us to spot the problem. Another action could be to check which libraries are linked using ldd to be sure, for instance, that you don't have multiple copies of Pinocchio.
Btw, could you provide more details of your setup (OS, Pinocchio version, example-robot-data version, etc)?

Thank you for your advices. It appears that re-installing Crocoddyl from scratch solved the problem :
Test project /home/sebastien/crocoddyl/cmake
      Start  1: test_states
 1/39 Test  #1: test_states ...................................   Passed    0.15 sec
      Start  2: test_actuation
 2/39 Test  #2: test_actuation ................................   Passed    0.04 sec
      Start  3: test_activations
 3/39 Test  #3: test_activations ..............................   Passed    0.01 sec
      Start  4: test_costs
 4/39 Test  #4: test_costs ....................................   Passed    2.66 sec
      Start  5: test_cost_sum
 5/39 Test  #5: test_cost_sum .................................   Passed    4.10 sec
      Start  6: test_contacts
 6/39 Test  #6: test_contacts .................................   Passed    0.15 sec
      Start  7: test_impulses
 7/39 Test  #7: test_impulses .................................   Passed    0.11 sec
      Start  8: test_multiple_contacts
 8/39 Test  #8: test_multiple_contacts ........................   Passed    1.24 sec
      Start  9: test_multiple_impulses
 9/39 Test  #9: test_multiple_impulses ........................   Passed    1.24 sec
      Start 10: test_actions
10/39 Test #10: test_actions ..................................   Passed    0.03 sec
      Start 11: test_diff_actions
11/39 Test #11: test_diff_actions .............................   Passed    0.39 sec
      Start 12: test_friction_cone
12/39 Test #12: test_friction_cone ............................   Passed    0.01 sec
      Start 13: test_boxqp
13/39 Test #13: test_boxqp ....................................   Passed    0.01 sec
      Start 14: test_solvers
14/39 Test #14: test_solvers ..................................   Passed   32.31 sec
      Start 15: pybinds-states
15/39 Test #15: pybinds-states ................................   Passed    2.19 sec
      Start 16: pybinds-actuations
16/39 Test #16: pybinds-actuations ............................   Passed    2.00 sec
      Start 17: pybinds-actions
17/39 Test #17: pybinds-actions ...............................   Passed    3.74 sec
      Start 18: pybinds-shooting
18/39 Test #18: pybinds-shooting ..............................***Exception: Child aborted  0.83 sec
      Start 19: pybinds-solvers
19/39 Test #19: pybinds-solvers ...............................   Passed    2.57 sec
      Start 20: pybinds-costs
20/39 Test #20: pybinds-costs .................................   Passed    5.48 sec
      Start 21: pybinds-contacts
21/39 Test #21: pybinds-contacts ..............................   Passed    1.32 sec
      Start 22: pybinds-impulses
22/39 Test #22: pybinds-impulses ..............................   Passed    1.75 sec
      Start 23: pybinds-force_cost
23/39 Test #23: pybinds-force_cost ............................   Passed    0.79 sec
      Start 24: pybinds-friction_cone_cost
24/39 Test #24: pybinds-friction_cone_cost ....................   Passed    0.70 sec
      Start 25: pybinds-momemtum_cost
25/39 Test #25: pybinds-momemtum_cost .........................   Passed    2.99 sec
      Start 26: pybinds-impulse_com_cost
26/39 Test #26: pybinds-impulse_com_cost ......................   Passed    0.58 sec
      Start 27: pybinds-squashing
27/39 Test #27: pybinds-squashing .............................   Passed    0.26 sec
      Start 28: example-python-double_pendulum
28/39 Test #28: example-python-double_pendulum ................***Exception: Child aborted  0.63 sec
      Start 29: example-python-boxfddp_vs_boxddp
29/39 Test #29: example-python-boxfddp_vs_boxddp ..............   Passed    5.23 sec
      Start 30: example-python-quadrotor
30/39 Test #30: example-python-quadrotor ......................   Passed    0.23 sec
      Start 31: example-python-quadrotor_ubound
31/39 Test #31: example-python-quadrotor_ubound ...............   Passed    0.31 sec
      Start 32: example-python-arm_manipulation
32/39 Test #32: example-python-arm_manipulation ...............   Passed    0.47 sec
      Start 33: example-python-quadrupedal_gaits
33/39 Test #33: example-python-quadrupedal_gaits ..............   Passed    4.17 sec
      Start 34: example-python-quadrupedal_walk_ubound
34/39 Test #34: example-python-quadrupedal_walk_ubound ........   Passed    3.51 sec
      Start 35: example-python-bipedal_walk
35/39 Test #35: example-python-bipedal_walk ...................   Passed    2.10 sec
      Start 36: example-python-bipedal_walk_ubound
36/39 Test #36: example-python-bipedal_walk_ubound ............   Passed    3.95 sec
      Start 37: example-python-humanoid_manipulation
37/39 Test #37: example-python-humanoid_manipulation ..........   Passed   32.60 sec
      Start 38: example-python-humanoid_manipulation_ubound
38/39 Test #38: example-python-humanoid_manipulation_ubound ...   Passed   26.30 sec
      Start 39: example-python-humanoid_taichi
39/39 Test #39: example-python-humanoid_taichi ................   Passed   91.73 sec

95% tests passed, 2 tests failed out of 39

Total Test time (real) = 238.90 sec

The following tests FAILED:
	 18 - pybinds-shooting (Child aborted)
	 28 - example-python-double_pendulum (Child aborted)
Errors while running CTest

I did :
make clean
sudo make uninstall
rm -rf cmake
git pull
git submodule update --init
cd cmake/
cmake .. -DBUILD_WITH_MULTITHREADS=ON
make
make install

My best guess is that I must have forgotten to clean the cmake directory in prior installations...

Yes, sometimes cmake clean is not enough. I always recommend to to rm -rf *.
Since everything seems to be resolved, could you close this issue?


[AP-3423] - Add GovukComponents gem

Pretty much every view in the application leans primarily on GOV.UK Design
System components.
This introduces the govuk-components gem to help us manage this complexity.
Rather than relying on copy-paste to ensure we render the design system
components accurately, we can render these resuable components. They are well
tested, customisable, and should reflect the design system accurately.
Link to story

Let me know what you think about this.
The gem is actively maintained and used across several services in DfE-Digital.
It will take a while to migrate entirely, but it's something that can be done incrementally and alongside other work. I would be tempted to migrate one component at a time, but it might make more sense from a UAT perspective to migrate by template.
Personally I think it's a no brainer in terms of shifting complexity and load outside the scope of our application. I'd expect it to reduce the maintenance load of the application and improve developer happiness, but I'm keen to hear other people's opinions too.

I suggested introducing this gem during a developer time meeting, card here

Thanks. I'm not sure who manages access to that Trello board, but I've requested access.

we should perhaps consider writing up tickets to move "bits" to using it now or wait to discuss in another developer time meeting, particularly with @RoseSAK - who was not present during the initial discussion

Agreed, the acceptance criteria of the task urges us to make a decision one way or the other:

"Done when a decision has been taken whether to proceed (subsequent tasks added), or not (pull request closed)."


chore(DEV-19412): [TimePicker] Add html native input time

Add HTML native input time
If the component receives the prop useNativeInput, a native time input will be rendered.
If the component DOES NOT receive the prop useNativeInput, an andt time component will be rendered.
PR Author Checklist

[x] PR title adheres to Conventional Commits
[x] I have finished reviewing the contents of this PR for accuracy
[x] If needed, I have included a Changeset and it follows Semantic Versioning principles
[ ] I have completed all of the above and have enabled auto-merge for this PR

Messages

:book:

packages/cascara/src/atoms/TimePicker/TimePicker.fixture.js: This fixture has been changed.

  Generated by :no_entry_sign: dangerJS against 83401d820b251d1677ca6dfd6f9badf4c95c6319


RFC: Adjust to eltype changes in DataArrays where the function now returns Union{T,NAtype}

These are the adjustments required for DataFrames's tests to pass after the changes in https://github.com/JuliaStats/DataArrays.jl/pull/280

No longer relevant?

Only relevant if somebody wants to make another (last?) release based on DataArrays.

Probably not worth it at this point. The DataArrays tag that introduces the eltype change still hasn't been merged, as it's expected to be breaking and folks wanted to know the extent of the breakage first. If we're moving away from DataArrays entirely, our last DataArrays-based release should be with DataArrays as it is now, perhaps upper bounded on 0.7.0. That would also make this PR obsolete.

I'm not sure whether the changes introduced in DataArrays 0.7.0 will be user-facing after this PR, but if so, it's likely not worth the churn to then change it again as we adopt Nulls.

We could still imagine replacing NA with null in DataArrays, and then tag a release of both the new DataArrays and DataFrames based on it, at the same time as a release based on master. That way people who need the performance of DataArray on Julia 0.6 could have it, while starting to port some of their code to Nulls.
Of course that's also a lot of work for an unclear gain, so I guess that depends on somebody being directly interested in it (maybe @andreasnoack is).

I believe @iamed2 was particularly interested in Union element types in DataArrays (i.e. Andreas' "stop lying" PR), so perhaps it would be good to hear from Eric as well from a user's perspective.

This isn't important if the ecosystem can stabilize around Nulls soon. And we only care about DataArrays insofar as they're the array type for DataFrames. If they stop being that, they're irrelevant to us.

Great, thanks for the input. I'm not sure it's worth churning DataArrays if we aren't going to use them here anymore. But the upside of no longer using them here is that we can update DataArrays independently of DataFrames, so we can move to Nulls there later if we want. (Assuming DataArrays will still be relevant once things settle on Nulls, which I imagine it won't be.)

Fine with me but has anybody made any benchmarks comparing operations on Array{Union{T,Null},N} to DataArray{T,N}? Just curious.

Just two small examples for now. (I'm testing DataArrays on 0.6 since DataStructures appears to be broken on master)
sum
# Union{Float64,Null}
julia> x = Union{Float64,Null}[rand() for i in 1:10^7];

julia> @time sum(x);
  0.065898 seconds (65.54 k allocations: 1.000 MiB)

# DataArrays
julia> y = @data rand(10^7);

julia> @time sum(x);
  0.007327 seconds (9 allocations: 320 bytes)

broadcast
# Union{Float64,Null}
julia> x = Union{Float64,Null}[rand() for i in 1:10^7];

julia> @time broadcast(+, x, 1);
  0.447139 seconds (20.00 M allocations: 381.479 MiB, 35.30% gc time)

julia> x[end] = Null()
null

julia> @time broadcast(+, x, 1);
  1.849494 seconds (40.00 M allocations: 762.941 MiB, 27.76% gc time)

# DataArrays
julia> y = @data rand(10^7);

julia> @time broadcast(+, y, 1);
  0.057529 seconds (61 allocations: 78.681 MiB, 20.38% gc time)

julia> y[end] = NA
NA

julia> @time broadcast(+, y, 1);
  0.054045 seconds (61 allocations: 78.681 MiB, 19.61% gc time)

Brutal

Having a brief glance, I imagine most of the cost is https://github.com/JuliaLang/julia/issues/23338, i.e. nothing getting inlined anywhere so the dynamic dispatch is killing performance.

@quinnj It seems to be allocating a lot of memory too though.
julia> using BenchmarkTools

julia> using Nulls

julia> using DataArrays

julia> x = Union{Float64,Null}[rand() for i in 1:10^7];

julia> y = @data rand(10^7);

julia> @benchmark sum($x)
BenchmarkTools.Trial:
  memory estimate:  763.94 MiB
  allocs estimate:  50065533
  --------------
  minimum time:     494.707 ms (5.28% GC)
  median time:      505.274 ms (5.27% GC)
  mean time:        506.101 ms (5.28% GC)
  maximum time:     526.756 ms (5.40% GC)
  --------------
  samples:          10
  evals/sample:     1

julia> @benchmark sum($y)
BenchmarkTools.Trial:
  memory estimate:  112 bytes
  allocs estimate:  2
  --------------
  minimum time:     5.421 ms (0.00% GC)
  median time:      5.887 ms (0.00% GC)
  mean time:        6.083 ms (0.00% GC)
  maximum time:     13.873 ms (0.00% GC)
  --------------
  samples:          819
  evals/sample:     1

Couldn't we just repurpose DataArrays to operate on Union{T, Null} instead?

Oh yes, it definitely will on 0.6; isbits Union arrays having their elements stored inline is 0.7 only.

Union{Null, T} performance is 10x better on Julia 0.7 compared with 0.6 for sum, and 2x better for broadcast without nulls (no clear difference when there's a null value). So the difference with DataArray is already less of an issue there, and could probably be reduced even more. But it would still be problematic to release the new Nulls-based DataFrames on Julia 0.6 given the poor performance.

Just to avoid confusion, my timings of Union{Null, T} were on 0.7 (from yesterday). Only the DataArrays timings were made on 0.6.

Ah, OK. So that would be even worse on 0.6. :-/

A possible intermediate solution would be to port DataArrays to Nulls (really, this just implies deprecating NA in favor of Null; PooledDataArray could also be deprecated in favor of CategoricalArray), and check that they are supported in DataFrames master, at least for most operations. Since DataFrames is now column type agnostic, it should already work, except that when a new column is created (concatenation, merge) an Array{Union{Null, T}} is created. We could add some temporary promotion rules so that DataArray wins. Then people could keep using DataArrays if they care about performance.

See https://github.com/JuliaStats/DataArrays.jl/pull/288 for a DataArrays port to Nulls.

For reference, things have of course improved a lot in 1.0, but we've still regressed a lot. We should probably file issues to track progress for these specific cases.
On Julia 0.6:
julia> x = DataArray(Union{Float64,Missing}[rand() for i in 1:10^7]);

julia> @btime sum(x);
  5.518 ms (2 allocations: 128 bytes)

julia> @btime broadcast(+, x, 1);
  53.678 ms (70 allocations: 78.68 MiB)

julia> x[end] = missing
missing

julia> @btime broadcast(+, x, 1);
  54.128 ms (70 allocations: 78.68 MiB)

julia> x[1] = missing
missing

julia> @btime broadcast(+, x, 1);
  52.828 ms (70 allocations: 78.68 MiB)

On Julia 1.0:
julia> x = Union{Float64,Missing}[rand() for i in 1:10^7];

julia> @btime sum(x);
  31.034 ms (1 allocation: 16 bytes)

julia> @btime broadcast(+, x, 1);
  127.155 ms (8 allocations: 76.29 MiB)

julia> x[end] = missing
missing

julia> @btime broadcast(+, x, 1);
  485.791 ms (19999498 allocations: 467.29 MiB)

julia> x[1] = missing
missing

julia> @btime broadcast(+, x, 1);
  734.313 ms (9 allocations: 85.83 MiB)

Closing anyway since the solution won't come from this PR.


Fix a false positive for Layout/ClosingParenthesisIndentation

This PR fixes a false positive for Layout/ClosingParenthesisIndentation when using keyword arguemnts.

Before submitting the PR make sure the following are checked:

[x] The PR relates to only one subject with a clear title and description in grammatically correct, complete sentences.
[x] Wrote good commit messages.
[ ] Commit message starts with [Fix #issue-number] (if the related issue exists).
[x] Feature branch is up-to-date with master (if not - rebase it).
[x] Squashed related commits together.
[x] Added tests.
[x] Ran bundle exec rake default. It executes all tests and runs RuboCop on its own code.
[x] Added an entry (file) to the changelog folder named {change_type}_{change_description}.md if the new code introduces user-observable changes. See changelog entry format for details.

Thanks!


Change to separate customizations for formatters

Hi!
I just stumbled on issue #131 and thought about your comment:

Wow, we should really find a way to make it easier to configure format-all.

When using Emacs' customization dialogs, currently a huge list of formatters is saved. Without looking at your code, I see the following problem:

it's not straight-forward to customize a single formatter
users with customized formatters won't be able to see and use formatters newly introduced by updates
or you'll have to add a lot of code to ensure that 2. does not happen

In other words, you have acquired some code debt. :smile:
I therefore suggest that you move from format-all-default-formatters to separate customizations such as format-all-formatter-python. This sounds like a nice task for LISP macros, although I'm not too familiar with elisp.
Best,
Martin

I agree that's the best move.

format-all-default-formatter-for-python
format-all-default-formatter-for-ruby
...

That lets people choose from a menu as well. The Python setting can have a menu of black and yapf.

format-all-formatters should still stay for fine-grained control.

Still haven't got around to this :-/ If anyone wants to try their hand at a PR, please let me know.


Confusing verbatim/ causes many SEVA in css

Hello guys,
In this command PS C:> $newcert = New-AdfsAzureMfaTenantCertificate -TenantId <tenant id such as contoso.onmicrosoft.com> -Renew $true
The tenant id format used in the sample can be misleading, many customers used the tenant with numeric format.
This can lead the customers to create a new certificate with an incorrect format which causes service disruption.
The following commands can be used to check what is your current configuration.
Connect-MsolService
Export-AdfsAuthenticationProviderConfigurationData -Name AzureMfaAuthentication -FilePath
I would be nice if you could add a note to clarify that the tenant format can change depending on the original set up.
Thanks for checking...
Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: f333e8cb-c046-ee06-d33d-d7dc8e8af695
Version Independent ID: 10838695-f60e-42a7-b0e8-eed7820fd333
Content: Configure AD FS 2016 and Azure MFA
Content Source: WindowsServerDocs/identity/ad-fs/operations/configure-ad-fs-and-azure-mfa.md
Product: windows-server
Technology: ad-fs
GitHub Login: @billmath
Microsoft Alias: billmath

This is directly related to my post - #6774 - which generated a SEV A when our certificate for Azure MFA link for ADFS expired - even though I followed the directions and there were no errors.  See support cases<PHONE_NUMBER>007432 and<PHONE_NUMBER>008145 (updated by @pmesen) for additional info.

@jweber14580 - Were you able to resolve the issue with certificates before this expired?
I believe we're about to face the same issue. A tech mistakenly created a new certificate with the tenantID, then created anther new certificate with the onmicrosoft.com domain, but no automatic rollover has occurrecd >4 days since they were created.

We hugely appreciate your contribution, feedback, and support to our technical documentation.
We’ve phased out GitHub Issues on this repository as the feedback mechanism for content and have replaced it with a new feedback system. For more information see, Content User Feedback. As part of this rollout, we are migrating your issue where our team will use your comments to improve the documentation experience. Unfortunately, this does mean you won't be able to access this issue after the migration is complete.
All issues will be migrated by Thursday 29th August 2024.
Once again, thank you for your valuable feedback, we hope you’ll continue to contribute using our new feedback system.


Move objects from BaseApp to OIOUBL [DK]

We are trying to install the OIOUBL extension in a W1 database, but for some reason there are 2 objects that are placed in the BaseApp in the DK localization. Shoudnt they be placed in the Extension OIOUBL? (or if shared with other danish extensios, it should be placed in a "BaseAppDK" extension that they can depend on).
The objects are
OIOUBLProfile.Table.al
VATReconciliation.Report.al

No problems found on install OIOUBL extension over W1 database. As for mentioned DK objects:

OIOUBLProfile.Table.al - is an obsolete table
VATReconciliation.Report.al is not related to OIOUBL


mWeb - Settings - Sign out  option not visible in smaller size devices

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Expected Result:
Sign out button is visible
Actual Result:
Sign out button isn't visible
Action Performed:

Go to https://staging.expensify.cash
Log in with any account
Go to Setting
Try to Log out

Workaround:
Unknown
Platform:
Where is this issue occurring?
Web
iOS
Android
Desktop App
Mobile Web ✔
Version Number: 1.0.51-0
Logs: https://stackoverflow.com/c/expensify/questions/4856
Notes/Photos/Videos: Issue reproduced in iPhone 6S and iPhone8
https://user-images.githubusercontent.com/43996225/119203636-891e7200-ba61-11eb-86bc-47af012c7936.mp4

Expensify/Expensify Issue URL:
View all open jobs on Upwork

Triggered auto assignment to @bfitzexpensify (AutoAssignerTriage), see https://stackoverflow.com/c/expensify/questions/4749 for more details.

Seems like we just need to make sure this page is in a scrollView, much like we do with other pages in Settings.

Issue reproducible today during KI retests

Upwork posting.

Triggered auto assignment to @johnmlee101 (Exported), see https://stackoverflow.com/c/expensify/questions/7972 for more details.

PROPOSAL
We can fix this by removing pointerEvents="box-none" at this point https://github.com/Expensify/Expensify.cash/blob/9f305176681385f2aa80f43de06fc6dffa7f0b83/src/pages/settings/InitialPage.js#L112
Thanks

Okay, I can make the sidebar scrollable to view the signout button.

@aliabbasmalik8 your proposal looks good.

@aliabbasmalik8 your proposal looks good.

can I start work?

Let's wait for @bfitzexpensify to accept the proposal on his end

Thanks @aliabbasmalik8, hired you on Upwork! Feel free to get started on a PR.

FIxed 🎉


Move SetContextMessageConsumer into libspirv namespace

Avoid polluting the global namespace.

Rebased, squashed, and pushed into master as c2999273


Possible deadlock condition

Hello,
I've noticed that the number of goroutines my application uses with discordgo seems to climb without coming back down.  Attached are some metrics from Prometheus showing the goroutines growing, as well as pprof output of the goroutines (zipped because Github comments do not support embedding SVGs).
I'm not entirely sure, but I think it might be somewhere around restapi.RequestWithLockedBucket...maybe there is a path that doesn't unlock the bucket?  I don't immediately see where that may be the case however.

pprof output: pprof001.svg.zip
The callback code from my project can be found here, if that's useful at all: https://github.com/ewohltman/ephemeral-roles/blob/master/pkg/callbacks/voiceStateUpdate.go
Thanks for your time and awesome library!

For additional information, this was observed on both the master and develop branches, using Go 1.13.

After further research and testing, I believe the issue was actually in my implementation and that discordgo is doing what it is supposed to with rate limiting.
Thanks again for a great library!


Mutation cache does not contain the mutationFn key

Describe the bug
Hi!
I'm currently implementing offline support for my application using the v4 version but I'm having a problem with the mutation cache.
It seems that in the hydration.js file, the mutationFn is returned in the dehydrateMutation function. This is causing the warning No mutationFn found.

Expected behavior
I expect my mutation to be cached correctly, and I expect the mutation function to be executed when resumed.
How often does this bug happen?
Every time
Screenshots or Videos
No response
Platform
iOS, Android
react-query version
4.0.0-beta.5
TypeScript version
No response
Additional context
No response

The problem is a conceptual one: How would you serialize a function to JSON, so that you can store it in e.g. localStorage? You cannot :)
The workaround is to use a default mutationFn for all mutations that you want to re-run after persistence.
I have documented this limitation in the docs here, and the offline example also uses this approach.

Right! ;) Then we will need to implement this for all required offline calls. Thanks!

I'm still getting the No mutationFn found warning.
Here's the code:
// App.tsx
const asyncStoragePersister = createAsyncStoragePersister({
  storage: AsyncStorage,
  throttleTime: 1000,
});

const queryClient = new QueryClient({
  defaultOptions: {
    queries: {
      cacheTime: 1000 * 60 * 60 * 24, // 24 hours
      staleTime: 2000,
      retry: 0,
    },
  },
});

queryClient.setMutationDefaults(trackKeys.all(), {
  mutationFn: (params) => {
    return track(params);
  },
});

return (
  <PersistQueryClientProvider
    client={queryClient}
    persistOptions={{ persister: asyncStoragePersister }}
    onSuccess={() => {
      console.log('PersistQueryClientProvider.onSuccess');
      queryClient
        .resumePausedMutations()
        .then(() => queryClient.invalidateQueries());
    }}
  >
     ...
  </PersistQueryClientProvider>
)

// useTrackMutation

export const trackKeys = {
  all: () => ['track'],
};

function useTrackMutation(options?: Options) {
  return useMutation({
    mutationKey: trackKeys.all(),
    ...options,
  });
}

Any idea of what's missing?
I'm using React native and @react-native-async-storage/async-storage for storage.

@TkDodo My bad, I actually have the QueryClient outside the render function. I updated the code

@TkDodo I'm following this tutorial that it is pretty similar to React Query documentation.
So what I'm doing to test this is that I go offline, I call the mutation, and then close and open the app. That's where I get the warning.
I logged the AsyncStorage content and this is what I got:
{
  "REACT_QUERY_OFFLINE_CACHE": {
    "content": {
      "buster": "",
      "timestamp":<PHONE_NUMBER>320,
      "clientState": {
        "mutations": [
          {
            "state": {
              "error": null,
              "failureCount": 0,
              "failureReason": null,
              "isPaused": true,
              "status": "loading"
            }
          },
          {
            "state": {
              "error": null,
              "failureCount": 0,
              "failureReason": null,
              "isPaused": true,
              "status": "loading"
            }
          },
          {
            "state": {
              "error": null,
              "failureCount": 0,
              "failureReason": null,
              "isPaused": true,
              "status": "loading"
            }
          }
        ],
        "queries": [
          {
            "state": {
              "data": "Hugo",
              "dataUpdateCount": 15,
              "dataUpdatedAt":<PHONE_NUMBER>923,
              "error": null,
              "errorUpdateCount": 0,
              "errorUpdatedAt": 0,
              "fetchFailureCount": 0,
              "fetchFailureReason": null,
              "fetchMeta": {},
              "isInvalidated": false,
              "status": "success",
              "fetchStatus": "idle"
            },
            "queryKey": [
              "async-storage",
              "vin"
            ],
            "queryHash": "[\"async-storage\",\"vin\"]"
          }
        ]
      }
    }
  }
}

I don't know if it is normal that the mutations don't have a query key even when I'm setting one.

where does this tutorial actually attach a mutationKey to the mutation? Searching for mutationKey on that page yields 0 results ...
FYI, our offline example does this here:
https://github.com/TanStack/query/blob/53813e45e126dcd8f846addf69c5b78322f868b8/examples/react/offline/src/movies.js#L23

where does this tutorial actually attach a mutationKey to the mutation? Searching for mutationKey on that page yields 0 results ...
FYI, our offline example does this here:
https://github.com/TanStack/query/blob/53813e45e126dcd8f846addf69c5b78322f868b8/examples/react/offline/src/movies.js#L23

You're right they don't do it in the tutorial but I found the source code and they do set a mutationKey
https://github.com/benoitpaul/react-native-offline-tanstack-query/blob/main/api.ts#L165
My use case is pretty similar to theirs with their addTodoWithIdMutationFn function, except that I don't need to store anything in the cache and I don't have any queries, I just need to send data to the backend and that's it

Have a look at our offline example. It works there.


Installation errors (execution policy and configuration command not found)

I followed the Installation instructions from https://psdatabaseclone.org/docs/getting-started/configuration/
When I get to the step to configure with Set-PSDCConfiguration, I get the following errors:
PS C:\WINDOWS\system32> Set-PSDCConfiguration
Set-PSDCConfiguration : The 'Set-PSDCConfiguration' command was found in the module 'PSDatabaseClone', but the module
could not be loaded. For more information, run 'Import-Module PSDatabaseClone'.
At line:1 char:1
+ Set-PSDCConfiguration
+ ~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (Set-PSDCConfiguration:String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CouldNotAutoloadMatchingModule

PS C:\WINDOWS\system32> Import-Module PSDatabaseClone
Import-Module : The following error occurred while loading the extended type data file: , C:\Program
Files\WindowsPowerShell\Modules\PSFramework\1.1.59\xml\PSFramework.Types.ps1xml: The file was skipped because of the
following validation exception: File C:\Program
Files\WindowsPowerShell\Modules\PSFramework\1.1.59\xml\PSFramework.Types.ps1xml cannot be loaded. The file C:\Program
Files\WindowsPowerShell\Modules\PSFramework\1.1.59\xml\PSFramework.Types.ps1xml is not digitally signed. You cannot
run this script on the current system. For more information about running scripts and setting execution policy, see
about_Execution_Policies at https:/go.microsoft.com/fwlink/?LinkID=135170..
At line:1 char:1
+ Import-Module PSDatabaseClone
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (:) [Import-Module], RuntimeException
    + FullyQualifiedErrorId : FormatXmlUpdateException,Microsoft.PowerShell.Commands.ImportModuleCommand

Setting execution policy with Set-ExecutionPolicy RemoteSigned will work to get past that error, but the instructions should say that this policy is assumed or provide a workaround.
That being said, after setting execution policy, I can't actually run the command; it's not recognized:
> Set-PSDCConfiguration
Set-PSDCConfiguration : The term 'Set-PSDCConfiguration' is not recognized as the name of a cmdlet, function, script
file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct
and try again.
At line:1 char:1
+ Set-PSDCConfiguration
+ ~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (Set-PSDCConfiguration:String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException

Seeking help on what I should try next :)

I have not been able to reproduce the error on any of my machines.
I even made a complete new VM and tried to install it with no problems at all.
If it's ok with you, we can maybe setup a call or something where I can check some things on your computer to see where this is coming from.

@sanderstad I got sidetracked with some things, sorry. Let me try installation again and document clearly my findings - I've got both a home machine and a work laptop that I was trying this on, and IIRC was getting different results based on which machine I was running. Let me get that extra data for you, and if after that it's still funky, we can try to set up some communication :)

I'm also having issues with importing and running. After installing the module in the CurrentUser scope, I cannot import the module by name.
Importing the psm1 file directly, I'm told that importing requires elevation.
When I hop to an elevated PowerShell prompt, I can import the psm1 file. However, running set-psdcconfiguration throws an error:

Test-Path : Cannot bind argument to parameter 'Path' because it is an empty string.
At line:218 char:35
+         if (-not (Test-Path -Path $Path)) {
+                                   ~~~~~
    + CategoryInfo          : InvalidData: (:) [Test-Path], ParameterBindingValidationException
    + FullyQualifiedErrorId : ParameterArgumentValidationErrorEmptyStringNotAllowed,Microsoft.PowerShell.Commands.Test
   PathCommand

This is version 0.3.135. $PSVersion 5.1.14393.3471 on Windows Server 2016

@sanderstad So here's some more data. I completely uninstalled my previous attempts and tried to document to the best of my abilities the commands run. I ran this on my desktop and laptop with the same results. I should also note that I ran this on my wifes desktop as well as her month-old laptop (so it's as "from scratch" as I can get without finding and loading a pristine VM). Here's the output of what was run on my desktop (forgive the long list of Azure modules when querying for installed modules). In it you'll see the current windows version (would vary slightly based on what machine I'm running), execution policy (I set to RemoteSigned before doing these), before and after installation with version numbers noted, the error that happens when trying to call Set-PSDCConfiguration, and the peculiar issue of Get-Modules not showing exported functions for PSDatabaseClone
https://gist.github.com/blitzmann/071fa6eba97c74c994315e9268cffbc2
I then started to go down the list of versions noted here: https://www.powershellgallery.com/packages/PSDatabaseClone/ from most recent. All of them produced the same issues until I installed 0.3.132, which gave a new error when attempting to set the configuration:
https://gist.github.com/blitzmann/6f5aef751d4d3f6dc452a6242f2bc62b
This however did generate some files in my current working directory (C:). Using the -Path option, I was able to generate the files without error (Set-PSDCConfiguration -InformationStore File -Path \\server\public). However, simply running  Set-PSDCConfiguration does not give the guided configuration as the image shows in https://psdatabaseclone.org/docs/getting-started/configuration/ (I'm aware tho that these screenshots may be from a newer version, which wont load)
Again, the above steps were performed on my desktop PC, personal laptop, wifes computer, and wifes laptop, with very similar results (only difference being minor Windows version build numbers). No version higher than 0.3.132 seemed to want to load the module when calling Set-PSDCConfiguration.
Ninja edit:
I also loaded up a Windows Server 2012 R2 evaluation image downloaded directly from Microsoft (2012 as this is the server version I'm planning on eventually implementing this on) using VirtualBox. I had to install Powershell 5 via Windows Management Framework 5.1 to be able to install PSDatabaseClone via the Install-Module (which is the broken installation method in question). I ran into the same issues as described above tho - all versions later than 0.3.132 just don't load and show no exported functions via Get-Module. I installed 0.3.132 and, while I did still get errors about Path variables not being set (which I'm not going to worry about right now until I can test the latest version), I was able to run the configuration against the SQL Server and successfully image my AdventureWorks2014 database.
I am running into cloning issues tho, but will investigate further and create new issues if support is needed as they aren't related to the initial installation and configuration :)

@blitzmann Is this still active due to all the changes that we did the last few days?

@sanderstad seems like installation from PSGallery is still not working correctly for latest version (0.3.136). Tried to install on my work laptop, and Get-Module still shows nothing getting exported for PSDatabaseClone
PS C:\WINDOWS\system32> get-installedmodule

Version    Name                                Repository           Description
-------    ----                                ----------           -----------
1.0.91     dbatools                            PSGallery            The community module that enables SQL Server Pro...
1.1.59     PSFramework                         PSGallery            General Scripting Framework, providing PowerShel...

PS C:\WINDOWS\system32> Install-Module -Name PSDatabaseClone

Untrusted repository
You are installing the modules from an untrusted repository. If you trust this repository, change its
InstallationPolicy value by running the Set-PSRepository cmdlet. Are you sure you want to install the modules from
'PSGallery'?
[Y] Yes  [A] Yes to All  [N] No  [L] No to All  [S] Suspend  [?] Help (default is "N"): y
PS C:\WINDOWS\system32> get-installedmodule

Version    Name                                Repository           Description
-------    ----                                ----------           -----------
1.0.91     dbatools                            PSGallery            The community module that enables SQL Server Pro...
0.3.136    PSDatabaseClone                     PSGallery            Module for cloning SQL Server databases
1.1.59     PSFramework                         PSGallery            General Scripting Framework, providing PowerShel...

PS C:\WINDOWS\system32> get-module

ModuleType Version    Name                                ExportedCommands
---------- -------    ----                                ----------------
Manifest   <IP_ADDRESS>    Microsoft.PowerShell.Management     {Add-Computer, Add-Content, Checkpoint-Computer, Clear-Con...
Manifest   <IP_ADDRESS>    Microsoft.PowerShell.Utility        {Add-Member, Add-Type, Clear-Variable, Compare-Object...}
Binary     <IP_ADDRESS>    PackageManagement                   {Find-Package, Find-PackageProvider, Get-Package, Get-Pack...
Script     <IP_ADDRESS>    PowerShellGet                       {Find-Command, Find-DscResource, Find-Module, Find-RoleCap...
Script     2.0.0      PSReadline                          {Get-PSReadLineKeyHandler, Get-PSReadLineOption, Remove-PS...

PS C:\WINDOWS\system32> Set-PSDCConfiguration
Set-PSDCConfiguration : The term 'Set-PSDCConfiguration' is not recognized as the name of a cmdlet, function, script
file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct
and try again.
At line:1 char:1
+ Set-PSDCConfiguration
+ ~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (Set-PSDCConfiguration:String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException

PS C:\WINDOWS\system32> get-module

ModuleType Version    Name                                ExportedCommands
---------- -------    ----                                ----------------
Script     1.0.91     dbatools                            {Select-DbaObject, Set-DbatoolsConfig, Add-DbaAgDatabase, ...
Manifest   <IP_ADDRESS>    Microsoft.PowerShell.Management     {Add-Computer, Add-Content, Checkpoint-Computer, Clear-Con...
Manifest   <IP_ADDRESS>    Microsoft.PowerShell.Utility        {Add-Member, Add-Type, Clear-Variable, Compare-Object...}
Binary     <IP_ADDRESS>    PackageManagement                   {Find-Package, Find-PackageProvider, Get-Package, Get-Pack...
Script     <IP_ADDRESS>    PowerShellGet                       {Find-Command, Find-DscResource, Find-Module, Find-RoleCap...
Manifest   0.3.136    PSDatabaseClone
Script     1.1.59     PSFramework                         {ConvertTo-PSFHashtable, Invoke-PSFCallback, Invoke-PSFPro...
Script     2.0.0      PSReadline                          {Get-PSReadLineKeyHandler, Get-PSReadLineOption, Remove-PS...

PS C:\WINDOWS\system32>

It's weird, PowerShell knows that Set-PSDCConfiguration should come from PSDatabaseModule, so it imports that, but then the function isn't available. Could it have to do with how things are being imported via the psm1 file? (I know nothing of how PS imports files 😞 )

Psgallery will only be updated when the master branch is merged with development.
Make sure you have a clean powershell window when importing from source
You can either import the psm1 or the psd1. It shouldn't matter. I use both methods all the time

Yeah, I'm able to import via my development source directory with Import-Module .\PSDatabaseClone.psm1 -Force just like you describe. That's how I've been running it for testing / development.
The issue is getting it to work through the module download via PSGallery, even through a clean shell. This should "just work" but for some reason doesn't seem to. The fact that the exported functions is blank when running Get-Module just let me to think maybe it's an issue with getting the functions in the first place.

Just to be clear, the PSGallery is not the same as what you've been testing.
The PSGallery only updates after a successful build when master gets merged with development. That has not yet happened because I only do the merge to master when things are tested.

Ah, sorry, I misread. I saw that 0.3.136 was released a day ago, and thought that it was dev being merged in. But yes, I'm aware the stuff I run locally isn't the same as whats on PSGallery.
Back to the original question:

Is this still active due to all the changes that we did the last few days?

Yes this issue is still active and unresolved. This particular issue relates to the fact that, when installed via PSGallery, the module doesn't load the functions correctly on any of the environments that I've tested it on. 😃  But, since most of the stuff that has been worked over the few days isn't on master, I wouldn't have expected the module that's on PSG to have changed much  😄

So if you load the module using development using
Import-Module .\PSDatabaseClone.psm1 -Force

Is the problem resolved? This is what powershell does in the background when you run
Import-Module PSDatabaseClone

@alevyinroc if you clone the repo and checkout the development branch, your problem should no longer occur.
Can you verify that?

The module is renamed as mentioned in #87
Can you please remove the module PSDatabaseClone from your computer and run
Install-Module dbaclone -Scope CurrentUser

So if you load the module using development using
Import-Module .\PSDatabaseClone.psd1 -Force

Is the problem resolved? This is what powershell does in the background when you run
Import-Module PSDatabaseClone

Yes, loading it through Import-Module .\PSDatabaseClone.psd1 -Force  works (which is loading form the git repo), but loading it through Import-Module PSDatabaseClone does not (which is from PSGallery). That's the problem, for some reason the PSGallery version just doesn't work - doesn't export any functions according to Get-Module
That being said, installing dbaclone from PSGallery works! First try! Fucking bizzare ¯_(ツ)_/¯ Confirmed on my home PC as well
Thanks for your time trying to troubleshoot with me, even though the actual cause is still a mystery lol

@sanderstad btw, do you have a chat like Gitter or Slack for development purposes? Someplace that is less formal than github issues to ask questions that pop up about things?

@blitzmann sure, you can find me on the sqlcommunity.slack.com channel
my name there is @sqlstad

The problems are fixed in the new release on PSGallery.
Please use dbaclone instead of PSDatabaseClone

The new module in the gallery is getting further for me, but I'm experiencing new errors when running functions from it. When I can re-test at work, I'll check open issues and my usage then open a new issue if warranted.


Charges page blank, timeline show null kwh

Installed a few days ago. 1.23.4.
I've done some sample charges just to see if I can get anything to show up in the charges section.
E.g. from the timeline

but if I go into the charges screen:

Appropriate log:
2021-06-23 13:00:04.742 car_id=1 [info] Start / :online
2021-06-23 13:00:04.761 car_id=1 [info] Connecting ...
2021-06-23 13:00:17.937 car_id=1 [info] Charging detected: -1 kW
2021-06-23 13:00:19.008 [info] GET https://nominatim.openstreetmap.org/reverse -> 200 (118.851 ms)
2021-06-23 13:00:19.025 car_id=1 [info] Charging / SOC: 81% / Home
2021-06-23 13:00:19.025 car_id=1 [info] Disconnecting ...
2021-06-23 13:00:58.844 [info] GET /
2021-06-23 13:00:58.847 [info] Sent 200 in 2ms
2021-06-23 13:00:58.862 [info] GET /
2021-06-23 13:00:58.865 [info] Sent 200 in 2ms
2021-06-23 13:01:06.659 car_id=1 [info] Charging / Stopped / 0.00 kWh – 1 min
2021-06-23 13:01:06.669 car_id=1 [info] Start / :online
2021-06-23 13:01:06.682 car_id=1 [info] Connecting ...
2021-06-23 13:02:37.338 [info] GET /
2021-06-23 13:02:37.340 [info] Sent 200 in 2ms
2021-06-23 13:03:26.645 car_id=1 [info] Charging detected: -3 kW
2021-06-23 13:03:26.766 car_id=1 [info] Charging detected: -4 kW
2021-06-23 13:03:26.766 car_id=1 [info] Fetch already in progress ...
2021-06-23 13:03:27.382 car_id=1 [info] Charging detected: -3 kW
2021-06-23 13:03:27.382 car_id=1 [info] Fetch already in progress ...
2021-06-23 13:03:27.526 [info] GET https://nominatim.openstreetmap.org/reverse -> 200 (101.885 ms)
2021-06-23 13:03:27.547 car_id=1 [info] Charging / SOC: 81% / Home
2021-06-23 13:03:27.547 car_id=1 [info] Disconnecting ...
2021-06-23 13:03:27.547 car_id=1 [info] Received stream data: %TeslaApi.Stream.Data{elevation: 112, est_heading: 46, est_lat: 52.<snip>, est_lng: -1.<snip> est_range: 233, heading: 45, odometer: 1659.5, power: -4, range: 269, shift_state: nil, soc: 81, speed: nil, time: ~U[2021-06-23 13:03:27.188Z]}
2021-06-23 13:06:48.269 car_id=1 [info] Charging / Stopped / 0.00 kWh – 3 min
2021-06-23 13:06:48.279 car_id=1 [info] Start / :online
2021-06-23 13:06:48.292 car_id=1 [info] Connecting ...
2021-06-23 13:09:54.116 car_id=1 [info] Suspending logging
2021-06-23 13:13:37.689 car_id=1 [info] Charging detected: -7 kW
2021-06-23 13:13:40.381 [info] GET https://nominatim.openstreetmap.org/reverse -> 200 (98.584 ms)
2021-06-23 13:13:40.398 car_id=1 [info] Charging / SOC: 81% / Home
2021-06-23 13:13:40.398 car_id=1 [info] Disconnecting ...
2021-06-23 13:15:30.895 car_id=1 [info] Charging / Stopped / 0.00 kWh – 2 min
2021-06-23 13:15:30.905 car_id=1 [info] Start / :online
2021-06-23 13:15:30.918 car_id=1 [info] Connecting ...
2021-06-23 13:18:35.699 car_id=1 [info] Suspending logging
2021-06-23 13:24:51.277 [info] GET /settings
2021-06-23 13:24:51.281 [info] Sent 200 in 3ms
2021-06-23 13:42:50.513 car_id=1 [info] Fetching vehicle state ...
2021-06-23 13:42:51.585 car_id=1 [info] Start / :asleep
2021-06-23 13:42:51.598 car_id=1 [info] Disconnecting ...

Any suggestions on where to look to see what I'm doing wrong?

Charge sessions with 0.00 kWh added are hidden. That was a feature requests IIRC.

Thanks that makes sense. I had a (small) charge last night and that appeared to work.

Any idea why the kwh always shows as null?

Can you put the attached query into the timeline dashboard query?
timeline.zip

Yep, that worked! Thanks.

For completeness.

The changed dashboard won't save, but this confirms the revised query works.

Created PR https://github.com/adriankumpf/teslamate/pull/1748


Support a way of adding a tld at runtime?

I don't know if you think it is a good idea, but what you think about ability to temporarily extend PublicSuffix with a new tld?
For example - lets say locally I have a domain called "foo.dev" which is not a valid TLD, but yet if I can say:
PublicSuffix.add_tld = "dev"
domain =  PublicSuffix.parse("foo.dev")
.. no errors here

In that case we just do it like this:
[1] pry(main)> PublicSuffix.valid?('foo.dev')
=> false
[2] pry(main)> PublicSuffix::List.default << PublicSuffix::Rule.factory('dev')`
=> # ...
[3] pry(main)> PublicSuffix.valid?('foo.dev')
=> true

I don't think an alternative way of doing this is really necessary?

Correct, you can append a new items at runtime using <<.
The method words on a list instance, therefore you can either use the PublicSuffix::List.default or any other instance of a list.


Added hashSalt support

Signed-off-by: Marcus Noble<EMAIL_ADDRESS>What this PR does / why we need it
There are sometimes occasions when the hash suffix generated for immutable resources isn't updated when it needs to be. An example is when the template for that resource changes but not the values being used for that resource.
This update to the hash function allows for forcing a new hash to be generated by setting .Values.hashSalt to a random string.
/cc @erkanerol this might also be useful for CAPO / CAPVCD
Checklist

[x] Update changelog in CHANGELOG.md.

Trigger e2e tests
/test create

/test create

/test create


Ensure changing CDS columns do not trigger errors

In https://github.com/ioam/holoviews/pull/3158 I made a change to the way bokeh events are handled, which sent each bokeh model change as a separate event and thereby avoided an error triggered by one event breaking a plot irreparably because subsequent errors would be ignored. Unfortunately that change introduced a noticeable delay between a CDS data and color mapper ranges being updated, resulting in undesirable flicker on updates. It was also a bit of a hack around the actual issue because the underlying issue would still trigger errors. This PR reverts that change and fixes the underlying problem.
It does this through a fairly involved change, so I'll step through it slowly:

It detects whether a glyph dataspec references a CDS.data column which does not yet exist
If this is the case it triggers a new ModelChangedEvent which replaces the data with empty columns which do contain the new column(s) and the old column(s)
Once the glyph has been updated to reference the new column(s), it sends another event containing the actual data

Going to merge so I can continue working on other PRs that depend on this.


Disarm doesn't actually disarm (although it appears as disarmed in Home)

My "Arrive Home" scene includes turning off the alarm. To test, I set alarm to Stay, run scene, and alarm shows as off in Home app but is actually still in Stay mode. I don't know if it's a limitation of the API.

Are you setting the alarm to Stay via phone or the Alarm Panel? If the latter, that's expected because the Home app polls Alarm.com for changes. This polling only happens when you refresh or relaunch the app. There might be a timed poll running in the background, but I'm not close enough to the code to confirm.

I suspect you're right about the polling interval. I'll increase the disarm timer window and see what happens.


feat(ui): node events logs queue

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 88 (0.0%)  changed or added relevant lines in 4 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.2%) to 26.911%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

lib/SocketEvents.ts
0
1
0.0%

src/store/mutations.js
0
6
0.0%

src/lib/utils.js
0
8
0.0%

lib/ZwaveClient.ts
0
73
0.0%

Totals

Change from base Build<PHONE_NUMBER>:
-0.2%

Covered Lines:
3668

Relevant Lines:
14368

💛  - Coveralls

cc @AlCalzone @kpine @jmgiaever

First draft looks good. A few things I'd change:

[ ] Filter out statistics updated events. They get NOISY and are exposed through the statistics anyways.
[ ] Filter out metadata updated events. Most users won't understand what to do with them, and they are indirectly exposed through the values anyways
[ ] Put multiline arguments below the event title. Currently the first line is behind the title, making it very hard to grasp
[ ] Indent the stuff that belongs to each event slightly, so it is easier to grasp visually
[ ] Distuinguish separate events visually: A bit of padding, alternating BG colors, horizontal lines (?)

Stylistically, there should only be a : if there is more following. The READY line in your example shows a : without anything else, which could be confusing.

The hover styling is a bit too strong and distracts from viewing the data. I'd rather see something like a border around the contents, or just nothing at all. Perhaps a much lighter color if you prefer this still.
The event data looks better with a smaller line height.
The 'alive' event (and 'sleep', going by your screenshot) is showing an argument of 0, which doesn't seem right. In comparison, the 'ready' event shows nothing. I'd expect the former to be the same as 'ready', since they have the same structure. Or are the docs incorrect?
dead/alive: (node: ZWaveNode) => void
ready: (node: ZWaveNode) => void
I'm wondering if the max queue size of 100 is too large as a default value.

I noticed this but I dunno why it happens

Oh, I guess the docs are out of date, but the answer is already here. 😅 It's the old status.
https://github.com/zwave-js/zwavejs2mqtt/blob/4982a2fe1fc58e4519e65d33bb8ccc9093016a94/lib/ZwaveClient.ts#L3202

Can be configured in settings but I could limit it more, maybe 50 ?

Just thinking of someone with a lot of nodes and events. What impact would saving N nodes * M events in cache be (50 nodes, 5000 max events; 100 nodes, 10,000 max events)? Might be nothing to worry about. Also how does the UI function with 100 events shown? Easy and fast to use? The default should be the "safest" option. I don't have the answer, just posing the question. 😃

Also how does the UI function with 100 events shown? Easy and fast to use

I should try but I don't think it would be a problem

Just tried with 200 items the UI has zero problems to show them, I think the same for 500 items but 100 I think it's enought

I'd do a few stylistic changes, here's an example for dark mode:

Here are the corresponding CSS changes I made for reference. BG color might need to be tuned for light mode:


Problem with Plunker demo for DataTable

Bug, feature request, or proposal:
The plunker demo for Data Table is not loading properly
What is the expected behavior?
It should load without errors
What is the current behavior?
Tere is a 404 error when trying to load cdk.ts, some modules aren't loaded correctly in the main module
What are the steps to reproduce?
Click on the plunker link in the demo page : https://material.angular.io/components/table/overview
What is the use-case or motivation for changing an existing behavior?
Viewing demo code to implement the component
Which versions of Angular, Material, OS, TypeScript, browsers are affected?
Current (documentation)
Is there anything else we should know?
Here is the fixed version : https://plnkr.co/edit/gk9KLraXnvTbBNLcPdEq

Close this issue, looks like people are participating on: https://github.com/angular/material2/issues/5569

Head's up- every plunker example I've opened so far has failed to load.
Getting errors loading platform-browser and cdk for multiple components (md-button, select, spinner, etc)

Will continue on #5569


Twig Extension Migration

Let's move all the Twig Extension from _twig-components into the proper Twig Extensions found in packages/core-php/src/TwigExtensions/. All can be placed in either the BoltCore.php or BoltExtras.php files found in that directory. We're done when the build stops failing.

@EvanLovely there's a couple things to chat about (and related to) this work*, but for the most part, Pattern Lab is now fully switched over to using the new Class-based extensions architecture** and is compiling without any errors - huzzah!!
I freakin LOVE how much cleaner and much more manageable this new setup is netting out. Having the ability to have reusable functions / helper utils across your Twig extension logic ROCKS! 🤘
*Things to chat / follow up on: autoload.php for class name resolution and the hairier parts of PHP Class namespaces, getting a little help to get some of the manifest file path resolution logic rolled into some of these, questions about the current state of the low quality base64 placeholder logic + image srcset logic in PL, the ratio object appearing to be missing in a few places, and the Bolt Icon component names being hard coded >_<
**There's a couple extensions I've kept around that I suspect we might need for a few missing features (like globbing files on the file system) -- want to double check on these first before removing.

So this error:
PHP Warning:  require(/home/travis/build/bolt-design-system/bolt/packages/core-php/src/../vendor/autoload.php): failed to open stream: No such file or directory in /home/travis/build/bolt-design-system/bolt/packages/core-php/src/TwigFunctions.php on line 5

was caused by the presence of require __DIR__ . '/../vendor/autoload.php'; in a package that is symlinked (due to being a custom path repo in composer.json repositories in PL. The folder core-php/vendor doesn't exist. I initially moved \Bolt\Layout from psr-0 to psr-4 and also removed those require statements. I'm getting a new error now:
Uncaught Error: Class 'Bolt\Drupal\Core\Template\Attribute' not found in /home/travis/build/bolt-design-system/bolt/packages/core-php/src/TwigFunctions.php:17

I see those are also psr-0, I'm wondering if moving them to psr-4 will help.

Wahoo! We got it!

You'll see on my commit b646bec (Adding specific version number to core-php so composer.lock doesn't have the current git branch as version number) that I've added this to core-php/composer.json:
+"version": "0.9.0",

The reason is b/c composer.lock would have the version number as the current branch name... which, well... yeah: you get it.

Just wanted to call this out so you were aware.

how does PL know about our local extension in this new setup?

Via twigExtensions in PL's config.yml - and since those packages are require-ed in composer.json, it can call that public static function and get back a Twig Extension.

Right right right - was just confirming that https://github.com/bolt-design-system/bolt/blob/011ee14a017b360624986e6bc4b5300249c36c1f/apps/pattern-lab--workshop/composer.json#L39 also still existed so the symlink in the vendor folder was still happening.
The updated config.yml in PL ties in these new extensions being symlinked in the monorepo...

Yep, we're still symlinked!

I'm merging this in as the remaining questions we have are all outside the scope of this PR.


[DPE-5417] Add check to ensure peer databag populated before reconciling mysqld exporter pebble layers

Issue
If a relation with the metrics endpoint is created before the charm is allocated, it is possible that the metrics-endpoint-created event runs before the leader-elected hook. The metrics-endpoint-created handler needs to use mysql which relies on values set in the leader-elected handler. This results in the charm going into an error state from which it can never recover (the charm continually re-runs the metrics-endpoint-created handler until it exits successfully)
Fixes: https://github.com/canonical/mysql-k8s-operator/issues/504
Solution
Add a check to ensure that the values in the app peer databag are set (ensuring leader-elected is run first)
Testing
Deployed COS + kubeflow locally, ensured the issue was reproducible. Added a hotfix in this PR, and ensured that the unit comes up successfully
unit-katib-db-0: 13:04:07 INFO juju.worker.uniter awaiting error resolution for "relation-created" hook
unit-katib-db-0: 13:04:36 INFO juju.worker.uniter awaiting error resolution for "relation-created" hook
unit-katib-db-0: 13:04:39 ERROR unit.katib-db/0.juju-log metrics-endpoint:66: Uncaught exception while in charm code:
Traceback (most recent call last):
  File "/var/lib/juju/agents/unit-katib-db-0/charm/./src/charm.py", line 888, in <module>
    main(MySQLOperatorCharm)
  File "/var/lib/juju/agents/unit-katib-db-0/charm/venv/ops/main.py", line 551, in main
    manager.run()
  File "/var/lib/juju/agents/unit-katib-db-0/charm/venv/ops/main.py", line 530, in run
    self._emit()
  File "/var/lib/juju/agents/unit-katib-db-0/charm/venv/ops/main.py", line 519, in _emit
    _emit_charm_event(self.charm, self.dispatcher.event_name)
  File "/var/lib/juju/agents/unit-katib-db-0/charm/venv/ops/main.py", line 147, in _emit_charm_event
    event_to_emit.emit(*args, **kwargs)
  File "/var/lib/juju/agents/unit-katib-db-0/charm/venv/ops/framework.py", line 348, in emit
    framework._emit(event)
  File "/var/lib/juju/agents/unit-katib-db-0/charm/venv/ops/framework.py", line 860, in _emit
    self._reemit(event_path)
  File "/var/lib/juju/agents/unit-katib-db-0/charm/venv/ops/framework.py", line 950, in _reemit
    custom_handler(event)
  File "/var/lib/juju/agents/unit-katib-db-0/charm/lib/charms/tempo_k8s/v1/charm_tracing.py", line 724, in wrapped_function
    return callable(*args, **kwargs)  # type: ignore
  File "/var/lib/juju/agents/unit-katib-db-0/charm/./src/charm.py", line 472, in _reconcile_mysqld_exporter
    if not self._mysql.is_data_dir_initialised():
  File "/var/lib/juju/agents/unit-katib-db-0/charm/./src/charm.py", line 200, in _mysql
    self.app_peer_data["cluster-name"],
  File "/var/lib/juju/agents/unit-katib-db-0/charm/venv/ops/model.py", line 1812, in __getitem__
    return super().__getitem__(key)
  File "/var/lib/juju/agents/unit-katib-db-0/charm/venv/ops/model.py", line 841, in __getitem__
    return self._data[key]
KeyError: 'cluster-name'
unit-katib-db-0: 13:04:39 ERROR juju.worker.uniter.operation hook "metrics-endpoint-relation-created" (via hook dispatching script: dispatch) failed: exit status 1
unit-katib-db-0: 13:04:39 INFO juju.worker.uniter awaiting error resolution for "relation-created" hook
unit-katib-db-0: 13:05:23 INFO juju.worker.uniter awaiting error resolution for "relation-created" hook

--- hotfix applied to unit charm code ---

unit-katib-db-0: 13:06:25 INFO juju.worker.uniter awaiting error resolution for "relation-created" hook
unit-katib-db-0: 13:07:16 INFO juju.worker.uniter awaiting error resolution for "relation-created" hook
unit-katib-db-0: 13:07:16 INFO juju.worker.uniter awaiting error resolution for "relation-created" hook
unit-katib-db-0: 13:08:29 INFO juju.worker.uniter awaiting error resolution for "relation-created" hook
unit-katib-db-0: 13:08:30 INFO juju.worker.uniter.operation ran "metrics-endpoint-relation-created" hook (via hook dispatching script: dispatch)
unit-katib-db-0: 13:08:32 INFO juju.worker.uniter.operation ran "grafana-dashboard-relation-created" hook (via hook dispatching script: dispatch)
unit-katib-db-0: 13:08:33 INFO juju.worker.uniter.operation ran "restart-relation-created" hook (via hook dispatching script: dispatch)
unit-katib-db-0: 13:08:34 INFO juju.worker.uniter.operation ran "upgrade-relation-created" hook (via hook dispatching script: dispatch)
unit-katib-db-0: 13:08:35 INFO juju.worker.uniter.operation ran "database-peers-relation-created" hook (via hook dispatching script: dispatch)
unit-katib-db-0: 13:08:35 INFO juju.worker.uniter found queued "leader-elected" hook
unit-katib-db-0: 13:08:37 WARNING unit.katib-db/0.juju-log Failed to check if cluster metadata exists from_instance='katib-db-0.katib-db-endpoints.kubeflow.svc.cluster.local'
unit-katib-db-0: 13:08:37 INFO juju.worker.uniter.operation ran "leader-elected" hook (via hook dispatching script: dispatch)
unit-katib-db-0: 13:08:38 INFO unit.katib-db/0.juju-log Setting up the logrotate configurations
unit-katib-db-0: 13:08:44 INFO unit.katib-db/0.juju-log Configuring instance
unit-katib-db-0: 13:08:44 INFO unit.katib-db/0.juju-log Installing plugin='audit_log'
unit-katib-db-0: 13:08:44 INFO unit.katib-db/0.juju-log Installing plugin='audit_log_filter'
unit-katib-db-0: 13:08:53 WARNING unit.katib-db/0.juju-log Failed to check if cluster metadata exists from_instance='katib-db-0.katib-db-endpoints.kubeflow.svc.cluster.local'
unit-katib-db-0: 13:08:53 INFO unit.katib-db/0.juju-log Creating cluster cluster-ae1e00e9160aba3af75a49c0f03deffb
unit-katib-db-0: 13:08:58 INFO juju.worker.uniter.operation ran "mysql-pebble-ready" hook (via hook dispatching script: dispatch)
unit-katib-db-0: 13:08:59 INFO unit.katib-db/0.juju-log Starting the log rotate manager
unit-katib-db-0: 13:08:59 INFO unit.katib-db/0.juju-log Started log rotate manager process with PID 1949
unit-katib-db-0: 13:08:59 INFO juju.worker.uniter.operation ran "database-storage-attached" hook (via hook dispatching script: dispatch)
unit-katib-db-0: 13:09:01 INFO juju.worker.uniter.operation ran "config-changed" hook (via hook dispatching script: dispatch)
unit-katib-db-0: 13:09:04 INFO juju.worker.uniter found queued "start" hook
unit-katib-db-0: 13:09:05 INFO unit.katib-db/0.juju-log Running legacy hooks/start.
unit-katib-db-0: 13:09:07 INFO juju.worker.uniter.operation ran "start" hook (via hook dispatching script: dispatch)
unit-katib-db-0: 13:09:10 INFO unit.katib-db/0.juju-log database:28: Kubernetes service katib-db-primary created
unit-katib-db-0: 13:09:10 INFO unit.katib-db/0.juju-log database:28: Kubernetes service katib-db-replicas created

Added integration test in 58f42cd38c6d6f19d2de0eb177ddc11622ebad82

Thank you the test, LGTM.
Please create a backlog to add COS full test (installing cos-lite bundle in parallel model and make sure grafana-agent is active and metrics/logs sent there.

Created https://warthogs.atlassian.net/browse/DPE-5482