Relabel BTRFS Content on container Creation

This change will allow us to run SELinux in a container with
BTRFS back end.  We continue to work on fixing the kernel/BTRFS
but this change will allow SELinux Security separation on BTRFS.
It basically relabels the content on container creation.
Just relabling -init directory in BTRFS use case. Everything looks like it
works. I don't believe tar/achive stores the SELinux labels, so we are good
as far as docker commit.
Tested Speed on startup with BTRFS on top of loopback directory. BTRFS
not on loopback should get even better perfomance on startup time.  The
more inodes inside of the container image will increase the relabel time.
This patch will give people who care more about security the option of
runnin BTRFS with SELinux.  Those who don't want to take the slow down
can disable SELinux either in individual containers or for all containers
by continuing to disable SELinux in the daemon.
Without relabel:

time docker run --security-opt label:disable fedora echo test
test

real    0m0.918s
user    0m0.009s
sys    0m0.026s
With Relabel
test
real    0m1.942s
user    0m0.007s
sys    0m0.030s
Signed-off-by: Dan Walsh<EMAIL_ADDRESS>

@tiborvass @crosbymichael @tianon @jfrazelle PTAL

makes sense for me
ping @docker/core-maintainers

Moved to code-review, unfortunately it needs a rebase.
Change LGTM.

@calavera Rebased.

it doesn't look like this compiles any more. Check https://jenkins.dockerproject.org/job/Docker-PRs/16830/console

How do I fix the golint problems with Windows?

I've opened #16666 to fix those issues.

Don't see how the Janky failures are related to my patch?

@rhatdan janky runs go-lint in every changed file you commit. For some reason, it looks like graph/windows/windows.go never went through go-lint before and it failed when your changes trigger the lint. It should be fixed if you rebased master into this branch.

rebased @calavera Any progress on this?

---> Making bundle: binary (in bundles/1.9.0-dev/binary)
Building: bundles/1.9.0-dev/binary/docker-1.9.0-dev
# github.com/docker/docker/daemon/graphdriver
daemon/graphdriver/plugin.go:32: cannot use proxy (type *graphDriverProxy) as type Driver in return argument:
	*graphDriverProxy does not implement Driver (wrong type for Create method)
		have Create(string, string) error
		want Create(string, string, string) error
Build step 'Execute shell' marked build as failure
[PostBuildScript] - Execution post build scripts.
[Docker-PRs-experimental] $ /bin/sh -xe /tmp/hudson466336133504869178.sh
+ set +x

@jfrazelle fixed

./docker_cli_external_graphdriver_unix_test.go:127: not enough arguments in call to driver.Create

daemon\graphdriver\windows\windows.go:133: d.resolveId undefined (type *Driver has no field or method resolveId)

ping @rhatdan can you rebase and address the errors?

A few more errors.

@cpuguy83 In order to get this patch to work.  I needed to reverse the order of
daemon.setHostConfig before daemon.createRootfs which is causing the problems. I am having someone on my team look into why this is causing the tests to be broken, since I am traveling.
We need the order reversed, because with this patch
daemon.createRootfs  calls graphdriver create which needs the SELinux labels initialized.  These are initialized in daemon.setHostConfig, if they are not reversed the labels do not get generated until too late.

@rhatdan any update on that? going through PRs to see if we can get some movement in the older ones :smile:

@thaJeztah Sorry been travelling, will start to look into this today.  Problem is the reversing of the
@@ -87,10 +87,10 @@ func (daemon *Daemon) create(params *ContainerCreateConfig) (retC *Container, re
 	if err := daemon.Register(container); err != nil {
 		return nil, err
 	}
-	if err := daemon.createRootfs(container); err != nil {
+	if err := daemon.setHostConfig(container, params.HostConfig); err != nil {
 		return nil, err
 	}
-	if err := daemon.setHostConfig(container, params.HostConfig); err != nil {
+	if err := daemon.createRootfs(container); err != nil {
 		return nil, err
 	}
 	defer func() {

Anyone have any idea why this is causing problem.

I moved the security check out of setHostConfig so I can call it before createRootFS.  Hopefully this fixes the test issues.

LGTM

LGTM

Why were the labels added to the Create method instead of putting it on Get? The interface to Create was updated to accommodate this but the reason is not clear.

You need the label at create time in order to recursively set the labels.  Get is too late.  We need the same thing to get labels for OverlayFS if the kernel ever gets fixed.

The label is only added to one Create call, and the next line has a call to Get. https://github.com/docker/docker/pull/16452/files#diff-1a1f3e7ad9b1d7584e2d3e7d0c4c3db9R995
Considering this label is done at the very end of Create, I don't see anything based on the code changes that wouldn't allow it to be done at the beginning of Get.

Then every get call would be relabeling, we only want this happening once.  Think docker create versus docker start.

I understand the difference, I am asking because this effects #17924 and we have to rebase and make interface updates as a result of this change. From my perspective, this is a btrfs specific change (possibly something similar for overlay in the future) which has now updated the interface to add mount labels on create, where previously create was not intended to mount. If relabeling must happen before first mount on btrfs then it is preferable to limit the change if that is possible. Except for on daemon.Mount, graphdriver.Get is normally called without a mount label, meaning this would only apply to start. From your previous comment, you are saying the main reason this must happen on create is that calling relabel for every start is either non-performant or is the incorrect behavior.
Somewhat separate question, why does this only apply to the init layer for the container and not any of the base layers or container writable layer?

Yes relabeling is going to be time consuming.  We time it out at about 1-2 seconds on a base image, but it could grow depending on the number of inodes in the image.  Since we only need to do this on create, no need to add this penalty to all Starts.
By only applying it to the init layer. we end up being able to continue to share the image. IE Does not cause a COW change for every container.  @rhvgoyal correct?

@rhatdan right. If we apply label on container rootfs (and not init layer), then docker commit will make all the image files part of the COW layer and that's not what we want.


Fix broken duration pressed functionality

Fixes #127.
Status:

[x] reproduce the bug
[x] write an integration test
[ ] figure out why the integration test is failing
[ ] fix the bug

Thoughts:

Add debug invocations to tick and flip.
Try to figure out a path for exactly when the duration is getting changed; may be reset to zero.


Bug (stable and dev) when --allow_root is passed and run as root user

Description
Currently when run as root and passing --allow_root, this flag is not passed through to segstats.py in recon-surf.sh , leading to an early termination of the script.
Steps to Reproduce
call run_fastsurfer as root and pass --allow_root flag.
Expected Behavior
Continue without any error as --allow_root was passed
Additional Context
Actually, I am not a great fan of all the root checks everywhere. I think it should suffice if we check this in the entry script(s) and no sub-scripts. So the fix is to remove that check and error from segstats.py and possibly mri_segstats.py , recon-surf.sh and maybe others.

Should be solved by #598 and in dev by #599


Blocked: VS Code hangs at startup after 1.54.2 upgrade (including insiders build)

VS Code Version: 1.55.0-insider (code-insiders-1.55.0-1615526749.el7.x86_64.rpm)
OS Version: CentOS Linux release 7.9.2009 (Core)

Steps to Reproduce:

Upgrade to code-1.54.2
Start VS Code.

or install the nightly code-insiders release and start code-insiders.

Does this issue occur when all extensions are disabled?: Yes

I start VS Code and after about a second I see the welcome screen. At that point, the process is locked up hard (hung). Does not respond. There is no obvious crash; the process does not exit, but interaction with the app causes the system present a "not responding" dialog where you can choose to force-kill it. Choosing to instead wait does not change the status of the process, even after a lengthy wait.
This is a show-stopper problem for me as I cannot use VS Code at all anymore.
I have downloaded and installed the insiders nightly version. I have followed the instructions on https://github.com/microsoft/vscode/wiki/Native-Crash-Issues#creating-and-symbolicating-local-crash-reports but it does not seem very helpful as only the clientid files are generated (because the app does not actually crash).
I have tried waiting longer, but that doesn't seem to help. Symptoms are that you cannot close the welcome page, you cannot uncheck the "show welcome" checkbox, the window close box does nothing, i.e. the application does not respond to events.
I also tried:

used ps axc | grep code to find and kill any code processes
deleted ~/.config/Code and ~/.config/Code - Insiders
ran code-insiders --disable-extensions . from an empty folder
rebooted the machine and repeated all this

Three possible clues:

Note that when I start code-insiders with --disable-extensions, I do get the small popup that tells me extensions are disabled, and even more telling is that seconds after the app is unresponsive, the popup auto-closes itself normally, meaning something there is still running normally. Just that all input and interactivity is blocked. For example, I cannot close that dialog or press the Reload Extensions button.
When I clean up previous runs for another test attempt, a kill <pid> on the first process ID is sufficient, I do not need to force-kill it with a kill -9 <pid>
A code-insiders --version reports the version after starting and cleanly exiting:

$ code-insiders --version
1.55.0-insider
06b54543d660353a04cdab6554d1f27266a537fd
x64

An attempt to resize the window does resize the frame (see image below) but does not resize the contents, as the app is frozen hard. (See whitespace on right and bottom of window in image below.):

A short time after attempting to interact, the system displays a "not responding" dialog and offers a forced quit.
This is the latest nightly insider as of Sunday March 14th 3pm, I used code-insiders-1.55.0-1615526749.el7.x86_64.rpm to install, and created an empty folder and used the following command to launch VS Code:
code-insiders --crash-reporter-directory ~/crashes .
VS code seems to be leaving process instances running, but I verified that none were running  using ps axc | grep code (or killed any that were) prior to launching the insiders build above.
The contents of the ~/crashes folder only contains a client_id file with an ID in it:
69c31889-2c99-4fed-94e2-f3275f581566
and a folder named exthost Crash Reports with another client_id file with an ID in it:
17b573a4-9c8a-47e0-9c6a-11c680d6e828
Note they are different IDs in spite of being named client_id, not sure if that is expected.
The machine is a CentOS 7 installation with the latest Chrome installed and it runs fine (I'm using it to submit this report):
Chrome 89.0.4389.82 (Official Build) (64-bit)
If there is a file-based way to disable the welcome screen before starting code, I could do that too, to determine if it is something related to rendering that page.

I assumed it was related and reported this in #118522, and it may be related, but now I think this is a separate problem. According to the original poster in that issue, VS Code actually crashes. That report is on Windows so may be a different issue since it actually crashes.

Note that I was not having an issue prior to my upgrade to 1.54.2, and the problem occurred on the very first launch after upgrading. I have not had a successful run since that upgrade, including after uninstalling 1.54.2, cleaning up the .config folder and reinstalling it, uninstalling it again, and installing the Insiders build. It is very likely a regression of some kind was introduced in 1.54.2. I do not know what version I was running before that, but I tend to keep Code very up to date, probably within a few days at least.

Can you provide the output of code --verbose, does version 1.53 https://code.visualstudio.com/updates/v1_53 work fine ?

More info:

Launching with code-insiders --disable-extensions --sync off --disable-gpu --verbose . on the empty folder produces several tracebacks (all appear to be the same), included below.
After launching the above, running code-insiders -s from another shell also hangs (or is blocked).

[main 2021-03-15T15:40:18.418Z] [uncaught exception in main]: Error: /lib64/libstdc++.so.6: version `CXXABI_1.3.9' not found (required by /usr/share/code-insiders/resources/app/node_modules.asar.unpacked/vscode-sqlite3/build/Release/sqlite.node)

I'll get you the full output next.

Here is the output from the above run with code-insiders --disable-extensions --sync off --disable-gpu --verbose .:
verbose.log

I think the culprit is /lib64/libstdc++.so.6: version CXXABI_1.3.9' not found`

After googling that error, one of the replies is:

CXXABI_1.3.9 is associated with gcc version 5.1. CentOS 7 has gcc 4.8.5. If you cannot find the software of your interest that is not linked against the newer version, your best bet will be to use a newer OS such as Fedora.

I am working on a government-provided laptop on a government project where CentOS 7 is (currently at least) the required platform. This kind of implies that maybe VS Code lost support for CentOS 7 in the 1.54.2 update.  Dependency ripple effect perhaps with an Electron upgrade?
I'll see if I can resolve the CXXABI library on CentOS 7 with an additional install.

Thanks for the details, reason to drop support is mentioned here https://github.com/microsoft/vscode/issues/115784#issuecomment-775364356 ,  workaround would be to install version of gcc that has required CXXABI support.
Merging to https://github.com/microsoft/vscode/issues/115784


Missing software licence

Hey xmssp,
thanks for providing the source code of this great tool. I wonder which software licence applies to your source code as there is no licence mentioned in your repository. Also it seems that GitHub does not have a default software license in case the repository itself does not provide one. Please give a short feedback if your source code can be used open source without any restrictions.
Kind regards, Marcel

Hi,
I am also interested in the license of this project. Without a license specification regular copyright laws apply, thus preventing others to use or extend this project in their own projects. Could you please add a license to this project or give some information, if it is possible to include this tool in other projects?
Thanks and kind regards,
Ulrich


Improve options for order of configuration pyproject.toml serialization and de-serialization

Currently the ser and deser for pyproject.toml is pretty raw. Some things aren't wrapped with Option when they should be. I believe the ordering of the information in the toml is also important.
I want to allow de-ser in really many arrangements, but be opinionated during serialization. The de-ser part is what should break if you move things around. I believe this only occurs when using certain formats like the array of object formats for authors.
I'll revisit this to help clarify this issue before adding the good first issues label.

Tag #247

Closing for now. Can revisit when it becomes an issue.


Consistent placement of quotes

for mev build. It's a minor thing, works either way, but would (hopefully) head off questions as to why it's in there two different ways.

Merging


Fix typo: device_client has never been initialized

device_client needs to be changed to client in this case.
Thank you for helping us improve the Azure IoT Python SDK!
Need Support

Have a feature request for SDKs? Please post it on User Voice to help us prioritize
Have a technical question? Please see IoT support options for information on posting questions on Microsoft Q&A and Stack overflow using the "azure-iot-sdk" tag.
Need Support? Every customer with an active Azure subscription has access to support with guaranteed response time.  Consider submitting a ticket and get assistance from Microsoft support team
Found a bug? Please help us fix it by thoroughly documenting it and filing an issue on GitHub (See below).

Here's a little checklist of things that will help it make its way to the repository: Note that you don't have to check all the boxes, we can help you with that.
This being said, the more you do, the quicker it'll go through our gated build!
-->
Checklist

[x] I have read the contribution guidelines.
[N/A] I added or modified the existing tests to cover the change (we do not allow our test coverage to go down).
If this is a modification that impacts the behavior of a public API

[N/A] I edited the corresponding document in the devdoc folder and added or modified requirements.

Reference/Link to the issue solved with this PR (if any)
N/A. Just trying to fix typo.
Description of the problem
In the sample code snippet, device_client needs to be changed to client in this case.
Please be as precise as possible: what issue you experienced, how often...
Description of the solution
device_client needs to be changed to client in this case.
How you solved the issue and the other things you considered and maybe rejected

/azp run

/azp run


Translations update from Hosted Weblate

Translations update from Hosted Weblate for Flashy/Flashy.
Current translation status:

Thanks for your support, @inson1! 😘

@CrazyMarvin I'm going to do 100%, someday :)


cant load more than 10 posts on someone's profile

pretty self explanatory

cant reproduce it, it works just fine on mobile and computers
maybe the user does not have more than 10 posts or you havent scrolled down all the way to the bottom


Error: Pagefind wasn't able to build an index.

Hello 👋
M1 Mac running Ventura 13.3 (22E252)
I have tried to set up pagefind on a smallish Hugo site.  What I did:

Created a partial and placed the code snippet from Getting started in this partial
Loaded this partial in the header of my site
Ran rm -rf public && hugo

Output
Start building sites …
hugo v0.111.3+extended darwin/amd64 BuildDate=unknown

                   | EN
-------------------+------
  Pages            | 338
  Paginator pages  |   0
  Non-page files   |  59
  Static files     |  28
  Processed images | 162
  Aliases          |   0
  Sitemaps         |   1
  Cleaned          |   0

Total in 1831 ms

Ran  npx -y pagefind --source public --serve

Output

Running Pagefind v0.12.0 (Extended)
Running from: "/Users/lim/CYF/Repos/curriculum"
Source:       "public"
Bundle Directory:  "_pagefind"

[Walking source directory]
Found 279 files matching **/*.{html}

[Parsing files]
Did not find a data-pagefind-body element on the site.
↳ Indexing all <body> elements on the site.

[Reading languages]
Discovered 0 languages:

[Building search indexes]
Total:
  Indexed 0 languages
  Indexed 0 pages
  Indexed 0 words
  Indexed 0 filters
  Indexed 0 sorts
Error: Pagefind wasn't able to build an index.
Most likely, the directory passed to Pagefind was empty or did not contain any html files.

Out of interest, I tried serving Hugo hugo server --disableFastRender and in the browser logged these errors:

Output
GET http://localhost:1313/_pagefind/pagefind-ui.js net::ERR_ABORTED 404 (Not Found)
localhost/:82
GET http://localhost:1313/_pagefind/pagefind-ui.css net::ERR_ABORTED 404 (Not Found)
Uncaught ReferenceError: PagefindUI is not defined at VM1728:3:5

I searched the docs for this error and could not find a reference to it. The closest thing I could find in this issues list was  #132  . I fiddled about a bit to no avail. I began installing Rust and wasm but came to my senses and have come to ask here instead. I'm probably just missing something small.
My feature branch is currently just a copy paste from the docs. Can someone please give me a steer? I'd like to use pagefind. Thanks for all your work on this.

Oh that's great! A bug that finds a bug. (Oh the perils of autocomplete.)
All fixed and working now. Thank you!


Fix <0 TBB and <0 category headers not showing

One bug from comments, and found another..
https://trello.com/c/UHqAAG8m/51-remove-zero-categories-from-cover-overbudgeting-because-they-are-useless-there

Thanks!


[MRG] Add a dev installation guide.

Reference issue
Attempt to fix #5767
What does this implement/fix?
I added a YAML environment configuration file that installs only the dependencies of MNE so that a developer can use its own forked version. I added a section in the contributing guide as well with some instructions explaining the whole process.

Maybe its a good idea to merge this info with this last bit of contributing.rst:
Submitting changes
------------------
Changes to code can be submitted using a standard GitHub Pull Request, as
documented in :ref:`using_github`.

Because what you do is 1, download the repo; 2, make sure that it works; 3, code something; 4 submit a PR.
regaring this using-github guide we found with @qb3 that this documentation is outdated and refers to trunk etc.. as in SVN.
I would suggest to not do such an exhaustive guide but more like a small paragraph pointing to the right github documentaion:
https://help.github.com/articles/creating-a-pull-request-from-a-fork/
in this manner we don't have to keep up with github website workflow updates

travis complains 'cos there's a typo.

https://travis-ci.org/mne-tools/mne-python/jobs/463389810#L3305
(do not worry, travis likes to complain)

here's another link that we might point want to point to
https://help.github.com/articles/github-flow/#following-the-github-flow

I’d rather keep the same env
If you do an editable install, it should override all other MNE
installations, no? Is that not the case?
On Sat 8 Dec 2018 at 09:20, Alexandre Gramfort<EMAIL_ADDRESS>wrote:

@agramfort commented on this pull request.
In doc/contributing.rst
https://github.com/mne-tools/mne-python/pull/5768#discussion_r240006167:

@@ -21,6 +21,30 @@ of the users who use the package.
page as the maintainers about changes or enhancements before too much
coding is done saves everyone time and effort!

+Installing developer version
+----------------------------
+
+The standard installation procedure can be found on the
+MNE website
+but if you want to install a developer version directly from the source
+code in order to contribute, you can follow those steps:
+

.. code-block:: console

$ git clone git@github.com:mne-tools/mne-python.git

$ cd mne-python

$ conda env create -f dev_environment.yml

$ conda activate mne-dev-py36

$ pip install -e .

or another option is to keep it like this but to add a test to make sure
that the diff from environment.yml and dev_environment.yml is just 2 lines
(change env title + remove mne)
thoughts?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/mne-tools/mne-python/pull/5768#discussion_r240006167,
or mute the thread
https://github.com/notifications/unsubscribe-auth/APHioqI1afVw9kB2jI2w3hglGRbZPLx5ks5u28qvgaJpZM4ZA2F8
.

--
Sent from my iPhone

https://github.com/pandas-dev/pandas has environment.yml and requirements-dev.txt

What should we do ? Pandas offers two routes for a dev setup. One with conda and one pip
Problem is that full mne with mayavi still requires conda afaik
I don’t know if we have a pip solution
Personally I use standard mne env, uninstall stable and setup cloned version

Problem is that full mne with mayavi still requires conda afaik

No, you can use the requirements.txt with pip and it should work. We have a Travis run that does this.

Arfff then we should advertise how to install with standard pip in main documentation
Wdyt?

I just pointed to pandas 'cos they use conda as a base, and then they have a dev in requirements.txt. But I don't like it (I jsut pointed at it).

Arfff then we should advertise how to install with standard pip in main documentation
Wdyt?

I would give single easy manner of doing it. Cover the case: I've no idea what I'm doing I just want the rubber to hit the road.
If this is conda, conda. If this is pip then pip. If I'm avble to criticise one over the other I've enough knoledge to reinterprete the instructions. Notice that none of us needed the dev instructions. Either 'cos someone helped us to set up or because we had enough experience (or got it in the way).
But having CIs with both I find it good. It help us to catch problems early stage, and if someone asks for how to do it we can point to it, or an 'expert' can check the CI for clues (We do that all the time. If we don't find how to install something just check the CI. CI scripts are some sort of documentation by themselves)

I'd rather just keep one as user and Dev I, whichever we think is better (simplest, best supported, etc.). Advanced users can always deviate

+10000 to @larsoner. If we are warried about sync, we can always write a test for it. (which I think we should)

@larsoner you're fine with this proposal to duplicate the conda env with an mne-dev?

No I would rather keep a single environment and instruct devs with a few more lines in the contributing section. It should be a few packages to install for doc building plus pip install -e

so you agree with what I said above? install standard mne env,
uninstall stable and install master?

Yes. pip install -e might even do the uninstall step for you already

Agreed
Guillaume can you update PR accordingly ?
Thx heaps

+1 for :ref:install_python_and_mne_python it's the clean sphinx solution

There is already a section Checking and Building documentation in doc/contributing.rst.

Apparently AppVeyor is not happy because of a download error.

thx @GuillaumeFavelier


fix day names with multiple entries

Descriptio
When there are entries on more than 1 day at once (most often occurs on startup), the bot only displays the name of the first mentioned day.

How to fix
https://github.com/Kaffeegeist/rector-bot/blob/04d0f472e6d14e7c7d464b3abc24e676d07f8266/src/commands.ts#L38
The day name should be taken from an entry that has the same date as the current, and not the 0th element

fixed by #17


Svg Attribute Testing?

Does this test library also support testing svg view attributes?
Say I have this svg:
viewImage : Int -> Int -> String -> Svg msg
viewImage width height url =
    image
        [ Attr.width (toString width)
        , Attr.height (toString height)
        , Attr.xlinkHref url
        ]
        []

The xlink:href attribute does not show up in the failed test.

Perhaps this is just a bug with certain svg attributes, and if so i can rename this issue/try to make a PR to fix the behavior.

Likely that attribute is generated in a different way to what you'd expect. Because those attributes are namespaced, they might not be gathered in the same way. See this -> https://github.com/elm-lang/svg/blob/master/src/Svg/Attributes.elm#L1182

It looks like elm-html-test doesn't support namespaced attributes yet.  Test.Html.Selector.attribute doesn't handle namespaced attributes and will return an invalid selector, and other parts of the code also appear not to handle namespaces yet (though elm-html-in-elm does handle namespaces).


Conv1d kernel setting bug

Describe the bug
Importing a working onnx model (working in pytorch) ends in an invalid matrix multiplication after flattening.
Here is the traceback:
thread 'predict_m6a::onnx::my_test' panicked at '=== Tensor Operation Error ===
  Operation: 'Matmul'
  Reason:
    1. The inner dimension of matmul should be the same, but got 75 and 25. Lhs shape [1, 75], rhs shape [25, 5].
', /Users/mrvollger/Desktop/repos/burn/burn-tensor/src/tensor/api/float.rs:196:9
stack backtrace:
   0: rust_begin_unwind
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library/std/src/panicking.rs:593:5
   1: core::panicking::panic_fmt
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library/core/src/panicking.rs:67:14
   2: core::panicking::panic_display
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library/core/src/panicking.rs:150:5
   3: burn_tensor::tensor::api::float::<impl burn_tensor::tensor::api::base::Tensor<B,_>>::matmul
             at /Users/mrvollger/Desktop/repos/burn/burn-tensor/src/tensor/api/float.rs:196:9
   4: burn_core::nn::linear::Linear<B>::forward
             at /Users/mrvollger/Desktop/repos/burn/burn-core/src/nn/linear.rs:83:22
   5: fibertools_rs::model::revio::Model<B>::forward
             at ./target/debug/build/fibertools-rs-531f3735a2cf7cfe/out/model/revio.rs:115:28
   6: fibertools_rs::predict_m6a::onnx::my_model
             at ./src/predict_m6a/onnx.rs:18:18
   7: fibertools_rs::predict_m6a::onnx::my_test
             at ./src/predict_m6a/onnx.rs:25:13
   8: fibertools_rs::predict_m6a::onnx::my_test::{{closure}}
             at ./src/predict_m6a/onnx.rs:24:14
   9: core::ops::function::FnOnce::call_once
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library/core/src/ops/function.rs:250:5
  10: core::ops::function::FnOnce::call_once
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library/core/src/ops/function.rs:250:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

And here is the model code generated for this by burn-import:
// Generated from ONNX "src/model/revio.onnx" by burn-import
use burn::nn::conv::Conv1d;
use burn::nn::conv::Conv1dConfig;
use burn::nn::Linear;
use burn::nn::LinearConfig;
use burn::nn::PaddingConfig1d;
use burn::record::Recorder;
use burn::{
    module::Module,
    tensor::{backend::Backend, Tensor},
};

#[derive(Module, Debug)]
pub struct Model<B: Backend> {
    conv1d1: Conv1d<B>,
    conv1d2: Conv1d<B>,
    conv1d3: Conv1d<B>,
    linear1: Linear<B>,
    linear2: Linear<B>,
    phantom: core::marker::PhantomData<B>,
}

impl<B: Backend> Default for Model<B> {
    fn default() -> Self {
        let record = burn :: record :: DefaultFileRecorder :: < burn :: record :: FullPrecisionSettings > :: new () . load ("/Users/mrvollger/Desktop/repos/fibertools-rs/target/debug/build/fibertools-rs-531f3735a2cf7cfe/out/model/revio" . into ()) . expect ("Record file to exist.") ;
        Self::new_with(record)
    }
}

impl<B: Backend> Model<B> {
    #[allow(unused_variables)]
    pub fn new_with(record: ModelRecord<B>) -> Self {
        let conv1d1 = Conv1dConfig::new(6, 30, 1)
            .with_stride(1)
            .with_padding(PaddingConfig1d::Valid)
            .with_dilation(1)
            .with_groups(1)
            .with_bias(true)
            .init_with(record.conv1d1);
        let conv1d2 = Conv1dConfig::new(30, 10, 1)
            .with_stride(1)
            .with_padding(PaddingConfig1d::Valid)
            .with_dilation(1)
            .with_groups(1)
            .with_bias(true)
            .init_with(record.conv1d2);
        let conv1d3 = Conv1dConfig::new(10, 5, 1)
            .with_stride(1)
            .with_padding(PaddingConfig1d::Valid)
            .with_dilation(1)
            .with_groups(1)
            .with_bias(true)
            .init_with(record.conv1d3);
        let linear1 = LinearConfig::new(25, 5)
            .with_bias(true)
            .init_with(record.linear1);
        let linear2 = LinearConfig::new(5, 2)
            .with_bias(true)
            .init_with(record.linear2);
        Self {
            conv1d1,
            conv1d2,
            conv1d3,
            linear1,
            linear2,
            phantom: core::marker::PhantomData,
        }
    }

    #[allow(dead_code)]
    pub fn new() -> Self {
        let conv1d1 = Conv1dConfig::new(6, 30, 1)
            .with_stride(1)
            .with_padding(PaddingConfig1d::Valid)
            .with_dilation(1)
            .with_groups(1)
            .with_bias(true)
            .init();
        let conv1d2 = Conv1dConfig::new(30, 10, 1)
            .with_stride(1)
            .with_padding(PaddingConfig1d::Valid)
            .with_dilation(1)
            .with_groups(1)
            .with_bias(true)
            .init();
        let conv1d3 = Conv1dConfig::new(10, 5, 1)
            .with_stride(1)
            .with_padding(PaddingConfig1d::Valid)
            .with_dilation(1)
            .with_groups(1)
            .with_bias(true)
            .init();
        let linear1 = LinearConfig::new(25, 5).with_bias(true).init();
        let linear2 = LinearConfig::new(5, 2).with_bias(true).init();
        Self {
            conv1d1,
            conv1d2,
            conv1d3,
            linear1,
            linear2,
            phantom: core::marker::PhantomData,
        }
    }

    #[allow(clippy::let_and_return)]
    pub fn forward(&self, input1: Tensor<B, 3>) -> Tensor<B, 2> {
        let conv1d1_out1 = self.conv1d1.forward(input1);
        let relu1_out1 = burn::tensor::activation::relu(conv1d1_out1);
        let conv1d2_out1 = self.conv1d2.forward(relu1_out1);
        let relu2_out1 = burn::tensor::activation::relu(conv1d2_out1);
        let conv1d3_out1 = self.conv1d3.forward(relu2_out1);
        let relu3_out1 = burn::tensor::activation::relu(conv1d3_out1);
        let flatten1_out1 = relu3_out1.flatten(1, 2);
        let linear1_out1 = self.linear1.forward(flatten1_out1);
        let relu4_out1 = burn::tensor::activation::relu(linear1_out1);
        let linear2_out1 = self.linear2.forward(relu4_out1);
        let softmax1_out1 = burn::tensor::activation::softmax(linear2_out1, 1);
        softmax1_out1
    }
}

To Reproduce
I tried to come up with a minimal example, but I am having difficulty doing so, so I am including the full onnx model (below) and the code I use to run it. I hope that is okay.
Load this onnx model using burn-import:
https://github.com/fiberseq/fibertools-rs/raw/onnx_burn/src/model/revio.onnx
And then test running the model with the following code:
use super::super::model::revio::Model;
use anyhow::Result;
use burn::tensor::backend::Backend;
use burn::tensor::{Shape, Tensor};
use burn_ndarray::NdArrayBackend;
type NdArrayB = NdArrayBackend<f32>;

pub fn my_model<B>() -> Result<()>
where
    B: Backend,
{
    let model: Model<B> = Model::new();
    let _shape = Shape::new([1, 6, 15]);
    //let _input = Tensor::<B, 3>::ones(_shape);
    let _input = Tensor::<B, 3>::zeros(_shape);
    let output = model.forward(_input);
    println!("{:?}", output);
    Ok(())
}

#[test]
fn my_test() {
    let _ = my_model::<NdArrayB>();
}

Desktop (please complete the following information):

OS: macOS
version: Main branch latest commit

Additional context
Sorry for bugging you guys again with my issues loading this model. Hopefully, it is productive beyond just my use case... Happy to provide any additional details or do additional coding if pointed in the right direction.
Thanks,
Mitchell

@mrvollger, thanks for filing the issue ticket. I have started looking into this.
I ran your onnx through onnxsim to trace the output shapes and I am sharing here with you so you can see the output shape at each node. onnxsim revio.onnx revio-sim.onnx
revio-sim.onnx.zip

Awesome, I was looking for a way to check all of those dimensions. Good to learn about this tool. I will resume my investigation as well, I feel there is a good chance that if there is a dimension mismatch it might be happening in my conv1d code...

Yep I think I made a mistake in my Conv1D code. Becuase your simulation says that the dims after the first conv1d should be [1 x 30 x 11] but in burn-import it has dims [1 X 30 X 15] after the first conv1d.
I will dig around to see if I can find what is likely my mistake in the conv1d.
sorry about this!

@mrvollger I found a bug in our Conv1d module implementation. I haven't dived in to find the root cause. It's not ONNX issue.
Here is a unit test that fails to catch:
    #[test]
    fn verify_output_shape() {
        let conv1d1: Conv1d<TestBackend> = Conv1dConfig::new(6, 30, 1)
            .with_stride(1)
            .with_padding(PaddingConfig1d::Valid)
            .with_dilation(1)
            .with_groups(1)
            .with_bias(true)
            .init();

        let input = Tensor::<TestBackend, 3>::zeros([1, 6, 15]);

        let output = conv1d1.forward(input);

        assert_eq!(output.shape(), [1, 30, 11].into());
    }

Errors out like this:
running 1 test
thread 'nn::conv::conv1d::tests::verify_padding' panicked at 'assertion failed: `(left == right)`
  left: `Shape { dims: [1, 30, 15] }`,
 right: `Shape { dims: [1, 30, 11] }`', burn-core/src/nn/conv/conv1d.rs:175:9
stack backtrace:

I traced the output shapes and that's how I discovered by comparing the outputs:
    #[allow(clippy::let_and_return)]
    pub fn forward(&self, input1: Tensor<B, 3>) -> Tensor<B, 2> {
        println!("input1: {:?}", input1.shape());

        let conv1d1_out1 = self.conv1d1.forward(input1);
        println!("conv1d1_out1: {:?}", conv1d1_out1.shape());

        let relu1_out1 = burn::tensor::activation::relu(conv1d1_out1);
        println!("relu1_out1: {:?}", relu1_out1.shape());

        let conv1d2_out1 = self.conv1d2.forward(relu1_out1);
        println!("conv1d2_out1: {:?}", conv1d2_out1.shape());

        let relu2_out1 = burn::tensor::activation::relu(conv1d2_out1);
        println!("relu2_out1: {:?}", relu2_out1.shape());

        let conv1d3_out1 = self.conv1d3.forward(relu2_out1);
        println!("conv1d3_out1: {:?}", conv1d3_out1.shape());

        let relu3_out1 = burn::tensor::activation::relu(conv1d3_out1);
        println!("relu3_out1: {:?}", relu3_out1.shape());

        let flatten1_out1 = relu3_out1.flatten(1, 2);
        println!("flatten1_out1: {:?}", flatten1_out1.shape());

        let linear1_out1 = self.linear1.forward(flatten1_out1);
        println!("linear1_out1: {:?}", linear1_out1.shape());

        let relu4_out1 = burn::tensor::activation::relu(linear1_out1);
        println!("relu4_out1: {:?}", relu4_out1.shape());

        let linear2_out1 = self.linear2.forward(relu4_out1);
        println!("linear2_out1: {:?}", linear2_out1.shape());

        let softmax1_out1 = burn::tensor::activation::softmax(linear2_out1, 1);
        println!("softmax1_out1: {:?}", softmax1_out1.shape());

        softmax1_out1
    }

The output:
     Running `target/debug/revio-burn`
input1: Shape { dims: [1, 6, 15] }
conv1d1_out1: Shape { dims: [1, 30, 15] }
relu1_out1: Shape { dims: [1, 30, 15] }
conv1d2_out1: Shape { dims: [1, 10, 15] }
relu2_out1: Shape { dims: [1, 10, 15] }
conv1d3_out1: Shape { dims: [1, 5, 15] }
relu3_out1: Shape { dims: [1, 5, 15] }
flatten1_out1: Shape { dims: [1, 75] }
thread 'main' panicked at '=== Tensor Operation Error ===
  Operation: 'Matmul'
  Reason:
    1. The inner dimension of matmul should be the same, but got 75 and 25. Lhs shape [1, 75], rhs shape [25, 5].
', /Users/dilshod/.cargo/git/checkouts/burn-acfbee6a141c1b41/8808ee2/burn-tensor/src/tensor/api/float.rs:196:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

I will keep looking for my padding bug, but I am having trouble identifying where I went wrong. I guess I shouldn't have used a sum test case since that misses the dimensions aspect and specifically padding issues. I will keep this thread updated.

@mrvollger I have a problem and I fixed it. It's working now.
The issue was incorrect kernel size set. I will submit a PR in a few minutes. Running checks locally. Here is a correct output:
    #[allow(dead_code)]
    pub fn new() -> Self {
        let conv1d1 = Conv1dConfig::new(6, 30, 5)
            .with_stride(1)
            .with_padding(PaddingConfig1d::Valid)
            .with_dilation(1)
            .with_groups(1)
            .with_bias(true)
            .init();
        let conv1d2 = Conv1dConfig::new(30, 10, 5)
            .with_stride(1)
            .with_padding(PaddingConfig1d::Valid)
            .with_dilation(1)
            .with_groups(1)
            .with_bias(true)
            .init();
        let conv1d3 = Conv1dConfig::new(10, 5, 3)
            .with_stride(1)
            .with_padding(PaddingConfig1d::Valid)
            .with_dilation(1)
            .with_groups(1)
            .with_bias(true)
            .init();
        let linear1 = LinearConfig::new(25, 5).with_bias(true).init();
        let linear2 = LinearConfig::new(5, 2).with_bias(true).init();
        Self {
            conv1d1,
            conv1d2,
            conv1d3,
            linear1,
            linear2,
            phantom: core::marker::PhantomData,
        }
    }

Here is a test output:
     Running `target/debug/revio-burn`
Tensor { primitive: NdArrayTensor { array: [[0.6664351, 0.33356494]], shape=[1, 2], strides=[2, 1], layout=CFcf (0xf), dynamic ndim=2 } }

https://github.com/burn-rs/burn/pull/658 fixes the bug. @mrvollger Please check out the PR locally to test if you can. This should work for you.

Confirmed your PR fixes my issues locally! Thanks!

Keeping it open till the PR is merged.


add status to progress callback

You're sending a status field in the websockets during the ffmpeg trim. See "ffmpeg_progress" and "ffmpeg_finished".

It would also be nice to get a status during the actual rendering, like "render_progress", and "render_finished".

Why do you need this when the current frame compared to the total frames achieves the same result?

@bluematter In order to give the user more granular progress.
Like when it's trimming, we can tell the user "Trimming audio...".


Adding ethernet drivers for modules like LAN8720

I am needing an ethernet RJ45 connector to create a wired connection, making this firmware + ESP32 a competitor to routers that suffer throughput performance when running a lot of small UDP packets.
As the title says, I would like to add support for ESP32 ethernet modules which have the LAN8720 chip and RJ45 jack built in. Like https://www.digikey.com/en/products/detail/seeed-technology-co-ltd/102991455/13572085
It would help me to have some trial code to add to the codebase, and give me a starting point to test and work my way through.
There are a lot of different projects for creating a router with an ESP32, but most of them are dated and don't work, or then don't compile.
Additional resources
https://github.com/Zelmoghazy/esp32-ethernet-lan8720
https://github.com/espressif/esp-iot-solution/blob/release/v1.1/examples/eth2wifi/main/ethernet2wifi_demo_main.c
https://github.com/makischu/ESP_eth2wifi

Hi @benreu  sorry for the long delay for answering.
There is a one project is out there, it my solve & give you what you looking for.
Check this project esper
This project is basically degined for add blocking but it uses same RJ45 setup. So try to reverse engineering this project will gives all what you looking for.
Please share is it's works for you...


System Large font size

Text on tabs is cut, when user set Large font size

Same Problem here. Also the icons (incl. text) are not centered vertically.

May be someone know workaround for this issue? Text is cut only on tabs, which were inactive on creation, on active tab text is Ok


Create example PR for review

This is a:

[x] ✨ New feature - new behaviour has been implemented
[ ] 🐛 Bug fix - existing behaviour has been made to behave
[ ] ♻ Refactor - the behaviour has not changed, just the implementation
[ ] ✅ Test backfill - tests for existing behaviour were added but the behaviour itself hasn't changed
[ ] ⚙ Chore - maintenance task, behaviour and implementation haven't changed

Description

Purpose - Adds a resource details page accessible from the resource list:

How to check -

Ensure there are some published resources
Visit the home page
Click on a resource title
See resource details per screenshot above

Links

CodeYourFuture/tech-products-demo#18

Author checklist

[ ] I have written a title that reflects the relevant ticket
[ ] I have written a description that says what the PR does and how to validate it
[ ] I have linked to the project board ticket (and any related PRs/issues) in the Links section
[ ] I have added a link to this PR to the ticket
[ ] I have made the PR to main from a branch named <category>/<name>, e.g. feature/edit-spaceships or bugfix/restore-oxygen
[ ] I have manually tested that the app still works correctly
[ ] I have requested reviewers here and in my team chat channel

[ ] I have spoken with my PM or TL about any parts of this task that may have become out-of-scope, or any additional improvements that I now realise may benefit my project
[ ] I have added tests, or new tests were not required
[ ] I have updated any documentation (e.g. diagrams, schemas), or documentation updates were not required

@textbook Sorry I don't have idea for your code, everything is good and this is your style. Just added some comments if you wanted to check I can add review, otherwise I don't know what do you expect to get as a review!


python312Packages.hstspreload: 2024.11.1 -> 2024.12.1

Diff: https://github.com/sethmlarson/hstspreload/compare/refs/tags/2024.11.1...2024.12.1

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
25.05 Release Notes (or backporting 24.11 and 25.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

nixpkgs-review result
Generated using nixpkgs-review.
Command: nixpkgs-review pr 361040

x86_64-linux

:white_check_mark: 4 packages built:

python311Packages.hstspreload
python311Packages.hstspreload.dist
python312Packages.hstspreload
python312Packages.hstspreload.dist


feat: node table UI optimizations

For more details about the reasons to these changes see #259
Streamlined v-data-table selection:

Tasks:

[x] use v-data-table (single) selection mode
[x] eliminate obsolete selection logic
[x] change item slot template to slot templates for specific columns to be able to use the standard table features (selection, row expansion)

Expandible rows: (keep node related data + actions related without scrolling)

Tasks:

[x] activate expandable rows (not yet pushed)
[x] move node-specific UI+code into NodeDetails.vue (not yet pushed)
[ ] fix broken showConfirm() in sendNodeAction() of NodeDetails.vue
[ ] use multi-selection to filter selected nodes by node id
[ ] move groups tab into NodeDetails.vue and remove the obsolete selection of the affected node
[ ] change terminology selectedNode -> currentNode or expandedNode
[ ] review the refactoring with special attention to the following questions:

Has everything that affects a selected node been moved from ControlPanel.vue to NodeDetails.vue?
Has there been moved too much which may be the case if e.g. fields/methods/... are used by both ControlPanel.vue and NodeDetails.vue (examples that have already been taken into account: node_actions, socket, apiRequest, showSnackbar)

Additional questions (to improve maintainability) came up during refactoring (to be addressed separately):

[ ] What is the preferred naming convention in the templates? There are different styles in use. E.g.

v-*, nodes-table, column-filter
Confirm, DialogSceneValue, ValueID
Suggestion: Decide on one style and document it as an ADR

[ ] Should we recommend a canonical (alphabetical) sorting for the following items:

component attributes in templates
properties, methods in vue code
PRO: It is always clear where to put a new field/method/attribute/...

resolves #259

Pull Request Test Coverage Report for Build 504694088

0 of 3 (0.0%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 23.382%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

src/components/nodes-table/nodes-table.js
0
3
0.0%

Totals

Change from base Build 504126915:
0.0%

Covered Lines:
2008

Relevant Lines:
8789

💛  - Coveralls

Pull Request Test Coverage Report for Build 504694088

0 of 3 (0.0%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 23.382%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

src/components/nodes-table/nodes-table.js
0
3
0.0%

Totals

Change from base Build 504126915:
0.0%

Covered Lines:
2008

Relevant Lines:
8789

💛  - Coveralls

@robertsLando may you have a look at the current draft of the PR?
There are 2 commits:

This refactors the selection to use the default functionality from v-data-table: https://github.com/zwave-js/zwavejs2mqtt/pull/332/commits/c3d2e0bcd94a753c8cca919d96c81f5dd66d612b
This moves the node details into expandable rows: https://github.com/zwave-js/zwavejs2mqtt/pull/332/commits/e3bdf4c82a91942447e3ea240b5d2a46059fc4b7

The refactoring to extract all node details to the new component NodeDetails.vue was almost straight forward with the exception to a broken showConfirm which I've not yet been able to resolve.
There are still more tasks planned (see description above) but the major work is already done.
Can you have a look at it and give me your feedback so far?

@robertsLando may you have a look at the current draft of the PR?
There are 2 commits:

This refactors the selection to use the default functionality from v-data-table: https://github.com/zwave-js/zwavejs2mqtt/pull/332/commits/c3d2e0bcd94a753c8cca919d96c81f5dd66d612b
This moves the node details into expandable rows: https://github.com/zwave-js/zwavejs2mqtt/pull/332/commits/e3bdf4c82a91942447e3ea240b5d2a46059fc4b7

The refactoring to extract all node details to the new component NodeDetails.vue was almost straight forward with the exception to a broken showConfirm which I've not yet been able to resolve.
There are still more tasks planned (see description above) but the major work is already done.
Can you have a look at it and give me your feedback so far?

@ahochsteger About components file naming I will follow vue guidelines: https://vuejs.org/v2/style-guide/#Single-file-component-filename-casing-strongly-recommended. So both MyComponent.vue and my-component.vue are ok. In this case I would use the first one with capital letters.

Should we recommend a canonical (alphabetical) sorting for the following items:

I don't care too much on this, usually I try to group them based on the use. I think there could be some vscode plugin to automatically sort them in alphabetical order anyway?
Also I was thinking about move the Groups/Debug/Scenes tabs to the side panel. Maybe this could be done in another PR. Toughts?

@ahochsteger About components file naming I will follow vue guidelines: https://vuejs.org/v2/style-guide/#Single-file-component-filename-casing-strongly-recommended. So both MyComponent.vue and my-component.vue are ok. In this case I would use the first one with capital letters.

Should we recommend a canonical (alphabetical) sorting for the following items:

I don't care too much on this, usually I try to group them based on the use. I think there could be some vscode plugin to automatically sort them in alphabetical order anyway?
Also I was thinking about move the Groups/Debug/Scenes tabs to the side panel. Maybe this could be done in another PR. Toughts?

Unfortunally I cannot see the result of this, could you also attach screenshots of the UI? This is good in general when there is an UI feature so I can also ask other users if they like it without asking to compile the branch :)

Sorry, of course screenshots are a must - they are in the description now :-).

Also I was thinking about moving the Groups/Debug/Scenes tabs to the side panel. Maybe this could be done in another PR. Toughts?

Yes, for Debug and Sceenes the side panel is a much better place.
For groups I've already moved them into the expandible row (using tabs there to switch between node details and groups). This would make the node selection within the groups UI obsolete :-).
What do you think?

Unfortunally I cannot see the result of this, could you also attach screenshots of the UI? This is good in general when there is an UI feature so I can also ask other users if they like it without asking to compile the branch :)

Sorry, of course screenshots are a must - they are in the description now :-).

Also I was thinking about moving the Groups/Debug/Scenes tabs to the side panel. Maybe this could be done in another PR. Toughts?

Yes, for Debug and Sceenes the side panel is a much better place.
For groups I've already moved them into the expandible row (using tabs there to switch between node details and groups). This would make the node selection within the groups UI obsolete :-).
What do you think?

@ahochsteger Actually I really like the expand row feature! Looks so great! Keep an eye on the chrome dev console to see if there are errors :) I will give this a try on Monday but looks promising! Good work @ahochsteger and thanks :pray:

@ahochsteger Actually I really like the expand row feature! Looks so great! Keep an eye on the chrome dev console to see if there are errors :) I will give this a try on Monday but looks promising! Good work @ahochsteger and thanks :pray:

@ahochsteger Actually I really like the expand row feature! Looks so great! Keep an eye on the chrome dev console to see if there are errors :) I will give this a try on Monday but looks promising! Good work @ahochsteger and thanks

Glad, that you like it!
Actually the errors in the chrome dev console guide me to the missing parts that have to be moved :-P.

@ahochsteger Actually I really like the expand row feature! Looks so great! Keep an eye on the chrome dev console to see if there are errors :) I will give this a try on Monday but looks promising! Good work @ahochsteger and thanks

Glad, that you like it!
Actually the errors in the chrome dev console guide me to the missing parts that have to be moved :-P.

For groups I've already moved them into the expandible row (using tabs there to switch between node details and groups). This would make the node selection within the groups UI obsolete :-).

I think groups should not be there, or if so you should remove the selector of the first node and leave just the target. But I think it would be better to keep them where they are or in another panel

For groups I've already moved them into the expandible row (using tabs there to switch between node details and groups). This would make the node selection within the groups UI obsolete :-).

I think groups should not be there, or if so you should remove the selector of the first node and leave just the target. But I think it would be better to keep them where they are or in another panel

For groups I've already moved them into the expandible row (using tabs there to switch between node details and groups). This would make the node selection within the groups UI obsolete :-).

I think groups should not be there, or if so you should remove the selector of the first node and leave just the target. But I think it would be better to keep them where they are or in another panel

That's what I meant with make the node selection within the groups UI obsolete.

For groups I've already moved them into the expandible row (using tabs there to switch between node details and groups). This would make the node selection within the groups UI obsolete :-).

I think groups should not be there, or if so you should remove the selector of the first node and leave just the target. But I think it would be better to keep them where they are or in another panel

That's what I meant with make the node selection within the groups UI obsolete.

While I'm moving stuff around I'm creating more smaller components (e.g. NodeDetails, AssociationGroups, ...) that just what is needed for that component which improves maintainability a lot.
As a side-effect the components may be re-used in other contexts and are easier to test (thinking at the test automation discussion we had).

While I'm moving stuff around I'm creating more smaller components (e.g. NodeDetails, AssociationGroups, ...) that just what is needed for that component which improves maintainability a lot.
As a side-effect the components may be re-used in other contexts and are easier to test (thinking at the test automation discussion we had).

While I'm moving stuff around I'm creating more smaller components (e.g. NodeDetails, AssociationGroups, ...) that just what is needed for that component which improves maintainability a lot.

Sure, nice! That was something on my TODOs list, I mean split the UI in smaller components, ATM the control panel was so huge!

While I'm moving stuff around I'm creating more smaller components (e.g. NodeDetails, AssociationGroups, ...) that just what is needed for that component which improves maintainability a lot.

Sure, nice! That was something on my TODOs list, I mean split the UI in smaller components, ATM the control panel was so huge!

@robertsLando I'm done with the PR :-).
The screenshots are updated and everything is now fully working as far as I can see.
From my PoV I'm really satisfied with the UX of the new nodes table.
The broken showConfirm (e.g. on "Replace failed node" or some Home Assistant actions) was a bit tricky to fix but I found a solution by propagating the $listeners down to all sub-components that make use of showConfirm,
It would be great if you could take a careful look at the changes so I did not break something I was not aware of or had not tested good enough :-).

@robertsLando I'm done with the PR :-).
The screenshots are updated and everything is now fully working as far as I can see.
From my PoV I'm really satisfied with the UX of the new nodes table.
The broken showConfirm (e.g. on "Replace failed node" or some Home Assistant actions) was a bit tricky to fix but I found a solution by propagating the $listeners down to all sub-components that make use of showConfirm,
It would be great if you could take a careful look at the changes so I did not break something I was not aware of or had not tested good enough :-).

@ahochsteger I have tested your changes and added some UI style fixes. Looks so good! :) What I really don't like now is the scenes/debug tab, expecially when a node is expanded it looks like another node details, would you like to try to move them in a separete page? /scenes and /debug routes

Yes, we can do it as part of this PR.
I try to find some time this evening to refactor them into separate pages.

@ahochsteger I have tested your changes and added some UI style fixes. Looks so good! :) What I really don't like now is the scenes/debug tab, expecially when a node is expanded it looks like another node details, would you like to try to move them in a separete page? /scenes and /debug routes

Yes, we can do it as part of this PR.
I try to find some time this evening to refactor them into separate pages.

@ahochsteger I'm working on it

@ahochsteger I'm working on it

@ahochsteger see #342

@ahochsteger see #342

@ahochsteger Before merging this could you give a look at my changes ?

@ahochsteger Before merging this could you give a look at my changes ?

@ahochsteger Give this a try (to all tabs, included the mesh one) and let me know your opinion about the fixes

@ahochsteger Give this a try (to all tabs, included the mesh one) and let me know your opinion about the fixes

@ahochsteger Before merging this could you give a look at my changes ?

Looks good to me :-).
I've tested the latest changes (including the bump to zwave-js 6.0.2).

@ahochsteger Before merging this could you give a look at my changes ?

Looks good to me :-).
I've tested the latest changes (including the bump to zwave-js 6.0.2).

:rocket:

:rocket:

very nice UI improvement, much easier to navigate between nodes.
One small remark, tool tip is missing from left menu

very nice UI improvement, much easier to navigate between nodes.
One small remark, tool tip is missing from left menu

debug menu can be at the bottom as normal end-user will not click on it

debug menu can be at the bottom as normal end-user will not click on it

Clicking start in the debug menu doesn't do anything on my end, I can submit an issue if someone confirm.

Clicking start in the debug menu doesn't do anything on my end, I can submit an issue if someone confirm.

@towerhand it is actually working on my side. Are you sire there are logs coming when you press start?

@towerhand it is actually working on my side. Are you sire there are logs coming when you press start?

Never mind, got it working. I'm not sure what happened but glad I asked before submitting an issue. Nice improvement to the UI, great job guys.

Never mind, got it working. I'm not sure what happened but glad I asked before submitting an issue. Nice improvement to the UI, great job guys.


Adopt Microsoft charts from stable and incubator repos before 13 November 2020?

Hello MSFT charts people 👋
You may know the stable and incubator chart repos will be archived on 13 November. See https://github.com/helm/charts/issues/21103 for context.
We're looking for new homes for the charts, with a preference for orgs who maintain the applications the charts install. Here are the MSFT charts I found from a repo search:

[ ] stable/msoms
[ ] stable/mssql-linux
[ ] incubator/azuremonitor-containers

Would you be interested in a PR to bring Microsoft charts from stable and incubator into this repo?
As a side note, we have some processes for distributed chart repos hosted on GitHub to help efficiently bring git history, package history, and GitHub actions for CI (chart testing) and CD (chart releasing). If helpful, this could be another PR. Please lmk.

We can help to move to the new repo

@scottrigby where would be the new repo ?

@vishiy hosted by this github repo's gh-pages branch, as it is now. The goal of this issue is to bring MSFT related charts from stable and incubator source code into this existing github repo, so they will become indexed as part of this existing helm repo (and also to improve this repo's automation etc with current helm github actions).

Hi, is there an update on the progress of moving these helm charts to a new home? The deprecation date has already passed and I can't find any official notice of this.

Hello,
I am also looking for an update on this. Not finding any official notice of migration. Artifact hub seems to be pointing to an invalid location now.

I'm bumping this issue again. Is there any plan on officially adopting the charts?
Alternatively, is there an alternative chart repository for these charts?

I'm bumping this issue again. Is there any plan on officially adopting the charts?
Alternatively, is there an alternative chart repository for these charts?


MegaMenu:  Engineering design support - data model validation

User Story

As a [user role below]:
DDS Designer working on the MegaMenu

I need to:
have engineering support while working on the designs

so that I can:
accelerate the design effort and improve the design quality

Additional information

This issues is created for the engineers to be involved during the megamenu design to validate that the existing data model will work with the designs.
The duration of this work is for the second week of sprint 20-14.  Support will also be needed in the first week of Sprint 20-15.  This effort is included in issue (#3123)

Acceptance criteria

[ ] Data model analysis and recommendation are provided to Olivia

This is our current data structure for the masthead menu:
mastheadNav: {
  links: arrayOf({
    title: string,
    url: string,
    hasMegapanel: boolean,
    hasMenu: boolean
    menuSections: arrayOf({
      heading: string,
      menuItems: arrayOf({
        title: string,
        url: string,
        megapanelContent: {
          headingTitle: string,
          headingUrl: string,
          description: string,
          quickLinks: {
            title: string,
            links: arrayOf({
              title: string,
              url: string
            })
          },
          feature: {
            heading: string,
            imageUrl: string,
            linkTitle: string,
            linkUrl: string
          },
        }
      })
    })
  })
}

Looking at the designs for model 3, I believe the current data structure will work for most of the design. We would need to add data sections/fields for the left section and the "View all ... " CTA.

Example of how it will look like for the design in the comment above
{
  links: [{
    title: "Products & Solutions",
    url: "..."
    hasMegapanel: true,
    hasMenu: true,
    viewAll: {
      title: "View All Products & Solutions",
      url: "..."
    }
    highlightedSections: {
      menuSections: [
        {
          menuItems: [
            {
              title: "Top brands",
              url: "...",
              megapanelContent: {
                quickLinks: {
                  links: [
                    {
                      title: "IBM Cloud",
                      url: "...",
                    },
                    {
                      title: "Watson",
                      url: "...",
                    }
                    { ..... }
                  ]
                }
              }
            },
            {
              title: "Industries",
              url: "...",
              megapanelContent: {
                quickLinks: {
                  links: [
                    {
                      title: "Aerospace and defense",
                      url: "..."
                    }
                    { ..... }
                  ]
                }
              }
            }
          ]
        },
        {
          menuItems: {
            title: "Product payment plans",
            url: "...",
            megapanelContent: {
              quickLinks: {
                links: [
                  {
                    title: "Loans",
                    url: "..."
                  },
                  { ..... }
                ]
              }
            }
          }
        }
      ]
    },
    menuSections: [{
      menuItems: [
        {
          title: "Artificial Intelligence",
          url: "...",
          megepanelContent: {
            quickLinks: {
              links: [
                {
                  title: "Customer Service AI",
                  url: "..."
                },
                { ..... }
              ]
            }
          }
        },
        {
          title: "Blockchain",
          url: "...",
          megepanelContent: {
            quickLinks: {
              links: [
                {
                  title: "Blockchain Cross Border Payments",
                  url: "..."
                },
                { ..... }
              ]
            }
          }
        }
        { ..... }
      ]
    }]
  }]
}


Custominzing TurtleBot3 OpenCR with one more additional Dynamixal Motor

ISSUE TEMPLATE ver. 0.4.0

Which TurtleBot3 platform do you use?

[Yes ] Waffle

Which ROS is working with TurtleBot3?

[ Yes] ROS 2 Dashing Diademata

Which SBC(Single Board Computer) is working on TurtleBot3?

[Yes ] Raspberry Pi 4b

Which OS you installed on SBC?

[Yes ] Ubuntu preinstalled server (18.04.5)

Which OS you installed on Remote PC?

[ Yes] Ubuntu 18.04 LTS (Bionic Beaver)

Specify the software and firmware version(Can be found from Bringup messages)

Software version: ROS2 Dashing on
Firmware version: OpenCR 1.4.18 , Dynamixal2Arduno : 0.4.7

Specify the commands or instructions to reproduce the issue.

ros2 launch turtlebot3_bringup robot.launch.py

Copy and Paste the error messages on terminal.

[INFO] [launch]: All log files can be found below /home/ubuntu/.ros/log/2021-05-24-15-01-21-107308-ubuntu-6202
[INFO] [launch]: Default logging verbosity is set to INFO
urdf_file_name : turtlebot3_waffle.urdf
[INFO] [robot_state_publisher-1]: process started with pid [6212]
[INFO] [hlds_laser_publisher-2]: process started with pid [6213]
[INFO] [turtlebot3_ros-3]: process started with pid [6214]
[robot_state_publisher-1] Initialize urdf model from file: /home/ubuntu/turtlebot3_ws/install/turtlebot3_description/share/turtlebot3_description/urdf/turtlebot3_waffle.urdf
[robot_state_publisher-1] Parsing robot urdf xml string.
[robot_state_publisher-1] Link base_link had 7 children
[robot_state_publisher-1] Link camera_link had 2 children
[robot_state_publisher-1] Link camera_depth_frame had 1 children
[robot_state_publisher-1] Link camera_depth_optical_frame had 0 children
[robot_state_publisher-1] Link camera_rgb_frame had 1 children
[robot_state_publisher-1] Link camera_rgb_optical_frame had 0 children
[robot_state_publisher-1] Link caster_back_left_link had 0 children
[robot_state_publisher-1] Link caster_back_right_link had 0 children
[robot_state_publisher-1] Link imu_link had 0 children
[robot_state_publisher-1] Link base_scan had 0 children
[robot_state_publisher-1] Link wheel_left_link had 0 children
[robot_state_publisher-1] Link wheel_right_link had 0 children
[robot_state_publisher-1] got segment base_footprint
[robot_state_publisher-1] got segment base_link
[robot_state_publisher-1] got segment base_scan
[robot_state_publisher-1] got segment camera_depth_frame
[robot_state_publisher-1] got segment camera_depth_optical_frame
[robot_state_publisher-1] got segment camera_link
[robot_state_publisher-1] got segment camera_rgb_frame
[robot_state_publisher-1] got segment camera_rgb_optical_frame
[robot_state_publisher-1] got segment caster_back_left_link
[robot_state_publisher-1] got segment caster_back_right_link
[robot_state_publisher-1] got segment imu_link
[robot_state_publisher-1] got segment wheel_left_link
[robot_state_publisher-1] got segment wheel_right_link
[hlds_laser_publisher-2] [INFO] [hlds_laser_publisher]: Init hlds_laser_publisher Node Main
[hlds_laser_publisher-2] [INFO] [hlds_laser_publisher]: port : /dev/ttyUSB0 frame_id : base_scan
[ERROR] [hlds_laser_publisher-2]: process has died [pid 6213, exit code 255, cmd '/opt/ros/dashing/lib/hls_lfcd_lds_driver/hlds_laser_publisher __node:=hlds_laser_publisher __params:=/tmp/launch_params_xeeaz84i'].
[turtlebot3_ros-3] [INFO] [turtlebot3_node]: Init TurtleBot3 Node Main
[turtlebot3_ros-3] [INFO] [turtlebot3_node]: Init DynamixelSDKWrapper
[robot_state_publisher-1] Adding fixed segment from base_footprint to base_link
[robot_state_publisher-1] Adding fixed segment from base_link to camera_link
[robot_state_publisher-1] Adding fixed segment from camera_link to camera_depth_frame
[robot_state_publisher-1] Adding fixed segment from camera_depth_frame to camera_depth_optical_frame
[robot_state_publisher-1] Adding fixed segment from camera_link to camera_rgb_frame
[robot_state_publisher-1] Adding fixed segment from camera_rgb_frame to camera_rgb_optical_frame
[robot_state_publisher-1] Adding fixed segment from base_link to caster_back_left_link
[robot_state_publisher-1] Adding fixed segment from base_link to caster_back_right_link
[robot_state_publisher-1] Adding fixed segment from base_link to imu_link
[robot_state_publisher-1] Adding fixed segment from base_link to base_scan
[robot_state_publisher-1] Adding moving segment from base_link to wheel_left_link
[robot_state_publisher-1] Adding moving segment from base_link to wheel_right_link
[turtlebot3_ros-3] [INFO] [DynamixelSDKWrapper]: Succeeded to open the port(/dev/ttyACM0)!
[turtlebot3_ros-3] [INFO] [DynamixelSDKWrapper]: Succeeded to change the baudrate!
[turtlebot3_ros-3] [ERROR] [turtlebot3_node]: Failed connection with Devices
[turtlebot3_ros-3] [INFO] [turtlebot3_node]: Add Motors
[turtlebot3_ros-3] [INFO] [turtlebot3_node]: Add Wheels
[turtlebot3_ros-3] [INFO] [turtlebot3_node]: Add Sensors
[turtlebot3_ros-3] terminate called after throwing an instance of 'rclcpp::exceptions::RCLError'
[turtlebot3_ros-3]   what():  could not create publisher: rcl node's context is invalid, at /tmp/binarydeb/ros-dashing-rcl-0.7.9/src/rcl/node.c:476
[ERROR] [turtlebot3_ros-3]: process has died [pid 6214, exit code -6, cmd '/home/ubuntu/turtlebot3_ws/install/turtlebot3_node/lib/turtlebot3_node/turtlebot3_ros -i /dev/ttyACM0 __params:=/home/ubuntu/turtlebot3_ws/install/turtlebot3_bringup/share/turtlebot3_bringup/param/waffle.yaml'].

Please describe the issue in detail.

I am customising the Turtlebot 3 with new additional Dynamixal motor to be added in front along with rear 2 wheels. I am using ROS 2 Turtlebot (https://github.com/ROBOTIS-GIT/turtlebot3/tree/ros2) and OpenCR Turtlebot3 ROS 2 Firmware (https://github.com/ROBOTIS-GIT/OpenCR/tree/master/arduino/opencr_arduino/opencr/libraries/turtlebot3_ros2). I have further changed (customised) the motor_driver.cpp and motor_driver.h file where Dynamixal2Arduino library is used to add the new motor id and other details such as functions. Changes are done in following file : https://github.com/ROBOTIS-GIT/OpenCR/blob/master/arduino/opencr_arduino/opencr/libraries/turtlebot3_ros2/src/turtlebot3/turtlebot3_motor_driver.cpp. For the third motor (named as centre) I have added following code in the motor driver file (only append code is shown) :

const uint8_t DXL_MOTOR_ID_FRONT
sync_write_param.id_count = 3;
sync_write_param.xel[CENTER].id = front_wheel_id_;    //**
sync_read_param.addr = 132;
sync_read_param.length = 4;
sync_read_param.id_count = 3;
sync_read_param.xel[CENTER].id = front_wheel_id_;    //**
bool Turtlebot3MotorDriver::set_torque(bool onoff)
{
bool ret = false;
sync_write_param.addr = 64;
sync_write_param.length = 1;
sync_write_param.xel[CENTER].data[0] = onoff;          //**
if(dxl.syncWrite(sync_write_param) == true){
ret = true;
torque_ = onoff;
}
return ret;
}
bool Turtlebot3MotorDriver::write_position(int32_t center_value)
{
bool ret = false;
sync_write_param.addr = 116;
sync_write_param.length = 4;
memcpy(sync_write_param.xel[CENTER].data, &center_value, sync_write_param.length);
if(dxl.syncWrite(sync_write_param)){
ret = true;
}
return ret;
}
// front_joint_value is the function of linear velocity and angular velocity coming from cmd_vel
dxl_comm_result= write_position(front_joint_value);

While uploading the firmware OpenCR shows no compilation error and new Dynamixal motor is also initialised but while launching the ROS2 node from robot.launch.py (from Turtlebot 3 ROS2 Library) the errors is displayed as shown above in section 8. Furthermore, to eliminate this error if I write following commands :
$ export OPENCR_PORT=/dev/ttyACM0
$ export OPENCR_MODEL=burger
$ rm -rf ./opencr_update.tar.bz2
$ wget https://github.com/ROBOTIS-GIT/OpenCR-Binaries/raw/master/turtlebot3/ROS1/latest/opencr_update.tar.bz2
$ tar -xvf opencr_update.tar.bz2
$ cd ./opencr_update
$ ./update.sh $OPENCR_PORT $OPENCR_MODEL.opencr
Then the error is eliminated but third motor doesn't work only other two motors work properly. I am sending the cmd_vel values from MATLAB which is also on ROS2 Network.
Can someone please explain, how I can make third motor work run properly without the above shown erro ? Do I need to change the code in other TurtleBot src code apart from MotorDriver.cpp & .h and what changes need to be done ?
Thanks

For ROS2 TurtleBot3, OpenCR works as a slave device like DYNAMIXEL.
OpenCR creates its own control table which consists of memory address and data byte in each address as below.
https://github.com/ROBOTIS-GIT/OpenCR/blob/master/arduino/opencr_arduino/opencr/libraries/turtlebot3_ros2/src/turtlebot3/turtlebot3.cpp#L109
Using this control table, OpenCR and the SBC such as Raspberry Pi communicate to each other with DYNAMIXEL Protocol 2.0.
In order to add another DYNAMIXEL module to the platform, you'll need to create a specific control table address that will represent the specific item of the DYNAMIXEL such as Goal Position and Present Position, then use these values to control DYNAMIXEL from Raspberry Pi.

Thanks for your reply. I will work on this issue and will close this as soon as it is solved.

I have been able to connect the OpenCR with customised dynamixel motor (third motor) but still facing issues to perform communication between OpenCR and Raspberry PI Turtlebot 3 main node.  I can now control motor by giving input in the OpenCR (motor_driver.cpp) while uploading OpenCR turtlebot3 firmware but unable to control it from inputs from the ROS node (/cmd_vel topic Twist commands). When I launch "bringup" I am getting following error  "Unable to connect devices"  :
Figure 1

Following are the changes (presented in concise manner) I did, while working on this issue :
Task I : OpenCR - Dynamixel Interface (working fine)
1. MotorDriver Dynamixal Control Table (changes in OpenCR/Libraries/Turtlebot3_ros2/Turtlebot3_motor_drover .h &  .cpp )
Figure 2

2.  Open CR Control Table & Callback (changes in OpenCR/Libraries/Turtlebot3_ros2/Turtlebot3 .cpp )
Figure 3

Post first and second step after uploading Turtlebot3_waffle.ino file on OpenCR, new dynamixel motor is running properly based on inputs to linear value and angular value inputs.
Task II : Rapberry Pi - OpenCR  Interface (query !!! error reference from figure 1 : Device not added)
_3.  Turtlebot3 ROS2 & Dynamixel Wrapper (changes in Turtlebot3_ros2/Turtlebot3_node/Turtlebot3 .cpp & ControlTable.hpp)
I have added same address of control item parameters in controltable.hpp for 3rd motor same as  added in OpenCR in figure 2. After colcon build, i am recieving error "" Device not added" which is related to TurtleBot3::check_device_status() and further to Dynamic SDK Wrapper bool DynamixelSDKWrapper::read_register. I would be thankful if you can suggest me what i am missing in this process which is causing that error ? Do I need to do some changes in Dynamixal SDK Wrapper.cpp code ? Is 3rd Motor device_id is not getting register causing broken communication ?
Figure 4

Thank you for taking time to solve this error.

For ROS2 TurtleBot3, OpenCR works as a slave device like DYNAMIXEL.
OpenCR creates its own control table which consists of memory address and data byte in each address as below.
https://github.com/ROBOTIS-GIT/OpenCR/blob/master/arduino/opencr_arduino/opencr/libraries/turtlebot3_ros2/src/turtlebot3/turtlebot3.cpp#L109
Using this control table, OpenCR and the SBC such as Raspberry Pi communicate to each other with DYNAMIXEL Protocol 2.0.
In order to add another DYNAMIXEL module to the platform, you'll need to create a specific control table address that will represent the specific item of the DYNAMIXEL such as Goal Position and Present Position, then use these values to control DYNAMIXEL from Raspberry Pi.

Hi @amaheshwari2509
Please understand that we do not provide a thorough support on customizing the source code.
I'm not sure how your system is designed, but assuming that cmd_vel is used to drive 2 DYNAMIXEL modules and to steer the third module.
I'll try to write an example regarding adding a third DYNAMIXEL to TurtleBot3, but please allow me some time as I have other imminent tasks to be done by this month.
I believe you are almost getting it, and just missing a few minor things.

Hi @amaheshwari2509
Please understand that we do not provide a thorough support on customizing the source code.
I'm not sure how your system is designed, but assuming that cmd_vel is used to drive 2 DYNAMIXEL modules and to steer the third module.
I'll try to write an example regarding adding a third DYNAMIXEL to TurtleBot3, but please allow me some time as I have other imminent tasks to be done by this month.
I believe you are almost getting it, and just missing a few minor things.

Sure, I acknowledge the same. Yes 'cmd_vel' is used for all the three motors (2 propulsion + 1 steering). Will look forward to your response in future. In meantime, I will also try to resolve "opencr-raspberrypi ros2 node communication" issue by myself.  Thank again for your time to guide through this issue.

Thanks for your understanding.
I should be able to put some time for this before the mid of July.
Please feel free to update any progress via this thread.

Hello @ROBOTIS-Will , hope you doing good. Did you got time to look into above issue ? As from my understanding for this issue, Rpi-OpenCR are unable to communicate which can be acknowledge from "Failed connection with devices." while launching. While debugging, I tracked for the function  call which might cause this i.e. check_device_status() , is_connected_to_devices() and to read_register() but unable to solve the same. Thanks a lot again.

@amaheshwari2509
While I was testing the Arduino code last week, I also experienced the OpenCR does not communicate well with the RPi and getting the connection failure when bringup.
It looks like the control table created in the OpenCR does not respond to the DYNAMIXEL instruction from the Raspberry Pi.
I'll investigate more on this and get back to you.
Sorry about your inconvenience.

@ROBOTIS-Will
I would like to thank you for looking into this issue in detail. I acknowledge your feedback on this bug issue. I will look forward to hear from you once it is solved from your side. Meanwhile, I  will also keep digging into this issue. If you could suggest me any particular bit to look out for , please let me know  and I shall also keep digging into this issue to resolve this.
Kind Regards

Hi @amaheshwari2509
Thank you for your help on this.
The Control Table may not be properly created in the OpenCR as I cannot detect ID 200 when I tried scanning the OpenCR with Wizard 2.0 through dev/ttyACM0 port. Or OpenCR may not even responding to the DYNAMIXEL Protocol.
Another possible cause of the issue is that the DynamixelSDKWrapper::read_register() in the dynamixel_sdk_wrapper.cpp fails retrieving the RX packet from the OpenCR. I still need to investigate on this, but I'm guessing that the problem is mainly caused by the OpenCR side.

Hello @ROBOTIS-Will
Has the bug related to RPi - OpenCR communication for ROS2 has been solved ? Looking forward to your reply.
Thanks

Hi @amaheshwari2509
Thank you for your help on this.
The Control Table may not be properly created in the OpenCR as I cannot detect ID 200 when I tried scanning the OpenCR with Wizard 2.0 through dev/ttyACM0 port. Or OpenCR may not even responding to the DYNAMIXEL Protocol.
Another possible cause of the issue is that the DynamixelSDKWrapper::read_register() in the dynamixel_sdk_wrapper.cpp fails retrieving the RX packet from the OpenCR. I still need to investigate on this, but I'm guessing that the problem is mainly caused by the OpenCR side.

Hi @amaheshwari2509
Thank you for the friendly reminder on this issue.
I'm still working on this issue found some discrepancy between the binary and the source code since 2019, but wasn't able to spend enough time to dig deeper due to an imminent hardware update related to TurtleBot3.
I'll keep you updated if I find something helpful to resolve the issue.
Thanks!

@amaheshwari2509
A small update while we're keep working on this.
Using the OpenCR Arduino Board Manager (1.4.12) with DYNAMIXEL2Arduino library (0.3.0) helps to resolve the issue.
Below fixes should be applied however.

modify the platform.txt file as below.

## Compile c files
recipe.c.o.pattern="{compiler.path}{compiler.c.cmd}" {compiler.c.flags} -mcpu={build.mcu} -DF_CPU={build.f_cpu} -DARDUINO={runtime.ide.version} -DARDUINO_{build.board} -DARDUINO_ARCH_{build.arch} {compiler.c.extra_flags} {build.extra_flags} {build.cpu_flags} {build.common_flags} {compiler.libs.c.flags} {includes} "{source_file}" -o "{object_file}"
###

## Compile c++ files
recipe.cpp.o.pattern="{compiler.path}{compiler.cpp.cmd}" {compiler.cpp.flags} -mcpu={build.mcu} -DF_CPU={build.f_cpu} -DARDUINO={runtime.ide.version} -DARDUINO_{build.board} -DARDUINO_ARCH_{build.arch} {compiler.cpp.extra_flags} {build.extra_flags} {build.cpu_flags} {build.hs_flag} {build.common_flags} {compiler.libs.c.flags} {includes} "{source_file}" -o "{object_file}"

Add TORQUE_ENABLE definition in the turtlebot3_motor_driver.h file as below.

#include <Dynamixel2Arduino.h>

#define TORQUE_ENABLE 1

@amaheshwari2509
A small update while we're keep working on this.
Using the OpenCR Arduino Board Manager (1.4.12) with DYNAMIXEL2Arduino library (0.3.0) helps to resolve the issue.
Below fixes should be applied however.

modify the platform.txt file as below.

## Compile c files
recipe.c.o.pattern="{compiler.path}{compiler.c.cmd}" {compiler.c.flags} -mcpu={build.mcu} -DF_CPU={build.f_cpu} -DARDUINO={runtime.ide.version} -DARDUINO_{build.board} -DARDUINO_ARCH_{build.arch} {compiler.c.extra_flags} {build.extra_flags} {build.cpu_flags} {build.common_flags} {compiler.libs.c.flags} {includes} "{source_file}" -o "{object_file}"
###

## Compile c++ files
recipe.cpp.o.pattern="{compiler.path}{compiler.cpp.cmd}" {compiler.cpp.flags} -mcpu={build.mcu} -DF_CPU={build.f_cpu} -DARDUINO={runtime.ide.version} -DARDUINO_{build.board} -DARDUINO_ARCH_{build.arch} {compiler.cpp.extra_flags} {build.extra_flags} {build.cpu_flags} {build.hs_flag} {build.common_flags} {compiler.libs.c.flags} {includes} "{source_file}" -o "{object_file}"

Add TORQUE_ENABLE definition in the turtlebot3_motor_driver.h file as below.

#include <Dynamixel2Arduino.h>

#define TORQUE_ENABLE 1

Thank You @ROBOTIS-Will  for your assistance in solving this issue. I will perform these changes and will test on my system. I would like to thank you for solving this issue and sharing your feedback during this process.
Best Regards


DI scoping issue when not using statics

I'm having trouble hooking Unleash up in IServiceCollection.  We're trying to use the DI IDisposable BeginScope() to store an IUnleashContextProvider backed by a Dictionary<string, string> as opposed to relying on HttpContext.Current.Items as in the README.  The reason is b/c we use the scope pattern outside of ASP.NET Core and would like to be able to use the same Unleash boilerplate code regardless of whether we're in an ASP.NET Core app, a generic host builder, or a simple console app (creating a scope per ASP.NET request, MQ message received, timer interval, etc.).
This almost works, except that the IUnleashContextProvider is a property on the UnleashSettings class.  IUnleash is registered as a singleton (per the docs), IUnleashContextProvider is registered as Scoped (also recommended in the docs and critical to capture request/message/etc-specific scope).  UnleashSettings is registered as a singleton.
When ASPNETCORE_ENVIRONMENT=development, IServiceCollection.BuildServiceProvider(true) is called by the framework, which ensures that we're not capturing scoped/transient dependencies in a singleton.  As far as I can tell it's unavoidable given the current architecture... I tried (see here: https://github.com/scottt732/UnleashClientTests/tree/master/UnleashClientTests).
DefaultUnleash(singleton) -> UnleashSettings(singleton) -> UnleashContextProvider(necessarily scoped, effectively captured) = invalid (single state would be shared across requests/race conditions).
The example code in the README ~gets around this issue by effectively having the UnleashContextProvider as a singleton which refers to HttpContext.Current?.Items.  It will happen to be there in an ASP.NET request because your BeginRequest handler puts it there, but this requires using a static for state and breaks the ability for the same configuration to work in different application types.
I think a solution to this would be to decouple DefaultUnleash into 2 pieces--the thing that sets up services & manages the timers and the thing that answers IsEnabled questions.  This would remove the IUnleashContextProvider from the UnleashSettings class.
// Configuration
services.AddSingleton<IStrategy, SomeStrategy>();
services.AddSingleton<UnleashSettings>(new UnleashSettings { ... });

// depends on settings & strategies, disposed at app end
services.AddSingleton<IUnleashServiceManager, UnleashServiceManager>();

// Request-specific
services.AddScoped<IUnleashContextProvider, DefaultUnleashContextProvider>();
services.AddScoped<IUnleash, DefaultUnleash>();

This would allow the IsEnabled calls to refer to the scoped IUnleashContextProvider

This decoupling completely makes sense, maybe a third piece, responsible to store the cached features (currently storing as a .json for now)

For sure.  I've been trying to wrap my head around what the implications are of having a .json cache on each node behind a load balancer as opposed to something like Redis.  Any thoughts there?  I haven't looked at what the other clients are doing.
Should I open another issue to track that?  I had one other about synchronously blocking on the first HTTP round-trip (so our app doesn't service any requests based on a stale cache/default values).
Also are you open to accepting pull requests for any of this?  I'm happy to get my hands dirty.

@scottt732 issue https://github.com/Unleash/unleash-client-core/issues/37 created only about decoupling strategies


chore: acir format cleanup

Refactor to remove redundancy in acir format (there used to be 4 copy pastes of 1 big function. shocking).
Remove composer_ as member var from acir composer for greater safety.
Fix a bug in the RAM constraint whereby setting invalid witness to true, hard baked a 0 constant into the pk causing subsequent runs to fail.

Benchmark results
All benchmarks are run on txs on the Benchmarking contract on the repository. Each tx consists of a batch call to create_note and increment_balance, which guarantees that each tx has a private call, a nested private call, a public call, and a nested public call, as well as an emitted private note, an unencrypted log, and public storage read and write.
L2 block published to L1
Each column represents the number of txs on an L2 block published to L1.

Metric
8 txs
32 txs
128 txs

l1_rollup_calldata_size_in_bytes
45444
179588
716132

l1_rollup_calldata_gas
222960
868196
3449372

l1_rollup_execution_gas
842047
3595304
22204741

l2_block_processing_time_in_ms
1049
3948
15581

note_successful_decrypting_time_in_ms
328
1010
3955

note_trial_decrypting_time_in_ms
29
107
135

l2_block_building_time_in_ms
9075
36135
150522

l2_block_rollup_simulation_time_in_ms
6743
26888
106364

l2_block_public_tx_process_time_in_ms
2288
9120
43691

L2 chain processing
Each column represents the number of blocks on the L2 chain where each block has 16 txs.

Metric
10 blocks
20 blocks
30 blocks

node_history_sync_time_in_ms
31626
77277
137261

note_history_successful_decrypting_time_in_ms
4619
13158
20146

note_history_trial_decrypting_time_in_ms
209
251
260

node_database_size_in_bytes
1195148
1901127
2754918

pxe_database_size_in_bytes
54187
108338
162578

Circuits stats
Stats on running time and I/O sizes collected for every circuit run across all benchmarks.

Circuit
circuit_simulation_time_in_ms
circuit_input_size_in_bytes
circuit_output_size_in_bytes

private-kernel-init
55.72852760736196
56577
14745

private-kernel-ordering
30.065184049079754
20137
8089

base-rollup
872
631604
810

root-rollup
38.048780487804876
4072
1097

private-kernel-inner
51.71296296296296
72288
14745

public-kernel-private-input
51.76311728395062
37359
14745

public-kernel-non-first-iteration
31.369598765432098
37401
14745

merge-rollup
0.9224806201550387
2592
873


Wintersday offset update part 1

Debug, Player, and Environment Fog pointers still need updating. Major features aside from fog all work.

@nightlark thank you, I'll update the offsets this weekend as I'm still recovering from my illness

@karliky I figured out how to find the pointer for fog.


attempt to fix -e in actions

@kylecarow, what's the deal with this PR?

@kylecarow, what's the deal with this PR?

This explains it https://github.nrel.gov/MBAP/fastsim/issues/310
Installing the package will go faster if it is non-editable