Add testing instructions to the readme

Hi!
I understand this repo contains test data for multiqc. Is there a recommended way of running these tests locally?
Where I'm at:

I've made some changes to multiqc
I want to test them

In multiqc dev docs, I see that github actions will likely run the testdata against my changes to multiqc: https://github.com/ewels/MultiQC/blob/master/.github/CONTRIBUTING.md#review-workflow
But I also want to modify testdata too. I want to add a test and run it against my branch of multiqc. Feels like adding anything in the readme could be helpful!
If I figure something out I'll PR it.

I did figure this out to test https://github.com/ewels/MultiQC/pull/2170
Here's what I did:
docker run -it --rm \
    -v /home/j/third-party/MultiQC:/src/MultiQC \
    -v /home/j/third-party/MultiQC_TestData:/src/MultiQC_TestData \
    -w /src/MultiQC \
    python:3.12 \
    bash

pip install .
cd /src/MultiQC_TestData/data/modules/kraken
multiqc -f .

Not sure if this is the recommended way. If it is, I can cut a PR to document.

I (and I think most people) just have the two repos cloned locally with MultiQC installed locally (pip install -e .). Then you can do whatever you want in the test data repo and push a PR with new data if / when you want to.
This setup also makes it easy to check out different PRs on both MultiQC and the test data for development.
It's low tech, but it's served me well for 8 years now 😅  Is there something you're missing with this kind of setup?
Phil

Yeah, I guess this ended up being a bit silly. It involved installing from source in one repo and running in another, which in hindsight, isn't all that bad.
Also, when you run it, do you usually cd into a specific testdata dir, or is there a test suite that covers all them? I noticed some pytest files, not sure if just blindly running pytest is the way to go.
Anyway, I think we can close this.

The pytest files are leftover from an aborted attempt to write unit testing for MultiQC which never went through. For now yes, I just cd into a specific directoriy.
There's an issue on the roadmap to set up proper unit testing, we're hoping to get to it in 2024 if all goes well: https://github.com/ewels/MultiQC/issues/2046


Update schedule/init.py

commit

Coverage remained the same at 100.0% when pulling 2ac7d9bc2738ef1d57351238f9bc5b91eaffe7f9 on chenxuefeng1207:master into c2f14068ca52535c8c98b8bbf15dea3efe9273d7 on dbader:master.

Coverage remained the same at 100.0% when pulling 2ac7d9bc2738ef1d57351238f9bc5b91eaffe7f9 on chenxuefeng1207:master into c2f14068ca52535c8c98b8bbf15dea3efe9273d7 on dbader:master.

Coverage remained the same at 100.0% when pulling 2ac7d9bc2738ef1d57351238f9bc5b91eaffe7f9 on chenxuefeng1207:master into c2f14068ca52535c8c98b8bbf15dea3efe9273d7 on dbader:master.

Thanks!


Issues with pom packaged modules in multi module build

I now can't reproduce all the failure scenarios I saw earlier, but here are the instructions to see what I can share.
$ mkdir /tmp/test
$ cd /tmp/test
$ git clone https://github.com/symphonyoss/symphony-rest-tools.git
Cloning into 'symphony-rest-tools'...
remote: Counting objects: 134, done.
remote: Compressing objects: 100% (58/58), done.
remote: Total 134 (delta 27), reused 61 (delta 5), pack-reused 51
Receiving objects: 100% (134/134), 50.08 KiB | 0 bytes/s, done.
Resolving deltas: 100% (39/39), done.
$ cd symphony-rest-tools
$ mvn clean install
[INFO] Scanning for projects...
Downloading: https://oss.sonatype.org/content/groups/public/org/symphonyoss/symphonyoss/6/symphonyoss-6.pom
Downloaded: https://oss.sonatype.org/content/groups/public/org/symphonyoss/symphonyoss/6/symphonyoss-6.pom (19 KB at 2.9 KB/sec)
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building Symphony REST tools 0.1.1-SNAPSHOT
[INFO] ------------------------------------------------------------------------
**<SNIP>**
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 18.624 s
[INFO] Finished at: 2017-09-17T16:01:11-07:00
[INFO] Final Memory: 37M/597M
[INFO] ------------------------------------------------------------------------
$ cd ..
$ git clone https://github.com/symphonyoss/symphony-rest-tools-ui.git
Cloning into 'symphony-rest-tools-ui'...
remote: Counting objects: 106, done.
remote: Compressing objects: 100% (56/56), done.
remote: Total 106 (delta 30), reused 97 (delta 24), pack-reused 0
Receiving objects: 100% (106/106), 27.75 KiB | 0 bytes/s, done.
Resolving deltas: 100% (30/30), done.
$ cd symphony-rest-tools-ui/
$ mvn clean install
[INFO] Scanning for projects...
**<SNIP>**
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO]
[INFO] symphony-rest-tools-ui
[INFO] bundles
[INFO] org.symphonyoss.symphony.tools.rest.ui.dependencies
[INFO] org.symphonyoss.symphony.tools.rest.ui.console
[INFO] org.symphonyoss.symphony.tools.rest.ui
[INFO] features
[INFO] org.symphonyoss.symphony.tools.rest.ui.feature
[INFO] org.symphonyoss.symphony.tools.rest.ui.console.feature
[INFO] products
[INFO] update
[INFO] org.symphonyoss.symphony.tools.rest.ui.product
**<SNIP>**
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] symphony-rest-tools-ui ............................. SUCCESS [  0.162 s]
[INFO] bundles ............................................ SUCCESS [  0.005 s]
[INFO] org.symphonyoss.symphony.tools.rest.ui.dependencies  SUCCESS [  0.926 s]
[INFO] org.symphonyoss.symphony.tools.rest.ui.console ..... SUCCESS [  0.606 s]
[INFO] org.symphonyoss.symphony.tools.rest.ui ............. SUCCESS [  0.190 s]
[INFO] features ........................................... SUCCESS [  0.006 s]
[INFO] org.symphonyoss.symphony.tools.rest.ui.feature ..... SUCCESS [  0.078 s]
[INFO] org.symphonyoss.symphony.tools.rest.ui.console.feature SUCCESS [  0.068 s]
[INFO] products ........................................... SUCCESS [  0.005 s]
[INFO] update ............................................. SUCCESS [  1.605 s]
[INFO] org.symphonyoss.symphony.tools.rest.ui.product ..... SUCCESS [ 19.220 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 29.239 s
[INFO] Finished at: 2017-09-17T16:02:12-07:00
[INFO] Final Memory: 98M/1069M
[INFO] ------------------------------------------------------------------------
$

So far so good, we need the S2-super-pom now
$ cd ..
$ git clone https://github.com/SymphonyOSF/S2-super-pom.git
Cloning into 'S2-super-pom'...
remote: Counting objects: 21, done.
remote: Compressing objects: 100% (19/19), done.
remote: Total 21 (delta 8), reused 12 (delta 1), pack-reused 0
Unpacking objects: 100% (21/21), done.
$ cd S2-super-pom/
$ mvn clean install
[INFO] Scanning for projects...
**<SNIP>**
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 3.265 s
[INFO] Finished at: 2017-09-17T16:07:52-07:00
[INFO] Final Memory: 16M/207M
[INFO] ------------------------------------------------------------------------
$ cd ..
$ cd symphony-rest-tools-ui/

Now edit pom.xml and uncomment the parent pom entry
        <!-- <parent>
                <groupId>org.symphonyoss.s2</groupId>
                <artifactId>S2-super-pom</artifactId>
                <version>1.0.1-SNAPSHOT</version>
        </parent> -->

Now building results in this error:
$ mvn clean install
**<SNIP>**
[INFO] ------------------------------------------------------------------------
[INFO] Building update 1.0.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-clean-plugin:3.0.0:clean (default-clean) @ update ---
[INFO] Deleting /private/tmp/test/symphony-rest-tools-ui/products/update/target
[INFO]
[INFO] --- tycho-packaging-plugin:1.0.0:build-qualifier-aggregator (default-build-qualifier-aggregator) @ update ---
[INFO] The project's OSGi version is <IP_ADDRESS>709172309
[INFO]
[INFO] --- maven-clean-plugin:3.0.0:clean (default-clean-1) @ update ---
[INFO]
[INFO] --- maven-remote-resources-plugin:1.5:process (process-resource-bundles) @ update ---
/private/tmp/test/symphony-rest-tools-ui/features/org.symphonyoss.symphony.tools.rest.ui.console.feature/.polyglot.build.properties
/private/tmp/test/symphony-rest-tools-ui/features/org.symphonyoss.symphony.tools.rest.ui.feature/.polyglot.build.properties
[INFO]
[INFO] --- maven-resources-plugin:3.0.2:resources (default-resources) @ update ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /private/tmp/test/symphony-rest-tools-ui/products/update/src/main/resources
[INFO] Copying 3 resources
[INFO]
[INFO] --- target-platform-configuration:1.0.0:target-platform (default-target-platform) @ update ---
[INFO]
[INFO] --- tycho-p2-publisher-plugin:1.0.0:publish-products (default-publish-products) @ update ---
[INFO]
[INFO] --- tycho-p2-publisher-plugin:1.0.0:publish-categories (default-publish-categories) @ update ---
[INFO]
[INFO] --- tycho-p2-publisher-plugin:1.0.0:attach-artifacts (default-attach-artifacts) @ update ---
[INFO]
[INFO] --- tycho-p2-repository-plugin:1.0.0:assemble-repository (default-assemble-repository) @ update ---
[INFO] Copying resources from /private/tmp/test/symphony-rest-tools-ui/products/update/target/classes to /private/tmp/test/symphony-rest-tools-ui/products/update/target/repository
[INFO]
[INFO] --- tycho-p2-repository-plugin:1.0.0:archive-repository (default-archive-repository) @ update ---
[INFO] Building zip: /private/tmp/test/symphony-rest-tools-ui/products/update/target/update-1.0.0-SNAPSHOT.zip
[INFO]
[INFO] >>> maven-source-plugin:3.0.0:jar (default) > generate-sources @ update >>>
[INFO]
[INFO] --- tycho-packaging-plugin:1.0.0:build-qualifier-aggregator (default-build-qualifier-aggregator) @ update ---
[INFO] The project's OSGi version is <IP_ADDRESS>709172309
[INFO]
[INFO] --- maven-clean-plugin:3.0.0:clean (default-clean-1) @ update ---
[INFO] Deleting /private/tmp/test/symphony-rest-tools-ui/products/update/target
[INFO]
[INFO] <<< maven-source-plugin:3.0.0:jar (default) < generate-sources @ update <<<
[INFO]
[INFO] --- maven-source-plugin:3.0.0:jar (default) @ update ---
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] symphony-rest-tools-ui ............................. SUCCESS [  0.742 s]
[INFO] bundles ............................................ SUCCESS [  0.024 s]
[INFO] org.symphonyoss.symphony.tools.rest.ui.dependencies  SUCCESS [  1.997 s]
[INFO] org.symphonyoss.symphony.tools.rest.ui.console ..... SUCCESS [  0.749 s]
[INFO] org.symphonyoss.symphony.tools.rest.ui ............. SUCCESS [  0.483 s]
[INFO] features ........................................... SUCCESS [  0.034 s]
[INFO] org.symphonyoss.symphony.tools.rest.ui.feature ..... SUCCESS [  0.174 s]
[INFO] org.symphonyoss.symphony.tools.rest.ui.console.feature SUCCESS [  0.181 s]
[INFO] products ........................................... SUCCESS [  0.026 s]
[INFO] update ............................................. FAILURE [  2.964 s]
[INFO] org.symphonyoss.symphony.tools.rest.ui.product ..... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 13.983 s
[INFO] Finished at: 2017-09-17T16:09:54-07:00
[INFO] Final Memory: 78M/1013M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-source-plugin:3.0.0:jar (default) on project update: Error adding directory to source archive. /private/tmp/test/symphony-rest-tools-ui/products/update/target/maven-shared-archive-resources isn't a directory. -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :update
$

This error was previously happening in bundles and features too, now it seems to work and I have no idea why. Also it was the case that if you delete the part of bundles/pom.xml between
<!-- Should not be needed START -->

and

<!-- Should not be needed END -->

the build would fail, but now it works, again I don't understand why.

Probably not helpful, but I tried the commands listed above here, and it worked for me too, even after installing the S2-super-pom (which I was surprised worked, given I didn't think I had access to the SymphonyOSF GitHub organisation).

I made the repos we are trying to open source public so you can see S2-super-pom and S2-common.
Did you get and error after uncommenting the parent in symphony-rest-tools-ui/pom.xml?

Yep:
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] symphony-rest-tools-ui ............................. SUCCESS [  2.843 s]
[INFO] bundles ............................................ SUCCESS [  0.023 s]
[INFO] org.symphonyoss.symphony.tools.rest.ui.dependencies  SUCCESS [  3.158 s]
[INFO] org.symphonyoss.symphony.tools.rest.ui.console ..... SUCCESS [  0.989 s]
[INFO] org.symphonyoss.symphony.tools.rest.ui ............. SUCCESS [  0.405 s]
[INFO] features ........................................... SUCCESS [  0.017 s]
[INFO] org.symphonyoss.symphony.tools.rest.ui.feature ..... SUCCESS [  0.141 s]
[INFO] org.symphonyoss.symphony.tools.rest.ui.console.feature SUCCESS [  0.582 s]
[INFO] products ........................................... SUCCESS [  0.017 s]
[INFO] update ............................................. FAILURE [  1.942 s]
[INFO] org.symphonyoss.symphony.tools.rest.ui.product ..... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 18.530 s
[INFO] Finished at: 2017-09-17T17:52:37-07:00
[INFO] Final Memory: 75M/857M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-source-plugin:3.0.0:jar (default) on project update: Error adding directory to source archive.: /Users/pmonks/Development/symphony/symphony-rest-tools-ui/products/update/target/maven-shared-archive-resources isn't a directory. -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :update

Thanks for the issue. I could reproduce the problem, which is caused by the https://github.com/symphonyoss/symphony-rest-tools-ui/blob/master/products/update/pom.xml (and similar) module, where the source-plugin fails because the target folder is not created during the build. For these modules, we need to skip the execution of the maven-source-plugin.
By running mvn package -Dmaven.source.skip=true I get another build issue, which makes me think that it fixes the current issue; @bruceskingle - could you please try this build command locally? If your build succeeds, I can make this parameter configurable and add the proper documentation in the README.md.
An alternative approach is to move the maven-source-plugin logic into a dedicated profile (ie -Pattach-sources), so that it doesn't get enabled in the default build lifecycle. I personally prefer the former solution, as it doesn't have any impact on the projects that are currently using this parent POM.

I've analysed further the issue... it seems that the maven-install-plugin ignores that the source-plugin is skipped, as it logs the following (right before failing):
[INFO] No primary artifact to install, installing attached artifacts instead.

The only short term fix for the issue is to isolate the maven-source-plugin logic into a dedicated profile called -Pattach-sources; I'll make a final test as soon as I deploy a SNAPSHOT version of the Parent POM.
In parallel, I'll try to fix the issue on the maven-install-plugin, shouldn't be very hard.

@bruceskingle - could you please using the following parent pom (on S2-super-pom project) and try again?
<parent>
	<groupId>org.symphonyoss</groupId>
	<artifactId>symphonyoss</artifactId>
	<version>8-SNAPSHOT</version>
</parent>

8-SNAPSHOT does not resolve and 7 and 7-SNAPSHOT don't fix the problem. Note that S2-super-pom points to the LLC artefactory repo, somehow your 7 and 7-SNAPSHOT versions are in there but 8 isn't.
Do I need to add your snapshot repo to the pom?
[ERROR] The build could not read 1 project -> [Help 1]
[ERROR]
[ERROR]   The project org.symphonyoss.s2:S2-super-pom:1.0.1-SNAPSHOT (/Users/bruce.skingle/symphony/git/S2-super-pom/pom.xml) has 1 error
[ERROR]     Non-resolvable parent POM for org.symphonyoss.s2:S2-super-pom:1.0.1-SNAPSHOT: Could not find artifact org.symphonyoss:symphonyoss:pom:8-SNAPSHOT in snapshots (https://repo.symphony.com/artifactory/libs-snapshot) and 'parent.relativePath' points at wrong local POM @ line 36, column 11 -> [Help 2]

Is it possible to control the activation of that plugin based on the pom packaging type?

Is it possible to control the activation of that plugin based on the pom packaging type?

Unfortunately I couldn't find a way to configure this activation, the only similar config is skipSource, see https://maven.apache.org/plugins/maven-source-plugin/jar-no-fork-mojo.html, though it leads to the issue on maven-install-plugin, as documented above.
Re. the test of 8-SNAPSHOT version, I forgot to ask you to add the following repository in your pom.xml (this won't be necessary as soon as I release version 8 on Maven Central), which is why you couldn't resolve it.
  <repositories>
        <repository>
            <id>sonatype-oss-public</id>
            <url>https://oss.sonatype.org/content/groups/public/</url>
            <releases>
                <enabled>true</enabled>
            </releases>
            <snapshots>
                <enabled>true</enabled>
            </snapshots>
        </repository>
  </repositories>

I can see the version 8-SNAPSHOT in the repository - https://oss.sonatype.org/#nexus-search;gav~org.symphonyoss~symphonyoss~~~
I'm quite confident this will fix the current issue.

Confirmed fixed, I haven't looked into what's happening in there but my build now works and I can remove the developers and other redundant sections in my aggregator poms, I have no idea why that problem came and then went.
I have just pushed a big set of commits including rewinding the versions on S2-common and S2-super-pom from 1.0.X to 0.1.X so if you have been building these I strongly suggest you clear out your local maven cache to avoid nasty surprises.
Thanks for your help with this.

Version 8 is now released on Maven Central, you can update the version and get rid of the sonatype-oss-public repository.
http://search.maven.org/#search|ga|1|symphonyoss


Curvenote --> MyST 🎉

This is a major push upstream of functionality that has been developed in curvenote over the last three years.
The major functionality added on this push is around pdf export, latex export, intersphinx support, citation support, docx export, and latex templating.
There are no changes to mystjs package beyond a few improvements of bugs to fix around code highlighting, all files are now living in a single git repo.
The changes have been highlighted here:
https://github.com/executablebooks/meta/issues/838
Directly after merging/testing this PR we will add another pass on docs, and issues for immediate next steps to continue cleaning things up.

Ok I am going to merge this in, as this PR is more about moving homes than actually reviewing the code. Future PRs will be smaller than 1900 commits!
That being said, anyone following along please feel free to continuing commenting on this PR if you have specific feedback about a package/file/etc.
There are many, many things that need to be fixed up, and the project is absolutely in beta (especially the CLI). We are aiming to make rapid progress over the coming weeks and months, and are also aiming to move into a more open process as things become more stable.
Please consider this a start in it's new EBP home! 🚀


Push notification

Notify the users when the transaction becomes successful
Notify the user once the hodling time is over.
I suggest push protocol to implement this feature.

Hey @SANTHOSH17-DOT I would like to work on this issue, can you assign it to me

Sure

@Yashs1623 Any updates on this issue?


having an error when trying to use the GPU instead of the CPU.

so I installed everything according to the documentation and everything was going smoothly until I tried enabling nvidia gpu support,
this is the command I ran on powershell (got it from the documentation):
$env:CMAKE_ARGS='-DLLAMA_CUBLAS=on'; poetry run pip install --force-reinstall --no-cache-dir llama-cpp-python
and this is the error I got:
$env:CMAKE_ARGS='-DLLAMA_CUBLAS=on'; poetry run pip install --force-reinstall --no-cache-dir llama-cpp-python
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Collecting llama-cpp-python
Downloading llama_cpp_python-0.2.46.tar.gz (36.7 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 36.7/36.7 MB 11.7 MB/s eta 0:00:00
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Collecting typing-extensions>=4.5.0 (from llama-cpp-python)
Downloading typing_extensions-4.9.0-py3-none-any.whl.metadata (3.0 kB)
Collecting numpy>=1.20.0 (from llama-cpp-python)
Downloading numpy-1.26.4-cp311-cp311-win_amd64.whl.metadata (61 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 61.0/61.0 kB ? eta 0:00:00
Collecting diskcache>=5.6.1 (from llama-cpp-python)
Downloading diskcache-5.6.3-py3-none-any.whl.metadata (20 kB)
Collecting jinja2>=2.11.3 (from llama-cpp-python)
Downloading Jinja2-3.1.3-py3-none-any.whl.metadata (3.3 kB)
Collecting MarkupSafe>=2.0 (from jinja2>=2.11.3->llama-cpp-python)
Downloading MarkupSafe-2.1.5-cp311-cp311-win_amd64.whl.metadata (3.1 kB)
Downloading diskcache-5.6.3-py3-none-any.whl (45 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 45.5/45.5 kB ? eta 0:00:00
Downloading Jinja2-3.1.3-py3-none-any.whl (133 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 133.2/133.2 kB ? eta 0:00:00
Downloading numpy-1.26.4-cp311-cp311-win_amd64.whl (15.8 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 15.8/15.8 MB 27.3 MB/s eta 0:00:00
Downloading typing_extensions-4.9.0-py3-none-any.whl (32 kB)
Downloading MarkupSafe-2.1.5-cp311-cp311-win_amd64.whl (17 kB)
Building wheels for collected packages: llama-cpp-python
Building wheel for llama-cpp-python (pyproject.toml) ... error
error: subprocess-exited-with-error
× Building wheel for llama-cpp-python (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [40 lines of output]
*** scikit-build-core 0.8.1 using CMake 3.28.3 (wheel)
*** Configuring CMake...
2024-02-22 11:25:06,603 - scikit_build_core - WARNING - Can't find a Python library, got libdir=None, ldlibrary=None, multiarch=None, masd=None
loading initial cache file C:\Users\Baraa\AppData\Local\Temp\tmpxvn8yp4z\build\CMakeInit.txt
-- Building for: Visual Studio 17 2022
-- Selecting Windows SDK version 10.0.22621.0 to target Windows 10.0.22635.
-- The C compiler identification is MSVC 19.39.33520.0
-- The CXX compiler identification is MSVC 19.39.33520.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.39.33519/bin/Hostx64/x64/cl.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.39.33519/bin/Hostx64/x64/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found Git: C:/Program Files/Git/cmd/git.exe (found version "2.43.0.windows.1")
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - not found
-- Found Threads: TRUE
-- Found CUDAToolkit: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.1/include (found version "12.1.66")
-- cuBLAS found
CMake Error at C:/Program Files/CMake/share/cmake-3.28/Modules/CMakeDetermineCompilerId.cmake:529 (message):
No CUDA toolset found.
Call Stack (most recent call first):
C:/Program Files/CMake/share/cmake-3.28/Modules/CMakeDetermineCompilerId.cmake:8 (CMAKE_DETERMINE_COMPILER_ID_BUILD)
C:/Program Files/CMake/share/cmake-3.28/Modules/CMakeDetermineCompilerId.cmake:53 (__determine_compiler_id_test)        C:/Program Files/CMake/share/cmake-3.28/Modules/CMakeDetermineCUDACompiler.cmake:135 (CMAKE_DETERMINE_COMPILER_ID)
vendor/llama.cpp/CMakeLists.txt:356 (enable_language)
  -- Configuring incomplete, errors occurred!

  *** CMake configuration failed
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for llama-cpp-python
Failed to build llama-cpp-python
ERROR: Could not build wheels for llama-cpp-python, which is required to install pyproject.toml-based projects

I had the exact same error. I re-installed CUDA and ran the command again, and it worked.

Thanks to jeff on the discord server he helped me solve the issue by copying some cuda files to a different visual studio folder.
"you might need to copy all the four files from C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.7\extras\visual_studio_integration\MSBuildExtensions, and paste them to C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\MSBuild\Microsoft\VC\v160\BuildCustomizations. These paths should be changed to your own."
and when I ran the command again it worked.

could you tell ,how to use on cpu ,i have issue with building wheel


checkstyle: 10.8.1 -> 10.9.1

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://github.com/checkstyle/checkstyle/releases.
meta.description for checkstyle is: Checks Java source against a coding standard
meta.homepage for checkstyle is: http://checkstyle.sourceforge.net/
Updates performed

Version update

To inspect upstream changes

Release on GitHub

Compare changes on GitHub

Impact

Checks done (click to expand)

built on NixOS
The tests defined in passthru.tests, if any, passed
found 10.9.1 with grep in /nix/store/iy5v0xgpc8i17l3yah157qkvi6fycv25-checkstyle-10.9.1
found 10.9.1 in filename of file in /nix/store/iy5v0xgpc8i17l3yah157qkvi6fycv25-checkstyle-10.9.1

Rebuild report (if merged into master) (click to expand)

1 total rebuild path(s)

1 package rebuild(s)

First fifty rebuilds by attrpath
checkstyle

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/iy5v0xgpc8i17l3yah157qkvi6fycv25-checkstyle-10.9.1 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A checkstyle https://github.com/r-ryantm/nixpkgs/archive/d3e4b26bb61355dc7089436576779b674c688e4a.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/iy5v0xgpc8i17l3yah157qkvi6fycv25-checkstyle-10.9.1
ls -la /nix/store/iy5v0xgpc8i17l3yah157qkvi6fycv25-checkstyle-10.9.1/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

1 package built:

checkstyle

Maintainer pings
cc @pSub for testing.

@ofborg eval

Result of nixpkgs-review pr 221776 run on aarch64-darwin 1


Do not import pych during install process.

There is a chicken/egg problem when importing pych.version in setup.py, which is used to install the pych module. Update setup.py to read the version.py file (not import it) and then grab the version/name.

Version number output correctly!


Update credits for Spanish translation

Did some update of the spanish translation credits:

Added surenames to @faliure, @alejandro8605 & @Pablosky12.
Added @leonelmore & @lfgdzdev as contributors (@lfgdzdev pushed his contributions throught @alejandro8605 account).
Added twitter handle for @Pablosky12.
Removed @pabloKz (is same person as @Pablosky12).

@leonelmore thank you for the PR! My guess is you wanted to change only README.md with credits, but right now this PR contains all changes you've done when you were translating content (and we have 78 files changed :scream: )
Could you please fix it so I could merge only credits change?

@leonelmore thank you for the PR! My guess is you wanted to change only README.md with credits, but right now this PR contains all the changes you've done when you were translating content (and we have 78 files changed 😱 )
Could you please fix it so I could merge only credits change?

@NataliaTepluhina sorry my bad! This PR was intended to be done in the local translation repo, I am closing it now.
Thank you!


Construction sites grant visibility

You can practically scout the entire map by placing construction sites.
The unbuilt one will act as scout towers.
It should be not allowed to build on unexplored terrain

yep, that's a bug


Auto logs out user in the case of a certain error from wso2. Fixes #266

This change is

Reviewed 2 of 2 files at r1.
Review status: all files reviewed at latest revision, all discussions resolved.

Comments from Reviewable

Review status: all files reviewed at latest revision, all discussions resolved.

Comments from Reviewable


soc: xtensa: nxp: invoke west sign only for SOF

west sign is used only for Sound Open Firmware (SOF) sample.
So, add this only if CONFIG_SOF is enabled.
Otherwise, when compiling other samples they're not building because of the rimage dependency.
Fixes: 5ae7bd84d3 ("soc: xtensa: nxp: invoke west sign at west build time")

Why not but I don't understand what problem this solves because I implemented --if-tool-available to make sure rimage is optional always - for instance for build-only twister tests that may or may not have rimage installed.

Why not but I don't understand what problem this solves because I implemented --if-tool-available to make sure rimage is optional always - for instance for build-only twister tests that may or may not have rimage installed.

Otherwise, when compiling other samples they're not building because of the rimage dependency.

That should not happen. Or did you mean: rimage is found while not wanted and it fails in some cases?

It fails, while building other samples, because it cannot find zephyr.ri.
Here's the error I got when compiling samples/subsys/ipc/openamp_rsc_table/:
nxa06898@lsv15040:~/data/zephyrproject/zephyr$ west build -p always -b nxp_adsp_imx8m samples/subsys/ipc/openamp_rsc_table/
-- west build: generating a build system
Loading Zephyr default modules (Zephyr base).
-- Application: /home/nxa06898/data/zephyrproject/zephyr/samples/subsys/ipc/openamp_rsc_table
-- CMake version: 3.21.1
-- Found Python3: /usr/bin/python3.8 (found suitable exact version "3.8.0") found components: Interpreter
...
-- Generating done
-- Build files have been written to: /home/nxa06898/data/zephyrproject/zephyr/build
-- west build: building application
ninja: error: 'zephyr/zephyr.elf', needed by 'zephyr/zephyr.ri', missing and no known rule to make it
FATAL ERROR: command exited with status 1: /usr/bin/cmake --build /opt/samba/nxa06898/zephyrproject/zephyr/build
nxa06898@lsv15040:~/data/zephyrproject/zephyr$

TBH is weird to me too, because, I don't get any error when building synchronization or hello_world samples.

Why not but I don't understand what problem this solves because I implemented --if-tool-available to make sure rimage is optional always - for instance for build-only twister tests that may or may not have rimage installed.

Otherwise, when compiling other samples they're not building because of the rimage dependency.

That should not happen. Or did you mean: rimage is found while not wanted and it fails in some cases?

It fails, while building other samples, because it cannot find zephyr.elf, needed by zephyr.ri. Here's the error I got when compiling samples/subsys/ipc/openamp_rsc_table/:
nxa06898@lsv15040:~/data/zephyrproject/zephyr$ west build -p always -b nxp_adsp_imx8m samples/subsys/ipc/openamp_rsc_table/
-- west build: generating a build system
Loading Zephyr default modules (Zephyr base).
-- Application: /home/nxa06898/data/zephyrproject/zephyr/samples/subsys/ipc/openamp_rsc_table
-- CMake version: 3.21.1
-- Found Python3: /usr/bin/python3.8 (found suitable exact version "3.8.0") found components: Interpreter
...
-- Generating done
-- Build files have been written to: /home/nxa06898/data/zephyrproject/zephyr/build
-- west build: building application
ninja: error: 'zephyr/zephyr.elf', needed by 'zephyr/zephyr.ri', missing and no known rule to make it
FATAL ERROR: command exited with status 1: /usr/bin/cmake --build /opt/samba/nxa06898/zephyrproject/zephyr/build
nxa06898@lsv15040:~/data/zephyrproject/zephyr$

TBH is weird to me too, because, I don't get any error when building synchronization or hello_world samples.

@marc-hb or it might be because for openamp_rsc_table sample it takes longer to build and get the .elf, compared with synchronization?
And, since the west build and west sign are done in parallel, the west sign starts before the .elf is done.

And, since the west build and west sign are done in parallel, the west sign starts before the .elf is done.

No that's not done in parallel, it would be really bad~ west sign has an explicit dependency on zephyr.elf as shown in this error message that you shared:
'zephyr/zephyr.elf', needed by 'zephyr/zephyr.ri', missing and no known rule to make it

My first impression is that some examples simply produce no zephyr.elf file for some reason.

samples/subsys/ipc/openamp_rsc_table/ fails to build much earlier with a clean tree. Can you please clone everything from scratch and try to reproduce?
cd zproj/zephyr
git checkout 75dfdb14a1ec
west update
west compare # all clean

rm -rf build/ # pristine works only most of the time
west build -p always -b nxp_adsp_imx8m samples/subsys/ipc/openamp_rsc_table/

-- west build: making build dir zproj/zephyr/build pristine
-- west build: generating a build system
Loading Zephyr default modules (Zephyr base).
-- Application: zproj/zephyr/samples/subsys/ipc/openamp_rsc_table
-- CMake version: 3.26.3
-- Found Python3: /usr/bin/python3.11 (found suitable exact version "3.11.3") found components: Interpreter
-- Cache files will be written to: .cache/zephyr
-- Zephyr version: 3.4.99 (zproj/zephyr)
-- Found west (found suitable version "1.0.99", minimum required is "0.14.0")
-- Board: nxp_adsp_imx8m
-- ZEPHYR_TOOLCHAIN_VARIANT not set, trying to locate Zephyr SDK
-- Found host-tools: zephyr 0.16.1 (zephyr-sdk-0.16.1)
-- Found toolchain: zephyr 0.16.1 (zephyr-sdk-0.16.1)
-- Found Dtc: /usr/bin/dtc (found suitable version "1.7.0", minimum required is "1.4.6")
-- Found BOARD.dts: zproj/zephyr/boards/xtensa/nxp_adsp_imx8m/nxp_adsp_imx8m.dts
-- Generated zephyr.dts: zproj/zephyr/build/zephyr/zephyr.dts
-- Generated devicetree_generated.h: zproj/zephyr/build/zephyr/include/generated/devicetree_generated.h
-- Including generated dts.cmake file: zproj/zephyr/build/zephyr/dts.cmake

warning: PRINTK (defined at subsys/debug/Kconfig:203) was assigned the value 'n' but got the value
'y'. See http://docs.zephyrproject.org/latest/kconfig.html#CONFIG_PRINTK and/or look up PRINTK in
the menuconfig/guiconfig interface. The Application Development Primer, Setting Configuration
Values, and Kconfig - Tips and Best Practices sections of the manual might be helpful too.

zproj/zephyr/samples/subsys/ipc/openamp_rsc_table/prj.conf:4: warning: attempt to assign the value 'n' to the undefined symbol PLATFORM_SPECIFIC_INIT
Parsing zproj/zephyr/Kconfig
Loaded configuration 'zproj/zephyr/boards/xtensa/nxp_adsp_imx8m/nxp_adsp_imx8m_defconfig'
Merged configuration 'zproj/zephyr/samples/subsys/ipc/openamp_rsc_table/prj.conf'

error: Aborting due to Kconfig warnings

CMake Error at zproj/zephyr/cmake/modules/kconfig.cmake:343 (message):
  command failed with return code: 1
Call Stack (most recent call first):
  zproj/zephyr/cmake/modules/zephyr_default.cmake:115 (include)
  zproj/zephyr/share/zephyr-package/cmake/ZephyrConfig.cmake:66 (include)
  zproj/zephyr/share/zephyr-package/cmake/ZephyrConfig.cmake:92 (include_boilerplate)
  CMakeLists.txt:7 (find_package)

-- Configuring incomplete, errors occurred!
FATAL ERROR: command exited with status 1: /usr/bin/cmake -DWEST_PYTHON=/usr/bin/python -Bzproj/zephyr/build -GNinja -DBOARD=nxp_adsp_imx8m -Szproj/zephyr/samples/subsys/ipc/openamp_rsc_table

samples/subsys/ipc/openamp_rsc_table/ fails to build much earlier in a clean tree: error: Aborting due to Kconfig warnings. Can you please clone everything from scratch and try to reproduce?
cd zproj/zephyr
git checkout 75dfdb14a1ec
west update
west compare # all clean

rm -rf build/ # pristine works only most of the time
west build -p always -b nxp_adsp_imx8m samples/subsys/ipc/openamp_rsc_table/

-- west build: making build dir zproj/zephyr/build pristine
-- west build: generating a build system
Loading Zephyr default modules (Zephyr base).
-- Application: zproj/zephyr/samples/subsys/ipc/openamp_rsc_table
-- CMake version: 3.26.3
-- Found Python3: /usr/bin/python3.11 (found suitable exact version "3.11.3") found components: Interpreter
-- Cache files will be written to: .cache/zephyr
-- Zephyr version: 3.4.99 (zproj/zephyr)
-- Found west (found suitable version "1.0.99", minimum required is "0.14.0")
-- Board: nxp_adsp_imx8m
-- ZEPHYR_TOOLCHAIN_VARIANT not set, trying to locate Zephyr SDK
-- Found host-tools: zephyr 0.16.1 (zephyr-sdk-0.16.1)
-- Found toolchain: zephyr 0.16.1 (zephyr-sdk-0.16.1)
-- Found Dtc: /usr/bin/dtc (found suitable version "1.7.0", minimum required is "1.4.6")
-- Found BOARD.dts: zproj/zephyr/boards/xtensa/nxp_adsp_imx8m/nxp_adsp_imx8m.dts
-- Generated zephyr.dts: zproj/zephyr/build/zephyr/zephyr.dts
-- Generated devicetree_generated.h: zproj/zephyr/build/zephyr/include/generated/devicetree_generated.h
-- Including generated dts.cmake file: zproj/zephyr/build/zephyr/dts.cmake

warning: PRINTK (defined at subsys/debug/Kconfig:203) was assigned the value 'n' but got the value
'y'. See http://docs.zephyrproject.org/latest/kconfig.html#CONFIG_PRINTK and/or look up PRINTK in
the menuconfig/guiconfig interface. The Application Development Primer, Setting Configuration
Values, and Kconfig - Tips and Best Practices sections of the manual might be helpful too.

zproj/zephyr/samples/subsys/ipc/openamp_rsc_table/prj.conf:4: warning: attempt to assign the value 'n' to the undefined symbol PLATFORM_SPECIFIC_INIT
Parsing zproj/zephyr/Kconfig
Loaded configuration 'zproj/zephyr/boards/xtensa/nxp_adsp_imx8m/nxp_adsp_imx8m_defconfig'
Merged configuration 'zproj/zephyr/samples/subsys/ipc/openamp_rsc_table/prj.conf'

error: Aborting due to Kconfig warnings

CMake Error at zproj/zephyr/cmake/modules/kconfig.cmake:343 (message):
  command failed with return code: 1
Call Stack (most recent call first):
  zproj/zephyr/cmake/modules/zephyr_default.cmake:115 (include)
  zproj/zephyr/share/zephyr-package/cmake/ZephyrConfig.cmake:66 (include)
  zproj/zephyr/share/zephyr-package/cmake/ZephyrConfig.cmake:92 (include_boilerplate)
  CMakeLists.txt:7 (find_package)

-- Configuring incomplete, errors occurred!
FATAL ERROR: command exited with status 1: /usr/bin/cmake -DWEST_PYTHON=/usr/bin/python -Bzproj/zephyr/build -GNinja -DBOARD=nxp_adsp_imx8m -Szproj/zephyr/samples/subsys/ipc/openamp_rsc_table

With a clean tree, the openamp_rsc_table sample is not compiling since there is no support for nxp_adsp_imx8m.
This sample is supported by STM32 - see https://github.com/zephyrproject-rtos/zephyr/tree/main/samples/subsys/ipc/openamp_rsc_table/boards.
I'm currently working on adding support for nxp_adsp_imx8m. Actually, I'm having all fixed, I'm at the part of code clean-up to put everything, for review, in upstream.
I've updated my repo to latest zephyr and I'm facing the compile error mentioned here
The first problem when compiling openamp_rsc_table sample for nxp_adsp_imx8m is the PLATFORM_SPECIFIC_INIT config, which is ARM specific (see here)  and we are compiling the sample for Xtensa HiFi4.
So, there are multiple issues to fix, in order to have this sample working on nxp_adsp_imx8m.
I'll send everything for review, soon.

Let's first make sure this fixes a real issue that can be reproduced in a clean tree.

As mentioned here, the openamp_rsc_table will not compile on a clean tree, for nxp_adsp_imx8m.
The fix proposed here, IMO is valid, since the west sign (or any other rimage commands) we want to use only for SOF.
Us, at NXP, we want to run other samples on HiFi4 DSP, not just SOF - https://eoss2023.sched.com/event/1LcNN
That's why I don't want to link all samples that will be compiled for HiFi with rimage (or a dependency for zephyr.ri).

The openamp_rsc_table sample has an elf but is called openamp_rsc_table.elf

Thanks! Now I can reproduce with
west build -p always -b nxp_adsp_imx8m samples/hello_world/ -- \
  -DCONFIG_KERNEL_BIN_NAME='"fubar"'

So you can forget about the more complex openamp_rsc_table example in this PR.
Unfortunately sign.py is hardcoded to zephyr.py but I may have some solution, I should know tomorrow. This needs to be fixed anyway.

Us, at NXP, we want to run other samples on HiFi4 DSP, not just SOF - https://eoss2023.sched.com/event/1LcNN
That's why I don't want to link all samples that will be compiled for HiFi with rimage

I understand you don't always want to run rimage but it seems much more flexible to turn rimage on/off with west config rather than a hardcoded dependency on SOF.
Also, it would be good to keep the divergence with soc/xtensa/intel_adsp/common/CMakeLists.txt to a minimum so they can be de-duplicated in the future.

I understand you don't always want to run rimage but it seems much more flexible to turn rimage on/off with west config rather than a hardcoded and somewhat artificial dependency on CONFIG_SOF.

I don't know that well west config, so I'm open to suggestions.
Thank you!

Also, it would be good to keep the divergence with soc/xtensa/intel_adsp/common/CMakeLists.txt to a minimum so they can be de-duplicated in the future (and rimage is definitely needed to run samples on Intel)

I agree!

I don't know that well west config, so I'm open to suggestions. Thank you!

Do NOT "install" rimage in your PATH
When you want to use it: west config rimage.path /path/to/build-rimage/rimage
When you want to stop using it: west config -d rimage.path.

I recently updated the rimage documentation in commit dd09b04dc33eb919

https://docs.zephyrproject.org/latest/develop/west/sign.html
https://docs.zephyrproject.org/latest/boards/xtensa/intel_adsp_cavs25/doc/intel_adsp_generic.html (not Intel-specific)
https://docs.zephyrproject.org/latest/develop/west/config.html

Unfortunately sign.py is hardcoded to zephyr.elf but I may have some solution, I should know tomorrow.

CONFIG_KERNEL_BIN_NAME fix submitted:

#59721

I understand you don't always want to run rimage but it seems much more flexible to turn rimage on/off with west config rather than a hardcoded and somewhat artificial dependency on CONFIG_SOF.

You later found that the root cause is CONFIG_KERNEL_BIN_NAME and that's unrelated to CONFIG_SOF.
west config offers the flexibility to turn rimage ON/OFF with zero code change. Tying rimage and CONFIG_SOF as this PR does would lose that flexibility: it would not be possible at all to use rimage on any other example/app - whatever its CONFIG_KERNEL_BIN_NAME is. I think that's way too restrictive.
This PR would also make CONFIG_KERNEL_BIN_NAME incompatible with CONFIG_SOF.
west build -p always -b nxp_adsp_imx8 ../modules/audio/sof/app/ --   -DCONFIG_KERNEL_BIN_NAME='"my_bin_name"'
 => build still fails the same even with this PR.

Not like I want to use CONFIG_KERNEL_BIN_NAME (I think it's a bad idea) but this example demonstrates that this PR is not aimed at the root cause.
So a totally different alternative would be:
if(CONFIG_KERNEL_BIN_NAME STREQUAL "zephyr")
  add_custom_command(
    west sign --if-tool-available ...

In other words: invoke west sign only for the CONFIG_urations it supports currently. How about it?

Unfortunately sign.py is hardcoded to zephyr.elf but I may have some solution, I should know tomorrow.

CONFIG_KERNEL_BIN_NAME fix submitted:

soc: adsp: cmake: fix sign.py when CONFIG_KERNEL_BIN_NAME is used #59721

I understand you don't always want to run rimage but it seems much more flexible to turn rimage on/off with west config rather than a hardcoded and somewhat artificial dependency on CONFIG_SOF.

You later found that the root cause is CONFIG_KERNEL_BIN_NAME and that's unrelated to CONFIG_SOF.
west config offers the flexibility to turn rimage ON/OFF with zero code change. Tying rimage and CONFIG_SOF as this PR does would lose that flexibility: it would not be possible at all to use rimage on any other example/app - whatever its CONFIG_KERNEL_BIN_NAME is. I think that's way too restrictive.
This PR would also make CONFIG_KERNEL_BIN_NAME incompatible with CONFIG_SOF:
west build -p always -b nxp_adsp_imx8 ../modules/audio/sof/app/ --   -DCONFIG_KERNEL_BIN_NAME='"my_bin_name"'
 => build still fails the same even with this PR.

Not like I want to use CONFIG_KERNEL_BIN_NAME (I think it's a bad idea) but this example demonstrates that this PR is not aimed at the root cause.
So a totally different alternative would be:
if(CONFIG_KERNEL_BIN_NAME STREQUAL "zephyr")
  add_custom_command(
    west sign --if-tool-available ...

In other words: invoke west sign only for the CONFIG_urations it supports currently. How about it?

@marc-hb I've tested #59721 on my samples for nxp_adsp_imx8m and is working perfectly.
Thanks for the fix!

Closing this since a better fix is in review #59721

So a totally different alternative would be:
if(CONFIG_KERNEL_BIN_NAME STREQUAL "zephyr")
add_custom_command(
west sign --if-tool-available ...
In other words: invoke west sign only for the CONFIG_urations it supports currently. How about it?

Submitted in:

#59726


Add config option to opt-out of generating CSRF token

This adds a config option ensure_csrf_token to opt-out of django-nextjs automatically generating a CSRF token to include in the request to the NextJS server.  It preserves the current behavior by defaulting to True.

The comment in _get_nextjs_request_cookies indicates this was done to support a GraphQL use case.  But in our project, the behavior is undesirable because of the side-effect of including a Set-Cookie header (see next bullet).

A new CSRF token value was being generated on every request.  This causes Django's CsrfViewMiddleware to include a Set-Cookie header on every response, making all responses from django-nextjs un-cacheable.  This updates the behavior to only generate a CSRF token if the settings.CSRF_COOKIE_NAME cookie is not already present in the request.

Thanks for the feedback, @danialkeimasi.  Please take a look again.

Looks good.


Implement transactional I2C interface and add example

I implemented the transactional I2C interface from https://github.com/rust-embedded/embedded-hal/pull/223 and added an example with a driver which does a combined Write/Read transaction.
This is based on previous work from #33 by @ryankurte, https://github.com/rust-embedded/rust-i2cdev/pull/51 and is similar to #35.

r? @nastevens
(rust_highfive has picked a reviewer for you, use r? to override)

Sorry, I haven't really been involved in these changes. Adding @ryankurte and @posborne instead as I believe they have.

@ryankurte Do you know why embedded_hal::blocking::i2c::transactional::Default ends up being imported here? That seems to be the cause for the conflicting implementation errors:
error[E0119]: conflicting implementations of trait `hal::prelude::_embedded_hal_blocking_i2c_Write` for type `I2cdev`:
   --> src/lib.rs:187:1
    |
187 | impl hal::blocking::i2c::Write for I2cdev {
    | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |
    = note: conflicting implementation in crate `embedded_hal`:
            - impl<E, S> hal::prelude::_embedded_hal_blocking_i2c_Write for S
              where <S as hal::prelude::_embedded_hal_blocking_i2c_Transactional>::Error == E, S: hal::blocking::i2c::transactional::Default<E>, S: hal::prelude::_embedded_hal_blocking_i2c_Transactional;
    = note: downstream crates may implement trait `hal::blocking::i2c::transactional::Default<_>` for type `I2cdev`

ahh, yeah whoops i should have caught that over on e-h. i believe it's because the trait bounds are different for Write (over address and errror) and Default (only over error), which gives the compiler nightmares.
you should be able to fix this in e-h by also binding the AddressMode across the default impl (same as is done in spi::transactional::Default for exactly the same reason), so it becomes pub trait Default<AddressMode, Error> {} and is used in all the impls and where clauses.
lmk how that goes / if you get stuck lmk and i can provide more guidance.

Ah, thank you! I added address mode support to the transactional traits over at https://github.com/rust-embedded/embedded-hal/pull/260.
However, the conflicting implementation error is still there. It disappears if I remove the explicit implementations of i2c::Read, i2c::Write and i2c::WriteRead, of course, but the point of the default implementation inside the i2c::transactional module was to make it optional for HALs to use it or not.
I still do not understand how the default implementation is being pulled in for I2cdev.
error[E0119]: conflicting implementations of trait `hal::prelude::_embedded_hal_blocking_i2c_Write` for type `I2cdev`:
   --> src/lib.rs:187:1
    |
187 | impl hal::blocking::i2c::Write for I2cdev {
    | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |
    = note: conflicting implementation in crate `embedded_hal`:
            - impl<A, E, S> hal::prelude::_embedded_hal_blocking_i2c_Write<A> for S
              where <S as hal::prelude::_embedded_hal_blocking_i2c_Transactional<A>>::Error == E, A: hal::blocking::i2c::AddressMode, S: hal::blocking::i2c::transactional::Default<A, E>, S: hal::prelude::_embedded_hal_blocking_i2c_Transactional<A>;
    = note: downstream crates may implement trait `hal::blocking::i2c::transactional::Default<u8, _>` for type `I2cdev`

ohh right, went back and looked and you've also gotta drop the error bound from the default type to that the transactional::Default trait, i believe this is required so there is no possibility of it existing over a foreign type.
sorry i missed that from the last advice, the following patch to e-h works for me.
diff --git a/src/blocking/i2c.rs b/src/blocking/i2c.rs
index 087d208..27b2092 100644
--- a/src/blocking/i2c.rs
+++ b/src/blocking/i2c.rs
@@ -334,12 +334,12 @@ pub mod transactional {

     /// Default implementation of `blocking::i2c::Write`, `blocking::i2c::Read` and
     /// `blocking::i2c::WriteRead` traits for `blocking::i2c::Transactional` implementers.
-    pub trait Default<A: AddressMode, E>: Transactional<A, Error = E> {}
+    pub trait Default<A: AddressMode>: Transactional<A> {}

-    impl<A, E, S> Write<A> for S
+    impl<A: 'static , E, S> Write<A> for S
     where
         A: AddressMode,
-        S: self::Default<A, E> + Transactional<A, Error = E>,
+        S: self::Default<A> + Transactional<A, Error = E>,
     {
         type Error = E;

@@ -351,7 +351,7 @@ pub mod transactional {
     impl<A, E, S> Read<A> for S
     where
         A: AddressMode,
-        S: self::Default<A, E> + Transactional<A, Error = E>,
+        S: self::Default<A> + Transactional<A, Error = E>,
     {
         type Error = E;

@@ -363,7 +363,7 @@ pub mod transactional {
     impl<A, E, S> WriteRead<A> for S
     where
         A: AddressMode,
-        S: self::Default<A, E> + Transactional<A, Error = E>,
+        S: self::Default<A> + Transactional<A, Error = E>,
     {
         type Error = E;

@ryankurte Interesting. That works, thank you!
I still do not understand why it is a problem if nobody imported transactional::Default, though.

i believe this is required so there is no possibility of it existing over a foreign type.
Could you elaborate or refer me to some docs about this?

Could you elaborate or refer me to some docs about this?

my thinking was that Address is sealed but Error is not, so you could plausibly create a new Error type somewhere else then try to impl Default over this, but, in digging into this more i have only confused myself further 😢 might have to chalk it up to experience painfully earned.
are we happy with / ready to land this then?

Yes, I am happy with the code itself as-is. However, to merge this we would need to make a new alpha release of embedded-hal and remove the patch to my branch.

I have updated this to use embedded-hal 1.0.0-alpha.4 and is now ready to merge.

Bors timed out again.

yeeeah, we really need to do #51 like all the other e-h repos

Hmm, but in this case Travis is not the problem. It is bors the one that timed out. Or am I missing something else?

i might be misunderstanding but i thought the bors timeouts were usually because travis doesn't schedule / execute the job quickly enough (because the ewg burns through hours)
bors retry

Ah, I see. That sounds possible. Thanks!


Add Kùzu graph database as a graph store

Hi,
This is a great project, looking forward to trying it out more and experimenting with some workflows!
I'm looking at the portions of the code that utilize networkx for the graph retrieval and/or visualization, and I was thinking, the existing use of Pandas DataFrames makes this workflow very amenable to using Kùzu, an embedded graph database that's very similar to DuckDB and LanceDB in philosophy (to be easy to deploy, and fully open source). Using a graph database with persistence and durability guarantees, rather than an in-memory database like NetworkX, is preferable. And the fact that Kùzu is embedded and open source makes it that much more simple and user-friendly, in the same way that LanceDB and DuckDB are.
It's trivial to read data into a Kùzu graph via Pandas, as described here. The benefit of using Kùzu, imo, over NetworkX, is that Kùzu can scale very well to out-of-memory data, and imo, it's the perfect compliment to LanceDB for users who are familiar with that database from a vector search perspective.
Additionally, it's also trivial to convert a Kùzu graph into a networkx Graph or DiGraph object, which can be used for all downstream workflows that require networkx objects.
The Microsoft GraphRAG repo also uses LanceDB as its default vector store, and the reason Kùzu isn't used there (they also leverage NetworkX for their graph computations) is that at the time of them writing their code, Kùzu wasn't well known enough. I think that's changing, as Kùzu is becoming more and more popular (disclaimer: I work at Kùzu).
I just wanted to create this issue so that this could be something that's on the roadmap, and I'd be happy to try out the framework more and offer my inputs as this project grows. Cheers!

that is malware to steal your account; do not under any circumstances download or run it. The post needs to be removed. If you have attempted to run it please have your system cleaned and your account secured immediately.

Look interesting enough. LanceDB was MS choice for simplicity I suppose. We will consider adding this to the roadmap.


handle context-specific tag incorrect

I have a signed data encoded by DER, but get strange output while parse
pub fn parse_signed_data(input: &[u8]) {
    let (_, top_object) = parse_der(input).unwrap();

    let pretty_object = top_object.as_pretty(4, 4);
    println!("{:?}", pretty_object);
}

The input
        let raw_bytes = [
            0x30, 0x82, 0x05, 0x20, 0x06, 0x09, 0x2A, 0x86, 0x48, 0x86, 0xF7, 0x0D, 0x01, 0x07, 0x02, 0xA0,
            0x82, 0x05, 0x11, 0x30, 0x82, 0x05, 0x0D, 0x02, 0x01, 0x01, 0x31, 0x0F, 0x30, 0x0D, 0x06, 0x09,
            0x60, 0x86, 0x48, 0x01, 0x65, 0x03, 0x04, 0x02, 0x01, 0x05, 0x00, 0x30, 0x0B, 0x06, 0x09, 0x2A,
            0x86, 0x48, 0x86, 0xF7, 0x0D, 0x01, 0x07, 0x01, 0xA0, 0x82, 0x03, 0x57, 0x30, 0x82, 0x03, 0x53,
            0x30, 0x82, 0x02, 0x3B, 0xA0, 0x03, 0x02, 0x01, 0x02, 0x02, 0x04, 0x77, 0xA4, 0xD5, 0xE2, 0x30,
            0x0D, 0x06, 0x09, 0x2A, 0x86, 0x48, 0x86, 0xF7, 0x0D, 0x01, 0x01, 0x0B, 0x05, 0x00, 0x30, 0x5A,
            0x31, 0x0B, 0x30, 0x09, 0x06, 0x03, 0x55, 0x04, 0x06, 0x13, 0x02, 0x43, 0x4E, 0x31, 0x0B, 0x30,
            0x09, 0x06, 0x03, 0x55, 0x04, 0x08, 0x13, 0x02, 0x53, 0x48, 0x31, 0x0B, 0x30, 0x09, 0x06, 0x03,
            0x55, 0x04, 0x07, 0x13, 0x02, 0x53, 0x48, 0x31, 0x0F, 0x30, 0x0D, 0x06, 0x03, 0x55, 0x04, 0x0A,
            0x13, 0x06, 0x71, 0x71, 0x71, 0x71, 0x71, 0x71, 0x31, 0x0F, 0x30, 0x0D, 0x06, 0x03, 0x55, 0x04,
            0x0B, 0x13, 0x06, 0x71, 0x71, 0x71, 0x71, 0x71, 0x71, 0x31, 0x0F, 0x30, 0x0D, 0x06, 0x03, 0x55,
            0x04, 0x03, 0x13, 0x06, 0x71, 0x71, 0x71, 0x71, 0x71, 0x71, 0x30, 0x1E, 0x17, 0x0D, 0x32, 0x30,
            0x31, 0x31, 0x30, 0x38, 0x30, 0x36, 0x33, 0x39, 0x34, 0x33, 0x5A, 0x17, 0x0D, 0x33, 0x30, 0x31,
            0x31, 0x30, 0x36, 0x30, 0x36, 0x33, 0x39, 0x34, 0x33, 0x5A, 0x30, 0x5A, 0x31, 0x0B, 0x30, 0x09,
            0x06, 0x03, 0x55, 0x04, 0x06, 0x13, 0x02, 0x43, 0x4E, 0x31, 0x0B, 0x30, 0x09, 0x06, 0x03, 0x55,
            0x04, 0x08, 0x13, 0x02, 0x53, 0x48, 0x31, 0x0B, 0x30, 0x09, 0x06, 0x03, 0x55, 0x04, 0x07, 0x13,
            0x02, 0x53, 0x48, 0x31, 0x0F, 0x30, 0x0D, 0x06, 0x03, 0x55, 0x04, 0x0A, 0x13, 0x06, 0x71, 0x71,
            0x71, 0x71, 0x71, 0x71, 0x31, 0x0F, 0x30, 0x0D, 0x06, 0x03, 0x55, 0x04, 0x0B, 0x13, 0x06, 0x71,
            0x71, 0x71, 0x71, 0x71, 0x71, 0x31, 0x0F, 0x30, 0x0D, 0x06, 0x03, 0x55, 0x04, 0x03, 0x13, 0x06,
            0x71, 0x71, 0x71, 0x71, 0x71, 0x71, 0x30, 0x82, 0x01, 0x22, 0x30, 0x0D, 0x06, 0x09, 0x2A, 0x86,
            0x48, 0x86, 0xF7, 0x0D, 0x01, 0x01, 0x01, 0x05, 0x00, 0x03, 0x82, 0x01, 0x0F, 0x00, 0x30, 0x82,
            0x01, 0x0A, 0x02, 0x82, 0x01, 0x01, 0x00, 0xA7, 0xE0, 0x84, 0x4E, 0x72, 0x99, 0xE0, 0x0F, 0x4A,
            0x91, 0x06, 0x08, 0x03, 0xC7, 0x8C, 0x10, 0xB5, 0x54, 0x20, 0x03, 0x63, 0x06, 0xCB, 0x03, 0x1F,
            0xB3, 0xA5, 0xAE, 0x5A, 0x4D, 0x47, 0xA2, 0x38, 0x3E, 0x57, 0xBD, 0xFF, 0x3F, 0x0E, 0xC0, 0xCD,
            0x82, 0x17, 0x18, 0x72, 0x71, 0xE4, 0x4B, 0xDE, 0xEC, 0x3E, 0xAF, 0xFD, 0x8D, 0x36, 0xF0, 0xFF,
            0x49, 0xE5, 0x5F, 0x6A, 0xB2, 0xE7, 0xCA, 0xBA, 0xF1, 0x0F, 0x65, 0x85, 0x92, 0xA6, 0xA2, 0xF9,
            0x3B, 0xBC, 0xA3, 0xC1, 0x46, 0x42, 0xDF, 0x34, 0x30, 0x3A, 0x90, 0xA5, 0x0A, 0xD2, 0x41, 0xD1,
            0xD7, 0x0F, 0xC0, 0xC2, 0x08, 0x41, 0x71, 0x16, 0x88, 0x4E, 0x4F, 0x5C, 0x48, 0xE1, 0xDE, 0x08,
            0xB2, 0x1F, 0x10, 0x3B, 0xB2, 0x25, 0x35, 0xDB, 0xCF, 0xFD, 0x16, 0x4F, 0x5A, 0x9D, 0x2F, 0xF6,
            0xCE, 0xC9, 0xB9, 0x57, 0xFE, 0x24, 0xE5, 0x52, 0x42, 0x02, 0x42, 0x1A, 0xE5, 0x42, 0xC3, 0xC5,
            0x5D, 0x4D, 0x31, 0x52, 0x22, 0x7A, 0x44, 0x67, 0x99, 0xD9, 0x8A, 0x41, 0x4E, 0xAC, 0x11, 0x80,
            0x66, 0x23, 0x47, 0x6D, 0xC3, 0x65, 0x14, 0x6C, 0x9D, 0x6F, 0x96, 0xD4, 0xDC, 0xC4, 0x85, 0x16,
            0xCC, 0x34, 0x14, 0xE7, 0x38, 0xA3, 0x9D, 0x7D, 0xB2, 0x51, 0xE1, 0xE1, 0x3C, 0x90, 0xE7, 0x26,
            0x7F, 0x12, 0x7C, 0x3B, 0xCC, 0xD3, 0x6A, 0x3F, 0xDF, 0x00, 0xFC, 0x1C, 0x7B, 0x2B, 0x48, 0x44,
            0x84, 0xA4, 0x9D, 0xEB, 0xE9, 0x2F, 0x8E, 0x26, 0x6A, 0x38, 0x6F, 0xBB, 0xF1, 0x1C, 0x2B, 0x11,
            0x29, 0x9D, 0xF0, 0x57, 0x91, 0x10, 0x60, 0xDD, 0xD2, 0xDC, 0x64, 0xAE, 0xDD, 0x94, 0x53, 0x40,
            0xBF, 0xDB, 0x11, 0xEA, 0xB0, 0x15, 0xDA, 0xD2, 0x8B, 0xAA, 0x08, 0x70, 0x4A, 0x21, 0x1E, 0x54,
            0x2C, 0xF9, 0x9E, 0x52, 0xD0, 0x6B, 0x13, 0x02, 0x03, 0x01, 0x00, 0x01, 0xA3, 0x21, 0x30, 0x1F,
            0x30, 0x1D, 0x06, 0x03, 0x55, 0x1D, 0x0E, 0x04, 0x16, 0x04, 0x14, 0xC5, 0x15, 0x25, 0x97, 0xD0,
            0x4B, 0xF9, 0x05, 0x87, 0x37, 0x95, 0x2F, 0xF1, 0x68, 0x1A, 0x22, 0xB3, 0xA7, 0x7F, 0xFC, 0x30,
            0x0D, 0x06, 0x09, 0x2A, 0x86, 0x48, 0x86, 0xF7, 0x0D, 0x01, 0x01, 0x0B, 0x05, 0x00, 0x03, 0x82,
            0x01, 0x01, 0x00, 0x54, 0x9A, 0x99, 0x39, 0x86, 0x3B, 0xC0, 0x91, 0xBB, 0x53, 0xA5, 0xB2, 0x2D,
            0x9B, 0xD3, 0x7E, 0xD1, 0x51, 0xB4, 0x5E, 0xC0, 0x36, 0x11, 0x2D, 0x52, 0x5C, 0x25, 0xDD, 0x91,
            0xD2, 0x5B, 0x42, 0xFA, 0xF6, 0x0E, 0x50, 0x30, 0x68, 0x2E, 0x86, 0xD9, 0x46, 0x19, 0x99, 0x42,
            0xD1, 0x87, 0x6F, 0xFF, 0x7E, 0xD2, 0x54, 0x93, 0x0D, 0x82, 0x6E, 0x85, 0xC2, 0x7D, 0x76, 0xA5,
            0x58, 0x57, 0x6C, 0x06, 0xE5, 0x8E, 0x05, 0xF7, 0x8B, 0x58, 0xD9, 0xBC, 0x64, 0xB0, 0x5E, 0x31,
            0x88, 0xFD, 0xB1, 0xDB, 0x63, 0x4B, 0xA1, 0x22, 0xD2, 0xA9, 0x71, 0x97, 0x34, 0x3D, 0x31, 0xE1,
            0xF7, 0xEE, 0xF4, 0xD4, 0x66, 0x6F, 0xB0, 0xD0, 0x9C, 0xDB, 0x82, 0x2A, 0x57, 0x08, 0xF7, 0x9F,
            0x33, 0x5B, 0xF3, 0x29, 0xF9, 0x21, 0xDB, 0xC6, 0x82, 0x4D, 0x09, 0x90, 0xBA, 0xA9, 0xD7, 0x7D,
            0x2B, 0xDD, 0xE8, 0xCA, 0x86, 0x21, 0xDF, 0xA5, 0xE3, 0xED, 0xED, 0x3B, 0x07, 0x2E, 0xDD, 0xBA,
            0x27, 0xB3, 0xAF, 0x4E, 0xBE, 0x09, 0x96, 0x14, 0x4A, 0xED, 0x5B, 0x5A, 0xB3, 0x6E, 0xD0, 0x1D,
            0x9E, 0x45, 0xC0, 0x70, 0xDF, 0x75, 0x7B, 0x77, 0xA1, 0x57, 0xC5, 0x76, 0x60, 0x58, 0x5C, 0xF2,
            0x2F, 0xAD, 0xF1, 0xAA, 0x15, 0xA6, 0x7D, 0x5A, 0x94, 0x95, 0xB7, 0x10, 0x81, 0xC5, 0xE4, 0x49,
            0xF1, 0x87, 0xB3, 0xA5, 0x97, 0xF8, 0xBE, 0xC4, 0xE0, 0xFD, 0x12, 0xCB, 0x14, 0x86, 0x98, 0xA0,
            0x73, 0xFE, 0xEF, 0x9B, 0xE0, 0x14, 0x6A, 0x0E, 0x06, 0xDE, 0x77, 0xE1, 0x5A, 0xC4, 0xF4, 0x4E,
            0xEB, 0x8D, 0x37, 0xF1, 0x3A, 0xCF, 0x4D, 0x4D, 0x36, 0xFC, 0xC0, 0x92, 0xA4, 0xE9, 0xA6, 0x49,
            0x7B, 0xCC, 0x93, 0x66, 0x51, 0x2A, 0x24, 0xD3, 0x15, 0x75, 0x9B, 0x6C, 0x4F, 0x60, 0x44, 0xE9,
            0x15, 0x59, 0x68, 0x31, 0x82, 0x01, 0x8D, 0x30, 0x82, 0x01, 0x89, 0x02, 0x01, 0x01, 0x30, 0x62,
            0x30, 0x5A, 0x31, 0x0B, 0x30, 0x09, 0x06, 0x03, 0x55, 0x04, 0x06, 0x13, 0x02, 0x43, 0x4E, 0x31,
            0x0B, 0x30, 0x09, 0x06, 0x03, 0x55, 0x04, 0x08, 0x13, 0x02, 0x53, 0x48, 0x31, 0x0B, 0x30, 0x09,
            0x06, 0x03, 0x55, 0x04, 0x07, 0x13, 0x02, 0x53, 0x48, 0x31, 0x0F, 0x30, 0x0D, 0x06, 0x03, 0x55,
            0x04, 0x0A, 0x13, 0x06, 0x71, 0x71, 0x71, 0x71, 0x71, 0x71, 0x31, 0x0F, 0x30, 0x0D, 0x06, 0x03,
            0x55, 0x04, 0x0B, 0x13, 0x06, 0x71, 0x71, 0x71, 0x71, 0x71, 0x71, 0x31, 0x0F, 0x30, 0x0D, 0x06,
            0x03, 0x55, 0x04, 0x03, 0x13, 0x06, 0x71, 0x71, 0x71, 0x71, 0x71, 0x71, 0x02, 0x04, 0x77, 0xA4,
            0xD5, 0xE2, 0x30, 0x0D, 0x06, 0x09, 0x60, 0x86, 0x48, 0x01, 0x65, 0x03, 0x04, 0x02, 0x01, 0x05,
            0x00, 0x30, 0x0D, 0x06, 0x09, 0x2A, 0x86, 0x48, 0x86, 0xF7, 0x0D, 0x01, 0x01, 0x01, 0x05, 0x00,
            0x04, 0x82, 0x01, 0x00, 0x4B, 0x60, 0xBF, 0xF7, 0xF0, 0x7F, 0x3E, 0x06, 0xBE, 0xCA, 0x8F, 0xAD,
            0x7E, 0xB6, 0x1B, 0xBE, 0xE9, 0x0D, 0x27, 0x02, 0x35, 0xF1, 0x31, 0xA6, 0xF7, 0xA6, 0x08, 0x6A,
            0x48, 0xE0, 0x77, 0xA9, 0xFA, 0x30, 0xB1, 0xDB, 0xA8, 0xD9, 0x70, 0x2A, 0xA2, 0x21, 0x96, 0x02,
            0x6F, 0x45, 0x94, 0xBD, 0x75, 0xA9, 0x1C, 0xF6, 0x04, 0x61, 0xB5, 0xCD, 0x39, 0xA5, 0xAB, 0xDD,
            0x58, 0xCD, 0x50, 0x01, 0x22, 0x55, 0x04, 0x1D, 0x5A, 0x6A, 0x7A, 0xD7, 0x30, 0xA0, 0xBD, 0x36,
            0x9E, 0x9D, 0xC9, 0x66, 0xBA, 0x13, 0x75, 0xB4, 0x46, 0x28, 0xA5, 0xF0, 0x1F, 0x09, 0x36, 0x9C,
            0x05, 0x01, 0x7A, 0x66, 0x94, 0x55, 0xD4, 0x12, 0x0D, 0xC0, 0x3F, 0x77, 0x31, 0xFD, 0x06, 0x41,
            0x48, 0x43, 0xC3, 0x17, 0x7B, 0x7F, 0x7D, 0x45, 0x39, 0x73, 0x00, 0x1E, 0xC2, 0x99, 0x3A, 0x25,
            0x59, 0xB2, 0x78, 0xDD, 0x3E, 0x86, 0x06, 0xF0, 0x6E, 0x84, 0x0C, 0xB6, 0x33, 0x69, 0x73, 0x9B,
            0x52, 0xBA, 0x0B, 0x5D, 0x2F, 0x0A, 0x01, 0x30, 0x42, 0x93, 0x2E, 0x16, 0x6B, 0x9A, 0xDB, 0x82,
            0xF2, 0x30, 0x90, 0x1D, 0xC6, 0x1B, 0xB2, 0x2E, 0x19, 0x97, 0x40, 0xEC, 0xE1, 0x3F, 0xF3, 0x56,
            0xD2, 0xD3, 0xAC, 0xA6, 0xF9, 0x07, 0x5E, 0xCF, 0xBE, 0x91, 0x1B, 0xEE, 0x75, 0x30, 0xEE, 0x2B,
            0x7C, 0x45, 0x52, 0x1A, 0x76, 0x2F, 0x67, 0xE7, 0xCC, 0x8B, 0xCD, 0x93, 0xAD, 0xE4, 0xED, 0xEE,
            0x28, 0x64, 0x2D, 0x08, 0x4F, 0xB2, 0xA5, 0xE8, 0x38, 0x85, 0xDE, 0x65, 0x40, 0x06, 0x68, 0x6D,
            0xF6, 0x13, 0xC5, 0xE1, 0x2F, 0xE8, 0xA1, 0x74, 0x20, 0x44, 0x56, 0xD2, 0x9C, 0xF4, 0x0B, 0x78,
            0xA2, 0x10, 0x9F, 0xAC, 0x81, 0x82, 0x26, 0xC6, 0x63, 0xBC, 0x02, 0x7D, 0x2F, 0x6C, 0xD4, 0x2F,
            0xBD, 0x08, 0xD0, 0xA7
        ];

The output
    Sequence[
        OID(OID(1.2.840.1135<IP_ADDRESS>))
        Unknown(EndOfContent,[30, 82, 5, d, 2, 1, 1, 31, f, 30, d, 6, 9, 60, 86, 48, 1, 65, 3, 4, 2, 1, 5, 0, 30, b, 6, 9, 2a, 86, 48, 86, f7, d, 1, 7, 1, a0, 82, 3, 57, 30, 82, 3, 53, 30, 82, 2, 3b, a0, 3, 2, 1, 2, 2, 4, 77, a4, d5, e2, 30, d, 6, 9, 2a, 86, 48, 86, f7, d, 1, 1, b, 5, 0, 30, 5a, 31, b, 30, 9, 6, 3, 55, 4, 6, 13, 2, 43, 4e, 31, b, 30, 9, 6, 3, 55, 4, 8, 13, 2, 53, 48, 31, b, 30, 9, 6, 3, 55, 4, 7, 13, 2, 53, 48, 31, f, 30, d, 6, 3, 55, 4, a, 13, 6, 71, 71, 71, 71, 71, 71, 31, f, 30, d, 6, 3, 55, 4, b, 13, 6, 71, 71, 71, 71, 71, 71, 31, f, 30, d, 6, 3, 55, 4, 3, 13, 6, 71, 71, 71, 71, 71, 71, 30, 1e, 17, d, 32, 30, 31, 31, 30, 38, 30, 36, 33, 39, 34, 33, 5a, 17, d, 33, 30, 31, 31, 30, 36, 30, 36, 33, 39, 34, 33, 5a, 30, 5a, 31, b, 30, 9, 6, 3, 55, 4, 6, 13, 2, 43, 4e, 31, b, 30, 9, 6, 3, 55, 4, 8, 13, 2, 53, 48, 31, b, 30, 9, 6, 3, 55, 4, 7, 13, 2, 53, 48, 31, f, 30, d, 6, 3, 55, 4, a, 13, 6, 71, 71, 71, 71, 71, 71, 31, f, 30, d, 6, 3, 55, 4, b, 13, 6, 71, 71, 71, 71, 71, 71, 31, f, 30, d, 6, 3, 55, 4, 3, 13, 6, 71, 71, 71, 71, 71, 71, 30, 82, 1, 22, 30, d, 6, 9, 2a, 86, 48, 86, f7, d, 1, 1, 1, 5, 0, 3, 82, 1, f, 0, 30, 82, 1, a, 2, 82, 1, 1, 0, a7, e0, 84, 4e, 72, 99, e0, f, 4a, 91, 6, 8, 3, c7, 8c, 10, b5, 54, 20, 3, 63, 6, cb, 3, 1f, b3, a5, ae, 5a, 4d, 47, a2, 38, 3e, 57, bd, ff, 3f, e, c0, cd, 82, 17, 18, 72, 71, e4, 4b, de, ec, 3e, af, fd, 8d, 36, f0, ff, 49, e5, 5f, 6a, b2, e7, ca, ba, f1, f, 65, 85, 92, a6, a2, f9, 3b, bc, a3, c1, 46, 42, df, 34, 30, 3a, 90, a5, a, d2, 41, d1, d7, f, c0, c2, 8, 41, 71, 16, 88, 4e, 4f, 5c, 48, e1, de, 8, b2, 1f, 10, 3b, b2, 25, 35, db, cf, fd, 16, 4f, 5a, 9d, 2f, f6, ce, c9, b9, 57, fe, 24, e5, 52, 42, 2, 42, 1a, e5, 42, c3, c5, 5d, 4d, 31, 52, 22, 7a, 44, 67, 99, d9, 8a, 41, 4e, ac, 11, 80, 66, 23, 47, 6d, c3, 65, 14, 6c, 9d, 6f, 96, d4, dc, c4, 85, 16, cc, 34, 14, e7, 38, a3, 9d, 7d, b2, 51, e1, e1, 3c, 90, e7, 26, 7f, 12, 7c, 3b, cc, d3, 6a, 3f, df, 0, fc, 1c, 7b, 2b, 48, 44, 84, a4, 9d, eb, e9, 2f, 8e, 26, 6a, 38, 6f, bb, f1, 1c, 2b, 11, 29, 9d, f0, 57, 91, 10, 60, dd, d2, dc, 64, ae, dd, 94, 53, 40, bf, db, 11, ea, b0, 15, da, d2, 8b, aa, 8, 70, 4a, 21, 1e, 54, 2c, f9, 9e, 52, d0, 6b, 13, 2, 3, 1, 0, 1, a3, 21, 30, 1f, 30, 1d, 6, 3, 55, 1d, e, 4, 16, 4, 14, c5, 15, 25, 97, d0, 4b, f9, 5, 87, 37, 95, 2f, f1, 68, 1a, 22, b3, a7, 7f, fc, 30, d, 6, 9, 2a, 86, 48, 86, f7, d, 1, 1, b, 5, 0, 3, 82, 1, 1, 0, 54, 9a, 99, 39, 86, 3b, c0, 91, bb, 53, a5, b2, 2d, 9b, d3, 7e, d1, 51, b4, 5e, c0, 36, 11, 2d, 52, 5c, 25, dd, 91, d2, 5b, 42, fa, f6, e, 50, 30, 68, 2e, 86, d9, 46, 19, 99, 42, d1, 87, 6f, ff, 7e, d2, 54, 93, d, 82, 6e, 85, c2, 7d, 76, a5, 58, 57, 6c, 6, e5, 8e, 5, f7, 8b, 58, d9, bc, 64, b0, 5e, 31, 88, fd, b1, db, 63, 4b, a1, 22, d2, a9, 71, 97, 34, 3d, 31, e1, f7, ee, f4, d4, 66, 6f, b0, d0, 9c, db, 82, 2a, 57, 8, f7, 9f, 33, 5b, f3, 29, f9, 21, db, c6, 82, 4d, 9, 90, ba, a9, d7, 7d, 2b, dd, e8, ca, 86, 21, df, a5, e3, ed, ed, 3b, 7, 2e, dd, ba, 27, b3, af, 4e, be, 9, 96, 14, 4a, ed, 5b, 5a, b3, 6e, d0, 1d, 9e, 45, c0, 70, df, 75, 7b, 77, a1, 57, c5, 76, 60, 58, 5c, f2, 2f, ad, f1, aa, 15, a6, 7d, 5a, 94, 95, b7, 10, 81, c5, e4, 49, f1, 87, b3, a5, 97, f8, be, c4, e0, fd, 12, cb, 14, 86, 98, a0, 73, fe, ef, 9b, e0, 14, 6a, e, 6, de, 77, e1, 5a, c4, f4, 4e, eb, 8d, 37, f1, 3a, cf, 4d, 4d, 36, fc, c0, 92, a4, e9, a6, 49, 7b, cc, 93, 66, 51, 2a, 24, d3, 15, 75, 9b, 6c, 4f, 60, 44, e9, 15, 59, 68, 31, 82, 1, 8d, 30, 82, 1, 89, 2, 1, 1, 30, 62, 30, 5a, 31, b, 30, 9, 6, 3, 55, 4, 6, 13, 2, 43, 4e, 31, b, 30, 9, 6, 3, 55, 4, 8, 13, 2, 53, 48, 31, b, 30, 9, 6, 3, 55, 4, 7, 13, 2, 53, 48, 31, f, 30, d, 6, 3, 55, 4, a, 13, 6, 71, 71, 71, 71, 71, 71, 31, f, 30, d, 6, 3, 55, 4, b, 13, 6, 71, 71, 71, 71, 71, 71, 31, f, 30, d, 6, 3, 55, 4, 3, 13, 6, 71, 71, 71, 71, 71, 71, 2, 4, 77, a4, d5, e2, 30, d, 6, 9, 60, 86, 48, 1, 65, 3, 4, 2, 1, 5, 0, 30, d, 6, 9, 2a, 86, 48, 86, f7, d, 1, 1, 1, 5, 0, 4, 82, 1, 0, 4b, 60, bf, f7, f0, 7f, 3e, 6, be, ca, 8f, ad, 7e, b6, 1b, be, e9, d, 27, 2, 35, f1, 31, a6, f7, a6, 8, 6a, 48, e0, 77, a9, fa, 30, b1, db, a8, d9, 70, 2a, a2, 21, 96, 2, 6f, 45, 94, bd, 75, a9, 1c, f6, 4, 61, b5, cd, 39, a5, ab, dd, 58, cd, 50, 1, 22, 55, 4, 1d, 5a, 6a, 7a, d7, 30, a0, bd, 36, 9e, 9d, c9, 66, ba, 13, 75, b4, 46, 28, a5, f0, 1f, 9, 36, 9c, 5, 1, 7a, 66, 94, 55, d4, 12, d, c0, 3f, 77, 31, fd, 6, 41, 48, 43, c3, 17, 7b, 7f, 7d, 45, 39, 73, 0, 1e, c2, 99, 3a, 25, 59, b2, 78, dd, 3e, 86, 6, f0, 6e, 84, c, b6, 33, 69, 73, 9b, 52, ba, b, 5d, 2f, a, 1, 30, 42, 93, 2e, 16, 6b, 9a, db, 82, f2, 30, 90, 1d, c6, 1b, b2, 2e, 19, 97, 40, ec, e1, 3f, f3, 56, d2, d3, ac, a6, f9, 7, 5e, cf, be, 91, 1b, ee, 75, 30, ee, 2b, 7c, 45, 52, 1a, 76, 2f, 67, e7, cc, 8b, cd, 93, ad, e4, ed, ee, 28, 64, 2d, 8, 4f, b2, a5, e8, 38, 85, de, 65, 40, 6, 68, 6d, f6, 13, c5, e1, 2f, e8, a1, 74, 20, 44, 56, d2, 9c, f4, b, 78, a2, 10, 9f, ac, 81, 82, 26, c6, 63, bc, 2, 7d, 2f, 6c, d4, 2f, bd, 8, d0, a7])
    ]

As show above, I think 0xA0 is not a unknown tag and the tag number should not be parsed as EOC, this is a context-specific tag.

I'm not sure if this function should process if hdr.class is DerClass::ContextSpecific
fn der_read_element_content_recursive<'a>(
    i: &'a [u8],
    hdr: DerObjectHeader<'a>,
    max_depth: usize,
) -> DerResult<'a> {
    match hdr.class {
        DerClass::Universal | DerClass::Private => (),
        _ => {
            let (i, content) = ber_get_object_content(i, &hdr, max_depth)?;
            let content = DerObjectContent::Unknown(hdr.tag, content);
            let obj = DerObject::from_header_and_content(hdr, content);
            return Ok((i, obj));
        }
    }
    match der_read_element_content_as(i, hdr.tag, hdr.len, hdr.is_constructed(), max_depth) {
        Ok((rem, content)) => Ok((rem, DerObject::from_header_and_content(hdr, content))),
        Err(Err::Error(BerError::UnknownTag)) => {
            let (rem, content) = ber_get_object_content(i, &hdr, max_depth)?;
            let content = DerObjectContent::Unknown(hdr.tag, content);
            let obj = DerObject::from_header_and_content(hdr, content);
            Ok((rem, obj))
        }
        Err(e) => Err(e),
    }
}

Hmm, parse_der may be too restrictive here. If I remember correctly, recursive parsing is not done, because this function has not way to determine if parsing is explicit or implicit from the encoding only (this function has no context). Maybe I should let parse_der try to parse a valid header, and descend if possible, but I'm unsure this would be a good idea.
The usual method with der-parser is to specify the expected tag and parsing function (for ex using parse_der_tagged_explicit). See for example krb5_parser.rs.
What are you trying to do? Maybe I can help if you give more details

@chifflier FWIW I am encountering a similar issue. In my case, I am attempting to parse a CMS (RFC 5652) data structure, specifically the signed and unsigned attributes types, where the structure of the data is not assigned within the RFC that I'm parsing.
I've observed that a context-specific [0] tag nested deeply within that structure yields the same EOC data pattern that @cuserdzy describes; this is an undesirable outcome in my case.

@chifflier FWIW I am encountering a similar issue. In my case, I am attempting to parse a CMS (RFC 5652) data structure, specifically the signed and unsigned attributes types, where the structure of the data is not assigned within the RFC that I'm parsing.
I've observed that a context-specific [0] tag nested deeply within that structure yields the same EOC data pattern that @cuserdzy describes; this is an undesirable outcome in my case.

@cuserdzy don't know if you're still facing this issue (or someone else may come along who does have a similar issue), but you can reparse the data blobs. The following code snippet could probably use some cleaning up, but it will reparse into such structures recursively:
fn reparse_unknown_o<'a>(o: BerObject<'a>) -> BerResult<'a> {
    // Workaround for https://github.com/rusticata/der-parser/issues/38.

    let new_o = match o.content {
        BerObjectContent::Sequence(v) => {
            let (_, new_v) = reparse_unknown_v(v)?;
            BerObject::from_header_and_content(o.header, BerObjectContent::Sequence(new_v))
        }

        BerObjectContent::Set(v) => {
            let (_, new_v) = reparse_unknown_v(v)?;
            BerObject::from_header_and_content(o.header, BerObjectContent::Set(new_v))
        }

        BerObjectContent::Optional(Some(boxed_obj)) => {
            let (_, new_o) = reparse_unknown_o(*boxed_obj)?;
            BerObject::from_header_and_content(
                o.header,
                BerObjectContent::Optional(Some(Box::new(new_o))),
            )
        }

        BerObjectContent::Tagged(class, tag, boxed_obj) => {
            let (_, new_o) = reparse_unknown_o(*boxed_obj)?;
            BerObject::from_header_and_content(
                o.header,
                BerObjectContent::Tagged(class, tag, Box::new(new_o)),
            )
        }

        BerObjectContent::Unknown(_tag, data) => {
            if o.header.class == BerClass::ContextSpecific && o.header.structured == 1 {
                // Tagged entries may contain multiple objects. The BerObjectContent enum
                // doesn't have a way to represent this directly, so we wrap this in an
                // implicit sequence.

                let mut objs: Vec<BerObject<'a>> = Vec::new();
                let mut i = data;
                while !i.is_empty() {
                    let (i2, obj) = parse_der(i)?;
                    let (_, obj) = reparse_unknown_o(obj)?;
                    objs.push(obj);
                    i = i2;
                }

                BerObject::from_header_and_content(o.header, BerObjectContent::Sequence(objs))
            } else {
                o
            }
        }

        _ => o,
    };

    Ok((&[], new_o))
}

fn reparse_unknown_v<'a>(v: Vec<BerObject<'a>>) -> BerResult<Vec<BerObject<'a>>> {
    let new_v: Result<Vec<BerObject<'a>>, nom::Err<BerError>> = v
        .into_iter()
        .map(|o| match reparse_unknown_o(o) {
            Ok((_i, new_o)) => Ok(new_o),
            Err(e) => Err(e),
        })
        .collect();

    match new_v {
        Ok(v) => Ok((&[], v)),
        Err(e) => Err(e),
    }
}

@cuserdzy don't know if you're still facing this issue (or someone else may come along who does have a similar issue), but you can reparse the data blobs. The following code snippet could probably use some cleaning up, but it will reparse into such structures recursively:
fn reparse_unknown_o<'a>(o: BerObject<'a>) -> BerResult<'a> {
    // Workaround for https://github.com/rusticata/der-parser/issues/38.

    let new_o = match o.content {
        BerObjectContent::Sequence(v) => {
            let (_, new_v) = reparse_unknown_v(v)?;
            BerObject::from_header_and_content(o.header, BerObjectContent::Sequence(new_v))
        }

        BerObjectContent::Set(v) => {
            let (_, new_v) = reparse_unknown_v(v)?;
            BerObject::from_header_and_content(o.header, BerObjectContent::Set(new_v))
        }

        BerObjectContent::Optional(Some(boxed_obj)) => {
            let (_, new_o) = reparse_unknown_o(*boxed_obj)?;
            BerObject::from_header_and_content(
                o.header,
                BerObjectContent::Optional(Some(Box::new(new_o))),
            )
        }

        BerObjectContent::Tagged(class, tag, boxed_obj) => {
            let (_, new_o) = reparse_unknown_o(*boxed_obj)?;
            BerObject::from_header_and_content(
                o.header,
                BerObjectContent::Tagged(class, tag, Box::new(new_o)),
            )
        }

        BerObjectContent::Unknown(_tag, data) => {
            if o.header.class == BerClass::ContextSpecific && o.header.structured == 1 {
                // Tagged entries may contain multiple objects. The BerObjectContent enum
                // doesn't have a way to represent this directly, so we wrap this in an
                // implicit sequence.

                let mut objs: Vec<BerObject<'a>> = Vec::new();
                let mut i = data;
                while !i.is_empty() {
                    let (i2, obj) = parse_der(i)?;
                    let (_, obj) = reparse_unknown_o(obj)?;
                    objs.push(obj);
                    i = i2;
                }

                BerObject::from_header_and_content(o.header, BerObjectContent::Sequence(objs))
            } else {
                o
            }
        }

        _ => o,
    };

    Ok((&[], new_o))
}

fn reparse_unknown_v<'a>(v: Vec<BerObject<'a>>) -> BerResult<Vec<BerObject<'a>>> {
    let new_v: Result<Vec<BerObject<'a>>, nom::Err<BerError>> = v
        .into_iter()
        .map(|o| match reparse_unknown_o(o) {
            Ok((_i, new_o)) => Ok(new_o),
            Err(e) => Err(e),
        })
        .collect();

    match new_v {
        Ok(v) => Ok((&[], v)),
        Err(e) => Err(e),
    }
}

I would add to the problem that I also share the same Unknown content for CMS data. And it get worse, as in X.509 ber sequence, with extra information (extension).
PKCS#7/CMS data contains a nested list of certificate X.509, which also has a context-specific [0] data. I have to manually parse the content as a sequence.
Why don't we support 0xA0 tag? It is in the specs anyway.
https://stackoverflow.com/a/43056617/13173256
P.S: the binary for my data is the signature blob in Mach-O (Apple binary) for signed binary. So I would expect the binary data is correctly encoded.

Hi,
It's not that tag 0xA0 is not supported: the problem is using the function parse_der, because it cannot automatically guess the contents of the tagged value. Without the ASN.1 grammar, it's not possible to know if the value is EXPLICIT or IMPLICIT, and so it cannot be decoded.
When dealing with such cases, the best way is to guide the parser by giving the expected functions, as DER parser design doc section explains.
Note: I have a crate (not yet published, but the code is mostly ready) to decode most PKCS formats, including PKCS#7 and CMS (the crate does not perform cryptographic operations). I intended to publish it later, but if you are interested I can push it to github.

FWIW I am unsubscribing as I no longer have the need for CMS parsing.

I would love to see the crate published. But I don't insist pushing something to the public when you're not feeling it. Having a good parser for most certificate standards is very useful. Thank you.
Just a guessing, I think you use serialization to make the decodable/codable (cryptographic standard) structs. Because manually parsing is more error-prone than serializing. And while serialization is experimental, it would be too early to publish code using the feature.

The initial problem is now fixed (master branch, to be released as 7.0.0), so I'm closing this issue.
Please open other issues if needed.
Thanks


Sync upstream 2023 06 30

Conflicts:
	.github/workflows/buf-lint.yml
	.github/workflows/buf.yml
	.github/workflows/ci.yml

deleted in mimir-prometheus.

	model/labels/regexp.go

no change: mimir-prometheus already optimised literal matches.

	tsdb/head.go
	tsdb/head_append.go
	tsdb/head_test.go

merged changes.

	tsdb/querier.go

no change: mimir-prometheus already optimised grouped matches.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.4 out of 5 committers have signed the CLA.:white_check_mark: roidelapluie:white_check_mark: cuishuang:white_check_mark: pracucci:white_check_mark: bboreham:x: metalmatzeYou have signed the CLA already but the status is still pending? Let us recheck it.


Crash - segmentation fault

I got new requirements installed and the app crashes as soon as it recognizes me. It does present a window and shows camera with unknown. Running MacOSX Monterey 12.0 Beta (21A5506j)
zsh: segmentation fault  python3 main.py --name Rudy
Rodolfo@RIP gen2-face-recognition % python3 main.py --name Rudy
Creating pipeline...
Creating Color Camera...
Creating Face Detection Neural Network...
Creating Head pose estimation NN
Creating face recognition ImageManip/NN
[14442C10210D60D700] [11.883] [NeuralNetwork(10)] [warning] Network compiled for 4 shaves, maximum available 13, compiling for 6 shaves likely will yield in better performance
[14442C10210D60D700] [12.127] [NeuralNetwork(10)] [warning] The issued warnings are orientative, based on optimal settings for a single network, if multiple networks are running in parallel the optimal settings may vary
Saving face...
Saving face...
Saving face...
Saving face...
Saving face...
Saving face...
Saving face...
Saving face...
Saving face...
Saving face...
Saving face...
Saving face...
Saving face...
Saving face...
Saving face...
Saving face...
Saving face...
Saving face...
Saving face...
Saving face...
Saving face...
Saving face...
Saving face...
Saving face...
Saving face...
Saving face...
Saving face...
Saving face...
Saving face...
Saving face...
Traceback (most recent call last):
File "main.py", line 254, in
frameIn = frameQ.tryGet()
RuntimeError: Communication exception - possible device error/misconfiguration. Original message 'Couldn't read data from stream: 'frame' (X_LINK_ERROR)'
Rodolfo@RIP gen2-face-recognition % python3 main.py --name Rudy
Creating pipeline...
Creating Color Camera...
Creating Face Detection Neural Network...
Creating Head pose estimation NN
Creating face recognition ImageManip/NN
[14442C10210D60D700] [8.005] [NeuralNetwork(10)] [warning] Network compiled for 4 shaves, maximum available 13, compiling for 6 shaves likely will yield in better performance
[14442C10210D60D700] [8.248] [NeuralNetwork(10)] [warning] The issued warnings are orientative, based on optimal settings for a single network, if multiple networks are running in parallel the optimal settings may vary
Saving face...
Saving face...
Saving face...
Saving face...
zsh: segmentation fault  python3 main.py --name Rudy
Rodolfo@RIP gen2-face-recognition % python3 main.py --name Rudy
Creating pipeline...
Creating Color Camera...
Creating Face Detection Neural Network...
Creating Head pose estimation NN
Creating face recognition ImageManip/NN
[14442C10210D60D700] [20.261] [NeuralNetwork(10)] [warning] Network compiled for 4 shaves, maximum available 13, compiling for 6 shaves likely will yield in better performance
[14442C10210D60D700] [20.504] [NeuralNetwork(10)] [warning] The issued warnings are orientative, based on optimal settings for a single network, if multiple networks are running in parallel the optimal settings may vary
Stack trace (most recent call last) in thread<PHONE_NUMBER>29056:
#7    Object "libsystem_pthread.dylib", at 0x7ff80dd195a8, in _pthread_start + 125
#6    Object "depthai.cpython-38-darwin.so", at 0x11af6a617, in void fmt::v7::detail::value<fmt::v7::basic_format_context<fmt::v7::detail::buffer_appender, char> >::format_custom_arg<std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000l> >, fmt::v7::formatter<std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000l> >, char, void> >(void const*, fmt::v7::basic_format_parse_context<char, fmt::v7::detail::error_handler>&, fmt::v7::basic_format_context<fmt::v7::detail::buffer_appender, char>&) + 89687
#5    Object "libc++abi.dylib", at 0x7ff80dcdde84, in __cxa_decrement_exception_refcount + 35
#4    Object "libc++abi.dylib", at 0x7ff80dcdbd16, in std::runtime_error::~runtime_error() + 94
#3    Object "libc++abi.dylib", at 0x7ff80dcdec93, in __cxa_guard_acquire + 115
#2    Object "libsystem_platform.dylib", at 0x7ff80dd2f06c, in _sigtramp + 28
#1    Object "depthai.cpython-38-darwin.so", at 0x11b0076ed, in backward::SignalHandling::sig_handler(int, __siginfo*, void*) + 13
#0    Object "depthai.cpython-38-darwin.so", at 0x11b00775c, in backward::SignalHandling::sig_handler(int, __siginfo*, void*) + 124
zsh: segmentation fault  python3 main.py --name Rudy
Rodolfo@RIP gen2-face-recognition % python3 main.py --name Rudy
Creating pipeline...
Creating Color Camera...
Creating Face Detection Neural Network...
Creating Head pose estimation NN
Creating face recognition ImageManip/NN
[14442C10210D60D700] [8.429] [NeuralNetwork(10)] [warning] Network compiled for 4 shaves, maximum available 13, compiling for 6 shaves likely will yield in better performance
[14442C10210D60D700] [8.672] [NeuralNetwork(10)] [warning] The issued warnings are orientative, based on optimal settings for a single network, if multiple networks are running in parallel the optimal settings may vary
Saving face...
Saving face...
Saving face...
Saving face...
Saving face...
Traceback (most recent call last):
File "main.py", line 254, in
frameIn = frameQ.tryGet()
RuntimeError: Communication exception - possible device error/misconfiguration. Original message 'Couldn't read data from stream: 'frame' (X_LINK_ERROR)'
Rodolfo@RIP gen2-face-recognition % pwd
/Users/Rodolfo/Downloads/depthai-experiments-master/gen2-face-recognition
Rodolfo@RIP gen2-face-recognition %

Hi @RudyAramayo ,
Sorry about the trouble here.  We should have put this as a known issue.  We have been debugging a tricky underlying bug that seems to be the root cause of this.  It seems that we just today nailed it - I think - and if so, @themarpe I think has the details and also instructions on how to test the fix.
Sorry again about the trouble here.
-Brandon

I love you product, im so excited to use it on my droid. I preordered the lite version.. (2 pack). Even better is that you are absolutely on top of these with quick response, thank you. I can't wait to perform my camera calibration and continue testing further. C++ integration should be a blast!

Can you believe the day that droids can recognize our entities and name them.. so exciting!

I think this is related to same issues as gen2-triangulation which isn't yet addressed unfortunately @RudyAramayo.
A potential fix is in the works, but we'll see whether it addresses the issue at hand as well.
Will keep you updated:)

I have been through the librealsense pathway with some of the RealSense chips. I was not happy with the CPU load they used and want to encourage you guys because of the simplicity and offloaded processing that is being performed. So critical to start a project without pegging the CPU right off the bat.

Thanks @RudyAramayo !

@RudyAramayo  could you try installing the following library, it contains some fixes for Script node related memory allocation:
python3 -m pip install -U --prefer-binary --user --extra-index-url https://artifacts.luxonis.com/artifactory/luxonis-python-snapshot-local "depthai==<IP_ADDRESS>.dev+7498afce36f34dae5482c2bbf42db317e3e53dd5"

Then modifying the script by adding the following (@alex-luxonis  suggested ):
script.setProcessor(dai.ProcessorType.LEON_CSS)

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
depthai-sdk 1.1.2 requires depthai>2, but you have depthai <IP_ADDRESS>+baacebe72eec4f257ddfc4d6cd45cb1725ecd6e4 which is incompatible.

i think you guys have handled the issue, let me test and get back to you... the above error is from the initial bug report.

i have understood where to add the code, will test now... i am new to the whole pipeline and what is going on ... very excited to learn something new this morning about how this script works... have not actually had time to read the scripts thuroughly yet.

script = pipeline.create(dai.node.Script)
script.setProcessor(dai.ProcessorType.LEON_CSS)

ITS WORKING!!! i have 50% recognition that its me with weird angles and the crash no longer occurs. had to reinstall the python_requirements.py and this is the output after installation, I also modified the script and didn't check if that change was necessary for this particular crash:
Rodolfo@RIP depthai-main % python3 install_requirements.py
pip 21.3.1 from /usr/local/lib/python3.9/site-packages/pip (python 3.9)
WARNING: Skipping opencv-python as it is not installed.
WARNING: Skipping opencv-contrib-python as it is not installed.
Requirement already satisfied: pip in /usr/local/lib/python3.9/site-packages (21.3.1)
Found existing installation: depthai <IP_ADDRESS>.dev0+7498afce36f34dae5482c2bbf42db317e3e53dd5
Uninstalling depthai-<IP_ADDRESS>.dev0+7498afce36f34dae5482c2bbf42db317e3e53dd5:
Successfully uninstalled depthai-<IP_ADDRESS>.dev0+7498afce36f34dae5482c2bbf42db317e3e53dd5
Looking in indexes: https://pypi.org/simple, https://artifacts.luxonis.com/artifactory/luxonis-python-snapshot-local/
Obtaining file:///Users/Rodolfo/Downloads/depthai-main/depthai_sdk (from -r requirements.txt (line 3))
Preparing metadata (setup.py) ... done
Ignoring opencv-python: markers 'platform_machine == "armv6l" or platform_machine == "armv7l"' don't match your environment
Ignoring opencv-contrib-python: markers 'platform_machine == "armv6l" or platform_machine == "armv7l"' don't match your environment
Collecting requests==2.24.0
Using cached requests-2.24.0-py2.py3-none-any.whl (61 kB)
Collecting argcomplete==1.12.1
Using cached argcomplete-1.12.1-py2.py3-none-any.whl (38 kB)
Collecting depthai==<IP_ADDRESS>.dev+7a0749a61597c086c5fd6e579618ae33accec8df
Downloading https://artifacts.luxonis.com/artifactory/luxonis-python-snapshot-local/depthai/depthai-<IP_ADDRESS>.dev0%2B7a0749a61597c086c5fd6e579618ae33accec8df-cp39-cp39-macosx_10_9_x86_64.whl (10.2 MB)
|████████████████████████████████| 10.2 MB 494 kB/s
Collecting opencv-python==<IP_ADDRESS>
Downloading opencv_python-<IP_ADDRESS>-cp39-cp39-macosx_10_13_x86_64.whl (40.3 MB)
|████████████████████████████████| 40.3 MB 331 kB/s
Collecting opencv-contrib-python==<IP_ADDRESS>
Downloading opencv_contrib_python-<IP_ADDRESS>-cp39-cp39-macosx_10_13_x86_64.whl (51.3 MB)
|████████████████████████████████| 51.3 MB 910 kB/s
Collecting certifi>=2017.4.17
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
Using cached urllib3-1.25.11-py2.py3-none-any.whl (127 kB)
Collecting idna<3,>=2.5
Using cached idna-2.10-py2.py3-none-any.whl (58 kB)
Collecting chardet<4,>=3.0.2
Using cached chardet-3.0.4-py2.py3-none-any.whl (133 kB)
Collecting numpy>=1.19.3
Downloading numpy-1.21.4-cp39-cp39-macosx_10_9_x86_64.whl (17.0 MB)
|████████████████████████████████| 17.0 MB 13.0 MB/s
Collecting blobconverter>=1.2.6
Using cached blobconverter-1.2.6-py3-none-any.whl (10 kB)
Collecting pytube>=11.0.1
Using cached pytube-11.0.1-py3-none-any.whl (56 kB)
Collecting PyYAML
Downloading PyYAML-6.0-cp39-cp39-macosx_10_9_x86_64.whl (197 kB)
|████████████████████████████████| 197 kB 59.5 MB/s
Collecting boto3
Downloading boto3-1.20.5-py3-none-any.whl (131 kB)
|████████████████████████████████| 131 kB 49.7 MB/s
Collecting jmespath<1.0.0,>=0.7.1
Using cached jmespath-0.10.0-py2.py3-none-any.whl (24 kB)
Collecting botocore<1.24.0,>=1.23.5
Downloading botocore-1.23.5-py3-none-any.whl (8.1 MB)
|████████████████████████████████| 8.1 MB 63.1 MB/s
Collecting s3transfer<0.6.0,>=0.5.0
Using cached s3transfer-0.5.0-py3-none-any.whl (79 kB)
Collecting python-dateutil<3.0.0,>=2.1
Using cached python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
Collecting six>=1.5
Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
Installing collected packages: six, urllib3, python-dateutil, jmespath, botocore, s3transfer, idna, chardet, certifi, requests, PyYAML, numpy, boto3, pytube, opencv-python, opencv-contrib-python, depthai, blobconverter, depthai-sdk, argcomplete
WARNING: The script chardetect is installed in '/Users/Rodolfo/Library/Python/3.9/bin' which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
WARNING: The scripts f2py, f2py3 and f2py3.9 are installed in '/Users/Rodolfo/Library/Python/3.9/bin' which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
WARNING: The script pytube is installed in '/Users/Rodolfo/Library/Python/3.9/bin' which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
WARNING: The script blobconverter is installed in '/Users/Rodolfo/Library/Python/3.9/bin' which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Running setup.py develop for depthai-sdk
Successfully installed PyYAML-6.0 argcomplete-1.12.1 blobconverter-1.2.6 boto3-1.20.5 botocore-1.23.5 certifi-2021.10.8 chardet-3.0.4 depthai-<IP_ADDRESS>.dev0+7a0749a61597c086c5fd6e579618ae33accec8df depthai-sdk-1.1.2 idna-2.10 jmespath-0.10.0 numpy-1.21.4 opencv-contrib-python-<IP_ADDRESS> opencv-python-<IP_ADDRESS> python-dateutil-2.8.2 pytube-11.0.1 requests-2.24.0 s3transfer-0.5.0 six-1.16.0 urllib3-1.25.11
Ignoring open3d: markers 'platform_machine != "armv6l" and platform_machine != "armv7l" and python_version < "3.9"' don't match your environment
Collecting ffmpy3==0.2.4
Using cached ffmpy3-0.2.4-py3-none-any.whl (6.0 kB)
Installing collected packages: ffmpy3
Successfully installed ffmpy3-0.2.4

85% recognition now....

I can confirm that it fails with RuntimeError: Communication exception - possible device error/misconfiguration. Original message 'Couldn't read data from stream: 'frame' (X_LINK_ERROR)'
After downgrading the deptai library through installing the requirements it works python3 -m pip install -r requirements.txt


Error in counting listing tasks using filter

In list filter tasks by category the count in the column does not update.

Proposal: This count could be performed on the client side, using JS, providing real-time count.

This doesn't happen anymore with the last master.


Preview stitching process

Would be useful to make necessary corrections before final stitching like IDS does. See the video here https://youtu.be/dp9_6r8AIKc

Timecode? It's a 30 minute video.
Do you mean the adding the trims or something else?

Or rather the entire widget of viewing all the stitch order and the sew color order information?


When the sending is complete, keep the total number of sms sent

When All sms are sent with success, we can see the green bar with "terminée", can you add the total number of SMS sent, and next to it, the number of response so far with the % it represents.

And we should set the bar green once all reachable people have been SMSed, currently, it stays blue, which is wrong.

TODO: when there are unreachable people, add it on the progress bar

duplicate of #166


raft: remove unused Error field

LGTM

Current coverage is 55.03% (diff: 100%)

Merging #1449 into master will increase coverage by 0.11%

@@             master      #1449   diff @@
==========================================
  Files            82         82
  Lines         12904      12904
  Methods           0          0
  Messages          0          0
  Branches          0          0
==========================================
+ Hits           7087       7102    +15
+ Misses         4830       4815    -15
  Partials        987        987

Powered by Codecov. Last update 3c887e4...50e0628


Add memory limits / reservations (:warning: OPS)

Add memory limits / reservations to traefik and monitoring services.
Part of https://github.com/ITISFoundation/osparc-ops-environments/issues/330
:warning: OPS ACTION REQUIRED:

Redeploy traefik and monitoring stack to propagate the limits

just to confirm: you took the new limitations introduced in this PR from some real-world evidence like prometheus metrics of running containers?

just to confirm: you took the new limitations introduced in this PR from some real-world evidence like prometheus metrics of running containers?

The memory limits are based on production deployments prometheus metrics