Questions about linking between entities in the beacon v2 default model

Hi all
I have a few questions around how entities are linked in the beacon model. @mbaudis kindly shared the documentation website with handy model diagram here: http://docs.genomebeacons.org/models/#introduction. Based on this it looks like you could explicitly link individuals to a particular cohort or dataset, but i don't believe there are any fields in these entities where you could store identifiers that would make these links explicit. Is there something I am missing here? I was perhaps thinking of a use case where you could search for a cohort of interest, then from that cohort get more information about individuals/analyses/variants, but I don't follow how that would be possible without linking.
The other question I have is around the cardinality of the relationships between objects. The foreign key type fields, (e.g. runId, biosampleId) seem to generally be string type variables, meaning they can only ever link to one object per field. I could imagine situations where you might want to have multiple objects linked in this field, e.g. an analysis performed on multiple runs or containing many individuals. So if I understand correctly, it currently represents a one way one-to-many relationship, e.g. an individual can have multiple analyses performed on it, but one analysis can't have multiple individuals involved. Are these kinds of relationships the only ones allowed within the beacon v2 model?
I also noticed that in the diagram there isn't a link displayed between runs and analyses, while you can have a runId stored in an analysis object. ( I understand these are still draft but wanted to point out to potentially help improve )
Thanks in advance for helping us understand the model!
CC: @victorskl

For now: Please consider the diagram ... as non-authoritative.
Then: For the purpose of the Beacon v2 default model I - personally, YMMV - consider the modelling of runs ... as overdoing it. The main commonality may be in some basic analysis info informing about date, platform/technology; everything else is beyond the scope of Beacon for now and should be left to handovers & custom fields.

The model was born with the intention of being easy to implement, barebones, althought it has been growing since then (so that premise is a bit compromised nowadays).
Also, the model is not a "mirror" of how it would be implemented in the backend, and different implementation options would be possible with internal schemas totally different from the one in the logical model.
For the linking between the collections (i.e. cohorts and datasets),  we were considering the scenario where there is lot of members in the collection, e.g. a cohort with 50K individuals, do we want a cohort document that lists 50K individual ids? probably not.
However, we are still having a way to get such individuals on demand by using the REST approach itself:
/cohort/{id}/individuals would return all the individuals in a cohort. How the translation of this REST endpoint to a backend store is solved is up to the implementer.
This will return 50K individual records, which could be overkill also if no filter is applied.
The other way around would be easier (listing collections to which an individual belongs to), but has not been included in the model so far.
For the runs and analyses having only one linked id, this is due to the simplicity we were looking at the beginning.
Although Michael is not a fan of having more objects, I believe this reflects his expertise in its preferred domain, but the requirement comes from genetic diagnose facilities in hospitals that need to have details on both wet and dry lab procedures.
In such use case, duplication are not a big deal, as they don't process 1000 samples at once.
If everything that is "required" is to uplift the id from string to array, that is probably something we can do that does not harm the principle but fits better with other use cases.
@mbaudis @Tom-Shorter what do you think?

These relationships will differ between beacon instances, some instances will have individual->biosample->genomicVariations with no direct link between individuals and genomicVariations whereas another instance could have individual->genomicVariations and not include any data about biosamples

We already try to reflect this in the model graph (needs some refinements e.g. for genomicVariations) through solid lines -> minimum link; dotted lines -> convenience link using existing ids. Suggestions appreciated!
http://docs.genomebeacons.org/models/#introduction

Hi all, thanks for the very informative discussion.
It makes sense to me for now that some of the linking e.g. individuals-> cohorts could be done 'behind the scenes' in the API framework rather than explicitly in metadata. In our case, we will probably have an explicit linking in our native data store that could then be translated by the API. ( I am probably showing I have much more experience with metadata models than API backends 😄  )
I also appreciate Tom's points around the potential differences in the experimental graph from different beacons depending on which objects are implemented. Would this potentially also raise complications when doing cross-beacon queries via network. i.e. could you query across different beacons if they had different graph configurations?
Also understand the model is simple by design. We are interested in using it as a base to build on for such use cases as a common model for cross-querying to assemble virtual cohorts across multiple repositories (Gen3/Stardog database) and potentially use as a base model to build tools for automated submission to EGA. So super valuable for us to figure out its limits and know what we will need to add in order to make it work for our custom uses as well as having the benefit of our data meeting a GA4GH standard.

@mshadbolt See also my comment on this issue - scoped parameters vs. "model free"[^1] filters here: https://github.com/ga4gh-beacon/beacon-v2-Models/pull/112#issuecomment-1088994969
So while the "logical model" should be kept in mind (and has been very similar throughout many GA4GH iterations & projects - with slight deviations - it doesn't say at all how your data should be organised. For progenetix.org we actually have this pretty close on the database level (document db...), with deviations in:

we store each variant call (actually as VRS location objects + addle. parameters) and collapse to the "prototype variation with caseLevelData Beacon model on export
analyses and runs are just one & parsed out on export
cohorts are annotated inline
any query does the full stack & stores all hits (on biosamples, individuals, variants ...)

But these are differences in handling/implementation, w/o contradictions. Now the representation of cohorts & their aggregated features/feature spaces is still a bit loose, but here Beacon will just act as facilitator (AFAIK).
[^1]: Not exactly but...

@Tom-Shorter The diagram code is actually this:

classDiagram

    analyses <-- genomicVariations : 1..n
    runs <-- analyses : 1..n
    biosamples <-- runs : 1..n
    individuals <-- biosamples : 1..n

    runs <.. genomicVariations : 1..n
    biosamples <.. genomicVariations : 1..n
    individuals <.. genomicVariations : 1..n
    biosamples <.. analyses : 1..n
    individuals <.. analyses : 1..n
    individuals <.. runs : 1..n

    cohorts o-- individuals : 1..n
    datasets o-- genomicVariations : 1..n
    datasets o-- runs : 1..n
    datasets o-- analyses : 1..n
    datasets o-- biosamples : 1..n
    datasets o-- individuals : 1..n

    class genomicVariations{
        analysisId
        runId
        biosampleId
        individualId
        variation
        clinicalInterpretations
        caseLevelData
        ...
    }
    class analyses{
        id
        runId
        biosampleId
        individualId
        analysisDate
        pipelineName
        aligner
        ...
    }
    class biosamples{
        id
        individualId
        biosampleStatus
        sampleOriginType
        histologicalDiagnosis
        collectionDate
        ...
    }
    class individuals{
        id
        sex
        diseases
        phenotypicFeatures
        ethnicity
        pedigrees
        ...
    }
    class runs{
        id
        biosampleId
        individualId
        runDate
        librarySource
        libraryStrategy
        platform
        ...
        }
    class datasets{
        id
        name
        description
        dataUseCondition
        info
        updateDateTime
        ...
    }
    class cohorts{
        id
        name
        cohortType
        cohortSize
        cohortDataTypes
        cohortDesign
        ...
    }

:-)


support nextjs

Sorry if it isn't the appropriate place to ask this.
But is there any plan for this lib support nextjs? or is it already possible to use it there?

Hi @eerison
That's the right place for this question.
I have plan to make the solution more agnostic, ie not dependent on Docusaurus.
The work is in progress, and I expect to have something by early summer.

I am working on this too within my company. We have a prototype ready and working which will go out in some weeks (nothing yet finalized).
We reused @graphql-markdown/tools for parsing schema and then we generate routes and return props. We're dealing with caching now to speed up some of the elaboration.

I have plan to make the solution more agnostic, ie not dependent on Docusaurus.
The work is in progress, and I expect to have something by early summer

Work is still in progress. Most of the ground work has been done, ie clean up the code and migrating to TS, adding documentation in the code.
I am exploring the next step that is moving away from a messy architecture (the latest clean up improve the code, but not the architecture), to a cleaner one based on NestJS. I don't know yet if NestJS will be the final solution, but I will for sure move the code into a "mature" NodeJS framework offering a better code structure, and DI.
A top of it, I'll add API documentation, so one can quickly reuse the libraries for custom export format.
:point_right:  This will be the 2.0.
Once the framework migration is done with feature parity to current 1.x, the next steps will be moving away from the current homemade schema parser to a proper AST parser (exploration has already be done, it is just a matter of implementation).
:point_right:  This will be the 2.0 or 2.1.
And, finally a template engine (tbd) to give more control on the output, and possibly generate other format than markdown.
:point_right:  This will be the 2.1 or 2.2.

Super excited for this! This would work really well with Nextra's Docs Theme

Any news on this? I'm using Nextra as well and this would be a life-saver! 👀

@LuchoTurtle - Working on it on and off... way slower than I'd like. I don't have an eta for now, but I'll try to provide an eta during August.


Quick fix for slurm artifacts

A quick and somewhat hacky fix for #221 .
We probably need to improve how the SLURM logs are added as artifacts.
We may want to do it in the bash script instead? I opened an issue

Codecov Report
Attention: Patch coverage is 0% with 2 lines in your changes missing coverage. Please review.

Project coverage is 46.13%. Comparing base (99ed754) to head (dcdf97c).

Files
Patch %
Lines

crabs/detector/evaluate_model.py
0.00%
2 Missing :warning:

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #222      +/-   ##
==========================================
- Coverage   46.16%   46.13%   -0.04%
==========================================
  Files          24       24
  Lines        1460     1461       +1
==========================================
  Hits          674      674
- Misses        786      787       +1

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Doesn't work with 1.18.2

https://paste.srnyx.xyz/mucexukiru.properties

use https://github.com/Blobanium/Loading-Timer/actions/runs/2012774123

Just used the build you sent me, I'm getting a similar error: https://paste.srnyx.xyz/elofikoteg.properties

Just used the build you sent me, I'm getting a similar error: https://paste.srnyx.xyz/elofikoteg.properties

it seems that the issue might be that if the mixin is failing to expose the bootstrap method properly, may be either an issue between either yarn or spongepowered idk.
Gonna reopen this issue for the time being. If you can find a way for this to work, feel free to make a PR.

this mod is able to count the launch time so maybe you can get some inspiration or help by the dev ?
https://modrinth.com/mod/dashloader

The refmap is not being automatically added, which means you have to manually edit the jar for one.


Add perf pipeline jobs to ci2

@adiaaida,
Thanks for having already signed the Contribution License Agreement. Your agreement was validated by .NET Foundation. We will now review your pull request.
Thanks,
.NET Foundation Pull Request Bot


Remove support for Rouge 1.x

This removes support for Rouge 1.x, along with the appraisals setup that comes with it.

Rouge is at version 3.x right now, which means either expanding the setup to include 3.x along with 2.x and 1.x, or dropping support for 1.x, which is unmaintained.

The appraisals setup is complicated and it’s not clear that bundle exec rake does not run all tests (it skips Rouge 1.x ones).

@barraq Can you take a look at this and let me know your thoughts/questions/concerns/objections?

Will merge this as I see no objection to this.
If the Appraisals setup is needed at a later point in time, it can always be reinstated easily.


Add Inkline to Libraries

Hello,
I'd like to add Inkline (https://inkline.io) to the list of Libraries. 🙏
Thanks a lot for maintaining this awesome project!

Hi there!
Nice library, however, not enough traction to justify adding a new one for now. There are already 15 libraries listed and every new library does not help end user to make a choice.
Feel free to reopen when you have more traction and/or special features to justify adding it.
Thanks.


Build failure when clang isn't default compiler

The commands in the readme Linux section says:

Your Linux distribution might default to gcc instead of clang, if that's the case invoke cmake with the following command:

But the following command matches the one above.
I tried the obvious to set CC to /usr/bin/clang and CXX to /usr/bin/clang++ prior to running cmake. cmake completes, but the build fails with some fairly basic #include file not found errors (things like  and ).
I'm building on Fedora 28 x86_64.

Whoops, looks like an issue in the documentation. Which version of clang are you using to build, and can you give the specific errors clang spits out?

I'm using clang 6.0.1.
All basic file not found errors:
../../libs/filaflat/include/filaflat/ShaderBuilder.h:20:10: fatal error: 'cstddef' file not found
../../libs/filaflat/include/filaflat/FilaflatDefs.h:21:10: fatal error: 'string' file not found
../../libs/math/include/math/TVecHelpers.h:25:10: fatal error: 'cmath' file not found
../../libs/filabridge/include/filament/SamplerInterfaceBlock.h:21:10: fatal error: 'map' file not found
I didn't look close enough to figure out how it is killing off the standard include paths.
/usr/bin/clang++  -DFILAMENT_DRIVER_SUPPORTS_VULKAN -I../../libs/filaflat/include -I../../libs/filabridge/include -I../../libs/utils/include -I../../third_party/robin-map/tnt/.. -I../../libs/math/include -std=c++14 -fstrict-aliasing -Wno-unknown-pragmas -Wno-unused-function -stdlib=libc++ -fcolor-diagnostics -O2 -DNDEBUG -fomit-frame-pointer -ffunction-sections -fdata-sections   -fvisibility=hidden -fPIC -MD -MT libs/filaflat/CMakeFiles/filaflat.dir/src/ShaderBuilder.cpp.o -MF libs/filaflat/CMakeFiles/filaflat.dir/src/ShaderBuilder.cpp.o.d -o libs/filaflat/CMakeFiles/filaflat.dir/src/ShaderBuilder.cpp.o -c ../../libs/filaflat/src/ShaderBuilder.cpp In file included from ../../libs/filaflat/src/ShaderBuilder.cpp:17: ../../libs/filaflat/include/filaflat/ShaderBuilder.h:20:10: fatal error: 'cstddef' file not found #include <cstddef>          ^~~~~~~~~ 1 error generated. [2/847] Building CXX object libs/filaflat/CMakeFiles/filaflat.dir/src/ChunkContainer.cpp.o FAILED: libs/filaflat/CMakeFiles/filaflat.dir/src/ChunkContainer.cpp.o  /usr/bin/clang++  -DFILAMENT_DRIVER_SUPPORTS_VULKAN -I../../libs/filaflat/include -I../../libs/filabridge/include -I../../libs/utils/include -I../../third_party/robin-map/tnt/.. -I../../libs/math/include -std=c++14 -fstrict-aliasing -Wno-unknown-pragmas -Wno-unused-function -stdlib=libc++ -fcolor-diagnostics -O2 -DNDEBUG -fomit-frame-pointer -ffunction-sections -fdata-sections   -fvisibility=hidden -fPIC -MD -MT libs/filaflat/CMakeFiles/filaflat.dir/src/ChunkContainer.cpp.o -MF libs/filaflat/CMakeFiles/filaflat.dir/src/ChunkContainer.cpp.o.d -o libs/filaflat/CMakeFiles/filaflat.dir/src/ChunkContainer.cpp.o -c ../../libs/filaflat/src/ChunkContainer.cpp In file included from ../../libs/filaflat/src/ChunkContainer.cpp:17: In file included from ../../libs/filaflat/include/filaflat/ChunkContainer.h:21: ../../libs/filaflat/include/filaflat/FilaflatDefs.h:21:10: fatal error: 'string' file not found #include <string>          ^~~~~~~~ 1 error generated. [3/847] Building CXX object libs/filaflat/CMakeFiles/filaflat.dir/src/Unflattener.cpp.o FAILED: libs/filaflat/CMakeFiles/filaflat.dir/src/Unflattener.cpp.o  /usr/bin/clang++  -DFILAMENT_DRIVER_SUPPORTS_VULKAN -I../../libs/filaflat/include -I../../libs/filabridge/include -I../../libs/utils/include -I../../third_party/robin-map/tnt/.. -I../../libs/math/include -std=c++14 -fstrict-aliasing -Wno-unknown-pragmas -Wno-unused-function -stdlib=libc++ -fcolor-diagnostics -O2 -DNDEBUG -fomit-frame-pointer -ffunction-sections -fdata-sections   -fvisibility=hidden -fPIC -MD -MT libs/filaflat/CMakeFiles/filaflat.dir/src/Unflattener.cpp.o -MF libs/filaflat/CMakeFiles/filaflat.dir/src/Unflattener.cpp.o.d -o libs/filaflat/CMakeFiles/filaflat.dir/src/Unflattener.cpp.o -c ../../libs/filaflat/src/Unflattener.cpp In file included from ../../libs/filaflat/src/Unflattener.cpp:17: In file included from ../../libs/filaflat/include/filaflat/Unflattener.h:20: In file included from ../../libs/filaflat/include/filaflat/ChunkContainer.h:21: ../../libs/filaflat/include/filaflat/FilaflatDefs.h:21:10: fatal error: 'string' file not found #include <string>          ^~~~~~~~ 1 error generated. [4/847] Building CXX object libs/filaflat/CMakeFiles/filaflat.dir/src/TextDictionaryReader.cpp.o FAILED: libs/filaflat/CMakeFiles/filaflat.dir/src/TextDictionaryReader.cpp.o  /usr/bin/clang++  -DFILAMENT_DRIVER_SUPPORTS_VULKAN -I../../libs/filaflat/include -I../../libs/filabridge/include -I../../libs/utils/include -I../../third_party/robin-map/tnt/.. -I../../libs/math/include -std=c++14 -fstrict-aliasing -Wno-unknown-pragmas -Wno-unused-function -stdlib=libc++ -fcolor-diagnostics -O2 -DNDEBUG -fomit-frame-pointer -ffunction-sections -fdata-sections   -fvisibility=hidden -fPIC -MD -MT libs/filaflat/CMakeFiles/filaflat.dir/src/TextDictionaryReader.cpp.o -MF libs/filaflat/CMakeFiles/filaflat.dir/src/TextDictionaryReader.cpp.o.d -o libs/filaflat/CMakeFiles/filaflat.dir/src/TextDictionaryReader.cpp.o -c ../../libs/filaflat/src/TextDictionaryReader.cpp In file included from ../../libs/filaflat/src/TextDictionaryReader.cpp:17: In file included from ../../libs/filaflat/src/TextDictionaryReader.h:20: In file included from ../../libs/filaflat/include/filaflat/ChunkContainer.h:21: ../../libs/filaflat/include/filaflat/FilaflatDefs.h:21:10: fatal error: 'string' file not found #include <string>          ^~~~~~~~ 1 error generated. [5/847] Building CXX object libs/filaflat/CMakeFiles/filaflat.dir/src/SpirvDictionaryReader.cpp.o FAILED: libs/filaflat/CMakeFiles/filaflat.dir/src/SpirvDictionaryReader.cpp.o  /usr/bin/clang++  -DFILAMENT_DRIVER_SUPPORTS_VULKAN -I../../libs/filaflat/include -I../../libs/filabridge/include -I../../libs/utils/include -I../../third_party/robin-map/tnt/.. -I../../libs/math/include -std=c++14 -fstrict-aliasing -Wno-unknown-pragmas -Wno-unused-function -stdlib=libc++ -fcolor-diagnostics -O2 -DNDEBUG -fomit-frame-pointer -ffunction-sections -fdata-sections   -fvisibility=hidden -fPIC -MD -MT libs/filaflat/CMakeFiles/filaflat.dir/src/SpirvDictionaryReader.cpp.o -MF libs/filaflat/CMakeFiles/filaflat.dir/src/SpirvDictionaryReader.cpp.o.d -o libs/filaflat/CMakeFiles/filaflat.dir/src/SpirvDictionaryReader.cpp.o -c ../../libs/filaflat/src/SpirvDictionaryReader.cpp In file included from ../../libs/filaflat/src/SpirvDictionaryReader.cpp:17: In file included from ../../libs/filaflat/src/SpirvDictionaryReader.h:20: In file included from ../../libs/filaflat/include/filaflat/ChunkContainer.h:21: ../../libs/filaflat/include/filaflat/FilaflatDefs.h:21:10: fatal error: 'string' file not found #include <string>          ^~~~~~~~ 1 error generated. [6/847] Building CXX object libs/filabridge/CMakeFiles/filabridge.dir/src/SibGenerator.cpp.o FAILED: libs/filabridge/CMakeFiles/filabridge.dir/src/SibGenerator.cpp.o  /usr/bin/clang++  -DFILAMENT_DRIVER_SUPPORTS_VULKAN -I../../libs/filabridge/include -I../../libs/utils/include -I../../third_party/robin-map/tnt/.. -I../../libs/math/include -std=c++14 -fstrict-aliasing -Wno-unknown-pragmas -Wno-unused-function -stdlib=libc++ -fcolor-diagnostics -fPIC -O2 -DNDEBUG -fomit-frame-pointer -ffunction-sections -fdata-sections -MD -MT libs/filabridge/CMakeFiles/filabridge.dir/src/SibGenerator.cpp.o -MF libs/filabridge/CMakeFiles/filabridge.dir/src/SibGenerator.cpp.o.d -o libs/filabridge/CMakeFiles/filabridge.dir/src/SibGenerator.cpp.o -c ../../libs/filabridge/src/SibGenerator.cpp In file included from ../../libs/filabridge/src/SibGenerator.cpp:17: In file included from ../../libs/filabridge/include/private/filament/SibGenerator.h:20: ../../libs/filabridge/include/filament/SamplerInterfaceBlock.h:21:10: fatal error: 'map' file not found #include <map>          ^~~~~ 1 error generated. [7/847] Building CXX object libs/filabridge/CMakeFiles/filabridge.dir/src/UibGenerator.cpp.o FAILED: libs/filabridge/CMakeFiles/filabridge.dir/src/UibGenerator.cpp.o  /usr/bin/clang++  -DFILAMENT_DRIVER_SUPPORTS_VULKAN -I../../libs/filabridge/include -I../../libs/utils/include -I../../third_party/robin-map/tnt/.. -I../../libs/math/include -std=c++14 -fstrict-aliasing -Wno-unknown-pragmas -Wno-unused-function -stdlib=libc++ -fcolor-diagnostics -fPIC -O2 -DNDEBUG -fomit-frame-pointer -ffunction-sections -fdata-sections -MD -MT libs/filabridge/CMakeFiles/filabridge.dir/src/UibGenerator.cpp.o -MF libs/filabridge/CMakeFiles/filabridge.dir/src/UibGenerator.cpp.o.d -o libs/filabridge/CMakeFiles/filabridge.dir/src/UibGenerator.cpp.o -c ../../libs/filabridge/src/UibGenerator.cpp In file included from ../../libs/filabridge/src/UibGenerator.cpp:17: In file included from ../../libs/filabridge/include/private/filament/UibGenerator.h:20: In file included from ../../libs/filabridge/include/filament/UniformInterfaceBlock.h:20: In file included from ../../libs/filabridge/include/filament/driver/DriverEnums.h:22: In file included from ../../libs/math/include/math/vec4.h:20: In file included from ../../libs/math/include/math/vec3.h:20: In file included from ../../libs/math/include/math/vec2.h:20: ../../libs/math/include/math/TVecHelpers.h:25:10: fatal error: 'cmath' file not found #include <cmath>          ^~~~~~~ 1 error generated. [8/847] Building CXX object libs/filaflat/CMakeFiles/filaflat.dir/src/MaterialChunk.cpp.o FAILED: libs/filaflat/CMakeFiles/filaflat.dir/src/MaterialChunk.cpp.o  /usr/bin/clang++  -DFILAMENT_DRIVER_SUPPORTS_VULKAN -I../../libs/filaflat/include -I../../libs/filabridge/include -I../../libs/utils/include -I../../third_party/robin-map/tnt/.. -I../../libs/math/include -std=c++14 -fstrict-aliasing -Wno-unknown-pragmas -Wno-unused-function -stdlib=libc++ -fcolor-diagnostics -O2 -DNDEBUG -fomit-frame-pointer -ffunction-sections -fdata-sections   -fvisibility=hidden -fPIC -MD -MT libs/filaflat/CMakeFiles/filaflat.dir/src/MaterialChunk.cpp.o -MF libs/filaflat/CMakeFiles/filaflat.dir/src/MaterialChunk.cpp.o.d -o libs/filaflat/CMakeFiles/filaflat.dir/src/MaterialChunk.cpp.o -c ../../libs/filaflat/src/MaterialChunk.cpp In file included from ../../libs/filaflat/src/MaterialChunk.cpp:17: In file included from ../../libs/filaflat/src/MaterialChunk.h:20: ../../libs/filaflat/src/BlobDictionary.h:20:10: fatal error: 'vector' file not found #include <vector>          ^~~~~~~~ 1 error generated. [9/847] Building CXX object libs/filabridge/CMakeFiles/filabridge.dir/src/UniformInterfaceBlock.cpp.o FAILED: libs/filabridge/CMakeFiles/filabridge.dir/src/UniformInterfaceBlock.cpp.o  /usr/bin/clang++  -DFILAMENT_DRIVER_SUPPORTS_VULKAN -I../../libs/filabridge/include -I../../libs/utils/include -I../../third_party/robin-map/tnt/.. -I../../libs/math/include -std=c++14 -fstrict-aliasing -Wno-unknown-pragmas -Wno-unused-function -stdlib=libc++ -fcolor-diagnostics -fPIC -O2 -DNDEBUG -fomit-frame-pointer -ffunction-sections -fdata-sections -MD -MT libs/filabridge/CMakeFiles/filabridge.dir/src/UniformInterfaceBlock.cpp.o -MF libs/filabridge/CMakeFiles/filabridge.dir/src/UniformInterfaceBlock.cpp.o.d -o libs/filabridge/CMakeFiles/filabridge.dir/src/UniformInterfaceBlock.cpp.o -c ../../libs/filabridge/src/UniformInterfaceBlock.cpp In file included from ../../libs/filabridge/src/UniformInterfaceBlock.cpp:17: In file included from ../../libs/filabridge/include/filament/UniformInterfaceBlock.h:20: In file included from ../../libs/filabridge/include/filament/driver/DriverEnums.h:22: In file included from ../../libs/math/include/math/vec4.h:20: In file included from ../../libs/math/include/math/vec3.h:20: In file included from ../../libs/math/include/math/vec2.h:20: ../../libs/math/include/math/TVecHelpers.h:25:10: fatal error: 'cmath' file not found #include <cmath>          ^~~~~~~ 1 error generated. [10/847] Building CXX object libs/filabridge/CMakeFiles/filabridge.dir/src/SamplerInterfaceBlock.cpp.o FAILED: libs/filabridge/CMakeFiles/filabridge.dir/src/SamplerInterfaceBlock.cpp.o  /usr/bin/clang++  -DFILAMENT_DRIVER_SUPPORTS_VULKAN -I../../libs/filabridge/include -I../../libs/utils/include -I../../third_party/robin-map/tnt/.. -I../../libs/math/include -std=c++14 -fstrict-aliasing -Wno-unknown-pragmas -Wno-unused-function -stdlib=libc++ -fcolor-diagnostics -fPIC -O2 -DNDEBUG -fomit-frame-pointer -ffunction-sections -fdata-sections -MD -MT libs/filabridge/CMakeFiles/filabridge.dir/src/SamplerInterfaceBlock.cpp.o -MF libs/filabridge/CMakeFiles/filabridge.dir/src/SamplerInterfaceBlock.cpp.o.d -o libs/filabridge/CMakeFiles/filabridge.dir/src/SamplerInterfaceBlock.cpp.o -c ../../libs/filabridge/src/SamplerInterfaceBlock.cpp In file included from ../../libs/filabridge/src/SamplerInterfaceBlock.cpp:17: ../../libs/filabridge/include/filament/SamplerInterfaceBlock.h:21:10: fatal error: 'map' file not found #include <map>          ^~~~~ 1 error generated. [11/847] Building CXX object libs/filabridge/CMakeFiles/filabridge.dir/src/SamplerBindingMap.cpp.o FAILED: libs/filabridge/CMakeFiles/filabridge.dir/src/SamplerBindingMap.cpp.o  /usr/bin/clang++  -DFILAMENT_DRIVER_SUPPORTS_VULKAN -I../../libs/filabridge/include -I../../libs/utils/include -I../../third_party/robin-map/tnt/.. -I../../libs/math/include -std=c++14 -fstrict-aliasing -Wno-unknown-pragmas -Wno-unused-function -stdlib=libc++ -fcolor-diagnostics -fPIC -O2 -DNDEBUG -fomit-frame-pointer -ffunction-sections -fdata-sections -MD -MT libs/filabridge/CMakeFiles/filabridge.dir/src/SamplerBindingMap.cpp.o -MF libs/filabridge/CMakeFiles/filabridge.dir/src/SamplerBindingMap.cpp.o.d -o libs/filabridge/CMakeFiles/filabridge.dir/src/SamplerBindingMap.cpp.o -c ../../libs/filabridge/src/SamplerBindingMap.cpp In file included from ../../libs/filabridge/src/SamplerBindingMap.cpp:17: In file included from ../../libs/filabridge/include/filament/SamplerBindingMap.h:20: In file included from ../../libs/filabridge/include/filament/EngineEnums.h:20: In file included from ../../libs/utils/include/utils/bitset.h:20: ../../libs/utils/include/utils/algorithm.h:22:10: fatal error: 'functional' file not found #include <functional>          ^~~~~~~~~~~~ 1 error generated. [12/847] Building CXX object libs/filaflat/CMakeFiles/filaflat.dir/src/MaterialParser.cpp.o FAILED: libs/filaflat/CMakeFiles/filaflat.dir/src/MaterialParser.cpp.o  /usr/bin/clang++  -DFILAMENT_DRIVER_SUPPORTS_VULKAN -I../../libs/filaflat/include -I../../libs/filabridge/include -I../../libs/utils/include -I../../third_party/robin-map/tnt/.. -I../../libs/math/include -std=c++14 -fstrict-aliasing -Wno-unknown-pragmas -Wno-unused-function -stdlib=libc++ -fcolor-diagnostics -O2 -DNDEBUG -fomit-frame-pointer -ffunction-sections -fdata-sections   -fvisibility=hidden -fPIC -MD -MT libs/filaflat/CMakeFiles/filaflat.dir/src/MaterialParser.cpp.o -MF libs/filaflat/CMakeFiles/filaflat.dir/src/MaterialParser.cpp.o.d -o libs/filaflat/CMakeFiles/filaflat.dir/src/MaterialParser.cpp.o -c ../../libs/filaflat/src/MaterialParser.cpp In file included from ../../libs/filaflat/src/MaterialParser.cpp:17: In file included from ../../libs/filaflat/include/filaflat/MaterialParser.h:20: In file included from ../../libs/filabridge/include/filament/EngineEnums.h:20: In file included from ../../libs/utils/include/utils/bitset.h:20: ../../libs/utils/include/utils/algorithm.h:22:10: fatal error: 'functional' file not found #include <functional>          ^~~~~~~~~~~~ 1 error generated. [13/847] Building CXX object libs/filaflat/CMakeFiles/filaflat.dir/src/ChunkInterfaceBlock.cpp.o FAILED: libs/filaflat/CMakeFiles/filaflat.dir/src/ChunkInterfaceBlock.cpp.o  /usr/bin/clang++  -DFILAMENT_DRIVER_SUPPORTS_VULKAN -I../../libs/filaflat/include -I../../libs/filabridge/include -I../../libs/utils/include -I../../third_party/robin-map/tnt/.. -I../../libs/math/include -std=c++14 -fstrict-aliasing -Wno-unknown-pragmas -Wno-unused-function -stdlib=libc++ -fcolor-diagnostics -O2 -DNDEBUG -fomit-frame-pointer -ffunction-sections -fdata-sections   -fvisibility=hidden -fPIC -MD -MT libs/filaflat/CMakeFiles/filaflat.dir/src/ChunkInterfaceBlock.cpp.o -MF libs/filaflat/CMakeFiles/filaflat.dir/src/ChunkInterfaceBlock.cpp.o.d -o libs/filaflat/CMakeFiles/filaflat.dir/src/ChunkInterfaceBlock.cpp.o -c ../../libs/filaflat/src/ChunkInterfaceBlock.cpp In file included from ../../libs/filaflat/src/ChunkInterfaceBlock.cpp:17: In file included from ../../libs/filaflat/src/ChunkInterfaceBlock.h:20: ../../libs/filabridge/include/filament/SamplerInterfaceBlock.h:21:10: fatal error: 'map' file not found #include <map>          ^~~~~ 1 error generated. [14/847] Building CXX object libs/math/CMakeFiles/test_math.dir/tests/test_fast.cpp.o FAILED: libs/math/CMakeFiles/test_math.dir/tests/test_fast.cpp.o  /usr/bin/clang++  -DFILAMENT_DRIVER_SUPPORTS_VULKAN -I../../libs/math/include -I../../third_party/libgtest/tnt/../include -std=c++14 -fstrict-aliasing -Wno-unknown-pragmas -Wno-unused-function -stdlib=libc++ -fcolor-diagnostics -O2 -DNDEBUG -fomit-frame-pointer -ffunction-sections -fdata-sections   -pthread -MD -MT libs/math/CMakeFiles/test_math.dir/tests/test_fast.cpp.o -MF libs/math/CMakeFiles/test_math.dir/tests/test_fast.cpp.o.d -o libs/math/CMakeFiles/test_math.dir/tests/test_fast.cpp.o -c ../../libs/math/tests/test_fast.cpp In file included from ../../libs/math/tests/test_fast.cpp:19: ../../third_party/libgtest/tnt/../include/gtest/gtest.h:54:10: fatal error: 'limits' file not found #include <limits>          ^~~~~~~~ 1 error generated. [15/847] Building CXX object libs/math/CMakeFiles/test_math.dir/tests/test_vec.cpp.o FAILED: libs/math/CMakeFiles/test_math.dir/tests/test_vec.cpp.o  /usr/bin/clang++  -DFILAMENT_DRIVER_SUPPORTS_VULKAN -I../../libs/math/include -I../../third_party/libgtest/tnt/../include -std=c++14 -fstrict-aliasing -Wno-unknown-pragmas -Wno-unused-function -stdlib=libc++ -fcolor-diagnostics -O2 -DNDEBUG -fomit-frame-pointer -ffunction-sections -fdata-sections   -pthread -MD -MT libs/math/CMakeFiles/test_math.dir/tests/test_vec.cpp.o -MF libs/math/CMakeFiles/test_math.dir/tests/test_vec.cpp.o.d -o libs/math/CMakeFiles/test_math.dir/tests/test_vec.cpp.o -c ../../libs/math/tests/test_vec.cpp In file included from ../../libs/math/tests/test_vec.cpp:19: ../../third_party/libgtest/tnt/../include/gtest/gtest.h:54:10: fatal error: 'limits' file not found #include <limits>          ^~~~~~~~ 1 error generated. [16/847] Building CXX object third_party/libgtest/tnt/CMakeFiles/gtest.dir/__/src/gtest-all.cc.o FAILED: third_party/libgtest/tnt/CMakeFiles/gtest.dir/__/src/gtest-all.cc.o  /usr/bin/clang++  -DFILAMENT_DRIVER_SUPPORTS_VULKAN -I../../third_party/libgtest/tnt/../include -I../../third_party/libgtest/tnt/.. -std=c++14 -fstrict-aliasing -Wno-unknown-pragmas -Wno-unused-function -stdlib=libc++ -fcolor-diagnostics -O2 -DNDEBUG -fomit-frame-pointer -ffunction-sections -fdata-sections   -pthread -MD -MT third_party/libgtest/tnt/CMakeFiles/gtest.dir/__/src/gtest-all.cc.o -MF third_party/libgtest/tnt/CMakeFiles/gtest.dir/__/src/gtest-all.cc.o.d -o third_party/libgtest/tnt/CMakeFiles/gtest.dir/__/src/gtest-all.cc.o -c ../../third_party/libgtest/src/gtest-all.cc In file included from ../../third_party/libgtest/src/gtest-all.cc:39: ../../third_party/libgtest/tnt/../include/gtest/gtest.h:54:10: fatal error: 'limits' file not found #include <limits>          ^~~~~~~~ 1 error generated. [17/847] Building CXX object libs/math/CMakeFiles/test_math.dir/tests/test_quat.cpp.o FAILED: libs/math/CMakeFiles/test_math.dir/tests/test_quat.cpp.o  /usr/bin/clang++  -DFILAMENT_DRIVER_SUPPORTS_VULKAN -I../../libs/math/include -I../../third_party/libgtest/tnt/../include -std=c++14 -fstrict-aliasing -Wno-unknown-pragmas -Wno-unused-function -stdlib=libc++ -fcolor-diagnostics -O2 -DNDEBUG -fomit-frame-pointer -ffunction-sections -fdata-sections   -pthread -MD -MT libs/math/CMakeFiles/test_math.dir/tests/test_quat.cpp.o -MF libs/math/CMakeFiles/test_math.dir/tests/test_quat.cpp.o.d -o libs/math/CMakeFiles/test_math.dir/tests/test_quat.cpp.o -c ../../libs/math/tests/test_quat.cpp ../../libs/math/tests/test_quat.cpp:18:10: fatal error: 'random' file not found #include <random>          ^~~~~~~~ 1 error generated. [18/847] Building CXX object libs/math/CMakeFiles/test_math.dir/tests/test_mat.cpp.o FAILED: libs/math/CMakeFiles/test_math.dir/tests/test_mat.cpp.o  /usr/bin/clang++  -DFILAMENT_DRIVER_SUPPORTS_VULKAN -I../../libs/math/include -I../../third_party/libgtest/tnt/../include -std=c++14 -fstrict-aliasing -Wno-unknown-pragmas -Wno-unused-function -stdlib=libc++ -fcolor-diagnostics -O2 -DNDEBUG -fomit-frame-pointer -ffunction-sections -fdata-sections   -pthread -MD -MT libs/math/CMakeFiles/test_math.dir/tests/test_mat.cpp.o -MF libs/math/CMakeFiles/test_math.dir/tests/test_mat.cpp.o.d -o libs/math/CMakeFiles/test_math.dir/tests/test_mat.cpp.o -c ../../libs/math/tests/test_mat.cpp In file included from ../../libs/math/tests/test_mat.cpp:17: ../../third_party/libgtest/tnt/../include/gtest/gtest.h:54:10: fatal error: 'limits' file not found #include <limits>          ^~~~~~~~ 1 error generated.

I experience the problem on ubuntu 18.04.

Thanks for reporting this issue. We only support compiling with clang and you need to have the clang STL installed (it should be mentioned in README.md, you need to install libc++-dev and possibly libc++abi-dev).
I'll fix the docs, it's supposed to prefix cmake with CC=… CXX=… like you did.

I just updated the README, please let me know if installing libc++-dev fixes the other issue for you.

On ubuntu 18.04, installing libc++-dev and libc++abi-dev makes the compilation work.  Now I get link errors though:
[ 13%] Linking CXX executable test_math
CMakeFiles/test_math.dir/tests/test_fast.cpp.o: In function `__clang_call_terminate':
test_fast.cpp:(.text.__clang_call_terminate[__clang_call_terminate]+0x2): undefined reference to `__cxa_begin_catch'

> clang++ --version
clang version 6.0.0-1ubuntu2 (tags/RELEASE_600/final)
Target: x86_64-pc-linux-gnu

Same issue here using the same setup.

You can fix this by passing -DFILAMENT_REQUIRES_CXXABI=true to cmake. We use this is our CI environment for the same reason, I just don't know in what cases it's sometimes necessary. What this flag does is add -lc++abi to link against libc++abi instead of GCC's runtime.

BTW if anybody know how to make this work properly, I'm all ears. I made linking against libc++abi optional because on the Linux distributions we use at work (Debian Rodete) using the flag breaks compilation. I do need to use it on Ubuntu 14 for continuous builds though. Not being a daily Linux user I'm not entirely sure what the problem is.

I did manage to get the release build to link on fedora by commenting out the -static-libgcc -static-libstdc++ in the CMakeLists.txt and using the -DFILAMENT_REQUIRES_CXXABI=true flag for cmake (unclear if that was needed).
It is unclear why it wasn't finding the static libraries (I installed libstdc++-static-8.1.1-5.fc28.x86_64).
My initial problem was caused by a missing libstdc++ devel package (I just assumed it had be installed previously).

The static link is there only for us when we need to distribute binaries. For your own machine... :))

Just got the build to work on Ubuntu 18.04. In addition to following the instructions in the readme for setting the compiler to Clang, I needed to pass the -DFILAMENT_REQUIRES_CXXABI=true flag and install libxi-dev, which seems to be required by the copy of SDL2 in the third_party directory.
Might be worth adding libxi-dev, cmake and clang to the list of dependencies, although it's fairly obvious from reading the rest of the readme that those last two are needed.

@thendrix This particular CI infrastructure uses Ubuntu 14, I do not control this. We actually develop on Debian Rodete.
@RoryDungan The README already mentions that cmake and clang are required (they're the first listed requirements :). Good catch about libxi-dev, I'll add it to the README.

It would be useful to specify the minimum versions as well. It took a few tweaks to build on Ubuntu 16.04. Starting with d9ba3998d2d986fa0b20db81fce27b5bdf0771a2, I followed these steps:
# install dependencies
sudo apt install cmake ninja-build clang-6.0 libglu1-mesa-dev libc++-dev libc++abi-dev libxi-dev

# apply patch to fix compile/link errors
git apply <<'EOF'
diff --git a/CMakeLists.txt b/CMakeLists.txt
index 8ca0d25..5fa5c27 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -74,13 +74,14 @@ endif()

 if (LINUX)
     set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -stdlib=libc++")
+    set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -isystem /usr/include/libcxxabi")
     if (FILAMENT_REQUIRES_CXXABI)
         # Required in CI environment with custom libc++ and libc++abi
         link_libraries("-lc++abi")
     endif()
     # To distribute our binaries, we must remove the dependency on libc++ and libgcc.
     if (CMAKE_BUILD_TYPE STREQUAL "Release")
-        link_libraries("-static-libgcc -static-libstdc++")
+#        link_libraries("-static-libgcc -static-libstdc++")
     endif()
 else()
     if (FILAMENT_REQUIRES_CXXABI)
EOF

# configure
CC=/usr/bin/clang-6.0 CXX=/usr/bin/clang++-6.0 cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=out/release/filament -DFILAMENT_REQUIRES_CXXABI=true -H. -Bout/cmake-release -G Ninja

# build
ninja -C out/cmake-release

The README does specify that clang 5.0 is the minimum.

Opps. I saw that just after I posted and I edited my comment, but you're way too fast!

Thanks to @cgmb the readme now checks whether clang is the compiler and that it is the right version.


feat: Add docker image ci

:tada: This PR is included in version 1.0.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Should we change hop_size to frame_rate?

The idea of hop_size might be confusing. Another proposal from @maxsolomonhenry is frame_rate (as number of frames per second).
What any user wants with an audio embedding for downstream use (e.g. for frame based transcription or sed) is that the embedding is based upon the prediction at every particular timestep. However, the input to the embedding might be variable length or use multi-scale centered frames.
The concern was that this distinction might not be clear and hop_size suggests classic overlapp add stuff.

After further discussion, we don't want to do this.


Configuration not reloading when Mappings are updated

Describe the bug
Our deployment(s) of emissary-ingress have been showing signs of not loading their config from Kubernetes state. For example, updating a mapping, then checking in the admin UI and the updated mapping is not loaded. Additionally the admin UI shows the config being many days old, even though mappings have changed. If you restart the deployment, the config will then be updated.
Configuration from 2023-09-27 13:49:44.491353 — 13 days, 1 hour, 37 minutes, 20 seconds ago
To Reproduce

Run emissary-ingress 3.7.2 on Kubernetes 1.25.8.
Apply a Mapping
Then, modify that mapping and re-apply it
Open the admin UI and you will see the old mapping config
Restart the emissary deployment and reload the admin ui, which will now show the right mapping data

Expected behavior
I expected the mapping to update in emissary-ingress as it always has. Unclear if this broke with our upgrade to 3.7 sometime in July.
Versions (please complete the following information):

Ambassador: 3.7.2
Kubernetes environment: kOps on AWS
Version: 1.25.8

Additional context
https://datawire-oss.slack.com/archives/CAULN7S76/p1696951814408599

confirm, version 3.7.0


module reload twice

Describe the bug
It works well when started first. Then when I saved file which trigger 'full-reload', the file reload twice, one with timestamp, and another is not.
Reproduction

start server
save file which trigger "full-reload"

System Info
System:
    OS: Windows 10 10.0.19043
  Binaries:
    Node: 14.17.3 - C:\Program Files\nodejs\node.EXE
    Yarn: 1.22.11 - C:\Program Files\nodejs\yarn.CMD
    npm: 6.14.13 - C:\Program Files\nodejs\npm.CMD
  Browsers:
    Edge: Spartan (44.19041.1263.0), Chromium (94.0.992.31)
  npmPackages:
    @vitejs/plugin-vue: ^1.2.2 => 1.4.0
    vite: ^2.3.4 => 2.6.1

Used Package Manager
yarn
Logs
No response
Validations

[X] Follow our Code of Conduct
[X] Read the Contributing Guidelines.
[X] Read the docs.
[X] Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
[X] Make sure this is a Vite issue and not a framework-specific issue. For example, if it's a Vue SFC related bug, it should likely be reported to https://github.com/vuejs/vue-next instead.
[X] Check that this is a concrete bug. For Q&A open a GitHub Discussion or join our Discord Chat Server.
[X] The provided reproduction is a minimal reproducible example of the bug.

@brucewar Does it still happen? If it does, please provide a reproduction with stackblitz or github.


Support for multiple outputs in prefect tasks

Prefect tasks can return multiple outputs and it would be nice if the FunnelResult supported this as well.
from typing import Tuple

import xarray as xr

from funnel import CacheStore
from funnel.prefect.result import FunnelResult
from prefect import task, Flow

store = CacheStore()

@task(result=FunnelResult(store, serializer='xarray.zarr'), target='foo')
def my_task() -> Tuple[xr.Dataset, xr.Dataset]:
    a = xr.DataArray([1, 2, 3], dims='x', name='foo-1').to_dataset()
    b = xr.DataArray([1, 2, 3], dims='x', name='foo-2').to_dataset()
    return a, b

with Flow('foo-flow') as flow:
    ds_a, ds_b = my_task()

flow.run()

This currently fails with the following error:
Traceback (most recent call last):
  File "/srv/conda/envs/notebook/lib/python3.9/site-packages/xarray/core/dataset.py", line 1398, in _construct_dataarray
    variable = self._variables[name]
KeyError: 0

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/srv/conda/envs/notebook/lib/python3.9/site-packages/prefect/engine/task_runner.py", line 876, in get_task_run_state
    value = prefect.utilities.executors.run_task_with_timeout(
  File "/srv/conda/envs/notebook/lib/python3.9/site-packages/prefect/utilities/executors.py", line 454, in run_task_with_timeout
    return task.run(*args, **kwargs)  # type: ignore
  File "/srv/conda/envs/notebook/lib/python3.9/site-packages/prefect/utilities/tasks.py", line 456, in method
    return run_method(self, *args, **kwargs)
  File "/srv/conda/envs/notebook/lib/python3.9/site-packages/prefect/tasks/core/operators.py", line 38, in run
    return task_result[key]
  File "/srv/conda/envs/notebook/lib/python3.9/site-packages/xarray/core/dataset.py", line 1502, in __getitem__
    return self._construct_dataarray(key)
  File "/srv/conda/envs/notebook/lib/python3.9/site-packages/xarray/core/dataset.py", line 1400, in _construct_dataarray
    _, name, variable = _get_virtual_variable(
  File "/srv/conda/envs/notebook/lib/python3.9/site-packages/xarray/core/dataset.py", line 158, in _get_virtual_variable
    raise KeyError(key)
KeyError: 0

But this has me thinking that the FunnelResult may need some slight modifications to handle multiple outputs. As a starting point, I'm curious if a modified artifact schema is necessary. In the example below I turn the json object into a list of artifacts
# /tmp/funnel_metadata_store/foo.artifact.json
[
  {
    "key": "foo[0]",
    "serializer": "xarray.zarr",
    "load_kwargs": {},
    "dump_kwargs": {},
    "created_at": "2021-12-03T23:19:11.186053"
  },
  {
    "key": "foo[1]",
    "serializer": "xarray.zarr",
    "load_kwargs": {},
    "dump_kwargs": {},
    "created_at": "2021-12-03T23:19:11.186053"
  }
]

I believe we could support this by

Allowing put() method to receive non-keyword arguments as values:  https://github.com/NCAR/esds-funnel/blob/82c4b984e2b97a1bbe07435367e7606e80bf9c8f/funnel/cache.py#L182-L186
I'm imagining
 def put(
     self,
     key: str,
     *value: typing.Any,

Extending the sidecar file schema to consist of a list of Artifact as you suggested above.

Modifying get() method to return a single value or a tuple of values depending on the number of artifacts recorded in the sidecar JSON file.

I have one question though. In your example, the type of returned values is homogeneous (which makes it easy to serialize using the same serializer). What should happen when the function returns multiple outputs with different types, say a tuple with a Pandas DataFrame and an xarray Dataset? Should we even support this use case?

What should happen when the function returns multiple outputs with different types, say a tuple with a Pandas DataFrame and an xarray Dataset? Should we even support this use case?

Honestly, I'm not sure how to handle this. We may want to share this use case (and the funnel project) with the Prefect developers and see what they say.

it appears that Prefect itself doesn't support caching for tasks that return multiple values. This conclusion is purely based on my attempts at tinkering with the caching and going over the docs. So, if I missed something, please let me know :).
Also, I started a discussion here: https://github.com/PrefectHQ/prefect/discussions/5201


SAK-33888 - Submitted/Ungraded submission counts in the assignments list are not correct for non-electronic assignments

Submitted/Ungraded submission counts in the assignments list are not correct for non-electronic assignments

@ern @bgarciaentornos Could you take a look?
I'm NOT specially happy with this implementation because the legacy logic is in the VM, I suggest to move the logic to the service, what do you think?
I've tested this in Master, with a converted 11.x instance and the results are fine now.

Hi @mpellicer I agree with your comment would be better if there was no logic in UI. I would suggest moving the logic to the service.

Hi @ern, I moved the logic from the VM to the service.
I've decided to create two new methods in the repository in order to not put logic in there, then you have different methods for electronic and non-electronic submissions. I could add a parameter like the previous implementation but I think this is cleaner.

So creating these new methods results in over populating the api and more code. It would be more ideal to create a single method with params maybe something like:
long countSubmittedSubmissionsForAssignment(String assignmentId, boolean graded, SubmissionType type);

I've rebased the changes, now it uses only one method with parameters.

Refactored this a bit as the method AssignmentRepository.countAssignmentSubmissions contained to much taint from the service. Now this method stands on its own and the service uses the method by supplying the correct params to get the result it desires.


Badge request for OpenMined and PySyft

:clipboard: OpenMined is a relatively famous community now that is working on federated learning. We need a logo for the same now. PySyft is one of their open-source products that is aimed at facilitating secure and private AI.
A clear and concise description of the new badge.

Which service is this badge for e.g: OpenMined
What sort of information should this badge show?
Provide an example in plain text e.g: OpenMined/PySyft

:link: Data
Where can we get the data from?

Is there a public API?
Yes. PySyft, OpenMined
Does the API requires an API key? No

:microphone: Motivation
OpenMined is a relatively famous community now that is working on federated learning. We need a logo for the same now. PySyft is one of their open-source products that is aimed at facilitating secure and private AI.

Hello @arghyadeep99 ! 👋🏻
Thanks for suggesting this. Our preferred way of enabling a new logo on Shields.io is to do it via SimpleIcons. See our documentation for a bit more information. 😉

@PyvesB aah yes, my bad, I forgot. The last request I made was back in 2019 for PyTorch, which was accepted, and it's been a while since then. Totally forgot the procedure. Will make the request at the appropriate place. Thanks a lot!

No worries! I see you've raised https://github.com/simple-icons/simple-icons/issues/5535 and https://github.com/simple-icons/simple-icons/issues/5536, will go ahead and close this issue.


USB Signal detection missing on alpha10

[x] I confirm that this is an issue rather than a question.

Bug report
The USB signal detection is missing from the web interface. It was there with alpha9 through.
Steps to reproduce
Open the web interface and search for signal detection.
What is expected?
Finding it.
What is actually happening?
Going crazy cause it's nowhere.
System

Hyperion Server:

Build:           coreelec-19 (Portisch-1333b74d4d/c908f00eaa-1628142929)
Build time:      Aug  5 2021 09:16:28
Git Remote:      git@github.com:CoreELEC/CoreELEC.git
Version:         2.0.0-alpha.10
UI Lang:         de (BrowserLang: de-DE)
UI Access:       default
Avail Capt:      v4l2,framebuffer,amlogic,qt
Database:        read/write

Hyperion Server OS:

Distribution:   CoreELEC (official): 19.2-Matrix_rc3
Architecture:   arm64
CPU Model:      Amlogic S905X rev b
CPU Revision:   0400
CPU Hardware:   Le Potato
Kernel:         linux (4.9.113 (WS: 32))
Qt Version:     5.13.0
Python Version: 3.8.9
Browser:        Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.114 Safari/537.36

Never mind, the update just resetted the config level from expert back to default.

@V10lator Please open a new issue for this.
Thanks a lot

@Paulchen-Panther I'll do tomorrow but just to clearify: You want me to open a new report for the issue that the config setting resetted with the update, right? I also found another setting (Capturing Hardware) to reset which was way more annoying as I had to reconfigure all subsettings from scratch. So do you want me to open two new bug reports or just one?

@V10lator No need to spend effort capturing issues on your end.
a) The Settings Level is only captured locally in your Browser. In case you deleted Cookies or used a different browser. You need to switch settings level. Nothing dependent on the Hyperion server part.
b) As outlined in the Release Notes,
https://github.com/hyperion-project/hyperion.ng/releases
the grabber is disabled and you need to enable it.
@portisch Might be worth to add this to the CoreElec Plug-in Release notes too.


Create a build task

index.js is nice, but we should be able to build the nitrogen.min.js with some tool.

Is scripts/build-module what you are looking for?


Gazebo crashes with Noetic when spawning model

After upgrading to Noetic, the simulator crashes upon spawning the drone model. See the error message below:

[Msg] 	- auv/thruster_0_joint
[Msg] 	- auv/thruster_1_joint
[Msg] 	- auv/thruster_2_joint
[Msg] 	- auv/thruster_3_joint
[Msg] 	- auv/thruster_4_joint
[Msg] 	- auv/thruster_5_joint
[Msg] 	- auv/thruster_6_joint
[Msg] 	- auv/thruster_7_joint
free(): invalid size
Aborted (core dumped)
[gazebo-2] process has died [pid 80110, exit code 134, cmd /opt/ros/noetic/lib/gazebo_ros/gzserver --verbose -e ode worlds/cybernetics_pool.world __name:=gazebo __log:=/home/strom/.ros/log/6b971a5e-fdfd-11eb-a3d1-7141ecb3f031/gazebo-2.log].
log file: /home/strom/.ros/log/6b971a5e-fdfd-11eb-a3d1-7141ecb3f031/gazebo-2*.log
[auv/urdf_spawner-6] process has finished cleanly

Offending line is 355 in /beluga_description/scripts/spawn_model:

        success = gazebo_interface.spawn_urdf_model_client(self.model_name, model_xml, self.robot_namespace, initial_pose, self.reference_frame, self.gazebo_namespace)

Also see line 20 here: http://docs.ros.org/en/groovy/api/gazebo/html/gazebo__interface_8py_source.html

https://answers.ros.org/question/168156/gazebo-crashes-when-spawning-a-robot-with-urdf/ Leaving this here for tomorrow :)

Got things working by updating the uuv_simulator-related folders here, but this indicates a greater problem of not being able to easily update our repo with updates in uuv_simulator..


[feature]: Stacked Create App with organization bundle identifier definition (non com.example)

Is your feature request related to a problem? Please describe.
Always when i create a new app, i need to manually update de bundle identifiers.
Flutter create app provides the --org flag that already perform this for us...
--org                    The organization responsible for your new Flutter project, in reverse
                             domain name notation. This string is used in Java package names and as
                             prefix in the iOS bundle identifier.
                             (defaults to "com.example")

Describe the solution you would like
stacked create app <app name> --org <org bundle identifier>
This would override the defaults com.example from current stacked create app that do not change the default behavior of flutter create app
Additional Context
No response

Hi @yangricardo, this feature is already in pre-release code, will be out very soon.

Stacked CLI v1.10.0 released including this feature.


Text input broken on Firefox Mobile after enabling this

if I enable this plugin, the text field to enter a new discussion or reply to a discussion is somewhat broken with Firefox Android. You can still type somewhat, but when the onscreen keyboard is up you cant see what you are typing and the overall layout of the text editor is just broken.

This is what it looks like on my forum.

This could well be an issue with Flarum in itself. I'm not taking action until beta 8 is released. The extension needs to be checked for consistency anyway.


correct pip install  syntax

with pip install from github for me had to be
pip install https://github.com/cosmo-epfl/scikit-cosmo/archive/refs/heads/main.zip
the one mentioned in the readme does not work
pip install https://github.com/cosmo-epfl/scikit-cosmo
Collecting https://github.com/cosmo-epfl/scikit-cosmo
Downloading https://github.com/cosmo-epfl/scikit-cosmo
- 180 kB 11.9 MB/s
ERROR: Cannot unpack file /tmp/pip-unpack-wv9lqln4/scikit-cosmo (downloaded from /tmp/pip-req-build-jcs6rwpu, content-type: text/html; charset=utf-8); cannot detect archive format
ERROR: Cannot determine archive format of /tmp/pip-req-build-jcs6rwpu

Does pip install git+https://github.com/cosmo-epfl/scikit-cosmo works for you? That should be the correct command.

yes pip install git+https://github.com/cosmo-epfl/scikit-cosmo works

Good to know, that's now the updated installation method in the README!


Unsupported attribute: eks_blueprints_addons/main.tf line 3688

Since the merge of the last PR this line is giving an error:
https://github.com/aws-ia/terraform-aws-eks-blueprints-addons/commit/b338f77384a79ff01dfba5fba4477533afe8fe51#diff-dc46acf24afd63ef8c556b77c126ccc6e578bc87e3aa09a931f33d9bf2532fbbR3688
╷
│ Error: Unsupported attribute
│
│   on .terraform/modules/eks_blueprints_addons/main.tf line 3688, in locals:
│ 3688:   wait_for_cert_manager = var.cert_manager.wait ? [module.cert_manager] : []
│     ├────────────────
│     │ var.cert_manager is object with no attributes
│
│ This object does not have an attribute named "wait".```

Can add this to your module args until a fix is merged
cert_manager = {
  wait = false
}

In my case that doesn't work as I am using this module as a transient dependency of terraform-aws-eks (well  to be honest - I don't know how to pass module args there, even tho it might be technically possilbe....)


Logged-in user's shipping address is not saved when placing order with registration

What is the motivation for adding / enhancing this feature?
Customer who filled in the shipping address, and checked the box createing account, would expect the shipping address saved for future use. Even customer saved their shipping address in user > my shipping adress section on topbar, the shipping address is not copying to their checkout page.
What are the acceptance criteria

...Save shipping address in customer's account if they chosed to register account during check out;
...customer saved shipping address should copy to the check out page everytime they check out.

Can you complete this feature request by yourself?
trying, not sure
Additional information

[ ] → Save shipping address in customer's account if they choose to register account during check out; - fixed
[ ] → customer saved shipping address should copy to the check out page everytime they check out. - it doesn't work Saved shipping address is not automatically copied to the checkout

@iwonapiotrowska can you please take care of this one?

@przspa I've assigned this issue to you if it won't be a problem?

@przspa i just wanted to check the status?;)

@pkarw for me

customer saved shipping address should copy to the check out page everytime they check out. - it doesn't work Saved shipping address is not automatically copied to the checkout

is worked, I can't replicate these issues or not know how it's should work @alinadivante can you check it or tell me more about these issues

@przspa the issue here is that when localStorage is cleared out the data is not being copied from data loaded from Magento (when user logs in we load api/user/me that should consist of addresses)


Single post, interesting people block should not appear

The block "Interesting people" on right sidebar should not appear on single post when logged:

Also there should be a bottom margin on sidebar right blocks

Yes, was going to report this. Thanks for fixing it. Also noticed: on Recommended Posts, there's no context menu (i.e. I can't right-click), nor Cmd + Click to open in a new tab. I don't always want to navigate away from the page I'm on right away.
I can file into another issue if you'd like.

@ryanbaer yeah probably should put it in another issue

Couldn't assign you, so here you go: https://github.com/busyorg/busy/issues/917

@ryanbaer thanks!

Fixed in #916


Wrong reset cause on brownout (ESP_RST_WDT instead of ESP_RST_BROWNOUT) (IDFGH-9468)

Answers checklist.

[X] I have read the documentation ESP-IDF Programming Guide and the issue is not addressed there.
[X] I have updated my IDF branch (master or release) to the latest version and checked that the issue is present there.
[X] I have searched the issue tracker for a similar issue and not found a similar issue.

IDF version.
4.4.4
Operating System used.
Windows
How did you build your project?
VS Code IDE
If you are using Windows, please specify command line type.
None
Development Kit.
TTGO (ESP32)
Power Supply used.
USB
What is the expected behavior?
After a brownout and the subsequent restart, ESP_RST_BROWNOUT should be returned by esp_reset_reason().
What is the actual behavior?
Instead, esp_reset_reason() returns ESP_RST_WDT.
Steps to reproduce.
Run any code that displays the result of esp_reset_reason(), then provoke a supply voltage instability. The console will correctly show "Brownout detector was triggered", the ESP32 will restart and esp_reset_reason() will show the wrong reason.
Debug Logs.
No response
More Information.
The IDF sets the reset reason correctly:
https://github.com/espressif/esp-idf/blob/release/v4.4/components/esp_system/port/brownout.c#L66
But it seems to get overwritten by the way the CPUs are reset (using the watchdog).

The console will show "Brownout detector was triggered", the ESP32 will restart and show the wrong reason.

@kriegste By "show the wrong reason", are you referring to the message printed by the ROM bootloader (rst:0x1 (RTC_WDT_RESET),boot:0x1f (SPI_FAST_FLASH_BOOT)) or the return value of esp_reset_reason function on the subsequent boot?

The return value of esp_reset_reason() is wrong.
I have not checked the bootloader message yet. But it is irrelevant since the app relies on esp_reset_reason(). Some background why this is important:
To make sure all is well after an OTA update, the app monitors each reset reason. If the reason is either ESP_RST_PANIC, ESP_RST_INT_WDT, ESP_RST_TASK_WDT or ESP_RST_WDT five times in a row, this is interpreted as being stuck in a boot loop. The app then attempts a roll back. Now, ESP_RST_BROWNOUT is a hardware problem and should not be the cause for a firmware roll back. So this cause should be distinguishable from the (regular) ESP_RST_WDT.

@kriegste the reset reason seems to be set correctly on v4.4.4 after a brownout is detected when I tested.
What could be happening though is that after setting the brownout hint the restart code could be getting stuck somewhere. (we enable the RTC WDT while doing the reset to protect us from this).
Are you able to reproduce this using a minimal example, e.g. hello world?

Strange. It seems to be a different issue.
I had the device running for days, developing and doing OTA updates, but apart from provoking brownouts, I never did a real power cycle.
After a proper power cycle (hardware reset), the problem disappeared... I can no longer reproduce it... The reset reason is correct now (ESP_RST_BROWNOUT).
Could it be that I messed up the RTC WDT so bad that even the (software) resets didn't fix it? I wonder how I can debug this.

@kriegste do you remember what the reset reason printed from the bootloader was? Do you have a full log of the whole process? If it was a SW reset then yes, the RTC related configs would not be cleared, but IDF should still reconfigure the RTC WDT on next boot.
Did your SW touch the RTC in any way (menuconfig/calling rtc wdt in your code/change)?

No, I do not see the bootloader log, I redirect the console messages to RAM using esp_log_set_vprintf() and esp_rom_install_channel_putc() access it via a HTTP server. So whatever happens before that, is lost.
The app does not touch the RTC WDT. The only "RTC" thing it uses is variables in RTC memory, aka RTC_NOINIT_ATTR, and lots of it.
It sounds plausible that brownouts can make the chip hang. But the message "Brownout detector was triggered" appeared every time. And now, try as I may, I cannot reproduce the issue any more.
One thing, though, I will attempt to do: Running an older version of my app compiled with 4.4.3 and then OTA upgrading to the newer 4.4.4 based version. This is what I did in the past weeks. I had the device running on my desk and also the IDF got updated during the time.

@kriegste any updates on this one? Are you still seeing this issue?

I have not seen it since upgrading to 4.4.4 and doing a proper power-cycle.

@kriegste That's good! Closing this one for now then, feel free to reopen if there is any new information.

I have reproduced it powering esp with a regulated dc power supply and dropping it's voltage to something around 1.3V.

@igrr This actually makes sense.  What will happen if the flash chip (3.3v) will stop working after the initial reset, during ROM code execution? Will the reset reason stay the same (brownout), or get overwritten by WDT timeout?

From my observation, a brownout sometimes results in a RTCWDT_RTC_RESET, same as the spurious RTCWDT_RTC_RESET during initial poweron on ESP32 rev1.
An RTCWDT_RTC_RESET will replace the original reset cause.
Also, I've noticed that the reset hint stored in RTC_RESET_CAUSE_REG is cleared by RTCWDT_RTC_RESET, adding to the confusion.
(this applies to the ESP32)

Is there already an issue (link?) for the

spurious RTCWDT_RTC_RESET during initial poweron
?
TTGO T-Display's ESP32 is indeed revision 1 as well.

Is there already an issue (link?) for the

spurious RTCWDT_RTC_RESET during initial poweron

https://www.espressif.com/sites/default/files/documentation/esp32_errata_en.pdf
3.1

I don't know if that's the expected behavior, but from some tests i got this:

If supply voltage drops to 2.3v and then back to 3.3v: got RTCWDT_BROWN_OUT_RESET

If supply voltage drops to 1.3v and then back to 3.3v: first got RTCWDT_BROWN_OUT_RESET then after a while RTCWDT_RTC_RESET(I think it's like @szmodz told here)

If supply voltage drops to 0.3v and then back to 3.3v: first got RTCWDT_BROWN_OUT_RESET then right after SW_CPU_RESET(This one I think doesn't make sense, not sure)

This should be re-opened.

I can't tell accurately since there is no FULL log before and after the brownout...
But here's my imagination:

The voltage drops under BOD threshold
Luckily BOD reset happen (or ISR triggered and do a SW reset as well as store the reason into RTC REG)
The flash fails to work. Most flash have a working voltage no larger than 2.3V. Some of them are 2.5, 2.7V. There is only a few allows 1.6V.
After reset, when 1st bootloader attempts to read from flash, it fails to read the contents correctly. If it takes too long, 1st bootloader will be reset by RTCWDT hardware, and try again.

If you provide the FULL log and the interval between each significant log, maybe I can try to explain more.. And your sdkconfig will also help a lot.
btw, I think under the situation of brownout, trying to erase/write anything to the flash are very dangerous. Maybe we should try to avoid it.

@ginkgm according to the WROVER-B/E datasheet, the "Recommended Operating Conditions" voltage is 3.0 to 3.6V, which is a bit tight TBH.
If you look at the original ESP-WROVER datasheet, which used 1.8V flash, the "Recommended Operating Conditions"  are much less restrictive.
But yeah, I also imagine something like that happens. Point is, the 1st bootloader crash after the initial reboot overwrites the initial reset cause, and also clears the SW defined RTC_RESET_CAUSE_REG.
I don't think anybody mentioned anything about erasing / writing flash other than monitoring the status of OTA update. If the new FW crashed after an OTA update, you generally want to know why it crashed.  Seeing RTC WDT as the reset cause can be confusing.
Personally, I leave the RTC watchdog disabled in the main FW, so at least I know that any RTC WDT crash must have been before "app_main".

@ginkgm The BOD detector is pretty much useless if you never actually get "ESP_RST_BROWNOUT" as the cause.
Let's assume we're dealing with buggy prototype HW, for example using an RS-485 transceiver powered from the same regulator as the ESP32. A short on the A, B lines causes large current flow, resulting in a voltage drop on the ESP32 VDD33, resulting in brownout, and what you get on the FW side is RTC WDT.
The purpose of the BOD detector is to identify power supply problems, is it not? It would be better if you saw either brownout or poweron as the cause, not RTC WDT.

@ginkgm

and the interval between each significant log

You mean the interval between the ESP32 ROM bootloader messages? That would require a custom logger attached to the ESP32 UART, which could also be buggy.

@ginkgm I assume the tight WROVER-B/E VDD33 requirements are due to the additional voltage drop on the VDD_SDIO regulator.

@ginkgm
The fact that the message string "\r\nBrownout detector was triggered\r\n\r\n" is placed in FLASH doesn't help either.
It should probably use the DRAM_STR macro.
https://github.com/espressif/esp-idf/blob/dc016f59877d13e6e7d4fc193aa5aa764547f16d/components/esp_system/port/brownout.c#L54

@szmodz

according to the WROVER-B/E datasheet, the "Recommended Operating Conditions" voltage is 3.0 to 3.6V, which is a bit tight TBH. It's above the max BOD threshold.
If you look at the original ESP-WROVER datasheet, which used 1.8V flash, the "Recommended Operating Conditions" are much less restrictive.

The main difference is that, ESP-Wrover uses 1.8V flash inside, while other modules, ESP-Wroom, ESP-Wrover-B/E uses 3.3V flash.
The 3.3V power is supplied via the input of ESP32 via a small resistor, while the 1.8V power is supplied via an internal regulator, which can tolerate lower input.

Seeing from the TRM: https://www.espressif.com/sites/default/files/documentation/esp32_technical_reference_manual_en.pdf
I know your requirements that knowing about a BOD reset is useful. But..
section 3.1.2, your reset seems to be a System Reset level one,  caused by RWDT System Reset. This will reset all the registers, including the one used to hold the reset reason. To avoid this behavior, you may try to turn it off , see: https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-guides/fatal-errors.html#rtc-watchdog-timeout.
But when brownout issue occurs, the behavior become undetermined, you will not be able to do very complex workaround.
More importantly, I'd like to know what causes the RTCWDT reset after the BOD reset. That's why I asked for the FULL log. Why the console output cannot be provided?

@ginkgm Are all necessary information in this log?
ets Jul 29 2019 12:21:46

rst:0xf (RTCWDT_BROWN_OUT_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:1
load:0x3fff0030,len:7008
load:0x40078000,len:15684
ho 0 tail 12 room 4
load:0x40080400,len:4516
0x40080400: _init at ??:?

entry 0x400806b8
I (30) boot: ESP-IDF v4.4.4 2nd stage bootloader
I (30) boot: compile time 08:36:31
I (30) boot: chip revision: v3.0
I (33) boot_comm: chip revision: 3, min. bootloader chip revision: 1
I (40) qio_mode: Enabling default flash chip QIO
I (45) boot.esp32: SPI Speed      : 80MHz
I (50) boot.esp32: SPI Mode       : QIO
I (54) boot.esp32: SPI Flash Size : 4MB
I (59) boot: Enabling RNG early entropy source...
I (64) boot: Partition Table:
I (68) boot: ## Label            Usage          Type ST Offset   Length
I (75) boot:  0 nvs              WiFi data        01 02 00009000 00005000
I (83) boot:  1 otadata          OTA data         01 00 0000e000 00002000
I (90) boot:  2 app0             OTA app          00 10 00010000 001b0000
I (98) boot:  3 app1             OTA app          00 11 001c0000 001b0000
I (105) boot:  4 spiffs           Unknown data     01 82 00370000 00060000
I (113) boot:  5 coredump         Unknown data     01 03 003d0000 00010000
I (120) boot:  6 reserved         Unknown data     01 fe 003e0000 00020000
I (128) boot: End of partition table
I (132) boot_comm: chip revision: 3, min. application chip revision: 1
I (139) esp_image: segment 0: paddr=00010020 vaddr=3f400020 size=3b1d0h (242128) map
��וBets Jul 29 2019 12:21:46

rst:0x10 (RTCWDT_RTC_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:1
load:0x3fff0030,len:7008
load:0x40078000,len:15684
ho 0 tail 12 room 4
load:0x40080400,len:4516
0x40080400: _init at ??:?

entry 0x400806b8
I (29) boot: ESP-IDF v4.4.4 2nd stage bootloader
I (29) boot: compile time 08:36:31
I (30) boot: chip revision: v3.0
I (33) boot_comm: chip revision: 3, min. bootloader chip revision: 1
I (40) qio_mode: Enabling default flash chip QIO
I (45) boot.esp32: SPI Speed      : 80MHz
I (50) boot.esp32: SPI Mode       : QIO
I (54) boot.esp32: SPI Flash Size : 4MB
I (59) boot: Enabling RNG early entropy source...
I (64) boot: Partition Table:
I (68) boot: ## Label            Usage          Type ST Offset   Length
I (75) boot:  0 nvs              WiFi data        01 02 00009000 00005000
I (82) boot:  1 otadata          OTA data         01 00 0000e000 00002000
I (90) boot:  2 app0             OTA app          00 10 00010000 001b0000
I (97) boot:  3 app1             OTA app          00 11 001c0000 001b0000
I (105) boot:  4 spiffs           Unknown data     01 82 00370000 00060000
I (112) boot:  5 coredump         Unknown data     01 03 003d0000 00010000
I (120) boot:  6 reserved         Unknown data     01 fe 003e0000 00020000
I (127) boot: End of partition table
I (132) boot_comm: chip revision: 3, min. application chip revision: 1
I (139) esp_image: segment 0: paddr=00010020 vaddr=3f400020 size=3b1d0h (242128) map
I (214) esp_image: segment 1: paddr=0004b1f8 vaddr=3ffb0000 size=04e20h ( 20000) load
I (221) esp_image: segment 2: paddr=00050020 vaddr=400d0020 size=e95ech (955884) map
I (484) esp_image: segment 3: paddr=00139614 vaddr=3ffb4e20 size=00be8h (  3048) load
I (485) esp_image: segment 4: paddr=0013a204 vaddr=40080000 size=15c80h ( 89216) load
I (519) esp_image: segment 5: paddr=0014fe8c vaddr=00000000 size=000f4h (   244)
I (530) boot: Loaded app from partition at offset 0x10000
I (530) boot: Disabling RNG early entropy source...
I (542) cpu_start: Pro cpu up.
I (542) cpu_start: Starting app cpu, entry point is 0x400811b4
0x400811b4: call_start_cpu1 at C:/esp/esp-idf/components/esp_system/port/cpu_start.c:149

I (0) cpu_start: App cpu up.
I (556) cpu_start: Pro cpu start user code
I (556) cpu_start: cpu freq: 240000000
I (556) cpu_start: Application information:
I (561) cpu_start: Project name:     mesh-shopBA
I (566) cpu_start: App version:      1.3.3
I (571) cpu_start: Compile time:     Jun  5 2023 13:15:21
I (577) cpu_start: ELF file SHA256:  dbb12184e25346a3...
I (583) cpu_start: ESP-IDF:          v4.4.4
I (588) heap_init: Initializing. RAM available for dynamic allocation:
I (595) heap_init: At 3FFAE6E0 len 00001920 (6 KiB): DRAM
I (601) heap_init: At 3FFBAC48 len 000253B8 (148 KiB): DRAM
I (607) heap_init: At 3FFE0440 len 00003AE0 (14 KiB): D/IRAM
I (614) heap_init: At 3FFE4350 len 0001BCB0 (111 KiB): D/IRAM
I (620) heap_init: At 40095C80 len 0000A380 (40 KiB): IRAM
I (627) spi_flash: detected chip: generic
I (631) spi_flash: flash io: qio
I (635) esp_core_dump_flash: Init core dump to flash
I (641) esp_core_dump_flash: Found partition 'coredump' @ 3d0000 65536 bytes
D (682) esp_core_dump_checksum: Total length of hashed data: 40000
I (682) esp_core_dump_flash: Core dump data checksum is correct
I (684) esp_core_dump_flash: Found core dump 40004 bytes in flash @ 0x3d0000
I (692) cpu_start: Starting scheduler on PRO CPU.
I (0) cpu_start: Starting scheduler on APP CPU.

The RTC WDT cannot be disabled in general. It's always activated by the ROM bootloader and the  ROM bootloader timeout cannot be changed.
<IP_ADDRESS> Flash Boot Protection
https://www.espressif.com/sites/default/files/documentation/esp32_technical_reference_manual_en.pdf
My workaround is to store the reset hint (RTC_RESET_CAUSE_REG) in RTC SRAM. This memory is not cleared by RTC WDT reset.
esp-idf already uses the RTC SRAM to store timekeeping info.

My workaround is to store the reset hint (RTC_RESET_CAUSE_REG) in RTC SRAM.

Of course that's 100% not foolproof. The BOD threshold remains below the FLASH recommended operating conditions, so while it should work better most times, it's by no means reliable.

Your log at least shows that there is a RTCWDT reset after the brownout reset. But the 1st/2nd bootloader doesn't help to print the PC before reset.. So we don't know what causes the RTCWDT before the reset.
Doubt whether there are some issue in the reset process that causes the 2nd bootloader failed to load the program again (in time) after reboot..


feat(runtime): support runtime replace image registry

Description
related:  #2305 #2306

client: runtime dockerize use new image rule
server: runtime version's image support replace registry

Modules

[ ] UI
[x] Controller
[ ] Agent
[x] Client
[ ] Python-SDK
[ ] Others

Checklist

[x] run code format and lint check
[x] add unit test
[ ] add necessary doc

Others LGTM


Recreate secret command in yargs

Problem to solve
We need to recreate the secret command in version 2 of the app.
Proposal
This command should work the same as it did in version 1.
We're not looking at supporting anymore OS types with this command, purely to replicate the same functionality from version 1 to 2.
The idea really is that we will advise people pass the token in manually by using the CT_TOKEN variable. As they would do when integrating Conventional Tools in CI.

@AdeAttwood What do we think about this one?
Been playing around with this and it might be best to just leave this one I think.
As we discussed previously, we want to push the use of passing CT_TOKEN in instead. Also with pushing the idea that Conventional Tools should be used in CI (when CT_TOKEN is required).
Also with not wanting to use execa (which is how this is done in 1.x) and child processes I think it might be best to leave this.

I think it might be best to leave this.

I don't mind. If we are on the fence about removing this feature in V2 then we should leave if off the list, if it becomes a probity we can add it back in. I don't think this is a big feature and will not be a significant amount of work to build in. We will still need to abstract the credentials logic, maybe bundle it into source control host interface like we have the source control.

I think it might be best to leave this.

I don't mind. If we are on the fence about removing this feature in V2 then we should leave if off the list, if it becomes a probity we can add it back in. I don't think this is a big feature and will not be a significant amount of work to build in. We will still need to abstract the credentials logic, maybe bundle it into source control host interface like we have the source control.

Cool, I'll move this into the Backlog milestone and we can move back in to 2.x if we think it's a requirement.


fixed the  "FACING THE ISSUE" positioning

I had corrected your issues for larger screens by removing fixed positions at particular positions.please review it and let me know if you have any corrections.
HOPE IT WORKS WELL.

@gitshanmukh123 attach snapshot of your work please.

INCREASING SCREEN SIZES


Load alternate configuration on unsupported versions

I am attempting to setup a tmux config that will work on various tmux versions. My first attempt was to load configuration conditionally in my ~/.tmux.conf file based on the tmux version, which works great but .tmux doesn't like not being at ~/.tmux.conf .. is there any way around this? Maybe adding a ~/.tmux.conf.unsupported (similar to ~/.tmux.conf.local) option that gets loaded instead of .tmux when it detects unsupported versions?

Hello @rjorgenson 👋
I'm not sure I understand what you're trying to achieve. If you're using tmux < 2.3 then you're really better of not using Oh my tmux! at all. After all it's just a matter of using another .tmux.conf file.

@gpakosz That is essentially the goal, but I deploy my dotfiles to quite a few systems and right now I have to do that logic at deploy time and put the right .tmux.conf in place, and then if that system ever changes down the line I have to manually put .tmux's .tmux.conf into place to start using it. The goal is to have a single configuration file that works on all systems and loads .tmux if it is supported. My initial attempt was to use a .tmux.conf that detected what tmux version I had and loaded configurations accordingly but .tmux doesn't work if it's not your ~/.tmux.conf file.

I don't know how to address that. I would find a way to deploy tmux > 2.3 (which was released in 2016) on the different hosts 🙂

These aren't typically hosts I have control of. Would it not be possible to detect the version just like you're doing now, and instead of killing the server load the alternate config and only load the rest of the config if the version is supported?

I don't think this is a thing I want to support.
But in the meantime you can try this ~/.tmux.conf (still assuming Oh my tmux! has been cloned in ~/.tmux)
# : << EOF

# put your base tmux config here
set -g default-terminal "screen-256color" # colors!
setw -g xterm-keys on
set -s escape-time 10                     # faster command sequences
set -sg repeat-time 600                   # increase repeat timeout
set -s focus-events on

# -- 8< ------------------------------------------------------------------------

run 'cut -c3- ~/.tmux.conf | sh -s _oh_my_tmux_if_supported'

# EOF
#
# # exit the script if any statement returns a non-true return value
# set -e
#
# unset GREP_OPTIONS
# export LC_NUMERIC=C
#
# _tmux_version=$(tmux -V | awk '{gsub(/[^0-9.]/, "", $2); print ($2+0) * 100}')
#
# _oh_my_tmux_if_supported() {
#   if [ $_tmux_version -ge 230 ]; then
#     tmux source ~/.tmux/.tmux.conf
#   fi
# }
#
# if [ x"$@" = x"_oh_my_tmux_if_supported" ]; then
#   _oh_my_tmux_if_supported
# else
#   cut -c3- ~/.tmux/.tmux.conf | sh -s "$@"
# fi

I'll give this a shot when I get a chance, but at first look this doesn't look any different than what I was trying to do, when ~/.tmux/.tmux.conf is sourced it will attempt to cut all of it's bit's out of ~/.tmux.conf but that's not where that file lives. This is the initial problem I ran into when trying to do a conditional load of .tmux.

To make things clear you should clone Oh my tmux! into ~/.tmux and instead of a ~/.tmux.conf → ~/.tmux/.tmux.conf symlink you should create a file named ~/.tmux.confwith the content given above 👆
That new ~/.tmux.conf file will act as a trampoline to ~/.tmux/.tmux.conf is tmux version is >= 2.3

Yes I get that, but when I tried to do that previously (granted in a more basic way, and I don't entirely understand everything that's going on in there but it doesn't appear like it would do anything differently than I was doing) this line https://github.com/gpakosz/.tmux/blob/760bf4fabda9a41f67f8ad7e202c2a66a2a3d4e1/.tmux.conf#L170 would not run because none of the code it's trying to pull out of ~/.tmux.conf and run exists in that file because all it has is the conditional sourcing.

After a few attempts at melting my brain trying to understand what that was doing it finally clicked! This works perfectly for my purposes, thanks!

@gpakosz Running into an issue with this solution, can you think of any reason why TPM would not be loading plugins with this config file but does with the original oh-my-tmux .tmux.conf in place? Sourcing from the same .tmux.conf.local with the below tpm settings in it? Running ~/.tmux/plugins/tmux-resurrect/resurrect.tmux after the session is up loads the plugin and I am able to restore and save my sessions but it's not loaded at runtime and continuum is also not loaded so sessions aren't saved. Everything works as expected when using the original .tmux.conf instead of this wrapper solution.
# -- tpm -----------------------------------------------------------------------

# while I don't use tpm myself, many people requested official support so here
# is a seamless integration that automatically installs plugins in parallel

# whenever a plugin introduces a variable to be used in 'status-left' or
# 'status-right', you can use it in 'tmux_conf_theme_status_left' and
# 'tmux_conf_theme_status_right' variables.

# by default, launching tmux will update tpm and all plugins
#   - true (default)
#   - false
tmux_conf_update_plugins_on_launch=true

# by default, reloading the configuration will update tpm and all plugins
#   - true (default)
#   - false
tmux_conf_update_plugins_on_reload=true

# /!\ do not add set -g @plugin 'tmux-plugins/tpm'
# /!\ do not add run '~/.tmux/plugins/tpm/tpm'

# to enable a plugin, use the 'set -g @plugin' syntax:
# visit https://github.com/tmux-plugins for available plugins
#set -g @plugin 'tmux-plugins/tmux-copycat'
#set -g @plugin 'tmux-plugins/tmux-cpu'
#set -g @plugin 'tmux-plugins/tmux-resurrect'
#set -g @plugin 'tmux-plugins/tmux-continuum'
#set -g @continuum-restore 'on'

# TPM configuration
set -g @plugin 'tmux-plugins/tmux-resurrect'
set -g @plugin 'tmux-plugins/tmux-continuum'

set -g @continuum-restore 'on'
set -g @continuum-save-interval '15'

Plugins are not loading because TPM inspects ~/.tmux.conf and follows source commands.
You can try:
# : << EOF

# put your base tmux config here
set -g default-terminal "screen-256color" # colors!
setw -g xterm-keys on
set -s escape-time 10                     # faster command sequences
set -sg repeat-time 600                   # increase repeat timeout
set -s focus-events on

source -q ~/.tmux.conf.local

# -- 8< ------------------------------------------------------------------------

run 'cut -c3- ~/.tmux.conf | sh -s _oh_my_tmux_if_supported'

# EOF
#
# # exit the script if any statement returns a non-true return value
# set -e
#
# _tmux_version=$(tmux -V | awk '{gsub(/[^0-9.]/, "", $2); print ($2+0) * 100}')
#
# _oh_my_tmux_if_supported() {
#   if [ $_tmux_version -ge 230 ]; then
#     tmux source ~/.tmux/.tmux.conf
#   fi
# }
#
# if [ x"$@" = x"_oh_my_tmux_if_supported" ]; then
#   _oh_my_tmux_if_supported
# else
#   cut -c3- ~/.tmux/.tmux.conf | sh -s "$@"
# fi

@rjorgenson Hello 👋
Did you give it a try?

@gpakosz sorry I thought I responded to this already. That did work for enabling the plugins in versions >=2.3 but it sourced the local config twice and sourced the local config on all versions. I ended up just reverting to using a different .tmux.conf on hosts with tmux <2.3 as it seems there's just not a good way to accomplish that. Thanks for all your help though! I even learned a bit about the voodoo that you do to get this to work =]

Hmm then I think what you could do is put the set -g plugin ... in the main .tmux.conf instead of source -q ~/.tmux.conf.local. This is the answer I should have made in fact!

@gpakosz I tested that out and it did work but has the side effect of doing a continuum restore every time I reload the config when not using dottmux, I was not able to figure out why it does this. I tried putting various pieces in different stages in the config but no matter where I put it when the config was reloaded and the _oh_my_tmux_if_supported function call loaded the unsupported tmux config file it would run a continuum restore. Manually restoring is definitely not a deal breaker for me as most machines hardly ever go down. Think i'll go with this approach, thanks!


You can't choose the original skins.

Modern Steve design is included, but it would be nice to use the originals.

Fixed, the version on master alongside the latest version of arcropolis will fix this.


TouchEvent.supported always true in Dartium

The following always returns true in Dartium (v45 and v50) when SDK 1.24.1 is used:
print("TouchEvent.supported: ${TouchEvent.supported}");
It works correctly in Chrome et al after dart2js, and also correctly in Dartium when SDK < 1.24 is used.
System: MBP, OSX Sierra

/cc @terrylucas

Dartium is no longer supported.


[ To-me: ' FacilityContxtRepo 《'(Assigned-to-me / Issue Tracker: installing-apps-github / GuilaMonstar.md)'》Flutterfire_apps_Cli]chrome-native://bookmarks/#> https://medium.com/googleplaydev

[ To-me: ' FacilityContxtRepo 《'(Assigned-to-me / Issue Tracker: installing-apps-github / GuilaMonstar.md)'》Flutterfire_apps_Cli]chrome-native://bookmarks/#> https://medium.com/googleplaydev
Publicado originalmente por @GuilaMonstar en https://github.com/Pixelkit/PixelKit-Bootstrap-UI-Kits/issues/32#issuecomment-1426880527

https://github.com/firebase/flutterfire/issues/10427


Support for MCUBoot's single application slot mode

It is possible that USB DFU class is used by MCUBoot itself while it has enabled CONFIG_SINGLE_APPLICATIO_SLOT. In this mode only application slot is the primary slot.
This patch assumes that if the secondary slot dosn't exist this mean modules should operates on the primary slots.
I avoided to use MCUBoot configuration output in zephyr code (so CONFIG_SINGLE_APPLICATIO_SLOT) - which I believe is a one of zephyr design principle.
Also be aware of the bug  #28504
PR includes https://github.com/zephyrproject-rtos/zephyr/pull/28519.
building
west build -d build/single_usb -b nrf52840dk_nrf52840 ./bootloader/mcuboot/boot/zephyr/ -t menuconfig -- -DDTC_OVERLAY_FILE=./boards/nrf52840_single_slot.overlay
set
CONFIG_SINGLE_APPLICATION_SLOT=y
CONFIG_BOOT_WAIT_FOR_USB_DFU=y
CONFIG_MULTITHREADING=y
CONFIG_SIZE_OPTIMIZATIONS=y

west build -d build/single_usb
testing
Similar as for USB DFU sample need to build for instance the hello_word sample with same DTS overlay file as MCUBoot.
west build -d build/hellow_single -b nrf52840dk_nrf52840 ./zephyr/samples/hello_world/ -t menuconfig -- -DDTC_OVERLAY_FILE=../../../bootloader/mcuboot/boot/zephyr/boards/nrf52840_single_slot.overlay
set
CONFIG_BOOTLOADER_MCUBOOT=y
west build -d build/hellow_single
west sign -d build/hellow_single/ -t imgtool -- -k ./bootloader/mcuboot/root-rsa-2048.pem
west flash -d build/single_usb/
sudo dfu-util --alt 0 --download ./build/hellow_single/zephyr/zephyr.signed.bin
console output:
dfu-util 0.9

Copyright 2005-2009 Weston Schmidt, Harald Welte and OpenMoko Inc.
Copyright 2010-2016 Tormod Volden and Stefan Schmidt
This program is Free Software and has ABSOLUTELY NO WARRANTY
Please report bugs to http://sourceforge.net/p/dfu-util/tickets/

dfu-util: Invalid DFU suffix signature
dfu-util: A valid DFU suffix will be required in a future dfu-util release!!!
Opening DFU capable USB device...
ID 2fe3:0100
Run-time device DFU version 0110
Claiming USB DFU Runtime Interface...
Determining device status: state = appIDLE, status = 0
Device really in Runtime Mode, send DFU detach request...
Resetting USB...
Opening DFU USB Device...
Claiming USB DFU Interface...
Setting Alternate Setting #0 ...
Determining device status: state = dfuIDLE, status = 0
dfuIDLE, continuing
DFU mode device DFU version 0110
Device returned transfer size 128
Copying data from PC to DFU device
Download        [=========================] 100%        16592 bytes
Download done.
state(2) = dfuIDLE, status(0) = No error condition is present
Done!

fixes #28502
fixes #28504

@utzig @jfischer-phytec-iot @emob-nordic Please revisit

DNM because #28459 is still not merged, and I am not convinced about it. See my comments there.

@carlescufi #28459 is not required for this PR at all.

DNM because #28459 is still not merged, and I am not convinced about it. See my comments there.

@carlescufi  The #28459 is cosmetic and we can deal with it later. This PR addresses feature request, and coincidentally a bug within compilation, reported by user here https://github.com/zephyrproject-rtos/zephyr/issues/28473.

@bpbradley I think this addresses the issue https://github.com/zephyrproject-rtos/zephyr/issues/28473, that has been reported by you.

@bpbradley I think this addresses the issue #28473, that has been reported by you, please join us in the review.

Yes this PR works! Thanks for the ping.

This PR introduces #ifdefry for making the compilation of zephyr dfu/boot/mcuboot module possible while using in MCUboot bootloader. In the future (not in this PR) I plan to rework this using https://github.com/zephyrproject-rtos/zephyr/issues/29963