Safety development concept (for IEC61508 certification)

Introduction
The quality and finally safety (IEC61508 SIL3 certification) and
security development of Zephyr OS requires

to follow a or several processes
to have a tool environment installed to complete tasks and support the processes.

This RFC defines a rough concept how the safety committee intends to setup, both,
tools and processes to do safety development.
Serveral areas, in example requirements management, are not restricted
to safety and shall be setup at the best of our current knowledge to suite
for the full extend of the ecosystem.
Nevertheless, in order to be able to finish with the safety certifcation task,
it must be accepted that the focus of the safety committee and WG and therefore
this RFC is safety.
Please feel invited

to suggest improvement
to disclose gaps
to ask questions when unclear
to state your opinion

Problem description
An overall concept how to do safety is not (fully) defined, described and known to the safety WG
and TSC.
It is necessary to achieve a common understanding.
Proposed change
Create an overall concept by defining processes and a tool environment that allows
to get 'main' in a 'safety certifiable ready' state of which an auditable can
be branched with basically no gaps to reach certification.
Doing safety on auditable and forward merge all code changes, all tests, requirements etc.
has shown to not work.
We start with an rough concept and iteratively detail and implement the different areas.
Detailed RFC
Proposed change (Detailed)
Overview

Requirements Management
For safety (IEC61508) the following specifications need to be written:
Requirements specifications

User Story (US) - not requested by 61508 but makes requirements more understandable
Software Requirements Specification (SRS)
Software Architecture & Interface Specification (SAIS)
Software Module Design Specification (SMDS)

Requirements management(SRS, SAIS, STS) is done using

Doorstop (https://doorstop.readthedocs.io/en/latest/,https://github.com/doorstop-dev/doorstop)
Doorhole (https://github.com/sevendays/doorhole) may serve as a GUI frontend for convenience

Storage location is zephyrproject-rtos/reqmgmt repo (https://github.com/zephyrproject-rtos/reqmgmt).
Test specifications

Software Test Specification (STS)
Software Integration Test Specification (SITS)
Software Module Test Specification (SMTS)

Remark: possibly replace term 'module' as in 61508 by 'component' as in 50128 or 'unit' as in unit test for SMDS and SMTS, since easily confused with git (sub-)module.
Definition: Module as in 61508 is in most cases referred to as a implementation / header file pair in a C/C++ context (my_module.c/.h).
Test specifications are written as doxygen comments to the unit-, integration- and software tests into the zephyrproject-rtos/zephyr repo (https://github.com/zephyrproject-rtos/zephyr).
The linkage to a Doorstop requirements (SRS, SAIS, SMDS) is done by doxygen tag referencing the requirement by its ID.
Requirements process
Adding requirements or test specifications uses the same workflow as any code change does:

Create a fork repo
Create a branch where all changes are done
Create a Github PR
Review the PR
Merge the PR

The standard github features (PR, diff, review) are used

since Doorstop uses single yaml files per requirement (less merging than a single file approach as e.g. with StrictDoc, but would still work)
the yaml file content is human readable
users are acustomed to the github process (no major changes compared to source code)

Requirements wording
Requirememts shall be

https://en.wikipedia.org/wiki/Requirement#Characteristics_of_good_requirements

and follow the 'Easy Approach to Requirements Syntax' (EARS) principle

https://alistairmavin.com/ears/
https://ieeexplore.ieee.org/document/5328509

Traceability
Traceability is created by a dediated 'tracedoc' tool.
The tool takes the

requirements specification from Doorstop (or any other requirements tool) in a defined csv based format as input
(other formats would be feasible but for simplicity shall not be considered in the beginning)
scans the zephyrproject-rtos/zephyr code base for dedicated doxygen comments with reference tags to the requirements
checks for dead links and missing traceability (report at the end of or highlighted in traceability documentation)
creates a doxygen / sphinx base requirements and traceability documentation

Remark: Intel has a prove-of-concept implemented.
By defining an open and simple input interface (csv) into tracedoc, requirements tools are not limited to Doorstop
and the requirements do not necessarily have to come from the same source (repo, system etc.)
Future: tracedoc could be extended to process multiple csv input files from different requirements tools and scan over multiple repos. We start with the simple 1:1 case.
Linkages
The following link roles will be used:

refines (in the requirements management tool (Doorstop):

software requirement <-- software architecture & interface requirement
software architecture & interface requirement  <-- software module design requirement

implements (Doxygen tag - exact syntax tbd):

software module design requirement <-- source code

verifies (Doxygen tag - exact syntax tbd):

software requirement <-- software test requirement
software architecture & interface requirement  <-- software integration test requirement
software module design requirement <-- software module test requirement

Safety Scope
Why a safety scope? The Zephyr OS is by far too big, that we can achieve a safety certification of the full code base.
Hence we need to restrict it to a smaller scope and provide measures to distinguish non-safety from safety (for us and for a user).
The safety scope as defined will be marked using two mechanisms:

The source files in the safety scope will carry an SPDX tag, similar to the SPDX license tag, indicating the safety relevance/scope

In that way safety relevant code is clear identified and immediately visible to developers and maintainers
Special (stricter) guidelines may apply to source code within the safety scope than to the rest of the code (e.g.: use of dynamic memory)
The SPDX tool can be used to verify that a built binary only comprises of sources out of the safety scope and no other
(similar to that a binary only comprises of sources under a certain license model (e.g. no GPL)

The maintainer.yml will define folders and file belonging to the safety scope

In that manner PRs affecting the safety scope can be auto-assigned to dedicated safety maintainers
Possibly to an include syntax also an exclude syntax need to be defined and implemented (/folder/A but not /folder/A/fileB.c)

Using both, SPDX and maintainer.yml, will be used to validate the safety scope against each other for plausibility / congruence (tool: scope validator, added to CI)
Coding Guidelines
The coding guidelines will be enforced as far as possible using a static code analyzer.
Guidelines which cannot be enforce by tooling, will be verified by code review prior to merging the github PR.
Static Code Analyzer (SCA)

CI

The project uses a commercial grade SCA (Parasoft cpptest is foreseen) in its CI, creating a violation report accessible via github PR for each development branch and main.
The violation report will be differential for a development branch, between the development branch and the origin the development branch was branched from, only showing and enforcing the newly introduced violations.
A full violation report should be available / visible but not be enforced to be cleaned up - in hope some feel motivated to fix low hanging violation fruits
(That could be skipped in CI and be left to the local SCA scans, where differential scans cannot be easily provided)

Local

Commercial SCAs typically have a rather high price tag for the community and the project providing floating license will be come expensive and requires an elaborated infrastructure (license server, access management, preventing from use outside of the project scope etc.)
Different companies use different commercial SCAs, they will be able to integrate their SCA.
The open source SCA "cppcheck" will be available and integrated into the build system to be used on a local machine. Even if the open source SCA does not have the same quality as a commercial grade solutions, it will be suitable to detect and this way fix 80 - 90% of the violations in a fast development cycle. The remaining 10 - 20% will have to be done using the CI integrate SCA with slower development cycles.
The local scans will only provide a full scan of the source files, showing all existing violations (whom ever introduced them).

Testing
Testing will be done according to IEC61508 on the three levels:

(full) software: verifying the software from a user perspective (black box testing)

The software is approached from a user API perspective

integration: verifying the interaction between dedicated modules (grey box testing)

A number of modules are used in a test application. The modules under test will be approached via theirs API, external dependencies will be provided by faking (FFF) or using a device simulation.

module: verifying single functions of a module including code coverage (white box testing)

module API:

Tests the module only via accessing the module API

module unit:

Takes the module-under-test as include into the test implementation file to allow access also to the local (static) functions
Limitation: testing caller functions on all possible return values, where the callee is in the same module will not be possible
in all cases (see module stubbing).

module stubbing:

functions in the module that call static functions of the module might not be fully testable without stubbing or faking the called function
(example: the called function interacting with some hardware which does not fail during tests and therefore always returning 0 = OK) but the callers reaction needs to be tested on error codes. That caller's code would not be reachable.
The function calls to module own functions need to be stubbed by suffixing the called function by "_stub" to allow to use fake function framework (FFF) to fake that called function (This will require a tool manipulating the source code file, suffixing the relevant function calls)

Testing folder structure
The /tests folder (as already existing) will resemble (except for the /tests folder itself) the repos folder structure and place the tests under dedicated sub-folders.

/_s_test, for "software-test", will be used as sub-folder for the (full) software tests
/_i_test, for "integration-test", will be used as sub-folder for the integration tests

at the top most level, which includes all participating modules

/_m_a_test, for "module-api-test", will be used as sub-folder for the module api tests

It will be under the module names folder
Which is, what we most have today

/_m_u_test, for "module-unit-test", will be used as sub-folder for the module unit tests

It will be under the module names folder

/_m_s_test, for "module-stubbing-test", will be used as sub-folder for the module unit tests

It will be under the module names folder

An example to depict the idea
(the shown existing tests might have been placed in the wrong category/location, creating the shown folder structure will
require a transition phase from the existing structure to the shown)

/tests

/_i_tests
/sched_subsys_xyz
/_s_tests

/queue
/semaphore
/sched

/deadline
/preempt

/

/kernel

/_i_test

/sched_mutex
/sched_queue

/queue

/_m_a_test
/_m_u_test

/semaphore

/_m_a_test
/_m_u_test

/sched

/_m_a_test (former schedule_api test?)
/_m_u_test

Dependencies
Introducing a system for quality to achieve a safety certification will introduce changes in various areas.
It has to be an iterative step by step process in which the direction very likely will have to be corrected
now and then.
Concerns and Unresolved Questions
Is part of this RFC's discussion to adjust the concept.
Alternatives
Is part of this RFC's discussion to adjust the concept.

That's a lot to discuss in a single RFC - how about we split this into several sub RFCs?

@gregshue - regarding the security and safety claims - as already discussed, there are no security claims attached to the safety claims. Both safety and security need a certain level of systematic capability. We aim for the systematic capability of SIL 3 (IEC 61508:2010), that also is enough for security, but does not make specific claims regarding security features, mechanisms, etc.

I repeated the question here to capture the topic within this issue. The question originated from the following statement in Embedded Software Development for Safety-Critical Systems, Second Edition (Chris Hobbs, CRC Press, 2020), p 10,

Safety is interlinked with security - the behavior of an insecure device is effectively unpredictable once a hacker has found a vulnerability.

This may be another subject appropriate for a topic RFC.

Regarding the choice of Doorstop, I think it is not a good one,
for the following reasons.
Doorstop is arguably unmaintained – its creator only sends Python
support patches from time to time. For any advanced patches from other
people, there are long unfinished threads hanging around with no
progress being made.
About "Doorstop uses single yaml files per requirement", this is not
so good granularity, if one considers that everything in the industry
is kept in documents (requirement specifications, test specifications,
verification control document, etc.). StrictDoc intentionally packages
requirements in .sdoc document files because people create and read
the whole documents and requirement sections. Maintaining separate
files without first-class document support is cumbersome. Take the
"level:" field for example
https://github.com/doorstop-dev/doorstop/blob/develop/reqs/REQ002.yml#LL4
– it is so inconvenient to manage it on the single req's YAML file
level.
Merge conflict argument works a little better with Doorstop, but I could
argue that if there is a conflict, this already means that the engineers
working on requirements went out of sync. To get a merge conflict in an
.sdoc file, you really have to work on the same or closely neighboring
requirements or someone has to start moving sections around, which
already indicates that people work without any synchronization.
Additionally, StrictDoc has made and is making a lot of progress recently:
see the GitHub project and compare it with the one for DoorStop.
In particular, StrictDoc has at least an 80%-working bridge to ReqIF export
and import.  I personally used it to successfully import Polarion-generated
requirements in ReqIF format. Doorstop doesn't have ReqIF support.

The following sentence misses one point: "Even if the open source SCA does not have the same quality as a commercial grade solutions, it will be suitable to detect and this way fix 80 - 90% of the violations in a fast development cycle."
It is not just that cppcheck has lots of false negatives, it also has lots of false positives.  The danger is that developers that have not been properly trained on the coding guidelines will not be able to recognize the false positives and will believe they have to to something about them.  In my experience, this always causes a decrease in code quality.

We need a requirements management system keeping with the composability and extensibility of Zephyr.

It needs to support a single "document" (e.g., StRS, SyRS, SyAID, SRS, SAID, etc.) that has requirements spread across multiple repositories in the workspace.
It needs to support "variation management" - tracking moderately different requirement sets for closely related products.

I have attempted to use Doorstop and found it at least unable to link to requirements in other repositories. I didn't look into variation management or aggregating requirements from multiple repositories.
@RobertoBagnara How well does StrictDoc support those needs?

Zephyr is not the first project for which a pre-qualification according to safety related standards is available. We did something similar for the RTEMS real-time operating system. The pre-qualification for RTEMS was performed using software development standards from the European Cooperation for Space Standardization: ECSS-E-ST-40C and ECSS-Q-ST-80C Rev.1. We took also other standards such as IEC 61508,
ISO 26262, DO-178, DO-330, DO-333, and Galileo Software Standard (GSWS) into account. The work started in 2020 and the first results are available on a ESA website: https://rtems-qual.io.esa.int/. The technical note QT-109 contains some background information. The latest documentation of the RTEMS pre-qualification process is available here: https://docs.rtems.org/branches/master/eng/req/index.html. A key observation is that the standards require a high degree of traceability from requirements to source code and tests and vice versa.
Interestingly, the proposal here has some overlaps with what we did. I can highly recommend the EARS syntax for requirements.
We also use Doxygen to make the link from the specification to the source code and vice versa. In a first pass, Doxygen is invoked to produce a tagfile. This tagfile is used to link requirements to source code. Then a file is generated which contains Doxygen comments for source code elements such as functions. The generated comments augment the existing Doxygen documentation and provide links to the specification.
For the requirements (or specification in general) we also started with Doorstop. However, while working with it, it turned out that some customization was required: https://docs.rtems.org/branches/master/eng/req/tooling.html. The RTEMS specification contains not just requirements, so we added a hierarchical type system for specification items: https://docs.rtems.org/branches/master/eng/req/items.html. These types are also specified by specification items: https://github.com/RTEMS/rtems-central/tree/master/spec/spec. It can be easily customized to meet project-specific requirements.
Real-time operating systems such as Zephyr or RTEMS are just a collection of state machines and data structures. For RTEMS the majority of the function specification is done by the specification of state machines. We use a specialized specification item for this: https://docs.rtems.org/branches/master/eng/req/howto.html#action-requirements. It contains the specification along with the corresponding validation test code. It needs a bit of learning to be able to write these files (example: https://github.com/RTEMS/rtems-central/blob/master/spec/c/req/posix-memalign.yml), however, it turned out to be very useful for the functional specification of RTEMS. We reach 100% branch and code coverage for the pre-qualified feature set of RTEMS through validation tests. Since we enumerate all state transitions, this yields more or less MC/DC coverage automatically.
Performance requirements are stated in terms of runtime measurement requests. The measurements are done using the RTEMS Test Framework. This test framework would also work with Zephyr.
We use Python modules for the tooling (BSD-2-Clause license): https://github.com/RTEMS/rtems-central/tree/master/rtemsspec.
Just let me know if you are interested in this work. Maybe it can be generalized so that it can be used also for Zephyr.

@sebhub

Zephyr is not the first project for which a pre-qualification according to safety related standards is available. We did something similar for the RTEMS real-time operating system.

Many thanks for reaching out to us.
We are aware of RTEMS in general, basically due to your evaluation of Doorstop and your decision to implement your own RQM tool for various reasons.
At least I have not looked into your detailed implementation and requirements of RTEMS, but appreciate your input.
The Zephyr OS code is existing and rewriting everything to yield MC/DC coverage will I assume effort-wise not work at least in the first place. It also is not demanded by IEC61508 SIL3, which we aim for.
Let me have a look at all the links you provided.

@RobertoBagnara

It is not just that cppcheck has lots of false negatives, it also has lots of false positives. The danger is that developers that have not been properly trained on the coding guidelines will not be able to recognize the false positives and will believe they have to to something about them. In my experience, this always causes a decrease in code quality.

The final check is the SCA in the CI that matters.
How do you suggest to provide fast SCA checks on a local developer machine with short cycle times with free access to the community?
Or how do you suggest the community shall do SCA runs? Just via CI?

We need a requirements management system keeping with the composability and extensibility of Zephyr.
1. It needs to support a single "document" (e.g., StRS, SyRS, SyAID, SRS, SAID, etc.) that has requirements spread across multiple repositories in the workspace.

2. It needs to support "variation management" - tracking moderately different requirement sets for closely related products.

I have attempted to use Doorstop and found it at least unable to link to requirements in other repositories. I didn't look into variation management or aggregating requirements from multiple repositories.
@RobertoBagnara How well does StrictDoc support those needs?

TL;DR: Currently, StrictDoc does not support this, but it could. In my experience, developers are very keen on receiving feedback and making StrictDoc more useful.  Rather, what is needed is a more detailed specification of how the feature-based requirements can be filtered.
Longer version:
The filtering has to happen based on some criteria.  Specific feature-based requirements are either:

Placed in a specific document, like features/ABC.sdoc
Have a specific field like FEATURE: ABC (makes most sense?)
Tagged with a given tag, such as #ABC
Have a specific requirement UID, like REQ-...-ABC-...
Placed in a specific section.

Once the convention is established, one can think of implementing a query mechanism that takes a given document, filters in/out given requirements and then the resulting document is a needed subset.  We did implement a similar mechanism in a day.
This feature definitely requires a good specification and a good implementation, so that it is not hard-coded to only Zephyr community as a single user.

@RobertoBagnara

It is not just that cppcheck has lots of false negatives, it also has lots of false positives. The danger is that developers that have not been properly trained on the coding guidelines will not be able to recognize the false positives and will believe they have to to something about them. In my experience, this always causes a decrease in code quality.

The final check is the SCA in the CI that matters.

If developers plagues by false positives worsen code quality, the final check will come too late.

How do you suggest to provide fast SCA checks on a local developer machine with short cycle times with free access to the community? Or how do you suggest the community shall do SCA runs? Just via CI?

As far as I know, no free tool has half-decent support of the MISRA coding standards.  Said that, yes, a CI infrastructure allowing developers to analyze their own branches/forks/whatever could do the job.

The filtering has to happen based on some criteria.

In the EARS syntax optional feature requirements are captured with the "Where ," fragment. This condition can be expressed as either a series of Where conditions (implicitly all must be true), or as a single condition with explicit logical "and" conjunctions.

We need a requirements management system keeping with the composability and extensibility of Zephyr.
1. It needs to support a single "document" (e.g., StRS, SyRS, SyAID, SRS, SAID, etc.) that has requirements spread across multiple repositories in the workspace.

The RTEMS specification graph can be built using from any number of directories, for example:

https://github.com/RTEMS/rtems-central/tree/master/spec-spec
https://github.com/RTEMS/rtems-central/tree/master/spec-glossary
https://github.com/RTEMS/rtems-central/tree/master/spec
https://github.com/RTEMS/rtems/tree/master/spec

The tools visit a configurable list of base directories and search recursively for specification items. The items are added to an item cache which associates an UID with exactly one item. The UID relates to the path of the file relative to the base directory. Specification item are stored in YAML or JSON files.

2. It needs to support "variation management" - tracking moderately different requirement sets for closely related products.

For RTEMS we solved this by adding an enabled-by  attribute to each specification item. For example, some requirements are restricted to SMP configurations: https://github.com/RTEMS/rtems-central/blob/master/spec/rtems/scheduler/req/get-processor-smp-only.yml. Other requirements depend on a particular SoC: https://github.com/RTEMS/rtems-central/blob/master/spec/bsp/sparc/leon3/req/errata-gr712rc-05.yml. These enabled-by attributes can be viewed as Where clauses in EARS.

@sebhub Thank you for the pointers. It looks to be very relevant content.

The RTEMS specification graph can be built using any number of directories, ...

From the RTEMS UID definitions it looks like RTEMS requires a (all?) specification(s) to be within a consistent, fixed directory tree. The Zephyr build system learns about the Zephyr module pathnames through the ZEPHYR_MODULES and ZEPHYR_EXTRA_MODULES variables. Any idea about the complexity of inserting a module reference parameter into the UIDs?

It needs to support a single "document" (e.g., StRS, SyRS, SyAID, SRS, SAID,
etc.) that has requirements spread across multiple repositories in the
workspace.

It needs to support "variation management" - tracking moderately different
requirement sets for closely related products.

@RobertoBagnara How well does StrictDoc support those needs?

Hello everyone,
I am the developer of StrictDoc. I became aware of this thread and thought I
could answer some of the questions about StrictDoc directly.
StrictDoc is in active development, and we are always happy to get feedback from
the users and implement all improvements suggestions that match our roadmap.
I started building StrictDoc in 2019 to implement some features that I was
missing in Doorstop and the main use case I had in mind originally was the
production of requirements documentation for spacecraft avionics projects
according to the ECSS standards. The ECSS-E-ST-40C suggests that a documentation
package for software projects consists of documents like:

Software System Specification (SSS)
Software Requirements Specification (SRS)
Interface Control Document (ICD)
Software Design Document (SDD)
...and some other documents, each having a very detailed predefined template.

StrictDoc is based on a Document concept that consists of sections/chapters or
requirements, and sections can consists of child sections or requirements.
Besides including requirements, sections can have blocks of free text. Free text
blocks provide all necessary glue to help a user to understand the requirements.
A document can also contain only sections and free text. Example: An ECSS SW
User Manual document normally does not have requirements but only free text
structured with sections (chapters).
An example of a typical SDoc file
StrictDoc deliberately does not cut the requirements to be one requirement per
one file because, to my original and still existing view, that breaks the idea
of a coherent requirements specification document that can be read and
understood as a whole.
After some time, I also implemented a Python library for ReqIF format, and it
was very easy to map the concept of StrictDoc's Document to ReqIF's
Specification and the concept of StrictDoc's Section to ReqIF's Spec Hierarchy.
The library is not coupled to StrictDoc and at some point I extracted it to be a
standalone building block: https://github.com/strictdoc-project/reqif.

It needs to support a single "document" (e.g., StRS, SyRS, SyAID, SRS, SAID,
etc.) that has requirements spread across multiple repositories in the
workspace.

StrictDoc supports
including Sections from other files.
It should be possible to build a StrictDoc document from a number of
contributing sections stored in separate files. StrictDoc does not support
including single requirements but that should be doable. In that case, I would
like to see a strong rationale/use case for going back to Doorstop's "single
req - single file" approach.
StrictDoc is not connected to a single repository concept in any way. You can
run StrictDoc against a given folder and it will recursively find all *.sdoc
files and collect them into a single documentation tree. Nested folders and
submodules are possible. The same should apply to finding the document fragments
but I admit I haven't tested this multi-repo case with section fragments yet.
@gregshue, could you elaborate a little more on which repositories do you have
in mind and how are you planning to maintain a single document that is spread
across several repositories? I can guess that from these child repositories, you
will not be cherry-picking single requirements to your parent document but
rather you will be including more coherent and self-contained sections with
requirements and free text, is it so?

It needs to support "variation management" - tracking moderately different
requirement sets for closely related products.

Formulated in this way, it is hard to give a general answer if StrictDoc is
capable of doing it but I would say we can certainly talk about this in more
detail: how do these variations look like and how StrictDoc could implement
this. One way of doing it is certainly by using a system of specialized fields
like @RobertoBagnara suggests in
his comment.
A document can be a result of a query applied to the specification tree,
filtering only the needed features.
With regard to this second question, I would also like to add some background
information to what @sebhub provided in
his comment.
@sebhub and I were looking into using Doorstop and were contributing to it
around the same time in 2019-2020. There is a very interesting conslusion by
https://github.com/doorstop-dev/doorstop/issues/471 that can shed light into why
RTEMS qualification initiative went to building the toolchain in the way that it
is implemented now.
Compared to StrictDoc's focus on the single-tree projects where in the end there
is one final product, RTEMS qualification effort had to be accomplished for a
number of selected hardware platforms which assumed high configurability of the
BSP parameters and build settings. The variation management requirement was
there from the very beginning and the qualification engineers selected very
early a subset of RTEMS of what would become a "profile for space applications".
This led to a "bottom-up" approach where the specification items were not
centered around the concept of documents but around the already existing source
files, directories and functional blocks of the RTEMS operating system. The
resulting RTEMS Qualification Data Package is generated from these smaller
specification items, and the final documents look like what ECSS standards
expects them to look like. A part of this toolchain is that the spec item files
(like
this one
from @sebhub's comment) support a number of special fields that emerged
according to the needs of the qualification effort and there is definitely no
code in Doorstop or StrictDoc that would be able to implement any of these out
of the box.
In the end, StrictDoc as a top-down system can produce Documents from smaller
fragments, and the RTEMS toolchain can produce Documents from composable
specification items. What matters for RTEMS is that its toolchain is tailored to
the RTEMS QDP needs very closely. To achieve something similar with StrictDoc,
custom code like that of RTEMS qualification toolchain will have to be written.
Having developed StrictDoc to what it is now and having observed the RTEMS
qualification progress as an outsider (The European space industry is largely an
RTEMS user), I can say that the bottom-up qualification approach of RTEMS is
certainly worth a very close look if Zephyr wants to do a similar qualification
exercise. At the same time, I would be happy to support the initiative from
StrictDoc tool side if the working group decided to include it somewhere in the
Zephyr toolchain.
To conclude, here are a few general bullet points:

I assume that whatever implementation strategy is chosen for Zephyr, the
working group will have to write custom code. If you are interested in doing
something with StrictDoc, keep in mind that some features can be implemented
very quickly (within a couple of evenings) and some can take more time. If I
knew more precisely what exactly is needed, I could answer and give reasonably
accurate estimates about the functionality missing in StrictDoc. Feel free to
introduce a temporary email thread or a GitHub issue to clarify what's needed
in a detailed discussion and then summarize the conclusions here.

I can imagine a hybrid solution where StrictDoc could play a limited role and
be used together with other tools such as RTEMS specs toolchain. In any case,
consider using ReqIF for
converting Zephyr requirements to ReqIF and vice versa. I want to believe that
the tool is becoming quite a capable Python parser/unparser for ReqIF. Once in
a few months, I still receive a ReqIF file with "a yet another edge case" from
the users of DOORS, Polarion, etc but these reports are becoming rare.

I have a basic description of how StrictDoc compares to some other tools
here.
In particular, I didn't see anyone to mention
Sphinx-Needs which is at
least as capable as StrictDoc and is very powerful due to its full
compatibility with the Sphinx plugin ecosystem.

I would like to include several people whose work was either mentioned in this
thread or is related to the topic of text-based requirements management:
@jacebrowning (Doorstop), @danwos (Sphinx-Needs), @anmavrid (FRET).

Thank you for your attention, and I hope that my input adds value to the
discussion.

Hi all,
thanks @stanislaw for mentioning Sphinx-Needs and for the great, detailed answer.
Let me try to introduce Sphinx-Needs, which in my understanding can also be used to solve your documentation requirements for safety and security docs.
First of all, I'm the current maintainer and founder of Sphinx-Needs, which was created ~6 years ago based on the needs of safety-relevant documentation at an Automotive OEM company (following ISO26262).
What is it?
In my understanding, Sphinx-Needs is more of an ALM toolbox for docs-as-code-based documentation, which contains safety-relevant documentation but is also used for architecture docs, test run docs or simple FAQs.
And as each team is following slightly different processes, with different tool and CI setups, Sphinx-Needs is highly customizable to fulfill nearly every dream regarding customization, data validation, and final representation.
Sphinx-Needs is an extension for the documentation generator Sphinx.
It uses and supports natively all Sphinx data formats and features.
So defining a requirement in rst or md syntax, and adding tables, images, math formulas, and own written solutions, is given by default.
This also means Sphinx-Needs objects (like requirements) can be defined everywhere in the documentation: all in one file, one per file, and between other paragraphs. It's up to your own rules how the structuring should look like.
You are also free to decide, what kind of objects you need. Is it a customer requirement, or more an epic with a linked user story?
And all of these decisions can be validated by Python based statements (needs-warnings) or even more complex by pydantic models via Sphinx-Modeling, which is an extension on top of Sphinx-Needs.
Another one is Sphinx-Test-Report to get test results into your docs and automatically link them with test cases.
The API and data model are completely open, so you are free to inject our own customizations.
With [needservices]}(https://sphinx-needs.readthedocs.io/en/latest/services/index.html) you can integrate external systems, for instance, to fetch GitHub issues and link them to requirements/specs.
Or you use the Variant Handling to maintain different options for different targets/customers for the same requirement.
And dynamic functions allows the automatic calculation of option values, like status or safety statements based on the linked elements.
Where is it used?
Sphinx-Needs is used in some bigger commercial Automotive ECU projects for SW development.
The biggest project I'm aware of is ~2.000 SW developers with a single documentation containing up to 130.000 Sphinx-Needs objects in ~22.000 rst/md files (Disclaimer: I and my team are providing commercial support for this).
Some projects are trying to introduce Sphinx-Needs for all steps in the V-Model:

Starting from below (SW dev part) and injecting more and more process steps from the left and right side (SW Arch, SW Testing, System Arch, ...). It's an ongoing process to make Sphinx-Needs available to support also the special requirements on the upper process steps.
What else
There is an online presentation available, which shows most features on some slides:
Documentation without frustration presentation.
Sorry, that I wasn't able to get into your special project requirements and provide already some solutions.
But in my understanding, StrictDoc and Sphinx-Needs are both capable to deal with your requirements.
And if not,  we hopefully have found some new contributors ;)

@stanislaw Good questions.  I don't have a solution in mind, but I have a pretty clear idea of the problem.
Let's look at this from the perspective of a manufacturer/product developer:

The IoT device fits within an overally solution that interfaces with natural people (HMI) and external automated agents (MMI, e.g., the electrical utility load-shed client). The natural people, the regulatory agencies, and the business developing the end product are all examples of Stakeholders. Stakeholders have needs that may be met by the system and impose some level of constraints on how the system meets those needs.
The System consists of everything that is provided that meets those needs.  For an IoT device, the system includes the remote service(s) associated with the device, along with all related cloud/desktop/mobile-based applications.
Devices, mobile apps, and cloud services are examples of System Elements. Each has specific responsibilities and implements a subset of the System-external and/or System-Internal interfaces.
A IoT Device (System Element) is composed of at least HW (e.g., mechanics, electronics, immutable boot code), configuration data (e.g., unit serial number), and typically has updatable embedded SW. Each of these are Sub-Elements.
A Device FW image may contain complete binary images for subordinate devices (e.g., radios, GPUs, asymmetric CPU cores, peer MPUs). These images may be proprietary, commercial, or OSS.
Note that product developers may want a single platform that supports configuring and building multiple Sub-Elements (bootloader, app) within each device along with supporting a set of distinctly different devices in the System (e.g., sensor vs actuator, initiator vs responder, scanner vs printer vs. MFP).
Zephyr is primed to be a reasonable framework for that platform. Part of the Zephyr solution is to support bootloader/app internal components and interfaces to be distributed across peer repositories. These repositories must coexist peacefully when viewed in aggregate in the same code tree, after filtering based on project configuration settings.
Note that the end user gets to choose the topology of the aggregate code tree. They also get to create whatever document tree they want, and which portions to use Zephyr RM tool for. They should have the option to develop and trace their entire system in detail with the Zephyr RM tool.

So ...

could you elaborate a little more on which repositories do you have in mind

I'm thinking of any/all of them. We need a requirements solution that scales to:

up to all levels of requirements decomposition;
across all potential repository distributions (e.g., zephyrproject-rtos, downstream OSS, commercial, private);
down to being in the middle of a larger solution.

how are you planning to maintain a single document that is spread across several repositories?

I don't have a proposal. I'm looking for suggestions too.

I can guess that from these child repositories

We cannot restrict these to being child repositories.

you will not be cherry-picking single requirements to your parent document but rather you will be including more coherent and self-contained sections with requirements and free text, is it so?

The solution needs to scale across both usages.

In the end, StrictDoc as a top-down system can produce Documents from smaller fragments,  and the RTEMS toolchain can produce Documents from composable specification items.

Fundamentally, a Zephyr application is a selection/configuration of composable subsystems & drivers (components) controlled via Kconfig & Devicetree. It is natural to model the RM for interfaces & components in the same way.

@gregshue thanks for the detailed problem description.
Sphinx-Needs together with Sphinx-Collections  have some concepts to deal with this kind of decentralized requirement distribution.
First of all, Sphinx-Needs has an exchange format, called needs.json.
This can be used to import objects from other projects and have a representation of them in the new project.
Or the objects from needs.json are just referenced, so that you can link objects to them, but the external objects have no representation in the new project. Links in this kind of referencing are later real HTML-links to the correct location of the referenced need in the external project.
A simple workflow looks like:

The above mechanism is good to share objects between Sphinx projects.
But it does not solve generic integration problems, where you have for instance standalone projects for components and need an integrated, overall system documentation, which shall include all the content (text, images, tables, ...) from the component projects.
For this, Sphinx-Collections was created, which integrates files from different projects into an overall project. Sources can be repositories, local folders, functions, and data provided by self-written functions.
Sphinx tags can be used to easily switch between different integration configurations.
A build call could look like: sphinx-build -t system_view -b html input/ output/
Some further links:

need.json details
needimport - filter and import objects from needs.json
external_needs config - how to allow objects to link to external reqs and co.
permalinks - track/link objects, even if they have moved

I hope this already helps to get a feeling about some mechanisms of object-handling between different docs-as-code projects.

@sebhub Thank you for the pointers. It looks to be very relevant content.

The RTEMS specification graph can be built using any number of directories, ...

From the RTEMS UID definitions it looks like RTEMS requires a (all?) specification(s) to be within a consistent, fixed directory tree. The Zephyr build system learns about the Zephyr module pathnames through the ZEPHYR_MODULES and ZEPHYR_EXTRA_MODULES variables. Any idea about the complexity of inserting a module reference parameter into the UIDs?

Each specification item (for example a requirement) needs a unique identifier (UID). For the RTEMS specification, the UID is derived from the file system path to the file which stores the specification item (in YAML or JSON format). The base directory and the file format extension is removed from the path to form the UID. For example spec/req/root.yml is loaded using the base directory spec which results in the UID /req/root. The specification items are loaded from the file system by recursively searching for files of a given format in a configurable list of base directories. The RTEMS specification and tooling is located in the rtems-central repository and the RTEMS sources added through a Git submodule.
The links in the specification item are unidirectional and have a role. This can be used to make the specification composable. For example, the requirement refinement relationship is stored in the specializing item and links to the more general requirement. So, the higher level parts of the specification do not need to know the specializations. You could for example have a higher level requirement like:
The system shall be extendable by modules.

Then in a specific module you may have this:
The network stack module shall extend the system to provide TCP/IP networking services.

When you build the documents for a given configuration, the links are traversed to compose documents specific to this configuration.
Another feature which helps to make the specification modular are proxy items. They can be used by higher level specification parts to refer to items which may be implemented in an architecture-specific or device-specific way. For example, the Gaisler IP library is available for SPARC and RISC-V systems. The generic part uses a proxy item /dev/grlib/if/store-32 to refer to some method which stores a 32-bit value to a register. This method is provided by an architecture-specific implementation /bsp/sparc/if/grlib-load-32. For a particular configuration, the proxy items are resolved to a specific item.
Another feature is the hierarchical glossary of terms. A glossary of terms is quite important for larger projects. However, some terms may be context-specific. An example are state machines specific to a certain functionality such as timers. For this you can create a hierarchy of glossaries:

https://github.com/RTEMS/rtems-central/blob/master/spec/glossary-general.yml

https://github.com/RTEMS/rtems-central/blob/master/spec/rtems/timer/glossary/group.yml

The terms defined by the Timer Manager glossary are used in the state machine specifications, for example /rtems/timer/req/reset.

@stanislaw thanks for your good overview. Yes, the RTEMS specification approach is very bottom up. It was a reverse engineering activity and not the development of a new product. For general requirements engineering I would not store individual requirements in individual files. This is just way to cumbersome and does not help you to discover a good set of high level requirements, user stories, etc. I think these two approaches could be easily combined. In the end, the specification is just a directed graph where each node is identified by an UID. I also belief that each projects needs a bit of specific tooling to be productive. It is important to have simple and open data formats which can be effectively used with Git (like reST, StrictDoc, YAML).

IMHO, this issue is a big leap into the right direction for allowing product creators in safety-critical domains adopt zephyr for their use-cases. That said I felt a bit lost in the discussions as it seems that ideas at least around process and actual tools are mixed up. Not saying they do not relate to one another, obviously they do. Still, disentangling the threads might help to focus the discussion.

process complexity Regarding the process I am concerned that an unmanageable thing might be created which I have seen to happen too often in product developments. And soon enough maintenance cost will dominate everything and hinder innovation. From my analysis that's one of the main reasons organisations are stuck on using decades-old technologie stacks. Manually (that's where the cost for many companies sits) maintaining a set of requirements in a seperate repo (even if pulled in through west) means submitting PRs against two repos in case something needs to change and is governed by a safety-clause. Given the frequency of changes that go into the kernel folder alone (and which would clearly be considered governed by a safety-clause) I am concerned about the general acceptance.
combinatorical complexity Another thing that I have thought about for some time now is how to deal with the 'superset' character of Zephyr. One of the hallmarks of Zephyr is that every aspect of it can be configued at compile time. The downside of this is that it creates an infinite (infinite for all practial purposes) configuration space. In the end downstream projects have to expect certain safety claims to hold as they have to build their safety case based on these assumptions and claiming validity in all end every conceivable configuration I consider not feasible. That begs the question which options to include and which to freeze if you wanted to use Zephyr in a safety-critical context, i.e. you have to use the dumb scheduler, not more than 16 threads, at least 128K stack_size etc.).
smart approach to design specification Lastly, even then a huge amount of test cases need to be generated (and kept up-to-date) and looking into methods how test cases can be (re-)generated  from (semi-)formal specifications may hold some additional value in that they keep the maintenance cost at an acceptable level. Examples include NASA's FRET or Lamport's TLA+

To sum up before going to call for some tools and rush into action some thought experiments how the process will play out over some time may help falling into avoidable pitfalls

some thought experiments how the process will play out over some time may help

Agreed, which is why I the first thing I contributed to #57703 are some needs from downstream users, particularly from product manufacturers. Please feel free to add others that you are aware of.

@sebhub I eventually managed to take a deeper look into what you guys are doing at RTEMS and it's really impressing.
Personally, I think it would make a lot of sense if FOSS projects targeting the safety-critical domain would eventually align on a 'standard' approach towards requirements engineering and how to provide a safety case for the FOSS under consideration. Reason being that many notified bodies (those agencies eventually responsible to asses the safety case) will probably have had none to very little exposure to FOSS yet. And the less familiar things look the more scrutiny auditors will put forth and hence whatever could be presented as a 'standard' way may help (even more so if someone would actually testify that the process is sound and apt for FOSS projects)
The RTEMS process and the content you have already produced using it could well serve as a reference for other projects to follow.
I wonder how much experience you have had with the process when it comes to adding updates to design elements that had been captured previously


Full Rollup using CMDT results in "SObject row was retrieved via SOQL without querying the requested field"

I'm trying out the full rollup using CMDT with different where clauses. The batch is failing because of the dreaded error SObject row was retrieved via SOQL without querying the requested field. I'm not sure if it matters, but the field I'm filtering on is a formula field.
I have 4 CMDT records for one calc item. Each one has a different where clause.
Here's my stack trace:
FATAL_ERROR System.SObjectException: SObject row was retrieved via SOQL without querying the requested field: [fieldname]
FATAL_ERROR Class.RollupEvaluator.WhereFieldCondition.getFieldValue: line 481, column 1
FATAL_ERROR Class.RollupEvaluator.WhereFieldCondition.equals: line 415, column 1
FATAL_ERROR Class.RollupEvaluator.AndConditionalGrouping.conditionallyEquals: line 374, column 1
FATAL_ERROR Class.RollupEvaluator.ConditionalGrouping.equals: line 336, column 1
FATAL_ERROR Class.RollupEvaluator.WhereFieldEvaluator.matches: line 134, column 1
FATAL_ERROR Class.Rollup.filter: line 2140, column 1
FATAL_ERROR Class.Rollup.: line 221, column 1
FATAL_ERROR Class.Rollup.RollupAsyncProcessor.: line 318, column 1
FATAL_ERROR Class.Rollup.loadRollups: line 1962, column 1
FATAL_ERROR Class.Rollup.getRollup: line 1844, column 1
FATAL_ERROR Class.Rollup.getDelegatedRollup: line 2037, column 1
FATAL_ERROR Class.RollupFullBatchRecalculator.execute: line 35, column 1

Also, I wish I could spend more time and actually provide a pull request for some of these issues. Sorry!

@dhoechst not to worry, you’ve been super helpful. There will always be a time for contributions! This error you’re reporting is present on v1.2.5, yes? I have some tests that are specific to formula fields working but the “where clauses” are something of a special beast. Are each of your calc item where clauses simply filtering on that one field, or are they more complex? Also - do you have any way to tell (this may be impossible so apologies in advance) if the error is occurring on the SECOND Rollup__mdt?
One potential way to validate this would be to try with only a single CMDT record but if that’s a pain to go and delete what you already have, no worries.
Also of note is the starting place for the error — the full batch recalculator. Were you trying to calculate the initial values for the parent records via the Rollup app when you got this error, or was it from a record being inserted/updated etc?
Thank you!

Well, I had managed to get v1.2.4 installed before you made the fix, so the line numbers in the stack trace correspond to that release. I'm still having the error in v1.2.5.
Here's a screen shot of the error with the new line numbers:

The where clause is simple: Has_SOX_Impact__c = TRUE. Just for grins, I was able to replicate the logic of the formula field in the where clause instead and now it is telling me a different field wasn't selected. As for which CMDT record it is failing, I'm not sure.

Oh, and to answer a question I missed. I'm trying to do a full rollup via the Rollup app.

Ok got it - that makes more sense. Should be able to patch this one up tomorrow, hopefully

@dhoechst you should be all set on this one!


Class name cannot be Object

Please fill in the entire template below.
TypeScript Version:  3.1
Search Terms:
class name cannot be Object
Expected behavior:
All browsers DO NOT generate errors for this line, and it is valid:
var c = class Object {}
Actual behavior:
Compiler error, preventing my large project from compiling with no way to turn it off.  This was working perfectly fine for almost 2 years until I upgraded to v3.1.
Playground Link:
https://www.typescriptlang.org/play/index.html#src=namespace Test {
    export class Object {
        x%3A number%3B
    }
}

We forbid this because we use the global Object in our helpers and expect it to be available, and so throw when people attempt to overwrite it.

See https://github.com/Microsoft/TypeScript/issues/17494 for details

Like everything else, that should be an option.  There are other things that TS emits as well, and you already have options to allow others to implement their own. My object class actually connects back to the original one so everything was working; now it's not.  You should not force this on nested namespaces. In the root perhaps, yes, but not many levels deep, otherwise you are saying that x.y.Object is not allowed as a class, but it is valid JS!!! (in every browser)

The error is there to mark that the emit is broken. Changing the emit to handle a used Object name was previously quite difficult (though now wouldn't be too bad in our new pipeline, maybe - would still be extensive, though). The error is there in lieu of changing the emit to be much more complicated because we've had few requests to support it over time (and more about how previous to TS 2.2 we silently broke code).


beta release tasks

delir-core

[x] shrink Project objects to plain object(no methods)
[x] add project modify methods into ProjectHelper
[ ] Frame buffer layer
[x] Multi audio mixing
[x] Layer effect plugin

standard plugins

[ ] html5-video-layer replace to avcodec-video-layer

Delir (frontend)

[ ] Plugin parameter controring on Preview canvas

Delir website

[ ] Write plugin develpment document

今後のスケジュールはこちら: https://trello.com/b/2WiN64z2/delir


Cannot send "People with existing access" share link to external guest user

When sharing a document by sending a link, if I set the type to "People with existing access" before picking an external guest user, the Send Link dialogue claims that the recipient does not have permissions to access the item:

I have confirmed that the external guest user in question has appropriate permissions. This error does not apply to internal users.
For reference, I'm attaching configs for my identity token issuer, SAML attributes and claims, and AzureCP claims mapping.
Identity token issuer:

SAML attributes and claims (Azure portal):

AzureCP

Any assistance with this would be greatly appreciated, as it is within the scope of my project to have Sharepoint Server function as close to Sharepoint Online as possible.
Thank you!

@hat-cat I did the exact same test as you described and it fixed the error message, but I didn't validate the user received the link (I tested with a fake email). Did you double check that the mail was synchronized the the local user profile in the current site collection?
Yes it's a good idea to properly sync profile properties but the test was anyway relevant.

@hat-cat any news on this?

Closing this issue due to lack of response, feel free to reopen it if needed


NLog Custom Target Guidance

Is there any guidance for how an NLog Custom target should log exceptions that occur in it's processing?
I can see an obvious infinite loop potential right off the bat for instance.

There is no guide, but those exceptions should be written to the InternalLogger, e.g.  InternalLogger.Log(exception, level, "Error has been raised.");.
In NLog we use helpers like exception.MustBeRethrown(), but those are internal.

I took a look at the targets on github and they don't do any error catching.  The internalLogger is great to know about though as I do need to do some logging.
One specific case that would be good to know about is how the normal targets handle errors which would result in a message not being logged.

One specific case that would be good to know about is how the normal targets handle errors which would result in a message not being logged.

Well that's handled in the base classes, like Target, or LayoutRenderer

Is this still an issue?


InfoWindow support

angular2-google-maps should support Info Windows:
https://developers.google.com/maps/documentation/javascript/infowindows
https://developers.google.com/maps/documentation/javascript/3.exp/reference#InfoWindow
As a standalone component:
<sebm-google-map [latitude]="lat" [longitude]="lng">

 <sebm-google-map-info-window [latitude]="lat" [longitude]="lng" (closeClick)="windowClosed()"
    [disableAutoPan]="true" [maxWidth]="120">

My Content

  </sebm-google-map-info-window>

</sebm-google-map>

And inside a marker:
<sebm-google-map [latitude]="lat" [longitude]="lng">

  <sebm-google-map-marker  [latitude]="lat" [longitude]="lng">
    <sebm-google-map-info-window>
      My Content

      can be HTML
    </sebm-google-map-info-window>
  </sebm-google-map-marker>

</sebm-google-map>

Component methods:
<sebm-google-map [latitude]="lat" [longitude]="lng">

  <sebm-google-map-info-window  [latitude]="lat" [longitude]="lng" #window>
    My Content
  </sebm-google-map-info-window>

  <button (click)="window.open()">Open InfoWindow</button>
  <button (click)="window.close()">Close InfoWindow</button>
  <button (click)="window.toggle()">Toggle InfoWindow</button>

</sebm-google-map>

+1

Hello friend I am using your Google Maps component and I love it , I wonder when it will be released the version 0.9.0 which contains InfoWindows ?


istioctl wait has an inconsistent usage message

the for flag is called for-distribution, and the resource specification should be updated...

fixed in the referenced pr


Merge branch-PPP-Marcus with Master

Codecov Report
Base: 72.15% // Head: 72.15% // No change to project coverage :thumbsup:

Coverage data is based on head (7536121) compared to base (0be3d6e).
Patch has no changes to coverable lines.

Additional details and impacted files
@@            Coverage Diff            @@
##             master      #16   +/-   ##
=========================================
  Coverage     72.15%   72.15%
  Complexity      399      399
=========================================
  Files            70       70
  Lines          1232     1232
  Branches        125      125
=========================================
  Hits            889      889
  Misses          311      311
  Partials         32       32

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


Fix last date on registration module

Coverage remained the same at 72.17% when pulling 1fa65ef9d4b7901bc9eb6c0813cf8dc98a02ab1e on edrisse:fix-last-date-on-registration-module into be86574f4fd31fc34decf5b72347732068ad0556 on esaude:master.


KAFKA-15981: update Group size only when groups size changes

Currently, we increment generic group metrics whenever we create a new Group object when we load a partition. This is incorrect as the partition may contain several records for the same group if in the active segment or if the segment has not yet been compacted.
The same applies to removing groups; we can possibly have multiple group tombstone records. Instead, only increment the metric if we created a new group and only decrement the metric if the group exists.
Committer Checklist (excluded from commit message)

[ ] Verify design and implementation
[ ] Verify test coverage and CI build status
[ ] Verify documentation (including upgrade notes)

The previous build had the error from test testNoGroupEpochBumpWhenStaticMemberTemporarilyLeaves() – org.apache.kafka.coordinator.group.GroupMetadataManagerTest from jdk 8
expected: <[Record(key=ApiMessageAndVersion(ConsumerGroupCurrentMemberAssignmentKey(groupId='fooup', memberId='_hAb4Z1NSHG2Xp42s7KO9Q') at version 8), value=ApiMessageAndVersion(ConsumerGroupCurrentMemberAssignmentValue(memberEpoch=-2, previousMemberEpoch=9, targetMemberEpoch=10, assignedPartitions=[TopicPartitions(topicId=YX9w07JjSp2Yw1ra6Bfkaw, partitions=[2]), TopicPartitions(topicId=brQVJtH8RsyF4BayziKeBQ, partitions=[3, 4, 5])], partitionsPendingRevocation=[], partitionsPendingAssignment=[], error=0, metadataVersion=0, metadataBytes=[]) at version 0))]> but was: <[Record(key=ApiMessageAndVersion(ConsumerGroupCurrentMemberAssignmentKey(groupId='fooup', memberId='_hAb4Z1NSHG2Xp42s7KO9Q') at version 8), value=ApiMessageAndVersion(ConsumerGroupCurrentMemberAssignmentValue(memberEpoch=-2, previousMemberEpoch=9, targetMemberEpoch=10, assignedPartitions=[TopicPartitions(topicId=brQVJtH8RsyF4BayziKeBQ, partitions=[3, 4, 5]), TopicPartitions(topicId=YX9w07JjSp2Yw1ra6Bfkaw, partitions=[2])], partitionsPendingRevocation=[], partitionsPendingAssignment=[], error=0, metadataVersion=0, metadataBytes=[]) at version 0))]>

Where the diff is the ordering between the two TopicPartitions in assignedPartitions field. I don't think my code touched this. Was this already flaky?

@jeffkbkim Yeah, I have noticed that failing test too. It is not related to your PR.

All the failed tests are not related. Merging to trunk and 3.7.


Wrong Lucene version in build.gradle

This looks wrong, the version is outrdated. If the Lucene analyzers module is not used at all, we should remove it: https://github.com/Tony-X/search-benchmark-game/blob/4402d42c906830e85d8d79a30ae776f204ade770/engines/lucene-9.5.0/build.gradle#LL17C4-L17C4

Hi @uschindler, thanks for taking a look a this.
I used the LengthFilter to remove excessively long tokens. So the analyzer module is needed.

This looks wrong, the version is outrdated.

Indeed the version is one minor release behind, now we have 9.6.0. We should upgrade. Do you believe this will lead to wrong performance conclusions? I did not spot major query performance changes by a quick glance of the changelog

Lucene is now up to 9.7.0, and Tantivy is 0.20.1 -- let's upgrade both?


IndexError: list index out of range

i met IndexError: list index out of range when trainning!
i delete the code related to distributed trainning and i met this problem when run the command
python train.py --dataset_path "./dataset/deepfashion" --batch_size 8 --exp_name "pidm_deepfashion"

when i print the path_names where the index error happen, it show this
print(path_names)
['fashion\WOMEN', 'Tees_Tanks', 'id_00003493', '04_1_front.png']
['fashion\WOMEN', 'Dresses', 'id_00003796', '01_1_front.png']
['fashion\WOMEN', 'Rompers_Jumpsuits', 'id_00001990', '02_4_full.png']
['fashion\WOMEN', 'Dresses', 'id_00003147', '01_7_additional.png']
['fashion\WOMEN', 'Sweaters', 'id_00007358', '04_1_front.png']
['fashion\WOMEN', 'Shorts', 'id_00001827', '05_1_front.png']
['fashion\WOMEN', 'Dresses', 'id_00002849', '01_3_back.png']
['fashion\WOMEN', 'Tees_Tanks', 'id_00001925', '21_3_back.png']
['fashion\MEN', 'Tees_Tanks', 'id_00002070', '01_2_side.png']
the path_names don't have index of 4.
What's the problem, the code or the dataset?
i download dataset as the tutorial and nothing error happened when processing data.
so What's the problem?


Image versioning  not work correctly

Versions
Versions / Env:
angular-cli: 6.0.8
node: 10.13.0
os: Window x64
Repro steps
The content of the src/ directory is as below:
src/
├── assets
│   ├── images
│   │    ├── assistance.png
│   │    ├── checked.png
│   │    ├── avatar.png
│   │    ├──table-desktop-blue.png
│   ├── fonts
│   │    ├──calibri.ttf
│   │    ├──cards21.ttf
├── app/
│   │    ├── home/
│   │    │    ├── home.module.ts
│   │    │    ├── rooms/
│   │    │    │    ├── rooms.component.html
│   │    │    │    ├── rooms.component.css
│   │    │    │    ├── rooms.component.ts
├── environments
├── index.html
├── styles.css
├── main.ts
├── favicon.ico
Build a production version. # ng build --prod --base-href=/myProject/
The content of the dist/ directory is as below: Notice that only two images hashed not all
dist/
├── assets
│   ├── images
│   │    ├── assistance.png
│   │    ├── checked.png
│   │    ├── avatar.png
│   │    ├──table-desktop-blue.png
│   ├── fonts
│   │    ├──calibri.ttf
│   │    ├──cards21.ttf
├── cards21.a3ca179e19afd9ff75c3.ttf
├── avatar.7b1f0eaea44dcf3ebaa5.png
├── table-desktop-blue.45bbb739b0ceafc92916.png
├── index.html
├── 1.9d5dbd91ee597bcea89c.js
├── common.8c9eaa448b01bdf489e1.js
├── main.d0df4fbb3790e577ce00.js
├── polyfills.1f682948bdc523f3cb0b.js
├── runtime.fb52e8a1c273f473121b.js
├── styles.8859be51ae08103a3fa6.css
├── favicon.ico
Noticeable Points:

Only two images & one font are hashed not all.
Images & fonts still appear in assets folder while these are hashed

I used all images in css file with background property

Hi, from the output it looks like you might be copying those files.
Copied assets don’t get fingerprinted. Only assets that are referenced directly in the sheetsheets get fingerprinted because otherwise the filename name will be different from the filename referenced and will cause a 404 error.

@alan-agius4 angular cli change file name inside css file too when i make build. image file name same as hashed file.

Can you please share you angular.json and a snippet of the code where you are using assistance.png?

angular.json
{
"$schema"<EMAIL_ADDRESS>"version": 1,
"newProjectRoot": "projects",
"projects": {
"pokerCoach": {
"root": "",
"sourceRoot": "src",
"projectType": "application",
"prefix": "",
"schematics": {},
"architect": {
"build": {
"builder": "@angular-devkit/build-angular:browser",
"options": {
"outputPath": "dist/pokerCoach",
"index": "src/index.html",
"main": "src/main.ts",
"polyfills": "src/polyfills.ts",
"tsConfig": "src/tsconfig.app.json",
"assets": [
"src/favicon.ico",
"src/assets"
],
"styles": [
"src/styles.css"
],
"scripts": []
},
"configurations": {
"production": {
"fileReplacements": [
{
"replace": "src/environments/environment.ts",
"with": "src/environments/environment.prod.ts"
}
],
"optimization": true,
"outputHashing": "all",
"sourceMap": false,
"extractCss": true,
"namedChunks": false,
"aot": true,
"extractLicenses": true,
"vendorChunk": false,
"buildOptimizer": true
}
}
},
"serve": {
"builder": "@angular-devkit/build-angular:dev-server",
"options": {
"host": "<IP_ADDRESS>",
"port": 4200,
"browserTarget": "pokerCoach:build"
},
"configurations": {
"production": {
"browserTarget": "pokerCoach:build:production"
}
}
},
"extract-i18n": {
"builder": "@angular-devkit/build-angular:extract-i18n",
"options": {
"browserTarget": "pokerCoach:build"
}
},
"test": {
"builder": "@angular-devkit/build-angular:karma",
"options": {
"main": "src/test.ts",
"polyfills": "src/polyfills.ts",
"tsConfig": "src/tsconfig.spec.json",
"karmaConfig": "src/karma.conf.js",
"styles": [
"styles.css"
],
"scripts": [],
"assets": [
"src/favicon.ico",
"src/assets"
]
}
},
"lint": {
"builder": "@angular-devkit/build-angular:tslint",
"options": {
"tsConfig": [
"src/tsconfig.app.json",
"src/tsconfig.spec.json"
],
"exclude": [
"/node_modules/"
]
}
}
}
},
"pokerCoach-e2e": {
"root": "e2e/",
"projectType": "application",
"architect": {
"e2e": {
"builder": "@angular-devkit/build-angular:protractor",
"options": {
"protractorConfig": "e2e/protractor.conf.js",
"devServerTarget": "pokerCoach:serve"
}
},
"lint": {
"builder": "@angular-devkit/build-angular:tslint",
"options": {
"tsConfig": "e2e/tsconfig.e2e.json",
"exclude": [
"/node_modules/"
]
}
}
}
}
},
"defaultProject": "pokerCoach"
}
avatar.png
.seat-user-avataar{     border-radius: 50%;     width: 33%;     height: 100%;     background: url("../../../assets/images/avatar.png") no-repeat;     background-size: cover;     position: absolute;     left: 0;     top: 0;     z-index: 1; }

So taking a look at your angular.json you are copying the following;
  "assets": [
      "src/favicon.ico",
      "src/assets"
 ],

Meaning copy these files as they are to the dist folder. These will not get finger printed. Because the build pipeline is unaware where they are being used.
In your stylesheet you are referencing avatar.png with the relative path. Meaning that this will be processed by webpack, get fingerprinted/hashed and the path and your stylesheet gets updated with the correct path and the file emitted will be something like avatar.7b1f0eaea44dcf3ebaa5.png which is correct.
Having the above configuration and usages will result in outputting 2 files named avatar.png one of them being finger printed and the other one not.
Unless you are using your assets in template files or somewhere outside of the CSS pipeline you should not be adding your files to the assets fields, as it will result in files being outputted which are not used.
See https://github.com/angular/angular-cli/wiki/stories-asset-configuration

thanks @alan-agius4, you are right stylesheet are referencing avatar.png with the relative path, but I used relative path for all images, so why only two images are hashed?
@alan-agius4 I work also with another project which also used relative path but not any single image hashed.

@alan-agius4  All images are used in css files.

This seems like a bug but we'll need to look at a reproduction to find and fix the problem. Can you setup a minimal repro please?
You can read here why this is needed. A good way to make a minimal repro is to create a new app via ng new repro-app and adding the minimum possible code to show the problem. Then you can push this repository to github and link it here.
This might be related to your directory structure so its really important to get an accurate repro to diagnose this.

@alan-agius4 please find Repo link below:
Image Version Issue Angular 6
@alan-agius4 Can you share alternate way if we need image versioning?

I had a look at your reproduction and there are two things

checked.png is under 10kb which in CLI version 6 get embedded in the CSS. Using DataUri. This doesn't apply for Angular CLI version 7.
You are placing your css assets under the assets which, these gets copied without fingerprinted, since you have the following in your angular.json

"assets": [
   ...
  "src/assets"
],

If you are using your assets directly in CSS, you don't need to add this in your angular.json. Note that in your reproduction these files are not being used.

why this closed?
So are you saying only image under 10k will be hashed? and will be put into the dist folder?

I would like cli to hash the image, but is there any way to change the output path?  like put them into the image folder under asset??


spinner rounding is failing badly

I tried to enter 99534 into spinner to attach to a process, and it converted it to 99533.99 which rounded down to ..32
and I'm seeing line 1192.9 in my debug list

This is the culprit.  reverting to horrible-but-functional sprintf-based version that works reliably.   there was a reason I didn't go this route before.
sp.Value = mat32.Truncate(sp.Value, sp.Prec)

I added a todo about making mat32.Truncate faster, but it probably isn't that much of a problem and thus it doesn't seem worth doing now.


feat: Add support for OLLAMA model provider and update .gitignore

Added an OLLAMA option, but since the docker container isn't in host, the ip address or hostname of the host running ollama must be specified to work.

nice one thank you.

how  to use ollama anyway. im a bit fuzzy


MSSQL 2016 SERVER MANAGEMENT STUDIO INSTALL IN HYPER-V WINDOWS SERVER 2016 CORE FAILS GETTING ERROR [0CBC:0470][2016-03-22T13:33:28]e000: MainViewModel.OnBundleAction: Bundle action failed

MSSQL 2016 SERVER MANAGEMENT STUDIO INSTALL IN HYPER-V WINDOWS SERVER 2016 CORE FAILS GETTING ERROR [0CBC:0470][2016-03-22T13:33:28]e000: MainViewModel.OnBundleAction: Bundle action failed

Successfully installed MSMQ from Powershell command then started installing MSQL
C:\install\SQLServer2016-x64-ENU\Tools>SSMS-FULL-SETUP.EXE

:::::::::::
:::::::::::
[0CBC:0470][2016-03-22T13:33:28]i319: Applied rollback package: SqlSupport_x86, result: 0x0, restart: None
[0FC0:0A40][2016-03-22T13:33:28]i329: Removed package dependency provider: {F6FDABD9-62CE-48D4-A10B-C22DE64A8CF3}, package: SqlSupport_x86
[0FC0:0A40][2016-03-22T13:33:28]i351: Removing cached package: SqlSupport_x86, from path: C:\ProgramData\Package Cache{F6FDABD9-62CE-48D4-A10B-C22DE64A8CF3}v13.0.1100.213
[0FC0:0A40][2016-03-22T13:33:28]i329: Removed package dependency provider: {5C7BA9C8-7A8B-4BAB-AEBB-17586F5C1546}, package: SqlSupport_x64
[0FC0:0A40][2016-03-22T13:33:28]i351: Removing cached package: DotNet46, from path: C:\ProgramData\Package Cache\83D048D171FF44A3CAD9B422137656F585295866
[0CBC:0470][2016-03-22T13:33:28]i000: MainViewModel.OnPackageActionProgress: Percent completed: 100, Overall progress: 100
[0FC0:0A40][2016-03-22T13:33:28]i330: Removed bundle dependency provider: {32b0803e-f6ab-4f8b-a06a-934142beba0f}
[0FC0:0A40][2016-03-22T13:33:28]i352: Removing cached bundle: {32b0803e-f6ab-4f8b-a06a-934142beba0f}, from path: C:\ProgramData\Package Cache{32b0803e-f6ab-4f8b-a06a-934142beba0f}
[0CBC:0470][2016-03-22T13:33:28]e000: MainViewModel.OnBundleAction: Bundle action failed
[0CBC:0470][2016-03-22T13:33:28]i399: Apply complete, result: 0x800713ec, restart: None, ba requested restart:  No
[0CBC:0FB0][2016-03-22T13:47:56]i000: MainViewModel.OpenUrl: Opening url: C:\Users\ADMINI~1\AppData\Local\Temp\SsmsSetup\SSMS-Full-Setup_20160322131627.log

Can you share a Dockerfile that can reproduce the problem?

Nevermind. I found the solution. It was because I was trying to install mssql2016 on server nano instead of full or core with required enabled features.
Thanks anyways.
Sent from my Samsung Galaxy smartphone.
-------- Original message --------From: Patrick Lang<EMAIL_ADDRESS>Date: 2016-05-03  7:24 PM  (GMT-05:00) To: Microsoft/Virtualization-Documentation<EMAIL_ADDRESS>Cc: mohajibrahim<EMAIL_ADDRESS>Author<EMAIL_ADDRESS>Subject: Re: [Microsoft/Virtualization-Documentation] MSSQL 2016 SERVER MANAGEMENT STUDIO INSTALL IN HYPER-V WINDOWS SERVER 2016 CORE FAILS GETTING ERROR [0CBC:0470][2016-03-22T13:33:28]e000: MainViewModel.OnBundleAction: Bundle action failed (#178)
Can you share a Dockerfile that can reproduce the problem?
—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub

Thanks for following up. I am going to close the issue, feel free to re-engage if needed.


sed command incorrect

This sed command
sed -i -e "s/<imgBuilderId>/$imgBuilderId/g" helloImageTemplateWin.json
needs to be replaces with something like this:
sed -i -e "s:<imgBuilderId>:$imgBuilderId:g" helloImageTemplateWin.json
The ImgBuilderId has slashes in it so using '/' as the delimiter in sed won't work.

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: cd4c4f41-2050-234e-e35b-750158cfba54
Version Independent ID: 09c0145e-fa2b-210d-d1a0-4acb8c790bc3
Content: Create a Windows VM by using Azure VM Image Builder - Azure Virtual Machines
Content Source: articles/virtual-machines/windows/image-builder.md
Service: virtual-machines
Sub-service: image-builder
GitHub Login: @kof-f
Microsoft Alias: kofiforson

@dwschulze
Thanks for the feedback! We will check and update the document as appropriate.

@dwschulze
Thanks for reporting this! We have created a PR for this issue and the changes should go live soon.

#signoff


Add Secure Password Encryption

Hi,
I would like to know if you plan to extend the encryption of this library to feature a PBKDF2 key stretching algorithm in addition with password salting. This would drastically improve the pin security. Implementations of this are very easy to find here for example: http://android-developers.blogspot.de/2013/02/using-cryptography-to-store-credentials.html
Thank you in advance!
Nico

Hello,
Why not. I just merged a Pull Request that implements Sha256 instead of Sha1.
Unfortunately I don't have the time right now to implement this additional security.
Would you be ok to write a Pull Request if you are interested in this feature?
Thank you in advance.
Cheers

Hi @olivierg13 and @ncioj10 ,
I have a custom AppLockImpl.java that does double sha256, encrypt sensitive information and stores it in Android SharedPreference securely.
The original question: https://github.com/OrangeGangsters/LolliPin/pull/72

...
Is there a plan to add the functionality of storing on device some sensitive information encrypted with password? In other words, in AppLockImpl.java, while calling setPasscode, we can also call something like setEncryptedMessage with it.
My use case of it is I have to store an encrypted version of access_token instead of user password. I'm currently using a custom version of AppLockImpl.
...

Huang

Hello,
I actually don't know which algorithm is the most performant (double sha256 versus PBKDF2WithHmacSHA1). Do you have informations about that?
It would be super interesting if we could find a comparison, in term of security and performances.
For your question, Lollipin is not meant to be a "storage" solution, meaning nothing in the code is by default meant to securely store sensitive information (we do so because we need it for the passcode, but that's all). If you want to add that feature, we could think about a way to properly architecture this "storage" solution? Don't hesitate to do a PR if you have some code to share.
Cheers

@olivierg13 and @ncioj10 ,
I found this article online that might help push forward our discussion: https://adambard.com/blog/3-wrong-ways-to-store-a-password/#comment-2582123806, and especially this comment: https://adambard.com/blog/3-wrong-ways-to-store-a-password/#comment-2582123806.
Best,

@trave7er
Very interesting, I guess pbkdf2 and bcrypt are really the solutions to go with for the best security.
They say Java ships directly with PBKDF2WithHmacSHA1, would be interesting to try on a large set of devices (from 4.0 to 6.x).

Hi,
atm I have very little time on my hands to do it myself but that would be the reference implementation (I would recommend a salt as well):
http://android-developers.blogspot.de/2013/02/using-cryptography-to-store-credentials.
The thing is, that PINs with 4 digits aren't that safe either way, but storing it nicely means making it a bit more dificult. Maybe I'll have the time in the next two weeks, then I'll make a pull request for it.

Hi,
I have a little time to implement https://github.com/scottyab/secure-preferences.
I already place my little code on my pull requests #94 ,

@tyangjawi03  Thank you for this nice implementation. I think your approach is the best, since we are dealing with 4 digit Pins, which are quite easily to brute force. This can be prevented better through obfuscation like with secure preferences in combination with high level encryption like the secure preferences you implemented.


from .core import * ImportError: No module named core

After doing
python3 setup.py install
When I import taso it gives
from .core import *
ImportError: No module named core

need to install cython

Can you make sure that you have Cython installed on your machine? Missing Cython returns a warning instead of an error, so many people may have missed that.
Meanwhile, make sure you run import taso in a directory other than taso/python.

works now
need to install cython


[BUG] sporadic out of order event handling with factory contracts

I have a factory contract, Parent, that emits an event with the address of two children contracts, A and B.  After A and B are created, the factory makes calls to them such that they emit events.
The log sequence is:

Parent:CreateEvent
A:ActionEvent
B:ActionEvent

In my ponder code, when I handle events from A and B, i need to do a find on the Parent event table.  If the create handler didn't fire, then there is no entry in the db.  So order of handling is critical.
This all works fine sometimes.  Ponder registers the children contracts, the event handling happens in order and all is good.
But sometimes, the handler for A and B fire without the Parent:CreateEvent ever firing.  I have verified that the logs are in order.  They are in order in the ponder_parse table.  But I can see in that table that the Parent event is not checkpointed even though the other two events, with higher logIndex values have been checkpointed.
This is particularly strange since the indexer should only handle events from A and B after it has seen the Create event.  So these children contracts are being registered, but the event that corresponds to their creation is not sent to the handler (sometimes).
I can't reliably reproduce the error on a given event.  If i nuke the db and run again, there is a chance that the ordering is correct.  Also, this only seems to be a problem in backfill.

Wow; thanks for the fast fix.  is there anything i can do before the next release to mitigate the issue?

No problem, thanks for the well written report, probably the best one we have ever gotten.
Gonna release in ~15 minutes


Verify streaming payload sizes

When a response has a streaming body with a Content-Length header, the SDK needs to verify that the full payload is received, or produce an error if not. For example, S3 GetObject can close the connection unexpectedly before all the bytes are received.
This might already be working correctly, but needs to be verified. Add an integration test.

I confirmed with @seanmonstar that hyper will error if the connection is closed before the body EOF is read. I believe that the exact check happens in this code for HTTP 1.1 connections. While I do agree that it's good to test your assumptions, I think we should rely on hyper testing itself to protect us from this.

That code you linked is rather checking if the user still cares about the body, and if not, to stop reading from the connection.
Instead, this is the code that returns an unexpected EOF.

I wrote a test and it's in PR

The PR is merged, pending release.

Linking the PR: #1906

There's no equivalent aws-sdk-rust issue for this, so I don't think we need to track release for it.


RSuccessTest.java:32-32: Improve test coverage and mutation...

The puzzle 8-dd02b080 (from #8) in atom-codegen/src/test/java/oo/atom/r/RSuccessTest.java (lines 32-32) has to be resolved: "Improve test coverage and mutation coverage for {@link RSuccess}"
The puzzle was created by Sergey Kapralov on 07-Sep-17.
Estimate: 15 minutes, role: IMP.
If you have any technical questions, don't ask me, submit new tickets instead. The task will be "done" when the problem is fixed and the text of the puzzle is removed from the source code. Here is more about PDD and about me.

@skapral please pay attention to this new issue.

@skapral/z everybody who has role DEV are banned at this job; I won't be able to assign anyone automatically; consider assigning someone manually or invite more people to the project, as explained in §51

@skapral/z everybody who has role DEV are banned at this job; I won't be able to assign anyone automatically; consider assigning someone manually or invite more people to the project, as explained in §51

@skapral/z everybody who has role DEV are banned at this job; I won't be able to assign anyone automatically; consider assigning someone manually or invite more people to the project, as explained in §51

@skapral/z everybody who has role DEV are banned at this job; I won't be able to assign anyone automatically; consider assigning someone manually (as in §19), or invite more people (as in §51), or remove the job from the scope (as in §14)

@skapral/z everybody who has role DEV are banned at this job; I won't be able to assign anyone automatically; consider assigning someone manually (as in §19), or invite more people (as in §51), or remove the job from the scope (as in §14)

@skapral/z everybody who has role DEV are banned at this job; I won't be able to assign anyone automatically; consider assigning someone manually (as in §19), or invite more people (as in §51), or remove the job from the scope (as in §14)

@skapral/z everybody who has role DEV are banned at this job; I won't be able to assign anyone automatically; consider assigning someone manually (as in §19), or invite more people (as in §51), or remove the job from the scope (as in §14)

@skapral/z everybody who has role DEV are banned at #30; I won't be able to assign anyone automatically; consider assigning someone manually (as in §19), or invite more people (as in §51), or remove the job from the scope (as in §14)

@skapral/z everybody who has role DEV is banned at #30; I won't be able to assign anyone automatically; consider assigning someone manually (as in §19), or invite more people (as in §51), or remove the job from the scope (as in §14)

@skapral/z everybody who has role DEV is banned at #30; I won't be able to assign anyone automatically; consider assigning someone manually (as in §19), or invite more people (as in §51), or remove the job from the scope (as in §14)


Please add CopyClip

CopyClip - Clipboard History Manager on the Mac App Store || https://itunes.apple.com/us/app/copyclip-clipboard-history/id595191960?mt=12

"CopyClip is the simplest and most efficient clipboard manager for your Mac. Running discreetly from your menu bar, this app stores all that you have copied or cut in the past, allowing you to quickly find that snippet of text you've been looking for."

Just checked this out and it looks dope. I've installed it myself, it should go on the list.

@damengilland Open a PR and request me as a reviewer.


add "Anrede"

please add a field "Anrede" for clients so we can address customers properly when sending invoices
e.g.

😅 👍🏼


Can't get scopeClaim

Claim scopeClaim = ClaimsPrincipal.Current.FindFirst("roles");
When I merged the code in the sample into my service, scopeClaim is always null.
I created two applications: QueryProbeClient and QueryProbeService, And configured as instructed.
On the client，I get Access Token through the following code:
string tenant = "queryprobe.onmicrosoft.com";             string aadInstance = "https://login.microsoftonline.com/{0}";             string authority = String.Format(CultureInfo.InvariantCulture, aadInstance, tenant);             string clientId = "19e4fe1c-d478-4a4b-928d-8122c17b5405";             string appKey = "****";             string todoListResourceId = "https://queryprobe.onmicrosoft.com/QueryProbeService";             AuthenticationContext authContext = new AuthenticationContext(authority);             ClientCredential clientCredential = new ClientCredential(clientId, appKey);             AuthenticationResult result = null;             try             {                 result = authContext.AcquireToken(todoListResourceId, clientCredential);             }             catch (Exception ex)             {                 throw ex;             }             return result;
I got the Access Token and added it to the http header.
On the Service, I added the following code to the startup.cs file:
public void Configuration(IAppBuilder app)         {             try             {                 app.UseWindowsAzureActiveDirectoryBearerAuthentication(                     new WindowsAzureActiveDirectoryBearerAuthenticationOptions                     {                         Tenant = "queryprobe.onmicrosoft.com",                         TokenValidationParameters = new TokenValidationParameters                         {                             SaveSigninToken = true,                             ValidAudience = "https://queryprobe.onmicrosoft.com/QueryProbeService"                         }                     });             }             catch (Exception ex)             {                 Logger.Info(ex.Message);                 throw ex;             }         }
However, I always get null when I get scopeClaim in the controller file.

This sample has been archived.
The newer one can be found at: https://github.com/Azure-Samples/active-directory-dotnetcore-daemon-v2


Add compressed texture support

Add Compressed Texture support to Lightning.
This implementation focused on the KTX1 container format to reduce the total amount of configuration required by an end user while loading images.
The WebGL renderer exposed the compressed images extensions so that at runtime a user can make the decision on what texture style to ask for.
It can contain any of the formats described and in use on the web targets. For our own testing, we utilized https://github.com/BinomialLLC/basis_universal and the UASTC style content, offline prepared to the compressed texture for the device.
This was based on a performance change with a reduction of 97% of the blocking time while loading a 1080p texture.

is now

This reduction in the blocking time of a frame render makes high-performance animations with images coming on screen, a possibility.

This is awesome! Thanks for contributing, we will review this as soon as possible


The windows documentation should use backslashes for path

Windows still uses backslashes for path

Oops :) Thank you!


DOC: Propose to use google style doc-strings for ophyd

See Google style docs

Why not numpydoc?

<personal_opinion>
I think that the google style is more compact and better read from ipython using the ? magic and just more pleasing on the eye. I don't like the underlining in the numpy style.
</personal_opinion>
However it is really @dchabot and @klauer's call as they are the architects, and we shouldn't probably bikeshed this too much. I do however think that user_api type functions need more than the usual terse string to be useful and these will face the general user.

Personally, I prefer the look of google docstrings. I made a quick (and far from perfect) auto-conversion script for both google/numpy styles so we can all decide:
google style branch
numpy style branch

The argument in favor of numpydoc is that it is the nominal standard of the scientific stack (numpy, scipy, most of the scikits-*, matplotlib (well, in the middle of a very slow conversion), pandas, ...) and is what we are using for the everything in Nikea.  There is value in reducing the cognitive load of switching between things like this.
I have been staring at numpydoc strings for several years now and I find the google style to be really cramped.  It just looks like a wall of text that I have to read, where as I am used to the white space patterns, and can jump around and answer things like 'what is the expected type of the third argument?' very quickly (find the third ':' down in the un-indented lines in the Parameters section).
I suspect that the features of numpydoc that I find most appealing are the exact same features that make it seem visually less pleasing :smiling_imp:

You can install numpydoc through either pip or conda.

Hi all,
Ok after reviewing the conversation, I agree with @tacaswell on the arguments about numpy style docs.
So.... As CTO I am deciding that ALL NSLS-II doc-strings will follow the Numpy style. Please follow this from now on across both the analysis and acquisition stacks.
Having consistency across the data collection and analysis I believe is also important and maintaining a standard is also very important.
Please make sure that from now on all docstrings follow the numpy style Numpy Docs
Installation of either numpydoc or napoleon is easy with conda or pip, and I don't see this a problem.
And please, no painting the bike shed ....


 So when you want to run a for loop in a separate goroutine, you have to do such thing:
go func() {
	for {
		// do something regularly
	}
}()

What if you could run a for loop like this ?
go for {
	// do something regularly
}

I know it's a little confusing when it comes to considering the scope but I think it looks clean and minimal.

This looks like related to #24210.

Why should we permit specifically go for and not any other kind of statement?
The current go statement simply accepts any function call.  That is easy to understand.  Supporting go with a for statement doesn't seem to me like a particularly common case, and I don't see we should add a special case for it.

Based on emoji voting this proposal is not popular.  Also as noted above it is not general.  For these reasons, this is a likely decline.  Leaving open for four weeks for final comments.

I know it's a little confusing when it comes to considering the scope but I think it looks clean and minimal.

The problem with both this proposal and the normal way of writing it (go func() { for { ... ) is that the stacktraces are ugly when you're debugging a server. It's nicer to give those permanent goroutines names.


Maintenance: Upgrade to node 12

It's now in long-term support.  The current build setup pins the dev and production node version in package.json, but floats to the latest LTS in Travis.  This means that when a new LTS is released, the build fails like this: https://travis-ci.org/studentinsights/studentinsights/jobs/600978116  This seems fine, and a healthy kind of prompt, so keeping it rather than fixing the version in Travis as well.

selfie

selfie


Replace white colours with #ece9e3

Replace atleast in the Pure Black theme, to make it ease on eyes.
#ece9e3 is also white but looks bettter.

#ece9e3 looks like this.