Change term - occurrenceStatus

Change term

Submitter: John Wieczorek (following discussion initiated by Steve Baskauf @baskaufs - see below)
Justification (why is this change necessary?): Clarity
Proponents (who needs this change): Everyone

Current Term definition: https://dwc.tdwg.org/list/#dwc_occurrenceStatus
Proposed new attributes of the term:

Term name (in lowerCamelCase): occurrenceStatus
Organized in Class (e.g. Location, Taxon): Occurrence
Current definition of the term: A statement about the presence or absence of a Taxon at a Location.
Proposed new definition of the term: A statement about the presence or absence of an Organism within a bounded place and time.
Usage comments (recommendations regarding content, etc.): Recommended best practice is to use a controlled vocabulary consisting of the two distinct concepts "present" and "absent". This term is not apt for breeding status, for which the term reproductiveCondition should be used. This term is not apt for threat status, for which one might consider using the Species Distribution Extension (http://rs.gbif.org/extension/gbif/1.0/distribution.xml - not part of the Darwin Core standard).
Examples: present, absent
Refines (identifier of the broader term this term refines, if applicable): None
Replaces (identifier of the existing term that would be deprecated and replaced by this term, if applicable): http://rs.tdwg.org/dwc/terms/version/occurrenceStatus-2017-10-06
ABCD 2.06 (XPATH of the equivalent term in ABCD or EFG, if applicable): not in ABCD

Discussion leading up to this change proposal can be found in Issue #238.

To facilitate use with the controlled vocabulary proposed in #342, Change Usage comments to "Recommended best practice is to use controlled value strings from the controlled vocabulary designated for use with this term, listed at (insert vocabulary page here). This term is not apt..." to be consistent with other terms having controlled vocabularies.
Due to the simplicity of the CV, I would leave the examples so as to not require people who know English to follow the link to the CV page. However, note that this term exists in both the dwc: and dwciri: namespaces. For the dwciri: variant of the term, the examples will need to be changed to the term IRIs once they are assigned.

 The establishment of a formal controlled vocabulary provides the framework for linking labels and definitions in many languages to the controlled value terms. Establishing the controlled vocabulary has no effect on the recommended controlled value strings, which are identical to those already in common use.

Has an existing dwciri: analog, so use the same changed definition and usage comments, but substitute IRI values in the examples.

I am sort of okay with the change in definition, given that an the Darwin Core definition of Organism can mean all members of a Taxon in a given area (and 'absent' only makes sense in that way), but I do not agree with the usage comments or the extremely restricted vocabulary (I will address that in #342). The comment that threat status is better dealt with in the Species Distribution Extension, while I agree with it, is neither here nor there, as the Species Distribution Extension goes with a Taxon Core and, when using a Taxon Core, occurrenceStatus is in the Species Distribution Extension too.

@nielsklazenga The vocabulary proposal here is only a formalization of what the precise vocabulary restriction was meant to be from inception. The term was only ever meant to be to distinguish presence from absence. I substituted Organism for Occurrence, because it isn't an Occurrence that was present (or absent), but more precisely an Organism was present, or any instance of an organism of a given Taxon was absent. The bottom line is that the vocabulary isn't really open for dispute unless you propose to change the semantics of the term. The usage comments were to nail down how to deal with things that people might confuse as occurrenceStatus.

@tucotuco, does not the definition of Organism in Darwin Core make any entry in a species checklist or flora etc. more or less an Occurrence? I agree that the terms related to breeding status and threat status, which were explicitly excluded in the usage comments should indeed not be in occurrenceStatus. However, terms like doubtful and excluded, which were in the previous GBIF Occurrence Status Vocabulary are statements about the presence or absence of an Organism in an area and therefore fall within the definition of occurrenceStatus. excluded can be mapped to absent, although I think there is still value in having it in the vocabulary, but where does one go with doubtful?

I commented on this here.

Sorry, but there is so much to catch up on here.
Discussion relevant to this proposal has developed in two distinct issues, this one and the proposal for the controlled vocabulary for occurrenceStatus. Though the two proposals are intimately related, they can be ratified, or not, independently. Even so, I would like to summarize the state of the proposals together here.
Is the proposed definition of occurrenceStatus satisfactory? Controversial
Current Definition: A statement about the presence or absence of a Taxon at a Location.
Proposed Definition: A statement about the presence or absence of an Organism within a bounded place and time.
Comments: This is the real crux of all issues that have been raised. The DwC Maintenance Group originally labelled this as a non-normative change, feeling that the change was to clarify the definition based on our understanding of the term. That understanding was based on the placement of the term within the Occurrence class and the thinking that went into the creation of the term to begin with. That thinking can be elaborated (beyond the actual normative definition) as
to make a definitive statement about whether any Organism given an Identification that posits it as being a member of a given Taxon was detected during an Event that had any members of that Taxon as a target of detection using a samplingProtocol.
Only what is known need be included in the Occurrence record (just because you don't have an explicit samplingProtocol doesn't mean you can't say something about occurrenceStatus). The current definition lacks any such rigor. The proposed definition isn't actually much better, but at least it tries to disambiguate that a Taxon is not something that can be present while an Organism is.
To me (also beyond the normative definition), a Taxon is an organizational hypothesis supported by discriminative evidence using organisms or derivatives of organisms as evidence for its validity. We use those hypotheses to label and group Organisms based on acceptable similarities of characters. A hypothesis isn't something that can be present in the occurrenceStatus sense. What can be present (or absent) is evidence of an Organism that would be classified as a given Taxon.  Damned semantics!
The actual definition of a Darwin Core Taxon doesn't reflect my personal opinion, however. The current definition of Taxon looks like a closeMatch synonym of Organism:
A group of organisms (sensu http://purl.obolibrary.org/obo/OBI_0100026) considered by taxonomists to form a homogeneous unit.
I think this definition is an underlying source of a real problem, because it reflects the superficial way in which we talk about what a Taxon is.
The rest of the controversy seems to be around the limits on the scope of an Organism. The definition of the Organism class includes no limit to scope,
A particular organism or defined group of organisms considered to be taxonomically homogeneous.
The definition of the term organismScope, meant to say explicitly what level of aggregation we are talking about, also holds no further constraint,
A description of the kind of Organism instance. Can be used to indicate whether the Organism instance represents a discrete organism or if it represents a particular type of aggregation.
Based on just the current definitions, it should be perfectly fine to have an Organism that includes every member of a Taxon at any scale, including the whole planet, but recommended practice would be to provide the scope explicitly.
An aside: To accept the entire continuum of scope as valid has a side benefit that it alleviates the arbitrary distinction of an Occurrence record from a record in a Checklist (of which there is no defined concept in Darwin Core, on purpose). The Checklist record indicates the accepted presence of at least one member of a Taxon within a bounded place and time, where the place and time, if provided, are usually captured in data set metadata or in a Species Distribution Extension. Because proponents of checklists treat them as a type of Taxon dataset for the purpose of sharing data, the illusion that occurrenceStatus can apply as easily to a Taxon as it does to any Organism that is a member of a Taxon arises.
Concretely, I suggest to amend the proposed definition of occurrenceStatus to:
A statement about whether any Organism identified as being a member of a given Taxon was detected during an Event that had any members of that Taxon as a target of detection.
If this is an acceptable definition, the concept extirpated could be captured in an Occurrence record with a geographic scope defined by the values in the  terms of the Location class in that record, with the temporal scope defined in the values in the terms of the Event class along with the detection methods employed, with the taxonomic scope of the detection effort in the values of the terms in the Identification and Taxon classes, and with absent in occurrenceStatus.
The concept of doubt as another vocabulary term is controversial in discussions. This concept (along with all others that are not present or absent) arose from the GBIF vocabulary for occurrenceStatus, which is a GBIF artifact, not generated from community activity through TDWG. You may be interested to know that there are a total of 19980 Occurrence records with the value 'doubtful' in all of GBIF (as of 2021-01-12), and all of these are from one dataset, the identity of which is being withheld to protect the innocent. You may also be interested to know that there are 3073 case-insensitive distinct values of occurrenceStatus in that same data snapshot.
There are many ways in which the specific nature of doubt can be expressed independent of a statement about presence or absence. The nature or source of the doubt would be lost if all that was left of it was the value doubtful in occurrenceStatus. For that reason, the label doubtful is probably a very poor choice. Here are some examples of ways to capture explicit sources of uncertainty about an Occurrence:

Occurrences that are doubtful because of the Identification should capture that uncertainty in some combination of identificationQualifier, identificationVerificationStatus, and identificationRemarks. The Organism was present, but the Identification is the thing that is doubtful, not its presence.

Occurrences that are doubtful because of uncertainty about the Location should capture that uncertainty in some combination of the Location terms to reflect the geographic scope within which the presence or absence was asserted.

Occurrences that are doubtful because of uncertainty about the timing should capture the temporal uncertainty with beginning and ending bounds in the Event terms, as should parameters that affect detectability(such as samplingEffort, sampleSizeValue, sampleSizeUnit, samplingProtocol).

Independent doubt about presence is a strange concept. If I don't know if I detected something or not, why am I bothering to say so? What knowledge am I imparting? I think the concept that is missing is not doubt at all, but rather a measure of certainty (likelihood). I can see that it might be useful to know that it is unlikely that something was at a given place and time, especially for large scales of place and time, because it can impart broadly useful expert knowledge. We don't have another way to capture that in Darwin Core that isn't essentially an occurrenceRemark or an interpretation of the results of a samplingProtocol and its measurement results.
I don't see categories of likelihoods as good candidates for values in a controlled vocabulary. No one would really know what they mean. It seems more the stuff of summary analysis and would be a confounding value for occurrenceStatus. I urge that this concept be captured in a separate term, if needed, just as other kinds of uncertainties are captured in Darwin Core.
On to the rest of the summary...
Does the proposal to have a controlled vocabulary for occurrenceStatus hold merit? 'Yes'
Should the values 'present' and 'absent' be in the controlled vocabulary? 'Yes'
Is the proposed definition of present satisfactory? 'No'
Comment: The definition will have to be in accord with the final accepted definition of occurrenceStatus. A generic definition that will almost certainly work regardless of other concerns is something like the following:
the target was detected within the given bounds of place and time
Is the proposed definition of absent satisfactory? 'No'
Comment: The definition will have to be in accord with the final accepted definition of occurrenceStatus. A generic definition that will almost certainly work regardless of other concerns is something like the following:
the target was not detected within the given bounds of place and time
Should there be additional values in the controlled vocabulary? Controversial
Comment: So far there is no consensus on the inclusion of any terms other than present and absent.

Many thanks, @tucotuco. One note of protocol for epic posts: always best to warn @timrobertson100 to go get a cup of tea up at the top.
I like how you addressed the relationship between Taxon and Organism as effectively representing alternate ends of the same spectrum.

The definition of the Organism class includes no limit to scope

I agree this is true in principle, but maybe not so much in practice.  We wrestled with this when Organism was first proposed (specifically whether it extended to "population", and I think it does require more wrestling.  To me, the distinction ought to be: when an aggregate Organism (i.e., more than one individual) is broad enough that a taxonomist has taken the trouble to assign a formal name to it, then it crosses into the ream of Taxon.

Based on just the current definitions, it should be perfectly fine to have an Organism that includes every member of a Taxon at any scale, including the whole planet, but recommended practice would be to provide the scope explicitly.

Hence my earlier comment that we should treat the scope of an Organism as extending from a single individual through various degrees of aggregates up to but not exceeding (or even including?) something that can be referenced by a formal taxonomic name.
That aside...

...an Organism within a bounded place and time

You addressed the Taxon-->Organism change (which I wholeheartedly agree with), but I wonder if you could comment whether the Location-->"bounded place and time" change might be better solved with a change of Location-->Event.  The crux of this (discussed on other issues here) is whether an Event can be defined as a "bounded place and time"; or perhaps more accurately, "an action that is bounded in place and time"?

To clarify, the proposal when the public review opened was:
A statement about the presence or absence of an Organism within a bounded place and time.
which, based on discussion so far, I proposed to amend to the following:
A statement about whether any Organism identified as being a member of a given Taxon was detected during an Event that had any members of that Taxon as a target of detection.
I maintain that a Taxon is a theoretical construct and an Organism is not, so that there is no way to have a continuum from one to another and no issue about where to draw the line in that respect. With respect to the scope of Organism, however, I don't feel there is a defensible line that can be drawn, nor can I imagine what is to be gained by making one. I guess I am not very creative. Clearly if I was a taxonomist I would be a lumper.

Thanks @tucotuco , I am a taxonomist and I have no problem at all with your characterisation of a Taxon. Our disagreement is in whether or not there can be an occurrence (of an Organism/Taxon, does not matter) independent of an (dwc:) Event.
occurrenceStatus has been successfully used for years in statements about the distribution of organisms, most notably with the GBIF Species Distribution Extension, which is used in Darwin Core Archives in combination with a Taxon Core. Even though we might want to replace the Taxon Core with something else, the distribution statements will always be an important part of Species Description data. The last proposed definition:

A statement about whether any Organism identified as being a member of a given Taxon was detected during an Event that had any members of that Taxon as a target of detection.

..., which I agree reflects one side of the discussion, mainly in issue #342, feels to me less like clarification and more like hijacking of a term that works perfectly well (and does not need extra clarity) for a different purpose.

I would like to try to summarize the controversy. I maintain that an Organism and a Taxon are conceptually distinct. One is the manifestation of a theory and the other a manifestation of biological entities. There is no continuum leading from one into the other and one is not semantically a subtype of the other, because theories are not living beings. Stated in another way, the attributes of a Taxon do not apply to an Organism, nor vice versa.
It isn't clear that there is agreement on this much. But there is more. Even if everyone accepts that the two classes are semantically distinct, I am of the opinion that the distinctness means that the term should not be a property of both classes. Granted, in Darwin Core we do not make the formal assignment of properties to Classes, we merely annotate the properties to be "organized in" a class. This was done on purpose in the absence of a rigorous community-wide conceptual schema for the Classes we manifest in various contexts. For Darwin Core, the conceptual schema was expected to be a separate exercise, after which formal assignments of properties to Classes could be made in the future. That future still hasn't arrived.
Thus, whereas nothing in Darwin Core prevents a term from being used as if it were a property of any Class, nor of it being assigned to no class at all (see Record-level terms), it is not a good way to prepare for a semantically rigorous future. Herein lies my principle objection. It isn't clear if there is any agreement about my position on this. If there is, it suggests that two distinct terms are necessary - one for Occurrence and one for Taxon.
Even if there is agreement on that last point we have contention built off the legacy built around the term to date. The name of the term and its organizational placement in Occurrence suggest that the term should apply to Occurrences only. That was indeed the (only) original purpose of the term, and the reason for the two original examples only. But the original definition is at odds with all the foregoing, being guilty of laziness in the use of the word Taxon and therefore opening the door for it to be used in a different way. Since the door was open, it was used in a different way. It was incorporated into the Species Distribution Extension, but it was given an entirely different definition (Term describing the status of the organism in the given area based on how frequently the species occurs.) and even a controlled vocabulary (https://tools.gbif.org/dwca-validator/vocabulary.do?id=http://rs.gbif.org/vocabulary/gbif/distribution_status/), none of which was done following the standards process in which Darwin Core is developed.  This was a regrettable development brought on by a poor definition on the part of the term, and an interpretation of the term in a different context without reconciliation in the context of the standard. In this proposal, we are trying to correct the problem with the definition of the term in the standard.
For my part, I would be perfectly happy to help with the correction of downstream problems that arose with the Species Distribution Extension re-defining the term for its own purposes, but I think the only reasonable way forward with that is to use a different term in that extension, whether or not that term (or its recommended controlled vocabulary) is also incorporated into Darwin Core.
Though I feel strongly about all of this, I recognize my part in creating the problem to begin with. I also recognize that I have to put my role as convenor and mediator before my role as a stakeholder in the community, so if I am the only one that has the viewpoints I have expressed, I am happy to sequester my objections from consideration of achieving consensus and let the proposal move forward without them. Even so, I am not sure we have achieved a clear consensus. Thoughts?

I would like to put forward my agreement with what @tucotuco has laid out and voice my support for occurrenceStatus being strictly related to occurrences, Organism and Taxon being conceptually distinct, and the definition of occurrenceStatus being amended to A statement about whether any Organism identified as being a member of a given Taxon was detected during an Event that had any members of that Taxon as a target of detection. Given this, I agree that the best path forward for the Species Distribution Extension would be adoption of a different term in that extension.

I would like to try to summarize the controversy. I maintain that an Organism and a Taxon are conceptually distinct. One is the manifestation of a theory and the other a manifestation of biological entities. There is no continuum leading from one into the other and one is not semantically a subtype of the other, because theories are not living beings. Stated in another way, the attributes of a Taxon do not apply to an Organism, nor vice versa.

OK, I could quibble with this, but only in an effort to play Devil's advocate (to wit, both Organism and Taxon classes could be construed as conceptually identical in the sense of "circumscribed set of one or more living things"; the only real difference being the formality of the label). HOWEVER, in the context of DwC (not to mention "common sense"), I am in full support of what @tucotuco asserts above.  My only concern is that a single instance of Organism can be scoped up to (and including) "every organism of a particular Taxon that ever has or ever will exist".  In other words, the only upper bound to the scope of an instance of Organism is that all members remain "taxonomically homogeneous".

It isn't clear that there is agreement on this much.

I'm pretty sure I'm the only one who floated the idea that Organism and Taxon could be interpreted as existing on the same continuum, and if so, then my comments above should dispense with any perceived disagreement.

But there is more. Even if everyone accepts that the two classes are semantically distinct, I am of the opinion that the distinctness means that the term should not be a property of both classes.

I completely agree with this opinion.

If there is, it suggests that two distinct terms are necessary - one for Occurrence and one for Taxon.

I'm still unclear on why we need something like this to be a property of a Taxon instance.  As long as we can define an Organism to include up to (and including) every member of a Taxon (as pointed out above), then we shouldn't have any problem capturing what most people think of as "Taxon-at-Location" instances in the form of Occurrence instances (which I view as Organism-at-Event instances; but could also be represented as Organism-at-Location instances).  So, unless I've misunderstood previous comments (@nielsklazenga ?), I think there is consensus that occurrenceStatus can remain as a property within the Occurrence class (and not within the Taxon class).

Thoughts?

Like @albenson-usgs, I'm in agreement with your main points, and am likewise OK with A statement about whether any Organism identified as being a member of a given Taxon was detected during an Event that had any members of that Taxon as a target of detection. as the definition for dwc:organismStatus.  I'm a little queasy about the that had any members of that Taxon as a target of detection part.  I understand what it's intention is (i.e., to clarify the meaning/context of an "absence"), but read literally it implies that you can't have a value of "present" for any Organisms identified to a Taxon that were not among the targets of detection.  Many are the times when I record an Organism as present at an Event, when I had no expectation of it occurring there/then (and, hence, could not have been the "target" of detection). Such is the fodder for new records...

I am not going to quibble either. I think it will be good to have different terms to indicate presence or absence during an Event and distribution. I think occurrenceStatus makes more sense for the latter, but that is the problem you have when you have URIs that are not opaque.
For me this has never been on whether this should be in Occurrence or not and I think it should not be, as then we would get the same problem with establishmentMeans and degreeOfEstablishment. I have no problem with the new term being in the Taxon class though.
I agree with @deepreef that that had any members of that Taxon as a target of detection is best left out of the definition, as that would make the term not apply to Specimens and Observations during non-targeted Events (that's the majority of Occurrences, I think).

OK, I had included that had any members of that Taxon as a target of detection to help circumscribe how one would arrive at an assertion of an absence. I see now what @deepreef and @nielsklazenga have identified as problematic. I think that elaboration can be solved better in a usage comment, leaving the actual definition simple, as proposed, or slightly amended based on discussions up to now to, A statement about the presence or absence of an Organism within an Event.
However, this definition still doesn't provide a solution for all points of view. It commits to the term being unambiguously a property (albeit informal) of an Occurrence, not of a Taxon. The current state of the term is contradictory with respect to its definition (based on Taxon) versus its organization (within Occurrence), which ambiguity leaves it open for interpretation and for the community practice that has come to pass. It may be that the only equitable solution is to deprecate the term entirely and create two new ones so that everyone is equally upset ;-), but at least the distinctions and uses and vocabularies would be clear.
Given that the problems go much broader than the revision of this term (@nielsklazenga has pointed out that there are similar conceptual issues with respect to establishmentMeans and degreeOfEstablishment, and there has been a lot of discussion about the scope of Organism and how to use that), I recommend that a task group be chartered to come up with a solution that clearly delineates all of the aspects of the problem that are tripping us up.
My personal hope (bolstered as not being entirely ridiculous by discussions here) is to demonstrate that a Taxonomic Checklist (to which a species distribution can be applied) is really just an inverted view of Occurrences (by putting the Taxon at the center) at different scales in time and place than is customary while dealing with observations, specimens, and material samples.
A bigger dream still is to enable ourselves to organize information around primary concepts (classes) and their data, connecting them in ways that make rigorous sense and allowing us to take advantage of the centralization of these themes to build community services. I hope in the not-too-distant future to be able to say, "Who would have guessed that occurrenceStatus would be the straw that broke the camel's back?"

I wrote a big-long reply, full of with and snark and all the usual stuff, then managed to close my browser before clicking the "Comment" button.  I'll cut to the chase:

I recommend that a task group be chartered to come up with a solution that clearly delineates all of the aspects of the problem that are tripping us up.

I think this is the most sensible approach.  Also, in addition to establishmentMeans and degreeOfEstablishment, the term pathway falls into this same vortex.
The rest of the stuff I had written before was more evangelizing on the concept of "Evidence-Based Checklists", prompted by @tucotuco 's comment about Taxonomic Checklists as inverted Occurrences, which is the principle we've been operating on for nearly a decade now.  A topic for another day...

I think we need to be pragmatic at this point.  Using GBIF as an example:
Occurrence record use

6,348 datasets covering 1,150,021,961 records populate this field with a value
All records are interpreted to present/absent only (inferred from other metadata when necessary) following the latest GBIF vocabulary

Species distribution extension use

352 dataset use distribution extensions with occurrence_status populated (covering 3,859,858 name_usage)
38 datasets use distributions with occurrence_status populated with values that are not PRESENT/ABSENT covering (covering 89,036 name_usage).

The values are according to the GBIF vocabulary from 2009 and populated as:
 occurrence_status | name_usages
-------------------+------------
 PRESENT           |  3757777
 COMMON            |     3399
 RARE              |     3019
 IRREGULAR         |     7619
 DOUBTFUL          |    69735
 EXCLUDED          |     5264
 ABSENT            |    13045
 NULL              |  1022671

For pragmatic reasons (it's been used for a decade), GBIF continues to use occurrenceStatus as the field name in the species distribution extension name the vocabulary distributionStatus. When a new edition is created, distributionStatus will be considered as the term name. This decision was taken to 1) avoid breaking existing datasets, 2) because the GBIF checklist indexing is maintained but not significantly developed at the moment, as we focus on the Catalogue of Life infrastructure (i.e. resource constraints).
Suggestion for consideration
The phrasing proposed in the issue of "taxon at a location" seems appropriate, as that really is what an Occurrence record is and is equally suitable for a species distribution use.
A pragmatic solution, allowing Darwin Core to be as reusable as possible could be to refine the usage comments to say:

Recommended best practice is to use a controlled vocabulary. When using this with an Occurrence the default vocabulary is recommended to be "present/absent" but can be extended by implementers with good justification.

Though the pragmatic solution proposed by @timrobertson100 takes occurrenceStatus in the exact opposite direction to the proposed changes in this issue, it would be a non-normative change with no bearing on existing implementations, and could be adopted instead of the original proposal without need for public review.

If all are fine with the non-normative change proposed by @timrobertson100, I can live with that solution as well.
However, I am very strongly in favor of the original direction @tucotuco had been pushing this.  I desperately hope we (TDWG community) are moving in the direction of @tucotuco's "bigger dream", and that following the proposal of @timrobertson100 now is understood to be only a pragmatic solution to simplify the path immediately in front of us, and that this issue will need to be resolved more robustly in the (not-too-distant) future. Moreover, I hope that this pragmatic solution does not preclude @tucotuco's suggestion for (and my endorsement of) forming a task group to come up with a more appropriate and robust solution.
That's my short response.
Much of my long response was already written, but lost when I closed the browser before clicking the "Comment" button yesterday.  As I indicated in my previous post, a lot of it relates to the idea of evidence-based checklists, and occurrences being the inversion of existing taxonomic checklists.
At the beginning of DwC, the predecessor of what is now called Occurrence was essentially the concept of specimen-in-collection (i.e., the original point was to be able to share and aggregate data about specimens in museum collections in a standardized and consistent way).  At some stage early on in the formation of the standard, the community decided the lowest-hanging fruit of scientific interest represented by museum specimens was as a source of information for Taxon-at-Location.  Once that bridge had been crossed (and perhaps also the main motivation for crossing it), additional records of unvouchered instances of Taxon-at-Location in the form of Observations came into scope, and the notion of Occurrence was formalized.
That transformation from specimen-in-collection to Taxon-at-Location was a fundamental and important step forward for DwC.  However, it was only one step on a longer journey.  As a community, I'd like to think that we're ready to make progress on the next step, which is an absolute prerequisite to realizing @tucotuco 's (and my, and many others') "bigger dream" (DwC as a true information model for biodiversity that can serve a much broader array of scientific uses).  That next step is to formalize the transition of Occurrence from Taxon-at-Location to Organism-at-Event.
This discussion had really given me hope that we had achieved critical mass to take that step. But perhaps we're not quite there yet.  If we can follow through with a Task group (perhaps integrated with the proposed task group for #314), then I have at least some hope for keeping the dream alive.
@tucotuco :  This issue might not have broken the camel's back (yet), but I hope at least the camel is in serious need of the services of a chiropractor.

Just to make myself really clear, I just want a property that I can use on taxon-area statements; it does not have to be occurrenceStatus. I totally accept that, if there are going to be two terms, occurrenceStatus goes with the presence/absence observations, which is the way it has been used much more frequently, including by myself. I would be happy with the distributionStatus mentioned by @timrobertson100.
There is an analogy with the "establishment means" terms here as well, as establishmentMeans as it is used now might better be called 'originStatus', whereas 'establishmentMeans' would better go with vector (this was indicated in the proposal at the time, so not some other problem I "discovered").
I think this is still a simple choice between one or two terms. I fail to see how the data model comes into it, or how anything @timrobertson100 suggested goes against the "bigger dream" of "Darwin core as a true information model for biodiversity", although I am not sure what 'true' means here.

I fail to see how the data model comes into it, or how anything @timrobertson100 suggested goes against the "bigger dream" of "Darwin core as a true information model for biodiversity"

Not to belabor, but what I meant was that we come to a clear agreement on what actual entity is meant for an instance of the Occurrence class.  At the moment, we do not have a clean working definition of what Occurrence instances represent (the definition per se is not so much the problem, but rather how records are actually presented by content providers).  Some content providers treat each unique instance of an Occurrence as a "specimen" (=instance of MaterialSample).  Some treat it as an instance of a Taxon-at-Location (e.g., "taxon-area").  Some treat it as an instance of Organism-at-Location. Some treat it as an instance of Organism-at-Event.  While this might seem like semantic technicalities, semantics matter (especially for the semantic web).
This sort of imprecision is tolerable so long as DwC is a "bag of terms" that are merely "organized" in classes.  What I meant by the "bigger dream" of "Darwin core as a true information model for biodiversity" is that each DwC class can achieve a precise definition in the context of an information model, and each term can be clearly mapped as a property of instances in exactly one of those precisely-defined classes.
For example, right now, a unique occurrenceID value might identify an instance representing unique combination of taxonID+locationID, or it might might identify an instance representing unique combination of organismID+locationID, or it might identify an instance representing unique combination of organismID+eventID, or it might represent some sort of "evidence" supporting any one of those things.
That degree of ambiguity in the conceptual entity represented by an instance uniquely identified by occurrenceID is one of the things that currently limits the power of information shared through the DwC standard.

I get that we need to distinguish between the primary occurrence data and the distribution data that is inferred from it, but what has it got to do with occurrenceStatus?

I get that we need to distinguish between the primary occurrence data and the distribution data that is inferred from it, but what has it got to do with occurrenceStatus?

From my perspective: it's not helpful to have a single term serve to represent a property of more than one class of "thing".  As @tucotuco  outlined, this particular term has a mixed interpretation (both through its definition history and its actual usage history) for application to both "primary" Occurrence data and taxon distribution data.  I think the goal would be to clarify its definition and usage to be more consistent to a single purpose.

I have already said that I am fine with two terms. My understanding of the nature of absence data is not enough to argue one way or another. It seems to me though that some people who want to make the change are uncomfortable with it. You cannot bring the data model into it though, as then you would have to do the same for several other terms.

Some content providers treat each unique instance of an Occurrence as a "specimen" (=instance of MaterialSample)

A "specimen" is an instance of PreservedSpecimen and/or Occurrence, not MaterialSample (see ABCD equivalent in https://dwc.tdwg.org/list/#dwc_Occurrence). It is not occurrenceStatus that is going to break the camel's back, but MaterialSample.

From my perspective: it's not helpful to have a single term serve to represent a property of more than one class of "thing".

That is diametrically opposite to the Standard Maintenance Specification, as well as circular, as you can always create a superclass that a property uniquely applies to.

I guess we'll just have to agree to disagree on some of these things.  But I definitely agree with this:

It is not occurrenceStatus that is going to break the camel's back, but MaterialSample.

Keeping in mind, of course, that the metaphorical broken camel's back is a good thing (i.e., progress...), in this context.

Hoping not to cause yet more trouble, may I ask how an absence occurrence should be interpreted if it contains further properties than just a location, time and "subject"?
If the Occurrence specifies a chicken with sex=female, basisOfRecord=PreservedSpecimen.  It would make absence data very hard to use. Should that be something to add to comments?

PreservedSpecimens are always present. A female chicken might be absent if it has an organismID. There might even be other female chickens around.

Just to elaborate a bit, the underlying assumption is that when an instance of PreservedSpecimen is represented (inappropriately, in my view, but we'll break that camel's back another time) as an instance of Occurrence, the implication is that the Occurrence is tied to the Event at which the associated Organism (or derivative of an Organism) was extracted from nature.  As such, I completely agree with @nielsklazenga that a PreservedSpecimen always implies present in the context of an Occurrence, and any records that combine basisOfRecord=PreservedSpecimen with occurrenceStatus=absent are logically inconsistent (and should be flagged as such).
I also fully agree that a female chicken could be absent, either as a specific Organism, or in the general sense ("we only saw male chickens").  One slight twist to this, though:  I always associate sex as a property of an Occurrence, not as an Organism, because so many organisms change sex throughout their lifetime.  Thus, even though birds do not change sex, I still wouldn't associate sex=female with any specific organismID.  But that's a subtle issue outside the context of the question that @mdoering posed.

Just to elaborate a bit, the underlying assumption is that when an instance of PreservedSpecimen is represented (inappropriately, in my view, but we'll break that camel's back another time) as an instance of Occurrence, the implication is that the Occurrence is tied to the Event at which the associated Organism (or derivative of an Organism) was extracted from nature.  As such, I completely agree with @nielsklazenga that a PreservedSpecimen always implies present in the context of an Occurrence, and any records that combine basisOfRecord=PreservedSpecimen with occurrenceStatus=absent are logically inconsistent (and should be flagged as such).
I also fully agree that a female chicken could be absent, either as a specific Organism, or in the general sense ("we only saw male chickens").  One slight twist to this, though:  I always associate sex as a property of an Occurrence, not as an Organism, because so many organisms change sex throughout their lifetime.  Thus, even though birds do not change sex, I still wouldn't associate sex=female with any specific organismID.  But that's a subtle issue outside the context of the question that @mdoering posed.

...and one more minor elaboration. A preserved specimen that is no longer in a collection can be indicated using the term disposition. That is a different kind of absence, not in the Occurrence, but in the material in a physical collection.

That is a different kind of absence, not in the Occurrence, but in the material in a physical collection.

So... before I had a better idea of the distinction between MaterialSample and Organism, we actually went through an experimental phase where we tracked the physical/temporal location of specimens using Occurrence records.  We start with the idea of an Organism in nature, encountered by another Organism in nature (two Occurrences at the same Event). In the vast majority of cases, one of those Organism instances is reliably identifiable to Homo sapiens sec. Wilson & Reeder 2005 (our go-to accordingTo for that particular species).  That Organism performs some sort of action that extracts (and usually leads to the demise of) the other Organism, at that particular Occurrence instance.  This extracted Organism then participates in another Occurrence instance, at the time and place where the human Organism prepares and photographs the extracted Organism (e.g., back at the field station).  As the extracted Organism continues to move through space and time (arriving at a Museum, being unpacked and curated, stored in a jar on a shelf in a room of a building somewhere, later taken off the shelf and moved to a laboratory for examination or subsequent imaging or whatever, then perhaps transported to a different Museum on loan, or being subsampled, etc., etc., etc.) ... each notable moment in the space-time trajectory of that extracted Organism is documented in the form of an Occurrence instance.
The concept actually worked extremely elegantly as a way of tracking a specimen through space and time -- no different than, say, a satellite tag affixed to a cougar or whale or shark or something.  Each Organism accumulates potentially dozens of Occurrence instances, only one of which represents the point of extraction from nature (which most DwC content providers are focused on as the only Occurrence of note for the Organism).  In this context, one could indeed score the "absence" of said Organism within a collection (e.g., when an inventory is done and the specimen is missing or disintegrated).
Both of these weird ways of looking at things:

the "collector" and the "collected" representing two Occurrence instances at the same Event, the former performing the action of collecting, and the latter performing the action of being collected; and
Tracking a PreservedSpecimen through space and time using Occurrence instances

dramatically simplified the data model and provided some very cool ways of parsing and representing patterns in the data.
However, of the two, I think the first still has merit, but the second breaks down when we clarify the distinction between Organism and MaterialSample.
Again, I digress...

We see two cases where basisOfRecord = preservedSpecimen and occurrenceStatus = absent in the GBIF publishing community.

Where the remains of a species are collected (e.g. bones)
Where environmental material (e.g. water, soil) is preserved as the evidence supporting the claim that the species does not exist there (e.g. some sequencing and clustering)

materialSample is likely a better option for 2. but due to the loose nature of DwC documentation it's understandable that someone assumes specimen may refer to environmental material.

Should that be something to add to comments?

I agree with @mdoering that expanding the comments to guide users when combining other terms is needed. We've identified disposition, basisOfRecord, trait related terms (sex, reproductiveCondition and we need to add lifeStage). I'd suggest we also cover those relating to measurements (e.g. individualCount), behaviour and occurrenceRemarks which could lead to confusing records.
(Aside - I feel much of the discussion on this thread belongs elsewhere to attract the visibility it deserves. This thread should focus on comments, and ideally proposals, that help refine occurrenceStatus only)

For 1 the How did it die? Task Group and a possible new term vitality may help? https://github.com/tdwg/how-did-it-die/issues/1.

this thread belongs elsewhere to attract the visibility it deserves

As a relative newcomer to TDWG, where is that exactly? I'm not trying to be difficult and I don't disagree that this has veered far from the original purpose. I've just had a hard time figuring out where the core discussions happen around the standard. Some discussions seem to happen in the GBIF Github, some discussions in the OBIS Github, some (but fewer) discussions in the TDWG Q & A repo, very few on the listserve (or maybe I'm not on the right one). This is actually the most discussion about the standard I've seen is only on issues where changes are proposed. Where should this discussion happen to get the most visibility?

@albenson-usgs This sort of discussion used to happen on the tdwg-content email list, and that list was once considered the "official" place to discuss issues related to proposed changes to standards. However, there were complaints that long threads like this one overwhelmed people's inboxes and it was suggested that they would be better documented in issue tracker comments like this, which also allow people to opt in or out of following them.
I think that using GitHub issues comments like this is a significant improvement over using the email list. If I am to busy on a given day to read everything that comes in as notifications, at a later time I can just go to the issue tracker page and scroll through the thread.
However, I agree that there are problems with the system once discussions veer away from direct comments about the proposal at hand. To some extent, that should probably be handled by opening a new issue more directly related to the other issues being discussed. But that still has the problem that discussions only include people who are paying attention to that repo or who are tagged in the thread. For example, I don't watch either the OBIS or GBIF repos, so I am unaware of discussions taking place there.
I would like to see some more attention be paid to how people can opt in to a more general system for discussion. That used to be tdwg-content, but with Slack, Twitter, the TDWG newsletter, and GitHub issues, the platforms that are used for communication are much more diffuse, with many people preferentially following some but not all of the venues. Despite the negative aspects of tdwg-content that I mentioned, it did have the advantage that everyone knew about it, everyone had email, and anyone could sign up for it. That isn't really the case for GitHub, Twitter, and Slack, which all require signing up for an account. I'm actually not sure how one gets subscribed to the TDWG newsletter. I get it, but don't know how I got subscribed, and one can't just post to it as one can in the other platforms, so it isn't really a discussion venue.

I really appreciate the rigor to which you have all been hammering out this change. I am very strongly supportive of how this change season has been put on here in the GitHub Issues for all to engage with and I thank you all for putting this crucial work in. I've been reading this thread specifically, to see where consensus might emerge, and whether I can still see a fit for my use case in the proposed solutions. That use case is basically: An animal acoustically detected and identified to the individual level by an electronic code recorded at a series of listening stations around the world. (MOTUS will have a similar model on the bird side of things)
Since these listening stations can sometimes decode random noise into a numeric code, and that code can match to a tag deployed on another animal somewhere in the world, there are filters that researchers apply to assess the likelihood that any given ping is truly of their tagged individual (who has for the sake of my use case, been identified taxonomically and issued an organismID at the point of their being tagged).
Since those filters are often not conclusive, I've got a range of possibilities when reporting (or not reporting) these detections that are flagged by filters. The occurrenceStatus field, if not one of the ways I should accomplish this reporting of imprecision (though not temporal, spatial, or taxonomic imprecision!), is at least a place where I have to make absolutely sure I don't mislead anyone. I wasn't sure how to apply the originally proposed changes to my niche example, and I figure there are possibly other ways to make my records speak clearly about false detections, but if there's a task team coming together and looking for edge cases to explore, ✋ .

@jdpye Fantastic eye-opening use case for Organisms (individuals even) that may or not be there, and why you would care to say something if you weren't sure they were there. This suggests that the controlled vocabulary proposal (https://github.com/tdwg/dwc/issues/342) is insufficient even without the issues about distinctions between Organism and Taxon, and the limits on the scope of Organism. This also supports the need for a Task Group.

this thread belongs elsewhere to attract the visibility it deserves

As a relative newcomer to TDWG, where is that exactly? I'm not trying to be difficult and I don't disagree that this has veered far from the original purpose. I've just had a hard time figuring out where the core discussions happen around the standard. Some discussions seem to happen in the GBIF Github, some discussions in the OBIS Github, some (but fewer) discussions in the TDWG Q & A repo, very few on the listserve (or maybe I'm not on the right one). This is actually the most discussion about the standard I've seen is only on issues where changes are proposed. Where should this discussion happen to get the most visibility?

The TDWG Darwin Core Q&A repository (and associated Darwin Core Hour of live seminars) was developed exactly for the purpose you described - for uncertainties about the standard to generate discussions that can be summarized in answers, documented, and inform changes (if necessary, and embodied in change requests here in this repository) to the standard.

At this point in the review process, my assessment is that will not achieve consensus on this issue as proposed. A more comprehensive solution (potentially involving two terms and their respective vocabularies) will be required and that is a perfect job for a task group.
Even so, I believe we can achieve consensus on an alternative, non-normative change proposed by @timrobertson100. I would appreciate it if we could assess that fully in the hopes of including it in this round of changes. To that end I have created a new term change issue. Please comment there on that specific proposal.

I am very strongly in favor of the original direction @tucotuco had been pushing this. I desperately hope we (TDWG community) are moving in the direction of @tucotuco's "bigger dream", and that following the proposal of @timrobertson100 now is understood to be only a pragmatic solution to simplify the path immediately in front of us, and that this issue will need to be resolved more robustly in the (not-too-distant) future.

I am tired of kicking cans on everything and allowing issues to pile up. I don't really have massive amounts of free time to devote to all of this, but I think it needs to be addressed sooner rather than later.

@Jegelewicz I understand the frustration, but we are following a community-defined and accepted practice and consensus is a part of that process. We can seek consensus, but we can't force it. The reason for deferring the proposal and recommending the task group for it is so it doesn't hold up the 39 other proposals in this massive cleanup effort for which there is consensus.

This proposal has been labeled as 'Controversial' and in need of a task group to for resolution. It is no longer part of an active public review.


[BUG] Fix binding of NULL value parameters in prepared statements

Hello.
I tried to use the pgcat in a project of mine and noticed that it is impossible to use the transaction mode with anonymous prepared statements with parameters under some conditions.
When I started digging, I've found out there is an error defined as follows:
pgcat_1             |   16:     0x56003524c7f9 - <pgcat::messages::Bind as core::convert::TryFrom<&bytes::bytes_mut::BytesMut>>::try_from::h6544c9b6f20afed0
pgcat_1             |   17:     0x56003514beb2 - pgcat::client::Client<S,T>::rewrite_bind::{{closure}}::hc9caf116b38777a6
pgcat_1             |   18:     0x560035150374 - pgcat::client::Client<S,T>::handle::{{closure}}::h067dca183cd6d153
pgcat_1             |   19:     0x560035148981 - pgcat::client::client_entrypoint::{{closure}}::h2c1c12e23450a320
pgcat_1             |   20:     0x560035189d20 - pgcat::main::{{closure}}::{{closure}}::h449e3005ae55b1cc

It occurred only when there has been a parameterized query and one of the parameters has been NULL.
Postgresql documentation states that there is one special occasion when the reported parameter length is -1: that is NULL value.

The length of the parameter value, in bytes (this count does not include itself). Can be zero. As a special case, -1 indicates a NULL parameter value. No value bytes follow in the NULL case.

BytesMut::with_capacity(param_len as usize);

is unusable provided param_len <= 0.
Please accept the PR or modify it in accordance to your code conventions (not a Rust programmer here) to be able to use parameterized statements consistently.

Great catch, thank you!


Improve UI Tests

The UI tests have become messy and confusing. I'm not convinced some of the tests are actually testing what should be being tested. I want to improve the quality of the UI tests to ensure they are robust.
There are also a lot of tests missing that should be created.
Implementations should be broken down into separate tests.

Improved tests for actions modal: https://github.com/johneastman/todo/commit/2906dd202fe01bb87411ed7321cc8caad8a88857

Moved to wiki


Some ideas for additional checks

There are guest accounts in the tenant that have not logged in for a long time
There are users in the tenant that have not logged in for a long time
There are admins in the tenant which have not logged in for a long time

Goodmorning @joostgrunwald ,
You can make custom inspectors regarding this request.
Your additional checks aren't really SMART defined.
Could you please define them SMART so I can see if there is a script available you can use?
Thanks in advance.
~LvdW


Create a project within readthedocs

Done, see http://wagtail-personalisation.readthedocs.io/en/latest/


Use bytes for file field on attachment

The idea of using the Bytes struct comes from being able to have more flexibility when adding attachments.
Quoting the docs it mentions that

Bytes values facilitate zero-copy network programming by allowing multiple Bytes objects to point to the same underlying memory. This is managed by using a reference count to track when the memory is no longer needed and can be freed.

I would honestly just add an impl Into<Bytes> for the from_bytes function but it would stop being const so it would also require a breaking change.

Ideally the struct would take &'a [u8] but at the time there were too many lifetime issues that I could not sort out. I went with Vec because I needed to release the change, and it was a temporary fix that ended up lasting longer than it should have. Since Bytes is still the same amount of allocations when using borrowed data, I don't see this change being necessary at the moment.

Since this isn't a change we are going to be making soon, I'm going to give this a close.


provide some ENV to set crossorigin attribute for JS chunk script tags

Is your proposal related to a problem?
I am using CRA for a website, which serve its script via CDN, by set PUBLIC_URL to CDN host , everthing works fine . But soon I realized  its imposiible for sentry-like tools to get Error detail for the crossOrigin attribute is not 'annoymous' in script tag.

Describe the solution you'd like

Just like  what we do with PUBLIC_URL , maybe it is convient to provide a ENV key like 'SCRIPT_ACCROSS_ORIGIN' ?
Or maybe more convinent is to detect  whether the PUBLIC_URL is an absolute path , and if does ,automatically generate scipt tag with acrossorigin="annoymous" ?
Describe alternatives you've considered

I have to turn my project into a  react-app-rewired or craco  version  to modify webpack config to achieve my goal;
Or write some ugly script to parse and modify my generated index.html  after build;
Additional context

There is a same issue  but soon closed with no solution


using with spring boot

As spring boot has its own task to create an executable jar I was wondering if this plugin will work with the bootRepackage task out of the box

Spring boot extends the gradle application plugin which is what gradle-docker is using so just add the spring boot plugin and the docker plugin and it'll work out of the box. Here's (relevant parts of) my build.gradle:
buildscript {
    repositories {
        mavenCentral()
    }
    dependencies {
        classpath "org.springframework.boot:spring-boot-gradle-plugin:1.2.2.RELEASE"
        classpath "se.transmode.gradle:gradle-docker:1.2"
    }
}

apply plugin: "spring-boot"
apply plugin: "docker"

applicationDefaultJvmArgs = [
        "-Dspring.profiles.active=docker,reverse-proxy"
]

distDocker {
    exposePort 8080
}

Is there any way if to tell distDocker don't use the spring-boot functionality ...i mean like ADD <>.tar file and ENTRYPOINT.
by default it keep on adding to distDocker even if we want to use war file and run with standlone tomcat container.

You could replace the distDocker task instead of extending it with task distDocker (overwrite: true)
See https://gradle.org/docs/current/userguide/more_about_tasks.html


[feature] [v0.2] [comet] Group spiketrains for testing

Hey!
@rgutzen and I have been thinking a lot about how to group spiketrains into some tests.
In our use case, we have a long list of spiketrains which were recorded in different days and sessions. These is fine for single unit kind of tests (FR, CV, LV, timescale...), but not so for pairwise tests. We want to apply the following pairwise tests:
a) Correlation coefficients (and actually take the mean/std to collapse it into a single unit metric)
b) Spike triggered population response (stPR, single unit metric versus the rest of the population!) (currently being implemented into elephant)
Grouping the spiketrains by date or area is relatively easy, because this information is included in the annotations. However, the main problem is that we do not know what is the best way to integrate it into NetworkUnit/SciUnit. We have considered the following options, but none of them has convinced us:

Create a grouped_test_xxx class that would inherit the corresponding pairwise test and overwrite the generate_predictions. This is quite blunt, since it requires rewriting all the tests of interest (so far only two, but maybe some other tests join the pairwise club) and is therefore not very generalizable.

Create a decorator to do the grouping before generate_predictions. This option has some potential. The decorator would add some boilerplate before and after the function, first to group the spiketrains then run the predictions in a loop and finally merge them back. The advantage is that it would not require editing the original tests. One big problem however is that the way that NetworkUnit is structured it produces the spiketrains inside of generate_predictions, which would be required for the grouping. So that if a decorator is to be used it would require separating this function, which is definitely not ideal! Following from this line some sort of hacky solution could exists like making the generate_predicitions call some other function and decorate that one.

Run the correlation for all spiketrain pairs and select only the relevant values. Unnecessary waste of computation, would not work for stPR anyway.

Add an option into the relevant tests to do this. Same as writing a custom inheriting test, not very general, but maybe it is cleaner.

Split the model instance into grouped child instances. This has some issues like how the model instance links to the memory/disk. However, if completely new instances are created for which the predictions are generated and then they are merged again. Not certain where this would have to happen (which is actually the point of this whole issue). I would suggest building this as a new test class, however due to the splitting into grouped model instances it would not need tweaking the original test code, which makes it more general. Might have to discuss this one more in detail.

In all, I would say that option 1 is the best so far even though it is not generalizable at all. Option 5 also has a lot of potential in my opinion, and if executed cleanly (such that the grouped model child instances are not messing with the parent) it can actually be very generalizable, but many open questions remain.
I believe this issue is quite fundamental to any pairwise metric, and can actually be a very useful feature not only in the context of our project.
@rgerkin please let us know if you have any thoughts on this :)
Cheers,
Aitor

As it stands now, is a single spike train the observation that is uses to instantiate the Test for most tests?  And does the observation contain a pair of spike trains for things like a correlation test?   If so, I would extend the observation to be a group of many spike trains, but with annotations.  You can create an observation_schema which validate_observation will use to make sure that what you have provided is reasonable.  For example, it might allow a list of spike trains, or also allow a dictionary where keys are group names and values are list of spike trains for that group.
Then I would have one general method to operate within a group, to produce something like an NxN correlation matrix (or whatever).  Another method could collapse this to a single mean/std pair, and another method could aggregate across mean/std pairs.

The application we have in mind is specific to a capability ProducesSpiketrains, so there is a call to the corresponding produce_spiketrains() function in  generate_prediction(). Thus, predictions are generally computed based on a list of spiketrains.
A normal test for a pairwise measure (e.g. correlation) would compute the NxN matrix, plus eventual further processing. Adding grouping according to some attribute of the spiketrains would require to already change the matrix creation somehow to generate a list of matrices [N1xN1; N2xN2; ...].
As I'm realizing now, a general solution might be to perform the grouping already in the function produce_spiketrains() which is defined in the model, and adapt the generate_prediction() function of the eligible tests to also accept lists of lists of spiketrains.
The remaining issue would just be how to elegantly invoke compute_score() for each group's prediction and observation (or 2nd predictions, since we are working mostly with M2MTests).

This sounds reasonable, but be sure to make the structure of whatever produce_spiketrains() returns consistent across test types that might require it.

The most sensible solution we could come up with is implemented in commit fc7817b5c855412fe3d9ac315f70faade850d062 which introduces the capability ProducesListsOfSpikeTrains which is used in the average_correlation_test.

Alternatively, you don't need to generate a entirely new Capability class just for a related kind of output; you can just add that method to the original one.  By default it could be:
class ProducesSpikeTrains(sciunit.Capability):
    def produce_spiketrains(self):
        raise NotImplementedError("")

    def produce_listsofspiketrains(self):
        raise NotImplementedError("")

or if you want a default implementation:
class ProducesSpikeTrains(sciunit.Capability):
    def produce_spiketrains(self):
        raise NotImplementedError("")

    def produce_listsofspiketrains(self):
        return list(self.produce_spiketrains())

which the model would then override.
The question of "how related" two capability methods have to be to belong in the same capability class is an open one, and I'm not saying your choice was wrong, but this is something to think about when you have this kind of situation.

@rgerkin, thanks, that's a good point. These capabilities are definitely related enough since the list of spiketrain-lists can always default to a list with a single element. I will update the scripts.

Awesome! :)


Add mean vessel radius to CSV output

From an user's message:
" (..) could you add mean vessel radius to the csv output?  I also am not sure how to use the histogram bin info, so maybe we can discuss that sometime soon."

Already filed in #106


🌐 Fix markdown error in Korean translation for docs/ko/docs/tutorial/*.md

I corrected minor Markdown syntax errors and fixed bold formatting as per the original text.
also modified some typo.
(most problem appeared by markdown, because of korean postposition grammer. anyway maybe it will be better)

Korean tutorial text is mostly broken.
This is why I started working on this, but there are still a lot of mistakes.
(i will do it again)

i did some mistake ( will fix😅)

Sorry for creating so much documentation.
(I should have broken up the work like the others.... 😅)
anyway, fixed all the grammatical markdown errors. It should be much more readable than before.
so, it's time to review & merge.
+)
I started doing this after reading Rafael's post in dev.to so I'll keep growing and making fewer mistakes!

If someone could clean up the expired comments by github-action about page, I'd appreciate it.

hi could you check my PR for fast-api-docs? Most docs are broken, so very hard to read
@hard-coders @jujumilk3 @junah201 @KaniKim

LGTM!

Hi @sodaMelon,
This was handled in PR #12509, so I'm closing this one, but thanks for the effort and interest! 🙌


Bug-When carousel active

i have found :
when carousel active all other relative or absolute text item(especially colored text ) will twinkling and blur .
i used:
AngularJS v1.4.7
angular-bootstrap v0.14.3
pls fix it !

@duongtuan1793, If you think you've found a bug, please follow these instructions.
Please be respectful of our time by fully clarifying exactly what your issue is and give us a reproducible plunker.  What you've typed above is not very helpful.  Thanks.
We will consider reopening when we have something to verify.


Contextual lexer fails to resolve ambiguity

Describe the bug
In a very specific circumstance, contextual lexer seems to resolve the wrong token.
To Reproduce
This is minimal reproducible example which I reduced from my actual use case:
grammar_str = '''

var_name : NAME

reaction: "***" var_name SEP_1
SEP_1: ";"

small_stmt : reaction
    | var_name
SEP_2 : ";"

root : small_stmt SEP_2

%import common.CNAME -> NAME
%import common.WS_INLINE
%ignore WS_INLINE

'''
from lark import Lark
lark = Lark(grammar_str, start='root', parser='lalr', lexer='contextual')
lark.parse('a;')

Output:
lark.exceptions.UnexpectedToken: Unexpected token Token('SEP_1', ';') at line 1, column 2.
Expected one of:
        * SEP_2

Note that if I remove the redundant var_name rule, the bug is gone. I can also add a priority to SEP_2 to fix the issue. But neither should be necessary, since there is no ambiguity here, right?
Additional note: I tried walking through the code in Lark. It seems that the lexer first output the correct token (Token('SEP_2')), but then after the token is passed into UnlessCallback, it is converted back into Token('SEP_1').
Finally, with my full grammar file, adding priority doesn't seem to work. Here is my original grammar:

var_name : NAME
    | "$" NAME
in_comp : "in" var_name
namemaybein : var_name [in_comp]

reaction_name : namemaybein ":"
reaction : [reaction_name] species_list ARROW species_list REACTION_SEP sum [in_comp]
species_list : species ("+" species)*
species : [NUMBER] var_name
ARROW : "->"
    | "=>"
REACTION_SEP : ";"

assignment : namemaybein "=" sum

annotation : var_name ANNOT_KEYWORD ESCAPED_STRING
ANNOT_KEYWORD .2: "identity" | "hasPart"   // use .2 to specify higher priority when matching

declaration : decl_modifiers decl_item ("," decl_item)*
decl_modifiers : VAR_MODIFIER
    | TYPE_MODIFIER
    | VAR_MODIFIER TYPE_MODIFIER
decl_item : namemaybein [decl_assignment]
decl_assignment : "=" sum

VAR_MODIFIER .2: "var" | "const"
TYPE_MODIFIER .2: "species" | "compartment" | "formula"

?sum : product
    | sum "+" product
    | sum "-" product

?product : power
    | product "*" power
    | product "/" power

?power: atom
    | power "^" atom

?atom : NUMBER
    | NAME
    | "-" atom                  -> neg
    | "(" sum ")"

?simple_stmt : [small_stmt] STMT_SEPARATOR
?small_stmt : reaction
    | assignment
    | declaration
    | annotation

STMT_SEPARATOR .2: ";"
    | NEWLINE

// a newline or a semicolon
model : ("model" | "module") NAME simple_stmt+ "end"

root : (simple_stmt | model)*

%import common.CNAME -> NAME
%import common.NUMBER
%import common.DIGIT
%import common.LETTER
%import common.WS
%import common.WS_INLINE
%import common.NEWLINE
%import common.ESCAPED_STRING
%ignore WS_INLINE

Parsing code const a; fails with
lark.exceptions.UnexpectedToken: Unexpected token Token('REACTION_SEP', ';') at line 2, column 8.
Expected one of:
        * STMT_SEPARATOR
        * EQUAL
        * COMMA
        * IN

Yes, this is a limitation of LALR(1). Calling FOLLOWS(var_name) returns {SEP_1, SEP_2}. It doesn't know how contextualize it for each path it takes.
But what I can't figure out, is why do you need both REACTION_SEP and STMT_SEPARATOR? Why not just use one name, or keep the semicolon anonymous? Merging them would certainly solve the issue.

You're right, it's not at all necessary now that I think about it. Thanks for clarifying though.


Adapt deposit and withdraw txs to new specs

Add common tests

I will address TTLs and from's type in a subsequent PR


removed implicit {} from inatepowers target

The implicit addition of { and } will prevent the usage of ANY, COSTAL, INVADERS and others.
using any will show the any-element

In order to keep the interface intuitive for the users (not requiring them to know when to use "{}" and when not to use them, I have implemented a fix in the latest commit that should resolve this issue. As such, this pull request will be closed.


Fixed the path management of tar command up on the case of internet download

If the MPI package is downloaded from the internet not blob, its path should be managed differently.

**MVAPICH MPI**
[OpenLogic CentOS 7.7 latest] [LISAv2 Test Results Summary]
[OpenLogic CentOS 7.7 latest] Test Run On           : 10/03/2019 23:54:32
[OpenLogic CentOS 7.7 latest] ARM Image Under Test  : OpenLogic : CentOS : 7.7 : latest
[OpenLogic CentOS 7.7 latest] Initial Kernel Version: 3.10.0-1062.1.1.el7.x86_64
[OpenLogic CentOS 7.7 latest] Final Kernel Version  : 3.10.0-1062.1.1.el7.x86_64
[OpenLogic CentOS 7.7 latest] Total Test Cases      : 1 (1 Passed, 0 Failed, 0 Aborted, 0 Skipped)
[OpenLogic CentOS 7.7 latest] Total Time (dd:hh:mm) : 0:4:15
[OpenLogic CentOS 7.7 latest]
[OpenLogic CentOS 7.7 latest]    ID TestArea             TestCaseName                                                                TestResult TestDuration(in minutes)
[OpenLogic CentOS 7.7 latest] -------------------------------------------------------------------------------------------------------------------------------------------
[OpenLogic CentOS 7.7 latest]     1 INFINIBAND           INFINIBAND-MVAPICH-MPI-2VM                                                        PASS               249.44
[OpenLogic CentOS 7.7 latest] 	InfiniBand-Verification-1-FirstBoot : eth0 IP : PASS
[OpenLogic CentOS 7.7 latest] 	InfiniBand-Verification-1-FirstBoot : IBV_PINGPONG : PASS
[OpenLogic CentOS 7.7 latest] 	InfiniBand-Verification-1-FirstBoot : PingPong Intranode : PASS
[OpenLogic CentOS 7.7 latest] 	InfiniBand-Verification-1-FirstBoot : IMB-MPI1 : PASS
[OpenLogic CentOS 7.7 latest] 	InfiniBand-Verification-1-FirstBoot : IMB-P2P : PASS
[OpenLogic CentOS 7.7 latest] 	InfiniBand-Verification-1-FirstBoot : IMB-IO : PASS
[OpenLogic CentOS 7.7 latest] 	InfiniBand-Verification-1-FirstBoot : IMB-RMA : PASS
[OpenLogic CentOS 7.7 latest] 	InfiniBand-Verification-1-FirstBoot : IMB-NBC : PASS
[OpenLogic CentOS 7.7 latest] 	InfiniBand-Verification-2-Reboot : eth0 IP : PASS
[OpenLogic CentOS 7.7 latest] 	InfiniBand-Verification-2-Reboot : IBV_PINGPONG : PASS
[OpenLogic CentOS 7.7 latest] 	InfiniBand-Verification-2-Reboot : PingPong Intranode : PASS
[OpenLogic CentOS 7.7 latest] 	InfiniBand-Verification-2-Reboot : IMB-MPI1 : PASS
[OpenLogic CentOS 7.7 latest] 	InfiniBand-Verification-2-Reboot : IMB-P2P : PASS
[OpenLogic CentOS 7.7 latest] 	InfiniBand-Verification-2-Reboot : IMB-IO : PASS
[OpenLogic CentOS 7.7 latest] 	InfiniBand-Verification-2-Reboot : IMB-RMA : PASS
[OpenLogic CentOS 7.7 latest] 	InfiniBand-Verification-2-Reboot : IMB-NBC : PASS
[OpenLogic CentOS 7.7 latest]
[OpenLogic CentOS 7.7 latest]
[OpenLogic CentOS 7.7 latest] Logs can be found at C:\LISAv2\AY29\TestResults\2019-03-10-23-54-28-3284
[OpenLogic CentOS 7.7 latest]
[OpenLogic CentOS 7.7 latest]
[OpenLogic CentOS 7.7 latest] 10/04/2019 04:10:13 : [INFO ] Creating 'C:\LISAv2\AY29\Azure-AY29-TestLogs.zip' from 'C:\LISAv2\AY29\TestResults\2019-03-10-23-54-28-3284'
[OpenLogic CentOS 7.7 latest] 10/04/2019 04:10:16 : [INFO ] C:\LISAv2\AY29\Azure-AY29-TestLogs.zip created successfully.
[OpenLogic CentOS 7.7 latest] 10/04/2019 04:10:16 : [INFO ] Analyzing results..
[OpenLogic CentOS 7.7 latest] 10/04/2019 04:10:16 : [INFO ] Copying all files back to original working directory: C:\jenkins-slave-executor-2\workspace\Azure\pipeline-cloudtest-manual@2.
[OpenLogic CentOS 7.7 latest] 10/04/2019 04:10:20 : [INFO ] Cleaning up C:\LISAv2\AY29
[OpenLogic CentOS 7.7 latest] 10/04/2019 04:10:21 : [INFO ] Setting workspace back to original location: C:\jenkins-slave-executor-2\workspace\Azure\pipeline-cloudtest-manual@2
[OpenLogic CentOS 7.7 latest] 10/04/2019 04:10:21 : [INFO ] LISAv2 exit code: 0
Intel MPI
[OpenLogic CentOS 7.7 latest] [LISAv2 Test Results Summary]
[OpenLogic CentOS 7.7 latest] Test Run On           : 10/03/2019 23:54:06
[OpenLogic CentOS 7.7 latest] ARM Image Under Test  : OpenLogic : CentOS : 7.7 : latest
[OpenLogic CentOS 7.7 latest] Initial Kernel Version: 3.10.0-1062.1.1.el7.x86_64
[OpenLogic CentOS 7.7 latest] Final Kernel Version  : 3.10.0-1062.1.1.el7.x86_64
[OpenLogic CentOS 7.7 latest] Total Test Cases      : 1 (1 Passed, 0 Failed, 0 Aborted, 0 Skipped)
[OpenLogic CentOS 7.7 latest] Total Time (dd:hh:mm) : 0:2:41
[OpenLogic CentOS 7.7 latest]
[OpenLogic CentOS 7.7 latest]    ID TestArea             TestCaseName                                                                TestResult TestDuration(in minutes)
[OpenLogic CentOS 7.7 latest] -------------------------------------------------------------------------------------------------------------------------------------------
[OpenLogic CentOS 7.7 latest]     1 INFINIBAND           INFINIBAND-INTEL-MPI-2VM                                                          PASS                157.4
[OpenLogic CentOS 7.7 latest] 	InfiniBand-Verification-1-FirstBoot : eth0 IP : PASS
[OpenLogic CentOS 7.7 latest] 	InfiniBand-Verification-1-FirstBoot : IBV_PINGPONG : PASS
[OpenLogic CentOS 7.7 latest] 	InfiniBand-Verification-1-FirstBoot : PingPong Intranode : PASS
[OpenLogic CentOS 7.7 latest] 	InfiniBand-Verification-1-FirstBoot : IMB-MPI1 : PASS
[OpenLogic CentOS 7.7 latest] 	InfiniBand-Verification-1-FirstBoot : IMB-IO : SKIPPED
[OpenLogic CentOS 7.7 latest] 	InfiniBand-Verification-1-FirstBoot : IMB-RMA : PASS
[OpenLogic CentOS 7.7 latest] 	InfiniBand-Verification-1-FirstBoot : IMB-NBC : PASS
[OpenLogic CentOS 7.7 latest] 	InfiniBand-Verification-2-Reboot : eth0 IP : PASS
[OpenLogic CentOS 7.7 latest] 	InfiniBand-Verification-2-Reboot : IBV_PINGPONG : PASS
[OpenLogic CentOS 7.7 latest] 	InfiniBand-Verification-2-Reboot : PingPong Intranode : PASS
[OpenLogic CentOS 7.7 latest] 	InfiniBand-Verification-2-Reboot : IMB-MPI1 : PASS
[OpenLogic CentOS 7.7 latest] 	InfiniBand-Verification-2-Reboot : IMB-IO : SKIPPED
[OpenLogic CentOS 7.7 latest] 	InfiniBand-Verification-2-Reboot : IMB-RMA : PASS
[OpenLogic CentOS 7.7 latest] 	InfiniBand-Verification-2-Reboot : IMB-NBC : PASS
[OpenLogic CentOS 7.7 latest]
[OpenLogic CentOS 7.7 latest]
[OpenLogic CentOS 7.7 latest] Logs can be found at C:\LISAv2\AT80\TestResults\2019-03-10-23-53-59-6062
[OpenLogic CentOS 7.7 latest]
[OpenLogic CentOS 7.7 latest]
[OpenLogic CentOS 7.7 latest] 10/04/2019 02:35:47 : [INFO ] Creating 'C:\LISAv2\AT80\Azure-AT80-TestLogs.zip' from 'C:\LISAv2\AT80\TestResults\2019-03-10-23-53-59-6062'
[OpenLogic CentOS 7.7 latest] 10/04/2019 02:35:47 : [INFO ] C:\LISAv2\AT80\Azure-AT80-TestLogs.zip created successfully.
[OpenLogic CentOS 7.7 latest] 10/04/2019 02:35:47 : [INFO ] Analyzing results..
[OpenLogic CentOS 7.7 latest] 10/04/2019 02:35:47 : [INFO ] Copying all files back to original working directory: C:\jenkins-slave-executor-3\workspace\Azure\pipeline-cloudtest-manual.
[OpenLogic CentOS 7.7 latest] 10/04/2019 02:35:51 : [INFO ] Cleaning up C:\LISAv2\AT80
[OpenLogic CentOS 7.7 latest] 10/04/2019 02:35:51 : [INFO ] Setting workspace back to original location: C:\jenkins-slave-executor-3\workspace\Azure\pipeline-cloudtest-manual
[OpenLogic CentOS 7.7 latest] 10/04/2019 02:35:51 : [INFO ] LISAv2 exit code: 0

Open MPI has also passed the file path check, but Open MPI 4.0.1 has MPI issue not the automation. No result available for Open MPI. I verified the successful logs though.

@LiliDeng @lizzha Can any of you sign off this PR? I can merge.

@juhlee-microsoft  After change, the code can work against blob and internet?

@juhlee-microsoft After change, the code can work against blob and internet?

Yes, I verified and post a few test results here. Two different cases have different number of directory path, so it should be handling differently.


Reading .yaml configuration from classpath using current threads classloader

The SnoopProducer reads the config .yaml file from classpath by using this.getClass().getClassloader() which IMHO is not appropriate within JEE. This should be done by using the current threads classloader. This way you don't make any assumptions whether the classloader which loaded the SnoopProducer is still available for loading the config.
Please see: http://jeecookbook.blogspot.de/2013/07/java-why-you-should-not-use.html

Thanks @svendiedrichsen


nixos-rebuild should enable flakes automatically when /etc/nixos/flake.nix is found

Describe the bug
nixos-rebuild cannot build a system flake in a system with flakes disabled unless explicitly passed the --flake option.
Steps To Reproduce
Steps to reproduce the behavior:

Have a flake-less NixOS.
Create a system flake in /root/os-flake.
Link it: ln -s /root/os-flake/flake.nix /etc/nixos
Run nixos-rebuild build

Error:
building the system configuration...
error: experimental Nix feature 'nix-command' is disabled; use '--extra-experimental-features nix-command' to override

OK, if you do that and try with this, it will fail too:
# nixos-rebuild build --extra-experimental-features "nix-command flakes"
/run/current-system/sw/bin/nixos-rebuild: unknown option `--extra-experimental-features'

Expected behavior
As NixOS detected /etc/nixos/flake.nix file, it should enable flakes support automatically.
Additional context
Workaround:
nixos-rebuild build --flake /root/os-flake

Notify maintainers

@Profpatsch
Metadata
[user@system:~]$ nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-linux"`
 - host os: `Linux 5.15.53, NixOS, 22.05 (Quokka), 22.05.1604.cf034a867e0`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.8.1`
 - channels(root): `"nixos-22.05"`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

@moduon MT-904

Yikes, I don't know what happened. Sorry!
I'll close this one, as the other one was triaged.


Draft: various small JSON schema fixes

Change Summary

Include default_factory results in default for json schema
Omit BaseModel docstring from json schema description due to verbosity

Related issue number
Fix #8350
Fix #8348
Checklist

[x] The pull request title is a good summary of the changes - it will be used in the changelog
[x] Unit tests for the changes exist
[x] Tests pass on CI
[ ] Documentation reflects the changes where applicable
[x] My PR is ready to review, please add a comment including the phrase "please review" to assign reviewers

Please review


bug: fix regression preventing -trimpath and trim of debug symbols

Fixes #181
I haven't validated it yet. I might need to split -tags from nobadger.

@wazerstar try the CI artifacts here: https://github.com/caddyserver/xcaddy/actions/runs/8971616569?pr=182

Tested

2024/05/06 17:59:27 [INFO] absolute output file path: C:\Users\Denni\Desktop\xcaddy\caddy.exe
2024/05/06 17:59:27 [INFO] Temporary folder: C:\Users\Denni\AppData\Local\Temp\buildenv_2024-05-06-1759.791475689
2024/05/06 17:59:27 [INFO] Writing main module: C:\Users\Denni\AppData\Local\Temp\buildenv_2024-05-06-1759.791475689\main.go
package main

import (
        caddycmd "github.com/caddyserver/caddy/v2/cmd"

        // plug in Caddy modules here
        _ "github.com/caddyserver/caddy/v2/modules/standard"
)

func main() {
        caddycmd.Main()
}
2024/05/06 17:59:27 [INFO] Initializing Go module
2024/05/06 17:59:27 [INFO] exec (timeout=0s): C:\Program Files\Go\bin\go.exe mod init caddy
go: creating new go.mod: module caddy
go: to add module requirements and sums:
        go mod tidy
2024/05/06 17:59:27 [INFO] Pinning versions
2024/05/06 17:59:27 [INFO] exec (timeout=0s): C:\Program Files\Go\bin\go.exe get -d -v github.com/caddyserver/caddy/v2@master
go: added github.com/beorn7/perks v1.0.1
go: added github.com/caddyserver/caddy/v2 v2.8.0-beta.1.0.20240502000919-7e2510ef43d3
go: added github.com/caddyserver/certmagic v0.20.1-0.20240423172519-140a6fa9202e
go: added github.com/caddyserver/zerossl v0.1.2
go: added github.com/cespare/xxhash/v2 v2.2.0
go: added github.com/go-task/slim-sprig v0.0.0-20230315185526-52ccab3ef572
go: added github.com/google/pprof v0.0.0-20231212022811-ec68065c825e
go: added github.com/google/uuid v1.6.0
go: added github.com/klauspost/cpuid/v2 v2.2.7
go: added github.com/libdns/libdns v0.2.2
go: added github.com/mholt/acmez/v2 v2.0.0-beta.2
go: added github.com/miekg/dns v1.1.58
go: added github.com/onsi/ginkgo/v2 v2.13.2
go: added github.com/prometheus/client_golang v1.19.0
go: added github.com/prometheus/client_model v0.5.0
go: added github.com/prometheus/common v0.48.0
go: added github.com/prometheus/procfs v0.12.0
go: added github.com/quic-go/qpack v0.4.0
go: added github.com/quic-go/quic-go v0.43.0
go: added github.com/zeebo/blake3 v0.2.3
go: added go.uber.org/mock v0.4.0
go: added go.uber.org/multierr v1.11.0
go: added go.uber.org/zap v1.27.0
go: added go.uber.org/zap/exp v0.2.0
go: added golang.org/x/crypto v0.22.0
go: added golang.org/x/exp v0.0.0-20240325151524-a685a6edb6d8
go: added golang.org/x/mod v0.16.0
go: added golang.org/x/net v0.24.0
go: added golang.org/x/sys v0.19.0
go: added golang.org/x/term v0.19.0
go: added golang.org/x/text v0.14.0
go: added golang.org/x/time v0.5.0
go: added golang.org/x/tools v0.19.0
go: added google.golang.org/protobuf v1.33.0
2024/05/06 17:59:28 [INFO] exec (timeout=0s): C:\Program Files\Go\bin\go.exe get -d -v
2024/05/06 17:59:30 [INFO] Build environment ready
2024/05/06 17:59:30 [INFO] Building Caddy
2024/05/06 17:59:30 [INFO] exec (timeout=0s): C:\Program Files\Go\bin\go.exe mod tidy -e
2024/05/06 17:59:31 [INFO] exec (timeout=0s): C:\Program Files\Go\bin\go.exe build -o C:\Users\Denni\Desktop\xcaddy\caddy.exe -ldflags -w -s -trimpath -tags nobadger
2024/05/06 17:59:37 [INFO] Build complete: .\caddy.exe
2024/05/06 17:59:37 [INFO] Cleaning up temporary folder: C:\Users\Denni\AppData\Local\Temp\buildenv_2024-05-06-1759.791475689


intermediate change broke 'docker build'

golang:alpine does not include 'git', see https://hub.docker.com/_/golang/
in deference to https://github.com/coreos/container-linux-config-transpiler/pull/28#issuecomment-284691551
Dockerfile builds a useful image that runs ct
build-scratch with Dockerfile.build-scratch pares that down to just ct

Gonna let this sit for a day or so in case anyone else wants to comment on it

:+1:

@jkt628 this needs rebasing, I'll merge as soon as you can get around to that

done and tested.
is it possible to push the final image (coreos/ct) to hub.docker.com?  or, with this change, the remote build triggers would automagically push the intermediate image.

I don't think I want to automagically push the image, unless it supports only pushing tagged releases. I'm not opposed to putting this on the docker hub though, I'll ask and see if that can happen.

you can choose the tag pushed into docker hub.  'latest' is expected to be just that, a build from the latest commit.  release tags are more difficult i think but do not consider me an expert.
perhaps 'latest' could always be the intermediate image and 'release' could always be the final scratch image.

I would want latest to point to the latest release, not the latest commit. I could always just add "build a new docker image, put in docker hub" to the release process.


feat: moves Go code and resources to repository root

Removes Python code and /go

@yana1205 I added you as a reviewer for some earlier feedback. I made some changes to the README.md based on the new org changes and the older Go README.md. I am running into some issues running the unit tests locally and in GitHub Actions using act. I added a new workflow that would run unit tests and some sanity check with go fmt and go vet, but the TestKustomize  unit test is failing. Wondering if running the unit tests requires some specific environment setup that I am missing.

Wondering if running the unit tests requires some specific environment setup that I am missing.

Thank you for pointing that out. I missed to document the prerequisites. It requires Kustomize and policy-generator plugin.
# Kustomize
curl -s "https://raw.githubusercontent.com/kubernetes-sigs/kustomize/master/hack/install_kustomize.sh"  | bash

# PolicyGenerator Plugin
GOBIN=${HOME}/.config/kustomize/plugin/policy.open-cluster-management.io/v1/policygenerator \
go install open-cluster-management.io/policy-generator-plugin/cmd/PolicyGenerator@latest

I will update it.


Create pull_request_template.md - 31

[Auto generated]
Number: [#31]

openshift-virtualization-tests(Rwk3z)[PR 31]: Closing issue for PR: Create pull_request_template.md.
PR was closed.


Add custom filtering with "exclude" to work with nodes/relations

Codecov Report

Merging #34 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master      #34   +/-   ##
=======================================
  Coverage   85.98%   85.98%
=======================================
  Files          18       18
  Lines         942      942
  Branches      175      175
=======================================
  Hits          810      810
  Misses         96       96
  Partials       36       36

Impacted Files
Coverage Δ

pyrosm/pyrosm.py
93.43% <ø> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a3b6d45...720a3f5. Read the comment docs.


Improve rst/docs linting/checks

description
There are a number of issues with the docs that could benefit from a cleanup and some CI to prevent recurrence.
One good example is the use of single ` rather than double `` for inline-literals in rst
Currently we have a custom rst checker which only checks the version_history file and not any other part of the docs
One complexity here is that the rst is dynamically built from parts of the repo, and also the referential nature of rst itself, means that some checks can only be done after the rst has been constructed.
I think we could have some pre/post build checks - eg the current version_history check runs against the rst source in the repo.
Some checks would probably require at least a simulated build to detect in addition to the static checks that a linter might provide.
I have looked at rst linters in the past and not found any that are the perfect solution, there are a few and my takeaway was that we could probably use a bit of each, but that requires more research.
Like the problem of linting yaml (which is also a docs linting problem of sorts) there is the issue that if we add/raise standards without a clear way to fix, this can be frustrating to developers

cc @kfaseela

I think we should pick up a rst style, document it. As it is much complex, we can use specific lint tools for specific problems.


Getting error while running the experiment

I have provide the cosmos db operator role to my experiment and even owner for that matter however I keep getting error as The
resources='/subscriptions/9747c5ed-210d-4efe-b300-41774a61016d/resourceGroups/chaosstudiotest/providers/microsoft.documentdb/databaseaccounts/avengerscomos' referenced by the experiment could not be found.
Am i missing something ? or something missing in the documentation?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 15956d27-28c1-5e3f-158c-522f091daca7
Version Independent ID: 038e143e-a919-0401-324e-6e9cc0d1dc0c
Content: Create an experiment that uses a service-direct fault with Azure Chaos Studio
Content Source: articles/chaos-studio/chaos-studio-tutorial-service-direct-portal.md
Service: chaos-studio
GitHub Login: @johnkemnetz
Microsoft Alias: johnkem

@RawishCRM
Thanks for the feedback! We are currently investigating and will update you shortly.

Hello @alexbuckgit , @johnkemnetz Could you please help in this matter.

Hi @SatishBoddu-MSFT,
Thank you for reaching out to us! We appreciate your interest in improving our documentation.
The feedback channel on GitHub is designed for updating documentation with known information.
The best way to get help is to create a support ticket. Our support team is available to assist you with any questions or issues you may have. You can create a support ticket by visiting the following link: https://azure.microsoft.com/support/create-ticket/
Once you have received assistance from our support team, we would be happy to document the answer to your question in the appropriate article for future reference.
#please-close


Epic: expand e2e tests that use branching

I don't think that e2e tests are the right place for expanded tests for branches. At first, those tests already exist https://github.com/neondatabase/cloud/blob/main/tests_e2e/tests/test_project.py#L400 At second, I think e2e tests are to test integrations and basic error-out scenarios, not to test feature corner cases. IMO it is better to expand branching tests suite around the pageserver itself -- tests would be fester and they'll have more access to the internal state (files, sizes, manual GC runs, etc).
Let's rename this epic? =)

killed in favor of https://github.com/neondatabase/neon/issues/1899


Question: bitcoind sync time

Hi,
Do you need bitcoind to be fully synced with the network for LND?
If so, how much time does it take for it to download the full chain?
Thanks

Hi, yes. About from 2 to 4 weeks to download.

As Amiga500 said, yes you need Bitcoin client synced. Sync takes 4 weeks more le less.


Spacing vs combing encoding issues

https://github.com/TypeNetwork/Amstelvar/issues/193 reports a number of encoding issues; we should have checks for them (or better, checks that catch those and more of their nature)

Let's try and define a heuristic for this. @dscorbett, would you agree with the following?

Glyphs which are mapped to a Unicode codepoint which has a General Category of Mn (Mark, normal) should always have horizontal advance of 0.

Additionally, if there is a GDEF table and it contains a Glyph Class Definition Table, these glyphs should be defined as marks.

Glyphs which are mapped to a Unicode codepoint which has a General Category of Sk (Modifier symbol) should not have horizontal advance of 0.

Additionally, if there is a GDEF table and it contains a Glyph Class Definition Table, these glyphs should be defined as bases.

That is a good start. I would add that gc=Mn glyphs should have anchors to be positioned as marks (assuming the font uses GPOS), that gc=Mn glyphs should be drawn at negative x coordinates (or positive x coordinates for RTL scripts) so that in the absence of mark positioning they still overstrike the correct base, and that gc=Sk glyphs should not have anchors to be positioned as marks.
Of course, there are false positives. It is reasonable to give marks that don’t go directly above or below, like U+0315 COMBINING COMMA ABOVE RIGHT and U+11722 AHOM VOWEL SIGN I, positive horizontal advance widths. U+1171E AHOM CONSONANT SIGN MEDIAL RA has gc=Mn but behaves like gc=Mc.