Graph Adaptations of Variants, References, and Reads APIs

This pull request seeks to merge a first shot at adapting the Variants, References, and Reads APIs to a graph reverence into the graph branch, which is not the main master branch. They will sit there until they are ready to replace the current non-graph-capable API.
This PR also includes some commits that have been merged into master since the graph branch was created, but you can ignore those.
The biggest changes are in the Variants API, which has been converted to a monoallelic model. Additionally, the References API now has support for References branching off of other References in the same ReferenceSet, and the Reads API has support for reads that are aligned to multiple References without being chimeric (because they follow adjacencies in the graph of References).
Closes #189.

How could we get the genotype of a sample? The Genotype object only keeps the totalCopyNumber but not the copy number of each allele. Call keeps the copy number of one allele but is not linked to Genotype.

I'm not sure if Reference and ReferenceSet have been completely translated into the Graph world. For example, what does isDerived mean in the context of a general Reference (i.e., a string of sequence grafted into the graph somewhere)? Also, I don't think sourceURI has any real meaning for a complex graph ReferenceSet, since it won't have been built from a FASTA file.

This is great progress, thanks for all the work Adam. I'm happy with most of the changes, but I wonder if we can tidy up the relationship between what we'd think of as a 'reference' now and what a reference will be under the graph model. Here is an alternative suggestion for a subset of the class hierarchy:

A ReferenceSet is a set of Reference objects, representing a canonical reference, e.g., GRCh38 (same as current definition).
A Reference is a sequence graph providing a canonical description of the known variation in a single genomic region (for example, human chromosome 1). The sequence graph is composed of a set of Segments (but does not have the IDs of these segments embedded within it). For a linear reference, there is exactly one Segment.
A Segment is a continuous chunk of sequence, which joins with other Segments to form a graph (equivalent to your current Reference object).

Under this model, there's a clear mapping between the old and the new when it comes to Reference objects. By using the more generic name Segment (or Fragment or whatever we end up using) to refer to the building blocks of the graph, I think we make the constructions for Alleles a little clearer (I find the idea of adding a new piece of Reference for each novel allele a bit strange).
Any thoughts?

@lh3 the Genotype can store genotypeLikelihoods if you have them. Otherwise you have to get all the Calls in the CallSet linked to the Variant you want a genotype for. (In other words, a Genotype and the appropriate Calls are linked by association to the same Variant and CallSet combination.)

I'm not sure if Reference and ReferenceSet have been completely translated into the Graph world. For example, what does isDerived mean in the context of a general Reference (i.e., a string of sequence grafted into the graph somewhere)? Also, I don't think sourceURI has any real meaning for a complex graph ReferenceSet, since it won't have been built from a FASTA file.

You are right that isDerived is going to be a bit useless for many References in the graph model. However, I'm not sure exactly what it's useful for in the main branch either. What are some examples of derived references? If a reference is derived, where do we find the URI and accession numbers of the derived sequence versus those of its parent? I think we should document a use case for it on the main branch before we decide whether we need it on References in a graph, and I've created issue #231 to collect opinions.
As for sourceURI, I'll change its definition slightly so that it gives the URI from which the sequence can be obtained by the client. This gives it a meaningful interpretation for References that weren't created by just importing a FASTA file, and provides an easy way to work around paging issues with long sequence strings. It also means that the client is guaranteed to be able to actually obtain the sequence of a Reference, which doesn't seem to be the case on the master branch: just because the API server originally fetched a FASTA from somewhere doesn't mean it's still there.

This is great progress, thanks for all the work Adam. I'm happy with most of the changes, but I wonder if we can tidy up the relationship between what we'd think of as a 'reference' now and what a reference will be under the graph model. Here is an alternative suggestion for a subset of the class hierarchy:

A ReferenceSet is a set of Reference objects, representing a canonical reference, e.g., GRCh38 (same as current definition).
A Reference is a sequence graph providing a canonical description of the known variation in a single genomic region (for example, human chromosome 1). The sequence graph is composed of a set of Segments (but does not have the IDs of these segments embedded within it). For a linear reference, there is exactly one Segment.
A Segment is a continuous chunk of sequence, which joins with other Segments to form a graph (equivalent to your current Reference object).

Under this model, there's a clear mapping between the old and the new when it comes to Reference objects. By using the more generic name Segment (or Fragment or whatever we end up using) to refer to the building blocks of the graph, I think we make the constructions for Alleles a little clearer (I find the idea of adding a new piece of Reference for each novel allele a bit strange).

Any thoughts?

Basically, this sounds like dividing the genome into per-chromosome boxes called References, in which Segments are linked together into one graph per box. Is that right?
This would work, and depending on how the meaning of isDerived gets settled and the eventual (non)implementation of getSequence it might be the best way to go to strip superfluous fields that sequence in a VariantSet will never need. But it presents a couple of problems:

How do we deal with variants that connect chromosomes?
In a similar vein, what about things like the pseudoautosomal region on chrX and chrY?
Can References overlap and describe the same genomic region? Can a chromosome be split into multiple References (maybe one for the PAR and one for the rest of Y)? If so, how are they connected?
Does every piece of sequence in a Reference have to come from the same FASTA and have the same set of accessions? If not, do we have to put those fields on the Segment/Fragment again? What data would that leave in Reference?
Would a Reference make more sense as a Fragment wrapper?

I'd favor leaving the overall model as is; it's the minimal change we need to enable graphs to be represented. Each existing Reference can just stay as is, with no parent and nothing attached. If we solve these problems and determine that we really have some useless-for-variants fields on Reference as implemented, we can do another refactoring.

@lh3 the Genotype can store genotypeLikelihoods if you have them. Otherwise you have to get all the Calls in the CallSet linked to the Variant you want a genotype for. (In other words, a Genotype and the appropriate Calls are linked by association to the same Variant and CallSet combination.)

@adamnovak Thanks. I see what you mean now. I am trying to walk through a query "get the genotype of sample NA12878 at position chr1:1000". Is that something like this:
// get callSet; suppose we know there is only one result
// Maybe we should also allow to search multiple samples in one go?
callSet = searchCallSets({"variantSetIds":["foo"],"name":"NA12878"}).callSets[0];
// get variant. Could we get two variants at the same position? Let's assume not as it is too complex.
variant = searchVariants({"referenceName":"chr1","start":9999,"end":1000,"callSetId":callSet.id}).variants[0];
// get genotype. Does the line below always return one result?
genotype = searchGenotypes({"callSetIds":[callSet.id],"variantIds":[variant.id]}).genotypes[0];
// get calls; could be multiple. Does the list also include the reference allele/call?
calls = searchCalls({"callSetIds":[callSet.id],"variantIds":[variant.id]}).calls;
// get allele sequences.
alleleIds = [];
for (call in calls) alleleIds.push(call.alleleId);
// I guess `SearchAllelesRequest` is missing an `alleleIds` request field?
// Is it always true that `calls.length == alleles.length`?
alleles = searchAlleles({"alleleIds":alleleIds}).alleles;
// get the genotype excluding uncalled alleles.
// Suppose we have a function `getAlleleSequenceById()` that returns the allele sequence.
finalGenotype = []; copyNumber = 0;
for (call in calls) {
  copyNumber += call.copyNumber;
  for (var c = 0; c < call.copyNumber; ++c)
    finalGenotype.push(getAlleleSequenceById(alleles, call.alleleId));
}
assert(copyNumber == genotype.totalCopyNumber); // this is a constraint
print(finalGenotype.join("/"));

There are a few additional questions which are also inlined in the code:

In a VariantSet, do we always get one Variant at each position? If we could get two variants, V1 and V2, I guess one of them will have totalCopyNumber==0. Right? Is it possible for two overlapping variants to share a same Allele?

When we get a list of alleles/calls, does it include the reference allele?

Does a 3-tuple (variantSetId,callSetId,variantId) unambiguously identify a Genotype object (i.e. two Genotype objects cannot have the same 3-tuple)? Does (callSetId,alleleId) unambiguously identify a Call object (i.e. for the same sample, two Calls cannot have the same alleleId)?

Should we allow to search multiple sample names in SearchCallSetsRequest? Currently we can only search one sample at a time. In SearchAllelesRequest, should we allow to retrieve Allele objects by their IDs? Currently there is no "alleleIds" field.

Is there a convenient way to get the allele sequence? For now, we need to go through Allele=>Path=>Segment=>Position=>Reference to get an allele sequence.

Finally, Genotype.genotypeLikelihoods[] is different from genotypes. We need single/multi-sample calling to move from likelihoods to genotypes.

Here is another approach towards a monolithic allele model that is closer to VCF and the v0.5 APIs. Just for thought.
record VariantRow { array<Genotype> genotypes; } // A variant row made of per-sample genotypes
record Genotype { int totCpyNum; array<Call> calls; } // A genotype made of per-allele calls
record Call { string callSetId; Path allele; int copyNumber; } // A call is a piece of sequence and its copy number in a sample
record AlleleRow {
  array<Call> calls; // An allele row made of per-sample calls (by definition)
  array<Genotype> genotypes; // optionally allow to get the associated genotypes (for response)
}

A VariantRow here is equivalent to a VCF line and a Variant in the v0.5 APIs. The difference is that we separate Genotype and per-allele Call. We could traverse all variants like what we do with VCF and v0.5 APIs. We could also traverse each allele with an AlleleRow iterator. We could remove the genotypes field, but it might be handy for some access patterns.
This model removes a lot of IDs and imposes an explicit hierarchy between objects that is not so obvious by linking objects with IDs. "Objects of objects" also reduces frequent ID lookups and probably helps performance.

I was mostly talking about the monolithic model above. I realize that I also have a question on the graph model itself. Currently, the actual allele sequence is a Path object, which is a list of regions (called Segment in the schema) on the Reference sequence. It seems that we cannot add an arbitrary allele sequence without adding a new Reference. Is it true? If it is, does this mean that we need to add ~30 million 1bp Reference objects to keep the ~30 million 1000g SNPs? Thanks.

Basically, this sounds like dividing the genome into per-chromosome boxes called  Reference s, in which  Segment s are linked together into one graph per box. Is that right?

That's it exactly. Isn't this what we do now (but the graph is a very simple one)?

This would work, and depending on how the meaning of  isDerived  gets settled and the eventual (non)implementation of  getSequence  it might be the best way to go to strip superfluous fields that sequence in a  VariantSet  will never need. But it presents a couple of problems:
How do we deal with variants that connect chromosomes?

I don't think there's a problem with this in principle. Both graphs are composed of (what I'll call for arguments sake here) Fragments. We make a new Fragment, which happens to have a Fragment from one graph at one join and a Fragment from the other in the other join. We don't update the graph here, because we're creating an allele, not updating the references.

In a similar vein, what about things like the pseudoautosomal region on chrX and chrY?
Can  Reference s overlap and describe the same genomic region? Can a chromosome be split into multiple  Reference s (maybe one for the PAR and one for the rest of Y)? If so, how are they connected?

Well, under the definition the graph is just a set of Fragments, so there's no reason Fragments can't be members of several different graphs (== Reference). There's issues with where we define the set of Fragments that a graph is composed of, but maybe an API getter with paging is the way to do this? If we attach a referenceId attribute, then we can only belong to one Reference, and we'd also have to have an alleleId if the Fragment is used to define an Allele.

Does every piece of sequence in a  Reference  have to come from the same FASTA and have the same set of accessions? If not, do we have to put those fields on the  Segment / Fragment  again? What data would that leave in  Reference ?

No, I would make Fragment as lightweight as possible. As @lh3 pointed out, we'll need an awful lot of them and they'll need to be very small to the feasible. A Fragment should just be an ID and two Joins, basically. The Reference is the container object, to which we attach all of the metadata and provenance information.

I'd favor leaving the overall model as is; it's the minimal change we need to enable graphs to be represented. Each existing  Reference  can just stay as is, with no parent and nothing attached. If we solve these problems and determine that we really have some useless-for-variants fields on  Reference  as implemented, we can do another refactoring.

I'm not sure I agree. Under the model I've proposed, we leave ReferenceSet and Reference exactly as they are. We just add in a new concept for how we get the sequence bases for the reference via a Fragment or a Path through multiple fragments. Existing implementations can easily be updated to support this, even if they don't support a graph reference.

@jeromekelleher Say we have the GRCh38 chromosomal assembly and one ~3Mbp HLA haplotype. What will your graph look like? Will adding another ~3Mbp HLA haplotype change the existing fragments in the old graph?

This is about the nomenclature issue.
I agree that Reference is awkward, and would be better applied to a (standard) set of sequence pieces.
There is a problem with Fragment for the name of the basic sequence piece, because it is the standard term for the piece of DNA which is put into a sequencing machine.  Read pairs come one at each end of a fragment.
I proposed on today's phone call to use the term Segment for the basic sequence piece, then switch what is now called Segment, which is an interval on the piece, to be called a SubSegment.  Adam (I think) suggests Slice, or perhaps SegmentSlice.
An alternative for the main piece is Contig.  I quite like this also.  It is an assembly term, but the concept there, of a simple linear piece of DNA, corresponds reasonably well to what we are discussing.  Would we then leave Segment as the term for an interval on a Contig? That seems OK to me.
So, I vote +1 for Contig for the primary piece,  and Segment as an interval on that.
I vote 0 for the Segment as the piece, and SegmentSlice as the interval.
And for completeness I vote -1 for Reference as the piece, and Segment as the interval (current position).
Can we get consensus?

Adam and I were discussing. We like:
(a) Segment for what is currently called a "Reference"
(b) Subsegment for what is currently called "Segment"
(c) Creating an extra object "Reference" that would wrap a Segment (old
name Reference), and be used for describing a Segment in the ReferenceSet,
with associated meta-data, like an accession or extra flags (see
"isDerived" flag issue).
I'm also open to the word Contig in place of Segment or Reference (and
Segment as an interval, as Richard suggests), but a contig in an assembly
is assumed to be a contiguous piece of DNA without scaffold gaps - and some
of the bits of the primary reference are definitely scaffolds, not contigs.
So I'm 0 on that proposal, on reflection.
On Wed, Jan 21, 2015 at 10:11 AM, Richard Durbin<EMAIL_ADDRESS>wrote:

This is about the nomenclature issue.
I agree that Reference is awkward, and would be better applied to a
(standard) set of sequence pieces.
There is a problem with Fragment for the name of the basic sequence piece,
because it is the standard term for the piece of DNA which is put into a
sequencing machine. Read pairs come one at each end of a fragment.
I proposed on today's phone call to use the term Segment for the basic
sequence piece, then switch what is now called Segment, which is an
interval on the piece, to be called a SubSegment. Adam (I think) suggests
Slice, or perhaps SegmentSlice.
An alternative for the main piece is Contig. I quite like this also. It is
an assembly term, but the concept there, of a simple linear piece of DNA,
corresponds reasonably well to what we are discussing. Would we then leave
Segment as the term for an interval on a Contig? That seems OK to me.
So, I vote +1 for Contig for the primary piece, and Segment as an interval
on that.
I vote 0 for the Segment as the piece, and SegmentSlice as the interval.
And for completeness I vote -1 for Reference as the piece, and Segment as
the interval (current position).
Can we get consensus?
—
Reply to this email directly or view it on GitHub
https://github.com/ga4gh/schemas/pull/198#issuecomment-70889813.

On Wed, Jan 21, 2015 at 7:21 AM, Jerome Kelleher<EMAIL_ADDRESS>wrote:

Basically, this sounds like dividing the genome into per-chromosome boxes
called Reference s, in which Segment s are linked together into one graph
per box. Is that right?
That's it exactly. Isn't this what we do now (but the graph is a very
simple one)?

I'm -1 on this, because I think we won't end up with nice separable
connected components by chromosome, because of inter-chromosomal
rearrangements. I think in a subsequent pull-request beyond this minimal
pull request it would make sense to define the notion of a subset of a
reference set (a kind of subgraph), which would allow one to talk about a
subgraph in the context of the large coordinate system - but I think this
should be done in a secondary pull request.
—

Reply to this email directly or view it on GitHub
https://github.com/ga4gh/schemas/pull/198#issuecomment-70855888.

On Tue, Jan 20, 2015 at 9:16 PM, Heng Li<EMAIL_ADDRESS>wrote:

Here is another approach towards a monolithic allele model that is closer
to VCF and the v0.5 APIs. Just for thought.
record VariantRow { array genotypes; } // A variant row made of per-sample genotypes
record Genotype { int totCpyNum; array calls; } // A genotype made of per-allele calls
record Call { string callSetId; Path allele; int copyNumber; } // A call is a piece of sequence and its copy number in a sample
record AlleleRow {
array calls; // An allele row made of per-sample calls (by definition)
array genotypes; // optionally allow to get the associated genotypes (for response)
}
A VariantRow here is equivalent to a VCF line and a Variant in the v0.5
APIs. The difference is that we separate Genotype and per-allele Call. We
could traverse all variants like what we do with VCF and v0.5 APIs. We
could also traverse each allele with an AlleleRow iterator. We could
remove the genotypes field, but it might be handy for some access
patterns.
This model removes a lot of IDs and imposes an explicit hierarchy between
objects that is not so obvious by linking objects with IDs. "Objects of
objects" also reduces frequent ID lookups and probably helps performance.

Hi Heng, this is cool. It's great you're thinking about how to use the API!
However, it seems to me that these are convenience objects that could be
added on top of the initial minimal pull request - i.e. they don't replace
objects in the core pull request. To me they should be added in a
subsequent pull request?

—
Reply to this email directly or view it on GitHub
https://github.com/ga4gh/schemas/pull/198#issuecomment-70785882.

(a) Segment for what is currently called a "Reference"
(b) Subsegment for what is currently called "Segment"
(c) Creating an extra object "Reference" that would wrap a Segment (old
name Reference), and be used for describing a Segment in the ReferenceSet,
with associated meta-data, like an accession or extra flags (see
"isDerived" flag issue).

I am +1 on Reference -> Segment. I'm -0 on Segment -> Subsegment. I feel like Segment->Subsegment denotes a subset relationship, e.g., a subset of a set yields another set, or a substring of a string yields another string. I'd prefer a name that was more explicit about the fact that the object just contains indices into a Segment/Reference; possibly a SegmentSlice or SegmentRange?

I'm also open to the word Contig in place of Segment or Reference (and
Segment as an interval, as Richard suggests), but a contig in an assembly
is assumed to be a contiguous piece of DNA without scaffold gaps - and some
of the bits of the primary reference are definitely scaffolds, not contigs.
So I'm 0 on that proposal, on reflection.

As the person who suggested Contig, I agree, it would be a leaky abstraction. I'd also prefer to not overload current terms, as that is confusing. I motion to retract the Contig proposal. ;)

Above other things, I am most interested in my question: will we need ~30 million 1bp Reference/Segment/Contig/Subsegment objects to describe the ~30 million 1000g SNPs?

Yes, absolutely, we will. That doesn't mean the back-end has to store it
that way, but yes. Do you see this is a big deal. Assuming each such record
is less than some small number of bytes I don't see this getting that big?
On Wed, Jan 21, 2015 at 11:29 AM, Heng Li<EMAIL_ADDRESS>wrote:

Above other things, I am most interested in my question: will we need ~30
million 1bp Reference/Segment/Contig/Subsegment objects to describe the ~30
million 1000g SNPs?
—
Reply to this email directly or view it on GitHub
https://github.com/ga4gh/schemas/pull/198#issuecomment-70904709.

Yes, we will need 30 million 1bp Reference/Segment/Contig/Subsegment objects.
I don't see that as a problem - we need some sort of object to represent the sequence difference.
I do see that we need to clean things up so that one does not need to make long series of calls to get
at very simple information.
Perhaps a solution to the debate about SubSegment or Slice is to actually have the subsegment be a segment, as
Frank wants.  This removes a layer of the structure, because then a Path can be an Array of Segments (as it is now).
Remember that we need the interval to describe, for example, reference alleles.
So, a Segment would then be an atomic basic segment, with its own DNA, or a subsegment defined as an interval on a Segment.
Actually, the extreme form of this would be to also swallow up the Path object, and allow a Segment to also alternatively be an
array of consecutive Segments.   This would remove yet another layer of the structure, and make simple Alleles much easier to
describe, as just a Segment, plus whatever additional information is wanted.
Richard
On 21 Jan 2015, at 19:29, Heng Li<EMAIL_ADDRESS>wrote:

Above other things, I am most interested in my question: will we need ~30 million 1bp Reference/Segment/Contig/Subsegment objects to describe the ~30 million 1000g SNPs?
—
Reply to this email directly or view it on GitHub.

--
The Wellcome Trust Sanger Institute is operated by Genome Research
Limited, a charity registered in England with number 1021457 and a
company registered in England with number 2742969, whose registered
office is 215 Euston Road, London, NW1 2BE.

I've put up what things would look like after the Segment -> Subsegment and Reference -> Reference/Segment changes.
I'm also interested in perhaps unifying Subsegment and Segment; at one point I wanted a Segment to be just a couple of SegmentJoinLocations and a Subsegment, but then I realized that that would let you start a Segment's coordinates at somewhere other than 0.

I was just skimming through this thread and thus may have missed something,
but the use of allele and genotype is confusing. As diploid organisms at
any single position an individual may have two different alleles. Is the
assumption that an AlleleRow is describing just one of the chromosome? And
the genotype is the genotype of the chromosome and not of the individual
(who may be either hetero or homozygous at that position)?
Apologies if this was part of an earlier thread, but this at first reading
appears to be an unusual definition/use of the term genotype which more
conventionally refers to an individual as a whole.
On Tue, Jan 20, 2015 at 9:16 PM, Heng Li<EMAIL_ADDRESS>wrote:

Here is another approach towards a monolithic allele model that is closer
to VCF and the v0.5 APIs. Just for thought.
record VariantRow { array genotypes; } // A variant row made of per-sample genotypes
record Genotype { int totCpyNum; array calls; } // A genotype made of per-allele calls
record Call { string callSetId; Path allele; int copyNumber; } // A call is a piece of sequence and its copy number in a sample
record AlleleRow {
array calls; // An allele row made of per-sample calls (by definition)
array genotypes; // optionally allow to get the associated genotypes (for response)
}
A VariantRow here is equivalent to a VCF line and a Variant in the v0.5
APIs. The difference is that we separate Genotype and per-allele Call. We
could traverse all variants like what we do with VCF and v0.5 APIs. We
could also traverse each allele with an AlleleRow iterator. We could
remove the genotypes field, but it might be handy for some access
patterns.
This model removes a lot of IDs and imposes an explicit hierarchy between
objects that is not so obvious by linking objects with IDs. "Objects of
objects" also reduces frequent ID lookups and probably helps performance.
—
Reply to this email directly or view it on GitHub
https://github.com/ga4gh/schemas/pull/198#issuecomment-70785882.

@selewis Call gives the state of one chromosome. Genotype combines Calls and describes two chromosomes in case of a diploid locus, just like what we normally mean.

To elaborate, a Call can also give the total copy number in a sample if it's in a GENOTYPE CallSet, as opposed to a HAPLOTYPE CallSet.

I've done some thinking about the relationship between Segment and Subsegment, and as @richarddurbin suggested I think that they really ought to be one object. I've done that, and I've moved a few things into common.avdl to make it work.
Instead of being identified objects, Segments now select subranges of sequences, with startJoin and endJoin fields to specify where those sequences are joined at their ends, if applicable. Sequences are not a new Avro type, but just strings with IDs that can be paged through with an API call.
You can specify a sequence graph as a set of Segments selecting the entirety of the involved sequences and giving all the joins, and you can also specify a Path as an ordered list of Segments (carrying only the joins followed along the path).
Each sequence can have one Reference, which contains a Segment covering the whole sequence and attaching it to other sequences with References. A sequence's Reference is addressable by the sequence ID.
A VariantSet comes with new sequences for the new bases it needs in its Alleles. I scrapped the idea of having any object referencing the VariantSet for this. Instead I now have an API call you can do on a VariantSet to page through a set of Segments describing the new pieces of sequence graph that it needs on top of its ReferenceSet. The Alleles will then be Paths through the sequence graph formed by the sequences with References in the ReferenceSet, and those described in this set of Segments from the VariantSet
I think this cleans things up tremendously. We now have just three types (Side, Position, and Segment) that you can use to describe a sequence graph. There's a natural correspondence between a sequence graph and a Path embedded in one. There's a single API endpoint for getting sequence base information. There are no more References in the Variants API (and no more ugly ID reference to a dependent API to support it).
What do people think?

On Wed, Jan 21, 2015 at 5:40 PM, adamnovak<EMAIL_ADDRESS>wrote:

I've done some thinking about the relationship between Segment and
Subsegment, and as @richarddurbin https://github.com/richarddurbin
suggested I think that they really ought to be one object. I've done that,
and I've moved a few things into common.avdl to make it work.
Instead of being identified objects, Segments now select subranges of
sequences, with startJoin and endJoin fields to specify where those
sequences are joined at their ends, if applicable. Sequences are not a new
Avro type, but just strings with IDs that can be paged through with an API
call.
You can specify a sequence graph as a set of Segments selecting the
entirety of the involved sequences and giving all the joins, and you can
also specify a Path as an ordered list of Segments (carrying only the
joins followed along the path).
Each sequence can have one Reference, which contains a Segment covering
the whole sequence and attaching it to other sequences with References. A
sequence's Reference is addressable by the sequence ID.

Hi Adam, I thought we agreed it would be good to allow a reference segment
to not necessarily cover the entirety of the underlying sequence - for
example when specifying a reference set for a subregion, like the MHC or
one of the other HGVM pilot regions?

Boring git engineering stuff: I think we need to rebase off upstream/master again Adam. Unfortunately, there's been some structural changes that have happened since you made this branch (#209 made some new files and added namespaces). So, some things like GAException moving into methods.avdl have happened. This is going to be tedious to resolve, but it's only going to get worse the longer we leave it, unfortunately.

@jeromekelleher Say we have the GRCh38 chromosomal assembly and one ~3Mbp HLA haplotype. What will your graph look like? Will adding another ~3Mbp HLA haplotype change the existing fragments in the old graph?

I think the graph will look like @richarddurbin's basic definition - we're really just working out exactly how we should translate that into objects and API calls. So, to answer your question, the graph for your example would consist of three Segments. Adding new Segments into the graph should not modify the old segments in any way - we just put in a new Segment, which knows where it starts and finishes.

I like the economy of removing some of the objects, and I think we're approaching something minimal that we can work to improve in subsequent PRs. There's a few major outstanding issues for me though:

I disagree with a Reference corresponding to exactly one Segment. Surely a Reference must correspond to set of segments? I completely agree that we might not have nice cleanly defined references corresponding to chromosomes, but surely there must be some disjoint subsets of the graph that we are happy are really disjoint? If we map a Reference to a set of Segments then we can reuse the existing API without modification. We just add a new API call getSegments(referenceId) or something similar, and we then have the information we need. All of the existing references API can be left as-is, and we can then slowly deprecate it, rather than insisting on a sharp transition to the graph enabled world.

Following on from this point, I think we should seek to edit the existing types and interfaces as little as possible, so that we can have the two APIs coexisting. I think this would ease adoption and make it easier for people to transition to the new API. It will also free us up to experiment more with the new types and calls, as we don't have to worry so much about how this will affect existing users.

Surely a Reference must correspond to set of segments?

Do you mean Reference or ReferenceSet? A Reference corresponds to one chromosome in the linear view. A ReferenceSet corresponds to a set of chromosomes.

Do you mean Reference or ReferenceSet? A Reference corresponds to one chromosome in the linear view. A ReferenceSet corresponds to a set of chromosomes.

A Reference. I'm arguing that we should define a reference as a set of Segments (which in the linear case will be a singleton set), describing the sequence graph for a particular region. @benedictpaten's point about us not being able to cleanly define a set of Segments (or a single connected graph) as corresponding to only one chromosome is a good one, and this will cause difficulties. However, I think this is less problematic than mapping today's concept of a reference onto a single Segment.

Hi Adam, I thought we agreed it would be good to allow a reference segment to not necessarily cover the entirety of the underlying sequence - for example when specifying a reference set for a subregion, like the MHC or one of the other HGVM pilot regions?

You're right, I forgot that use case. I've added a comment explicitly allowing it.

@jeromekelleher It sounds like what you want might be better implemented as composable ReferenceSets? The current everyday concept of a "reference" is something like GRCh38, which would be implemented as a ReferenceSet in our API. If you want to use a smaller part of GRCh38 for some analysis of just a single chromosome or region, you would implement that as a ReferenceSet as well.
What we call a Reference in our API has a 1:1 correspondence with FASTA records in GRCh38, and a FASTA record is a single sequence, which can be described with a single Segment.
Maybe these names are confusing, and we should have Reference as the object that ReadGroups are mapped to, and ReferenceSegment as the annotated sequence? But then that's another refactoring that would be completely incompatible with the current non-graph API.

This isn't supposed to be closed; I pushed the wrong branch to the wrong place while bringing the graph branch even with master (so I could have my rebased commits show as a diff against master) and Github helpfully closed it. I'll see if I can get it opened again.

OK, #233 is going to replace this thread. Sorry again for closing it prematurely.

There is a lot of detail on this thread. But the one take home for me was
Adam's comment:
"just three types (Side, Position, and Segment)"
Simplicity, economy, and beauty matter a lot. It strikes me that as a pure
form, these three are the fundamental elements of these graphs.
-D
On Thu, Jan 22, 2015 at 6:26 PM, adamnovak<EMAIL_ADDRESS>wrote:

OK, #233 https://github.com/ga4gh/schemas/pull/233 is going to replace
this thread. Sorry again for closing it prematurely.
—
Reply to this email directly or view it on GitHub
https://github.com/ga4gh/schemas/pull/198#issuecomment-71138011.


Add helm chart

I expected to see some usual helm staff we have in other repos. But instead, I see a bunch of customization. Is this a new way to manage helm values?

I expected to see some usual helm staff we have in other repos. But instead, I see a bunch of customization. Is this a new way to manage helm values?

At the end of the day everything lands in helm/management-cluster-admission/templates/kustomize-out.yaml (generated) when make manifests is being run.
Kubebuilder uses kustomize, so I think it's beneficial to start getting into this especially since we can generate valid helm charts which are on par with what we have right now.

I'm fine with that. Just wanted to know if there was some discussion or decision has been done to use more kustomize now. And also what it covers - operators only or all the projects?

Let's continue in Slack: https://gigantic.slack.com/archives/C010L893ZFW/p1617879384015500?thread_ts=1617877651.015100&cid=C010L893ZFW


Support minimum LOD for ESRI layers

Some ESRI Mapserver define a minimum LOD in its capabilities.  Any tile request made for a tile on a smaller LOD, will result in a failed tile request.  In the previous implement, this would prevent the proper display of children tiles on levels inside the lod validity range (i.e. level>=minLod && level<=maxLod).
The solution is to flag as disabled tiles on LOD outside the validity range.  Disabled tiles can have visible children per definition.
To reproduce the issue, following key-in can be used:
fdt attach arcgis maplayer https://tiles.arcgis.com/tiles/IMCZpp2qXhYVmRXp/arcgis/rest/services/Stockholm_%C3%96P_Stadsutvecklingskartan/MapServer stockholm
Dataset is geo located in Stockholm/Sweden.
A sample BIM can be found in attachment of this bug

@RBBentley I think you are reviewing the latest revision, your comments are about a previous commit.

@Mergifyio backport release/2.18.x


Implement MITW Track 2

Some references are as follows:

https://www.youtube.com/watch?v=LE4QSMknWrQ
https://www.youtube.com/watch?v=h_Xj3TBJtsc
https://www.youtube.com/watch?v=q9to8L9cEko
https://www.youtube.com/watch?v=OrTasgrtToM

At this moment, just implement the Patient resource to be compatible with the scenario 2.


  PLEASE DON'T ASK FOR WORKAROUNDS OR TIMELINES - THIS IS FREE SOURCE CODE
v2 no longer has a package for UWP support and comes built into new Shiny nuget
Please check all of the platforms you are having the issue on (if platform is not listed, it is not supported)

[ ] iOS
[ ] Android
[ x] UWP

PLATFORMS THAT YOU CANNOT FILE ISSUES ON RIGHT NOW: Tizen, watchOS, tvOS, macOS
Feature Library
beta components are not fully done - issues MUST include full reproducible sample and stacktrace or they will be closed without explanation!!!

[x] Core
[ ] Locations
[ ] HTTP Transfers
[ ] Notifications
[ ] Push
[ ] Push - Azure Notification Hub
[ ] Push - Firebase (beta)
[ ] Sensors
[ ] AppCenter Logging Integration
[ ] Location Sync (beta)
[ ] Photo Sync (alpha)
[ ] BluetoothLE (beta)
[ ] Beacons (beta)
[ ] SQLite Integration

Version of Library
v2
Version of OS(s) listed above with issue
All
Steps to Reproduce

Deploy it

Expected Behavior
Not getting a rejection from nuget
Actual Behavior
Getting a rejection from nuget

fixed in latest previews


Feature request: prediction streaming

Hi there! I’ve recently started playing with Replicate in SwiftUI and this library has been super helpful.
In my previous custom implementation of the Replicate HTTP API, I was doing some manual polling to get a stream of updates in predictions:
// Illustrative pseudocode
Task {
  while prediction.completedAt == nil {
    do {
      try await Task.sleep(nanoseconds: 1_000_000_000)
      prediction = try await fetchPrediction(prediction.id)
    } catch {
      print(error)
    }
  }
}

// 1 second
print(prediction.output) // ["Hello", "!", " "]

// 2 seconds
print(prediction.output) // ["Hello", "!", " ", "As", " ", "a", " ", "helpful", ...]

It'd be great if this library included a polling/streaming method alongside the wait method, especially for chat-like models.

Hi @daneden. That's a great idea! We recently added support for streaming prediction output to Replicate. #41 added an overload to Client.createPrediction that accepts a stream parameter. When set to true, the created prediction will include a "stream" key in its urls property.
My next step is to create a Swift implementation of EventSource that conforms to AsyncSequence and you could iterate over directly.

Please support Streaming and EventSource as the Replicate JavaScript code you provided here.
https://github.com/replicate/replicate-javascript#streaming
Now even that you have a stream parameter, it doesn't make any sense.
Because we still need to make the stream logic code by ourself.


[Vue warn]: Template compilation error: Tags with side effect (

Hi there!
I just started using this package, and run into this warning in Google Chorme console:
[Vue warn]: Template compilation error: Tags with side effect (<script> and <style>) are ignored in client component templates.
5  |          <input type="hidden" name="widget_form_id" value="1">
6  |
7  |          <script>
8  |                  window.addEventListener('load', () => {
9  |                      setTimeout(() => {
10 |                          document.querySelectorAll('input[data-purpose="honey_javascript"]')
11 |                              .forEach(input => {
12 |                                  if (input.value.length > 0) {
13 |                                      return;
14 |                                  }
15 |
16 |                                  input.value = "here is my key string";
17 |                                  input.dispatchEvent(new Event('change'));
18 |                              });
19 |                      }, 3000)
20 |                  });
21 |              </script>
  at <App>

My .blade.php file looks like this:
    <form action="/" method="post">
        @csrf
        <x-honey />
        <x-honey-recaptcha />
        ....
    </form>

This blade file is returned as view, and this view is inserted into the body element of HTML DOM.
Any idea?

@netdjw I created a PR that fixes this.


BIGTOP-2586: hadoop-processing tests.yaml should include PPA for Amulet

Add source PPA for Juju bundles' tests.yaml (both hadoop-processing and
spark-processing).

This went in with PR #167


Support pinned stories

Description

Corresponding backend PR: https://github.com/source-academy/stories-backend/pull/70.
Also did some refactoring and added a new field for editing the story title.
Type of change

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update
[x] Code quality improvements

How to test

Checklist

[x] I have tested this code
[ ] I have updated the documentation

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 9 (0.0%)  changed or added relevant lines in 2 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.02%) to 36.064%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

src/pages/stories/Stories.tsx
0
2
0.0%

src/pages/stories/NewStory.tsx
0
7
0.0%

Totals

Change from base Build<PHONE_NUMBER>:
-0.02%

Covered Lines:
5452

Relevant Lines:
14162

💛  - Coveralls

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Warning: This coverage report may be inaccurate.
This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

For more information on this, see Tracking coverage changes with pull request builds.
To avoid this issue with future PRs, see these Recommended CI Configurations.
For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

0 of 9 (0.0%)  changed or added relevant lines in 2 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.02%) to 36.064%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

src/pages/stories/Stories.tsx
0
2
0.0%

src/pages/stories/NewStory.tsx
0
7
0.0%

Totals

Change from base Build<PHONE_NUMBER>:
-0.02%

Covered Lines:
5452

Relevant Lines:
14162

💛  - Coveralls


Consider verifying custom domain for GitHub Pages

https://docs.github.com/en/pages/configuring-a-custom-domain-for-your-github-pages-site/verifying-your-custom-domain-for-github-pages

Done.


OCPBUGS-21880: CVE-2023-39325 - Bump golang.org/x/net to v0.17.0 - 4.13

Forcing a merge to ensure this gets picked up in the next rebuild. This will ensure that it meets the bug resolution deadline (today). This is low risk because this image is in tech preview.


Fixed-HomePage

Removed Duplicate Elements
Re-enabled Progress-Bar, Chatbot and Scroll up Button
Before
https://github.com/user-attachments/assets/aa6acf2f-4499-45d8-baed-3570083d32cc
After
https://github.com/user-attachments/assets/ba3be01f-114d-441e-a5b4-82982c82d882
Please Merge the PR with level3 label
Thankyou!!

Please resolve merge conflicts

Please resolve merge conflicts
@vishanurag
Resolved merge Conflicts.
Please Merge the PR.


fix(#183): defaults if frame empty

closes #183

PR-Codex overview
This PR focuses on enhancing the handling of empty input in the GitHub mentions processing, introducing a test case to ensure that the output is correct when no mentions are available.
Detailed summary

Added to-ghmentions-empty.csv to test with empty input.
Introduced test_returns_zero_records_on_empty_input in test_ghmentions.py to verify behavior with empty input.
Updated gh_mentions.py to set readme_pmentions and readme_imentions to 0 if the frame is empty.

✨ Ask PR-Codex anything about this PR by commenting with /codex {your question}

@rultor merge

@rultor merge

@h1alexbel Done! FYI, the full log is here (took me 10min).


[SPARK-25805][SQL][TEST] Fix test for SPARK-25159

The original test would sometimes fail if the listener bus did not keep
up, so just wait till the listener bus is empty.  Tested by adding a
sleep in the listener, which made the test consistently fail without the
fix, but pass consistently after the fix.

LGTM

Test build #97892 has finished for PR 22799 at commit 65be98b.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97892/
Test PASSed.


Feature #3120: PID file

This Pull Request add support for PID file (issue #3120) by adding new settings on app.ini:
[pid]
; Is Gogs have to create a PID file?
ENABLED  = false
; Location of the PID file to create
PATH     = RUNNING_PID
; Is Gogs can override existing PID file?
OVERRIDE = false

Seems pretty bad overall. PID files are to be placed in /var/run (or /run on modern systems), so the default should be something like /var/run/gogs.pid. The PID file should be deleted on exit. The option not to overwrite the pid file could cause problems, when the gogs daemon crashes and won't restart, because the PID file still exists. Also, gogs won't start when it cannot write the pid file for other reasons.
Doing this via the init manager (systemd, upstart, sysvinit) would be better.

@Marco01809 do you suggest Gogs should not take care of PID files?

It wouldn't be wrong to have gogs write a PID file, all other daemons do it as well and it certainly helps when dealing with an older init manager like sysvinit.
It's just that this specific implementation is not very well done, and in the times of systemd, writing pid files isn't absolutely necessary anymore. (I have posted how to do it with systemd in https://github.com/gogits/gogs/issues/3120#issuecomment-242244807)

@Marco01809 thanks a lot!


Add new “in question” status

Adds [?] support as of v1.1 of the spec.

Ran into this missing syntax today, anything preventing this from being merged? Happy to help out if I can

No, I just forgot about it… 🙃

I just released 1.4.0 containing this fix. Note that it might take a while for the release to be available on Sublime package control.


improve performance

[ ] (https://github.com/neovim/neovim/pull/5269) use update diffs instead of fetching entire buffer
[ ] same with sublime on_modified - not sure how to efficiently diff.
[ ] batch vim round-trips using nvim_atomic
[x] binary vim rpc (binary msgpack in cb9ac7ef8a680835afe3fa242339de674d98de77)

[ ] utf-8 enc/decode in C instead of list comp in Python

[x] optimize ActualVim.visual
[x] #52

If we're too slow to issue an update after pressing a key, Windows won't draw the frame.

@lunixbochs https://github.com/neovim/neovim/pull/5269 (https://github.com/neovim/neovim/pull/7917) was merged.

thanks, pushed initial support in master


Use of undefined constant LARAVEL_START

Unit tests give the error
REMOTE_ADDR is not defined
This is fixed by : https://github.com/aschmelyun/larametrics/pull/47
After this fix, this error occurs on unit test. There is a similiar issue here : https://github.com/barryvdh/laravel-debugbar/issues/15 but I could not find declaration of LARAVEL_START in vendor/autoload.php too. The only declaration I can find is in index.php and artisan.
root@239a484e966e:/var/www/html/project# php vendor/bin/phpunit
PHPUnit 7.5.1 by Sebastian Bergmann and contributors.

.E                                                                  2 / 2 (100%)

Time: 822 ms, Memory: 16.00MB

There was 1 error:

1) Tests\Feature\ExampleTest::testBasicTest
ErrorException: Use of undefined constant LARAVEL_START - assumed 'LARAVEL_START'

/var/www/html/project/vendor/aschmelyun/larametrics/src/LarametricsRouteServiceProvider.php:66
/var/www/html/project/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php:29
/var/www/html/project/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php:75
/var/www/html/project/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php:31
/var/www/html/project/vendor/laravel/framework/src/Illuminate/Container/Container.php:564
/var/www/html/project/vendor/laravel/framework/src/Illuminate/Foundation/Application.php:966
/var/www/html/project/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php:191
/var/www/html/project/vendor/laravel/framework/src/Illuminate/Foundation/Testing/Concerns/MakesHttpRequests.php:352
/var/www/html/project/vendor/laravel/framework/src/Illuminate/Foundation/Testing/Concerns/MakesHttpRequests.php:168
/var/www/html/project/tests/Feature/ExampleTest.php:17

ERRORS!
Tests: 2, Assertions: 1, Errors: 1.

This is a well-documented issue with PHPUnit tests and Laravel packages. You can add the following to your TestCase.php and it should resolve the issue:
if (!defined('LARAVEL_START')) {
    define('LARAVEL_START', microtime(true));
}


HIR-3 test

@drulacosmin Please sign the Contributor License Agreement!
Click here to manually synchronize the status of this Pull Request.
See the FAQ for frequently asked questions.

zxzxz

zxz


packages.microsoft.com Debian Packages does not verify

Strictly speaking this is not an issue with msphpsql but I couldn't find anywhere appropriate to file this.
It appears that the Packages list on packages.microsoft.com was updated on 25th February, but the InRelease file for it was not (last update 19th Feb).
Files known to be affected:
https://packages.microsoft.com/debian/9/prod/dists/stretch/InRelease
https://packages.microsoft.com/debian/9/prod/dists/stretch/main/binary-amd64/
It also looks like the Release was updated, but not the Release.gpg
This makes it impossible to install safely on Debian.
Hit:1 http://security-cdn.debian.org/debian-security stretch/updates InRelease
Get:2 https://packages.microsoft.com/debian/9/prod stretch InRelease [3232 B]
Ign:3 http://cdn-fastly.deb.debian.org/debian stretch InRelease
Get:5 https://packages.microsoft.com/debian/9/prod stretch/main amd64 Packages [38.0 kB]
Err:5 https://packages.microsoft.com/debian/9/prod stretch/main amd64 Packages
  Writing more data than expected (38208 > 37999)
  Hashes of expected file:
   - Filesize:37999 [weak]
   - SHA512:194ded11a350aab88b7fc1c5fee5dd6746a6bd0136ee96ad2290fc9459086daf75fbeda376ab00c0455cbeab6e64fa405f82ef0991b28322f29f30dab2aaaede
   - SHA256:369f15b8a43c29e6adf6db3329576366123c6bb749ee311f5099752de06474ea
   - SHA1:87f67045f0db46ee582998ba1d432b47900f5992 [weak]
   - MD5Sum:38eeded94abf10a911e570233047151b [weak]
  Release file created at: Tue, 19 Feb 2019 18:30:09 +0000
Hit:4 http://cdn-fastly.deb.debian.org/debian stretch-updates InRelease
Hit:6 http://cdn-fastly.deb.debian.org/debian stretch Release
Fetched 3232 B in 0s (6394 B/s)
Reading package lists...
E: Failed to fetch https://packages.microsoft.com/debian/9/prod/dists/stretch/main/binary-amd64/Packages.bz2  Writing more data than expected (38208 > 37999)

@andrewnicols Yes we are aware of the issues with the repos and they are being fixed. Please stay tuned, it should not be long before they are fixed.

Thanks for the prompt response @david-puglielli. Have a great day.

The latest ODBC driver should now be in the repos and the problems fixed. Please try again and let us know if there are any issues. Thanks!

Closing issue, please reopen if necessary.


Revert "Fix hashtag regex blocking transform to heading"

@2wheeh
Unfortunately, I made a mistake here - some Meta surfaces support multiple leading "#" characters in a hashtag, so this would break them. Instead, we might need a generic way to override the regex.
Reverts facebook/lexical#5376

some Meta surfaces support multiple leading "#" characters in a hashtag, so this would break them.

This was I worried... Thanks for check.
Could you reopen #5366 ?


[Serviceability] Trace server updates by machine id

Include machine information (id) as part of the request that is updating data on the server. This will be helpful in diagnosing issues.

How will this be exposed to the user?

Its not needed to expose it to the user from the product but it will be useful to get the information from the server (through logs or any other means) while diagnosing the issue

Reopening this as @Tyriar requested to reset machine id after turning off.

Added following headers

X-Client-Name

code-insiders , code-insiders-web, code, code-web

X-Client-Version

X-Machine-Session-Id - Reset when user session is cleared on server

X-User-Session-Id - Session id that received from the server

X-Machine-Id


html links in individual slide

I'm building a flipster that has html elements in each slide. Some of those elements need to have html links to other pages, but it appears that the click state is being overwritten for the whole slide. The link url shows up in the status bar at the bottom of the browser, but the link is not registered. Any way to make this work? I thought maybe putting a high z-index on the link would work, but no change. Thanks

This repository is going into very-low-maintenance mode and declaring backlog bankruptcy


⚠ ClamAV has degraded performance

In e55df15, ClamAV (https://spamassassin.apache.org/) experienced degraded performance:

HTTP code: 200
Response time: 87 ms

Resolved: ClamAV performance has improved in 55f19db.


Specify minimum MicroManager Version

With build gamma1 20210211 running the example code gave me this error:
2021-07-01 15:11:56.781 | DEBUG    | pymmcore_remote._client:ensure_server_running:118 - No server found, creating new mmcore server
2021-07-01 15:11:57.382 | DEBUG    | pymmcore_remote._client:__exit__:56 - closing pyro client

---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
~\AppData\Local\Temp/ipykernel_8868/2357010630.py in <module>
      4     # 'demo' is a special option for the included CMMCorePlus
      5     # that loads the micro-manager demo config
----> 6     mmcore.loadSystemConfiguration("demo")
      7     print(mmcore.getLoadedDevices())

~\mambaforge\envs\control\lib\site-packages\Pyro5\client.py in __call__(self, *args, **kwargs)
    474         for attempt in range(self.__max_retries + 1):
    475             try:
--> 476                 return self.__send(self.__name, args, kwargs)
    477             except (errors.ConnectionClosedError, errors.TimeoutError):
    478                 # only retry for recoverable network errors

~\mambaforge\envs\control\lib\site-packages\Pyro5\client.py in _pyroInvoke(self, methodname, vargs, kwargs, flags, objectId)
    241                     return _StreamResultIterator(streamId, self)
    242                 if msg.flags & protocol.FLAGS_EXCEPTION:
--> 243                     raise data  # if you see this in your traceback, you should probably inspect the remote traceback as well
    244                 else:
    245                     return data

OSError: Line 7: Device,DHub,DemoCamera,DHub
Failed to load device "DHub" from adapter module "DemoCamera" [ Failed to load device adapter "DemoCamera" from "C:\Program Files\Micro-Manager-2.0gamma\mmgr_dal_DemoCamera.dll" [ Incompatible device interface version (required = 70; found = 69) ] ]

which was resolved when I updated to the current nightly build.
One small thing of note is that I had to restart my jupyter kernel in order to have the micromanager installer be able to replace mmgr_dal_DemoCamera.dll which may be a helpful thing to note if the fix for above is ever included in a troubleshooting section.

yeah it's an issue, I asked Nico about this actually... unfortunately, there's no ideal way to know what device interface any given download from micromanger has without checking the git repo, or downloading at looking.  And it's not just a minimum version either, it's a maximum too since we have to be working with a pymmcore version that matches the library version.
there is an install_mm.py helper script in the root directory here that will help install a specific version for you ... but that too will go out of date when pymmcore updates itself (fortunately, pymmcore puts the device interface in the version number)
anyway, it would be a great feature here to a) make the helper script install a compatible version for the given pymmcore device interface (that might require a little manual annotation) and b) provide a perhaps an even more useful error message when someone runs into this


Governance. Can't delegate on Kusama

Description
When user try to delegate on Kusama it stack in loading state.
https://github.com/user-attachments/assets/d83a61e1-dc80-4b9f-be21-e53a43c2d4b2
Steps to Reproduce

Go to Governance page
Select Kusama
Delegate to all tracks

Expected Behavior
Successfully submit the operation
Actual Behavior
Infinity loading state
Environment
https://github.com/novasamatech/nova-spektr/commit/b3df41c6c90545340866518c64736bcda4d6d21a

After long time the app have got a status - invalid but it still in ubmitting state:

android implementation

Status verification: ✅

Delegate on Kusama

Found during testing:

https://github.com/novasamatech/nova-spektr/issues/2262

Status verification: ✅


Creating cluster using private registry fails as kubelet fails to pull the image rancher/pause-amd64:3.0

Rancher versions:
rancher/rancher:vv2.0.0-rc1
Steps to Reproduce:

Push all images to a private registry
Create a custom cluster using the private registry in UI

kubelet, kube-proxy and all the components get pulled from the registry successfully on the host. kubelet is running but fails to pull the rancher/pause-amd64:3.0 image from the private registry
kubelet does not have the information to pull from the private registry.
kubelet logs below:
E0426 22:22:59.122859   13648 pod_workers.go:186] Error syncing pod 1775a30e-499c-11e8-8aac-962f3e87879d ("rke-network-plugin-deploy-job-nkcdr_kube-system(1775a30e-499c-11e8-8aac-962f3e87879d)"), skipping: failed to "CreatePodSandbox" for "rke-network-plugin-deploy-job-nkcdr_kube-system(1775a30e-499c-11e8-8aac-962f3e87879d)" with CreatePodSandboxError: "CreatePodSandbox for pod \"rke-network-plugin-deploy-job-nkcdr_kube-system(1775a30e-499c-11e8-8aac-962f3e87879d)\" failed: rpc error: code = Unknown desc = failed pulling image \"registrytest:5000/rancher/pause-amd64:3.0\": Error: image rancher/pause-amd64:3.0 not found"
W0426 22:23:02.563059   13648 cni.go:171] Unable to update cni config: No networks found in /etc/cni/net.d
E0426 22:23:02.563322   13648 kubelet.go:2120] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized
I0426 22:23:03.090067   13648 kuberuntime_manager.go:385] No sandbox for pod "rke-kubedns-addon-deploy-job-7rjds_kube-system(50594e38-499c-11e8-8aac-962f3e87879d)" can be found. Need to start a new one
I0426 22:23:03.121792   13648 kube_docker_client.go:348] Stop pulling image "registrytest:5000/rancher/pause-amd64:3.0": "Pulling repository registrytest:5000/rancher/pause-amd64"
E0426 22:23:03.121862   13648 remote_runtime.go:92] RunPodSandbox from runtime service failed: rpc error: code = Unknown desc = failed pulling image "registrytest:5000/rancher/pause-amd64:3.0": Error: image rancher/pause-amd64:3.0 not found
E0426 22:23:03.121913   13648 kuberuntime_sandbox.go:54] CreatePodSandbox for pod "rke-kubedns-addon-deploy-job-7rjds_kube-system(50594e38-499c-11e8-8aac-962f3e87879d)" failed: rpc error: code = Unknown desc = failed pulling image "registrytest:5000/rancher/pause-amd64:3.0": Error: image rancher/pause-amd64:3.0 not found
E0426 22:23:03.121954   13648 kuberuntime_manager.go:647] createPodSandbox for pod "rke-kubedns-addon-deploy-job-7rjds_kube-system(50594e38-499c-11e8-8aac-962f3e87879d)" failed: rpc error: code = Unknown desc = failed pulling image "registrytest:5000/rancher/pause-amd64:3.0": Error: image rancher/pause-amd64:3.0 not found
E0426 22:23:03.122113   13648 pod_workers.go:186] Error syncing pod 50594e38-499c-11e8-8aac-962f3e87879d ("rke-kubedns-addon-deploy-job-7rjds_kube-system(50594e38-499c-11e8-8aac-962f3e87879d)"), skipping: failed to "CreatePodSandbox" for "rke-kubedns-addon-deploy-job-7rjds_kube-system(50594e38-499c-11e8-8aac-962f3e87879d)" with CreatePodSandboxError: "CreatePodSandbox for pod \"rke-kubedns-addon-deploy-job-7rjds_kube-system(50594e38-499c-11e8-8aac-962f3e87879d)\" failed: rpc error: code = Unknown desc = failed pulling image \"registrytest:5000/rancher/pause-amd64:3.0\": Error: image rancher/pause-amd64:3.0 not found"

The node is in unavailable state as the network plugin is not ready

currently, we are not supporting being able to use private registries with credentials for RKE clusters. UI should remove that option.
instead, the setting of private registry should be used.

in v2.0.0-rc2, this setting should be removed.

Tested with v2.0.0-rc3. The fields in the UI to enter private registry credentials have been removed.
Created a cluster using private registry without providing credentials. Cluster was created successfully.


Sometimes orientation is wrong

When the source video is UIImageOrientationUp, the output video's orientation is not the same as source.

Did you try using
func rotate(rotateDegree: LLRotateDegree)
or maybe the information is wrong from your camera..
If youre using LLSimple Cam try
var fixOrientationAfterCapture: Bool
and
public var useDeviceOrientation: Bool


[Feature Request] Message-to-Data Format Converter

Required prerequisites

[X] I have searched the Issue Tracker and Discussions that this hasn't already been reported. (+1 or comment there if it has.)
[ ] Consider asking first in a Discussion.

Motivation
Develop a flexible system for converting agent messages into structured data formats, leveraging the power of Pydantic for standardization and customization.

Generalized Data Format Support:

Define a set of reusable Pydantic objects for commonly used data formats, such as pre-training, instruction tuning, and conversion tasks.
Provide the ability for users to pass custom Pydantic models for data conversion to suit their specific requirements.

Message-to-Data Conversion:

Implement a converter capable of transforming agent messages into the defined structured data formats seamlessly and efficiently.

Solution
No response
Alternatives
No response
Additional context
No response

BaseMessage -> structured object
string -> structured object
structured object -> BaseMessage


Add clusterLogForwarding

Checklist

[x] Keep pull requests small so they can be easily reviewed.
[x] Update the documentation.
[x] Categorize the PR by setting a good title and adding one of the labels:
bug, enhancement, documentation, change, breaking, dependency
as they show up in the changelog
[x] Link this PR to related issues.

Implementation LGTM overall, I cannot judge whether it works correctly / as expected since I'm not familiar with the log forwarding stuff.

I can confirm, that the compiled output results in the same configuration as curently in test on appuio lab.

I'll soon have another functional enhancement to this component, so this doesn't need to be released immediatly.


validate location from fwrj

just prevent situation happened using T0_CERN_CH

@amaltaro, Alan, please review

Code looks Ok, I'm just not sure whether we should really break the component if this case happens again. I'd instead print a warning and fail the job (possible adding a new error/message in WMExceptions).

I think we should crash the component. If job runs somewhere we don't have the record of,

it is completely screwed up
Or we don't have the proper site record in our table.
I think it would be mostly 2 case (although proper way is to have SSB record), then we need add the site in the table manually and restart the component.

If we just print error and move on, we need to do manual surgery when case 2 cause the problem.
I think it actually shouldn't happen (at least very often). But it happens it might be easier to fix by crashing Component

I will update the patch to fix unit tests.

If we just print error and move on, we need to do manual surgery when case 2 cause the problem.

we do not move on. We permanently fail this job, report failures to wmstats and someone  fix either a) the assignment or some black magic script that was screwing with jobs or b) fix resource control/sitedb.
I'm just trying to get rid of ALL the unneeded component crashes.
On the other hand, I agree that this should happen not more than once/twice per year, so it's ok'ish to crash the component if you prefer (at least we would be able to make a hot fix and not waste cpu time)

e do not move on. We permanently fail this job, report failures to wmstats and someone fix either a) the assignment or some black magic script that was screwing with jobs or b) fix resource control/sitedb.

I see, I think a) case it makes sense to failed the jobs  but b) case, it maybe better to fix resource and not redo the whole processing.  I am guessing most case will be b). but I might be wrong.

Patch looks good, though I'm running a couple quick workflows just to make sure nothing breaks.
If it's Ok, I think this one deserves to be backported to 1.0.14

@ticoann feel free to merge this one

test this please


Update to VIER CVG Extension 4.3.0

These are two bug fixes, which would be great to be included in the current 4.2.0 version.
Since this includes a new node now, let's make it 4.3.0

@alexteusz It would be nice to have this included before the marketplace is updated.

@alexteusz 0af2f5f5fca8e9a716d3b658e57525a93217913b originally also contained the upgrade of @cognigy/extension-tools, but the extension fails to import in the trial environment, when I pull 0.15.1, while it works fine with 0.14.0.


Support for latest API level

Hello,
We are currently using your library in one of our applications. Actually we need a favour from your side. According to the Google Play Policy, all apps have to provide support for at least API level 26. It seems like yours is not as per the new google play policy. We are targeting API level 28. So can you please provide us support for API level 28 in your current repository?

Please check out the Android official docs. We just want to make sure that all the libraries support at least API level 26 and we are targeting API level 28. Please update as soon as possible.


Fix #167: Introduce SEMVER_SPEC_VERSION

This PR fixes #167 and implements the global SEMVER_SPEC_VERSION constant.

Similar to https://github.com/k-bx/python-semver/pull/164#issuecomment-541443311
Updating CHANGELOG.rst will be nice.

Thanks @tomschr


Build Failed - Cannot invoke method lastIndexOf() on null object & compileSdkVersion is not specified. Please add it to build.gradle

Describe the bug
A clear and concise description of what the bug is.
To Reproduce
Steps to reproduce the behavior:

Go to '...'
Click on '....'
Scroll down to '....'
See error

Expected behavior
After completing fcm integration, i can't build the app. I got 2 exceptions,
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.

Where:
Script 'F:\Ionic Cordova\goMart - Full App\platforms\android\cordova-plugin-fcm-with-dependecy-updated\gomartapp-FCMPlugin.gradle' line: 22

What went wrong:
A problem occurred evaluating script.

Cannot invoke method lastIndexOf() on null object

Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to
get full insights.
==============================================================================

2: Task failed with an exception.

What went wrong:
A problem occurred configuring project ':app'.

compileSdkVersion is not specified. Please add it to build.gradle

Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to
get full insights.
==============================================================================

Get more help at https://help.gradle.org

BUILD FAILED in 8s
Command failed with exit code 1: F:\Ionic Cordova\goMart - Full App\platforms\android\gradlew cdvBuildDebug -b F:\Ionic Cordova\goMart - Full App\platforms\android\build.gradle
[ERROR] An error occurred while running subprocess cordova.
    cordova.cmd build android --debug exited with exit code 1.

    Re-running this command with the --verbose flag may provide more information.

Environment:

Developer OS and version: Windows 10 Pro
Mobile OS and version: Android 9
Cordova CLI version (cordova --version): 10.0.0

Package.json
{
"name": "XXXX",
"version": "1.1.0",
"author": "Shivam ITS",
"homepage": "https://shivamits.com/",
"scripts": {
"ng": "ng",
"start": "ng serve",
"build": "ng build",
"test": "ng test",
"lint": "ng lint",
"e2e": "ng e2e"
},
"private": true,
"dependencies": {
"@angular/common": "~10.0.0",
"@angular/core": "~10.0.0",
"@angular/forms": "~10.0.0",
"@angular/platform-browser": "~10.0.0",
"@angular/platform-browser-dynamic": "~10.0.0",
"@angular/router": "~10.0.0",
"@ionic-native/camera": "^5.30.0",
"@ionic-native/core": "^5.0.0",
"@ionic-native/fcm": "^5.31.1",
"@ionic-native/file": "^5.30.0",
"@ionic-native/file-transfer": "^5.30.0",
"@ionic-native/google-plus": "^5.30.0",
"@ionic-native/splash-screen": "^5.0.0",
"@ionic-native/status-bar": "^5.0.0",
"@ionic/angular": "^5.0.0",
"@ngx-translate/core": "^13.0.0",
"@ngx-translate/http-loader": "^6.0.0",
"animate.css": "^4.1.1",
"ng-lazyload-image": "^9.1.0",
"rxjs": "~6.5.5",
"tslib": "^2.0.0",
"zone.js": "~0.10.3"
},
"devDependencies": {
"@angular-devkit/build-angular": "~0.1000.0",
"@angular/cli": "~10.0.5",
"@angular/compiler": "~10.0.0",
"@angular/compiler-cli": "~10.0.0",
"@angular/language-service": "~10.0.0",
"@ionic/angular-toolkit": "^2.3.0",
"@types/jasmine": "~3.5.0",
"@types/jasminewd2": "~2.0.3",
"@types/node": "^12.11.1",
"codelyzer": "^6.0.0",
"com.razorpay.cordova": "^1.4.12",
"cordova-browser": "^6.0.0",
"cordova-plugin-camera": "^5.0.1",
"cordova-plugin-device": "^2.0.2",
"cordova-plugin-fcm-with-dependecy-updated": "^7.8.0",
"cordova-plugin-file": "^6.0.2",
"cordova-plugin-file-transfer": "^1.7.1",
"cordova-plugin-googleplus": "^8.5.1",
"cordova-plugin-ionic-keyboard": "^2.2.0",
"cordova-plugin-ionic-webview": "^4.2.1",
"cordova-plugin-splashscreen": "^5.0.2",
"cordova-plugin-statusbar": "^2.4.2",
"cordova-plugin-whitelist": "^1.3.3",
"jasmine-core": "~3.5.0",
"jasmine-spec-reporter": "~5.0.0",
"karma": "~5.0.0",
"karma-chrome-launcher": "~3.1.0",
"karma-coverage-istanbul-reporter": "~3.0.2",
"karma-jasmine": "~3.3.0",
"karma-jasmine-html-reporter": "^1.5.0",
"protractor": "~7.0.0",
"ts-node": "~8.3.0",
"tslint": "~6.1.0",
"typescript": "~3.9.5",
"web": "0.0.2"
},
"description": "An eCommerce App",
"cordova": {
"plugins": {
"cordova-plugin-whitelist": {},
"cordova-plugin-statusbar": {},
"cordova-plugin-device": {},
"cordova-plugin-splashscreen": {},
"cordova-plugin-ionic-webview": {
"ANDROID_SUPPORT_ANNOTATIONS_VERSION": "27.+"
},
"cordova-plugin-ionic-keyboard": {},
"cordova-plugin-googleplus": {
"REVERSED_CLIENT_ID": "XXXX",
"WEB_APPLICATION_CLIENT_ID": "XXXX",
"PLAY_SERVICES_VERSION": "15.0.1"
},
"cordova-plugin-camera": {
"ANDROID_SUPPORT_V4_VERSION": "27.+"
},
"cordova-plugin-file": {},
"cordova-plugin-file-transfer": {},
"com.razorpay.cordova": {},
"cordova-plugin-fcm-with-dependecy-updated": {}
},
"platforms": []
}
}

I tried to add variables,
    "ANDROID_DEFAULT_NOTIFICATION_ICON": "@mipmap/ic_launcher",
    "ANDROID_FIREBASE_BOM_VERSION": "26.0.0",
    "ANDROID_GOOGLE_SERVICES_VERSION": "4.3.4",
    "ANDROID_GRADLE_TOOLS_VERSION": "4.1.0",
    "IOS_FIREBASE_MESSAGING_VERSION": "~> 7.0.0"

But, Now getting this following issue.

Configure project :app
FCMPlugin: Support for Gradle v4 or lower is deprecated. Please upgrade to a newer version.
FAILURE: Build failed with an exception.

What went wrong:
Could not determine the dependencies of task ':app:compileDebugJavaWithJavac'.

Could not resolve all task dependencies for configuration ':app:debugCompileClasspath'.
Could not find com.google.firebase:firebase-messaging:null.
Required by:
project :app

Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to
get full insights.

I think I just ran to the same issue after upgrading to version 7.8.0:
> Configure project :app
FCMPlugin: Support for Gradle v4 or lower is deprecated. Please upgrade to a newer version.
FAILURE: Build failed with an exception.

* What went wrong:
Could not determine the dependencies of task ':app:compileDebugJavaWithJavac'.
> Could not resolve all task dependencies for configuration ':app:debugCompileClasspath'.
   > Could not find com.google.firebase:firebase-messaging:null.
     Required by:
         project :app

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 2s
Command failed with exit code 1: /Users/mikkoville/project/onskebo/platforms/android/gradlew cdvBuildDebug -b /Users/mikkoville/project/onskebo/platforms/android/build.gradle
Error: Command failed with exit code 1: /Users/mikkoville/project/onskebo/platforms/android/gradlew cdvBuildDebug -b /Users/mikkoville/project/onskebo/platforms/android/build.gradle
    at makeError (/Users/mikkoville/project/onskebo/platforms/android/cordova/node_modules/execa/lib/error.js:59:11)
    at handlePromise (/Users/mikkoville/project/onskebo/platforms/android/cordova/node_modules/execa/index.js:114:26)
    at processTicksAndRejections (internal/process/task_queues.js:97:5)
[ERROR] An error occurred while running subprocess cordova.

Thanks, @mvaljento. It's resolved!


Migrate Brief.copied_from_brief_id to Brief.is_a_copy

Migrate data, drop field.

And the model change here can go in to the previous PR (https://github.com/alphagov/digitalmarketplace-api/pull/587).


Storage service shouldn’t fail an ingest if the bag tracker is temporarily unavailable

This has happened a couple of times in the last fortnight – the bag tracker is down, an app goes to talk to it, it gets a connection error and the entire ingest fails.
Our apps should cope with this error and retry when the bag tracker is back up hopefully.

The exact error is

akka.stream.StreamTcpException: Tcp command [Connect(storage-prod-bags-api.storage-prod:8080,None,List(),Some(10 seconds),true)] failed because of java.net.UnknownHostException: storage-prod-bags-api.storage-prod


playability with touching kings (ornicar/lila#1583)

Chasing ornicar/lila#1583

This is because kings can not threaten other kings: https://github.com/ornicar/scalachess/blob/856aa08bebbd6f363ba9676270041eda20aa1dfa/src/main/scala/Board.scala#L47

The castling validity check does not use this kind of filter (and is thus unaffected):
https://github.com/ornicar/scalachess/blob/1bd6bfea86465418a3caa290106bc47b694b131b/src/main/scala/Actor.scala#L120

Other calls:

[X] Variant.scala
[X] RacingKings.scala
[X] Atomic.scala

This may have also been the last use of the filter argument in scalachess. None found in lila, too.

Variant specific considerations from the commit message:

Most variants, including standard chess: Consider it a check when
kings are touching.
Antichess: Overrides everything. There are no checks.
Atomic: Touching kings are not a check. Instead the king is even
protected.


[TS MIGRATION]Typescript migration for MoneyReportHeader

Details
Migrating the MoneyReportHeader component from jsx to tsx

Fixed Issues
Typescript migration for MoneyReportHeader

$ #25108
PROPOSAL: #25108 (comment)
Tests

Create a Money Request
Go to created Money Request
Verify that you can see the header
Verify clicking on 3 dot menu shows the pop-over

[x] Verify that no errors appear in the JS console

Offline tests
Same as Tests steps.

QA Steps
Same as Tests steps.

PR Author Checklist

[x] I linked the correct issue in the ### Fixed Issues section above
[x] I wrote clear testing steps that cover the changes made in this PR

[x] I added steps for local testing in the Tests section
[x] I added steps for the expected offline behavior in the Offline steps section
[x] I added steps for Staging and/or Production testing in the QA steps section
[x] I added steps to cover failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[x] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)
[x] I tested this PR with a High Traffic account against the staging or production API to ensure there are no regressions (e.g. long loading states that impact usability).

[ ] I included screenshots or videos for tests on all platforms
[ ] I ran the tests on all platforms & verified they passed on:

[ ] Android: Native
[ ] Android: mWeb Chrome
[ ] iOS: Native
[ ] iOS: mWeb Safari
[x] MacOS: Chrome / Safari
[ ] MacOS: Desktop

[x] I verified there are no console errors (if there's a console error not related to the PR, report it or open an issue for it to be fixed)
[x] I followed proper code patterns (see Reviewing the code)

[x] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick)
[x] I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
[x] I verified that comments were added to code that is not self explanatory
[x] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[x] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method

[x] If any non-english text was added/modified, I verified the translation was requested/reviewed in #expensify-open-source and it was approved by an internal Expensify engineer. Link to Slack message:

[x] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[x] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is approved by marketing by adding the Waiting for Copy label for a copy review on the original GH to get the correct copy.
[x] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[x] I verified the JSDocs style guidelines (in STYLE.md) were followed

[x] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[x] I followed the guidelines as stated in the Review Guidelines
[x] I tested other components that can be impacted by my changes (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar are working as expected)
[x] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[x] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[x] I verified that if a function's arguments changed that all usages have also been updated correctly
[x] If any new file was added I verified that:

[x] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[x] If a new CSS style is added I verified that:

[x] A similar style doesn't already exist
[x] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(theme.componentBG))

[x] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[x] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[x] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[x] If the PR modifies a component or page that can be accessed by a direct deeplink, I verified that the code functions as expected when the deeplink is used - from a logged in and logged out account.
[x] If the PR modifies the form input styles:

[x] I verified that all the inputs inside a form are aligned with each other.
[x] I added Design label so the design team can review the changes.

[x] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[x] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.

Screenshots/Videos

Android: Native

Android: mWeb Chrome

iOS: Native

iOS: mWeb Safari

MacOS: Chrome / Safari

MacOS: Desktop

@KrAbhas Please, resolve lint and TS errors. If the TS issue appears due to not migrated files, follow the rules described in the guidelines.
Also please add screenshots for every platform.

Lint issues have been resolved, I am testing and adding the screenshots, will be done in a couple of hours. Will let you know once completed.

@KrAbhas Please, resolve lint and TS errors. If the TS issue appears due to not migrated files, follow the rules described in the guidelines.
Also please add screenshots for every platform.

Please update PR name to follow the issue name, so
[TS migration] Migrate 'MoneyReportHeader.js' component to TypeScript

I will attach the screenshots for mobile devices soon, after resolving some of the setup issues. Almost solved, due to new Mac setup facing some trouble.

Added all screenshots and videos.

@KrAbhas can you merge main, please?

There was a conflict due to relationship parent->current->child , where parent and child and expecting slightly different types, so I followed to approach to match type of all three with parent, @VickyStash please review as you have made changes to child component.

@rlinoz , everything updated. The lint error is coming from some other files(not changed in this PR), so will be fixed as part of other issues I guess.

Thanks @KrAbhas, I believe the lint fix was merged, would you mind merging main once more?

@rlinoz Yes it's fixed. Thanks!

Oh I approved, but lint is failing

Yes I am still waiting on the lint fixes, will start with the checklist then.
@KrAbhas Can you please fix the lint issue?

Reviewer Checklist

[x] I have verified the author checklist is complete (all boxes are checked off).
[x] I verified the correct issue is linked in the ### Fixed Issues section above
[x] I verified testing steps are clear and they cover the changes made in this PR

[x] I verified the steps for local testing are in the Tests section
[x] I verified the steps for Staging and/or Production testing are in the QA steps section
[x] I verified the steps cover any possible failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[x] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)

[x] I checked that screenshots or videos are included for tests on all platforms
[ ] I included screenshots or videos for tests on all platforms
[ ] I verified tests pass on all platforms & I tested again on:

[ ] Android: Native
[ ] Android: mWeb Chrome
[ ] iOS: Native
[ ] iOS: mWeb Safari
[ ] MacOS: Chrome / Safari
[ ] MacOS: Desktop

[ ] If there are any errors in the console that are unrelated to this PR, I either fixed them (preferred) or linked to where I reported them in Slack
[x] I verified proper code patterns were followed (see Reviewing the code)

[x] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick).
[x] I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
[x] I verified that comments were added to code that is not self explanatory
[x] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[x] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method
[x] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[x] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is approved by marketing by adding the Waiting for Copy label for a copy review on the original GH to get the correct copy.
[x] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[x] I verified the JSDocs style guidelines (in STYLE.md) were followed

[x] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[x] I verified that this PR follows the guidelines as stated in the Review Guidelines
[x] I verified other components that can be impacted by these changes have been tested, and I retested again (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar have been tested & I retested again)
[x] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[x] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[x] If a new component is created I verified that:

[x] A similar component doesn't exist in the codebase
[x] All props are defined accurately and each prop has a /** comment above it */
[x] The file is named correctly
[x] The component has a clear name that is non-ambiguous and the purpose of the component can be inferred from the name alone
[x] The only data being stored in the state is data necessary for rendering and nothing else
[x] For Class Components, any internal methods passed to components event handlers are bound to this properly so there are no scoping issues (i.e. for onClick={this.submit} the method this.submit should be bound to this in the constructor)
[x] Any internal methods bound to this are necessary to be bound (i.e. avoid this.submit = this.submit.bind(this); if this.submit is never passed to a component event handler like onClick)
[x] All JSX used for rendering exists in the render method
[x] The component has the minimum amount of code necessary for its purpose, and it is broken down into smaller components in order to separate concerns and functions

[x] If any new file was added I verified that:

[x] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[x] If a new CSS style is added I verified that:

[x] A similar style doesn't already exist
[x] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(theme.componentBG)

[x] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[x] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[x] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[x] If the PR modifies a component or page that can be accessed by a direct deeplink, I verified that the code functions as expected when the deeplink is used - from a logged in and logged out account.
[x] If the PR modifies the form input styles:

[x] I verified that all the inputs inside a form are aligned with each other.
[x] I added Design label so the design team can review the changes.

[x] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[ ] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.
[ ] I have checked off every checkbox in the PR reviewer checklist, including those that don't apply to this PR.

Screenshots/Videos

Android: Native

Android: mWeb Chrome

iOS: Native

iOS: mWeb Safari

MacOS: Chrome / Safari

MacOS: Desktop

One more conflict to go, then we can merge.

@KrAbhas Please resolve conflicts

@rlinoz some problem is coming from master in unchanged file related to typecheck. please tag me once fixed, I will take a fresh pull.

:hand: This PR was not deployed to staging yet because QA is ongoing. It will be automatically deployed to staging after the next production release.

🚀 Deployed to staging by https://github.com/rlinoz in version: 1.4.27-0 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

🚀 Deployed to production by https://github.com/thienlnam in version: 1.4.27-1 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
failure ❌

🕸 web 🕸
success ✅


Query string not ignored when using query string specifier

Hi!  First of all big thanks for such a great and easy to use tool!
I have a problem with not expected behavior of -q flag on v0.9.6. Consider this line from our nginx logs:
2016-03-01T03:00:41+03:00	example.com	GET	/Login	a=b&c=d	<IP_ADDRESS>	Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.116 Safari/537.36	302	0.263

a=b&c=d corresponds to $args variable.  This is corresponding goaccess config:
date-format %Y-%m-%d
time-format %H:%M:%S
log-format %dT%t+%^\t%^\t%m\t%U\t%q\t%h\t%u\t%s\t%T

goaccess -p cfg -f sample -q gives this URL in Top Requests table: /Logina=b&c=d.  There is no ? between path and query string but even if we add it to the log we still get query string in the table (/Login?a=b&c=d).
Isn't it expected to omit what is matched by %q when -q parameter used?
And there is another little question with this situation.  I can move it to another issue if you want.  As you can see it is possible to have only arguments from query string without ? in nginx logs.  Maybe add ? to the content of %q in such situations?

Hi, Currently, in order to avoid the overhead of adding the query string to every URL path and then removing it, it does the opposite. It will append it after it has checked that there's no query string to strip. I'm trying to understand the use case on this. Is there any reason why you would like to append it and then remove it?
As far as the question mark ? at the beginning of the string, I can prepend it if the first character is not ?. I don't recall if Apache's %q already contains ?...
Thanks.

I'm trying to understand the use case on this. Is there any reason why you would like to append it and then remove it?

I just wanted to report that query string from %q is not removed with -q (no query string) parameter.  Here is my use case.  I was experimenting with reports and wrote log-format with %q.  Then I wanted to see what the report would be without query strings and applied -q parameter.  However, query string wasn't removed.
I agree, no real use case here.  After the report was tested I don't need -q argument because I have appropriate log-format.  So maybe there is no real problem here.

The latest commit adds ensures the '?' is prepended to the query string if it's not present.
Thanks for pointing this out!


Docker integration tests do not use the upstream repo

The docker integration tests do not use the upstream IML repo from Jenkins.
Instead they are using the mfl-devel repo. This is because the base.repo needs to be overridden in /etc/iml-docker/setup prior to starting the swarm.

In addition, we need to keep the upstream copr repo chosen for the test run as there are bits there that are needed.


Structure and cleanup

Large PR that focuses on the overall structure and flow of the spec, aligning with all the great new changes, and doing lots of different clean up.
In particular, it focuses on #195 #194 and #152
It is marked Work In Progress as there is more to do, but making a PR so people can see what is coming.

I think this PR should point to dev and not master?

Ah, yes. Changed, thanks!

Ok, this is good for review.

Ok, this is good for review.


有没有大佬遇到过 a problem RuntimeError: Cannot load the vocabulary from the model directory

有没有大佬遇到过 a problem Loading faster whisper model medium for device None after RuntimeError: Cannot load the vocabulary from the model directory

@zhangzijie666, hello. Does the  vocabulary.txt file exist in your model path ?
If it is missing or not in the correct format, it may cause the error above.

在下载model的时候目录里应该有应该vocabulary.json文件，把它一起下载下来，放在model.bin同目录下


Version for Windows Server without desktop experience (GUI)

Problem Description
Current installer does not work for Windows Server without desktop experience (GUI).
Proposal
Provide a version that installs and works on CLI-only Windows Server installations.
Alternatives
I don't see any.
Additional context
I cannot install Windows Server with GUI due to constraints.

I believe the current installer should also work from the command line.
https://mitmproxy.org/downloads/ also has standalone binaries.

The exe installer does not work. I didn't know about the zip version. Not mentioned on the installation page. Will try the zip version and feedback.


Add isize and usize types

The decision.

Thanks! The .tmLanguage is generated using AAAPackageDev though. I've added your changes in the JSON file, too.

When is it planned to remove the int/uint and i/u terms from this highlighter?
It was suggested to remove them immediately from the vim syntax highlighter (rust-lang/rust#20881), as this shows their deprecation.
Also, when does this get pushed up to github/linguist?
Thanks.

Good idea. If someone opens a PR I will merge it right away, otherwise I
will do it in the following weeks ;)
Don't know about linguist. Are they using this plugin?

Yeah, I got here from linguist. This plugin is linked as a submodule in https://github.com/github/linguist/tree/master/vendor/grammars. I assume that means githubs syntax highlighting is using this plugin.
I'm not a Sublime Text user, so I don't really want to start hacking at it without a way to test.

@jhasse I think we should wait a little bit, at least until the beta version of Rust.


Add GPTLocalhost for using AnythingLLM in Microsoft Word

Pull Request Type

[x] ✨ feat
[ ] 🐛 fix
[ ] ♻ refactor
[ ] 💄 style
[ ] 🔨 chore
[x] 📝 docs

What is in this change?
This repo is great and I have integrated AnythingLLM with Microsoft Word through a local Word Add-in. I wonder whether it is suitable to list this Word Add-in in README so that users have one more reason to use AnythingLLM. Thank you for your time and consideration.

Thank you. The demo of using AnythingLLM in Microsoft Word has been uploaded to YouTube here.